The Interledger Community 🌱: Little Webby Press

Little Webby Press — Final Grant Report

Andre Alves Garzia — Tue, 13 Jul 2021 18:12:22 +0000

The devices shown are all viewing Chapter 5 of Moby Dick, which was assembled by Little Webby Press, that is shown on the main computer screen above.

Project Update

Little Webby Press has reached it's desired minimum lovable project status, and is ready for general usage. Authors can use the webapp to assemble eBooks and hotsites from their manuscripts thus enabling them to focus their time and effort into their craft instead of having to deal with desktop publishing tooling and trivia.

The project is feature complete in terms of the features that were planned for this grant — more about them below — and I'm happy to announce that we managed to generate eBooks with 100% of epubcheck compliance, which is harder than it should be.

EBooks generated with Little Webby Press can be hosted and distributed on each authors own platform, while still being distributed by major vendors such as Kobo and Amazon. The Web version of the eBooks can be WebMonetised, thus providing yet another funding opportunity for content creators.

At the moment, Little Webby Press, is most useful to Indie Authors and those in the self-publishing ecosystem including writers doing Web Novels and Fanfics as well. The system is flexible enough to cater to those audiences without forcing them to leave whatever other platforms they're already present on.

A Bit Of Context

To understand our progress, it is important to understand the workflow used by authors when using our application. They write their manuscript using as many plain-text files as they want to. This is a common workflow used by other major players in this ecosystem such as Leanpub and Scrivener. These files are collated and assembled into a manuscript, which goes into revision and edits until it is good enough for publishing.

Each author has their own preferred text editor, we're not in the text editing game, we let them use whatever they want as long as it can save into a format that Little Webby Press understands.

To assemble an eBook or a WebSite for the eBook, the author can:

Drag & drop the folder with the manuscript into Little Webby Press.
Click a button to load a folder and manually select the folder with the files.

Once the files are loaded, an interface is shown for the author to verify if everything is correct. If they're happy with it, they can click to generate the eBook or to generate the WebSite. The assembled deliverables will be downloaded to their machine.

In essence, the system was designed to stay away from the user and provide as low friction as possible. It takes an author less than two minutes to use Little Webby Press to assemble their book. Their time is spent with their content, not with our system.

I decided to explain this all to you because to craft a system to be used in a way that let the user do what they want and move away from it as fast as possible is not usually a UX constraint. There having been multiple iterations over the feature set and workflow to remove steps and reduce time spent on the webapp. Little Webby Press has been designed to get out of the way.

Progress on objectives & Key Activities

The easiest way to follow the progress on objectives is if we break them down by core feature. The overlap with our key activities was so big that I opted to fold these two sections into a single one. Below, I'll create a subsection per core feature in our proposal. I'll describe what it is and what is the current status of the implementation.

eBook generation

At the core of Little Webby Press is the eBook generation. In our original proposal, we planned to generate EPUB3 and MOBI formats as EPUB3 was used by all companies out there and Amazon required MOBI. This changed while we were developing the project, it was a minor miracle to be honest, as Amazon took an unexpected change of policy and decided that all eBooks should be sent to them in EPUB3 format, and that they would convert them to MOBI on their own. Suddenly, we didn't needed to implement MOBI anymore.

Without the complexity of having to implement two different eBook formats, we could focus our attention into EPUB3 which is the default standard for eBooks.

Epubcheck is a tool used industry-wide to verify EPUBs for compliance with the spec. It is a very strict tool, some say it is too strict, and getting a book to validate with 100% approval is quite hard. Not only because it is a complex spec, but also because the content of the book can affect compliance.

Our Moby Dick sample has been reaching 100% compliance for a while now proving that Little Webby Press can generate good EPUBs.

Important: Epubcheck compliance is good but it is not needed for your book to be compatible with most devices. Each device is usually way less strict than epubcheck in what they accept and are able to display. The only store I am aware that rejects books based on compliance with epubcheck is Apple iBooks.

WebMonetization

We can't add WebMo to EPUB files until someone creates eReaders that follow WebMo specs. Little Webby Press has always been more than just an EPUB generator. That feature was important because without it no author would be interested in using our system, but LWP offers more.

A core feature is our ability to generate a hotsite for your book and also a web version of the eBook. An author can actually disable the EPUB generation if they're so inclined and go with a pure web deliverable.

Both the hotsite and web version of the book are WebMonetized.

There is more to do in the future regarding WebMo, our roadmap doesn't end with the end of the grant. The next step for this feature is enabling the author to unlock extra content for readers who have WebMo enabled.

Multiple templates

I'm not a designer. Doing good templates is hard for me and even though the generic default template used in Little Webby Press is OK and good enough for daily usage, we didn't want to lock users into having only my simplistic designs available for them.

Instead of commissioning someone to write a couple of good templates for us — something I might still do in the future, maybe with some new grant — I opted to provide a mechanism for authors to use their own custom template.

Authors who are happy with the default template can simply keep using it.
Creators who know HTML/CSS can create their own template and use them.
Creators who have enough funds can commission a designer to create bespoke templates for them, and use them with Little Webby Press.

Important: There are mechanisms in place allowing future templates to be shipped with Little Webby Press. The system already looks for extra templates, they're just not there.

Multiple markup languages

Most SaaS and applications in this ecosystem when they support a markup language, they work with Markdown, and Markdown only.

Even though Markdown is easy to learn (it is what we use to write these reports for example) it is considered by some to be too simplistic and even buggy.

Instead of forcing our own personal choice on our users, we opted to provide multiple options instead. This is a theme with Little Webby Press options and mechanisms over policies and decisions, provide the user with options and get out of the way. It is a very unopinionated web app.

Little Webby Press accepts manuscript files written in: Markdown, Asciidoctor, Textile, and even HTML chunks.

As far as I am aware, there is no other eBook generation web app that accepts all these formats. There is pandoc but that is a command-line tool with a steep learning curve.

Documentation

Our proposal had a huge emphasis on documentation. It is one of the pillars of Little Webby Press, allowing creators easy access to the information they need.

A cool aspect of our documentation is that it was assembled by Little Webby Press itself. In Computer Science this is called eating your own dogfood, ours was delicious. We wrote two books about how to use Little Webby Press and they alongside our samples, templates, and video tutorial compose our documentation site.

No user accounts

Little Webby Press is not a startup. We don't have a business plan, or a path to grow that passes anywhere near being a company. This is an open source project designed to be easy and cheap for me to run.

All the processing happens in the front-end. Little Webby Press webapplication is a set of static HTML5 files. They don't require expensive servers or DevOps to keep working. One can actually download the exact same files from our repo and run everything locally if they so wish.

Since everything is done in the browser, there is no need for the authors data to leave their computer. This makes LWP very private and enables us to work without having user accounts.

Communications and marketing

Posted about it on my blog: Launching the MVP for Little Webby Press • AndreGarzia.com
Created an introductory video about it, which is linked on the post.

What’s next?

This is the end of our funded period, but not the end of our journey. NaNoWriMo is just around the corner, and Little Webby Press will be ready to welcome all the thousands of novel writers that are a part of that wonderful community. These were just the first chapters, the setup for our adventure is now ready, and there is a lot to do.

We're extremely thankful for all the support from everyone at Grant For The Web. You folks made all this possible. Without your support, I wouldn't have been able to find the time and safety required to focus on creating this tool. Thank you ❤️

Little Webby Press MVP launched

Andre Alves Garzia — Fri, 09 Jul 2021 16:33:12 +0000

This appeared first on my own personal blog.

Today I’m launching the initial version of Little Webby Press, an online service to build eBooks and a website for the eBook. It allows the writer to go from a folder with manuscript files to a static website and eBook with a single click. Check the video below for a quick introduction, and to see it working:

How it works

The author creates a folder on their own machine and use a plain-text editor to create their manuscript files inside that folder. They create a book configuration file — usually in TOML, but YAML and JSON are also supported — that sets all the configuration options for the Little Webby Press workflow.

The folder is loaded into the Web Application and the author can then generate an eBook and a Website based on their manuscript. If they notice they want to change something, or they progressed in their writing and want to generate a new eBook, they can simply reload the web app, open the folder again, and use the action buttons to generate the deliverables. Every time there is a change to the manuscript folder, it needs to be reloaded. There is no way for Little Webby Press to monitor your disk files. Still reloading is as simple as just dragging and dropping the folder on the UI again.

Why does it exists?

Mostly because I wanted to build it. I have been publishing books for many years now, and decided it was time for me to be on control of my tools. Another important aspect is that I wanted to build something that worked purely on the client-side so that it could be offered for free and without user accounts.

There are many tools out there to generate eBooks and even more tools to generate static websites. They range from very simple to use SaaS such as Leanpub, to complex command-line tools such as Pandoc. There is a place for everyone and I wanted to create something that reduced friction between writing your book and actually getting something you can share.

Some Highlights

You can write your manuscript files using Markdown, Asciidoctor, Textile, or raw HTML chunks. You can mix and match them in the same folder as long as you use the correct file extension for each format. I'm not aware of any other SaaS that allows you to write in all these formats.

The configuration file can be written in TOML, YAML, or JSON. Again providing the author with the flexibility to use whatever configuration format they’re more comfortable with.

It supports custom templates—even though the documentation about them is not yet ready—allowing the author full control over the workflow. Basically, add a _theme folder inside the manuscript folder, and place your custom template inside it as it’s own folder. For example, if your template is named custom, then the folder would be _theme/custom/ and the configuration would have a line theme = “custom”. I will document that shortly.

Even though the feature above is not documented, Little Webby Press has extensive documentation that was written using Little Webby Press itself.

There is initial support for WebMonetization. I think that it is a great standard that could foster a sustainable Web that does not rely in Ads or selling user data.

I’m developer-curious, tell me about the source

The source-code is available at the Little Webby Press Github repository. The license for the source is MIT, so use it any way you want. There is no backend, everything is processed and assembled in the frontend. I need to be honest here and make a bit of a disclaimer:

This is a project that I started to make for myself, and then decided it was cool enough to be something I’d share with others. I got a grant from Grant For The Web to build it, and that is how I could afford to devote time to this project.

The source-code is a bit messy. There are two major functions, one to generate the epub and another to generate the website, it is basically waterfall-oriented. Waterfalls are pretty, sit down, enjoy the view.

Yep, I said it. The source is not the pristine webapp you’re used to see when someone go public with a project. But, hey, I’m a writer first and a developer second. I’m happy with my source, and my results. That’s enough for me.

The whole system is built with Rollup from Svelte components and some extra JS files. There is a fake filesystem in memory, courtesy of BrowserFS, that allows the waterfalls mentioned above to be coded as if we were dealing in NodeJS with a local disk filesystem. It is all a lie, there is no file system, but don’t tell that to the routines, they’re fragile already.

Once those waterfalls finish assembling everything on the in-memory filesystem, a zip file is created with JSZip and downloaded. You can host the deliverables in any server that supports static files.

Final words

This was just a simple post to make it real. I have been working in this for a while, and a lot of it was rushed because I lost one of my jobs just before the Pandemic hit. Both the Pandemic and that job loss caused an impact in my life, and it took me a while to recover and get this rolling. The grant was fundamental for allowing me the time to breath and the safety to spend time researching better workflows and test driving this with my own books.

There are lots of opportunities for improvement, and I’ll keep working on this. Little Webby Press is a project I love and use, I think it can grow to be a wonderful alternative for authors everywhere. I hope you enjoy it too.

We're almost ready!

Andre Alves Garzia — Sun, 06 Jun 2021 18:33:11 +0000

Little Webby Press is almost ready for general usage. Today I got simple WebMonetization working on the static websites generated by LWP.

In the screenshot above, you can see Chapter 2 of Moby Dick on the screen and the Coil WebExtension verifying that WebMo is indeed working.

It is the final sprint to get everything ship shape for the final report and release of the tool.

I also posted a small teaser video about it on Mastodon.

Little Webby Press — Grant Report 1

Andre Alves Garzia — Mon, 08 Mar 2021 13:49:20 +0000

Life Update

The last months have been hard for me and thus for the project. Before I embarked on this endeavour, I had two fixed jobs which provided an steady income. Due to multiple factors one of the jobs is gone and the other is severilly diminished. This impacted the amount of hours I could put on Little Webby Press since I needed to sort out my situation post-grant. I'm writing this just as a bit of contextualization, these projects we're all building don't exist in isolation and unforseen situations elsewhere in our lives end up impacting how we deliver our little gems. I do plan to ask for a no-cost extension because of that. Now, onto proper project updates...

Project Update

The objective of Little Webby Press is to help writers deliver eBooks and Static Websites from their manuscript. The Static Website is WebMonetised and is the part of the project that effectivelly hooks into Grant For The Web from a technical perspective, still the eBooks part is the harder part to build. Effective eBook generation that passes the dreaded Adobe Epubcheck is damn hard and a simple online search will guide you to many developers crying out their frustrations about those strict tests.

It has been a bit more than a month that Little Webby Press is finally able to generate eBooks that comply with those tests. That is very rare for eBook generators and I consider it a major victory. It is important to notice that many of the eBooks you buy out there are not compliant with those tests and work anyway. Those tests are very strict and missing some marks does not mean your book is broken.

Little Webby Press is able to generate EPUB3 books with good styling, hyperlinked table of contents, and no erros or warnings on epubcheck. This means that the eBook part of the project can be considered done for now.

Onto Static Site generation. I spent more than a month pursuing a novel approach which used epub.js from FuturePres to render the real eBook on the website. After I got it done, I quickly realised that this was not the way to go. Yes, the eBook renders well and behaves well, but it feels like an applet embeded in a page. I can't really put into words, there is nothing wrong with their reader, it is a fantastic piece of work. It is just that I think that a true static site book like those generated by gitbook or mdbook (sample mdbook here) is better.

So I changed to work on that. This makes it easier to unlock bonus content using WebMonetisation since I control all the assets being generated.

Progress on objectives

✅ compliant EPUB3 generation.
✅ multiple templates.
⚒️ static site generation.
- a minimal site can be generated and supports WeMo.
- needs better templates.
- needs code for unlocking bonus content.
🐌 documentation has not been touched yet as the UX and patterns are still evolving.
- Documentation in form of videos and text is a large part of this grant and work will begin on it as soon as I have better templates for static site generation.
🛑 MOBI and PDF generation.
- When I originally wrote the proposal, I added the need to generate MOBI (to support selling eBooks on Amazon) and PDF because people like PDFs.
- Since that time, Amazon switched their requirements to EPUB3 from MOBI which means I don't need to generate MOBI anymore.
- I've found an online service called epub.to that doesn't require accounts and has a compatible privacy policy (as in they don't save anything) that I can leverage for PDF generation.

Key activities

The current key activity is evolving the templates and the static generator to the point that a beta can be launched. I'm very close but I'm not a web designer, as in my layouts are not really pretty. I'm working with this in mind and making it possible to add custom layouts and even bundling them later with Little Webby Press. This way I can later accept template contributions without the need to change anything or even recompile the code.

Communications and marketing

I've got very low energy right now. I'm living in a new country (the UK) for the past two years and haven't yet established many friendships here. Due to lockdown, I've been isolated and living on groundhog day mode for many months.

At the moment, I don't really have the willpower to engage into doing a well structured communication and marketing campaign. I understand it is needed, but until I have a better template for the static generation and at least some documentation, I don't think it is very effective. I'm trying to devote my time to fixing that instead.

What’s next?

There is a lot of work to do. The bulkier part of it is the comprehensive documentation I have planned, which goes beyond just using the app but into being an effective web writer.

I will probably need a no-cost extension to be able to finish all I have planned.

The most immediate task at the moment is finishing the static generator.

What community support would benefit your project?

In the near future, I'd probably benefit from template contributions (they're just HTML + CSS).

Resumed working; Updates

Andre Alves Garzia — Tue, 05 Jan 2021 18:41:32 +0000

I resumed working on Little Webby Press today after taking an extended pause for the holidays and some peace of mind.

Some aspects of building an static generator seem simple at first glance, but are actually quite complex. A good example is generating a Table of Contents for a book. Upon a cursory inspection, one might assume that since each chapter is a markdown file, that just looping through all files and assembling an array is enough. Trouble happens when you stop to think a bit deeper about this problem.

Authors have their own ideas about what constitutes a chapter, and what should be included in a Table of Contents. In the sample book being used – Moby Dick – the author of the markdown files used a combination of <h3>for a hat and <h1>for the chapter title. Meaning that to assemble a working label for the Table of Contents, LWP needs to find the first <h3>in the HTML and then look for the <h1> that immediately follows it. Once LWP has all this information, it can render a nice Table of Contents that looks like:

This is how a chapter source looks:

As you can verify, there is a small heading level 3 followed by a heading level 1 to mark the chapter header. The configuration for that book had to take that into account:

[toc]
prefix = "h3"
label = "h1"
match = "first"

The problem is that each book is its own island and other authors will want a different behavior. Some will want LWP to find multiple ToC entries per chapter. This is common for non-fiction, specially for programming books. Others will not use a little hat or prefix before the entry label.

To accomodate the most common use cases, I had to do a major overhaul of the Table of Contents routines and templates. I'm happy that the sample is being generated with the correct ToC, but there is still work to be done before I am able to start implementing the site generation and web monetization.

In my own subjective opinion, generating the website and WebMo stuff will be much easier than actually generating a valid EPUB3 file.

In another news, Amazon changed their approach to digital publishing and is now telling authors to upload EPUB3 files instead of MOBI files.

This is a major change since Amazon used to require a MOBI file while all the other players required an EPUB. This is because in the early days Amazon bought mobipocket who created the MOBI format for ebooks on Palm Devices IIRC. They were the only vendor requiring the outdated MOBI format. Kindle devices still don't know how to render EPUB files though, what happens is that Amazon KDP (their digital publishing service) will convert from EPUB to MOBI on their backend. If you still want to ship files to be sideloaded on a Kindle, you still need to generate a MOBI.

For me this signals they moving away from MOBI and maybe implementing EPUB renderers in the future devices. Because of this change, I'm changing the requirement of LWP to generate MOBI to a very low priority. MOBIs are only useful for sideloading now, and it is very hard to implement a whole new binary file generation format. Specially since Amazon MOBI spec is terribly undocumented.

Side-note: Amazon ships a desktop app that can convert EPUB to MOBI. This app should work well with EPUB files generated by LWP. This moves implementing MOBI on LWP to even lower priority.

Regarding WebMo, I think I'll have to opt to a naive approach to unlock chapters for monetized users. The generated site will be static, so there is no backend to ~~protect~~ hide content from the user. I'll rely on client-side JS (which is inspectable) to switch which chapters they show based on presence of WebMo. As long as I explain to the author that enterprising users will be able to find the unlockable content anyway, I think I'm OK.

My plan is to frame this as unlocking extra goodies and fun stuff. Things that there would cause no harm to the author if exposed to non WebMo readers.

I still need to do some work before being able to touch this anyway, anyone has feedback or comments?

Little Webby Press original teaser video

Andre Alves Garzia — Tue, 01 Dec 2020 20:45:02 +0000

This is the original teaser video I made after creating the proof-of-concept webapp that would become Little Webby Press.

In this video I explain what Little Webby Press is, why we need such application, and how it works.

I plan to post many more videos here as development goes on. I hope you all like them.

Little Webby Press: Testing book generation

Andre Alves Garzia — Tue, 17 Nov 2020 16:43:22 +0000

Been testing book generation today and debugging with multiple devices.

Check this GIF to see how book generation with Little Webby Press works. For some weird reason I can't embed it on the post :-/

My whole working area was taken over by eReader devices today.

My table full of devices

This is a closer look of chapter five of Moby-Dick.

Kobo Mini, Lev (rebranded Bookeen), Kobo Forma

I think it is looking great 😁

Little Webby Press: first successful book generation

Andre Alves Garzia — Fri, 06 Nov 2020 18:52:14 +0000

PARTY TIME ✨ 🍰

It is time for celebration, for the first time since I started programming Little Webby Press, a book generated by the tool passed the validation from epubcheck without any warnings or error.

There is still a lot of code to build. I don’t have a good way to generate the table of contents and index, I also need better markup for the cover as well. After that is solved, it will be time to work on the static site generator for the books.

Moby Dick sample just passed validation.

Metadata is displaying correctly.

Cover needs work.

The book content is rendering well.