The Interledger Community 🌱: Storage To The People

Storage to the People Final Report

Robert Friedman — Sat, 04 Dec 2021 03:43:30 +0000

Project Update

We are very pleased to announce the completion of our Grant for the Web project to apply the web monetization framework to the data storage use case.

Our final product for this funded project was a plugin to Etherpad that allows a user to seamlessly save a version of their pad to a participating data storage provider if they have Coil running in their browser. Permanent.org is the initial participating data storage provider for the purpose of this grant.

The system in action can be found at https://pad.opentechstrategies.com. We elected to install the etherpad plugin on a non-Permanent Etherpad server as a way to demonstrate separation between pad and storage provider.

We believe this decentralized approach is an essential feature of our result and this workflow is a demonstration of a more general solution that can be extended to any content-generating application. Our next application of this model will be with the Hyperaud.io project.

The plugin is fully functional for the purpose of closing out this funded project, but there are some important user experience improvements we still need to make in order to make this application user friendly and useful for our members and the general public.

Here are a couple of important notes that will help you test and use this plugin while we continue to iterate towards a truly finished product:

If you have the Coil extension installed and configured in your browser, you can log in with your Permanent credentials using the Etherpad plugin under the export menu, enable sync with your account, and see etherpad changes show up in your default Permanent Archive under the Apps > Etherpad workspace folder.
Syncing will time out after an hour and must be manually re-initiated because we still need to improve our token refresh and expiration approaches. You can do that by deleting the “permanentToken” cookie from your browser. Once it is deleted, you will be able to login again. this will be addressed in upcoming releases.
Logging in takes you out of the pad you were just in and you must manually return to that pad yourself so save/bookmark the pad name/url before logging in to Permanent. We will improve this with an auto-redirect in future releases.
The Permanent login screen is not currently styled yet and appears as the simple, default FusionAuth login UI. Don’t be concerned by that.
We still have some important cleanup work to do in our Permanent node-sdk and Etherpad plugin repos: working branches will be merged in the coming weeks.
OTS is planning some maintenance on their Etherpad server. Although some of that can happen without downtime, they will wait a few days after this post to do it.

We will be updating our Etherpad knowledge base article to reflect the current state of the plugin shortly, but the current article serves as a reasonable guide for using the plugin.

Please, take a moment to give our plugin a test for yourself and let us know what you think! Please see the community support section at the end of this post for ways that you can contribute to this work.

Progress on objectives

In this project we set out to achieve the following goals:

Design a workflow that builds on the Web Monetization protocol
Upgrade the Permanent API to support that workflow
Build a simple demonstration application that uses the protocol to store content
Document everything
Improve understanding of ecosystem gaps

We’re very pleased to report that we largely completed all 5 of these goals during our initial grant period. However, in order to extend the scalability of our solution, we requested a strategic no-cost extension to achieve additional development goals.

In the course of our work we discovered that for the Permanent API to properly support the Coil workflow and integrate with a generic demonstration application on an external server, we would have to make some very big changes to the Permanent authentication framework. So we took a strategic pause on the core objectives in the spring and spent the last six months educating ourselves on how to best tackle this overhaul and executing on it. In effect, we ended up adding a sixth objective:

#6 Upgrade the Permanent authentication framework to to support third-party applications and services.

We can now say that we have completed all six of these objectives.

Key activities

Our team consisted of four key people (moving counter clockwise in the screenshot, starting from the top left):

Jason Owen, Software Engineering Consultant, Open Tech Strategies – engineering lead on web monetization framework and Etherpad plugin;
Cecilia Krum, Director of Engineering, Permanent Legacy Fdn – technical direction and interface with Permanent.org API;
Robert Friedman, Executive Director, Permanent Legacy Fdn – project strategy and grant management;
James Vasile, Principal, Open Tech Strategies – research lead, rapid prototyper, documentarian;

Some combination of this team met weekly to review goals and priorities, report progress, assign tasks, and make direct progress on grant objectives through the winter and spring of 2021. At the start of the no-cost extension, most of the work became very focused on the implementation of the authentication framework. We joined the Grant for the Web community call in June 2021 to share our work as well as joining a Coil conference. We also presented some of our key findings on ecosystem gaps to the Grant for the Web advisory committee to inform future grant making.

Together, we completed the following activities in support of our objectives:

Design a workflow that builds on the Web Monetization protocol

We completed the first iteration of this work during the research phase of the funded period. We made an early discovery that Coil was the only viable web monetization payer we could leverage for this work. We honestly could have spent months digging deeper into web monetization, but we knew that building the Etherpad integration would teach us more than static research, and we very much wanted to deliver a usable proof-of-concept.

Upgrade the Permanent API to support that workflow

We made critical upgrades to the login functionality of the Permanent API to achieve the goals of this grant. We added endpoints to our API that allows a 3rd party app to retrieve information about a user’s login status and their currently active archive as well as a mechanism to endow space to a user’s account equivalent to the Etherpad file size. The core mechanic here operates under the assumption that Etherpad files are very small and the Coil payment received from a user’s activity more than covers the cost of storage.

Build a simple demonstration application that uses the protocol to store content

The main tangible output of our work is a repository containing an Etherpad plugin that implements web-monetized (Coil-based) micropayments to authorize permanent storage of pad content. That repository is available on GitHub, and we published its contents under a GPLv3 license.

The plugin we built demonstrates the functionality we originally envisioned to connect a generalized web-monetized application for content creation (or curation) to a storage layer capable of accepting web monetized payments. The plugin contains generalized functionality that could be plugged into any Etherpad instance.

The initial prototype installation of this system that we demonstrated at the Grant for the Web community call in June was hosted at https://pad.staging.permanent.org. However, we chose to host the final version of the system at https://pad.opentechstrategies.com in order to demonstrate a separation between pad and storage provider.

While Etherpad does not require a user to create an account with the host, users will be required to create accounts with Coil and the storage provider in order to save out a pad.

Document everything

Full documentation for installing this plugin is available in the project Readme visible in our GitHub repository. We documented our work in a Permanent blog post, multiple web monetization posts (introduction, interim report, etherpad justification, ecosystem analysis, etc.), and in a slide presentation of ecosystem gaps to the Grant for the Web advisory committee.

Ecosystem Gaps

In the course of adopting Web Monetization, we began with an evaluation of Web Monetization. We came to respect its privacy characteristics, even as we considered how to give users some insight into the trustability of the system. We considered a number of payment structures one might build on top of Web Monetization and the difficulty in creating new models using the current infrastructure. As we studied and implemented, we came to understand that the Web Monetization ecosystem was missing some elements we wished existed.

Coil is well-suited to replace advertising revenue models. It produces advertising-like revenue, which has some characteristics that make it difficult to replace fee-for-service revenue models. In particular, it is difficult to know how much a user has paid. Permanent's pay-once model provides space to users in proportion to fees paid. Those fees are low relative to the long timeframe of service offered, which makes them well suited to micropayments.

Coil, by contrast, provides payment in proportion to service usage, and that usage is proportional to the time spent active on a page. Sites do not get instant feedback on which user paid which amounts, which is what they would need to enact fee-for-service models. A Coil-enabled website monetizes content and can enable premium features, but it has little control over how much it charges and has little ability to scale premium service to fees. That limits Permanent's ability to increase storage quotas reliably.

As a site owner choosing to include Coil as a payment option for our services, we were hamstrung by a lack of granular transaction data. Without the ability to know immediately who paid what amounts for what services, there are strict limits to the types of things we can do. Coil's approach to privacy and security simply does not allow it. This is a point in favor of that approach to privacy. And yet, it made Coil a difficult fit for our current business model.

That realization left Permanent with several options.

We could abandon Coil and build our own web monetization payments infrastructure. That promised to be an insurmountable effort, much larger than the current project (but one we hope to see happen eventually).
We could write our own browser extension and try to leak information to our server. This also would be a large amount of effort, and the goal of this project was never to make a less privacy-protecting Coil.
We could look to a micropayments system that is not built on web monetization. This would take us beyond the scope of the project, but because our goals were research-related, we did examine other solutions. There are a number of them. None are mature, complete solutions that would enable us to serve users effectively and immediately.
We could adjust how Permanent allocates storage for users in the face of inexact payment streams, essentially fitting our business model to the available infrastructure.

After examining each of these options, we chose to adjust Permanent's backend to work with users, even when we cannot tie payments directly to a specific user. We did the math on payments versus the amount of storage required to save Etherpads, and decided we would come out ahead. We plan to test usage over time and verify our numbers which were necessarily based on some assumptions.

This scheme works for small files like Etherpad pads but it breaks down as the file size increases.

Ultimately, we adjusted our business model to suit the existing Coil infrastructure. That's not ideal, as we ended up with worse product-market fit than our usual offerings. An ecosystem limited to Coil's offering of time-based payments decoupled from specific users is missing some necessary pieces. We wanted to see support for one-time-payments, streaming, and various forms of tipping. More broadly, we wanted to see bundled subscriptions that allow several services to make joint offerings.

In addition to those big-picture difficulties, we observed some minor missing elements of the ecosystem:

Information for users. As currently constructed, a user who installs Coil has very little ability to see how it changes their online experience. Where did my $5 go? What sites did I support? What benefits did I receive? All of that is opaque, which might make Coil a difficult sell as a value proposition.
We struggled during development from a lack of test infrastructure. We wanted mock wallets and accounts so we could demo all this end-to-end, dig into the details, and understand what was happening. Especially in a system where developers cannot see much information about transactions, we wanted a test environment with more inspection capabilities.
The need to install a browser extension continues to be a barrier. Although many users have installed ad-blocking extensions, the vast majority of browser users have not. Ad-block penetration at around a quarter of all users has been relatively steady for several years. And once we step beyond ad-blocking, the numbers only go down. Coil has momentum and investment that could overcome reluctance to install extensions. The greatest chance for non-Coil web monetization solutions to reach wide adoption might be as an additional feature of Coil, not as a separate extension.

Authorization Upgrade

In order to make the etherpad plugin specifically and 3rd party web monetized applications generally compatible with the Permanent platform when hosted independently from the Permanent.org tech stack, we needed to upgrade our authentication framework.

Using the time granted by our no cost extension, we undertook a research inquiry to determine the best implementation option for managing authorization. We consulted with experts in the auth community, including D. Keith Casey, Jr. Based on our research we determined the best solution to be a 3rd party authentication provider and after evaluating Auth0 as an option, we selected FusionAuth as our vendor.

We published the work we did to migrate our authentication data to both Auth0 and FusionAuth in our GitHub repository.

Communications and marketing

Our communications and marketing efforts were focused on inviting others to help us build this solution together. We distributed our messages via the web monetization community forem, community calls, and advisory board meetings, as well as the Permanent.org member testing group, blog, knowledge base, social media and email newsletter. We also did all of this work in the open in our GitHub repo and documented our results there.

Through several posts on the Web Monetization Community forem, we shared project updates, insights into our work, discoveries about the web monetization ecosystem, and invitations to evaluate our prototype (see our project page). We joined a Grant for the Web community call on June 24, 2021 and provided our etherpad instance for note taking as a way to highlight and demonstrate the results of our work. It was an honor to have our prototype pad instance highlighted in this way.

Additionally, our project was selected to be featured on the Coil developer site and we participated in the Coil Web Monetization Workshop on July 28, 2021.

During the course of settling on Etherpad as our demo app, we also reached out to the folks building the amazing docs.plus tool! Our conversation with Edward Saperia was particularly inspiring and expanded our understanding of what was possible to build on top of Etherpad! We consulted with experts in the auth community, including D. Keith Casey, Jr.

Our project began and ended with user research. Before any design began, we conducted informal interviews with non tech-adjacent individuals in our community to understand how they would expect this kind of functionality to work for them. Once we had a prototype solution in place, we began to user test with our member community.

We communicated with our members about the etherpad plugin work using Permanent resources. We published this blog post and shared it [via social media]. We made a call for our members to provide feedback on our prototype in our monthly newsletter. And we created a dedicated knowledge base article to help folks understand how the prototype works. Now that we have a final product live on our production server, we are ready to take another shot at communication and marketing. We hope to add this as a prominent feature on our marketing site home page.

What’s next?

Our work on this grant has concluded, but we're not done with Etherpad, the plugin, or web monetization. We want to explore using our plugin with Docs.plus, another GFTW grantee that is already using Coil. One question we have about this path is how Coil works when a user is consuming services from multiple providers.

In addition, there are opportunities to help users better manage their archives of etherpad documents on Permanent.org. If we imagine adding permanence to etherpads across the internet, that's a lot of documents. At that scale, users might need tools that support sophisticated curation of etherpad collections.

We aimed this work at simple, MVP functionality. There are features we would like to add that allow portability of documents between pads, storage to multiple locations, and improved access controls.

More broadly, we could expand this approach to many different types of content, not just etherpad documents. In fact we have already started to explore other compelling use cases thanks to the progress made possible by this grant.

Our most compelling partnership in development today is with the Hyperaud.io team who are currently working on their own Grant for the Web funded project to integrate web monetization into their video transcription application. In this scenario, Permanent.org serves as both a storage layer as well as a streaming platform.

We’re also excited about work we’ve done with Theirstory.io as well as conversations we’ve started with Ponga.com and Webrecorder.org. Each of these applications demonstrates a unique use-case for a web-monetized content to storage workflow.

This work has pushed us toward considering how web monetization fits into Permanent’s overall vision for products and services.

What community support would benefit your project?

There are three key requests for our Web Monetization Community to help support this work:

Host it: If you host an Etherpad instance or are interested in hosting one, we would love to have our plugin installed on your instance.
Use it: keep your meeting notes in Etherpad and use our instance when you do, then save your pads out to Permanent.org to preserve them. If you are an application developer, we’d love to explore opportunities to integrate with our API.
Improve it: our Etherpad plugin is an open source project and we’re eager for members of this community to help us improve it.

Please feel free to reach out with any comments or questions you might have about how to participate.

Additional comments

We would like to thank the Grant for the Web team for creating this truly valuable opportunity to expand the capabilities of Permanent.org. We are so grateful to have been granted the time and resources to work on a unique project. We hope this work expands the capabilities of web-based applications to build a decentralized content distribution and storage ecosystem.

Storage to the People: Come Try It Out!

Thomas Visco — Thu, 24 Jun 2021 14:24:35 +0000

Hello everyone!

We’re really excited for today’s GFTW community call! As part of the call, we’ll be using Permanent’s Etherpad Prototype for collaborative note taking: An exciting milestone for our project.

A Quick Recap on Storage to the People

The main tangible output of our work is a repository containing an Etherpad plugin that implements web-monetized (COIL-based) micropayments to authorize permanent storage of pad content. That repository is available on GitHub, and we published its contents under a GPLv3 license.

We have a demo instance of the pad running on the Permanent staging environment. The pad is not meant for serious use just yet because documents created there can only be saved to the Permanent staging instance at present. Anyone who wants to demo the app can check out instructions on our Knowledge Base.

The plugin we built does demonstrate the functionality we originally envisioned. It is, however, just a demo. We plan to go further. We hope to host a polished version of this app on the Permanent production environment after the completion of auth improvements.

Come Test It Out

We’d love to get feedback from folks. If interested in providing perspective, fill out this brief survey so we can follow-up with you.

We’ll keep you posted and our documentation updated as we discover bugs, roadmap improvements, and build towards the eventual incorporation of a polished version of this prototype into Permanent’s production environment.

Storage to the People – Grant Report 2ish

Robert Friedman — Thu, 03 Jun 2021 20:47:35 +0000

Howdy Web Monetization Friends,

We're back with a fresh update. TL;DR: we're gonna go on a (figurative) summer engineering road trip to take our project outcomes on a little detour. Less Thelma and Louise going off a cliff and more Blues Brothers jumping the bridge.

A quick refresher – the goals of our project were to:

Design a workflow that builds on the Web Monetization protocol
Upgrade the Permanent API to support that workflow
Build a simple demonstration application that uses the protocol to store content
Document everything
Improve understanding of ecosystem gaps

While we have made significant progress on these goals in the allotted grant period (so far), our investigations and prototypes revealed an opportunity to significantly extend the impact of our work.

However, doing so will require us to first take a little roadmap detour in our core product at Permanent.org. That will require more time to get it done right.

First, Our Demo Application

In a fortuitous twist of fate, we chose to build an Etherpad extension as our demonstration application. I've written about the logic behind that choice in past posts.

We believed that building on Etherpad was a solid win-win opportunity for the Permanent.org feature set as well as to the open source ecosystem.

However, the real win-win was that we could get away with using Coil even though it's not designed for a file storage use case like ours. We didn't quite realize that about Coil at the start of this journey, so it was a nice coincidence.

Why will Coil work for Etherpad? That's because Etherpad files are vanishingly small. Even though Coil payments are proportional to the time spent on site and not proportional to the total data transferred in a given session, the small Coil payments cover the small costs of typical Etherpad files.

So, since Coil can sufficiently cover the cost of pad storage, the simple Etherpad plugin concept we cooked up has real potential to be a fully viable web monetized feature in the Permanent.org app as well as an extension that could be used with any Etherpad instance in existence. Yay!

However, there’s a catch.

Second, Upgrades to Auth

If we only wanted to web monetize an Etherpad instance hosted on a Permanent.org server, we'd be done. However, the beauty of Etherpad (vs. walled garden, privacy vacuum products from the internet giants) is that there are instances hosted all over the web.

Don't trust Permanent.org? No sweat, use the riseup.net Etherpad instance instead (which we have been doing for over a year ourselves actually).

However, in order for an Etherpad instance hosted on a server that is not in the Permanent.org stack to make a successful data transfer to Permanent.org for storage, we have to make some critical improvements to our authorization and authentication infrastructure.

These improvements go well above and beyond the anticipated API improvements scoped out for this project and would change how our own microservices communicate with each other as well.

We want to do this work and we need to do this work. It's good for Permanent. We also want to do it under the banner of Grant for the Web and stick around in this great community.

Therefore we will be working under a no-cost extension to give us ample time to upgrade our auth and publish a fully supported, web monetization extension to Etherpad.

We'll keep you posted while we make these important improvements.

Web Monetization Isn't Just Content

James Vasile — Thu, 13 May 2021 22:13:12 +0000

Web Monetization and COIL are very focused on a specific model of paying for content. They do a very good job of replicating a familiar, specific, revenue-generating interaction between users and websites. In order for web monetization to take off, it needs to support other models, other interactions. At Permanent.org, our expertise is storage. We store things on long time scales for fixed, up-front payments. Pay once, keep it forever. Your great-great-great grandkids will thank you!

When we looked at web monetized content creation, we instantly saw possibilities beyond content, beyond creation, and beyond the web. The COIL approach of charging end-user consumers to pay creators serves an important function. It replaces the advertising economic model with something more equitable and privacy-respecting. We believe web monetization can pay for other things too: services, for example. And because storage is our thing, we looked at how the current web monetization infrastructure handles storage as a service.

We also looked beyond what most end-users think of as “the web”. Web monetization doesn’t have to be confined to end-user websites. It can apply anywhere you can do http, which these days is pretty much everywhere except embedded IOT devices. And even those smaller edge computing devices increasingly speak web.

In our project, we took a web application, Etherpad Lite. We added some hooks to Permanent’s web API, and are using web monetization as the payment mechanism to allow users to permanently store the contents of Etherpad documents. There’s a lot to do before our current proof-of-concept work is ready to deploy across the web, but we validated some basic ideas:

Web monetization can do micropayments for services on the backend, not just for content on the front end.
We can pay third parties instead of the content-hosting site-- the micropayments don’t go to the etherpad host. They go to pay for storage at a place the user chooses.
We can integrate Permanent storage as a service in other apps. We can introduce permanence and user control of their own data to whole chunks of the web that are currently ephemeral or out of users’ hands.

Our next steps are to make some API changes on the Permanent side, improve the plugin so anybody can use it in any etherpad installation, and perhaps even make the integration work with another GFTW project, docs.plus.

Of course, we didn’t do all this work just to make a nifty etherpad plugin. We wanted to explore the possibilities. We’ll cover what we learned in our next blog post. The challenges we faced point to gaps in Web Monetization infrastructure, and a lot of our next steps will involve trying to figure out which gaps to fill next. COIL was never supposed to serve every possible need. Designing the next piece to sit next to COIL in the ecosystem will enable web monetization to take some big steps forward in adoption and application.

Web Monetization Back In The Day

James Vasile — Thu, 06 May 2021 21:20:17 +0000

Over at Permanent.org we're thinking a lot about what types of infrastructure we need beyond COIL. There are services that don't match up well with COIL. Of all the revenue models for delivering content and services on the web, COIL focuses on a specific slice. It is quite good at replacing pages with ads on them. It is less good at fee-for-service.

In my research through prior efforts to meet those other usage scenarios, I came across some old and informative and inspiring patents that proposes processes for setting prices on content, paying for them, and splitting that payment among several parties. Those patents are the result of a lot of research and is a good starting point for thinking about content providers who will not be satisfied taking whatever unpredictable amount COIL sends them.

COIL does one thing well. If that one thing isn't you're thing, drop me a line. Maybe we can work together to fill that gap in the ecosystem infrastructure!

Storage to the People — Grant Report 1

Robert Friedman — Thu, 08 Apr 2021 17:52:16 +0000

Project Update

Howdy y'all! We're so excited to be sharing this project update with you. TL;DR: things are going great and we've made good progress on all our stated objectives for a really unique web monetization use case: content preservation and more generally, data storage. After quite a bit of R&D we've identified some important opportunities to expand the Web Monetization ecosystem and improve how Permanent.org interfaces with 3rd party applications.

We set out to explore web monetization and micropayment business models for permanent digital storage. Specifically, we proposed a set of deliverables that would build a demonstration application that relied on web monetized micropayments to secure storage for a user. After much discussion on the team and with the GFTW program team, we chose Etherpad as our target application and decided to integrate web monetization as a plugin. You can learn more about why we chose Etherpad in this post.

We're excited to get to the sharing part. Here's what we've done so far.

Progress on objectives

In our original proposal, we set out to accomplish a defined list of objectives.

Design a workflow that builds on the Web Monetization protocol
Upgrade the Permanent API to support that workflow
Build a simple demonstration application that uses the protocol to store content
Document everything
Improve understanding of ecosystem gaps

The status of each of these objectives is “Nearing Completion” with the exception of documentation and gaps. We're knocking out our punchlist of minor improvements to the demo app and in the midst of building out updates to the API. We still have quite a bit of work left to do on documentation and communication. We need to organize our learnings to meaningfully improve the community's understanding of web monetization ecosystem gaps.

Key activities

Our team consisted of three key people, plus me (moving counter clockwise in the screenshot, starting from the top left):

Jason Owen, Software Engineering Consultant, Open Tech Strategies – engineering lead on web monetization framework and Etherpad plugin;
Cecilia Krum, Director of Engineering, Permanent Legacy Fdn – technical direction and interface with Permanent.org API;
Robert Friedman, Executive Director, Permanent Legacy Fdn – project strategy and grant management;
James Vasile, Principal, Open Tech Strategies – research lead, rapid prototyper, documentarian;

Some combination of this team met weekly to review goals and priorities, report progress, assign tasks, and make direct progress on grant objectives. Together, we completed the following activities:

Design a workflow that builds on the Web Monetization protocol

Upgrade the Permanent API to support that workflow

We made critical upgrades to the login functionality of the Permanent API to achieve the goals of this grant. We identified several more significant improvements that could orient Permanent more broadly around web monetization and 3rd party integration in general. Those changes are beyond scope, but streamlining authorization for automated use would benefit Permanent in multiple ways. We have added those changes to the Permanent development roadmap, and plan to complete them after the grant period, but before the end of 2021.

Build a simple demonstration application that uses the protocol to store content

Document everything

This task is ongoing!

Communications and marketing

We already shared a brief summary of our project goals with the web monetization community a few weeks ago in this post.

We've also been working in the open on GitHub, where anyone is welcome to review our work and contribute. However, the code repository is currently sparse on documentation, which we will address in our next sprints as we approach the end of the project.

Of course, our project began and ended with user research. Before any design began, we conducted informal interviews with non tech-adjacent individuals in our community to understand how they would expect this kind of functionality to work for them. At the end of this work we will circle back to users and present them with our prototype for feedback to understand whether the features are a value add and where we can make meaningful improvements to the user experience.

Finally, as we wrap up documentation, we plan to host a series of webinars, advertised broadly but specifically to the web monetization community to familiarize different target audiences with our new found understanding of the problem we sought to explore, the gaps in web monetization, how the Permanent.org model can add an important preservation tool to the creator toolbox, and how we designed and built our specific demo application for Etherpad.

What’s next?

In summary our next steps include the following:

Wrap up development on the demo application shortly;
Produce documentation in our GitHub repo and as a formal report for the web monetization community;
Host a series of webinars to inform and test our assumptions and prototypes;

What community support would benefit your project?

Fundamentally, we are calling for all people interested in the broad issues of digital preservation and how new pay for storage models can help us not only save what's already published, but create new channels for all people to control and preserve their digital legacies.

If you are a digital archivist or have a general interest in the digital dark ages, check out this blog post we did with the Long Now Foundation and get in touch to tell us what you think!

If you are an open source developer who wants to get involved with work that can have a transformative impact on the preservation of digital content and our model for understanding history, or just like to get your hands dirty in shaping early-stage projects that have real scale potential, get in touch to tell us what you think! Or dive right into our GitHub repos!

If you are a content creator, family historian, or digital pack rat then check out Permanent.org and tell us what you think!

Additional comments

Relevant links/resources (optional)

Why we chose Etherpad as our demo application

Robert Friedman — Thu, 08 Apr 2021 17:39:49 +0000

When we were crafting the proposal that would become Storage to the People, our team contemplated the requirements that would make a good demonstration application for web monetized storage but left the contours of that application fairly broad.

Generally speaking, we wanted to build something that would scale easily. This might be because a baked-in audience for an existing open source app already exists or because the use-case for preservation was implicit to the workflow of an app and could be nicely inserted into existing workflows.

As examples, we considered streaming video conference apps like Jitsi, photo sharing apps for mobile devices (of which most well known examples are proprietary), email clients like Thunderbird, audio transcription projects like Hyperaud.io, or document creation suites like Libre Office.

The video streaming and audio transcription options were the most attractive to us. Fortunately, by the time our grant proposal was approved, we had already struck up a successful partnership with TheirStory, a video- conference-based oral history platform. Their application fit the contours of an ideal use-case and their commitment to a historical content capture mission makes them a natural partner for Permanent. You can learn more about our work together in this blog post.

Our partnership with TheirStory motivated us to create an SDK that would expose the private Permanent.org API to 3rd party platforms. This added considerable fuel to the GFTW fire at exactly the moment we needed it.

When we got started with our project, we quickly learned more about how the current web monetization ecosystem works. One implication was that we would be dependent on the Coil model for transactions. This narrowed our options because Coil calculates payments using time on site, not volume of data transferred. This meant our demo app would have to deposit files small enough in size to render any Coil payment sufficient to cover the cost of storing the file.

Then it dawned on our team: why not use something we already use in our own workflows. For the last year, the Permanent team has been relying on the riseup.net Etherpad instance to take open notes during team meetings in order to improve transparency and accessibility in our partner projects. The main drawback with Etherpad was that we could not easily save the files to a persistent storage platform. The workflow was just cumbersome enough that nobody was downloading the Etherpad files or copying the text to more durable repositories.

Etherpad also already has a baked in audience that would help our work scale. So it was settled. We consulted the GTFW Program Team and they gave us their blessing to go ahead with our prototype.

Web Monetization For Data

James Vasile — Mon, 15 Feb 2021 22:20:32 +0000

Much of the web is ephemeral. Your digital social life streams by, and every time you look, it's changed again. So much of our online selves disappears, even the parts we want to preserve. This is especially true on a multi-generational timeframe. Without your active participation, your online self slowly fades. Permanent.org's mission is to stem the tide, to help preserve your online self for future generations. We are excited to be working on a project to use web monetization to pay for storage of temporary text documents from Etherpad Lite. As part of that project, we are considering ways web monetization could extend beyond replacing ads.

We come at this with an eye toward long-term sustainability. Permanent is a non-profit focused on helping people preserve their digital legacies across generations. We steward permanent storage of your documents, media, and data, and are working on a set of tools to help you manage digital archives for posterity.

Our web monetization project explores new ways to integrate Permanent storage into services on a micropayments basis. We're starting with Etherpad Lite, an ephemeral service that hosts temporary text documents ("pads") and enables collaborative editing. By integrating web monetization and Permanent, we can make pads endure. Your work ends up safely in your Permanent archive, and eventually you will be able to take it to other Permanent-enabled Etherpad instances. Our work will thus enable some short-term data resilience that leads to long-term preservation.

Etherpad Lite is just the start. In the longer term we will look for more ways to improve temporary services by adding portable, permanent storage.

What's more important than what we're building is why we proposed this project. We're having a lot of fun adding Permanent, portable storage to Etherpad Lite documents. It's so much fun, in fact, that we have to keep reminding ourselves we're not actually here to just build nifty features into Etherpad Lite. This is a research project. The real goal is to better understand how web monetization works with new types of online services.

We don't quite fit the typical use for web monetization described on the Coil website. We're not monetizing content. We're not replacing an ad-supported business. The dominant web monetization infrastructure offered by Coil does not quite match up with our needs. Identifying all those gaps is why we're here. By the time we're done, we hope to have a good idea of what the missing ecosystem pieces are, and that is the deliverable we're most excited about. It's a roadmap to expanding web monetization to new types of offerings. Once we have that, the real work (and the real fun) begins.

If you want to follow along or jump in and play, the work lives on github. Come join us!