The Interledger Community 🌱: Jan Zheng

Instill Science — Grant Report #2

Jan Zheng — Thu, 02 Feb 2023 00:22:22 +0000

Project Update

Instill is a community-driven project aimed at helping early- and mid- career scientists advance their research careers.

Initially, we were excited about becoming a micro-publications and for highly technical topics, like procedures for performing specific calculations, assays, and setting up experiments. Going into last year, we heard from many early researchers, especially those from the Global South, that they wanted some help getting their manuscripts cleaned up for journal publication. For researchers, getting a paper published in a decent journal can set the trajectory for the rest of the careers, so we pivoted to the “Peer Preview” system to help more researchers get published in better journals.

In academic publishing, once a raw “manuscript” submitted for publication, other scientists recruited from the field volunteer by reading and giving feedback for the author (this is called “peer review”). After a few back and forth emails between the peer reviewer, author, and sometimes journal editor, the paper is either accepted for publication or rejected from publication.

Our “Peer Preview” system intended as a quick “temperature check” to see if a paper was ready to be submitted for the official peer review process, which can take weeks to months to publication. We partnered with a nascent phage journal (Mary Ann Liebert’s PHAGE journal), and paired our community of experienced researchers with early researchers to Peer Preview their papers. The interest for Peer Previews was overwhelming, but we quickly realized that looking through papers before publication wasn’t very helpful. Many of the manuscripts we saw had fundamental shortcomings in experimental design and thesis, which required more than just copy editing.

Towards the middle-end of last year, we realized that we’d have trouble scaling up the more narrow “Peer Preview” system, since many papers needed more discussion on the scientific approach and experimental design. We instead decided to take the best parts of our learnings and lay them on top of familiar community tools like Stack Overflow and dev.to. Essentially, we realized that the way forward was to merge our v1 and v2 concepts, and build an Academia-centric system that is heavily inspired by existing tools. Once this platform becomes technically more mature, we will re-create but generalize the “Peer Preview” system to cover all sorts of abstract and scientific reviews.

In the latest iteration of the platform, currently live at https://instill.xyz, we built a concept we call a “super-portable micro-community” where users are able to share research, pose questions, and discuss science. The platform currently mimics popular forum-like interactions like Hacker News, but we’re using this platform as a launchpad to prototype better research-specific user flows, based on our past and future learnings.

Progress on objectives

Last year, we continued launching and supporting more academic, community events. Overall, we spent a lot of our time and efforts building out prototypes for peer review and hosting academic events to get an idea of how to support peer review at scale. We built a system that received hundreds of abstracts to be reviewed by a small group of peer reviewers.

Figure 1. Abstract submission portal and Retool-based Abstract Assignment dashboard (abstracts not shown due to personally-identifiable information)

Our system, based on the Peer Preview system built on top of our Airtable and Retool interface we submitted in our previous progress report, was able to handle 600+ abstract submissions and poster uploads from researchers around the world for VoM (https://isvm.org), a virus conference in Portugal of almost a thousand attendees. The system allowed a small committee of senior scientists to quickly approve and assign the abstracts into various tracks of the conference. We also successfully hosted a large virtual conference series for VoM called “iVoM” (1100+ registrants) in which we experimented with various community interactions (to various degrees of success).

The Super-Portable Micro-Community Tool

We also built and deployed our “super-portable micro-community” forum tool, which is currently hosted on our homepage (https://instill.xyz).

After the Elon-Twitter acquisition, there has been an explosion of Twitter-like, forum-like, and blog-like projects like Post News, Postcard, t2.social (among others), and an eagerness to try these tools out. There’s a large contingency of the “Science Twitter” crowd that we’re aiming to tap into.

Inspired by the movement towards decentralized spaces like Mastodon, we decided our community system and its data should be easily managed by the owner of the community. It should be easy to moderate and manage, and use familiar tools. Unlike Mastodon, we don’t necessarily the contents to be decentralized. Instead, we need to make it easy for many groups to create their own “flavors” of communities where they can control what goes on in their own community (not unlike old-school forums and bulletin boards).

Fig 2. Version 3 of the Instill community. The community currently focuses on the bacteriophage research and phage therapy community; the screenshot on the left has been cropped to be shorter, as the main page is too long because pagination hasn’t been implemented yet

We made some early decisions that this tool should be very easy for anyone to clone and launch as their own, with their own backend. This meant that we didn’t want complicated or expensive setups like Docker. We wanted it to be embeddable in as many places as possible, including places like a Notion page or as a mere iframe. We also wanted users to have full user-friendly control over the data, which is why it uses Airtable as the data source.

Of course, this does mean we will run into scaling issues very quickly, if we were to get lots of users. The benefit of being Academia-focused is that we’ll probably never run into scaling issues (joking aside, we will definitely be working on data adapters for more scalable databases like Cloudflare D1, Pocket Base, Supabase, LiteFS but maybe also even less-scalable ones like Notion and Google Sheets).

Currently, our system is live, and we’re slowly onboarding more users while squashing bugs and redesigning flows. Our current community is specifically aimed at the phage community, but we are happy to add communities more as we’ve fleshed out most of our user interactions. We’ve implemented a few interactions like like creating a Post, a Link, a Q&A-style “vote for the answer” flow, poll flow. We’re now making it easy to add and change interaction styles through a configuration file, so the community can easily morph from looking like Twitter to something like Hacker News or Instagram.

Towards the end of the year, we get to build and test our workflows for a couple of academic conference events, and a “Biobank Working Group” initiative, where we will use Instill to aid in scientific discussions and get a couple of manuscripts published.

Key activities

In the past year, we’ve made strides in both academic community-building and in developing Instill out as a community platform:

Built and launched a completely “portable, customizable, embeddable, micro” community concept that takes after inspirations like Hacker News, Stack Overflow and dev.to. Here’s what we mean with our terms:
1. Portable means that any team should easily and cheaply create and manage their own community space. This led to decisions to create an API-first community framework deployed as a serverless Sveltekit project on Vercel (as opposed to using Docker, AWS, and so on). We also built the UI on top of “data adapters” that lets us connect tools like Airtable as the database (and eventually support “real” databases like Postgres). This portability eventually means other should be able to create their own Instill instances, using their own Vercel and Airtable accounts, completely separately from the core project, and with full control of their instance and data. We want the experience to be similar to launching old phpBB forums and Discourse, but without the technical overhead. In order to keep the data small and portable, we decided to keep the schema flat and simple, borrowing ideas from Reddit’s “RDF triple” architecture. This makes it easy to port the database into document databases like CouchDB in the future, but also unlikely places like Notion or Google Sheets. The main table tracks “Posts” — any comment, reply, or blog post is a “Post”. The second table holds User accounts and profile data. Events like “likes”, votes, and reactions are stored in the third table, which allows to relate any number of Events to any Posts. Events can be quantified, so the plan is to sync our IOU and tipping events to a ledger system like Formance or TigerBeetle, so that eventually they can be resolved and paid using Rafiki / the Web Monetization layer.
2. Customizable means that different instances should be allowed to set the “interaction styles” to the owner of the community’s liking. Interaction styles change the way users use the community — e.g. Instagram, Substack, Twitter, Postcard, and Post.news all have different “styles” and user interfaces but are at their core the same interactions and data model. Customizations should allow a community to combine the best of Substack and Twitter, to fit their needs. To address this design goal, we’ve applied an “API-first” principle where every user interaction is handled by an API, in which interactions are sent to the database using “data adapters”. This allows us to design and prototype interactions (like reactions, upvoting, polling, marking as answer etc.) through the API and Events table, rather than having to add custom tables and columns in the database.
3. Embeddable means that the tool should easily be embeddable (and fully usable) in any web environment. Similar to Disqus, Instill should be embeddable as a comment section to any content, like other Sveltekit projects, static web pages, or even in Notion pages. (I tried to get iframe embedding working in dev.to and the Interledger community, but couldn’t get it to work, since it doesn’t seem to allow for iframes). Getting embeddability to work meant having to create a slightly unfamiliar account sign up and sign in pattern, and then designing every interaction element around that pattern. For example, because the app needs to operate in iframes, we can’t rely on local storage or cookie data to persist — so any forum interaction like posting a topic or comment needs to potentially prompt the user for account information. Every time a user “posts” anything to the API, it needs to re-verify the account. We’re hesitant to add complex caching, key-value or redis on top, as that would make the system more complicated to deploy.
4. Micro means that we want to design around small communities first, and being ok with using tools that aren’t scalable. For example, we want anyone to be able to make the community and data their own. This usually means that the data can’t be that large, and consequentially the community can’t be that large (for now). We want users to be able to “bring your own database” — hence the use of Airtable to store all data, including account information and encrypted passwords: using Airtable lets users easily moderate their communities. Though in the future we’ll adapters for other tools like Google Sheets and Notion, but also other database projects like Pocket Base, D1, and Supabase, for now the idea is to be not scalable, but be able to easily support a small community of ~300 active users, which is the size of our Slack group. Few academic communities ever grow past 1,000 users, so we don’t need to address the scaling issue at the moment.
In terms of attracting users and fulfilling our mission of making research more equitable, we’ve:
1. Worked to encourage users to post about their lab issues and small findings. This has proven challenging, since asking the community questions implies that “you are junior and inexperienced.” The tech field has somehow managed to overcome this barrier with Stack Overflow, but much of academia still runs on appearances and status games instead of getting results. To get around this, we’re collectively creating a Biobank Working Group to address issues that generally everyone in the field has, and eventually posting questions as the Working Group. This should hopefully make people less afraid of posting questions.
2. Helped researchers improve their “Peer Preview” submissions by giving them tips on how to improve their papers and research. Peer Previews are what we called our system for peer reviewing preprints (drafts of scientific manuscripts before they’re submitted to an academic journal). This proved tougher than we thought, as the relationship quickly turns into a “mentor / mentee” relationships, which becomes hard at scale. Even with the systematic peer review interface, we couldn’t get to “automating” the peer reviews because each paper was so different. In the end, we thought it might better to shift focus toward answering questions about basic scientific approach, rather than fix the paper. If we can change the way the researchers think about the approach and framing, we can catch problems before they make it on paper.
On our Science Communication side, we’ve:
1. Prototyped and built a system for “massive peer review,” which received 600+ abstracts and let a core group of scientists accept and place each abstract into various tracks of the Viruses of Microbes conference in Portugal. The system also generates a database directory of all posters, and creates an abstract book printout of each abstract. (Screenshots in the previous section)
2. Successfully hosted a virtual events series with almost 1200 registrants (and 400~ active participants across events) (Screenshots in the previous section)

Communications and marketing

In order to gain trust and relevance within academia (specifically the phage therapy space), we’ve collaborated with academic societies on several fronts. For our partners, we’ve built science education tools that cover lab basics; we built a scientific abstract and poster peer review system for a major European virus conference with almost a thousand attendees and we launched a couple of online science communications series.

The overall purpose was to establish strong relationships, and build trust within the established inner circles, understand the user needs (of both young and experienced researchers, as well as publishers and academic societies) within our space, and indirect marketing, to get our users familiar with using our tools.

By launching science-focused events and tools with our partners, we can both get valuable insights, and also embed ourselves as the technical and community partner of these influential organizations.

Viruses of Microbes: iVoM Virtual Phage Conference Series

https://ivom.phage.directory — We were central to organizing an academic-centric conference, posting videos online, eliciting questions and generating responses from the audiences.

Viruses of Microbes Poster & Abstract Upload & Review System for Portugal conference

https://vom.phage.directory — The goal of this system was to explore ways to scale up Instill v2 in a way that allows a small staff to review a large number (though shorter, in abstract form) of scientific content. The system handled more than 600 individual submissions from around the world, and using a panel of judges filtered and selected submissions into various tracks for the Viruses of Microbes conference in Portugal

What’s next?

For this year, we are continuing to build tools, host events, and experimenting with building community within academic research. In the last few years, we built one-off prototypes in tools like Retool to quickly iterate and learn from our users. This year, we’ll integrate these learnings into the core Instill community platform.

Most of our efforts this year will go into expanding features on the platform, and using the platform to accomplish a few use cases. We’ll also be expanding Instill to support hosting educational material and videos, and hosting live event streams before, during, and after academic conferences (like a small conference-specific Twitter thread).

Biobank Working Group

We are working with a few research labs and academic societies around the world (ISVM, Phages for Global Health, Phage Australia) to create a working group to discuss and establish standards on building phage and bacterial biobanks, which are collections of phages and bacteria. These discussions will involve things like lab standard operating procedures, lab safety, data collection, shipping and handling, paperwork, etc.

We are planning to host moderated and guided asynchronous discussions on Instill throughout the year, bi-monthly virtual calls, and an in-person meetup at the 2023 Viruses of Microbes meeting in Tbilisi, Georgia. The meeting notes, agenda, virtual meeting videos, and other materials will all be posted within the Biobank Working Group page on Instill, and every post will receive DOIs and permalinks, and will be citable units for the standards papers that will published by the group. All of the posts will be publicly available. If all goes well, we aim to publish our first paper either in Q3 or Q4 of 2023.

Phages for Global Health Online

In 2021, we partnered with Phages for Global Health to build a micro site with a curriculum designed around teaching lab skills for working with bacteriophages (https://open.phage.directory). We spent last year rethinking how to reorient the curriculum and content to be more community-centric, including encouraging user-uploaded videos, benchmarking tasks (e.g. different groups around the world perform a lab task like a plaque assay on the same phage and bacteria, and post their results with the rest of the group). We hope to relaunch PGH Online this year, with our new curriculum, towards this new direction. We are also building out the platform to support “playlists” in order to group content and video posts around various topics.

Evergreen Phage Meeting 2023

In 2021 we also created and hosted Evergreen, a hybrid phage conference in Olympia, WA. We experimented with some community features, but eventually moved all conversations to Slack. This time around, we will use Instill as the basis for before- and after-conference conversations. We’ll explore setting up conference presentations to be released recorded before the conference, so we can maximize face-to-face meetings during the conference. Just like last year, conference posters and abstracts will be searchable and tied to attendees presentations, but this year we want to add things like poster voting, polls for activities, and “pop quizzes” for different talks during the event, to make it a bit more interactive for those tuning in from elsewhere.

Expanding Instill Features

While we’re keen to build out our partner sites, we’ll be adding more community interaction styles, like short-form text posts and threads (e.g. tweets and tweet threads), image-only posts (e.g. Instagram), event sequences (e.g. similar to what The Verge posts during conferences or while covering Apple releases).

An area we are still excited to build out the points-based rewarding and bounty system we’ve outlined previously. Since we want to be end-users of Rafiki, we’ve been waiting for the system to become more stable and easy to integrate with. To allow us to develop tipping interactions separately, we’ve explored a ledger system like Formance (or TigerBeetle, a recently-launched project within the Interledger ecosystem). By using a database-backed, points-based ledger, we can separately test out tipping features without direct integration into a payments platform like Rafiki — instead, we can then “resolve” the transactions by letting users “pay out” the final amount, by connecting their payment pointers using the Rafiki and Web Monetization framework. Using a database ledger allows us to freely prototype and design user flows around payments with fake points, without worrying about running afoul of any laws. Ideally, we end up partnering with more mature groups using Rafiki, so we can connect our ledger system with their implementation (hopefully they’ve figured out the legal side of things as well).

Finally, we might look into launch another instance for an eager (and potentially non-academic) community to try it out (if you’re reading this and interested, send me a DM through Forem!)

What community support would benefit your project?

As we’re steadily adding use cases within academia and science, we’re eager to expand outside of academia. Specifically, we want to collaborate more with ILP, Web Monetization, and Coil-adjacent groups — in other words, this community!

We think the uptake for exploring the Events system, Formance ledger and ILP and Rafiki use cases will initially be much stronger and apparent in an Interledger community. Once the product reaches more stability, we’d love to work with other groups that have ILP and Rafiki figured out, so we can explore prototyping tipping and bounty interactions. Onboarding and getting to prototyping will be much faster with a community already familiar with Interledger; especially one already well-versed with Rafiki.

Towards the end of this year, we hope to:

Work with a couple Interledger communities familiar with Rafiki and that might have community needs and a “test net” set up for us to plug into (Snake Nation comes to mind!)
Find other Interledger projects which might roll out small-scale forum, Substack, or comments-like interactions for their products or platforms
Connect with Interledger projects interested in working within academic research biotech and deep tech, since we’ve gained a lot of insights into how to work with and deploy new projects in this space

Additional comments & relevant links/resources

Our core project website and main deployed instance for v3 can be found at: https://instill.xyz.

Github repo for v3 is here (still poorly documented, sorry!): https://github.com/janzheng/instill-v3

Our public-facing prototypes and projects from last can be found at:

iVoM, a virtual conference series for VoM: https://ivom.phage.directory/
VoM Abstract Portal: https://vom.phage.directory
Sadly, the peer review dashboards contain too much personally-identifiable information to fully share. We’ll consider to create a “sample deployment” at some point, if there’s interest

Our partnership projects we’re launching this year with Instill-integration can be found at:

Evergreen Phage Meeting 2023: https://evergreen.phage.directory
Phages for Global Health Online: https://open.phage.directory. Phages for Global Health, the partner organization, can be found here: https://phagesforglobalhealth.org
The Biobank Working Group will be a collaboration of members between the International Society for Viruses and Microbes (isvm.org), Phages for Global Health, and various other academic institutions (the list is still fluid)
Phage Australia Network* (phageaustralia.org) will host discussions, topics, and share data between member labs on Instill towards the end of the year.

Instill Science — Grant Report #1

Jan Zheng — Sat, 09 Oct 2021 07:13:54 +0000

Project Update

At the beginning of the grant, we had this idea that we could build a better, peer-led Peer Review system. We spoke to scientists who were eager to participate. But once we had manuscripts in hand, it turns out that people love the idea of helping others improve the quality of their science and research output — not the idea of conducting peer review. Even merely mentioning "peer review" elicited a lot of negative reactions!

We've had to change our approach several times, and re-think how we could create a science community that could help itself raise the bar for the scientific and publishing output.

Along the way, we've successfully organized a week-long, hybrid conference for 400+ bacteriophage researchers from around the world, many from countries in the Global South; we've built a rapid web app prototyping system for community building (and week-long, hybrid conferences) on top of Airtable, Notion, Retool, and Supabase; we've interviewed students, post-docs, professors, and journal editors about fixing Peer Reviews; and we've re-thought, pivoted, scrapped, and restarted our concept many, many times.

But now we're finally converging on an idea that seems to work, and we have 146 early adopters to try our latest iteration on! 😬 😅

Progress on objectives

Peer Reviews are a difficult subject

When we first started, we set out to build a community for more constructive, helpful peer reviews, meant for the manuscript authors. Most scientists have a complex relationship with peer reviews. Every scientist acknowledges that it's broken, and most have even thought about ways to "fix" it.

Peer reviews of today are both a feedback practice and a gatekeeping practice. As a feedback practice, they help authors improve their manuscripts and research, by suggesting copy changes, additional experiments, etc. As a gatekeeping practice, they are supposed to help science maintain its rigor and higher standards. In practice however, this amounts to an unpaid obligation that robs researchers of several hours, if not days, of valuable time. This practice also ends up selectively biasing scientific output towards Western labs, and creates large barriers that block out smaller, underserved and/or less-funded labs.

After talking to students, professors, and even science journal editors, we quickly understood that the term "peer review" comes with a lot of baggage. Everyone already has existing notions of what peer review is and isn't. Any mention of "improving peer reviews" was met with immediate negativity. Everyone balked at the idea of a community that encouraged more peer reviews. So we had to dance around the word.

Everyone we talked to loved the idea of a community that encouraged researchers to give each other manuscript feedback in an effort to improve quality. You just couldn't call it "peer review." We also found that many responded well to the idea of creating a structured, time-limited (30 minutes max) feedback system, where both the author and "Peer Previewers" understand the expectations going in.

We also found that generally, barriers to publishing high quality research exist prior to the point of publication. Usually, the labs that are having trouble publishing are also the labs that have trouble accessing expensive equipment (like sequencers) and data analysis capabilities.

So, after multiple interviews, multiple redesigns, and multiple reconsiderations of the core value proposition of Instill, we realized we needed to be a more generalized resource for scientists of all stages. We also realized that "underserved" researchers could come from unexpected areas, sometimes even from well-resourced labs.

Improving Access, Finding Collaborators, and Removing Barriers, for underserved researchers

Instead of just focusing on improving manuscript output and publication quality, we've zoomed out to look at creating better resources for science communities around the world. From this community peer review workshop series conducted across Africa (https://scholarlykitchen.sspnet.org/2021/08/23/guest-post-best-practices-and-innovative-approaches-to-peer-review-in-africa/), the authors point out the need to "provide opportunities for [members] to engage in debate-style learning, horizontal peer-to-peer mentorship, individual and collective reflection."

Through our conversations with phage researchers and with our partner Crowdfight, we realize that peer-to-peer training, mentorship programs, structured guidance, and efforts to increase representation are all necessary building blocks. Activities like both top-down or peer-led mentorship, peer-led discussions, as well as experimental and data sharing collaborations between scientists will lead to higher quality research output for all groups involved.

As such, we set out to find strategic partners that fit our mission. Through our partnerships, we hope to build a strong advisors and mentorship network.

Developing the right incentives

After scrapping the idea of "peer review" we also needed to rethink incentives. We found that researchers were not primarily motivated by money, but by winning "status games" that prove the credibility and contributions of their careers. These "games" include the publishing game, as measured in H-index, impact factor, and publication records, which all serve as a proxy for someone's research output. They can also involve other types of contributions that can paint a well-rounded career. These include mentorships, invited talks and seminars, teaching, and aspects of community outreach. (More on status games by Eugene Wei: https://www.nfx.com/post/status-games/)

Originally, we started with the idea that high quality, peer-led constructive peer reviews could be paid for, since they would help a lab save a lot of time and grief from the slow and unpredictable peer review process. Though scientists often like to posit the idea of paying for a peer review in conversation and in social media, this squarely did not resonate with anyone we spoke to, given a real scenario where they had the chance to be paid.

A payment structure, or even a tipping or bounty system paints the peer review as a job: as a "paid service to be rendered" rather than a community service chore that every scientist needs to perform "for the greater good." Any sorts of direct payment destroys the illusion that peer reviews are a necessary obligation. We also found that prices are very hard to establish for an audience that exists in virtually every economy around the world... $20 can be overwhelming for someone in Africa but also underwhelming for someone in Switzerland.

We also considered using Coil for performing tasks and "services" like peer reviewing, but because Coil only rewards time spent on page, it doesn't fit as a payment structure.

Instead, we were looking for Reddit, Stack Overflow, and Medium for inspiration from their coin and upvoting mechanisms, as a proxy for how much work an individual is contributing to the platform. Payouts could then be made based on the number of votes, which would incentivize more positive behavior from the members.

We also very recently came across an a16z article on reputation-based rewards systems (https://future.a16z.com/reputation-based-systems) that we could see working with our platform. We could potentially create a payout system based on amount of work performed (for example, number of Peer Previews conducted per month) relative to the entire "pot" of that month. The size of the pot could be relative to the number of participants, or members, of the community.

Implementing these ideas is difficult given the current state of Web Monetization and Coil, but we're very eager to build out some of these ideas in Rafiki.

Establishing a long-term community-led infrastructure

To address our requirements of building a sustainable, affordable, easily operated community specifically for scientists, we looked through the many options that currently exist. We found that existing solutions are either too complicated and expensive to set up (Forem), too opinionated (Discourse), or too closed or enterprise-facing (Circle.so).

We then turned to inspirations like Hacker News, Product Hunt, Stack Overflow and Quora, to identify how they operate at the intersection of "social media" and technical resources and knowledge bases, and created a content and feature inventory.

We had to develop our own architecture, based on off-the shelf tools that have generous freemium offerings, like Airtable and Notion, and using them to develop simple schemas that can easily be ported from Airtable to various other databases like MongoDB, Postgres, or even Google Sheets. We also needed the tool to be natural sciences friendly, which meant that both user and data interfaces need to be user-friendly.

Even though the current prototype stack runs on Airtable and Notion APIs, and is built on top of Retool, the idea is to eventually move to something more robust and free or cheap to host. The idea is to eventually run a Forem-like micro platform for small science communities, but with a fully no-code experience.

Key activities

Evolving Peer Reviews through User Centered Design

When we started, we knew that peer reviews are often a touchy subject, and that we would need to evolve our concept as we spoke to more people. During the design process, we interviewed a variety of members in our Phage Directory community, from PhD students all the way to senior professors and even journal editors.

Through several iterations and conversations, we landed on the idea of a "Peer Preview". The name is chosen to reflect and invoke the idea of structured, faster peer review model, meant to help manuscript authors get a second pair of eyes before submitting to a journal for full Peer Review.

The Peer Preview system starts out as a request from a manuscript author. Members of Instill respond to the Peer Preview request, and if selected, give the author structured feedback through the feedback form. As part of this system, we developed the "SCOR Card" system to help researchers quickly assess papers across significance, clarity, originality, and rigor, based on this paper: https://arxiv.org/pdf/2109.09774.pdf. Requesters are given a dashboard where they can monitor the feedback as it rolls in.

Building a formalized mechanism for finding the right collaborator

We are also developing a Collaborator Search system where we want to make it easier for researchers to find other labs to collaborate with, which includes sharing data, samples, performing new experiments, etc. in order to further develop research and experiments. Through our conversations, we found that many labs from the Global South lack resources and equipment that we take for granted in the West. This could range anywhere from Illumina sequencers to pipette tips, filters, and 24/7 access to electricity. Through a formalized mechanism to find trusted collaborators to send samples, we could help accelerate labs' research progress.

Designing Stack Overflow for natural sciences

Currently we run a Slack group for phage biologists (https://phage.directory/slack), and we monitor and answer several massive WhatsApp groups' phage questions. We have wanted to build a forum system specifically meant for natural sciences, that borrows the best ideas from Stack Overflow, Quora, Discourse, Circle.so, Hacker News, Product Hunt and Reddit. This tool addresses needs similar to those that Stack Overflow are addressing — the "basic" questions that everyone has, but don't know who to ask or who to turn to. This kind of initiative will hopefully help more researchers get past the years-long initial learning curve that most biology PhD students encounter. Furthermore, this platform makes more sense for Coil monetization than the other ones.

Building Communities & Partnerships

To get closer to our target communities, we've partnered with many communities in the phage biology space. We've found groups that align with our mission, and we all share the aim of improving access to science resources. Through the partnerships, we've learned about the challenges their members are facing, which have informed our product designs.

Our partners are:

Phagebiotics Research Foundation: https://www.phagebiotics.org, which runs the Evergreen Phage Meeting, and trains many undergrad students and researchers from underserved areas of the world.
PHAGE Journal, by Mary Ann Liebert: https://home.liebertpub.com/publications/phage/652
Phages for Global Health: https://www.phagesforglobalhealth.org, which trains many phage researchers in Africa and South East Asia
Crowdfight https://crowdfightcovid19.org, which boasts more than 45,000 community members and volunteers in the fight againt COVID

As we grow and develop our project, we aim to partner with and onboard more communities, such as the Africa Phage Forum, the International Bacteriophage Research Consortium, and others.

Evergreen, an international hybrid conference for phage scientists

Through our community-building efforts, we were approached by the Phagebiotics Research Foundation (https://www.phagebiotics.org) to be the technical partner for the Evergreen Meeting (https://evergreen.phage.directory). Evergreen, one of the major academic organizations in phage research, wanted to run a hybrid conference for phage researchers internationally, which we'd never run (or attended) before.

We joined forces because we both share the aim to provide access to mentorship and resources for researchers from underserved countries (India, SE Asia and African countries). This event introduced us the many new phage communities, as well as challenges that researchers from the Global South face.

For our partnership, we built the web infrastructure for conference that hosted about 500 participants from all over the world... in about two weeks. We devised a ticketing system, payment flow, organized world wide meet and greets, and made many introductions among many phage researchers.

To accomplish this, we devised a system based on Airtable and Notion that let us quickly and collaboratively input the data, like updating the schedule in real-time, adding conference abstracts and 100+ hours of webinars. This system eventually became the backbone of Instill's prototyping system.

Evergreen Github: https://github.com/janzheng/phdir-seminars-evg

Evergreen website (it's still live, and soon we'll make all webinars open access): https://evergreen.phage.directory

Technical Activities

We've built (or heavily modified) a set of tools that powers our rapid prototyping system. We originally looked at pre-built communities like Forem and Circle.so, but deemed them too limited, complicated, or too inflexible.

We therefore decided to roll our own group of tools that let us test our different iterations and value propositions early and quickly. In our setup, we use Airtable as the main document data source, Notion as our site CMS, Retool for rapid UI and workflow development, and Supabase for data like scores, upvotes, and comments. The entire system is deployed on Vercel as a serverless platform, with some code running as APIs on top of Cloudflare Workers. This setup keeps our server and website costs to an absolute minimum. **

The Github repos that power Instill:

Instill Github (latest version): https://github.com/janzheng/pdn-instill
Cytosis - an Airtable API wrapper: https://github.com/janzheng/cytosis
svelte-notion (forked) - A Svelte renderer for Notion blocks: https://gitlab.com/janzheng/svelte-notion.git
notion-api-worker (forked) - A Cloudflare Worker for fetching and caching Notion blocks: https://github.com/janzheng/notion-api-worker

Communications and marketing

Because peer review and manuscript feedback is a sensitive topic to many researchers, we haven't made a lot of announcements about our product launch. We launched our first version to our newsletter, Capsid & Tail (https://phage.directory/capsid/instill-launch), and tweeted about our product. In total, our early adopter list has 146 members. However, we haven't onboarded all of them yet, as we've been speaking with many of them one-on-one to make sure that our product is appropriately addressing their needs.

Most of our marketing, communications and lead generation are organic. Through our community and our partnership work, we are already connected with thousands of researchers within the phage research field. Our partnership with Crowdfight extends our reach to their network of 45,000+ scientists.

What’s next?

With a better understanding of how researchers prefer to give and receive feedback, we're onboarding users onto both our Peer Preview and Collaborator Search programs.

From the Product-side, we're:

Building a topics/commenting knowledge base system, along the lines of Stack Overflow and Circle.so, but designed for tight-knit science communities.
Continuing to evolve our Peer Preview and Collaborator Search concepts with more interviews with researchers and design iterations.
Designing and building an upvoting, incentive, and rewards system with our researchers. We plan to use both Web Monetization/Coil/Rafiki, in addition to designing ways to show how much impact each community member Is providing the community.

We're also excited to onboard hundreds of researchers from our waiting list, but we're doing this carefully as to not overwhelm our system. Once we feel the product is maturing, we'll be expanding our system into more research areas with the help of Crowdfight.

We will also document more of our learnings and findings, product ideas, and our technical work in our blog at https://instill.xyz/blog

What community support would benefit your project?

The most time consuming areas have been understanding how to implement advanced features of Web Monetization (e.g. direct payments), and other wallet technologies, past Coil features. The other very time-consuming area has been coming up with a simple flow of getting users to adopt a Web Monetized wallet and subscribe to Coil.

We would love if there were some technical communities, working groups, or workshops in the Web Monetization that have successfully built production-ready payment systems other than Coil. I'd also be interested to learn if anyone's thought about creating DAOs with Coil, Web Monetization, and XRP.

Also, if you'd like to collaborate in any way, reach out to us on Twitter (@phagedirectory)!

Additional comments

We're eagerly waiting to adopt a direct payment system (e.g. Rafiki). With a system like this, users could tip each other and create bounties for new work. With direct payments, the platform itself could also set aside a rewards system based on how many points users have accrued within a given time. The payment system could also collect membership dues every month, which would be used to sustain the project and pay back members.

And we'd like to thank all the professors, PhD students, journal editors for getting us to where we are, as well as the Grant for the Web!

Relevant links/resources (optional)

Here's a list of links mentioned throughout the report:

Our website: https://instill.xyz

The Github repos that powers Instill:

Instill Github (latest version): https://github.com/janzheng/pdn-instill
Cytosis - an Airtable API wrapper: https://github.com/janzheng/cytosis
svelte-notion (forked) - A Svelte renderer for Notion blocks: https://gitlab.com/janzheng/svelte-notion.git
notion-api-worker (forked) - A Cloudflare Worker for fetching and caching Notion blocks: https://github.com/janzheng/notion-api-worker

A few readings that have taught us a lot:

Inconsistency in Conference Peer Review: Revisiting the 2014 NeurIPS Experiment. https://arxiv.org/pdf/2109.09774.pdf
Best Practices and Innovative Approaches to Peer Review in Africa: https://scholarlykitchen.sspnet.org/2021/08/23/guest-post-best-practices-and-innovative-approaches-to-peer-review-in-africa/
A Novel Framework for Reputation-Based Systems: https://future.a16z.com/reputation-based-systems/
Status Games: Engineering Scarcity in a World of Abundance: https://www.nfx.com/post/status-games/

Other projects that inspire us:

Peer Community in: https://peercommunityin.org/
PREreview: https://prereview.org/
ASAPbio: asapbio.org