The Interledger Community 🌱: Sabine Schaller

Rafiki Updates - August 2024

Sabine Schaller — Wed, 14 Aug 2024 09:25:40 +0000

Once again, there has only been one Rafiki release over the course of the last month, Alpha 15. And there have only been two big themes, which are quite related: Telemetry and Performance Testing.

Telemetry

We have added the telemetry stack (OpenTelemetry collector, Prometheus and Grafana) as part of our local playground. This is also a first step to performance testing Rafiki. Soon, we will be able to run load/stress tests in the local playground, and be able to see the metrics data from our tests.

We also added a histogram metric to the telemetry service and use it for measuring the completion time of Open Payments outgoing payments. These metrics related to transaction time have been added to our public Grafana dashboard. Here is a preview:

Performance Testing

As a first step, we added the ability to collect traces in Rafiki, specifically for HTTP requests, our GraphQL Admin API and Postgres. These traces are pushed to the OpenTelemetry collector, which proceeds to push traces to Grafana Tempo. This leads to some nice visualizations:

Outlook to our next release

The next release will include a couple more telemetry metrics and a k6 script to run performance testing. Additionally, it will include an upgrade to the latest Tigerbeetle version which also led to a bunch of optimizations within Rafiki.

Rafiki Work Week 2024

Finally, I don't want to miss the opportunity to give you some details about our upcoming Rafiki Work Week. During the week of August 26, about 30 of us, including guests from JoPACC and Peoples Clearing House (PCH), will gather in Cluj, Romania, to work on the following topics:

multi-tenant Rafiki
settlement simulation in the test wallet
documentation
and whatever is urgently required by JoPACC and PCH

I can't wait to share the outcomes of the week with you next time!

Rafiki Updates - July 2024

Sabine Schaller — Wed, 10 Jul 2024 10:40:49 +0000

Summer time is here, meaning that we are moving at pretty much only half speed due to multiple events and vacation time. However, we still managed to put one Rafiki release out there - Alpha 14.

Updated Error Handling

We simplified the Admin APIs by moving error statuses out of mutation responses, and instead, we now throw proper GraphQL errors. That is obviously a breaking change, however, it allows us to easily enable additional logging on the Admin APIs.

Update of Environment Variables

Another breaking change removes environment variable defaults and requires them to be set by the integrators. This ensures that especially sensible variables are thoroughly considered before being set, e.g. ILP address, the Rafiki instance name, or whether telemetry should be enabled or not.

Additional Properties on the Wallet Address

Open Payment wallet addresses allow for additional properties when requesting them. However, so far, Rafiki did not allow ASEs to add any properties to the wallet address that were not required in Open Payments. We have changed the wallet address creation API to accept additional properties. Furthermore, ASEs can specify whether they want those properties to be for internal use only or whether they want them to be returned on the Open Payments wallet address GET request.

Upcoming Changes

We are currently working on adding more telemetry metrics to the collector. Once that is done, we'll shift our focus to performance testing - the actual last requirement for a Beta release (pinky promise 😉).

Rafiki Updates - June 2024

Sabine Schaller — Wed, 12 Jun 2024 08:37:52 +0000

Slowly but surely, conference season and vacation season is creeping up on us, meaning that we are moving slower than we had been hoping to. But it's not like we've dropped everything and gone to the beach. We still put out Alpha 12 and 13, so let me introduce you to the most important additions and changes.

Breaking Changes

A first major change we made was to move the grant interaction endpoints to a new server on a different port. To clarify, we refer to the grant interaction endpoints as the endpoints that the identity provider consumes to inform the auth server about the user's decision. Previously, these endpoints were on the same port as the GNAP endpoints. Now, in the spirit of introducing more security measures, the account servicing entities do not have to expose the grant interaction endpoints anymore but can keep them private to their internal network.

Additionally, we removed some of the environment variable defaults on the auth server because we were seeing a bunch of nodes coming online that only used default values which can be very harmful (and costly) when done in production.

Additional Properties on the Wallet Address

Previously, the only property that could be passed when creating a wallet address via the Admin APIs was a public name. The Open Payments spec, however, allows for any additional properties. We have updated the Admin API mutation to also allow for additional properties to be added to the wallet address.

❗WARNING: Be careful what you add there since that endpoint is not auth protected but available to the public!

DevEx Enhancements

Since we want to invite our community to also contribute to Rafiki, we spent some time improving our developer experience. Previously, whenever code changes were made in Rafiki, one had to re-build the code to try it out. We have now introduced hot server reloading within the localenv docker containers whenever code changes are detected. That way, new features or bug fixes can be tried out and debugged on the fly.

Making the MASE a better place

We have always neglected the Mock Account Servicing Entity, a.k.a. MASE, because we were just using it for early demos and sometimes for debugging. However, we did realize that people were looking at the code for integration inspiration so we decided to give the MASE a face lift. Thanks to one of our newest team members Arpi, it now looks and feels more like an actual bank or wallet interface and also adds some features like adding and editing accounts from the frontend.

What's upcoming

We are still working on the refactor of the backend Admin APIs that will change error handling. Furthermore, we are also in the process of removing harmful environment variable defaults on the backend. Once these two breaking changes are merged, we should be finally ready for a Beta release. We are closer than ever but we also want to be diligent in the process so we have been making sure to fix a bunch of little bugs on the way.

Additionally, we have started working on collecting some additional telemetry metrics that will give us even more insight into the network. Stay tuned!

Rafiki Updates - May 2024

Sabine Schaller — Wed, 08 May 2024 08:25:48 +0000

Still no beta 🐌, but 2 new alpha releases we can share with you, Alpha 9 and Alpha 10. Let’s have a look at what’s new and what’s the holdup.

Autopeering is fixed

We don’t know if anybody even noticed but Rafiki’s local playground autopeering didn’t work for quite some time because tunnelmole disallowed multiple open tunnel connections at the same time without a subscription. Since we didn’t want every developer looking into Rafiki to buy a tunnelmole subscription, we looked for alternatives and landed at another open source tool called localtunnel. While localtunnel requires user confirmation when accessing tunneled front-end pages, Rafiki is mainly backend software, hence localtunnel worked as an alternative for us.

Tigerbeetle production release

While we are still trying to finally make it to Beta, our friends at Tigerbeetle have released their first production release! Congratulations to Joran Dirk Greef and team 🥳! Of course we also upgraded our version of Tigerbeetle that Rafiki comes with.

Open Payments Quoting is optional

We have always introduced Open Payments as a three-step process.

Create an Incoming Payment
Create a Quote
Create an Outgoing Payment

While this makes perfect sense for e.g. e-commerce payments, i.e. once-off payments. It doesn’t necessarily work that well for recurring payments, e.g. subscriptions or Web Monetization payments. In these instances, payments are pre-approved based on an existing outgoing payment grant. This means a user does not need to explicitly approve a payment on a consent screen, meaning we can remove the extra round trip for quote creation, since it does not add value for the client.Hence, we have allowed for the creation of outgoing payments without a quote if those outgoing payments are created with such a long-lived grant. The ILP rate probe still happens under the hood, however.

Security features

This was always supposed to be the last missing piece before we moved to Beta. And don’t get me wrong, it was definitely the biggest missing piece. We have secured the Admin UI using Ory Kratos and the Admin APIs with a shared secret between the ASE and Rafki, that is used to sign request signatures, just like webhook requests are currently already secured. If this secret is set within Rafiki, it expects all incoming Admin API requests to be signed with that secret.

What’s the holdup?

So if the security features are done, why is there still no Beta release of Rafiki? The main reason is that we are currently working on a couple of breaking changes and Beta is supposed to be at least somewhat stable. So let me walk you through those upcoming breaking changes:

We are moving the grant interaction endpoints to a different server/port. That way, this port does not need to be exposed because it’s only communicating with the ASE’s identity provider.
We have opted to transition away from treating errors as data within the GraphQL Admin APIs and we are now returning errors as described in the GraphQL specification. The reason for the transition was the inconsistency in queries (that didn’t treat errors as data) and mutations.

🤞There will be a Beta release next month! 🤞

Rafiki Updates - April 2024

Sabine Schaller — Wed, 10 Apr 2024 07:22:08 +0000

A month has passed and a new release was just published - Alpha 8. Let’s zoom into what’s new.

Integrator Goodies

For those of you running or planning to run Rafiki yourselves, we have added two features that give you more optionality. First of all, you can disable Rafiki’s support for SPSP, in case you won’t allow direct ILP access for third parties (which is probably the case for almost all integrators). Furthermore, we have added the ability to run database migrations manually. With this in place, if Rafiki is installed on kubernetes cluster, it is possible to leverage Init Containers for database migrations.

Integration Tests

This last release of Rafiki Alpha 8 also includes a set of integration tests that go through several Rafiki to Rafiki payment flows, check that all the endpoints behave as expected, and that all the webhook events are fired accordingly. These can be triggered manually, but they are also part of our CI pipeline. We plan to expand these tests and are open to suggestions on what to add.

Release Pipeline

Thanks to the newest addition to the Interledger team, @golobitch, we are ready to move our releases to the next level. As soon as Rafiki is ready for Beta, we have an improved and flexible release pipeline in place to automate the process using Github Workflows.

Security Fixes

So, “when is Rafiki ready for Beta?”, you may ask. The answer to that is: As soon as all our security additions are merged in. Alpha 8 already contains some security features added to our GraphQL servers, easily enabled thanks to GraphQL Armor, that limit the depth of a query and disables field suggestions. Additionally, we set the X-Frame-Options header to SAMEORIGIN for the Rafiki frontend to prevent clickjacking.

We currently also have an open pull request to secure the Rafiki Admin UI with Ory Kratos. Finally, @feraltc is busy securing the Rafiki Admin APIs.

Upcoming Changes

I really hope that this month is the month that we release our first Beta version. We will definitely push for it! Additionally, we will be working on making quoting within Open Payments and Rafiki optional to allow for a smoother flow in Web Monetization. Finally, we will add a bunch of logs to Rafiki to ramp up our developer and integrator experience.

Rafiki Updates - March 2024

Sabine Schaller — Wed, 13 Mar 2024 08:35:40 +0000

Big news! We have released 2! new versions of Rafiki - Alpha 6 and 7. Alpha 6 was the feature heavy release and we’ll focus on this one here. Alpha 7 was a follow-up release to fix a migration bug that Alpha 6 introduced. For more details, read the Alpha 6 and Alpha 7 release notes.

Telemetry

The biggest feature within the Alpha 6 release is our new telemetry feature. Rafiki will collect, sanitize, and report test network data to a central service run by the Interledger Foundation. This allows us to gain insights into the adoption of ILP and the growth of the network in general. We are testing it in testnet first, before we release this feature for livenet deployments of Rafiki. If you want to learn more about our telemetry, please check out our docs.

Payment Liquidity

Alpha 6 also deprecates event liquidity in favor of payment liquidity. So far, liquidity management for incoming and outgoing payments was handled based on the webhook events fired when these Open Payments resources were created and completed. From feedback we have gathered from our integrators, payment liquidity is considered to be more intuitive. Hence, we have deprecated the event liquidity endpoints and added new endpoints to deposit and withdraw liquidity from incoming and outgoing events. For more details, check out the Admin API documentation.

Postman adieu, hello Bruno

Over the months, we have become more and more frustrated with Postman and its hybrid synchronization with its own servers and Github. Furthermore, generating request signatures for Open Payments in Postman pre-request scripts has been cumbersome due to the limitations of WebCrypto. And let’s not get me started on the costs of Postman! We have now fully completed our transition to Bruno, an Open Source API client that only syncs using git and uses very readable collection files. Additionally, it allows us to load npm packages, which will remove the dependency on an AWS lambda function for signature generation in our pre-request scripts. I call this a total success!

Upcoming Changes

It’s time to tighten our security. We are working on securing Rafiki’s Admin APIs as well as the Admin UI. Additionally, we are introducing protection against commonly known GraphQL attack vectors. With the help of our new devops engineer, Tadej, we will improve our release pipeline such that we are ready for our first beta release as soon as all those security issues are fixed.