The Definitive Guide to Understanding The Mercury's Stack Vision
Learn about our innovations at a technical level and what it means for the whole ecosystem.
Last updated
Learn about our innovations at a technical level and what it means for the whole ecosystem.
Last updated
It has been a while since I wanted to publish a part like this one in the documentation, and I think that right now is the correct time to do that since I'm seeing many infra interested in building for Soroban.
Let's start by a very important point behind our tech.
A fairly common misconception in the space is that Mercury is closed source and cannot be self-hosted. While it's true that parts of our codebase are closed source and our data layer is hosted on a centralized database there are a few considerations to be done:
The core processors that work with data are open source.
The data that is input for these processors is data that is directly derived from the network.
Does this mean that I can univocally trust Mercury's data to be correct and not tampered?
No.
This however means that you can assert the validity of Mercury's data to be correct and not tampered, hence univocally trust it or not.
How so?
As long as you trust the underlying network, you are trusting the data that gets directly fed as input for Mercury's processors. Mercury's processors (the zvm and svm fork) are open source, i.e you can run what mercury runs at a low level on your computer. This means that you can audit the effects and past data that is held on Mercury whenever and however you want.
Can this be done right now?
Yes and no. While it is possible, there's currently no verifier running. There's a couple of reasons for that:
different clients might be looking into specialized data to be verified, creating a universal verifier for a downstream system is not simple because all clients have different requirements.
there isn't demand for us to be creating various kinds of verifiers that aid different kinds of clients. If however you're a client and, even if there's no organic demand, would like to have one developed and are up for sponsoring the development let us know.
Our stack is open source and outcomes of our cloud computations verifiable.
What I think is difficult to grasp as a concept is that our VMs, more specifically the Zephyr Virtual Machine pose as a completely new way for developers to interface with blockchain networks.
Building anything as simple as a soroban transaction builder is simpler when run on Zephyr as compared to running a script that fetches data from the RPC and simulates + builds the transaction client side. When talking about transaction builders that behave in response to downtstream data or alert systems, then there's no room for competition.
This is because of a few things primarily:
The zvm is a (developer) abstraction layer. All actions you can think of are either facilitated on the host or on the sdk. Not to mention the ease when working with types and contract sdk helpers (for stellar and really any rust smart contract ecosystem).
Programmable multidimensional data availability. You can access both state and historical (custom aggregated) data at the same time, in the way you want to.
Significantly scalable. Having programmable network interfaces historically meant that anyone who wanted to run their programmable workflow should have hosted a server, full node and related configurations. This is however not the case with Zephyr. As a sandboxed execution environment, you can deploy multiple workflows to various providers.
[plus] The zvm is heavily memory based as a VM. While this makes it less efficient that strict host-based VMs (like the svm), it makes development significantly easier. You can use pretty much all of rust's crates, the std, etc.
Zephyr is not Mercury. You can run it self-hosted or on our instances. If you're building automations, data infrastructure, aggregations, RPC forks, monitoring, etc, I highly encourage you to use Zephyr. Again, there are a few main reasons for this:
It will reduce your workload by at least 80%. I've seen a lots of projects that are trying to re-invent the wheel when it comes to accessing data, formulating automations, or similar processes. This will only likely lead to less efficient/functional implementations that have taken much more time to develop.
Increases the ecosystem's growth. If there are features missing or things you don't like about zephyr, you're welcome to contribute. Even just sharing what it is that you do not like or features that you'd like to see is enough. Together we're stronger, and can make both zephyr, Mercury, and your own products better.
Don't want to work with Rust? Zephyr's dependance on rust is relative at guest-level, you can build with any language that compiles to webassembly, and the SDK is dependant on a small set of host functions, making creating SDKs for other languages (python, typescript, etc) fairly simple.
If you're building anything that interfaces in some way with the network, you can DM me either on discord, telegram or X (heytdep).
This is a bold take that needs some arguments to back it up.
RPCs are a crucial piece of the infrastructure. They allow you to simulate and submit transactions, interact with the chain, get events, blocks, etc. The only fundamental flaw that the jsonrpc API has in the first place is that it's fully static. You can't programmatically work with these APIs, and for any significant workflow that goes beyond showing events or submitting transactions you'll have to add logic client side or server side.
This means higher latency for obtaining what you actually need, more codebase to maintain, client-side code/dependency mess, and much slower development times.
What if you could do what the RPC does but programmatically?
At its core, many of zephyr's functionality isn't too different from the RPC's. Both have ledger state/chain state and transitions/changes access and do processing on it. The main difference is that one executes a static hardcoded process when processing transitions or user requests, while the other one (Zephyr) safely runs custom processes defined by arbitrary code.
I would personally love to start seeing developers (that aren't xyclooLabs) using zephyr to build full APIs for their network interfaces. This doesn't mean that they should not have a backend and just completely rely on Zephyr instances. Most produciton projects do have a backend and multiple servers that execute various workflows. But there are better alternatives that just reinventing everything.
If you don't want to depend on any kind of service (not a 3rd party RPC or zephyr instance):
use zephyr locally and connect it directly into your own database through the db abstraction implementation.
If you are up to relying on 3rd party services (e.g are ok getting data from a 3rd party RPC) but don't want to have the database and data retrieveal API on an external server:
deploy a zephyr program that does all the custom processing you require and write to your database. This can currently only be done through web requests, but see .
If we see that our vision is shared and our work further supported, there are some clear next steps for us to go towards a direction where zephyr becomes more popular to write network interfaces:
Right now, shipping fully featured instances that can run locally might be a bad decision business-wise for us. Bigger and funded companies can just take advantage of our tech without proper retribution and (potentially licensing). However, in a setup where there is enough knowledge about Zephyr, who is behind it, and where there's teams or funding backing it up, we'd love to work on creating fully featured ready-to-run zephyr instances that are modular and composable to help the host customize the instance's behaviour (VM stack, caching, database, etc).
This would enable companies, users and dApps to run zephyr infrastructure that anyone can use to:
index data.
write transaction simulation endpoints.
APIs
automations.
And depending on the instance setup, users will be able to choose between the provider, the database implementation, etc. This makes zehpyr completely trustless. Instead of having dApps choose which RPC to connect to, they can choose which zephyr infra provider to connect to, and incredibly simplify the client-side development, speed and latency.
While I've worked hard already on improving the developer experience, there's always more you can do, especially in a complex project such as the zephyr virtual machine. Some of these improvements are:
Better webapp UI and functionality.
More freedom and customizability around database configurations, reads and writes.
Key-value object storage.
Improved CLI.
Dedicated instances.
Better error handling and debugging.
Local testing for ledger-state based programs.
Data streams and custom database reads are a major milestone for a relatively low-hanging fruit. Due to the highly modular design on the ZVM, we can support any kind of database and read/write target. This is work we intend to prioritize in the future once we reach a sustainable state.
This is a tough topic for us. On one hand, our success in the Stellar ecosystem lies in the deep technical knowledge we have around it. On the other hand, zephyr's highly modular design makes it perfect to integrate with as many chains as possible. If you're a team with deep technical expertise of another chain, please reach out to us!
I've mostly talked about zephyr here, but Retroshades still remains the to-go way to approach data indexing on Soroban specifically. It is the most advanced, fast and technically sound way of indexing your smart contracts. The same thing that I've said about zephyr instances also takes into account Retroshades!
I expect to be updating this section of the docs quite frequently, so I suggest to check this every now and then if you're interested in knowing how our vision is expanding.