Traditionally, nodes assume one of two roles: “transmitter” or “producer/transmitter” of blocks, however, there is a third role that must be treated separately, and that is the one played by nodes whose purpose is to resolve queries about the state local, or serve as a data source for tools such as accounting data.
This development consists of a new node to maintain an up-to-date copy of the ledger and respond to customer inquiries, while requiring a small fraction of the network resources compared to the aforementioned nodes.
I published an article about this development almost a year ago (1).
A healthy Cardano ecosystem will require node diversity to make the entire ecosystem more resilient, and an alternative Rust node would be ideal.
However, it is a complex task that requires planning and many resources, such as a multi-year project with multidisciplinary engineers on the team. That is why it is not possible to achieve the goal with a single Catalyst proposal, and it is necessary to divide the project into several parts.
The Proposal
This proposal presented by the TxPipe team, in Catalyst’s Fund10, was included in the OSDE category: ‘Open Source Dev Ecosystem’.
The team intends to build on Dolos new functionality, moving towards a full implementation, and as the next big feature, plans to add business logic to perform “Phase 1 Validations” of block transactions, providing as a benefit a higher level of self-responsibility for the correctness of the chain.
Broadly speaking, “Phase 1 Validation” is defined as all checks required for a transaction to be added to the ledger, excluding the execution of Plutus scripts.
Nodes are currently not compensated for this work, and transactions that do not pass this validation are rejected without being included in a block and without paying any fee or guarantee.
All validation rules are specified and implemented in the Cardano Ledger. The specification is divided into several documents, one for each era, incrementally describing the modifications and additions that each era introduced, and therefore to obtain a complete and accurate definition of the Phase 1 validation rules, it is necessary to perform a complete reverse engineering work that requires studying the specification documents of all the eras (Byron, Shelley, Mary, Alonzo and Babbage) and the Haskell source code of the ledger implementation.
The large set of Phase 1 rules can be divided into groups based on the aspects to which they refer, and the contextual information needed to check them. The team chose to organize the rules into the following categories:
- Basic transaction structure: CDDL compliance, integrity hashes, size limits, etc.
- Basic ledger rules: input UTxOs existence, current slot in validity interval, etc.
- Fees and collateral: correct amounts according to protocol parameters.
- Output value: input/output balance, collateral balance, min lovelace per output, etc.
- Witnesses: required signatures, scripts, redeemers and data.
- Phase-1 scripts: native script execution.
- Staking: credentials and withdrawals, etc.
- Other: protocol parameter update proposals, bootstrap witnesses, etc.
For this proposal, the team plans to implement the validation rules in the first five categories, as this rule set is a significant part of the current ledger, and given budget constraints, is a reasonable increment on the way to a node Rust fully functional.
With this implementation it will be possible to synchronize all the public networks (mainnet, preprod and preview) with a significant coverage of the general ledger rules. This will pave the way for subsequent iterations where the remaining aspects of “Phase 1 Validation” can be completed and “Phase 2 Validation” fully implemented.
The Key Metric
The team considers the following dimensions to measure the success of the project:
- Activity on the Github open source repository through metrics such as, but not limited to: # of issues, clones, external contributors, stars, visitors, etc.
- Number of downloads of binary versions
- Web traffic on the documentation site
- Number of external repositories that include this project as a dependency
Being a project of Open Source, the results will be easily verifiable.
The final product will be:
- An LTS version of the engine available for download in various formats (binary, Docker image, etc.).
- A CLI (Command Line Interface) binary that will serve as an entry point for developers
- A documentation website with usage and deployment instructions
- An executable testing framework to run Phase 1 validation unit tests on the provided code.
- A Rust library that can be used to evaluate Phase 1 validations on arbitrary transaction data.
- A video tutorial showing how to run Dolos in different environments and configurations.
The Milestones
Milestone 1: Reverse Engineering (1 month)
- Evaluate available specs and documentation for ledger rules
- Evaluate existing Haskell implementation of the ledger module
- Document findings and definition of required rules
Milestone 2: Business logic implementation (2 months)
- Define module interface for ledger rule execution
- Prepare code scaffold for all required rules
- Define unit-tests for all required rules (all red)
- Implement business rules until all unit-tests pass (all green)
- Release testing framework documentation
Milestone 3: Dolos integration (1 month)
- Connect ledger rules into the existing Dolos data pipeline
- Execute full passes over Mainnet, PreProd and Preview
- Capture performance benchmarks and compare against previous Dolos version
- Release new versions of Dolos
Milestone 4: Tooling integration (1 month)
- Abstract ledger rules as a reusable Rust library in Pallas repo
- Implement a CLI (command line interface) for evaluating arbitrary transactions
- Release binary versions of the CLI
The Budget
The total requested is ₳ 182,142
—Breakdown by resource type:
- Rust developers: 1 FTE x 4 months = ₳ 128,571
- Haskell developers: 1 FTE x 1 month = ₳ 32,143
- Technical writers: 1/5 FTE x 5 months = ₳ 10,714
- Project manager: 1/5 FTE x 5 months = ₳ 10,714
FTE = full-time equivalent
—Breakdown by milestone:
- Milestone 1: ₳ 36,429
- Milestone 2: ₳ 72,855
- Milestone 3: ₳ 36,429
- Milestone 4: ₳ 36,429
The Team
The team has developed Pallas, a Rust library for Cardano that is used by several high-profile projects in the community (such as: cncli and Aiken).
With funding from Catalyst, he has developed Oura, an off-chain data integration tool for Cardano used by many community projects (such as: Pool.io and dcSpark’s Carp, etc), and Dolos, a minimalist version of Cardano node. which is slowly being released to the community as a beta version.
He has also created Demeter, a cloud hosting platform for Cardano infrastructure with several high-profile clients (such as: JPG.store,SummonPlatform y TeddySwap).
Those responsible for this project:
- Santiago Carmuega: Project Lead, part-time Rust developer and lead for Dolos integration
- Franco Luque: Cardano blockchain engineer and Haskell developer
- Alejandro Gadea: Cardano blockchain engineer and Haskell developer
- Emmanuel Gunther: Cardano blockchain engineer and Haskell developer
- Federico Weill: will be responsible for project management.
- Florencia Luna: will be responsible for technical writing of tutorials and documentation.
The Interview
LiberLion: What was your motivation for choosing Cardano to develop your work?
Santiago: Over the past 15 years, I’ve dedicated a significant portion of my career to the development of distributed systems. Around two years ago, my partner Federico Weill and I ventured into the realm of blockchain technology. Among the various options, Cardano stood out to us primarily due to its robust technical foundations, including its non-custodial proof-of-stake, the Ouroboros consensus protocol, and the extended UTXO model. As we interacted with the developers, SPOs and entrepeneurs, we realized the true potential of this ecosystem relied on its community. Our conviction grew stronger; not only did we see it as an exciting ecosystem to explore and contribute to, but we also identified an opportunity to build a sustainable business model within it. So, we made the decision to dedicate our efforts to helping this ecosystem flourish, that’s why we created TxPipe.
LiberLion: Why did you choose to develop infrastructure? It is easier to be profitable in the crypto ecosystem with the creation of end products for the mass public.
Santiago: Our decision to focus on infrastructure stemmed from our strengths and expertise in that area. It may come across as naive, but our primary motivation isn’t profit. We’re committed to building compelling technology that equips developers with the tools they need to create and innovate. We firmly believe that by offering valuable, user-friendly tools, we’ll naturally generate revenue over time. There’s a unique sense of fulfillment that comes from creating tools that others find useful, and we’re confident that we can deliver an infrastructure that many developers will appreciate and leverage.
LiberLion: If you had to choose only one negative feature of Cardano, what is the one that makes it more difficult, in general, for developers to build products?
Santiago: While I wouldn’t label it a “negative” feature per se, I would say the Extended UTxO model poses the most significant challenge when it comes to building on Cardano. This is essentially a new paradigm, one for which there is no established experience, best practices, or patterns to lean on. In comparison, the account model utilized by many other blockchains tends to be more developer-friendly. Don’t get me wrong, I’m fully convinced that in the long run, the Extended UTxO model is the superior approach. However, until the developer ecosystem matures, this model does impact product development quite significantly.
LiberLion: What is TxPipe’s business model? How do you finance your developments without Catalyst? How do you sell your infrastructure products to other developers?
Santiago: Our principal revenue stream is derived from Demeter.run, the Platform-as-a-Service (PaaS) we’ve developed. Our modus operandi is straightforward: we create open-source tools that anyone can download, customize, and operate. However, if you prefer to focus on your core business and product development without the hassle of managing infrastructure, we can handle that aspect for you via Demeter.run. Our platform is designed for those developers who would rather avoid dealing with infrastructure intricacies. Think of it as an alternative to services like Firebase or AWS, but specifically tailored for Cardano infrastructure.
LiberLion: How does having nodes with different functionalities affect the performance of the blockchain, since not all nodes will run Dolos?
Santiago: It’s worth noting that we’re very far from the point where Dolos will be part of the consensus network, if that ever happens. The short-term objective of Dolos is to serve developers who need access to low-level, raw data from the blockchain without the requirement of extensive resources (CPU, RAM). This setup doesn’t impact the chain’s performance, but it undoubtedly improves the lives of developers and projects with these data requirements. In the long run, I hope that Dolos will be picked up by a larger entity or team that can provide the level of resources needed to turn it into a node capable of producing blocks and participating in the consensus of mainnet. If this happens, performance, reliability, and security of the software will be top priorities. I’m confident that nobody would risk affecting the chain’s performance until these areas are addressed.
LiberLion: The blockchain is open source and permissionless, but Have you thought that Cardano’s founding entity, IOG, might call for censorship of SPOs for your node-level development, if Dolos interferes in any way with the founders’ objective?
Santiago: The introduction of an alternative node would indeed be a disruptive event that must be carefully planned and agreed upon by the ecosystem. As it stands now, I see no potential for a fully realized alternative node unless one or more of the funding entities takes the lead on the development. I’m uncertain about how the chain-wide governance mechanism will evolve, and while I’m optimistic, I’m also aware of the numerous challenges that must be overcome to ensure a reasonable and fair use of the Cardano treasury. If executed correctly, it could serve as a means to pursue node diversity. I really hope for this outcome.
Website: https://txpipe.io
You can read the original proposal in IdeaScale
. . .
