gOuroboros: Cardano Ouroboros Protocol Server support in Golang

Proof of Reputation:

  • Working Product or MVP:  Blink Labs has developed foundational infrastructure tailored for Go developers, enabling them to start projects without the need for extensive groundwork.
  • Previous Proposals and Completions: The BlinkLabs team has historically emphasized infrastructure development and has been actively engaged in community projects. Their notable contributions include 2 proposals for Fund 9, which emphasized enhancing developer support and streamlining the integration process for Go-based solutions.

Context:

1. Project Description:

This is the introduction of the project proposal, which gives an overall understanding of what the project is about.

Overview: 

The project in focus is the continued development of the gOuroboros open-source Golang library, which is engineered to communicate using the Ouroboros protocol with a Cardano Node. This library also facilitates the reading of Cardano’s ledger data structures, emphasizing server-side messages.

Purpose: 

The central objective of the project is to enhance and expand the capabilities of the gOuroboros library. Initially, the library provided functionalities for client-side Ouroboros protocol communications. The current proposal aims to integrate server-side support, enabling the library to facilitate the development of software entities that can act as active participants within the Cardano network. By decoding messages into actionable transaction types, developers can devise applications that trigger specific actions upon observing particular events on the network, such as the operation of blockchain indexers or trading bots.

Background:

The name “Ouroboros” denotes a suite of protocols forming the foundation of the Cardano blockchain. Comprising several specialized mini-protocols, Ouroboros handles distinct communication tasks broken down into messages, which are essential for facilitating interactions with a Cardano Node.

The gOuroboros library had its genesis as the go-ouroboros-network. Funded partially by a proposal from Catalyst Fund 9, this initiative led to the creation of the client side of the library. Consequently, developers could craft client applications in Golang that used the Ouroboros protocol for interactions with a Cardano Node. Subsequent developments and adaptations have seen the gOuroboros library being incorporated into various projects, spanning both open-source and commercial domains, with notable mention of its usage in numerous Blink Labs projects.

Now, there’s an ambition to embark on the next phase of development – the integration of server-side support. With this, server applications could be designed in Go to serve client data using the Ouroboros protocol directly. The present proposal, under Project Catalyst Fund 10, is a plea for funding this pivotal open-source development, effectively bringing full circle the Ouroboros functionalities in Go. Upon successful implementation, developers will have the privilege of accessing the comprehensive range of Ouroboros mini-protocols right from their Go applications.

2. Project Team:

An understanding of who is behind the project can shed light on their capability to execute it:

Team Members:

The gOuroboros project is helmed by the seasoned engineering team at Blink Labs. Key members leading the charge include:

  • Aurora Gaffney (GitHub Profile): Aurora has been earmarked as the primary developer for the project.
  • Chris Gianelloni (GitHub Profile): Chris will be undertaking roles in code review, documentation, and support.

While these two will be at the forefront, additional staff members from Blink Labs may be engaged in supporting roles, particularly concerning administrative functions like project management and community reporting.

Skills and Expertise: Blink Labs boasts a conglomerate of Principal / Staff Engineers with an aggregate experience surpassing 40 years in the domain. Their skill sets span a wide range of development aspects, ensuring comprehensive expertise for the gOuroboros project. Such vast experience places the team in a good stead to understand the nuances and intricacies of the project and develop robust solutions in line with the objectives set.

Track Record: Blink Labs is no stranger to the open-source domain and has been contributing for decades. Their dedication to transparency is evident in their approach, with all their works being open to public scrutiny.

The testament to their capabilities is visible in their previously completed projects under Catalyst Fund 9:

  • Ouroboros network (NtC – node-to-client) in Golang: This directly paves the way for the current proposal, laying down the groundwork.
  • Cardano Submit API in Golang: Another significant project showcasing their expertise in the field.

Moreover, the team’s cohesiveness, built over years of collaboration, ensures precise time and cost estimation, vital for project execution. Notably, the gOuroboros project has seen a steady development pace for 18 months, both in funded and non-funded capacities. With dedicated funding, Blink Labs is poised to enhance the development speed, bringing their vision to life more promptly.

3. Project Details:

In-depth information about the project and how it plans to address the issues at hand:

Methodology: 

The overarching ambition for the gOuroboros project is to pave the way for a flourishing Golang ecosystem within the Cardano framework. For this ambition to be realized, the gOuroboros library needs to boast versatile functionalities, thereby ushering in a myriad of downstream projects.

To make this vision a reality, Blink Labs has outlined the following four strategic milestones:

  • Client completion and automated testing: To ensure the client side is not just functionally robust but also error-resilient.
  • Era support: Catering to the Cardano blockchain’s evolving eras.
  • Transaction attributes: Deep-diving into the intricate attributes of each transaction.
  • Server support: Establishing a robust server-side capability to make the gOuroboros library holistic.

To maintain transparency and allow for public tracking, Blink Labs will use GitHub for creating specific issues pertaining to each work item. These items will then be meticulously managed using a project Kanban board on GitHub.

Timeline: 

The project is structured with each milestone corresponding to a month. Therefore, given the four milestones, the entire project is slated for a four-month duration. While some tasks might overlap and run in parallel, software releases will be orchestrated as and when new functionalities are integrated. This ensures that downstream projects can promptly harness these new capabilities.

Milestones Overview:

  • Initial Foundation: The project has been established and is available under the Apache 2.0 license on GitHub. This provides a strong base for future work. Given that it’s already accessible, the transparency of the project is commendable.
  • Milestone 1: This focuses on the client side of node-to-node and node-to-client mini-protocols, with an emphasis on protocol client support and additional tests for several protocol clients. Support for more LocalStateQuery protocol queries and responses is also included, ensuring a comprehensive client-side functionality.
  • Milestone 2: This milestone tackles support for various eras, which would ensure the library’s versatility across different versions of Cardano’s blockchain. The inclusion of eras from Byron to Babbage demonstrates forward-thinking and comprehensive coverage.
  • Milestone 3: The emphasis here is on completing support for all transaction attributes. The detailed listing of transaction attributes—from fees to voting procedures—shows meticulous planning.
  • Milestone 4: Completing the server side of the protocols ensures the library can effectively interact with Cardano nodes, both from a client and a server perspective.

Ressources:

The core resource allocation is centered around human capital:

  • Engineering: The primary funding driver, accounting for 40,000 ADA per month. The allocated time per milestone for engineering is roughly equivalent to 75% of a full-time software engineer, translating to approximately 120 hours.
  • Project Management: Allocated 6,250 ADA monthly, with an expected commitment of around 10 hours per milestone. This ensures that the project stays on track, and the logistics are handled seamlessly.

The totality of the project budget is pegged at 46,250 ADA per month, seamlessly bifurcated between engineering and project management.

Deliverables and Outcomes:

Each milestone, though distinct, forms a part of a cohesive whole. Though these tracks of work might sometimes converge, the final outcome for each milestone will always be its seamless integration into the main software project. This integration will be complemented by generating and publishing documentation on pkg.go.dev, the Go language’s premier developer documentation portal.

For the intricacies of tracking, the gOuroboros project will be leveraging GitHub issues. All work scoped within this proposal is set to be meticulously tracked through a dedicated Kanban board on GitHub Projects, accessible here. This ensures transparency and offers stakeholders a real-time view of the project’s progress.

4. Impact Assessment:

Understanding the potential impact of the project is crucial for its selection:

Target Audience: 

Golang boasts a burgeoning developer community that has grown substantially, as evidenced by its increased traction on platforms such as GitHub. The gOuroboros library aims to bridge the gap between this vast pool of Golang developers and the Cardano blockchain. By offering tools that allow for seamless interaction with Cardano nodes and ledger readings, the project opens the gates for a significantly larger segment of developers, thereby expanding Cardano’s reach and developer ecosystem.

Measurable Outcomes:

The primary benchmark for success is rooted in the gOuroboros library’s capability to support a comprehensive range of client and server-side functionalities. The project is deemed successful once the library can fully support all operations of both NtC (node-to-client) and NtN (node-to-node) mini-protocols, in conjunction with reading basic ledger information and all transaction attributes.

Sustainability: 

The gOuroboros library is developed as an open-source endeavor, ensuring complete transparency and public access. Each phase of development and associated tasks will be available for public scrutiny on the gOuroboros repository. Furthermore, to make progress tracking accessible, a dedicated GitHub Projects Kanban board (link) is in place. The proactive communication approach of the engineering team, with weekly meetings and regular status updates, further underscores the project’s commitment to transparency.

In addition to this, Blink Labs leverages the gOuroboros library for various projects, including but not limited to open-source initiatives on GitHub, like nview and tx-submit-api. By weaving gOuroboros into closed-source partner projects like the Maestro’s dApp Platform, the project’s relevance is further cemented within the broader ecosystem.

In essence, the long-term impact of gOuroboros lies not just in its immediate utility, but in its potential to attract a broader swath of developers to the Cardano ecosystem, opening the door for innovative applications and heightened engagement.

5. Risk and Contingency Planning:

Every project faces potential risks and challenges. How are these addressed?

Risk Identification: 

1. Dependency Risks:

  • Unforeseen Delays in External Implementation: As mentioned, the project is contingent upon the completion of Conway era support and the Peer sharing protocol by IOG. Delays or changes in IOG’s implementation plans can push back the project’s timeline.
  • Incomplete or Ambiguous Specifications: If the spec document provided by IOG lacks clarity or misses out on crucial details, it could lead to misinterpretations and flawed implementations in the gOuroboros library.

2. Technical Challenges:

  • Interoperability Issues: While gOuroboros aims to communicate with Cardano nodes, discrepancies in communication protocols or standards could arise.
  • Scalability Concerns: As the Cardano ecosystem grows, the library needs to handle increased data structures and transactions without performance degradation.

3. Resource Constraints:

  • Unavailability of Key Team Members: The project leans heavily on specific team members (e.g., Aurora Gaffney and Chris Gianelloni). Their unavailability due to unforeseen circumstances might impact the project.
  • Budget Overruns: Unanticipated technical challenges could lead to increased engineering time, potentially straining the budget.

4. Adoption and Integration Challenges:

  • Resistance from Golang Developers: Even with a functioning library, the wider Golang community might be slow or resistant to adopting it, especially if they are unfamiliar with the Cardano ecosystem.
  • Competing Solutions: Other organizations or communities might release similar or more advanced Golang libraries for Cardano, overshadowing gOuroboros.

Risk Mitigation:

  • Regular Communication: One of the most effective ways to mitigate dependency risks is by maintaining frequent communication with the organizations or entities on which the project is dependent. In this case, regular check-ins with IOG can help the team stay updated on the status of the implementation or the spec document.
  • Alternative Research: In cases where there might be delays or uncertainties regarding the spec document or implementation, the team can actively research alternative sources or approaches that might provide the necessary information or guidance.
  • Parallel Development: Even as the team awaits the necessary implementations or documents, they can focus on other parts of the project that are not reliant on these dependencies. This ensures that the project timeline doesn’t come to a standstill and other milestones can be achieved in parallel.

6. Budget Estimation:

A breakdown of the project’s budget provides insight into its financial feasibility:

Detailed Budget: 

The gOuroboros project is well-structured into clear milestones, each with its defined tasks and goals. From the previous section, we know that the monthly budget is 46250 ADA, with a clear allocation to Engineering (40000 ADA) and Project Management (6250 ADA).

Based on the provided timeline of 4 months, we can provide an estimated total budget for the project:

  • Total Budget for Engineering (4 months): 160,000 ADA
  • Total Budget for Project Management (4 months): 25,000 ADA
  • Overall Total Budget (4 months): 185,000 ADA

This budget covers the entirety of the project, from client-side protocols, through the support for various eras, then to transaction attributes, and finally to server-side support.

7. Q&A with Chris from BlinkLabs

Q1: How was the idea for gOuroboros conceived?

A: gOuroboros emerged from our recognition of a void in the Cardano ecosystem for Go applications, informed by our collaboration with Boris on ‘toasty drop’ and our extensive work in stateful operations with ‘construct’. Our rich background in platform engineering and cloud-based solutions, epitomized by our prior identity ‘cloud struct’, directed us to establish gOuroboros, a Go implementation of the Cardano network protocol. This grassroots approach allowed us to communicate directly with the protocol, and with funding from ‘fun nine’, we developed client software to directly query the blockchain, an innovation later integrated by our partner, Maestro, into their v1 DAPP platform.

Our present trajectory is focused on gOuroboros’ server-side capabilities. While our client-side architecture enables querying a node, we’re forging ahead to answer those queries from the server side. Drawing from past experiences, like the logging tool crafted by ‘pi’ from Sunday swap, our aim is to not just emulate node functionalities but to create specific, targeted node capabilities. gOuroboros is our endeavor to provide a foundational toolkit to developers, enabling more nuanced interactions and applications within the Cardano ecosystem.

Q2: Why is introducing the Go language important to the Cardano ecosystem?

A: Go, as a language, was conceived at Google in 2007 with the aim to cater to the requirements of massive-scale product and service developments. It was architected keeping in mind modern machines with multiple cores handling vast workloads and had a significant emphasis on network concurrency. This made it particularly apt for systems running at the heart of Google, from their core search functionalities to critical services in Site Reliability Engineering.

One of the notable strengths of Go lies in its concurrent, network-optimized nature which perfectly balances performance with readability. Furthermore, Go boasts a robust standard library. An illustration of its capabilities can be seen with the Kubernetes container orchestration system, which is penned in Go. Another advantage is that Go applications compile into binaries that are straightforward to distribute. Its support for cross-compiling to a multitude of platforms, ranging from Windows and Linux to MacOS and even FreeBSD, is commendable. This innate characteristic makes it profoundly portable for pure Go applications.

Our team at Blinks Labs has a strong foundation in open-source, container orchestration, and large-scale distributed systems. Our tryst with Kubernetes, especially its deployment at scale in production scenarios, acquainted us with Go. This soon became our preferred language for application development.

Statistical insights from GitHub, specifically GitHut 2.0, reveal the soaring popularity of Go. In the first quarter of 2023 alone, Go repositories witnessed a growth of 10.423% in pull requests and a 12.107% increase in repository stars year-on-year. Contrasting this with Haskell, which is the foundational language for several pivotal components of Cardano software, there was only a meager 0.201% rise in pull requests and a 0.238% increment in repository stars. To give a clearer perspective, the Go developer community overshadows the Haskell developer community by more than four times, based on the number of GitHub repositories for each language. These figures, albeit used as a proxy since community size is challenging to determine accurately, convey our aspiration. We are keen to bridge the prevalent divide between Go developers and the Cardano blockchain, aiming to magnetize a larger developer audience towards Cardano.

Q3: If you don’t receive funding, what’s your next move?

A: If we don’t receive full funding, we’ll likely continue as we did in the past — by primarily self-funding through Blink Labs. My co-founder and I have been committed to this project, investing personally in its development. Even in ‘fun nine’, where we asked for retroactive funding, a significant portion of the work had been accomplished prior to securing those funds. Our aim has always been to ensure the progress of our projects, even if it means covering costs out of pocket. We deeply believe in the potential of introducing Go into the Cardano ecosystem, considering its widespread use in other blockchains. Our hope is that, with our foundational work, when developers are ready to incorporate Go alongside other languages for blockchain development, they’ll have the tools they need to kickstart their projects on Cardano.

Q4: What are the key differences between Haskell and Go?

A: The primary differences between Haskell and Go lie in their design philosophies, performance, and the problems they’re optimized to solve. Go, emerging from Google, was specifically designed for highly concurrent network-based programs, making it apt for distributed systems like blockchains. Its design allows for binaries to be produced rather than interpretation at runtime, resulting in easier production, distribution, and typically, a lower memory footprint. A practical illustration of this is our rewriting of the Cardano submit API, originally in Haskell, into Go; our Go version consumed half the memory, had lower latency, and compiled significantly faster.

On the other hand, Haskell is renowned for its capabilities in formal verification, allowing developers to mathematically prove the correctness of their code. While Go doesn’t possess this exact capability, our approach utilizes Haskell’s proven correctness as a benchmark. By implementing the same protocols and specifications in Go as in Haskell, and ensuring both produce matching outputs, we leverage a comparison-based proof of correctness. Essentially, if Haskell’s output, which is mathematically proven to be correct, matches Go’s output, it serves as a validation of Go’s correctness in that context.

Q5: How do you intend to monetize this open-source initiative?


A: We’re currently not self-sufficient and fund our open-source efforts largely out of pocket, supplemented by revenue from commercial code-writing engagements. This approach, as seen with our work for Maestro, helps validate and enhance our open-source projects. Our future strategy revolves around an open-core business model: the core software remains free, while we charge for premium features, support, and custom solutions built atop it. Our aim is to establish ourselves as the experts for specific niches, like Go-based applications for Cardano. While our ultimate goal is to make this initiative full-time, we’re also exploring options like Catalyst funding to support our open-source endeavors.

Q6: How does one approach a project using Go?

A: Approaching a project using Go, from our perspective at Blink Labs, isn’t just about diving into the language. Our aim is to build an ecosystem where developers can capitalize on their existing knowledge. This means collaborating with teams like Gimbal Labs, who are crafting an outstanding educational platform. We’re excited about creating project-based learning modules in Go to fit into such platforms.

Consider the Plutus courses as an analogy. Some developers have reconstructed the Plutus example app in Python, offering another avenue to achieve the same output. Similarly, our objective is to allow Go developers to access project-based learning tailored to their expertise. Instead of directing them to first learn Haskell, Rust, or TypeScript, we’re providing tools in a language they’re already familiar with.

Our current focus is on establishing the foundational infrastructure, ensuring Go developers can leap directly into creating, without being bogged down by the foundational groundwork. While we’re in these early stages, our vision for the next couple of years is to expand this offering, enabling more developers to smoothly transition into projects using Go.

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Posts