Gossamer Ongoing Development Grant

Previous discussion: https://polkadot.polkassembly.io/post/1260

Treasury Proposal: Gossamer Ongoing Development Grant
Proponent: AFRI, 1dvvq2sZg55LfLqFmegU493oUaPWxJfjiYysP1eyovkX6hP (1413)
Beneficiary: CHAINSAFE, 14gaKBxYkbBh2SKGtRDdhuhtyGAs5XLh55bE5x4cDi5CmL75
Date: 09.09.2022
Requested amount: USD 1,998,117.60 in DOT on the day of submission using EMA30 rate by Subscan (Oct 1st, 2022: EMA30 6.815 USD/DOT)
Scope: Software development / Protocol

Short description: Gossamer is a Polkadot host written in Go: a framework to build and run nodes for different blockchain protocols compatible with the Polkadot ecosystem, mainly but not limited to the Polkadot and Kusama relay chains. This proposal addresses the Polkadot treasury to consider a grant for the ongoing development by ChainSafe's Gossamer team for another 12 months after the successful collaboration with the Web3 Foundation Grants Program on the initial steps of this development.

Context: ChainSafe Systems [0], founded in 2017 in Toronto, is a Canadian blockchain development studio with more than 100 employees worldwide. Our vision is to move the various blockchain ecosystems closer, building bridges and increasing client diversity by providing alternative protocol implementations. With a strong emphasis on protocol engineering, we have been engaged by various clients and foundations, among others, in the Ethereum, Polkadot, COSMOS, and Filecoin ecosystems.

Initially funded through a Web3 Foundation grant [1], our protocol team started implementing a Go-Lang Polkadot host in January 2019. Ever since three years and 1300 merged pull requests later, the client has seen more than 40 contributors. As a result, a Gossamer node today can act as an alternative full node for chains such as Polkadot or Kusama. However, it is not yet feature complete. A treasury allocation should cover the pending developments with approval from the community. We want to continue developing and maintaining Gossamer to extend its capabilities further to grow and stabilize the Polkadot ecosystem with an alternative client implementation that is actively used by the community.

The current team consists of seven full-time protocol engineers and a dedicated project manager. The source code is available on GitHub [2] under the terms of an LGPL v3.0 license.

While the Web3 Foundation has entirely covered funding for the development in the past, this will be our first treasury proposal. Hopefully, this will secure our future operations, allow us to scale the team, and continue to broaden our scope while fostering our presence in the Polkadot ecosystem.

Problem Statement: The Substrate blockchain framework enables developers to quickly assemble blockchains for many different use cases. However, it is locking the ecosystem down onto a single programming language, Rust, competing for resources in areas where collaboration is desired.

Offering alternative frameworks in other programming languages should be a high-valued goal of the Polkadot community as it opens up the ecosystem to a larger pool of engineers. Thus far, Gossamer is not yet a generalized framework for building blockchains in Go; however, this will always be our central mission for the project.

Objective: The overall goal is for Gossamer to provide the functionality needed to bring true client diversity to the Polkadot ecosystem. This entails, but is not limited to:

A full-node client capable of peering, synchronizing and seeding the most prominent relay chains and parachains of the Polkadot ecosystems.
Block-production and finality capability for relay chains and parachains (validators, collators).
API services to allow sharing chain and node states through standardized interfaces.
Runtimes generated in alternative programming languages such as Go.

The primary focus of Gossamer development has been implementing the protocols of the Polkadot host, the client architecture that hosts the Wasm runtime of a Polkadot-like relay chain. The first milestone on the Gossamer roadmap is to allow members of the Polkadot community to stake cryptocurrency behind a Gossamer block-authoring node on the Polkadot Network.

Deliverables: The objectives of this proposal can be broken down into the following milestones in no particular order.

Validator specification: Gossamer currently runs in single-node development environments or in sync with other Gossamer nodes or older Substrate versions. A desirable goal for the next year would be implementing the latest validator specification, which breaks down into three major components.
- Transaction queue: Gossamer must be able to receive user-submitted extrinsics [3] and deliver them to the network. Furthermore, Gossamer must implement the transaction queue logic [4] to handle user-submitted transactions efficiently.
- Block production: Gossamer must be able to author blocks using the latest BABE block production protocol [5].
- Block finality: Gossamer must be able to participate in the latest GRANDPA block finalization protocol [6].
Sync modes: Gossamer can currently conduct a full bootstrap synchronization to the tip of the chain. This means that the node can use the chain specification [7] of a Polkadot-like network to connect to that network and to import and verify the blocks that make up the network's canonical chain; these blocks will be stored in Gossamer's key-value database. A desirable goal for next year's development would be to update BABE and GRANDPA implementations to the latest specification and provide alternative synchronization modes.
- State Integrity: Gossamer must implement Polkadot's Patricia-Merkle trie [8] and use it to generate proofs correctly.
- Fork Choice: Gossamer must tip-sync with a Polkadot-like network, which implies tracking multiple candidate forks at the head of the chain and following the best according to the specification.
- Warp Sync: Besides providing a full-synchronization service, it would be desirable to provide other modes such as warp sync.
- DB Imports: A Gossamer node should be able to bootstrap with a database snapshot at a given block height.
RPC APIs: Gossamer must implement the same WebSocket JSON-RPC APIs as the Substrate Polkadot implementation (PSP-6, [9]); this means that the Polkadot-JS API [a] should be able to work seamlessly with a Gossamer host.
Telemetry: A Gossamer client should be able to natively connect to any Polkadot Telemetry Service [b] to announce its presence. These Go-hosts should be identifiable by the Telemetry UI reporting the existence of alternative clients.
Performance: To make the client attractive, it's crucial to adjust the reality to the user's performance expectations concerning CPU usage, memory usage, and synchronization time. Metrics can be comparable with Parity's implementation.

The focus for the next development cycle is matching the latest specification to be able to bootstrap a full Polkadot node and participate in consensus, e.g., on a public testnet such as Westend. Success metrics can be defined by passing the respective test suites provided by the Web3 Foundation; the organization will partner on this proposal as technical advisor and deliverables auditor to ensure the completed milestones are technically sound. If unavailable, an additional deliverable could entail providing test suites for the respective deliverable above.

Please note that funds are requested from the community treasury. The Web3 Foundation technical team has no control over these funds and will not be rewarded as a reviewer of the milestones. The team will serve the sole purpose of evaluating deliverables in alignment with the community approval of this proposal. This role is based on their past participation in this project. It is expected for the ChainSafe team to publish deliverable reports after the Web3 Foundation's technical evaluation and before continuing with subsequent milestones.

Payment Conditions: The requested amount for a 12-months ongoing development grant is USD 1,998,117.60 in DOT on the day of submission using the EMA30 rate by Subscan [c]. The allocation is asked for as a single, up-front payment to cover our ongoing development costs and for the team to be able to provide a runway to their developer resources.

The following resource table outlines the expected costs for the Gossamer team. The table consists of:

the current base cost to operate the team,
an 18% operational overhead to cover the costs of running the project as a team within ChainSafe Systems (e.g., operations, offices, equipment, tooling, infrastructure, legal, travel),
the potential to grow the team sustainability by another two engineers over the course of the following year,
as well as a 3% margin to protect against market volatility of the underlying DOT assets.

             | E2*    | E3*    | E4*    | TPM*   | Base Cost        | Cost +18% overhead |
| ---------- | ------ | ------ | ------ | ------ | ---------------- | ------------------ |
| Month 0    | 3 FTE* | 3 FTE* | 1 FTE* | 1 FTE* | USD   120,000.00 | USD     141,600.00 |
| Month 1    | 3 FTE* | 3 FTE* | 1 FTE* | 1 FTE* | USD   120,000.00 | USD     141,600.00 |
| Month 2    | 3 FTE* | 3 FTE* | 1 FTE* | 1 FTE* | USD   120,000.00 | USD     141,600.00 |
| Month 3    | 3 FTE* | 3 FTE* | 1 FTE* | 1 FTE* | USD   120,000.00 | USD     141,600.00 |
| Month 4    | 3 FTE* | 3 FTE* | 1 FTE* | 1 FTE* | USD   120,000.00 | USD     141,600.00 |
| Month 5    | 3 FTE* | 3 FTE* | 1 FTE* | 1 FTE* | USD   120,000.00 | USD     141,600.00 |
| Month 6    | 4 FTE* | 4 FTE* | 1 FTE* | 1 FTE* | USD   154,000.00 | USD     181,720.00 |
| Month 7    | 4 FTE* | 4 FTE* | 1 FTE* | 1 FTE* | USD   154,000.00 | USD     181,720.00 |
| Month 8    | 4 FTE* | 4 FTE* | 1 FTE* | 1 FTE* | USD   154,000.00 | USD     181,720.00 |
| Month 9    | 4 FTE* | 4 FTE* | 1 FTE* | 1 FTE* | USD   154,000.00 | USD     181,720.00 |
| Month 10   | 4 FTE* | 4 FTE* | 1 FTE* | 1 FTE* | USD   154,000.00 | USD     181,720.00 |
| Month 11   | 4 FTE* | 4 FTE* | 1 FTE* | 1 FTE* | USD   154,000.00 | USD     181,720.00 |
| ---------- | ------ | ------ | ------ | ------ | ---------------- | ------------------ |
| USD Sum    |        |        |        |        | USD 1,644,000.00 | USD   1,939,920.00 |
| +3% margin |        |        |        |        |                  | USD      58,197.60 |
| ---------- | ------ | ------ | ------ | ------ | ---------------- | ------------------ |
| USD Total  |        |        |        |        |                  | USD   1,998,117.60 |
                                                                     ====================

[*] Abbreviations: E2 - Engineer II (Mid-level Engineer), E3 - Engineer III (Senior Engineer), E4 - Engineer IV (Technical Lead), TPM - Technical Project Manager, FTE - Full-Time Employee

The beneficiary of this proposal is ChainSafe Systems Inc, Toronto, Canada. The on-chain address is 14gaKBxYkbBh2SKGtRDdhuhtyGAs5XLh55bE5x4cDi5CmL75. The funds' manager is Hatcher Lipton, COO at ChainSafe Systems, reachable through hatcher@chainsafe.io for any questions regarding the payment conditions.

For any questions about the technical nature of this proposal, you can message me, Afri Schoedon, Head of Protocol Engineering at ChainSafe Systems, at afri@chainsafe.io, or Dan Forbes, Technical Project Manager for the Gossamer team, at dan.forbes@chainsafe.io.

References:

#273·Gossamer Ongoing Development Grant