OnFinality High Performance Public Infrastructure (Q1 2022)

This is a post to indicate our intentions to submit a full on chain submission to the Polkadot treasury for Q1 operating costs. Please feel free to review and get in contact with OnFinality if you have any questions

Proponent: 12mRyiCp9zdh1wEVW5gLLiFBxDPKks72rRXmSupyEK3VAMLf
Date: 27th April 2022
Requested DOT: 6,800.169 DOT
Short description: Ongoing costs for the running of high performance, scalable, and reliable public infrastructure for Polkadot, Statemint and Westend.
Raw Data: https://docs.google.com/spreadsheets/d/1XhhzxTO8pC9tQMG12Uf8NnHZah6O_TIhMaerzsEBAVY/edit?usp=sharing

Original Motivation

Over the last 3 months (January, February and March 2022) we've provided a scalable free public API service so that everyone can access public pools of managed Polkadot, Statemint, and Westend nodes. This provides the following three outcomes:

• Grow the ecosystem by reducing the barriers to entry to access a high performance nodes
• Help scale existing projects in the ecosystem by providing infrastructure services that can be depended on
• Assist with the testing of all networks by providing valuable data back to the community on how validator nodes are running and scaling as the network traffic increases (we’ve made GitHub issues based on behaviour we’ve observed)

Service Details

This last quarter has been huge for Polkadot and OnFinality, the demand for our services continues to be another unprecedented period. January had a larger activity than the rest of the quarter which also had our highest daily Polkadot response count (for Q1) of over 340 million responses in a single day. Raw data is here.

This quarter was all about service stabilisation as part of the promise we made to this ecosystem to provide enterprise grade services to the community. Our services all exceeded our 99.9% target, with 99.99% uptime for Polkadot over the last 90 days. This is an important measure as we surpassed our largest ever day, with over a billion API responses processed by our service in a single 24 hour period in late January.

You’ll note that there was a significant increase in the amount of data egress over the previous quarter (55% growth to 110TB). We’ve been constantly scaling our nodes to meet this rising level of demand while maintaining our target 99.9% uptime (actual was 99.99%). We are currently working on new load balancing solutions to better route this traffic to different node types and expect that we will soon be able to handle more traffic with less resources.

Between 2022-01-01 and 2022-03-31 our Polkadot API service has:

• Served an unprecedented total of 17,841,881,373 (17 billion) responses
• Accumulated over 110 TB of data egress (up 44% on Q4, 2021)
• The highest daily total was 340,585,751 responses in a single 24 hour period
• Our rolling 7 day average at the end of this period was over 201 million responses each day.
• Westend had over 465 million API responses in the period
• Statemint had over 302 million API responses in the period
• All time totals for Polkadot is 1,919,138,536,720 (1.9 Trillion) since November, 2020
• At the end of this period the 90 day uptime for Polkadot, Statemint, and Westend was 99.99%, 99.98%, and 100%

The full proposal is here