Summary

**Proponent:** [Subsquid Lab Official](https://polkadot.subscan.io/account/13bfKSQXoBn3AMLtZaW6BKv797fqZzsD3PYF6xpJDir3tdoE) - 13bfKSQXoBn3AMLtZaW6BKv797fqZzsD3PYF6xpJDir3tdoE

**Beneficiary:** [Subsquid Lab Official](https://polkadot.subscan.io/account/13bfKSQXoBn3AMLtZaW6BKv797fqZzsD3PYF6xpJDir3tdoE) - 13bfKSQXoBn3AMLtZaW6BKv797fqZzsD3PYF6xpJDir3tdoE

**Contact Details:** 

Subsquid Labs GmbH

6300 Zug, Switzerland

[mf@subsquid.io](mailto\:mf@subsquid.io) 

**Short description:** Ongoing development and maintenance costs for public SQD Archives, providing critical data infrastructure for Polkadot and Kusuma and other public Substrate-based chains in Q4 2024.  

**Archive raw data:** [Archive Infrastructure metrics](https://docs.google.com/spreadsheets/d/1Rj_-4KqM2BZ7pHptIBryLZqcOmiQqi7PdFjBLcN25w4/edit?usp=sharing) - October, November, December

**Requested amount:**  **$308,704**

**Previous proposals:** [https://polkadot.polkassembly.io/referenda/1526](https://polkadot.polkassembly.io/referenda/1526)

# Motivation



**A) Archives** 



SQD Archives are a vital component of the Polkadot and Kusama ecosystem’s data infrastructure, offering scalable access to historical on-chain data for public Substrate-based chains. These archives serve as performant data pipelines for major dApps, APIs, and developer tools, supporting critical ecosystem projects such as:


* Tanssi Network
* Talisman Wallet
* Polimec
* Polkassembly
* Apillon
* Hydration
* Giant Squid API (maintained by LimeChain, see details below)
* SubWallet 
* FiDi
* KodaDot
* RMRK
* ChainSafe (in particular the Multix multi-sig, as well as other projects)
* Phala Network
* Stellaswap
* Polkascan 



By providing free indexing services for the following public chains, SQD lowers barriers for developers and projects building on Substrate, fostering innovation and adoption:



* polkadot
* kusama
* asset-hub-kusama
* asset-hub-polkadot
* bridge-hub-polkadot
* collectives-polkadot
* Westend


Despite transitioning to a fee-based model for private data services to ensure sustainability, SQD remains committed to covering infrastructure costs for public archives, ensuring open access to indexed Substrate Events, Extrinsics, Storage Items, and EVM logs. This commitment aligns with Polkadot’s vision of a decentralized, accessible ecosystem.



**B) Contribution**



During Q4’2024, we made significant contributions to the Polkadot ecosystem: 

* Infrastructure Scaling: Expanding archive capacity to meet growing data demand. 
* Squid SDK Enhancements: Introducing advanced indexing features, including support for new Substrate pallets and improved error-handling mechanisms. 
* Decentralized Network Growth: Increasing SQD Worker Node participation to enhance network resilience. 



# Achievements

**Growth in Archival Data Demand, Network Expansion and Further Decentralization**

In Q4 2024, demand for archival data grew steadily, rising from 11.35 TiB to 11.68 TiB. At the same time, the number of active Worker Nodes in the SQD network increased from 1,139 in October to 1,600 by the end of December—a 40.47% increase. This expansion strengthens the SQD network’s infrastructure, contributing to greater decentralization and resilience across the Polkadot ecosystem.


By the end of 2024, the SQD network had scaled to approximately 1,600 active Worker Nodes, securely storing around 900 TB of data and serving roughly 11 TB daily. This robust, decentralized infrastructure significantly enhances data processing and storage capabilities, delivering improved security, fault tolerance, and scalability for the Polkadot network.



**Squid SDK improvements empowering Polkadot Developers:**

* **Additional unique support for Revive pallet:** Enabling the tracking of contracts deployed to the PolkaVM. 
* **Indexing efficiency upgrades:** Implementation of a novel mechanism to skip and log problematic blocks-those containing invalid or non-decodable data on Substrate based chains, ensuring uninterrupted indexing and improved resilience of the data pipeline. 


# Statistics and data

 <img height="300" width="506" src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXfR5GffoKYtzjPZmyabzbBFfF6vck2Zideo3kZDzv8q0VGBM1Qzd4-ptg_fnfD_t_pGOe6GJqgcfqy3SVBchT4R5P7BbrzFGdGva37uR7ZG3-4dhgjM5upRmW4XGrj7spG9Eia9zw?key=OtINmgDGvVWqK5axEzaXJs4e" />


Over the course of Q4 2024:

* Served a total of **11,668,177,888,249**  (\~10.06 TiB) of data from substrate based chains.
* Served **2,336,249**  archive requests from substrate based archives.
* The highest daily total data transfer recorded for Substrate was **382.57 GiB** within a single 24-hour period (2024-10-05).



## Cost Breakdown

This proposal encompasses the combined costs for running Archives for Polkadot, Kusama, and other public chains, along with the development contributions, amounting to a total of **$308,704**  [Raw data can be seen here](https://docs.google.com/spreadsheets/d/1Rj_-4KqM2BZ7pHptIBryLZqcOmiQqi7PdFjBLcN25w4/edit?gid=466566330#gid=466566330)

The proposal is submitted towards the Polkadot treasury. Here is a summary of the running costs for Archives, in a tabular format:

| Service | Cost | VAT 8.1% | Subtotal |
| ---------------------------------- | ----------- | ----------- | ----------- |
| Compute Engine (GCE) | $92,630.90 | $100,134.00 | $100,134.00 |
| OVHCloud | $28,419.03 | $30,720.97 | $30,720.97 |
| Networking | $2,963.02 | $3,203.02 | $3,203.02 |
| Kubernetes Engine (GKE) | $436.23 | $471.56 | $471.56 |
| Cloud Storage (GCS) | $63.21 | $68.33 | $68.33 |
| Cloud SQL | $396.38 | $428.49 | $428.49 |
| Cloud DNS | $18.01 | $19.47 | $19.47 |
| Cloud Logging | $13,506.84 | $14,600.89 | $14,600.89 |
| Cloud Key Management Service (KMS) | $2.99 | $3.23 | $3.23 |
| Artifact Registry | $336.01 | $363.23 | $363.23 |
| | | | |
| Engineers & Maintenance\* | $144,000.00 | $155,664.00 | $155,664.00 |
| Total | | | $305,677.20 |




Contribution costs: 

| Service | Cost / h | Quantity | VAT 8.1% | Subtotal |
| ---------------------------- | -------- | -------- | --------- | --------- |
| Revive pallet support | $100.00 | 16.00 | $1,729.60 | $1,729.60 |
| Indexing efficiency upgrades | $100.00 | 6.00 | $648.60 | $648.60 |
| Product management | $100.00 | 3.00 | $324.30 | $324.30 |
| Project management | $100.00 | 3.00 | $324.30 | $324.30 |
| Total | | | | $3,026.80 |

Propose and approve a spend of treasury funds.

## Dispatch Origin

Must be [`Config::SpendOrigin`] with the `Success` value being at least
`amount` of `asset_kind` in the native asset. The amount of `asset_kind` is converted
for assertion using the [`Config::BalanceConverter`].

## Details

Create an approved spend for transferring a specific `amount` of `asset_kind` to a
designated beneficiary. The spend must be claimed using the `payout` dispatchable within
the [`Config::PayoutPeriod`].

### Parameters
- `asset_kind`: An indicator of the specific asset class to be spent.
- `amount`: The amount to be transferred from the treasury to the `beneficiary`.
- `beneficiary`: The beneficiary of the spend.
- `valid_from`: The block number from which the spend can be claimed. It can refer to
  the past if the resulting spend has not yet expired according to the
  [`Config::PayoutPeriod`]. If `None`, the spend can be claimed immediately after
  approval.

## Events

Emits [`Event::AssetSpendApproved`] if successful.

#1661 SQD (fka Subsquid) - Public Data Indexing Infrastructure for Polkadot and Kusama (Q4 2024)

No Active Proposals