We are excited to introduce our proposal for the Substrate Subsquid Migration to ArrowSquid & Indexing Upgrades. This project targets enhancing the performance and functionality of Subsquid, a vital component providing access to on-chain data within the Substrate ecosystem. Through this proposal, we aim to improve the speed, efficiency, and indexing capabilities of Subsquid, ultimately benefiting developers and users across various chains such as Polkadot, Kusama, Moonbeam, and Acala.

Subsquid currently provides the community with access to on-chain data via GraphQL gateways, serving as data sources for APIs and on-chain data exploration tools. The main objective of this project is to migrate the existing Substrate Subsquid ecosystem to the ArrowSquid processor as well as enhance its indexing capabilities. This migration will enhance the speed and efficiency of all 50+ public squid endpoints.

Currently, all Giant Squid Main squids utilize the previous Substrate processor, Firesquid. The recent upgrade to the ArrowSquid processor introduces significant improvements, such as the ability to ingest unfinalized blocks directly from a network node, resulting in near real-time APIs with subsecond latency. However, this upgrade to the processor resulted in breaking changes and all squids that want to use it have to be adjusted to conform to the breaking changes.

Additionally, Substrate squids currently lack support for indexing Staking and Crowdloans pallets, which contain critical information for various chains in the Substrate ecosystem, including Polkadot, Kusama, Moonbeam, and Acala. The migration will also improve the performance of Polkascan as it depends directly on the data and performance from GiantSquid

The project aims to address these challenges by:

- Migrating the existing giant-squid-main codebase to utilize the ArrowSquid processor, improving speed and efficiency across all supported chains.
- Enhancing indexing capabilities by implementing indexing of calls and events from the Staking and Crowdloans pallets, enabling access to vital on-chain information.

The project will be executed in two milestones. The first one is focused on migrating giant-squid-main to ArrowSquid, which includes upgrading the dependencies, configuration, and endpoints adaptations, as well as testing and deployment. The second milestone will be focused on indexing Staking and Crowdloan pallets calls and events.

For a detailed description of our proposal, please visit: [Polkadot Treasury - Subsquid Upgrades Proposal](https://docs.google.com/document/d/148ZnlnG7nrDRVhS-VQQ-Ld21u78qaXWqcVsV366EPn4/edit)

**Companies Background:**

Subsquid is providing the community with access to on-chain data in the form of GraphQL gateways and it is doing it completely for free for the time being. Such endpoints are being used both as data sources for complex APIs and as an on-chain data exploration tool. The purpose of the proposal is to ask for funds to cover the cost of running the infrastructure, given the utility they provide to the community.

LimeChain is a software development company founded in 2017. Positioned to handle end-to-end product development with the capability to manage projects from conceptualization and design to the complete implementation of a certain product. LimeChain specializes in blockchain technology, with an emphasis on developing blockchain-related and infrastructure solutions. In the context of the Polkadot ecosystem, the expertise is focused on developing infrastructure implementations, including [Gosemble, a framework for building Substrate compatible Runtimes in Go](https://github.com/LimeChain/gosemble), [Fruzhin, a Host implementation in Java](https://github.com/limechain/fruzhin), a framework for runtimes in AssemblyScript, a framework for runtimes in AssemblyScript. On top of that, Limechain has also implemented a [Parachain Validation Conformance Testing suite](https://github.com/LimeChain/polkadot-conformance).

Subsquid Migration to ArrowSquid & Indexing Upgrades

Treasury

<p>The project is completed, however we have identified as possible pallet improvements to the Giant Squid project is the support for XCM transactions, Conviction Voting, Nomination Pools pallet. We plan to increase the overall adoption of the tool by designing and implementing a home screen page, where users can interact with the different chains, without having to open the GraphQL interface.</p>

This proposal aims to upgrade the Subsquid service by transitioning the Giant Squid Main from the Firesquid processor to the newer ArrowSquid Substrate processor, which is expected to enhance data processing speed and efficiency for over 50 public squid endpoints. This upgrade allows for near real-time responsiveness by directly ingesting unfinalized blocks, enhancing the service's capability.

amount

beneficiary

- **Problem Statement:** Enhancing the performance and functionality of Subsquid, a key component providing on-chain data access within the Substrate ecosystem, targeting developers and users across various chains.
- **Significance:** Improving speed, efficiency, and indexing capabilities of Subsquid to benefit developers and users on chains like Polkadot, Kusama, Moonbeam, and Acala.
- **Proposed Solution:** Migrating Subsquid to the ArrowSquid processor, enhancing indexing capabilities for Staking and Crowdloans pallets to provide vital on-chain information.
- **Expected Outcomes:** Improved speed, efficiency, and indexing capabilities across supported chains, benefiting developers and users with enhanced on-chain data access.

#618 Subsquid Migration to ArrowSquid & Indexing Upgrades

No Active Proposals