[Dune](https://dune.com/browse/dashboards) is the industry leader in highly shareable dashboards widely used by industry leaders, data analysts, marketers, and builders across the broader Web3 ecosystem.  For example here is [Brian Armstrong sharing a Dune dashboard about Base](https://twitter.com/brian_armstrong/status/1690158283632586752), compared to [Moonbeam sharing dolpha.com](https://twitter.com/MoonbeamNetwork/status/1702721492834755061), [Astar sharing subscan](https://twitter.com/WatanabeSota/status/1731863081943277787) or [Polkadot sharing DotLake](https://twitter.com/Polkadot/status/1722980186189070760):

<table>
<tr>
<td>
<img src="https://cdn.polkaholic.io/366/366-coinbase.png" alt="Polkadot" width="400">
</td>
<td>
<img src="https://cdn.polkaholic.io/366/366-moonbeam.png" alt="Polkadot" width="400">
</td>
</tr>
<tr>
<td>
<img src="https://cdn.polkaholic.io/366/366-astar.png" alt="Polkadot" width="400">
</td>
<td>
<img src="https://cdn.polkaholic.io/366/366-polkadot.png" alt="Polkadot" width="400">
</td>
</tr>
</table>
 
Common to the above is that Web3 ecosystem participants want to share on-chain data, which Dune is optimized for, better than a block explorer ([Subscan - #311](https://polkadot.polkassembly.io/referenda/311)) or expensive dashboards  ([Dolpha.com - #341](https://polkadot.polkassembly.io/referenda/341)).  Colorful Notion has built [substrate-etl - #248](https://polkadot.polkassembly.io/referenda/248), here proposed to be ingested by Dune as *substrate-etl 2.0* in 2024.

##  Proposal - Onboard Polkadot + Kusama Data on Dune (12 months, 3/15/24-3/14/25)

Colorful Notion proposes to onboard the Polkadot + Kusama relay chains by mid-March 2024, available for the community for at least _2 years_.  This proposal funds the _first_ year after a PoC is completed in early 2024, for 3/15/24 - 3/14/25.  Onboarding Polkadot will be Dune’s second non-EVM ecosystem integration, after Solana.  Separate from this OpenGov proposal, Colorful Notion has negotiated pricing for all parachains with Dune, beyond the Polkadot and Kusama Relay Chain.  This will enable Polkadot + Kusama Substrate data to be represented in Dune for both relay chains (using "polkadot" and "kusama") and all Polkadot parachains in an official integration with hourly indexing. 

# Demonstration: Polkadot Staking Dashboard

In December 2023, Colorful Notion developed a proof of concept with key Dune engineers for the Polkadot Relay Chain with a few representative datasets.    Here is one of the first Polkadot dashboards built in Dune: 

[https://dev.dune.com/mkchungs/polkadot-staking-validator](https://dev.dune.com/mkchungs/polkadot-staking-validator)

<img src="https://cdn.polkaholic.io/366/366-staking.png" alt="Polkadot" width="100%">


Once data is ingested from Colorful Notion’s substrate-etl into Dune, this dashboard can be rapidly constructed within a couple hours from SQL knowledgeable engineers such as Colorful Notion cofounder [@mkchungs](https://polkadot.polkassembly.io/user/mkchungs).  Here is one query:

<img src="https://cdn.polkaholic.io/366/366-staking-rewards.png" alt="Polkadot" width="100%">

Any on-chain dataset from Polkadot or Kusama involves writing a query such as the against some substrate-etl 2.0 dataset.  Colorful Notion’s commercial terms have been developed to allow for a certain amount of storage ingesting a certain number of chains so that “Dune wizards” can query these tables, views, and materialized views and produce shareable charts like the above.   

Everyone with a Dune account can query and build these dashboards in Dune.  While Google BigQuery can be connected to open source Apache Superset (and we have, see our W3F grant work on [Deep Account Analytics](https://github.com/w3f/Grants-Program/pull/1286) and inkubator-supported [ChainIDE + Polkaholic.io WASM Contract Explorer Integration](https://github.com/use-inkubator/Grant-Milestone-Delivery/blob/master/deliveries/chainide-polkaholic-integrated-wasm-contract-explorer-integration-milestone_1.md)), Dune is more usable and more shareable.  

Whereas much of 2022 involved developing Polkaholic.io and 2023 involved substrate-etl 1.0 bringing data into Google BigQuery, Colorful Notion will have 2024 involve robust ingestion of Polkadot and Kusama data into Dune and ensuring the community can use it to support blockchain analytics and marketing.

# Proposal Details


[Full Proposal](https://docs.google.com/document/d/1UMpD152JmvHm6O9Q76TlF8GR1O_FRJydjf7bndClj-k/edit)

[Cost Breakdown](https://docs.google.com/spreadsheets/d/1c7-3kdfAjWdtWTkx2anEQgYpToUiWo3vav1WrapFuEI/edit#gid=1479719106)

[Slides](https://docs.google.com/presentation/d/1wulreQof9V7QwCTuJZ9e64eDHhzAtqTS/edit#slide=id.g2a79f725924_0_8)

**Requested Amount:** 325,000 DOT [$1,968,200]

# December 2023 Decision For DOT Tokenholders: Dune Now or late 2024?

Colorful Notion has been advocating a Polkadot Data Alliance Bounty along with Parity Data throughout much of 2023 (see [this from CN](https://docs.google.com/document/d/1ryC6dxcd9tiQsB7KiCc2BY_TwBJ5jKloGfCyVAGCkKo/edit#heading=h.gjdgxs) and [this from Parity Data](https://docs.google.com/document/d/1fA5ARHy-frzgZC66rniKZ5o7CSbDvCTkS--kWaMzuMs/edit#heading=h.foftxwdr3st2)).   In November, we finally decided to submit #248 and reach out to Dune as an outstanding goal of both teams.   From the DOT Tokenholder point of view, once it is accepted that an ecosystem wide Dune integration is desirable and both teams are competent (as we do), we believe the decision is largely a tradeoff between Dune NOW vs Dune LATER:

<img src="https://cdn.polkaholic.io/366/decision.png" alt="Polkadot" width="100%">

A simplified comparison is as follows:

<img src="https://cdn.polkaholic.io/366/366-comparison.png" alt="Polkadot" width="100%">

Which boils down to the following choice:

* If you believe you want a Dune Integration NOW (early 2024), vote Aye on #366.
* If you believe you want a Dune Integration LATER (or don’t want it at all), vote Nay on #366.

We believe the answer is to integrate Dune NOW to support marketing and general ecosystem growth.

# About Colorful Notion

Colorful Notion is led by [Sourabh Niyogi](https://forum.polkadot.network/u/sourabhniyogi) and [Michael Chung](https://forum.polkadot.network/u/mkchungs).  We have been active in the Polkadot ecosystem over the last 2 years and look forward to building with others in Polkadot 2.0’s “map reduce” ubiquitous computing future. 


## Updates 

* 12/21/2023 - [AAG #93 - et-l brute?](https://www.youtube.com/watch?v=3jlA51DLz2o) 
* 12/22/2023 - PoC of Polkadot Relay Chain completed -- see [Polkadot Proof-of-Concept Dashboard](https://dune.com/sourabhniyogi/polkadot-proof-of-concept)
* 1/3/2024 - Happy New Year 2024!   Basic CN-Dune terms are being developed by Dune General Counsel; Partially redacted summary of Terms used for MSA draft may be posted here, _with permission_.
* 1/4/2024 - CN received MSA from Dune.  Publicly sharable CN-Dune terms are included below for transparency:

![image](https://i.ibb.co/Rb5Nb3V/4113-sourabhniyogi-1704397042754.png)

* 1/5/2024 - 3 Order forms developed for MSA.
* 1/9/2024 - [Relay Chain/Parachain tables](https://docs.google.com/spreadsheets/d/1eG14AFNpJBx8Ng46XIy19hM1R_POcT94t5mvfWaQoFE/edit#gid=1994869400) (shared with Dune permission)
* 1/11/2024 - MSA finalized
* 3/19/2024 - Dune public announcement: https://twitter.com/DuneAnalytics/status/1770087920994603325

# Ongoing Dashboard development in Q2 2024: 

[Polkadot Wiki PR](https://github.com/w3f/polkadot-wiki/pull/5933)

## Polkadot Relay Chain

Polkadot Governance:
* General: https://dune.com/substrate/polkadot-gov
* Refs:  https://dune.com/substrate/polkadot-referendum	

Polkadot Staking: https://dune.com/substrate/polkadot-staking	

## Polkadot Parachains:

AssetHub:  https://dune.com/substrate/assethub		

Interlay:  https://dune.com/substrate/interlay		

Astar:  https://dune.com/substrate/astar-dapp-staking		

Bifrost:  https://dune.com/substrate/bifrost		

HydraDX:  https://dune.com/substrate/hydradx		

Centrifuge:  https://dune.com/substrate/centrifuge		

Moonbeam:  https://dune.com/substrate/moonbeam-governance	

Stellaswap:  https://dune.com/substrate/stellaswap	

Acala:  https://dune.com/substrate/acala		

Unique: https://dune.com/substrate/unique		

Phala:  https://dune.com/substrate/phala		

Neuroweb:  https://dune.com/substrate/neuroweb		

Manta:  https://dune.com/substrate/manta		

Mythos: https://dune.com/substrate/mythos

Collectives:  https://dune.com/substrate/collectives		

Nodle:  https://dune.com/substrate/nodle		

Invarch:  https://dune.com/substrate/invarch	

## Kusama Relay Chain 

Kusama Staking: https://dune.com/substrate/kusama-staking	

Kusama Governance:
* General: https://dune.com/substrate/kusama-gov
* Refs: https://dune.com/substrate/kusama-referendum	

## Kusama System Chains / Other

Coretime:  https://dune.com/substrate/coretime_kusama		

DED: https://dune.com/substrate/dotisded


For the latest of ongoing ingestion status and dashboard catalyst, see this [public sheet](https://docs.google.com/spreadsheets/d/1eG14AFNpJBx8Ng46XIy19hM1R_POcT94t5mvfWaQoFE/edit#gid=2015037)
	

- **Problem Statement:** Web3 ecosystem participants desire to share on-chain data efficiently, which Dune excels at compared to block explorers or expensive dashboards.
- **Significance:** Enhancing Dune's capabilities to onboard Polkadot and Kusama data can significantly improve blockchain analytics and marketing within the ecosystem.
- **Proposed Solution:** Colorful Notion proposes to integrate Polkadot and Kusama relay chains into Dune by mid-March 2024, enabling hourly indexing and representation of all Polkadot parachains.
- **Expected Outcomes:** By integrating Polkadot and Kusama data into Dune, users can efficiently query and build shareable dashboards, supporting marketing efforts and ecosystem growth.

#581 substrate-etl 2.0: Polkadot + Kusama Data on Dune (12 months, 3/15/24-3/14/25)

No Active Proposals