Subgraph Evaluation: Tutorial for Curators
The process of evaluating a subgraph is an open experience that the curator’s group is elaborating on up to now. At this moment, there is no step by step tutorial that could help curators and The Graph community as a whole in this fundamental task. On The Graph testnet, the Curators Program is on Phase 2 which is about evaluating subgraphs on the completeness, accuracy, and complexity criteria in order to assure that they are capable of supporting an application’s data needs.
Find basic available information for the subgraph evaluation.
1. Find your subgraph:
1) in The Graph Explorer manually search for subgraph keywords in the Graph Explorer.
2) in the list or just navigate by direct link: https://thegraph.com/explorer/subgraph/[subgraph_name]
Here you will have basic information about the subgraph.
Here several points that could be useful.
Id (Manifest):
ID field contains the id of the subgraph manifest (subgraph.yaml) that is stored in IPFS. Moreover from here you can navigate to other related files stored in IPFS. You can read more information about manifest description here https://thegraph.com/docs/define-a-subgraph#the-subgraph-manifest
The manifest is a YAML file that defines the smart contracts, events, blocks and calls that the indexer should be paying attention to. In addition, the manifest maps event data to entities. Entities are objects that contain data and are the data stores that queries request data from. However, the data stored within entities is pulled directly from the blockchain and as a result is inherently unorganised.
Think of the manifest as an instruction manual that tells the indexers what information to pay attention to and how it should be treated. As new blocks are created the information held in the manifest is used to listen and index new events pertaining to a specific subgraph.
The manifest is a good place to assess the accuracy of the subgraph by ensuring that the correct contracts are referenced, the most updated ABI files are being used and the correct event/call handlers are inputted. It also shows how raw data is mapped to entities helping the curator assess whether the naming convention of entities is easy to understand.
Having the manifest you can do your further evaluation — read the Ethereum contract, validate graphql schema, check mappings, etc.
GitHub:
Some subgraphs do not have a link to the github repo, but each of them basically contains a github account.
You can also go to ‘subgraph.yaml’ (it’s the manifest file) on the subgraph Github page and compare it to the data obtained from https://ipfs.io. These two should be the same. Then you can also check ‘schema.graphql’ file to see what sort of graphql types are defined there. These typing define the structure of the data that you can query in The Explorer Playground.
The Graph Explorer
The Graph Explorer and its GraphQL playground is a useful way to explore and query deployed subgraphs on the hosted service.
Some of the main features are detailed below: https://thegraph.com/docs/query-the-graph#using-the-graph-explorer
In subgraph page the curators should research:
Schema
The schema of your data source — that is, the entity types, values, and relationships that are available to query — are defined through the GraphQL Interface Definition Langauge (IDL).
GraphQL schemas generally define root types for queries, subscriptions and mutations. The Graph only supports queries. The root Query type for your subgraph is automatically generated from the GraphQL schema that’s included in your subgraph manifest.
The schema will provide information on how complete and how useful the subgraph is. It will show the entities that can be queried and the values that are held within each entity. An assessment of the schema may highlight missing datasets that would be useful in the context of a particular subgraph. Not all subgraphs aim to be complex and therefore, the curator should take into account the ultimate goal of the subgraph when deciding on its efficacy.
Queries (Mappings):
In your subgraph schema you define types called Entities. For each Entity type, an entity and entities field will be generated on the top-level Query type. Note that query does not need to be included at the top of the graphql query when using The Graph.
More info here: https://thegraph.com/docs/graphql-api#queries
The mappings file transforms the lower level Ethereum data into the desired entities that are specified in the GraphQL schema file. To elaborate further, the manifest extracts the raw Ethereum data, and the mappings manipulate this data into the form that is set out in the schema.
The mappings file will provide a lot of great information to curators who are able to follow the AssemblyScript language used. It can provide a curator with insight into the complexity and usefulness of the subgraph by detailing how the raw data is being transmuted.
The Etherscan
The Etherscan can help to validate the subgraph complexity.
1. In the manifest file https://ipfs.io/ipfs/QmQ3yZiAvHff8xWWsjwsAD5pvUhD5TKHgARFaisTUAeT4s find which contracts your subgraph is using and which mappings it has. In my case
2. Navigate to the https://etherscan.io/ or https://dashboard.tenderly.co/explorer and search for the contract addresses from the manifest file.
3. Verify contract solidity code — how big and complex it is.
Summarize evaluation of Subgraph
At this point, you should have a pretty good understanding of what events the subgraph is recording from the contract and what data it is recording from these events. Now you will be able to evaluate the graph and answer the following questions and fill answers to such table:
· What the subgraph does?
· Does this subgraph look production-ready?
· Does this subgraph look like it would be useful to others?
· What changes would you make to the schema including additions or modifications to entities, fields, field types, relationships, or any other improvements?
Subgraphs are only as valuable as they are complete, accurate, and complex enough to support an application’s data needs. A very important skill of any curator is to evaluate subgraphs making sure they are ready for any dApp to use.
Thank you for reading, I hope it helps! Good luck!