Don't trust, verify: Indexing ENS Domains with Subsquid
Tags: tutorial, node, blockchain, typescript
Published: 2023-03-21 10:27:00

---
title: Don't trust, verify: Indexing ENS Domains with Subsquid
published: false
description: Thanks to Subsquid, **you can build your own indexer, that is able to map every** `tokenId` **to its corresponding ENS name**, allowing for querying in the opposite direction (get ENS name from `tokenId` ).
tags: tutorial, node, blockchain, typescript
cover_image: https://miro.medium.com/v2/resize:fit:4800/format:webp/1*vDfor7fuV7QQSmM52tWx2w.png
published: true
published_at: 2023-03-21 10:27 +0000
canonical_url: https://medium.com/subsquid/dont-trust-verify-indexing-ens-domains-with-subsquid-e1fa267923db
---

### Summary

I leveraged Subsquid's indexing framework to build my very own API that maps an ENS name to its `tokenId` and vice versa. Why this is needed or important? Read below 😉

The project discussed in this article is available at this repository, on the `ens-workshop` branch:

[**github.com/RaekwonIII/ethereum-name-service-indexing/**](http://github.com/RaekwonIII/ethereum-name-service-indexing/)

The repository is also configured, so you can run the project on [Gitpod](https://gitpod.io/#https://github.com/RaekwonIII/ethereum-name-service-indexing/tree/ens-workshop):

[![](https://gitpod.io/button/open-in-gitpod.svg align="center")](https://gitpod.io#https://github.com/RaekwonIII/ethereum-name-service-indexing/tree/ens-workshop)

### Introduction

{% embed https://www.youtube.com/watch?v=kfKBNy79zxA %}

I have recently published a series of videos and articles to educate about Blockchain Data Indexing.

It specifically discusses Ethereum, although it's applicable to all the EVM networks that Subsquid is available for, plus Substrate networks, with a couple of adaptations I'll probably discuss in a future article. It's a full course consisting of multiple ["lessons"](https://medium.com/subsquid/blockchain-data-indexing-a-full-course-lesson-1-cc0356528a57?source=your_stories_page-------------------------------------), introducing [Subsquid](https://subsquid.io/), the basic concepts, and a few practical tutorials to practice using the SDK.

While creating these tutorials, I choose [Ethereum Name Service](https://ens.domains/) as an example, because it's a famous project, and quite frankly, also because I take these changes to study some subjects I am interested in (sue me! 😛).

What I didn't realize was that, in doing so, I ended up discovering not one, but two peculiar aspects of the NFTs created by the project, and consequently, two strong cases in favor of blockchain data indexing, and more specifically, two great examples of Subsquid's superiority in this aspect.

### Ethereum Domains are NFTs…but they are odd

First of all, this needs to be clarified: whenever someone is registering a domain name on the [Ethereum Name Service app](https://app.ens.domains/), they are minting a Non-Fungible Token from the ENS collection.

This token looks and behaves like a common ERC-721 token: as a matter of fact, the project migrated the registrar in 2019, and since then, it's a contract that is compliant to this standard.

It does, however, have its peculiarities.

#### Token ID calculation

Unlike usual NFT collections, the `tokenId` of ENS name is the `uint256`representation of the hash of the label (`vitalik` for `vitalik.eth`). If someone wanted to derive the `tokenId` for a given ENS name (in this case: `vitalik`, which computes to 79233663829379634837589865448569342784712482819484549289560981379859480642508), they would have to do it like this:

```javascript
const ethers = require('ethers')
const BigNumber = ethers.BigNumber
const utils = ethers.utils
const name = 'vitalik'
const labelHash = utils.keccak256(utils.toUtf8Bytes(name))
const tokenId = BigNumber.from(labelHash).toString()
```

On the flip-side, deriving the ENS name from the `tokenId` is not as easy. The reason for this was that the fixed-length hash allows to register infinite length of names, but the trade-off is that, because the name is hashed, this is a one-way operation: you cannot obtain ENS names from a `tokenId`.

All of this is explained in greater detail on [ENS documentation page](https://docs.ens.domains/dapp-developer-guide/ens-as-nft).

#### Token URL and metadata

Simply put, the ENS contract does not have `.tokenURI`. The project has, instead, created a [separate metadata service](https://metadata.ens.domains/docs) which NFT marketplaces like OpenSea can fetch metadata for ENS such as registration data, expiration date, name length, etc.

The problem with this service? It’s using the `tokenId` to obtain the metadata.

#### Downsides of these oddities

Let's say you want to visualize the [Nyan cat image that Vitalik has associated with his vitalik.eth name](https://opensea.io/assets/ethereum/0x57f1887a8bf19b14fc0df6fd9b2acc9af147ea85/79233663829379634837589865448569342784712482819484549289560981379859480642508): how would you do that?

So this is exactly the adventure I embarked on.

### Let's get building!

The project in [Lesson 2 of the Blockchain Data Indexing course](https://link.medium.com/HuACn2CU0wb) (you can find the project on [this GitHub page](https://github.com/RaekwonIII/ethereum-name-service-indexing)) was already indexing ENS tokens, but in order to keep it simple and don't introduce too many variables to the learning process, it stopped at tracking transfers of tokens.

```javascript
const processor = new EvmBatchProcessor()
.setDataSource({
  chain: process.env.RPC_ENDPOINT,
  archive: "https://eth.archive.subsquid.io",
})
.addLog(contractAddress, {
  filter: [
    [
      events.Transfer.topic,
    ],
  ],
  data: {
    evmLog: {
      topics: true,
      data: true,
    },
    transaction: {
      hash: true,
    },
  },
});
```

#### Modifying the schema

Because the objective is to improve the current project and save additional data, we need to change the schema first. I have added new fields, such as `name`, `imageURI`, `uri` and the registrar expiration date as `expires` :

```javascript
type Token @entity {
  id: ID!
  owner: Owner
  name: String
  imageURI: String
  uri: String
  expires: BigInt,
  transfers: [Transfer!]! @derivedFrom(field: "token")
  contract: Contract
}

type Owner @entity {
  id: ID!
  ownedTokens: [Token!] @derivedFrom(field: "owner")
}

type Contract @entity {
  id: ID!
  name: String! @index
  symbol: String!
  totalSupply: BigInt!
  tokens: [Token!]! @derivedFrom(field: "contract")
}

type Transfer @entity {
  id: ID!
  token: Token!
  from: Owner
  to: Owner
  timestamp: BigInt!
  block: Int! @index
  transactionHash: String!
}
```

> After the publishing of Lesson 2, Subsquid published a new version of our CLI, with **improved commands**, make sure to install it with
> 
> `npm i -g @subsquid/cli@latest`
> 
> And if you are working on the same project from the lesson, make sure to add this [`commands.json` file](https://github.com/RaekwonIII/ethereum-name-service-indexing/blob/ens-workshop/commands.json) to your root folder.

Following these changes, we need to update the TypeScript models, so let's run the command:

```bash
sqd codegen
```

#### Processor configuration

What is truly interesting for the ENS project, is the ability to index the `NameRegistered`and `NameRenewed` events as well. This way, we are able to capture every time a name is registered, renewed and when it is transferred to a different owner.

For this, we need to change the part where `EvmBatchProcessor` class is instantiated and configured:

```javascript
const processor = new EvmBatchProcessor()
  .setDataSource({
    chain: process.env.RPC_ENDPOINT,
    archive: "https://eth.archive.subsquid.io",
  })
  .addLog(contractAddress, {
    filter: [
      [
        events.NameRegistered.topic,
        events.NameRenewed.topic,
        events.Transfer.topic,
      ],
    ],
    data: {
      evmLog: {
        topics: true,
        data: true,
      },
      transaction: {
        hash: true,
      },
    },
  });
```

Of course, we also need to adjust how we process each item, because we will have logs generated by three different events, with three different topics, and they have to be treated separately:

```javascript
processor.run(new TypeormDatabase(), async (ctx) => {
  const ensDataArr: ENSData[] = [];
  for (let c of ctx.blocks) {
    for (let i of c.items) {
      if (i.address === contractAddress && i.kind === "evmLog") {
        if (i.evmLog.topics[0] === events.NameRegistered.topic) {
          const ensData = handleNameRegistered({
            ...ctx,
            block: c.header,
            ...i,
          });
          ensDataArr.push(ensData);
        }
        if (i.evmLog.topics[0] === events.NameRenewed.topic) {
          const ensData = handleNameRenewed({
            ...ctx,
            block: c.header,
            ...i,
          });
          ensDataArr.push(ensData);
        }
        if (i.evmLog.topics[0] === events.Transfer.topic) {
          const ensData = handleTransfer({
            ...ctx,
            block: c.header,
            ...i,
          });
          ensDataArr.push(ensData);
        }
      }
    }
  }

  await saveENSData(
    {
      ...ctx,
      block: ctx.blocks[ctx.blocks.length - 1].header,
    },
    ensDataArr
  );
});
```

Notice the three different `handle*` functions. They are not too different from each other, but I'd argue they are still different enough to deserve to be separate:

```javascript
function handleNameRegistered(
  ctx: LogHandlerContext<
    Store,
    { evmLog: { topics: true; data: true }; transaction: { hash: true } }
  >
): ENSData {
  const { evmLog, block, transaction } = ctx;

  const { id, owner, expires } = events.NameRegistered.decode(evmLog);

  const ensData: ENSData = {
    id: `${transaction.hash}-${evmLog.address}-${id.toBigInt()}-${
      evmLog.index
    }`,
    to: owner,
    tokenId: id.toBigInt(),
    timestamp: BigInt(block.timestamp),
    expires: expires.toNumber(),
    block: block.height,
    transactionHash: transaction.hash,
  };

  return ensData;
}

function handleNameRenewed(
  ctx: LogHandlerContext<
    Store,
    { evmLog: { topics: true; data: true }; transaction: { hash: true } }
  >
): ENSData {
  const { evmLog, block, transaction } = ctx;

  const { id, expires } = events.NameRenewed.decode(evmLog);

  const ensData: ENSData = {
    id: `${transaction.hash}-${evmLog.address}-${id.toBigInt()}-${
      evmLog.index
    }`,
    tokenId: id.toBigInt(),
    timestamp: BigInt(block.timestamp),
    expires: expires.toNumber(),
    block: block.height,
    transactionHash: transaction.hash,
  };

  return ensData;
}

function handleTransfer(
  ctx: LogHandlerContext<
    Store,
    { evmLog: { topics: true; data: true }; transaction: { hash: true } }
  >
): ENSData {
  const { evmLog, block, transaction } = ctx;

  const { from, to, tokenId } = events.Transfer.decode(evmLog);

  const ensData: ENSData = {
    id: `${transaction.hash}-${evmLog.address}-${tokenId.toBigInt()}-${
      evmLog.index
    }`,
    from,
    to,
    tokenId: tokenId.toBigInt(),
    timestamp: BigInt(block.timestamp),
    block: block.height,
    transactionHash: transaction.hash,
  };
  return ensData;
}
```

#### Augment on-chain data with external APIs

We are getting to the juicy bit here.

Finally, I have explored the `tokenId` and `tokenURI` issues mentioned above, and changed the `saveENSData` function to:

* Fetch the token metadata from [ENS metadata service](https://metadata.ens.domains/docs)
    
* Process this information and extract the name of a given token (establishing the `tokenId` ↔️ `name` relation) and the URL of the image
    
* Save this data in the `Token` entity, on the database
    

```javascript
async function saveENSData(
  ctx: BlockHandlerContext<Store>,
  ensDataArr: ENSData[]
) {
  const tokensIds: Set<string> = new Set();
  const ownersIds: Set<string> = new Set();

  for (const ensData of ensDataArr) {
    tokensIds.add(ensData.tokenId.toString());
    if (ensData.from) ownersIds.add(ensData.from.toLowerCase());
    if (ensData.to) ownersIds.add(ensData.to.toLowerCase());
  }

  const transfers: Set<Transfer> = new Set();

  const tokens: Map<string, Token> = new Map(
    (await ctx.store.findBy(Token, { id: In([...tokensIds]) })).map((token) => [
      token.id,
      token,
    ])
  );

  const owners: Map<string, Owner> = new Map(
    (await ctx.store.findBy(Owner, { id: In([...ownersIds]) })).map((owner) => [
      owner.id,
      owner,
    ])
  );

  for (const ensData of ensDataArr) {
    const {
      id,
      tokenId,
      from,
      to,
      block,
      expires,
      transactionHash,
      timestamp,
    } = ensData;

    // the "" case handles absence of sender, which means it's not an actual transaction
    // it's the name being registered
    let fromOwner = owners.get(from || "");
    if (from && fromOwner == null) {
      fromOwner = new Owner({ id: from.toLowerCase() });
      owners.set(fromOwner.id, fromOwner);
    }

    // the "" case handles absence of receiver, which means it's not an actual transaction
    // it's the name being renewed
    let toOwner = owners.get(to || "");
    if (to && toOwner == null) {
      toOwner = new Owner({ id: to.toLowerCase() });
      owners.set(toOwner.id, toOwner);
    }

    const tokenIdString = tokenId.toString();
    let token = tokens.get(tokenIdString);
    if (token == null) {
      token = new Token({
        id: tokenIdString,
        contract: await getOrCreateContractEntity(ctx.store),
      });
      tokens.set(token.id, token);
    }
    token.owner = toOwner;

    if (expires) {
      token.expires = BigInt(expires);
      if (expires * 1000 < Date.now()) {
        // ctx.log.info(`Token ${tokenId} has expired`);
        continue;
      }
      let name = "",
        uri = "",
        imageURI = "";
      try {
        // https://metadata.ens.domains/docs
        let tokenURI = `https://metadata.ens.domains/mainnet/${contractAddress}/${tokenId}`;
        const meta = await axios.get(tokenURI);

        imageURI = meta.data.image;
        name = meta.data.name;
        uri = meta.data.url;
      } catch (e) {
        ctx.log.warn(`[API] Error during fetch token ${tokenId} metadata`);
        if (e instanceof Error) ctx.log.warn(`${e.message}`);
      } finally {
        token.name = name;
        token.uri = uri;
        token.imageURI = imageURI;
      }
    }

    if (toOwner && fromOwner) {
      const transfer = new Transfer({
        id,
        block,
        timestamp,
        transactionHash,
        from: fromOwner,
        to: toOwner,
        token,
      });

      transfers.add(transfer);
    }
  }

  await ctx.store.save([...owners.values()]);
  await ctx.store.save([...tokens.values()]);
  await ctx.store.save([...transfers]);
}
```

The important part is where the metadata service API is called, using `axios` library:

```javascript
let tokenURI = `https://metadata.ens.domains/mainnet/${contractAddress}/${tokenId}`;
const meta = await axios.get(tokenURI);
```

I want to take a moment to emphasize the ability to call arbitrary external services to obtain additional information, and add to the data we extract from the blockchain.

This way, we are building a service, providing data that would not be possible to produce by purely relying on the blockchain.

### Don't trust, verify

The [ENS documentation acknowledges the shortcomings of their architectural choice](https://docs.ens.domains/dapp-developer-guide/ens-as-nft#deriving-ens-name-from-tokenid), and provides an alternative, which is using a third party service, or downloading this service's dataset.

One might brush this off as non-consequential, but if you truly subscribe to the motto *“Don’t Trust, Verify”*, being able to obtain this information independently, in a decentralized way, and without relying on third party services is paramount.

#### Local indexing

If you have followed along and want to try this at home, you should delete the previous migration file:

```bash
rm -rf db/migrations/*js
```

Launch the database Docker container and generate a new migration file:

```bash
sqd up
sqd migration
```

Launch the processor:

```bash
sqd process
```

And finally, in a separate terminal window, launch the GraphQL service:

```bash
sqd serve
```

We can, for example, test out our initial question: "what is the image linked to `vitalik.eht` ENS name?" with this query:

```graphql
query MyQuery {
  tokens(where: {name_contains: "vitalik"}) {
    expires
    id
    imageURI
    name
    uri
    owner {
      id
    }
  }
}
```

![](https://cdn-images-1.medium.com/max/1600/1*l0MiKwvXmqucIFf1x6RSow.png)

And if we copy paste the `imageURI` in our browser, we'll eventually land on vitalik's Nyan cat animated image:

![](https://cdn-images-1.medium.com/max/1600/1*o2hZOgamCR4xv_P1lHCBtA.png)

#### Gitpod

Another option, as mentioned at the start of the article, is to run the project on [Gitpod](https://gitpod.io/#https://github.com/RaekwonIII/ethereum-name-service-indexing/tree/ens-workshop) by clicking on the related link on the [project's README file](https://github.com/RaekwonIII/ethereum-name-service-indexing/blob/ens-workshop/README.md):

![](https://cdn-images-1.medium.com/max/1600/1*nuql7MBQFRqWRzoJnhWrmQ.png)

Which will launch the entire project in a cloud-hosted execution environment for us.

#### Conclusions

What I showed you here is that, thanks to Subsquid, **you can build your own indexer, that is able to map every** `tokenId` **to its corresponding ENS name**, allowing for querying in the opposite direction (get ENS name from `tokenId` ).

If you found this article interesting, and you want to read more, please [follow me](https://linktr.ee/raekwoniii) and most importantly, Subsquid.

**Subsquid socials:**

[Twitter](https://www.twitter.com/subsquid) | [Discord](https://discord.gg/subsquid) | [LinkedIn](https://www.linkedin.com/company/subsquid) | [Telegram](https://t.me/subsquid) | [GitHub](https://github.com/subsquid) | [YouTube](https://www.youtube.com/@subsquid)