The mysteries of GraphQL clients' cache - The Introduction
Tags: graphql, opensource, tutorial, webdev
Published: 2022-09-07 07:56:50

Recently I completed an internship at Microsoft where I had the privilege to work with people who are experts in their fields, more specifically in the field of GraphQL. I had numerous opportunities to learn from them and now I would like to teach you something. Let me take you on a journey through the midst of GraphQL clients & their cache. 

>This is a two-part series, where I'll talk about GraphQL client caches and compare the client's cache performance.
>
Part 1: [The mysteries of GraphQL clients' cache - The Introduction](https://dev.to/andycko/the-mysteries-of-graphql-clients-cache-3j3m-temp-slug-7832453?preview=704245c70cba2f6b37ca453f41aaaed0a29c1a423b89f5f5038162d893caa34c6344981ce974c421bafb6f3cf4036cbef634bcc07704523aa2c16364)
Part 2: [The mysteries of GraphQL clients' cache - The Showdown](https://dev.to/andycko/the-mysteries-of-graphql-clients-cache-the-showdown-4211)

---

> "There are only two hard things in Computer Science: cache invalidation and naming things."
-- Phil Karton

As every blog post that even just remotely mentions the subject of caches, I too have felt the need to include this infamous quote. Regardless if we are going to speak about cache invalidation or not, this quote always brightens up the mood a little bit.

## A quick intro to GraphQL clients
The conventional (but not only) way to use GraphQL is as an API that communicates through HTTP POST requests. Because the response of a GraphQL API is a bit more complex than a simple REST response, libraries have been built to make your life easier when communicating with such APIs. Libraries such as Apollo client, Relay, URQL, etc., help you automatically handle things like batching, **caching**, constructing queries, managing UI state and much more. 

With GraphQL becoming ever-so-popular, the support and development for these have skyrocketed in the past few years. When starting a new project, there is plenty to choose, but the question is, which one is the best for you?

## Caching - not the [geo](https://www.geocaching.com/play) one
As "hinted" in the name of this post, our main focus today is going to be caching. Caching in GraphQL, as anywhere else, is used to save data received from some data layer (eg. server) so the next time our application needs that data, it can read it from our cache memory instead of doing another network request to our data layer. This way, we can save precious resources and minimize client load times. The above-explained type of cache can also be called an In-Memory cache.

However, implementing and maintaining a cache is not as easy as it sounds and every client has their way of doing it. I have researched mainly two popular clients - **Apollo client** and **Relay**. 

Before I start speaking about this thought, let me explain what is data normalization, and why is it important to us.

### Data normalization
Data normalization is the process of restructuring some data to reduce redundancy. You might have heard about this from relational databases, where you have 5 normal forms to get through before you can even start thinking about being happy.. :) Both clients have a normalized cache (which is the de facto standard now) and they need to perform this process to convert the JSON blob they receive from the GraphQL server into a relational structure. The algorithm used for normalization varies between them, however, the principles remain the same. 


### Apollo client
As a GraphQL client - Apollo is the more simplistic and flexible of the two. It provides an easier way of getting started, a bit more comprehensive documentation, and perhaps better community support. However, when it comes to cache implementation, it is, unfortunately, lacking behind. 

However, before we get to compare the performance of the two, let me explain how Apollo's cache works. As mentioned previously, they are using an In-Memory cache and this consists of two main parts - **`EntityStore`** and **`ResultCache`**.

**`EntityStore`** is the main cache which holds normalized data in a flat lookup table, therefore when data is read from it, it needs to be de-normalized.

**`ResultCache`** has been introduced to help with the denormalization problem. Therefore, when the first read of a query is executed against the EntityStore the de-normalized data is memoized in the ResultCache and this then makes all upcoming reads of the identical query very fast. However, this comes with the overhead of having to write into the ResultCache on every first read operation on some query that has not yet been memoized. (The match has to be 1:1 exact here)

#### Normalization algorithm
As previously mentioned, Apollo maintains a normalized In-Memory cache by default, and now I'd like to explain in a few words how their normalization algorithm works. 

Firstly, let's say we have a GraphQL query to get all the users, which looks like this:
```{graphql}
query getAllUsers {
  users {
    id
    firstname
    lastname
  }
}
```

When a GraphQL response reaches the client, it comes in a JSON format, looking something like this:
```{json}
{
  "data": {
    "users": [
      {...},
      {...},
      {...},
    ]  
  }    
}
```
The contents of the `data` object are the actual response, therefore from now on, we will only work with what is inside of `data`. 
Now, the first step to normalize this data would be to split our `users` Array into single objects.
```{json}
{
  "__typename": "User",
  "id": 1,
  "firstname": "John",
  "lastname": "Doe"
}

{...}

{...}
```
Notice, that a `__typename` field has been added to our response, even though we have not requested it in our query. This is because Apollo client requests this field automatically, even if you don't explicitly do so. In the next step, you will see why.

Now that we have extracted all our objects, we can perform another step of normalization, which would be creating a globally unique identifier for every object, so that it can be saved in a key-value type lookup table (Hashmap). By default, Apollo client uses a composite key made of the `__typename` + `id`. Now you see, why Apollo client requested the `__typename` field automatically. However, not all objects in our response always have a unique `id` which could be in the composite key, therefore Apollo gives us the possibility to choose which fields we want to use for creating this unique identifier key using the `typePolicies` setting in the `InMemoryCache` [config](https://github.com/apollographql/apollo-client/blob/main/src/cache/inmemory/inMemoryCache.ts#L67).

Once that we have created the unique identifier for an object, we need to look into it and find any nested objects. Should there be any, they will need to be extracted, and assigned with a unique identifier key, which will then be placed in their original position as a reference. See the following example:
```{json}
{
  "__typename": "User",
  "id": 1,
  "firstname": "John",
  "lastname": "Doe",
  "address": {
    "__typename": "Address",
    "id": 1,
    "line1": ...,
    ...
  }
}

// The cache lookup table would look something like this after the normalization of the above object.

{
  "User:1": {
    "id": 1,
    ...
    "address": {
      "__ref": "Address:1"
    }
  },
  "Address:1": {
    "id": 1,    
    ...
  }
}
```

>However, it is important to mention, that should these be simply scalar fields, let's say an array of objects, which are not a GraphQL type, then these will **not** get extracted from the object.

And now, you know the overview of how Apollo performs normalization, not that hard, right? :) 

Next, let's jump over to our friend Relay.

### Relay
As previously mentioned, Apollo is the more simplistic and versatile one, whereas Relay is the more optimized and narrow-scoped one. What I mean by this, is that whereas Apollo Client has frameworks for multiple languages, Relay was made specifically for React and is highly optimised. 

However, when it comes to cache, they are not that different from their core implementations. If you forget about Apollo's `ResultCache` for a minute, then they are indeed very similar. However, since Relay is based on granular `fragments` instead of whole `queries` as Apollo is, it does not need anything such as a `ResultCache`. This is why Apollo client had to add another layer of complexity to their `InMemoryCache` to optimize for second reads. (Read re-renders of whole queries are very frequent in apollo, and therefore the memoization process performed by the `ResultCache` helps to speed this up)

**`Store`** is the one source of truth for an instance of `RelayRuntime` which holds a collection of entities presented by the `RecordSource` type. These are (as before) a collection of normalized records belonging to a single query/mutation/etc. The query goes through a normalization process and its entities are extracted and saved into `Records` which are then all collected in one `RecordSource`. The `RecordSource` object is then merged into the `Store` and subscribers (observers) to a fragment that was affected are notified.

#### Normalization algorithm
Since I have provided an in-depth explanation of the normalization algorithm in Apollo, I am not going to do the same here, as they are similar in their nature. However, one thing, that I believe is worth mentioning, is how they choose their unique identifiers for normalized records.

As you already know, Apollo client uses the composite key of `__typename` + `id` fields and only extracts nested objects if they have GraphQL types (not scalars). However, Relay, on the other hand, takes a different approach, where **every** nested object, is extracted into a `Record` and is assigned a `DataId`. This is a globally unique identifier in the scope of the cache and it can be made from the `id` field of the objects, or if they don't have such fields, it can be based on the path to the record from the nearest object with an id (such path-based ids are called client ids). Thanks to this logic, even nested scalar objects are always extracted and normalized.

## Next up
Now that I have explained the basics of how GraphQL client cache works, I would like to show you a showdown between the clients, comparing their caches head-to-head. Use this link to read more about that in Part 2 - [The mysteries of GraphQL clients' cache - The Showdown](https://dev.to/andycko/the-mysteries-of-graphql-clients-cache-the-showdown-3mpe-temp-slug-8775347?preview=cb1919c6d006563cba0c6feef0275c95fd198dde21ae1d8220b9f9f3472f896386b2e9c06fc7cde54780670f571a3d0b27a2f49922f554a518bdc522).