Web Caching - ETag/If-None-Match
Tags: webdev, cache, performance, tutorial
Published: 2023-11-21 15:40:00

> [Support code](https://github.com/didof/post-caching)

In the previous [post](https://dev.to/didof/the-art-of-efficient-web-browsing-public-resources-27hl), we explored the usefulness of the `Last-Modified` Response Header and `If-Modified-Since` Request Header. They work really well when dealing with an endpoint returning a file.

What about **data retrieved from a database** or **assembled from different sources**?

<center>
<table border="1">
    <thead>
        <tr>
            <th>Request</th>
            <th>Response</th>
            <th>Value example</th>
        </tr>
    </thead>
    <tbody>
        <tr>
            <td><code>Last-Modified</code></td>
            <td><code>If-Modified-Since</code></td>
            <td><code>Thu, 15 Nov 2023 19:18:46 GMT</code></td>
        </tr>
        <tr>
            <td><code>ETag</code></td>
            <td><code>If-None-Match</code></td>
            <td><code>75e7b6f64078bb53b7aaab5c457de56f</code></td>
        </tr>
    </tbody>
</table>
</center>

Also here, we have a *tuple of headers*. One must be provided by the requester (`ETag`), while the other is returned by the sender (`If-None-Match`). The value is a **hash generated on the content of the response**.

If you want to go directly to using headers, go to the [endpoint](#endpoint). Otherwise, observe (but don't spend too much time on) the implementation.

## Preparation

For simplicity, we use an in-memory DB. It is exposed via the endpoint `/db`. It contains a list of `posts`. Each post contains a `title` and a `tag`. Posts can be added via `POST`, and modified via `PATCH`. 

Retrieval is via a `GET` function, which optionally filters by `tag`.

{% details <code>src/db.mjs</code> %}
```js
import { getJSONBody } from "./utils.mjs";

const POSTS = [
  { title: "Caching", tag: "code" },
  { title: "Headers", tag: "code" },
  { title: "Dogs", tag: "animals" },
];

export function GET(tag) {
  let posts = POSTS;
  if (tag) posts = posts.filter((post) => post.tag === tag);
  return posts;
}

export default async function db(req, res) {
  switch (req.method) {
    case "POST": {
      const [body, err] = await getJSONBody(req);
      if (err) {
        res.writeHead(500).end("Something went wrong");
        return;
      }

      POSTS.push(body);
      res.writeHead(201).end();
      return;
    }
    case "PATCH":
      const [body, err] = await getJSONBody(req);
      if (err) {
        res.writeHead(500).end("Something went wrong");
        return;
      }

      POSTS.at(body.index).title = body.title;
      res.writeHead(200).end();
      return;
  }
}
```
{% enddetails %}

{% details <code>src/utils.mjs</code> %}
```js
export function getURL(req) {
  return new URL(req.url, `http://${req.headers.host}`);
}

export async function getJSONBody(req) {
  return new Promise((resolve) => {
    let body = "";
    req.on("data", (chunk) => (body += chunk));
    req.on("error", (err) => resolve([null, err]));
    req.on("end", () => resolve([JSON.parse(body), null]));
  });
}
```
{% enddetails %}

## Endpoint

By registering the *db*, we will be able to modify the content of the responses in real-time, appreciating the usefulness of ETag.
Also, let's register and create the `/only-etag` endpoint.

```js
// src/index.mjs
import { createServer } from "http";
import db from ".src/db.mjs";
import onlyETag from "./src/only-etag.mjs";
import { getURL } from "./src/utils.mjs";

createServer(async (req, res) => {
  switch (getURL(req).pathname) {
    case "/only-etag":
      return await onlyETag(req, res);
    case "/db":
      return await db(req, res);
  }
}).listen(8000, "127.0.0.1", () =>
  console.info("Exposed on http://127.0.0.1:8000")
);
```

The `onlyETag` endpoint accepts an optional query parameter `tag`. If present, it is used *to filter the retrieved posts*.
Thus, the template is loaded in memory.

{% details <code>src/views/posts.html</code> %}
```html
<html>
  <body>
    <h1>Tag: %TAG%</h1>
    <ul>%POSTS%</ul>
    <form method="GET">
      <input type="text" name="tag" id="tag" autofocus />
      <input type="submit" value="filter" />
    </form>
  </body>
</html>
```
When submitted, the form uses as `action` the current route (`/only-etag`) appending as query parameter the `name` attribute. For example, typing `code` in the input and submitting the form would result in `GET /only-etag?name=code`), **No JavaScript required**!

***
{% enddetails %}

And the posts are injected into it.

```js
import * as db from "./db.mjs";
import { getURL, getView, createETag } from "./utils.mjs";

export default async (req, res) => {
  res.setHeader("Content-Type", "text/html");

  const tag = getURL(req).searchParams.get("tag");
  const posts = await db.GET(tag);

  let [html, errView] = await getView("posts");
  if (errView) {
    res.writeHead(500).end("Internal Server Error");
    return;
  }

  html = html.replace("%TAG%", tag ?? "all");
  html = html.replace(
    "%POSTS%",
    posts.map((post) => `<li>${post.title}</li>`).join("\n")
  );

  res.setHeader("ETag", createETag(html));

  res.writeHead(200).end(html);
};
```

As you notice, before dispatching the response, the **ETag** is generated and included under the `ETag` Response header.

```js
// src/utils.mjs
import { createHash } from "crypto";

export function createETag(resource) {
  return createHash("md5").update(resource).digest("hex");
}
```

> Changing the content of the resource changes the <b>E</b>ntity **Tag**.

Performing the request from the browser you can inspect the Response Headers via the Network tab of the Developer Tools.
```http
HTTP/1.1 200 OK
Content-Type: text/html
ETag: 4775245bd90ebbda2a81ccdd84da72b3
```

If you refresh the page, you'll notice the browser adding the `If-None-Match` header to the request. The value corresponds of course to the one it received before.
```http
GET /only-etag HTTP/1.1
If-None-Match: 4775245bd90ebbda2a81ccdd84da72b3
```

As seen in the previous posts per `Last-Modified` and `If-Modified-Since`, let's instruct the endpoint to deal with `If-None-Match`.
```js
export default async (req, res) => {
  res.setHeader("Content-Type", "text/html");

  retrieve (filtered) posts; // as seen before
  load html; // as seen before
  fill template; // as seen before

  const etag = createETag(html);
  res.setHeader("ETag", etag);
  const ifNoneMatch = new Headers(req.headers).get("If-None-Match");
  if (ifNoneMatch === etag) {
    res.writeHead(304).end();
    return;
  }

  res.writeHead(200).end(html);
};
```

Indeed, subsequent requests on the same resource return `304 Not Modified`, instructing the browser to use previously stored resources. Let's request:
- `/only-etag` three times in a row;
- `/only-etag?tag=code` twice;
- `/only-etag?tag=animals` twice;
- `/only-etag`, without tag, once again;

![A screenshot of the Developer Tools showing what is listed above.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bspcf1w7gguslu07v6qa.png)

> The presence of the query parameter determines a change in response, thus in **ETag**.

> Notice the last one. It does not matter that there have been other requests in the meantime; the browser keeps a map of requests (including the query parameters) and ETags.

### Detect entity change

To further underscore the significance of this feature, let's add a new post to the DB from another process.
```bash
curl -X POST http://127.0.0.1:8000/db \
-d '{ "title": "ETag", "tag": "code" }'
```

And request again `/only-etag?tag=code`.
![The same cached response is not useful anymore since the returned ETag is now different.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xpo0hmpoord616pokabe.png)
After the *db* has been updated, the same request generated a different **ETag**. Thus, the server sent the client a new version of the resource, with a newly generated **ETag**. Subsequent requests will fall back to the expected behavior.

The same happens if we modify an element of the response.
```bash
curl -X PATCH http://127.0.0.1:8000/db \
-d '{ "title": "Amazing Caching", "index": 0 }'
```

![The same behaviour seen before is replicated if the content changes once again.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/960g0w3ycv1boii1976b.png)

> While ETag is a more versatile solution, applicable regardless of the data type since it is content-based, it should be considered that **the server must still retrieve and assemble the response**, then pass it into the hashing function and compare it with the received value.

Thanks to another header, `Cache-Control`, it is possible to optimize the number of requests the server has to process.