A few sneak peeks into Hey.com technology (VI - Template page updates)
Tags: rails, ruby, webdev, javascript
Published: 2020-06-26 18:13:02

> _**Update as of Dec 2020**: now that the Hey support code has been extracted and officially released under the name **[Hotwire](https://hotwire.dev/)**, we know that this article actually speaks about the **[Turbo Streams](https://turbo.hotwire.dev/handbook/streams)**. Basecamp renamed and polished a few things but the principles stay the same as described here._

In parts [III](https://dev.to/borama/a-few-sneak-peeks-into-hey-com-technology-iii-turbolinks-frames-5e4a) and [IV](https://dev.to/borama/a-few-sneak-peeks-into-hey-com-technology-iv-turbolinks-frames-continued-2bn) we’ve seen how **partial page updates** can be done via “Turbolinks frames“ (the `<turbolinks-frame>` element), either automatically or upon request. That was nice and cool but that’s not the only way to do partial updates that Hey supports. In fact, I think that it’s even not the nicest and coolest way to do them! At least when compared to today’s topic - **the `<template>` element**… 

This element (and associated JavaScript code) handles page updates in many places in Hey. Let’s now study a use case with just about the perfect level of complexity − non-trivial but not too complex: the “Mark seen” function. But first of all, we need to talk about how the main email listing is organized in the Imbox HTML code.

### Email listing in Imbox

We all know that the Imbox page has two lists of emails: the new ones and the previously seen ones below. But how do they look in the HTML source? Let’s point our dev tool picker:

![Mails listing in Imbox HTML](https://dev-to-uploads.s3.amazonaws.com/i/ohllat3k83kqglv15zcy.png)

Oh, it’s just a single list! A **single flat list of `<article>` elements**. How do the new and previously seen emails differ then? As is apparent from the image, the already seen emails have a `data-seen="true"` attribute, whereas the new ones don’t.

OK, but how is the “PREVIOUSLY SEEN” header made then? Turns out it’s pure CSS and it’s a neat trick (see the image below): the CSS rule with the header in a `:before` pseudo-class targets the first element in the `.postings` (emails) list that has the `data-seen` attribute set. (Either the very first such element in the list when there are no new emails, or the first element after another that has not this attribute set.) So, simply adding this attribute to the email element prepends it with the header!

![The Previously seen CSS rules](https://dev-to-uploads.s3.amazonaws.com/i/j39cw16hz5q75sllrkdb.png)

I think that this is a nice example of a **presentation that is handled by a simple HTML structure with a few “sprinkles” of specific data attributes, CSS rules or a bit of JavaScript** instead of e.g. handling all this logic in a more complex JS code. It’s an approach which uses the combined strengths of the “classic” technologies that, overall, have been here since many many years ago and are thus very well supported, tested and familiar to devs. No need to learn a new big framework every one or two years! This **“composition pattern”** can be seen …just about everywhere in Hey and I find it very sensible and appealing. And, most importantly, it plays very well with partial page updates via template elements…

### The "Mark seen" action analysis

So what goes on when you select a new email and click on the “Mark seen” popup menu item? Let’s have a look!

![Mark seen menu item](https://dev-to-uploads.s3.amazonaws.com/i/r6jalivqkencvwupajgx.png)

Not surprisingly, the Mark seen item is a **standard HTML form**. It triggers a POST action to the server and the only data it sends along is the IDs of the email(s) selected. As Hey employs Turbolinks, the form is submitted asynchronously via AJAX by default. 

> BTW, the `data-remote="true"` attribute used to do a similar thing in [Rails UJS](https://github.com/rails/rails-ujs), included by default [in ActionView](https://github.com/rails/rails/commit/ad3a47759e67a411f3534309cdd704f12f6930a7). Rails UJS, or the corresponding ActionView JS code, is **not** included among Hey JS scripts so I assume, the new Turbolinks handles the async form posting in a different way and this attribute is actually unused.

Anyway, this is not just a standard async form posting, there is one important detail going on: when Turbolinks code intercepts the form `submit` event, it first **dispatches its own custom event** called `turbolinks:before-fetch-request`. (This can be seen in the `turbolinks/dist/fetch_request` module, which is probably transpiled from TypeScript or similar language and is unfortunately not properly source-mapped, so it’s harder to grasp…) Still before actually fetching the AJAX request, this custom event is handled by JS code in `initializers/page_updates.js` and it **modifies the `Accept` header in the request**:

![Turbolinks modifies the Accept header](https://dev-to-uploads.s3.amazonaws.com/i/xavopoimmincxww6ngxc.png)

What does it mean? The `Accept` header [tells](https://tools.ietf.org/html/rfc7231#section-5.3.2) the server what Media types (i.e. “types of data”, often called “MIME types”) the browser expects to receive back in the response. Turbolinks adds the `"text/html; page-update"` media type. Standards-wise, this is a common `text/html` media type with a custom [parameter](https://tools.ietf.org/html/rfc7231#section-3.1.1.1) denoting that the **browser will accept a “page update” response from the server** and that the response should generally be a HTML response, of course!

So, to sum it up, the AJAX form submit request adjusted by Turbolinks looks like this in the Network tab:

![The Accept header in form submit request](https://dev-to-uploads.s3.amazonaws.com/i/qouzq86nzwag9fbnymvt.png)

And the server indeed responds with the “page update” media type as can be seen in the `content-type` response header:

![The content-type in form submit response](https://dev-to-uploads.s3.amazonaws.com/i/bwx0mrds8htws16vphy1.png)

So what does such response look like? Let’s take a look at its body:

![Form submit response body](https://dev-to-uploads.s3.amazonaws.com/i/739yc7w3bav1phncsq3b.png)

Oh at last, here they are, the `<template>` elements!

### Processing the page update response

Right after Turbolinks receives the response, it again triggers a custom event, `turbolinks:before-fetch-response` (in `fetch_request.js`), which allows some special treatment of the response. This event is again handled in the same initializer as was the request event − `page_updates.js` − in the `handlePageUpdates` method. This method first checks for the custom media type in the response `content-type` header and if it is present, it calls the **`processPageUpdates` method from a tiny custom library called `page-updater`**:

![Page update response processing in Turbolinks](https://dev-to-uploads.s3.amazonaws.com/i/rs8nv099yvyyiimeabm1.png)

> I assume the reason why this JS code is not part of Turbolinks but is a separate library is that it is, indeed, independent − you can employ such a thing anywhere, you just need to render proper response content on your server.

The `page-updater` library is pleasantly small and concise. All it does is it extracts all `<template>` elements that have the `data-page-update` attribute set, **parses them into individual page update commands and executes them**, one by one. 

The commands are encoded in the `<template>` elements in a simple way:

```html
<template data-page-update="command#html-id">
...
</template>
```

where **`command` is the operation** that is about to be run and `html-id` is the… you guessed it… **HTML ID** of the element the operation should be run against. Optionally, the `<template>` element can also have its own content, which is needed for some commands. **There are five distinct page update commands defined: append, prepend, replace, update and remove**. They are quite self-explaining, I think, maybe I’ll just add that the `update` command leaves the target element intact and only updates its content whereas `replace` takes away the content as well as the target element itself. Perhaps, it’ll be best to show a picture instead of a ”thousand words“:

![Page update commands definitions](https://dev-to-uploads.s3.amazonaws.com/i/6cb2k84pl2rie0bcfrv9.png)

> Unfortunately, **there is no `morph` command**, i.e. you cannot make only the smallest possible changes to the element (such as changing only its class or attribute), you have to resort to updating the whole thing. I see this as a disadvantage when compared to [StimulusReflex](https://docs.stimulusreflex.com/morph-modes) which leverages the awesome [morphdom library](https://github.com/patrick-steele-idem/morphdom) to effectively morph elements on the page. 

And by the way, as it turns out, **`<template>` element is [defined in the HTML standard](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/template)** and denotes “an element for holding HTML that is not to be rendered immediately when a page is loaded but may be instantiated subsequently during runtime using JavaScript”. It seems like a perfect fit for what this element actually does in Hey!

### The “Mark seen” response processing

So, let’ get back to the “Mark seen” action. In the response body image above we can see that the response contains two page update commands:

1. **remove** the email element from the page,
2. **prepend** the new version of the email element (given in the response) at the beginning of emails list.

Have you spotted anything weird here? How come the returned mail element is put _at the beginning_ of the emails listing? We already know that we need this email element somewhere in the _middle_ of the list, as it’s a single flat list and we still have some unseen emails at the top!

_Eh, you know what? This is getting long and I’m going to cowardly cut this post here. I have a lot more thoughts about this topic! While I generally like this approach very much, I can see some possible caveats, too. Next time, I’ll finish the partial page updates analysis as well as try to discuss this pattern overall, and compare it to the “Turbolinks frames” pattern. Stay tuned and in the mean time you may try to solve the puzzle with mails ordering…!_ 😉