Familiar day #3 - The Revenge of the JavaScript
Published: 2019-06-20 23:25:28

For people who are new, I'm blogging along as I'm building my new tabletop roleplaying assistant. 

## Where are we at?

I've set some goals. I've created some terraform configuration and a Google Cloud account. I deployed a Compute Engine instance that does nothing. 

## What are we doing today?

We're going to create some storage - we're going to set up a [Terraform Backend](https://www.terraform.io/docs/backends/). These are great for a few reasons, avoiding local configuration, avoiding storing sensitive values on disk (or accidentally committing them to Git 😱) and allowing collaboration with multiple people. It needs a Google Cloud Storage bucket to work, so we're going to create that with the CLI - we could set it up with terraform too, but we don't want to accidentally destroy it when tearing down resources.

Then, we're going to hopefully start some coding, and create a server that actually returns a response. I might get around to deploying it, so let's see.

We have to use the `gsutil` tool that came with the Google Cloud SDK we installed earlier to create Cloud Storage. You can find instructions to follow along [here](https://cloud.google.com/storage/docs/creating-buckets#storage-create-bucket-gsutil).

**⚠️Warning: you get free REGIONAL storage in US regions only⚠️**.

The code we're going to run is:
`gsutil mb -p [PROJECT_ID] -c regional -l [REGION] -b on gs://[BUCKET_NAME]/`

-b means bucket policy only, which reading the documentation seems to mean that you can't modify individual objects' Access Control Lists, which as our Terraform backend will contain sensitive data seems to be what we're after - we can lock down the bucket.

You can now see the bucket has been created in the [console](https://console.cloud.google.com/storage/browser).

Now, I'm going to update my terraform configuration to set this bucket as the backend. I'm going to add these lines to my variables.tf file I set up last time.

```terraform
variable "backend_bucket_name" {
  type = string
  default = "MY_BUCKET_NAME"
}
```

I still haven't decided if I want to commit this file, as it contains project and bucket names. I will probably write a quick script that interactively sets it up for users later.

Let's add the backend to our main.tf file 

```terraform
terraform {
  backend "gcs" {
    bucket  = var.backend_bucket_name
    prefix  = "terraform/state"
    credentials = var.service_account_location

  }
}
```
*Note: the syntax highlighting doesn't work because they changed how expressions work in Terraform 0.12. It compiles correctly, I promise!*

So all we need to do is run `terraform init`. Or is it - oops, I get this error:

```
Initializing the backend...

Error: Variables not allowed

  on main.tf line 10, in terraform:
  10:     bucket  = var.backend_bucket_name

Variables may not be used here.
```

Aaagh. I want to keep the terraform files generalizable. Perhaps this needs to be another thing that I have a script automatically generate for people who want to deploy by themselves! I'm keeping my project id and bucket name secret and out of version control.

I added it to another file, backend.tf, where I added my configuration, which looks a little like this 

```terraform
terraform {
  backend "gcs" {
    bucket  = "MY_BUCKET"
    prefix  = "terraform/state"
    credentials = "~/.gcp/terraform-sa.json"
  }
}
```

Terraform init now gives me the promising 
```
Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "local" backend to the
  newly configured "gcs" backend. No existing state was found in the newly
  configured "gcs" backend. Do you want to copy this state to the new "gcs"
  backend? Enter "yes" to copy and "no" to start with an empty state.
```

I said yes, because why not, re-ran `terraform plan`, and voila

```
No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.
```

Perfect! All our state is now stored inside Google Cloud. I removed my local .tfstate and re-ran `terraform plan` once more to check. All good.

Checking on the Google Cloud Console:
![Our terraform state!](https://thepracticaldev.s3.amazonaws.com/i/ofvn3x160yvj19wt9w9s.png)

Awesome.

##### A note on why I started with Terraform
I know it's a little boring, but I'm starting as I mean to go on. I don't want to create any infrastructure I can't reproduce. Even with this blog, I don't want to have to go back and work out what I did to deploy X bit of software. It's also good practice to have infrastructure as code for everything, and allows your infrastructure to be version controlled and changed in an auditable way.

### What now?

Well, I could set up some more resources (maybe Cloud Firestore), but I don't really know what I need right now. Let's start writing some code and get things working locally, then work out what we'll need.

I've set up a list of my planned features on [GitHub](https://github.com/NickBlow/familiar), reproduced below for posterity. They will probably change.
- Real time chat
- Real time shared virtual tabletop (VTT)
- Pluggable rules engine
- In game news and campaign journal
- Character sheets with programmable macros
- Push notifications
- Customisable random generators. Automatic integration into other components TBD.
- Map building tools (TBD)
- Importing stat-blocks with computer vision (TBD)
- Developer API (TBD)

Some of these are obviously more aspirational than the others, but at a high level I have a good vision of what I want to build.

If I wanted to do things truly [lean](http://theleanstartup.com/principles), which I am a fan of, I'd focus on building the smallest possible unit of value, get it out to users ASAP, refine my ideas and iterate. I'm still going to do it somewhat lean, but I'm building a lot of this for myself, which means that *I'm* the user. So to some extent, if I think it's good, it *is* good. If other people like it, great.

Having a look at the features, there's a few things that jump out. The real time aspect is interesting from an engineering point of view, as I'm currently working on a Go library capable of handling GraphQL subscriptions over a HTTP/2 Server Sent Events transport. I'll probably open source it separately and then use it in this project.

The virtual tabletop is going to be an enormous undertaking, so we'll leave that for later. The same goes for some of the more speculative features (the ones I've marked as TBD).

The other thing that's interesting is the in game news and campaign journal. I see this as a pseudo-blogging platform, but with lasers and dragons. It also feels like the most complete 'product' by itself. There are a million chat apps, the rules engine and character sheets are cool but need the virtual tabletop to be complete.


### Some problems

First off, I'm going to use [LitElement](https://lit-element.polymer-project.org/). All major browsers support shadow DOM, it's fast, it builds off native web functionality (namely WebComponents). The other option I'd take seriously is React, but I'm betting on WebComponents in the long run. Unlike the [Polymer Library](https://polymer-library.polymer-project.org/), which often felt like there was a 'Polymer' way of doing things, LitElement feels much more interoperable with vanilla HTML5 and vanilla Javascript.

However, not all is roses in LitElement (although, the same can be said of React). A lot of this is to do with some buggy implementations of ShadowDOM in various browsers, standards that are not quite ready yet. It also doesn't work in JSDOM, which makes it a little painful to test. The other issue is a lack of server side rendering. There's a [library](https://github.com/popeindustries/lit-html-server) that does server side rendering for [LitHTML](https://github.com/Polymer/lit-html), which is the rendering library that underpins LitElement. 

This *can* cause issues with CDNing (assuming we can get it cheaply, free was one of the goals after all!), but stuff like rehydrating content is non-trivial, and we can cache all the javascript really well anyway. It's really fast too.

#### The worst issue

Obviously we need a text editor for our campaign journal feature.

The biggest issue is the [getSelectionAPI](https://developer.mozilla.org/en-US/docs/Web/API/DocumentOrShadowRoot/getSelection) in shadow DOM. It's poorly supported across browsers, and [doesn't work at all in Safari](https://bugs.webkit.org/show_bug.cgi?id=163921). There's a new api getComposedRange, which at time of writing is [still not finalized](http://pbtynpj.com/github_/w3c/webcomponents/issues/79). Obviously working out selections is a really important feature of text editors, so most of them don't work out of the box. There's a [polyfill](https://github.com/GoogleChromeLabs/shadow-selection-polyfill) that doesn't really work exactly the same way, and modern editors who use many features of the [Selection](https://developer.mozilla.org/en-US/docs/Web/API/Selection) object also fail, as it only polyfills part of the Selection object. 

I've seen this polyfill work in [QuillJS](https://quilljs.com/) as part of the [Vaadin Rich Text Editor](https://github.com/vaadin/vaadin-rich-text-editor) component, but the exact details of their implementation is minified and therefore obfuscated, so I don't know how complicated it will be to get it working. The other option is passing it is as a [slot](https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_templates_and_slots) into our WebComponents, which means that it's going to be rendered as the element's light DOM. It does however then need to be passed down all the way from the top level. We could also render it in an `iframe`, but that has its own set of issues.  

Additionally, we could mix-and-match technologies, and render the campaign builder pages with just HTML/CSS (potentially via LitHTML), or even React. This means we have to be very careful about our CSS, but we're going to control the entire app, we don't need to necessarily make a fully isolated text editor. To start with, we'll try the `slot` method, and if that doesn't work then we can revisit this decision.


### Let's write some code.

I've rambled enough. I'll create a 'hello world' Lit Element page, get it to render, then call it a day.

I've initialised an npm package, and run `npm install lit-element --save`. We don't need the web components polyfill as we're not supporting old Edge (if you're on Edge, grab the Chromium based [Edge Insider](https://www.microsoftedgeinsider.com/en-us/)). 

We're going to try [ParcelJS](https://parceljs.org), but we might have to switch to webpack later. I created a 'frontend:dev' script `parcel frontend/index.html` in the `package.json`, and I created index.html in the frontend folder for now (I might have to change that when I decide how to serve it). Eventually I might have multiple `package.json` files in the project and maybe manage it with [LernaJS](https://github.com/lerna/lerna), but let's see how unwieldy it gets.

The content of index.html
```html
<html>
<body>
  <script src="./js/app.js"></script>
</body>
</html>
```

And in app.js for now
```javascript
console.log('hello world')
```

I'm just going to quickly check parcel works by running `npm run frontend:dev`

![Console log](https://thepracticaldev.s3.amazonaws.com/i/6kgknozfs4snyimp6u6d.png)

Awesome. That's easy though. Can it run our WebComponent? I'm just going to grab the example from the [LitElement](https://lit-element.polymer-project.org/) home page. It has fancy things like Javascript Decorators and tagged template literals. We'll switch it to TypeScript later.

```javascript
import { LitElement, html, property, customElement } from 'lit-element';

@customElement('simple-greeting')
export class SimpleGreeting extends LitElement {
  @property() name = 'World';

  render() {
    return html`<p>Hello, ${this.name}!</p>`;
  }
}
```
This example has a few things going on. It's using LitElement as a base component, which at some point extends the base HTMLElement, making it a WebComponent. It's using decorators to define a property (something you can get and set on the element), and to define the element tag name (`simple-greeting`). The render function used a tagged template literal, so the string template will be passed to the LitElement `html` function. The render function itself handles doing the minimal changes to the DOM, and updating when a property (in this example `name`) changes.


Copy pasting it into VSCode is already giving us an error, saying we need to enable it in our jsconfig or tsconfig. Let's ignore that for one second and see if parcel handles it.

We need to update index.html to include our custom tag to test it.

```html
<html>
<body>
    <simple-greeting><simple-greeting/>
  <script src="./js/app.js"></script>
</body>
</html>
```

Running frontend:build again doesn't work:
```

Server running at http://localhost:1234 
🚨  /Users/nickblow/Personal/familiar/frontend/js/app.js:3:0: Support for the experimental syntax 'decorators-legacy' isn't currently enabled (3:1):
```

Oh dear. Let's add a [tsconfig](https://www.typescriptlang.org/docs/handbook/tsconfig-json.html), and see if that fixes it. Worst case we'll move to WebPack.

###🕒🕒🕒30 minutes later🕒🕒🕒

Aaaand I'm reminded why I hate the Javascript ecosystem again. I ended up changing so much configuration I may as well have been using WebPack, so that'll be the next task. I ran into [this](https://github.com/parcel-bundler/parcel/issues/839) issue, and the only fix was to add `last 1 Chrome versions` to my `browserslistrc` (for [browserslist](https://www.npmjs.com/package/browserslist)). That's awful. I will fix this with WebPack properly later. We want to keep ES6, but we also want to support a few more browsers. [Open WC](https://github.com/open-wc/open-wc/) has a whole bunch of good advice, examples and defaults which we'll use.

In addition I added (a lot of this was trial and error and I need to clean it up): 

tsconfig.json
```json
{
    "compilerOptions": {
        "module": "es6",
        "noImplicitAny": true,
        "removeComments": true,
        "preserveConstEnums": true,
        "allowJs": true,
        "allowSyntheticDefaultImports": true,
        "sourceMap": true,
        "moduleResolution": "node",
        "target": "es2015",
        "experimentalDecorators": true,
    },
    "include": [
        "frontend/**/*"
    ],
    "exclude": [
        "node_modules",
        "**/*.spec.ts"
    ]
}
```
I tried installing various babel plugins and presets, but we'll use those later when it comes to WebPack. [This issue](https://github.com/Polymer/lit-element/issues/54) also had some useful info.

On the plus side: 
![Our first web component running](https://thepracticaldev.s3.amazonaws.com/i/wrn93565vcpw90owd116.png)

We have a live webcomponent running.

### Next time
I'll configure WebPack, and hopefully we can get some real JS and CSS going!