SPAC: Publishing Apps
Tags: javascript, framework
Published: 2020-12-28 07:33:15

_SPAC is a custom JavaScript framework for client-side, single-page web applications. It stands for "Stateful Pages, Actions and Components". Its design goal is to provide robust and simple entities that help you to structure apps. Pages and components provide the HTML, JavaScript functions and UI interactions. Actions govern external API calls. You define these entities in plain JavaScript, load up the central controller, and your app is ready to be served. Read the development journey of SPAC in my series: <https://admantium.com/category/spac-framework/>._

The past articles explained the components and features of SPAC. Now it’s time to go through the steps of publishing your app.

_This article originally appeared at [my blog](https://admantium.com/blog/spac08_publishing_apps/)_.

## Assumptions & Observations

At the beginning of the development journey, I envisioned the following:

> SPAC.js is intended to be a lightweight, self-initializing framework for publishing client-side applications. Its foundation is the controller, the central entity that resolves pages, components and actions. The controller is the only tool that you need to include in a JS. When started it, will serve the index page.

Over the last weeks, each entity was developed and supported by an extensive test suite. Then I assembled a very simple demo app with one page, components and action. And... the framework just did not work.

The troubles I ran into were interesting and rewarding to understand and solve. This blog post is about these learnings.

## Modules: (ES5 != Node)

The module syntax of node is not the same as that of ES5 or ES6 that runs in a browser. The expressions `require()` and `module.exports` are node specific. My code base was using Node exports, which were incompatible with ES5. I was unaware of this issue because the unit tests relied on Babel to transpile the code to a compatible version. So, to make it running in a web browser, I need to transpile to that environment. Which leads me to ...

## Imports & Bundling

A bundler takes a complex project, computes a dependency graph, and provides an optimized, transpiled for the target environment compatible version of your project.

[Webpack](https://webpack.js.org/concepts/) was the first bundler that came to my mind. I absorbed the documentation, provided a sample config, and could transpile my projects. Then I created a simple HTML page and manually included the transpiled version. This version could not be executed because the Node module export syntax could not be understood by native webpack. From a blog post I understood that a Babel config is only applied after Webpack bundled the code, so my original problem could not be solved.

After some more research, I discovered [Snowpack](https://www.snowpack.dev/#import-npm-packages) - and it could transpile my code base without any additional configuration. All SPAC entities were available in the browser. And then I executed `Controller.init()` which uses the Node module `fs` to recursively traverse files in a directory. For the time being, I tried to get the `fs` node module working with snowpack, following this documentation about [polyfilling NPM packages](https://www.snowpack.dev/#import-npm-packages), but could not get it working.

Stop for a moment.

Javascript running in a browser should not be allowed to traverse local files. This traversal is server-side, not client side!

This finding is particularly interesting. Initially, I considered these options:

- Dynamic Imports: ES5 supports a [dynamic import() statement](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import). This statement needs a filename, and this file is asynchronously fetched from the server. Therefore, the server-side actually needs to deliver individual JavaScript pages.
- Pre-Build Imports: Before bundling the app, I use a helper script that traverses the app directories, determines the pages, and then adds them to an inventory file. During bundling, the controller reads the inventory, and executes static imports of these file. Then, the application is bundled.
- Static Imports: All entities, including pages, need to statically import their required assets. Then, bundling "just" packs the application into the target format.

After some thought, and another try to create a self-initializing controller, the solution became a combination of all the above ideas:

- Inventory: Before building, a script detects all pages, and creates a file called `inventory.js`
- Imports: During the `init` phase, the controller loads all required pages from the inventory. These imports are dynamic at execution time, but...
- Bundling: ... the bundling determines and executes all imports before the code is assembled. Then, a bundled, optimized version of the app source code is produced.

## Changing how the Controller Works

Before the app is started, we `bootstrap` the inventory with the following function.

```js
export default function bootstrap (rootDir) {
  const inventory = { pages: [], components: [], actions: [] }
  Object.keys(inventory).forEach(entity => {
    const files = fs.readdirSync(path.join(rootDir, entity), {
      withFileTypes: true
    })
    const fullPath = path.join(path.resolve(rootDir), entity)
    files.forEach(file =>
      inventory[entity].push(path.join(fullPath, file.name))
    )
  })
  return inventory
}
```

This functions traverses, in the `rootDir`, the subdirectories `/pages`, `/components` and `/actions`, and collects a list of all contained files. The filenames will be stored with their full path to make bundling easier.

Then, the `controller.init()` uses this inventory to create the internal map objects.

```js

init() {
  this._initMap(Page, 'pages', /Page.js/)
  this._initMap(Action, 'actions', /Action.js/)
  this._initMap(Component, 'components', /Component.js/)
}

_initMap (parentClass, mapType, pattern) {
  this.inventory[mapType].forEach(async filePath => {
    try {
      if (!filePath.match(pattern)) {
        throw new Error()
      }
      const name = filePath
        .split('/')
        .pop()
        .replace(pattern, '')
      const clazz = (await import(`${filePath}`)).default

      if (clazz.prototype instanceof parentClass) {
        if (parentClass === Page) {
          const route = `/${name.replace(/([a-zA-Z])(?=[A-Z])/g, '$1-').toLowerCase()}`
          this[mapType].set(name, { route, clazz })
        } else {
          this[mapType].set(name, { clazz })
        }
      } else {
        throw new Error()
      }
    } catch (e) {
      console.error(e)
      throw new (class EntityLoadError extends Error {
        message = `Entity ${parentClass.name} from path ${filePath} could not be loaded`
      })()
    }
  })
}
```

This method traverses each directory, and checks each file. If the file does not have a name that ends with its type, or if the export does not contain a class of the requested, it is not imported, but an error is thrown. If all checks are passed, class object is dynamically loaded and added to the corresponding map.

## Building & Bundling Commands

With these changed, SPAC apps can be built. For convenience, the necessary steps are exposed as npm commands.

Building the app consists for two steps. First, the `bootstrap` command creates the inventory files. This step needs to happen on the machine at which SPAC is installed, because it requires some core NodeJS libraries that cannot be imported or transpiled to the browser. Second, the `build` command will initiate bundling the complete application code. You need to transfer the bundled file to a web server, or for local development you can use the `dev` command which starts a snowpack build-in server.

## Snowpack Config File

Snowpack is a bundler that comes with a robust default configuration. You can change several aspects with a `snowpack.config.js` file. In order to control the final layout, and details of the bundling process, I use the following config file.

```js
module.exports = {
  mount: {
    public: '/',
    src: '/src'
  },
  devOptions: {
    bundle: true,
    clean: true
  },
  installOptions: {
    treeshake: true
  },
  buildOptions: {
    out: 'build',
    clean: true,
    metaDir: '/core',
    webModulesUrl: '/lib'
  }
}
```

The config file is separated into four sections with the following meaning.

- `mount`: Configure additional folders to be served in your build, where `src` is the absolute path in your project, and `public` the folder to which these files will be copied
- `devOptions`: Control how the `dev` command works, here I add options to `clean` the cache and to use the `bundled` version of the code. This option is important to save you valuable time when your builds are not working - figure out the errors rather earlier.
- `installOptions`: During the bundling step, I use `treeshake` to eliminate redundant and dead code in the application and libraries
- `buildOptions`: The bundled source code is copied to `out`, but before new files are copied, everything is deleted with the `clean` option. Then, all additional libraries are installed at the `webModulesUrl` folder, and the `metaDir` defines where snowpack modules will be installed.

When using all of the above options, the build directory has the following structure:

```bash
build
├── core
│   └── env.js
├── img
│   └── favicon.ico
├── index.html
├── lib
│   ├── import-map.json
│   └── spac.js
├── src
│   ├── actions
│   │   ├── SearchApiAction.js
│   │   ├── ...
│   ├── components
│   │   ├── ApiSearchBarComponent.js
│   │   ├── ...
│   ├── globals
│   │   └── icons.js
│   ├── index.js
│   ├── inventory.json
│   ├── inventory.json.proxy.js
│   └── pages
│       ├── IndexPage.js
│       ├── ...
├── ...
└── style.css
```

Snowpack takes care to minify the bundled source code, but does not obfuscate the code - for this, you need to configures the `@snowpack/webpack-plugin` and provide a custom webpack configuration.

## The Final Frontier: Caching Dynamic Imports

During the development, I figured out that bundling does not pre-load the dynamic imports. Instead, they are issued at runtime, against the webserver providing your application. This does not impair functionality, but results in several more HTTP request from the users’ browser back to the server. To prevent this, my current idea is to modify the bootstrap command with a preload option that will store the file content of pages, components and actions. Then, at runtime, these strings would be evaluated. How? JavaScript supports running strings as code with `eval()`, but this has severe security implications as detailed in [mdn documentation](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval#Never_use_eval!). There are alternatives to consider. But for the time being, this is an ongoing development.

## Conclusion

This article covered an interesting development phase of SPAC: Figuring out how to serve the application from a web server to the browser. I discovered the important syntactical differences of NodejS modules, present on the machine on which you develop your application, and commonJS modules that run in your browser. Also, dynamic imports of files differ: On your machine, the file system is accessed, inside the browser, HTTP requests are made. This difference is also solved through the bundler. I continued to explain convenient commands that will bootstrap, build and deploy the application code, and finally detailed the bundlers configuration options.