Loading WASM as ESM in NodeJS
Tags: webassembly, node, javascript
Published: 2019-06-21 13:17:52

# What is a module?
The [module](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules) in JavaScript is a logical namespace within which we will define the functions or/and values. Then we can export these functions or/and values and import them into some other namespaces.

In the NodeJS world, we have CommonJS modules.

# What is CommonJS?
[CommonJS modules](https://nodejs.org/docs/latest/api/modules.html) were created for server and Desktop. With CommonJS the import and export syntax looks something like this:

```js
// importing from a node_modules
const lodash = require('lodash');
// importing from another module
const localFunction = require('./some-module').someFunction;

// Exporting 
module.exports = {
    exportValue: someValue,
    exportFunction: someFunction
}
```

In servers, (mostly) all the necessary JavaScript files are located in the filesystem. This means they can be loaded synchronously. So the CommonJS module system is

* compact
* allows synchronous loading
* built for the servers

But CommonJS does not have _live bindings_. CommonJS modules have dynamic structure, this makes it extremely difficult to static checking, optimizing, eliminating dead code with the bundlers.

The bundlers or loaders ecosystem does some intelligent hacks to make this happen.

Also in CommonJS, it is very hard to identify and fix cyclic module dependencies. [Sometimes it might lead to infinite loops](https://nodejs.org/api/modules.html#modules_cycles).

# ES Modules

On the other side, the web has a highly **undefined** network. That introduces latency which makes it hard to work. But still, the web is the most awesome thing happened.

There were multiple attempts made to make the module system for the web. But finally, ES2015 gave us [ESModules](https://www.ecma-international.org/ecma-262/6.0/#sec-modules) (or ECMAScript Modules).

```js
// Importing a node modules
import lodash from 'lodash';
// Importing a function from another ESModule
import {someFunction} from './some-module';

// Exporting values or functions
export const exportValue = someValue;
export function exportFunction = someFunction;
```

The ESModules are built for the web. That is they have the support for **asynchronous** loading. 

It is also important to have clear, concise and compact statements that are easy to understand both for people writing it and for the loaders or bundlers.

ESModules are 
* more compact 
* loads asynchronously
* built for the web
* cyclic module dependencies are managed efficiently
* static structure makes it easy to check, optimize, eliminate dead code

The ECMAScript modules are slowly stabilizing itself in the NodeJS ecosystem. It definitely took a while but it is all for good. We planned and deliver ESModules in NodeJS. Check out more details [here](https://github.com/nodejs/modules/blob/master/doc/plan-for-new-modules-implementation.md).

Currently, JavaScript that we write for Node and browser environment is different. This makes it difficult for library authors, developers and others. Making JavaScript isomorphic between Node and Browser will be awesome. It will reduce a lot of boilerplate code.

Bringing ESModules to NodeJS makes us bridge the gap between Node and browser.

Narrower the bridge is better the ecosystem will be.

# ESModules comes to Node.js

In browsers, we differentiate the ESModules in the script tag using `type="module"`. Likewise in the NodeJS world, we will differentiate the ESModules using `.mjs` extension.

We can import the `.mjs` files using the `import` syntax.

```js
// add.mjs
export let methodCalled = 0;

export function add(x, y) {
    methodCalled++;
    return x+y;
}
```

```js
// main.mjs
import assert from 'assert';
import {methodCalled, add} from './add.mjs';

assert(methodCalled, 0); 
assert(add(13, 31), 44);
assert(methodCalled, 1);
```

We can compile and run the above code using `node --experimental-modules main.mjs`.

The `experimental-modules` flag specifies the Node to load the `main.mjs` file as an ESModule.

### No default resolution

Currently, the modules implementation does not resolve to `index` file or add extensions `.mjs`. That is 
```js
import {add} from './add'; // is not a valid ESM import
                          // nor it looks for index file in add folder

```

### No mixing syntaxes in .mjs files

With the current implementation, you cannot mix and match the syntax. That is the `.mjs` files should only use `import` statements to import.

```js
const assert = require('assert');
             ^
ReferenceError: require is not defined
```

### Loading js files inside mjs files

The ESModules JavaScript file(`.mjs`) can import the CommonJS file(`.js`). 

To import the `.js` we have to use the [createRequire](https://nodejs.org/api/modules.html#modules_module_createrequire_filename) function.

```js
// add.js

let methodCalled = 0;

function add(x, y) {
    methodCalled++;
    return x+y;
}

module.exports = {
     methodCalled,
     add
}

```

Inside the `.mjs` file let us import the `add.js` file.

```js
//main.mjs

import { createRequire } from 'module';      // ---1
const require = createRequire(import.meta.url); // ---2

const { add } = require('./add.js'); // ---3

console.log(add(13, 10)); // 23

```
1. We import the `createRequire` from the `module`. [The `module` is available inside the NodeJS code](https://nodejs.org/api/modules.html#modules_the_module_object_1).

2. Then we define the `require`. The require uses `import.meta.url`. Check out more about `import.meta` [here](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import.meta)

3. Load the library using the `require` function.

Then we can use the add function, just like any other imported function. 

### Loading mjs files inside js files
It is not possible to do this.

# How does ESModules work?

There is an absolutely awesome blog from Lin Clark [here](https://hacks.mozilla.org/2018/03/es-modules-a-cartoon-deep-dive/).

There are three phases in the ESModules loading:
    1. Fetch and Parse
    2. Link
    3. Evaluate

#### Fetch & Parse

As the name indicates in this phase the JavaScript file mentioned is fetched from any URL given. The URL can be a remote location (usually in browsers) or an absolute File URL. Once fetched, the file is parsed.

During parsing the dependencies(or modules) are identified progressively. Then it fetches all the modules and parses them. The parsing ensures the JavaScript is having a valid syntax.

The phase ends with creating a `Module record`. Consider Module record as an instance that holds all the things that are defined inside the module. Things like import, export and others. 

#### Linking phase

During this phase, the links to export and import are mapped using the module record. The linking will just link the values to a location rather than to a value. This enables live bindings for the imports. 

So the values that are imported will always reflect the live value.

#### Evaluate

During this phase, 
* the module's lexical scope is initialized
* functions are hoisted
* function declarations are initialized
the JavaScript code is evaluated and the values are filled in the memory location.


# Enters the WebAssembly Modules

[WebAssembly](https://webassembly.org/) is the cool new kid in the block. It brings maintainable performance and native code to the browser. 

# ESM in WASM

Currently, the [ESModules integration for WebAssembly Modules](https://github.com/WebAssembly/esm-integration/tree/master/proposals/esm-integration) is in Stage1. 

Let us see the main difference between loading a WebAssembly Module as an ES Module over the JavaScript.

There are three phases in the ESModules loading (similar to JavaScript):

* Fetch and Parse
   - The binary format is parsed and validated.
* Link
   - No Function initialization happens here
* Evaluate
   - Initialize the modules
   - Run the start function
   - Function declarations are initialized

# Loading WASM as ESM in NodeJS

Let us first create a WebAssembly Module. The easiest and hackiest way to generate a WebAssembly Module is using the WebAssembly Text Format.

Create a file called `add.wat` with the following contents

```wast
(module
  (func $add (param $p1 i32) (param $p2 i32) (result i32)
    local.get $p1
    local.get $p2
    i32.add)
  (export "add" (func $add))
)
```

Convert the above WebAssembly Text Format into WebAssembly Module using [wabt](https://github.com/WebAssembly/wabt)

```bash
/path/to/wabt/build/wat2wasm add.wat -o add.wasm
```

It creates `add.wasm`

```wasm
00 61 73 6d 01 00 00 00 01 07 01 60 02 7f 7f 01
7f 03 02 01 00 07 07 01 03 61 64 64 00 00 0a 09
01 07 00 20 00 20 01 6a 0b
```

Now we can import the WebAssembly Module like an ESModule. Let us create a file called `index.mjs` with the following contents.

```js
import * as M from './add.wasm';

console.log(M.add(10, 13)); // 23
```

We can run the above code using two flags one for enabling the 
`experimental-modules` and `experimental-wasm-modules`.


```bash
node --experimental-modules --experimental-wasm-modules index.mjs
```

Check out the [commit](https://github.com/nodejs/node/commit/bbc254db5db672643aad89a436a4938412a5704e) where it landed in the Node core.

Reference links to explore further about `--experimental-modules`

[NodeJS announcement](https://medium.com/@nodejs/announcing-a-new-experimental-modules-1be8d2d6c2ff)

I hope this gives you a head start into ESModules. If you have any questions/suggestions/ feel that I missed something feel free to add a comment.

If you like this article, please leave a like or a comment.

You can follow me on [Twitter](https://twitter.com/sendilkumarn) and [LinkedIn](https://linkedin.com/in/sendilkumarn).

Thanks [@MylesBorins](https://twitter.com/MylesBorins) for the awesome review. :)