Rust and Wasm Side-by-Side
Tags: webassembly, rust, software, compiling
Published: 2019-12-22 20:13:24

---
title: Rust and Wasm Side-by-Side
published: true
date: 2018-10-26 15:42:57 UTC
tags: wasm,rust,software,compiling
canonical_url: http://www.jwir3.com/rust-and-wasm-side-by-side/
---

## Background
 ![Rust and Wasm Side-by-Side](http://www.jwir3.com/content/images/2018/10/crab-298346_1280.jpg)

At my work, we're considering using [WebAssembly](https://webassembly.org/) (hereafter abbreviated as WASM) because it allows us to cross-compile just about any language for use on the web. WebAssembly is a "binary instruction format for a stack-based virtual machine". Basically, this means it's a binary language designed to be run anywhere, but generally speaking, it's used right now within web browsers as a replacement for Javascript modules. One of the languages we are considering using as a source language is [Rust](https://doc.rust-lang.org/book/2018-edition/index.html), a new(ish) language designed for type safety, performance, and concurrency.

As such, I've been reading up the Rust programming language, and, more importantly, how to [compile Rust to WebAssembly](https://rustwasm.github.io/book/). After learning from the example module given in the RustWasm book, I decided to convert one of my personal npm libraries from JS -\> Rust <sup><a href="#fn1" id="fnref1">[1]</a></sup>. The first step in this endeavor was to convert over the existing tests that I had in mocha/chai to Rust's native test format, so that as I added code, I could verify that it was working as I expected it to work <sup><a href="#fn2" id="fnref2">[2]</a></sup>. However, the ability to run these tests became an issue when I moved to a WASM target.

## The Problem

One of the big downsides to compiling Rust to WASM is that there isn't a good way to debug code within the browser, and have it link back to the Rust source. The Rust-WASM book notes this in [the section on debugging](https://rustwasm.github.io/book/game-of-life/debugging.html#using-a-debugger):

> Unfortunately, the debugging story for WebAssembly is still immature. On most Unix systems, DWARF is used to encode the information that a debugger needs to provide source-level inspection of a running program. There is an alternative format that encodes similar information on Windows. Currently, there is no equivalent for WebAssembly.

Instead, they recommend using testing (specifically, automated testing) to identify regressions _before_ they make it into the build. This is a great idea, and one I obviously support, given my original writing of tests prior to implementing my library. However, there's one big downside to writing tests in Rust when compiling to WASM, as detailed in the [next subsection of the Rust-WASM book](https://rustwasm.github.io/book/game-of-life/debugging.html#avoid-the-need-to-debug-webassembly-in-the-first-place):

> Note that in order to run the #[test]s without compiler and linker errors, you will need to comment out the crate-type = "cdylib" bits in wasm-game-of-life/Cargo.toml.

Wait... what? In order to run my Rust tests, I need to _make changes to my source repository just to get it to build_? This is a bit of a non-starter for me, because I want to run on a continuous integration system, where I'm able to build the Rust package, run the tests, build the WASM binary, and then deploy to NPM, all _automatically_.

In other words, what I want to be able to do is something like this:

```
# These commands compile (and subsequently test) the native Rust code
$ cargo build
$ cargo test

# This command builds the wasm module
$ cargo build --target=wasm32-unknown-unknown
```

The key here, though, is that I want to be able to do this _without_ making any source file or build-file changes.

## Possible Solutions<sup><a href="#fn3" id="fnref3">[3]</a></sup>

I spent quite a bit of time trying to determine what I _could_ do here. One thought that came to mind was that I could have a pre-build script that runs that performs this commenting-out of `#[wasm_bindgen]` attributes manually. How this would work is, before building, the script would make copies of everything in the local directory to a sub-directory. It would then switch the `crate-type` in `subdirectory/Cargo.toml` to `lib`, instead of `cdylib` and comment out any instances of `#[wasm_bindgen]`. Needless to say, I didn't want to go this route - it seemed incredibly fragile and error-prone.

Next, a coworker of mine suggested I separate the native Rust code from the WASM aspect of the code, and make the WASM library utilize the code from the native library, similar to how [geotoy](https://www.github.com/fitzgen/geotoy) handles this<sup><a href="#fn4" id="fnref4">[4]</a></sup>. This is a great solution, but, when I set up my library, I couldn't for the life of me figure out how to import an enum or struct into the WASM portion of the library.

Basically, geotoy has a setup where it uses the WASM library to expose a set of _functions_ that serve as the API for the library. These functions then utilize the data structures in `src/lib.rs`. However, the data structures _themselves_ aren't exposed. With `cratchit`, what I want to expose is the `Account` and `AccountsChart` data structures, as well as the `Currency` and `AccountType` enumerations _directly_, instead of a set of gateway functions that utilize these data structures <sup><a href="#fn5" id="fnref5">[5]</a></sup>. I suspect there probably _is_ a way to get it to expose the structs, but I wasn't able to figure it out.

## Final Solution

The solution I eventually arrived at doesn't separate the libraries per se. Instead, it makes the `#[wasm_bindgen]` attributes conditional on whether or not you're building for a WASM target.

The first thing you need to do is make sure you're using _both_ `cdylib` _and_ `rlib` (or `lib`) in your `Cargo.toml`:

```
[lib]
crate-type = ["cdylib", "rlib"]
```

Next, (this might be specific to me), I had to add `#![feature(custom_attribute)]` to the top of my crate attributes (i.e. at the top of `src/lib.rs`:

```
#![feature(custom_attribute)]

extern crate cfg_if;
extern crate json;
extern crate wasm_bindgen;

use wasm_bindgen::prelude::*;
use cfg_if::cfg_if;
use std::collections::HashMap;

...
```

Next, wherever you previously used `#[wasm_bindgen]`, make it conditional on the target architecture:

```
/// Use this instead of #[wasm_bindgen]
#[cfg_attr(target_arch = "wasm32", wasm_bindgen)]
```

Finally, and this might, again, be something that is specific to my use case, you need to pull types out of submodules and into the root crate `src/lib.rs`. In other words, in my `src/lib.rs`, I had the following:

```
pub mod accounts;
pub mod currency;
```

And I had two files, `src/currency.rs` and `src/accounts.rs`, which defined types related to accounts and currencies, respectively. `wasm_bindgen` didn't appear to like this, so I moved this code into `src/lib.rs` and removed references to the modules in the tests.

What I did to test was not use `wasm-pack` initially. Instead, I ran `cargo +nightly build --target=wasm32-unknown-unknown` to ensure that no errors were present, and that it created a .wasm file in the appropriate `target` directory:

```
$ cargo +nightly build --target=wasm32-unknown-unknown
...
$ ls -al target/wasm32-unknown-unknown/debug
total 9520
drwxr-xr-x@ 13 scottj staff 416 Oct 26 10:15 .
drwxr-xr-x 3 scottj staff 96 Oct 26 10:14 ..
-rw-r--r-- 1 scottj staff 0 Oct 26 10:14 .cargo-lock
drwxr-xr-x 8 scottj staff 256 Oct 26 10:14 .fingerprint
drwxr-xr-x 3 scottj staff 96 Oct 26 10:14 build
-rw-r--r-- 1 scottj staff 122 Oct 26 10:15 cratchit.d
-rwxr-xr-x 2 scottj staff 3549051 Oct 26 10:15 cratchit.wasm
drwxr-xr-x 13 scottj staff 416 Oct 26 10:15 deps
drwxr-xr-x 2 scottj staff 64 Oct 26 10:14 examples
drwxr-xr-x 3 scottj staff 96 Oct 26 10:15 incremental
-rw-r--r-- 1 scottj staff 125 Oct 26 10:15 libcratchit.d
-rw-r--r-- 2 scottj staff 1311774 Oct 26 10:15 libcratchit.rlib
drwxr-xr-x 2 scottj staff 64 Oct 26 10:14 native
```

And, of course, I made sure the tests ran without any changes:

```
$ cargo test
running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/debug/deps/test_accounts-54475044840749bd

running 6 tests
test account_type_from_integer ... ok
test account_creation ... ok
test account_type_from_string ... ok
test adding_top_level_accounts_to_accounts_chart ... ok
test getting_all_account_ids_in_a_chart ... ok
test creating_accounts_chart_from_json ... ok

test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/debug/deps/test_currency-d2d59a6d3436c103

running 1 test
test currency_translation_from_string ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

   Doc-tests cratchit

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
```

Now, when you want to build your WASM module, you can run:

```
wasm-pack build
```

And, it will create the packaged WASM library for you in `pkg`.

## Bonus: Run tests on Travis and deploy WASM to NPM

This was another area that took a bit of figuring out for me. The problem I ran into with Travis-CI was that it doesn't install `wasm-pack` by default. As such, you need to install it manually in a script. However, if you have `cache: cargo` enabled, it will hang if `wasm-pack` was previously installed. Thus, I added the following to my `.travis.yml` file to conditionally install `wasm-pack`:

```
before_script: |
  if hash wasm-pack 2>/dev/null; then
      echo "Wasm-pack already is installed"
  else
      curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
  fi
```

Now, you can add the following to your `script` section build both the WASM module and the native Rust module:

```
script:
- cargo clean
- cargo build
- cargo test
- wasm-pack build --target=nodejs
```

Note the section that calls `wasm-pack build` has an additional argument: `--target=nodejs`. If you want to test locally using `npm link`, you will need this argument, since it packages the WASM module with a `main` parameter in the `package.json`.

And, finally, add the deployment logic<sup><a href="#fn6" id="fnref6">[6]</a></sup>:

```
before_deploy:
- cd pkg
deploy:
  provider: npm
  email: <your email address>
  on:
    tags: true
  skip_cleanup: true
  api_key:
    secure: <YOUR_API_KEY>
```

Note that this deployment logic will only deploy on tagged releases, so you may want to change that if you want different behavior.

One last thing to realize: this currently requires _nightly_ rust. So, you'll need to make sure you're using nightly Rust locally, and you need to make sure that, if you're building on travis, that you allow `stable` and `beta` Rust to fail:

```
rust:
- stable
- beta
- nightly
matrix:
  allow_failures:
    - rust: stable
    - rust: beta
```

* * *
<section>
<ol>
<li id="fn1">
<p>The library in question is <a href="http://www.github.com/jwir3/cratchit">Cratchit</a>, which is a library designed to encapsulate accounting data for an accounting application running in electron. It's not important that you have familiarity with this library, but if you want to follow along, feel free. <a href="#fnref1">↩︎</a></p>
</li>
<li id="fn2">
<p>I'm going to gloss over the rest of the adventures regarding me porting to Rust, since it's not super interesting, and involves me fighting a lot with the borrow checker to get things to work. <a href="#fnref2">↩︎</a></p>
</li>
<li id="fn3">
<p>If you're not interested in my musings about the problem and <em>how</em> I came to the solution, feel free to skip to the section on the solution I used, later on in this post. <a href="#fnref3">↩︎</a></p>
</li>
<li id="fn4">
<p>The coworker in question is <a href="https://github.com/johshoff">Johannes Hoff</a> who also is a contributer on the geotoy repo. Interestingly enough, the original author of geotoy, <a href="http://www.github.com/fitzgen">Nick Fitzgerald</a>, was an intern at Mozilla when I first started there. It's amazing to me sometimes how small this world is, given that I haven't seen fitzgen in years, and one of my coworkers happens to have a solution to a problem I have, in collaboration with this <em>other</em> person I worked closely with. Sometimes the wheel of fate astounds me. <a href="#fnref4">↩︎</a></p>
</li>
<li id="fn5">
<p>There probably is a good reason for exposing gateway functions, rather than the underlying data structures themselves, which I will discover at some future point, but for now, my chosen solution seems to be a little cleaner. <a href="#fnref5">↩︎</a></p>
</li>
<li id="fn6">
<p>Make sure that you encrypt your api key. You can do this by retrieving an api key from travis-ci.org and then running: <code>travis encrypt &lt;api_key&gt; --add deploy.api_key</code>. I waited until I had the configuration otherwise the way I wanted it before doing this. <a href="#fnref6">↩︎</a></p>
</li>
</ol>
</section>