Rust and WebAssembly Serverless functions in Vercel
Tags: rust, webassembly, serverless, jamstack
Published: 2021-08-23 15:07:12



[Vercel](https://vercel.com/) is a leading platform for developing and hosting [Jamstack](https://jamstack.org/) applications. Unlike traditional web apps, where the UI is dynamically generated at runtime from a server, a Jamstack application consists of a static UI (in HTML and JavaScript) and a set of serverless functions to support dynamic UI elements via JavaScript.

There are many benefits to the Jamstack approach. But perhaps one of the most significant benefits is performance. Since the UI is no longer generated at runtime from a central server, there is much less load on the server and we can now deploy the UI via edge networks such as CDNs.

However, the edge CDN only solves the problem of distributing the static UI files. The backend serverless functions could still be slow. In fact, popular serverless platforms have well-known performance issues, such as slow cold start, especially for interactive applications. That's where WebAssembly could help.

With [WasmEdge](https://github.com/WasmEdge/WasmEdge), a cloud-native WebAssembly runtime [hosted by the CNCF](https://www.secondstate.io/articles/wasmedge-joins-cncf/), developers can write high-performance serverless functions deployed on the public cloud or on edge computing nodes. In this article, we will explore how to use WasmEdge functions, written in Rust, to power a Vercel application backend.



## Why WebAssembly in Vercel Serverless?

The Vercel platform already has a very easy-to-use [serverless framework](https://vercel.com/docs/runtimes) for deploying functions hosted in Vercel. As we discussed above, the reason to use WebAssembly, and WasmEdge, is to **further improve performance**. High-performance functions written in C/C++, Rust, and Swift can be easily compiled into WebAssembly. Those WebAssembly functions are much faster than JavaScript or Python commonly used in serverless functions. 

However, if raw performance is the only goal, why not just compile those functions to machine native executables (Native Client or NaCl)? Vercel already runs these functions safely in application containers like Docker or Firecracker.


> Our vision for the future is to run WebAssembly as an alternative lightweight  runtime side-by-side with Docker and other containers in cloud native infrastructure. WebAssembly offers much higher performance and consumes much less resources than Docker-like containers or Firecracker-like microVMs. But for now, Vercel only supports running WebAssembly inside other containers.


Running WebAssembly functions inside Docker-like containers offer advantages over running NaCl programs directly in Docker. 

For starters, WebAssembly provides fine-grained runtime **isolation for individual functions**. A microservice could have multiple functions and support services running inside a Docker-like container. WebAssembly can make the microservice more secure and more stable. 

Second, the WebAssembly bytecode is **portable**. Developers only need to build it once and do not need to worry about changes or updates to the underlying Vercel serverless container (OS and hardware). It also allows developers to reuse the same WebAssembly functions in other cloud environments.

Third, WebAssembly apps are easy to deploy and manage. They have much less platform dependencies and complexities compared with NaCl dynamic libraries and executables. 

Finally, the [WasmEdge Tensorflow API](https://www.secondstate.io/articles/wasi-tensorflow/) provides the **most ergonomic way** to execute Tensorflow models in the Rust programming language. WasmEdge installs the correct combination of Tensorflow dependency libraries, and provides a unified API for developers.

Enough with the concepts and explanations. Without further ado, let's jump into the example apps!


## Prerequisite 

Since our demo WebAssembly functions are written in Rust, you will need a [Rust compiler](https://www.rust-lang.org/tools/install). Make sure that you install the `wasm32-wasi` compiler target as follows, in order to generate WebAssembly bytecode.


```
$ rustup target add wasm32-wasi
```


The demo application front end is written in [Next.js](https://nextjs.org/), and deployed on Vercel. We will assume that you already have the basic knowledge of how to work with Vercel.


## Example 1: Image processing

Our first demo application allows users to upload an image and then invoke a serverless function to turn it into black and white. A [live demo](https://vercel-wasm-runtime.vercel.app/) deployed on Vercel is available.

![vercel-webassembly](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fupa9o4kmriyx1zeg5z8.gif)

Fork the [demo application’s GitHub repo](https://github.com/second-state/vercel-wasm-runtime) to get started. To deploy the application on Vercel, just [import the Github repo](https://vercel.com/docs/git#deploying-a-git-repository) from [Vercel for Github](https://vercel.com/docs/git/vercel-for-github) web page.

This repo is a standard Next.js application for the Vercel platform. The backend serverless function is in the [`api/functions/image_grayscale`](https://github.com/second-state/vercel-wasm-runtime/tree/main/api/functions/image-grayscale) folder. The [`src/main.rs`](https://github.com/second-state/vercel-wasm-runtime/blob/main/api/functions/image-grayscale/src/main.rs) file contains the Rust program’s source code. The Rust program reads image data from the `STDIN`, and then outputs the black-white image to the `STDOUT`.


```
use hex;
use std::io::{self, Read};
use image::{ImageOutputFormat, ImageFormat};

fn main() {
  let mut buf = Vec::new();
  io::stdin().read_to_end(&mut buf).unwrap();

  let image_format_detected: ImageFormat = image::guess_format(&buf).unwrap();
  let img = image::load_from_memory(&buf).unwrap();
  let filtered = img.grayscale();
  let mut buf = vec![];
  match image_format_detected {
    ImageFormat::Gif => {
        filtered.write_to(&mut buf, ImageOutputFormat::Gif).unwrap();
    },
    _ => {
        filtered.write_to(&mut buf, ImageOutputFormat::Png).unwrap();
    },
  };
  io::stdout().write_all(&buf).unwrap();
  io::stdout().flush().unwrap();
}
```


You can use Rust’s `cargo` tool to build the Rust program into WebAssembly bytecode or native code. 


```
$ cd api/functions/image-grayscale/
$ cargo build --release --target wasm32-wasi 
```


Copy the build artifacts to the `api` folder. 


```
$ cp target/wasm32-wasi/release/grayscale.wasm ../../
```



> Vercel runs [`api/pre.sh`](https://github.com/second-state/vercel-wasm-runtime/blob/main/api/pre.sh) upon setting up the serverless environment. It installs the WasmEdge runtime, and then compiles each WebAssembly bytecode program into a native `so` library for faster execution. 


The [`api/hello.js`](https://github.com/second-state/vercel-wasm-runtime/blob/main/api/hello.js) file conforms Vercel serverless specification. It loads the WasmEdge runtime, starts the compiled WebAssembly program in WasmEdge, and passes the uploaded image data via `STDIN`. Notice [`api/hello.js`](https://github.com/second-state/vercel-wasm-runtime/blob/main/api/hello.js) runs the compiled `grayscale.so` file generated by [`api/pre.sh`](https://github.com/second-state/vercel-wasm-runtime/blob/main/api/pre.sh) for better performance.


```
const fs = require('fs');
const { spawn } = require('child_process');
const path = require('path');

module.exports = (req, res) => {
  const wasmedge = spawn(
      path.join(__dirname, 'wasmedge'), 
      [path.join(__dirname, 'grayscale.so')]);

  let d = [];
  wasmedge.stdout.on('data', (data) => {
    d.push(data);
  });

  wasmedge.on('close', (code) => {
    let buf = Buffer.concat(d);

    res.setHeader('Content-Type', req.headers['image-type']);
    res.send(buf);
  });

  wasmedge.stdin.write(req.body);
  wasmedge.stdin.end('');
}
```


That's it. [Deploy the repo to Vercel](https://vercel.com/docs/git#deploying-a-git-repository) and you now have a Vercel Jamstack app with a high-performance Rust and WebAssembly based serverless backend. 


## Example 2: AI inference

The [second demo](https://vercel-wasm-runtime-cozpr5z84-wangshishuo1.vercel.app/) application allows users to upload an image and then invoke a serverless function to classify the main subject on the image.

![Vercel WebAssembly](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/21gee7zfif3j7cxtvcme.gif)

It is in [the same GitHub repo](https://github.com/second-state/vercel-wasm-runtime) as the previous example but in the `tensorflow` branch. Note: when you [import this GitHub repo](https://vercel.com/docs/git#deploying-a-git-repository) on the Vercel website, it will create a [preview URL](https://vercel.com/docs/platform/deployments#preview) for each branch. The `tensorflow` branch would have its own deployment URL.

The backend serverless function for image classification is in the [`api/functions/image-classification`](https://github.com/second-state/vercel-wasm-runtime/tree/tensorflow/api/functions/image-classification) folder in the `tensorflow` branch.  The [`src/main.rs`](https://github.com/second-state/vercel-wasm-runtime/blob/tensorflow/api/functions/image-classification/src/main.rs) file contains the Rust program’s source code. The Rust program reads image data from the `STDIN`, and then outputs the text output to the `STDOUT`. It utilizes the WasmEdge Tensorflow API to run the AI inference. 


```
pub fn main() {
    // Step 1: Load the TFLite model
    let model_data: &[u8] = include_bytes!("models/mobilenet_v1_1.0_224/mobilenet_v1_1.0_224_quant.tflite");
    let labels = include_str!("models/mobilenet_v1_1.0_224/labels_mobilenet_quant_v1_224.txt");

    // Step 2: Read image from STDIN
    let mut buf = Vec::new();
    io::stdin().read_to_end(&mut buf).unwrap();

    // Step 3: Resize the input image for the tensorflow model
    let flat_img = wasmedge_tensorflow_interface::load_jpg_image_to_rgb8(&buf, 224, 224);

    // Step 4: AI inference
    let mut session = wasmedge_tensorflow_interface::Session::new(&model_data, wasmedge_tensorflow_interface::ModelType::TensorFlowLite);
    session.add_input("input", &flat_img, &[1, 224, 224, 3])
           .run();
    let res_vec: Vec<u8> = session.get_output("MobilenetV1/Predictions/Reshape_1");

    // Step 5: Find the food label that responds to the highest probability in res_vec
    // ... ...
    let mut label_lines = labels.lines();
    for _i in 0..max_index {
      label_lines.next();
    }

    // Step 6: Generate the output text
    let class_name = label_lines.next().unwrap().to_string();
    if max_value > 50 {
      println!("It {} a <a href='https://www.google.com/search?q={}'>{}</a> in the picture", confidence.to_string(), class_name, class_name);
    } else {
      println!("It does not appears to be any food item in the picture.");
    }
}
```


You can use the `cargo` tool to build the Rust program into WebAssembly bytecode or native code. 


```
$ cd api/functions/image-classification/
$ cargo build --release --target wasm32-wasi
```


Copy the build artifacts to the `api` folder. 


```
$ cp target/wasm32-wasi/release/classify.wasm ../../
```


Again, the [`api/pre.sh`](https://github.com/second-state/vercel-wasm-runtime/blob/tensorflow/api/pre.sh) script installs WasmEdge runtime and its Tensorflow dependencies in this application. It also compiles the `classify.wasm` bytecode program to the `classify.so` native shared library at the time of deployment.

The [`api/hello.js`](https://github.com/second-state/vercel-wasm-runtime/blob/tensorflow/api/hello.js) file conforms Vercel serverless specification. It loads the WasmEdge runtime, starts the compiled WebAssembly program in WasmEdge, and passes the uploaded image data via `STDIN`. Notice [`api/hello.js`](https://github.com/second-state/vercel-wasm-runtime/blob/tensorflow/api/hello.js) runs the compiled `classify.so` file generated by [`api/pre.sh`](https://github.com/second-state/vercel-wasm-runtime/blob/tensorflow/api/pre.sh) for better performance.


```
const fs = require('fs');
const { spawn } = require('child_process');
const path = require('path');

module.exports = (req, res) => {
  const wasmedge = spawn(
    path.join(__dirname, 'wasmedge-tensorflow-lite'),
    [path.join(__dirname, 'classify.so')],
    {env: {'LD_LIBRARY_PATH': __dirname}}
  );

  let d = [];
  wasmedge.stdout.on('data', (data) => {
    d.push(data);
  });

  wasmedge.on('close', (code) => {
    res.setHeader('Content-Type', `text/plain`);
    res.send(d.join(''));
  });

  wasmedge.stdin.write(req.body);
  wasmedge.stdin.end('');
}
```


You can now [deploy your forked repo to Vercel](https://vercel.com/docs/git#deploying-a-git-repository) and have a web app for subject classification.


## What's next

Running WasmEdge from Vercel’s current serverless container is an easy way to add high-performance functions to Vercel applications. If you have created an interesting Vercel function using WasmEdge, please[let us know and we will send you a WasmEdge SWAG](https://forms.gle/ozSNLz1MCJTNj9n18)!

Going forward an even better approach is to use WasmEdge as the container itself. There will be no Docker and no Node.JS to bootstrap WasmEdge. This way, we can reach much higher efficiency for running serverless functions. WasmEdge is [already compatible with Docker tools](https://www.secondstate.io/articles/manage-webassembly-apps-in-wasmedge-using-docker-tools/). If you are interested in joining WasmEdge and CNCF for this exciting work, [let us know](https://github.com/WasmEdge/WasmEdge#contact)!