The future of distributed computing; transitioning microservices to WebAssembly(Wasm) via RPC.
By: Timothy Mccallum
Tags: Mobile, Virtual Machine, Microprocessor, Rust, Webassembly

1. Write custom Rust code and compile to Wasm

Install Rust

Create a new Rust project

cd ~

cargo new --lib add_numbers_via_wavm

cd add_numbers_via_wavm

Edit the Cargo.toml file; add the lib section & also add the dependencies as shown below

[lib]

name = "adding_lib"

path = "src/adding.rs"

crate-type =["cdylib"] [dependencies]

serde_json = "1.0"

Add the necessary Wasm software and configuration, at the command line

rustup target add wasm32-wasi

rustup override set nightly

Create a new file called ~/.cargo/config and place the following build text into this newly created config file

[build]

target = "wasm32-wasi"

Write a custom Rust program with a couple of different functions that we can call. In our case below the functions “double” and “triple” which take an integer each and multiply by 2 and 3 respectively.

use serde_json::json; pub extern fn print(answer: i32){

let the_answer = json!({

"Result": answer});

println!("{}", the_answer.to_string());

} #[no_mangle]

pub extern fn double(x: i32){

let z = x * 2;

print(z);

} #[no_mangle]

pub extern fn triple(x: i32){

let z = x * 3;

print(z);

}

The above code can be compile with the following command

cargo build --release

2. Set up a Remote Procedure Call (RPC) server

I found a neat C++ RPC Server called rpcsrv. The aim here is to use this C++ RPC Server to accept HTTP POST and convert them into system calls via C++.



sudo apt-get -y upgrade

sudo apt-get install autoconf

sudo apt install libevent-2.1-6

sudo apt-get install libevent-dev

sudo apt-get install libtool

cd ~

git clone

cd ~/rpcsrv

git submodule update --init

./autogen.sh

./configure

make

sudo make install sudo apt-get updatesudo apt-get -y upgradesudo apt-get install autoconfsudo apt install libevent-2.1-6sudo apt-get install libevent-devsudo apt-get install libtoolcd ~git clone https://github.com/jgarzik/rpcsrv.git cd ~/rpcsrvgit submodule update --init./autogen.sh./configuremakesudo make install

Start the RPC using the following commands

sudo ./rpcsrvd --listen-port 8080

3. Define custom services on the RPC server

Before we go any further I would just like to briefly discuss the use of JSON. I briefly explored a brilliant concept of the univalue. UniValue is high performance RAII C++ JSON library and universal value object class. One which I need to find more time to investigate thoroughly. For convenience in this demonstration, I used a combination of UniValue and rapidjson. Again, more time is need to explore this to ensure the best method of data interchange and interoperability.

The following code was used to install rapidjson.



cd ~/rapidjson

git submodule update --init

mkdir build

cd ~/rapidjson/build/

cmake ..

sudo cp -rp ~/rapidjson/include/rapidjson /usr/include/ cd ~

https://github.com/Tencent/rapidjson.git cd ~/rapidjsongit submodule update --initmkdir buildcd ~/rapidjson/build/cmake ..sudo cp -rp ~/rapidjson/include/rapidjson /usr/include/

After installing rapidjson, I then modified the top of the original C++ API file to include the rapidjson functionality in the rpcsrv codebase.

#include "rapidjson/document.h"

#include "rapidjson/writer.h"

#include "rapidjson/stringbuffer.h"

#include <iostream>

using namespace rapidjson;

At this stage we were able to go ahead and use the rapidjson functionality in our C++ code. Below is an example of how the original echo function was modified for this demonstration

//

// RPC "echo"

//

static UniValue myapi_1_echo(const UniValue & jreq,

const UniValue & params) { // Receive and parse incoming parameters

string s = params.write();

const char * theJson = s.c_str();

Document d;

d.Parse(theJson); // Assign parameters to C++ variables and print to console

Value & theService = d["Service Name"];

Value & theType = d["Type"];

Value & theFunctionName = d["Execution"]["FunctionName"];

Value & theArgument = d["Execution"]["Argument"];

cout << endl;

cout << "Received a new request to execute a service on Wasm Virtual

Machine..." << endl;

cout << s.c_str() << endl;

cout << endl;

cout << "Service Name is: " << theService.GetString() << endl;

cout << "Service Type is: " << theType.GetString() << endl;

cout << "Wasm function is: " << theFunctionName.GetString() << endl;

cout << "Wasm function argument: " << theArgument.GetString() <<

endl;



// Construct and execute the call to Wasm Virtual Machine

string commandString = "wavm run --abi=wasi --function=";

commandString += theFunctionName.GetString();

commandString += " ";

commandString += "~/add_numbers_via_wavm/target/wasm32

wasi / release / adding_lib.wasm ";

commandString += " ";

commandString += theArgument.GetString();

cout << "

";

cout << "Executing command ... " << endl;

string theWasmResults =

execute_function_and_return_output(commandString);



// Print the result to console

cout << "Results are as follows ...: " << endl;

cout << theWasmResults << endl;

UniValue result(theWasmResults);

cout << "Finished." << endl; // Return the results back to the caller of the RPC

return jrpcOk(jreq, result);

}

4. Install a Wasm Virtual Machine (WAVM)

WAVM uses LLVM to compile WebAssembly code to machine code with close to native performance.

The following instructions were used to install WAVM

sudo apt-get install gcc

sudo apt-get install clang

wget https://github.com/WAVM/WAVM/releases/download/nightly%2F2019-11-04/wavm-0.0.0-prerelease-linux.deb

sudo apt install ./wavm-0.0.0-prerelease-linux.deb

5. Execute your custom WebAssembly (Wasm) code remotely via HTTP Post (i.e. Curl, Python etc.)

The execution can be performed by any mechanism which is capable of producing HTTP POST. For example everything from GUIs like Postman right through to Linux curl command and of course both interpreted and compiled code bases like Python and Java respectively.

Here is an example of the calling code using Curl in the linux command line

Curl - passing in valid JSON

tpmccallum$ curl --header "Content-Type: application/json" --request POST --data '{"jsonrpc":"2.0","method":"echo","params":{"Service Name": "Double the digits","Type": "Execution","Execution": {"FunctionName": "double","Argument": "10"}}, "id":1}' http://123.456.78.9:8080/rpc/1

When looking at this calling code, remember that our Rust program, from which the Wasm originates, has two functions; “double” and “triple”. Our additional layer of RPC means that these original functions are now defined as two individual services.

As you can see above, we are not only specifying which of the services we wish to call, but also specifying the single argument required. When this POST is executed across the web the RPC server directly calls the WAVM and then a JSON result object is returned to the calling code.

Returns valid JSON

{

"jsonrpc": "2.0",

"result": {

"Result": 20

},

"id": 1

}

The return object is completely configurable, this is just a simple example which returns the result of the computation.

RPC Server Output

This RPC server output is optional and was just created for demonstration purposes. It still demonstrates that the RPC server has a hand in passing the JSON back and forth. There are opportunities for additional formatting etc. to be built into the RPC layer (which are over and above the Rust and Wasm code)

Received a new request to execute a service on Wasm Virtual Machine... {"Service Name":"Double the digits","Type":"Execution","Execution":{"FunctionName":"double","Argument":"10"}} Service Name is: Double the digits

Service Type is: Execution

Wasm function is: double

Wasm function argument: 10 Executing command ... Results are as follows ...:

{"Result":20} Finished.

Python - passing in valid JSON

System setup

sudo apt-get install python-pip

pip install json-rpc

pip install requests

We now pass in the JSON, to the Python code, in a way that describes which service we want. In this case we want to double the number 10 i.e. calling “FunctionName”: “double” with “Argument”: “10”.

>>>import requests

>>>import json



>>>url = "http://123.456.78.9:8080/rpc/1"

>>>payload = {

"jsonrpc":"2.0","method":"echo","params":{"Service Name":"Double the digits","Type": "Execution","Execution": {"FunctionName": "double","Argument": "10"}}, "id":1

}

>>>response = requests.post(url, json=payload).json()

As we can now see, the response returns the result of the Wasm execution i.e. “Result”:20.

>>> print response {u'jsonrpc': u'2.0', u'result': u'{"Result":20}, u'id': 1}

We can try this again, by calling the other service i.e. “FunctionName”: “triple” with “Argument”: “10”

>>>url = "http://123.456.78.9:8080/rpc/1"

>>>payload = {

"jsonrpc":"2.0","method":"echo","params":{"Service Name":"Triple the digits","Type": "Execution","Execution": {"FunctionName": "triple","Argument": "10"}}, "id":1

}

>>>response = requests.post(url, json=payload).json()

Again, we can see that this response is the correct result for the chosen service.

>>> print response {u'jsonrpc': u'2.0', u'result': u'{"Result":30}', u'id': 1}

This article has demonstrated the use of Wasm via RPC. By day, I am a passionate open source blockchain software researcher and core developer for SecondState (Dallas, Austin, Beijing, Shanghai, HongKong and Taipei). If you would like to learn more about Wasm and other technologies which can improve your operation please get in touch via email or GitHub.

References

Arpaci-Dusseau, R.H. and Arpaci-Dusseau, A.C., 2018. Operating systems: Three easy pieces. Arpaci-Dusseau Books LLC.

Rossberg, A., Titzer, B., Haas, A., Schuff, D., Gohman, D., Wagner, L., Zakai, A., Bastien, J. and Holman, M. (2018). Bringing the web up to speed with WebAssembly. Communications of the ACM, 61(12), pp.107–115.

Tech.ebayinc.com. (2019). WebAssembly at eBay: A Real-World Use Case. [online] Available at: https://tech.ebayinc.com/engineering/webassembly-at-ebay-a-real-world-use-case/ [Accessed 20 Nov. 2019].