How to run WebAssembly/WASI application (Spin) with Dapr on Kubernetes
Tags: wasi, kubernetes, dapr, webassembly
Published: 2023-09-04 09:09:16

## Introduction

As far as we know, [WebAssembly (WASM)](https://github.com/WebAssembly/spec) / [WebAssembly System Interface (WASI)](https://github.com/WebAssembly/WASI) is gaining traction these days. It comes with a lot of benefits such as faster startup times (no cold-start like serverless, start-up around milliseconds), near-native performance (WASM format), lightweight (very small sizing), convenience and versatility (build once run everywhere promise), security (sandbox by default).

But WASM/WASI itself is only strong for the computing workload (e.g. int + int => int) just like [serverless v2](https://www.fermyon.com/blog/next-generation-of-serverless-is-happening) in the cloud computing world. It is appropriate for heavy and ephemeral calculation/transformation tasks or some kind of trigger tasks by external components such as cloud services. And it might delegate persistent or message communication tasks to its old brothers (containers). That's the reason [Containers and WebAssembly should come and complement together](https://www.docker.com/blog/why-containers-and-webassembly-work-well-together/), not suited for stand-alone tasks when we intend to use it to build cloud applications.

Having said that, we can use many of the existing containers built by famous companies like Redis, Kafka, and RabbitMQ... Then we can run WebAssembly (WASI) components on Kubernetes with [containerd-wasm-shims](https://github.com/deislabs/containerd-wasm-shims). But with the very limitation of components for persistence data, message broker, networking management, etc. we might think about how can we leverage some kind of OSS from CNCF like [Dapr](https://github.com/dapr/dapr) or [KEDA](https://github.com/kedacore/keda) to amplify its power. Imagine that we can use Dapr for service invocation, data binding, pub/sub, distributed workflow, you name it, so we can build out some battery-included applications on the cloud. 

This article is the first step in addressing the described above. And it works because of the release of [containerd-wasm-shims v0.9.0](https://github.com/deislabs/containerd-wasm-shims/releases/tag/v0.9.0) just yesterday by DeisLabs team. In this release, there is a very cool feature called `Linux Container Side-by-Side in a Pod: You can now run Linux containers alongside WebAssembly containers within the same Kubernetes Pod`, and we can use it to run the Dapr [sidecar model](https://learn.microsoft.com/en-us/azure/architecture/patterns/sidecar) which runs side-by-side with [Spin component](https://github.com/fermyon/spin). We will deep dive into how to make it work in a minute.

## Sample application (Spin)

In this post, we would like to introduce a very simple application written on Spin which returns a list of products (drinks or foods) as below.

```
GET {{host}}/v1-get-item-types HTTP/1.1
content-type: application/json
```

```json
[
  {
    "image": "img/CAPPUCCINO.png",
    "itemType": 0,
    "name": "CAPPUCCINO",
    "price": 4.5
  },
  // ...
  {
    "image": "img/CROISSANT_CHOCOLATE.png",
    "itemType": 9,
    "name": "CROISSANT_CHOCOLATE",
    "price": 3.5
  }
]
```

The code in Rust language:

```rs
use anyhow::Result;
use bytes::Bytes;
use serde::{Serialize, Deserialize};
use serde_json::json;
use spin_sdk::{
    http::{Params, Request, Response},
    http_component,
};

#[derive(Debug, Serialize, Clone)]
#[serde(rename_all = "camelCase")]
struct ItemType {
    name: String,
    item_type: i8,
    price: f32,
    image: String,
}

#[derive(Debug, Deserialize)]
struct GetItemByTypeModel{
    types: String,
}

impl TryFrom<&Option<Bytes>> for GetItemByTypeModel {
    type Error = anyhow::Error;

    fn try_from(value: &Option<Bytes>) -> std::result::Result<Self, Self::Error> {
        match value {
            Some(b) => Ok(serde_json::from_slice::<GetItemByTypeModel>(b)?),
            None => Err(anyhow::anyhow!("No body")),
        }
    }
}

#[http_component]
fn handle_product_api(req: Request) -> Result<Response> {
    println!("{:?}", req.headers());

    let mut router = spin_sdk::http::Router::default();
    router.get("/", health_handler);
    router.get("/v1-get-item-types", get_item_types_handler);
    router.get("/v1-get-items-by-types", get_item_by_types_handler);
    router.handle(req)
}

fn health_handler(_req: Request, _params: Params) -> Result<Response> {
    Ok(http::Response::builder()
        .status(200)
        .body(Some("".into()))?)
}

fn get_item_types_handler(_req: Request, _params: Params) -> Result<Response> {
    let items = json!(get_item_types());
    let result = bytes::Bytes::from(items.to_string());
    Ok(http::Response::builder()
        .header("Content-Type", "application/json")
        .status(200)
        .body(Some(result))?)
}

fn get_item_by_types_handler(req: Request, _params: Params) -> Result<Response> {
    let Ok(model) = GetItemByTypeModel::try_from(&req.body().clone()) else {
        return Ok(http::Response::builder()
        .status(http::StatusCode::BAD_REQUEST)
        .body(None)?);
    };

    let mut temp: Vec<ItemType> = Vec::new();

    for i in get_item_types() {
        let parts = model.types.split(',');

        let ii = i.clone();
        for j in parts {
            if ii.item_type.to_string().as_str() == j {
                temp.push(ii.clone())
            }
        }
    } 

    let result = bytes::Bytes::from(json!(temp).to_string());
    Ok(http::Response::builder()
        .header("Content-Type", "application/json")
        .status(200)
        .body(Some(result))?)
}

fn get_item_types() -> Vec<ItemType> {
    vec![
        ItemType {
            name: "CAPPUCCINO".to_string(),
            item_type: 0,
            price: 4.5,
            image: "img/CAPPUCCINO.png".to_string(),
        },
        // ... 
        // ...
        ItemType {
            name: "CROISSANT_CHOCOLATE".to_string(),
            item_type: 9,
            price: 3.5,
            image: "img/CROISSANT_CHOCOLATE.png".to_string(),
        },
    ]
}
```

The whole source code for `product-api` can be found at https://github.com/thangchung/dapr-labs/blob/feat/spin-refactor/polyglot/product-api/src/lib.rs

## Package the Spin application into Container

To make it work on Kubernetes, we need to package the application with `docker buildx` and the target should be `wasi/wasm`. Let's do it below

Login into GitHub artifact hub:

```sh
docker login ghcr.io -u <your username>
```

It asks you to provide the password (PAT), please go to your developer profile to generate it.

Then we are ready to build the wasm/wasi container:

```dockerfile
FROM --platform=${BUILDPLATFORM} rust:1.67 AS build
RUN rustup target add wasm32-wasi
COPY . /product
WORKDIR /product
RUN cargo build --target wasm32-wasi --release

FROM scratch
COPY --from=build /product/target/wasm32-wasi/release/product_api.wasm /target/wasm32-wasi/release/product_api.wasm
COPY ./spin.toml /spin.toml
```

Let builds the image above:

```sh
cd product-api
docker buildx build -f Dockerfile --platform wasi/wasm,linux/amd64,linux/arm64 -t ghcr.io/thangchung/dapr-labs/product-api-spin:1.0.1 . --push
```

This action will build everything that is needed for the Spin app can run on a container, and then push it into the container hub. After that, you can see it at https://github.com/thangchung/dapr-labs/pkgs/container/dapr-labs%2Fproduct-api-spin/124547580?tag=1.0.1

## Prepare your Kubernetes (K3d) cluster

Install k3d into your Ubuntu:

```sh
wget -q -O - https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash
```

Make it shim to be able to run WASM/WASI workloads (slight, spin, wws, lunatic):

```sh
k3d cluster create wasm-cluster --image ghcr.io/deislabs/containerd-wasm-shims/examples/k3d:v0.9.0 -p "8081:80@loadbalancer" --agents 2
kubectl apply -f https://github.com/deislabs/containerd-wasm-shims/raw/main/deployments/workloads/runtime.yaml
kubectl apply -f https://github.com/deislabs/containerd-wasm-shims/raw/main/deployments/workloads/workload.yaml
echo "waiting 5 seconds for workload to be ready"
sleep 15
curl -v http://127.0.0.1:8081/spin/hello
curl -v http://127.0.0.1:8081/slight/hello
curl -v http://127.0.0.1:8081/wws/hello
curl -v http://127.0.0.1:8081/lunatic/hello
```

If you can `curl` it with 200 statuses, then everything is okay. Move on!

Refs: https://github.com/thangchung/containerd-wasm-shims/blob/main/deployments/k3d/README.md

## Install Dapr 1.11.2 into K3d

In this post, we use Dapr v1.11.2, and you should see as:

```sh
dapr --version
CLI version: 1.11.0
Runtime version: 1.11.2
```

For a demo purpose only, so we install it via Dapr CLI to `k3d` cluster (on production, we might need to use Dapr Helm chart with HA mode):

```sh
dapr init -k --runtime-version 1.11.2
```

Wait a second for Dapr installed on your cluster, then we continue to install Redis component into K3d cluster as well (not need redis for now, but might need in the next posts).

```sh
helm install my-redis oci://registry-1.docker.io/bitnamicharts/redis --set architecture=standalone --set global.redis.password=P@ssw0rd
```

Now we create some Dapr components which bind with Redis above:

```sh
kubectl apply -f components-k8s/
```

Query it:

```sh
kubectl get components
```

Should return:

```
NAME            AGE
baristapubsub   16h
kitchenpubsub   16h
statestore      16h
```

Okay, let's move on to the final step.

## Run Spin app with Dapr on Kubernetes (k3d)

We create the yaml file as below:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: product-api
spec:
  replicas: 1
  selector:
    matchLabels:
      app: product-api
  template:
    metadata:
      labels:
        app: product-api
      annotations:
(1)        dapr.io/enabled: "true"
(1)        dapr.io/app-id: "product-api"
(1)        dapr.io/app-port: "80"
(1)        dapr.io/enable-api-logging: "true"
    spec:
(2)      runtimeClassName: wasmtime-spin
      containers:
        - name: product-api
          image: ghcr.io/thangchung/dapr-labs/product-api-spin:1.0.1
          command: ["/"]
          env:
          - name: RUST_BACKTRACE
            value: "1"
          resources: # limit the resources to 128Mi of memory and 100m of CPU
            limits:
              cpu: 100m
              memory: 128Mi
            requests:
              cpu: 100m
              memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
  name: product-api
spec:
  type: LoadBalancer
  ports:
    - protocol: TCP
      port: 5001
      targetPort: 80
  selector:
    app: product-api

```

(1): daprized the application so that Dapr control plane will inject the sidecar automatically for us.
(2): [`Wasmtime`](https://github.com/bytecodealliance/wasmtime) with `Spin` is chosen for this application.

Then we apply it to create the `product-api`:

```sh
kubectl apply -f iac/kind-spin/product-api-deploy.yaml
```

Check it work:

```sh
kubectl get po
```

You should see:

```
NAME                         READY   STATUS    RESTARTS       AGE
my-redis-master-0            1/1     Running   8 (134m ago)   16h
product-api-8ccbc56b-gvlc2   2/2     Running   0              78m
```

If you notice, now we can have a sidecar (2/2) work very well with Spin app (WASM/WASI). Thanks, [Mossaka](https://github.com/Mossaka) and the [Deislabs team](https://github.com/deislabs) for working [very hard](https://github.com/containerd/runwasi/pull/156) on [it](https://github.com/deislabs/containerd-wasm-shims/pull/121).

If you tail the logs of `product-api`-`daprd`, you should see:

```sh
kubectl logs pod/product-api-8ccbc56b-dxfmj --namespace=default --container=daprd --since=0
```

```log
...
time="2023-09-04T08:47:10.311158673Z" level=info msg="Dapr trace sampler initialized: DaprTraceSampler(P=0.000100)" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.311260921Z" level=info msg="Initialized name resolution to kubernetes" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.311556754Z" level=info msg="Loading components…" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.313604323Z" level=info msg="Component loaded: kubernetes (secretstores.kubernetes/v1)" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.318570002Z" level=info msg="Component loaded: baristapubsub (pubsub.redis/v1)" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.321000955Z" level=info msg="Component loaded: kitchenpubsub (pubsub.redis/v1)" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.3210958Z" level=info msg="Waiting for all outstanding components to be processed" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.325038524Z" level=info msg="Using 'statestore' as actor state store" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.325088749Z" level=info msg="Component loaded: statestore (state.redis/v1)" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
...
time="2023-09-04T08:47:10.352600037Z" level=info msg="application protocol: http. waiting on port 80.  This will block until the app is listening on that port." app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
...
time="2023-09-04T08:47:16.169135624Z" level=info msg="actor runtime started. actor idle timeout: 1h0m0s. actor scan interval: 30s" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime.actor type=log ver=1.11.2
time="2023-09-04T08:47:16.169271184Z" level=info msg="Configuring workflow engine with actors backend" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime.wfengine type=log ver=1.11.2
time="2023-09-04T08:47:16.169288176Z" level=info msg="Registering component for dapr workflow engine..." app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:16.169440257Z" level=info msg="initializing Dapr workflow component" app_id=product-api component="dapr (workflow.dapr/v1)" instance=k3d-wasm-cluster-agent-1 scope=dapr.contrib type=log ver=1.11.2
time="2023-09-04T08:47:16.17126385Z" level=info msg="dapr initialized. Status: Running. Init Elapsed 5864ms" 
```

Amazing, it could daprized and run `product-api` successfully. I struggled to make it work for a month (before `containerd-wasm-shims v0.9.0`), and now it is like a dream 🙌.

Now we get the services:

```sh
kubectl get svc
```

You should get:

```
NAME                TYPE           CLUSTER-IP      EXTERNAL-IP                        PORT(S)                               AGE
kubernetes          ClusterIP      10.43.0.1       <none>                             443/TCP                               16h
my-redis-headless   ClusterIP      None            <none>                             6379/TCP                              16h
my-redis-master     ClusterIP      10.43.109.123   <none>                             6379/TCP                              16h
product-api-dapr    ClusterIP      None            <none>                             80/TCP,50001/TCP,50002/TCP,9090/TCP   81m
product-api         LoadBalancer   10.43.4.58      172.19.0.2,172.19.0.3,172.19.0.4   5001:32134/TCP                        81m
```

For demo purpose, we do a port forward our `product-api: 10.43.4.58` on `5001` port to host machine.

And using `curl`, we can achive our purpose like a breeze:

```
###
GET http://localhost:5001/v1-get-item-types HTTP/1.1
content-type: application/json
```
Return

```json
HTTP/1.1 200 OK
content-type: application/json
content-length: 776
date: Mon, 04 Sep 2023 08:27:42 GMT

[
  {
    "image": "img/CAPPUCCINO.png",
    "itemType": 0,
    "name": "CAPPUCCINO",
    "price": 4.5
  },
  ...
  {
    "image": "img/CROISSANT_CHOCOLATE.png",
    "itemType": 9,
    "name": "CROISSANT_CHOCOLATE",
    "price": 3.5
  }
]
```

Or,

```
###
GET http://localhost:5001/v1-get-items-by-types HTTP/1.1
content-type: application/json

{
  "types": "1,2,3"
}
```

Return:

```json
HTTP/1.1 200 OK
content-type: application/json
content-length: 241
date: Mon, 04 Sep 2023 08:29:19 GMT

[
  {
    "image": "img/COFFEE_BLACK.png",
    "itemType": 1,
    "name": "COFFEE_BLACK",
    "price": 3.0
  },
  {
    "image": "img/COFFEE_WITH_ROOM.png",
    "itemType": 2,
    "name": "COFFEE_WITH_ROOM",
    "price": 3.0
  },
  {
    "image": "img/ESPRESSO.png",
    "itemType": 3,
    "name": "ESPRESSO",
    "price": 3.5
  }
]
```

Ta-da!!! It worked ❤️

This is just the beginning of a series of posts about how can we run WASM/WASI with Dapr on Kubernetes. More fun to come soon.

> The source code of this sample can be found at https://github.com/thangchung/dapr-labs/tree/main/polyglot

WebAssembly, Docker container, Dapr, and Kubernetes better together series: [part 1](https://dev.to/thangchung/webassembly-docker-container-dapr-and-kubernetes-better-together-part-1-demo-scenario-with-coffeeshop-application-28i2), [part 2](https://dev.to/thangchung/webassembly-docker-container-dapr-and-kubernetes-better-together-part-2-build-and-run-the-coffee-shop-backend-services-3c0e), [part 3](https://dev.to/thangchung/webassembly-docker-container-dapr-and-kubernetes-better-together-part-3-daprize-the-coffee-backend-services-239n), [part 4](https://dev.to/thangchung/webassembly-docker-container-dapr-and-kubernetes-better-together-part-4-package-and-deploy-to-kubernetes-4bi8).