Building Multi-Architecture Container Images
Tags: cloudnative, docker, containers, azure
Published: 2023-09-06 01:14:56

---
series: Multi-Architecture Container Images
title: Building Multi-Architecture Container Images
description: Unlock the power of multi-architecture container images with this guide. Learn to build, deploy, and optimize your containerized applications for diverse processor architectures
published: true
date: 2023-09-05 13:00:00 UTC
tags: cloudnative, docker, containers, azure
canonical_url: https://paulyu.dev/article/building-and-deploying-multi-arch-container-images-guide/
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yox2n41fxudjznm961td.jpeg
---

## Introduction

Over the past several years, we've seen the emergence of the **ARM64** architecture, which is gaining popularity due to its energy efficiency and performance benefits. We often seen these processors in mobile devices, such as smartphones and tablets. We also seen them in our IoT devices, such as smartwatches and smart TVs. Now, we are starting to see increased adoption of **ARM64** processors in servers and personal computers.

This emergence of the **ARM64** architecture, means that we now have two major processor architectures to consider when building and deploying our containerized applications in order to maximize our application's reach. 

In this article, we'll explore the importance of multi-architecture (multi-arch) container images, how to build them, and how to deploy them on Azure Kubernetes Service (AKS).

## Why multi-arch container images matter

When distributing a containerized application, broad compatibility across different platforms is essential for widespread adoption. Whether you're sharing your container image with your team or the wider community, ensuring it runs seamlessly on various architectures is critical.

When it comes to cloud computing, cloud providers offer a wide range of processor architectures, each with its own set of benefits and drawbacks. For example, we know that **ARM64** processors are more energy efficient than **x86_64** processors. This means that running your containerized application on an **ARM64** processor will consume less power than running it on an **x86_64** processor. This can lead to significant cost savings for your organization.

By creating multi-arch container images, you create a single container image that contains multiple versions of your application, each targeting a specific OS architecture. This allows the client OS to determine the appropriate architecture to utilize.

## Acronyms galore

Before we dive into building multi-arch container images, it is important to understand some of nomenclature used to identify OS processor architectures. When it comes to identifying OS processor architectures, it is full of acronyms and can be a bit dizzying to decipher and it can be challenging to keep track of all the different processor architectures. 

Here's what we need to know:

- **x86** is a family of instruction set architectures based on the Intel 8086 CPU. The is the most widely used architecture and is supported by most operating systems. **x86_64** is a 64-bit version of the x86 architecture. 
- **ARM** is a family of instruction set architectures based on the ARM CPU. This is a newer architecture that is gaining popularity due to its energy efficiency and performance benefits. **ARM64** is a 64-bit version of the ARM architecture and you might also see it identified as **aarch64**.

So if you see, **x86_64** just know that its synonymous with **AMD64**, and **aarch64** is synonymous with **ARM64**.

## Approaches to build multi-arch container images

There are a few strategies for building container images that support multiple architectures. The first one is to build a separate image for each architecture. This approach has one huge drawback; you'll need to maintain multiple images and keep them in sync. From an end-user perspective, this means that they will need to know which image to use for their specific host OS architecture.

Another approach is to build each container image separately, then [combine them into a single image using the Docker manifest command](https://learn.microsoft.com/azure/container-registry/push-multi-architecture-images#build-and-push-a-multi-arch-image). This ultimately gives you a multi-arch container image; however, this approach is cumbersome and error-prone. 

And, do you really want to update the manifest file every time you build a new image?!? Probably not... there is a better way.

A better approach is to use a tool like [Docker Buildx](https://docs.docker.com/build/architecture/) to build a single image which can run on multiple architectures. Docker Buildx is a tool that [simplifies the process of building multi-arch container images](https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/). With this tool, you can use a single command to build multiple container images, each targeting a specific architecture, and combine the images into a single multi-arch image.

![This is the way](https://assets.paulyu.dev/articles/2023/07/streamline-network-observability-aks-addon-terraform-guide/this-is-the-way.jpg)

## Multi-arch container images in action

To demonstrate how to build multi-arch container images, we'll use a simple Rust application that I wrote. You can find the source code for this application [here](https://github.com/pauldotyu/osinfo). This application is a simple command line tool that prints out information about the OS it is running on.

### Testing the application locally

I'll start by cloning the repo and running the app locally to see what it does:

```bash
git clone https://github.com/pauldotyu/osinfo.git
```

```bash
cd osinfo
```

```bash
cargo run
```

> You will need Rust installed on your machine to run this application. You can install Rust by following the instructions [here](https://www.rust-lang.org/tools/install).

I am running this M1 Mac (Apple Silicon) and this is what I see:

```text
Type: Mac OS
Version: 13.5.1
Edition: None
Codename:
Bitness: 64-bit
Architecture: arm64
```

### Testing the application on GitHub Codespaces

If you have access to [GitHub Codespaces](https://docs.github.com/en/codespaces/developing-in-codespaces/creating-a-codespace-for-a-repository), open the repo in a Codespace and run the app.

You should see the following output which is different from what I saw on my M1 Mac:

```text
Type: Debian
Version: 11
Edition: None
Codename: bullseye
Bitness: 64-bit
Architecture: x86_64
```

### Running the application on AKS

What happens if I build a container from my M1 Mac, push it to Azure Container Registry (ACR), and attempt to run it on an AKS cluster?

Let's find out...

I'll need an AKS cluster, so I'll use the [Azure CLI](https://docs.microsoft.com/cli/azure/install-azure-cli) to create one:

```bash
# Create a resource group
az group create \
  -n rg-multiarch \
  -l eastus

# Create an Azure Container Registry
ACR_NAME=$(az acr create -n acrmultiarch${RANDOM} \
  -g rg-multiarch \
  --sku Standard \
  --admin-enabled \
  --query name -o tsv)

# Create an AKS cluster
az aks create \
  -g rg-multiarch \
  -n aks-multiarch \
  --attach-acr $ACR_NAME

# Connect to the AKS cluster
az aks get-credentials \
  -g rg-multiarch \
  -n aks-multiarch
```

> I am running these commands on my M1 Mac.

#### Run a container image built on **ARM64** architecture

There is a `Dockerfile` in the repo that I can use to build a container image. I'll use the `docker build` command to build and push the image from my M1 Mac:

```bash
docker build -t $ACR_NAME.azurecr.io/osinfo:v0.1-arm64 .
docker push $ACR_NAME.azurecr.io/osinfo:v0.1-arm64
```

> When creating a Dockerfile that will be used to build multi-arch container images, you'll need to ensure that the base image(s) supports the architectures you are targeting. For example, I am using is `rust:1.72-bullseye` and `rust:1.72-slim-bullseye` images and they both support `linux/amd64` and `linux/arm64` platforms. You can find more information about these image [here](https://hub.docker.com/layers/library/rust/1.72.0-bullseye/images/sha256-e431500c0bd21997da93fbb6cc3a20833f1307306e18982eeb503b6f432ee9f5?context=explore) and [here](https://hub.docker.com/layers/library/rust/1.72-slim-bullseye/images/sha256-20e595a6dde0524def375640ac3583ea759b2ac67ec92f96fc326d3dcf457854?context=explore)

With the container pushed to ACR, run the container on the AKS cluster:

```bash
kubectl run osinfo1 --image=$ACR_NAME.azurecr.io/osinfo:v0.1-arm64 --restart=Never
```

If you run the command `kubectl get po osinfo1`, you'll see that the pod is in a `Error` state.

```text
NAME     READY   STATUS   RESTARTS   AGE
osinfo1  0/1     Error    0          7s
```

Run the command `kubectl logs osinfo1`, and you'll see the following error message:

```text
exec ./osinfo: exec format error
```

This means that the container image is not compatible with the architecture of the AKS node. The node is running on the **x86_64** architecture and the container image is built for the **ARM64** architecture. This is expected since I built the container image on my M1 Mac, which is running on the **ARM64** architecture.

#### Run a container image built on **AMD64** architecture

Let's try building the container image on an **x86_64** machine and see what happens. This time, I'll use my GitHub Codespaces to build and push the container image by running the `docker build`  and `docker push` commands as I did on my M1 Mac:

```bash
# Login to Azure
az login

# Get ACR name
ACR_NAME=$(az acr list -g rg-multiarch --query "[].name" -o tsv)

# Build and push container image
docker build -t $ACR_NAME.azurecr.io/osinfo:v0.1-amd64 .
docker push $ACR_NAME.azurecr.io/osinfo:v0.1-amd64
```

Run the container image on AKS:

```bash
# Get AKS credentials
az aks get-credentials --resource-group rg-multiarch --name aks-multiarch

# Run container image on AKS
kubectl run osinfo2 --image=$ACR_NAME.azurecr.io/osinfo:v0.1-amd64 --restart=Never
```

If you run the `kubectl get po osinfo2` command, you should see the pod is in a `Completed` state.

```text
NAME     READY   STATUS      RESTARTS   AGE
osinfo2  0/1     Completed   0          20s
```

If you run `kubectl logs osinfo2`, you'll see the following output:

```text
Type: Debian
Version: 11.0.0
Edition: None
Codename: 
Bitness: 64-bit
Architecture: x86_64
```

So we can see here that where the container image was built matters. If you build the container image on an **x86_64** machine, it will run on an **x86_64** machine. If you build the container image on an **ARM64** machine, it will run on an **ARM64** machine.

#### Build and run a multi-arch container image

Now, let's use Docker Buildx to build a multi-arch container image. I'll use the `docker buildx` command to build and push the image from my M1 Mac which has Docker Desktop installed. Buildx uses emulation to build container images for different architectures. This means that you can build container images for architectures that are different from the host machine.

If you are using Docker Desktop, Buildx should be enabled by default. You can verify this by running the following command:

```bash
docker buildx version
```

But to enable Buildx to build for multiple platforms, you'll need to run the following command:

```bash
docker buildx create --use
```

When building container images using Buildx, you'll need to know which "platform" the container image is targeting. Docker uses the combination of `os/arch` to identify the platform in which an image can run on. For example, `linux/amd64` is the platform for running on Linux on x86_64 architecture and `linux/arm64` is the platform for running on Linux on ARM64 architecture.

Run the following command to build the a multi-arch container image:

```bash
docker buildx build --platform linux/amd64,linux/arm64 --push -t $ACR_NAME.azurecr.io/osinfo:v0.1-multi .
```

Run the following command to confirm the container image supports multiple architectures:

```bash
docker manifest inspect $ACR_NAME.azurecr.io/osinfo:v0.1-multi
```

You should see output similar to the following and you will notice both **AMD64** and **ARM64** are included in the output. This means that your container image can run on either architecture.

```json
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.oci.image.index.v1+json",
   "manifests": [
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 1055,
         "digest": "sha256:76781cd09ce46d5c0cf13b0c834c3d39e649c6c392c42343284ec851125f819b",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 1055,
         "digest": "sha256:26ebcb5ac14907804552477bf01f2892cdaebac06035e7561df4c5c61e6d8c30",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 566,
         "digest": "sha256:875f61309b8199a0582faa4ebe8aaca9412e9982774a3bf9d9d24f5a61d7f020",
         "platform": {
            "architecture": "unknown",
            "os": "unknown"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 566,
         "digest": "sha256:76e5d0269adffa14b470b22856c653f466644b213b53d5fac0461fff88942362",
         "platform": {
            "architecture": "unknown",
            "os": "unknown"
         }
      }
   ]
}
```

AKS supports multiple node pools with different host OS architectures. We can tell Kubernetes to run a pod on a specific type of node by using a `nodeSelector` to ensure it runs on node that has the appropriate architecture.

Run the following command to run the container image on a node that has the **x86_64** architecture:

```bash
kubectl run osinfo3 --image=$ACR_NAME.azurecr.io/osinfo:v0.1-multi --restart=Never --overrides='{"apiVersion":"v1","spec":{"nodeSelector":{"kubernetes.io/arch":"amd64"}}}'
```

Run `kubectl get po osinfo3` and the `kubectl logs osinfo3` commands, you'll see the output is the same as the previous example. It ran successfully on the **x86_64** node.

Now, let's add a new ARM64 node pool to our AKS cluster, and see what happens when we run the container image on the new node pool.

```bash
az aks nodepool add \
  -g rg-multiarch \
  --cluster-name aks-multiarch \
  --name armpool \
  --node-vm-size Standard_B2ps_v2
```

Run the container image and use a `nodeSelector` again to ensure it runs on an **ARM64** node:

```bash
kubectl run osinfo4 --image=$ACR_NAME.azurecr.io/osinfo:v0.1-multi --restart=Never --overrides='{"apiVersion":"v1","spec":{"nodeSelector":{"kubernetes.io/arch":"arm64"}}}'
```

If you run the `kubectl logs osinfo4` command, you should see the following output:

```text
Type: Debian
Version: 11.0.0
Edition: None
Codename: 
Bitness: 64-bit
Architecture: aarch64
```

This means that we've successfully built a multi-arch container image that can run on both **x86_64** and **ARM64** architectures 🎉

## Conclusion

In this article, we explored the importance of multi-arch container images and how to build them using the Docker Buildx tool. Docker Buildx is valuable tool that simplifies the process of building multi-arch container images. By creating a single container image that supports multiple architectures, you can ensure that your application can run seamlessly on diverse hosts. This enables you to reach a wider audience, increase the adoption of your application, and potentially save some money in the cloud.

The knowledge we have gained here on how to use Docker Buildx will help us as we take a step further into automation. Because, running these `docker buildx build` commands locally is not ideal either. 

So stay tuned for an upcoming article where I will show you how to build and publish multi-arch container images to Azure Container Registry using [GitHub Actions](https://docs.github.com/en/actions) and the [Build and push Docker images](https://github.com/marketplace/actions/build-and-push-docker-images) GitHub Action 🚀

If you have any feedback or suggestions, please feel free to reach out to me on [Twitter](https://twitter.com/pauldotyu) or [LinkedIn](https://www.linkedin.com/in/yupaul).

Peace ✌️

## References

- [Docker Multi-Architecture Images](https://docs.docker.com/desktop/multi-arch/)
- [Docker manifest command documentation](https://docs.docker.com/engine/reference/commandline/manifest/)
- [Docker Buildx](https://docs.docker.com/buildx/working-with-buildx/)