Zig Makes Go Cross Compilation Just Work
Tags: zig, go, cpp, systems
Published: 2021-01-24 22:06:28

**See also the followup post: https://zig.news/kristoff/building-sqlite-with-cgo-for-every-os-4cic**

For the last couple of months I worked on a redesign of https://ziglang.org. [Among other things](https://kristoff.it/blog/first-zig-website-redesign/), **the site was ported to Hugo**, a popular static site generator written in Go. Everything went smoothly, but I did encounter a snag when setting up the deploy pipeline: **I could not build Hugo for x86_64 Linux from my Apple Silicon Mac mini!** 

![Go cross-compilation failed](https://dev-to-uploads.s3.amazonaws.com/i/gvwu5amzos85iximb4k2.png)<center><i>The failed build.</i></center>


## How could this be?
Go does have the ability to compile a project for another platform, you just need to specify `GOOS` and `GOARCH` when running `go build`, like I did in the screenshot above. The problem is the remaining environment variable: `CGO_ENABLED`, which caused the build command to fail.

### Hugo C extensions
It just so happens that Hugo can be built with or without a set of C extensions used to manipulate CSS and other assets. If you want the C extensions (like in my case), you need to enable [cgo](https://golang.org/cmd/cgo/), a piece of the Go toolchain that handles compilation and linking of C code.

## Pesky C code
Compiling C code has always been a bit of a nuisance, and especially so when it comes to cross compilation. **If you search for `how to cross compile cgo` you will find a long list of suffering and hopelessness**. This is what Dave Cheney replied to one of such questions on Stack Overflow:

![Dave Cheney recommends giving up on cross compiling cgo (he was not wrong at the time)](https://dev-to-uploads.s3.amazonaws.com/i/p2lqp6uurcb6td7z7212.png "Dave Cheney recommends giving up on cross compiling cgo (he was not wrong at the time)")

Well, a few years plus a lot of collective effort later, **I'm happy to show you how to cross compile trivially** :)

## Say hello to Zig
Zig is a new programming language that has no runtime, no macros, [a radical compile-time metaprogramming system](https://kristoff.it/blog/what-is-zig-comptime/), and seamless C interoperability. You can even import C header files directly and immediately use all the definitions in your Zig code, without needing any glue / bindgen. 

Even better, **Zig is a full-fledged C/C++ cross compiler** that leverages LLVM. The crucial detail here is what Zig includes to make cross compilation possible: **Zig bundles standard libraries for all major platforms** (GNU libc, musl libc, ...), **an advanced artifact caching system**, and it has a **flag-compatible interface for both clang and gcc**.

This means that Zig is a dependency-free, in-place replacement for your current C/C++ compiler that allows cross compilation out-of-the-box. Just **download a Zig tarball, extract it somewhere, and boom: you can now cross compile** to your heart's content.

Let's see how to use Zig from Go.

# How to use Zig to cross compile Hugo (with C extensions)

**First of all, [you need to download Zig](https://ziglang.org)**. You can either get a tarball as mentioned above, or have your favorite package manager install everything for you. You can even find Zig in [Homebrew](https://formulae.brew.sh/formula/zig#default) (Mac) and [Chocolatey](https://chocolatey.org/packages/zig) (Windows).

**You also need to make sure `zig` is present in your `PATH`**, so that you can call the compiler from any directory. If you're not sure how to do it, [check out the Getting Started](https://ziglang.org/learn/getting-started/) guide. Package managers should take care of `PATH` for you, if you decide to go that route.

To test if you have setup Zig correctly, run `zig version` in a terminal, it should reply with something similar (i.e. it should not error out, but the version might be different of course):

```
0.7.1
```

### Invoking Zig from Go
We're finally at the climax! How hard is it to call into Zig when compiling a cgo project?

**If you have Go version 1.18 or above**, then you only have to tell Go to use Zig to compile C/C++ code.

If you want to cross compile for x86_64 Linux, for example, all you need to do is add `CC="zig cc -target x86_64-linux" CXX="zig c++ -target x86_64-linux"` to the list of env variables when invoking `go build`. In the case of Hugo, this is the complete command line:

```
CGO_ENABLED=1 GOOS=linux GOARCH=amd64 CC="zig cc -target x86_64-linux" CXX="zig c++ -target x86_64-linux" go build --tags extended
```

__That's it!__ Really. Trivial cross compilation indeed.

### Important notes
As you probably noted, **we've repeated the cross compilation target multiple times**. 

Unfortunately, Go doesn't provide this information to the C/C++ compiler, so it's up to us to provide that little bit of glue. This means that **if you want a different target, you will have to change both Zig invocations and the `GOOS`/`GOARCH` variables**.

Another important detail is that **Zig calls `x86_64` what Go calls `amd64`**. That's the most notable difference in naming conventions, so keep that in mind.

Finally, you may be interested in knowing that Zig can also accept a third option when specifying the target architecture: the libc ABI.

For Windows, you want gnu (e.g. `x86_64-windows-gnu`) because that will use Zig's bundled MinGW-w64 instead of trying to find an MSVC installation. Note: there's a problem with targeting Windows, tracked in [this](https://github.com/ziglang/zig/issues/7874) issue ([downstream issue](https://github.com/golang/go/issues/43886#issuecomment-766483975)).

For Linux, you probably want musl (e.g. `x86_64-linux-musl`) because your resulting binary will be statically linked and thus work on all Linux distributions. However, if you prefer to interact with the system glibc, such as on Ubuntu, you can specify gnu (e.g. `x86_64-linux-gnu`).

The Zig language reference contains [the full list of supported targets](https://ziglang.org/documentation/master/#Targets).

## For older versions of Go
**For versions of Go lower than 1.18**

This workaround consists of 2 bash scripts that wrap the two Zig commands into single-argument commands (it might seem silly, but that's what the bug is about). 

Here are the steps:

### 1. Create the scripts
__zcc__
```sh
#!/bin/sh
ZIG_LOCAL_CACHE_DIR="$HOME/tmp" zig cc -target x86_64-linux $@
```

__zxx__
```sh
#!/bin/sh
ZIG_LOCAL_CACHE_DIR="$HOME/tmp" zig c++ -target x86_64-linux $@
```
### 2. Make the scripts executable
```shell
$ chmod +x zcc zxx
```

### 3. Add the scripts to `PATH`
If you don't know how to do it, it's the same procedure explained in the [Getting Started](https://ziglang.org/learn/getting-started/) guide: you want to add to `PATH` the directory containing `zcc` and `zxx`.

### 4. Use the scripts as your C/C++ compiler
Just specify `CC="zcc" CXX="zxx"` when building and you're good to go! Here's the full command line for Hugo:
```
CGO_ENABLED=1 GOOS=linux GOARCH=amd64 CC="zcc" CXX="zxx" go build --tags extended
```

This is what I did in my case and, well, it just worked.
![The successful build](https://dev-to-uploads.s3.amazonaws.com/i/iwn7awp4tpqy47kz0xwl.png "The successful build")<center><i>The successful build.</i></center>

## Conclusion
I think Andrew (the creator of Zig) captured the conclusion perfectly in this Tweet.

{% tweet 1349864095399383040 %}

### Is this solution Go-only?
This should be easy to infer, but to be absolutely clear: no, this is not a feature designed specifically for Go.

**Zig can be used as a C/C++ cross compiler directly or from other toolchains**. 

**Zig can also be used by [cc-rs](https://crates.io/crates/cc), a Rust crate used for shelling out to a C/C++ compiler, for example**.

If you want a more detailed explanation [read this blog post by Andrew](https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-clang.html).

### Couldn't you just download a prebuilt Hugo executable?
I have my own [small fork of Hugo](https://kristoff.it/blog/first-zig-website-redesign/#implementation-details) where I added a custom integration with zig-doctest, a tool that both tests and renders to html the real output of most of the code snippets present on the website.

{% github kristoff-it/zig-doctest no-readme %}

In other words, I had to build my own executable.
Originally [the CI on GitHub](https://github.com/ziglang/www.ziglang.org) would build Hugo every run, but that took 4 mins out of a 5 mins total runtime. After this change, we can now deploy in about 1 minute, with most of the time spent testing Zig code snippets, as should be.

### Oh no, it doesn't work!
You tried but got blasted with errors anyway? 

There are two possibilities: either you did something wrong, or we did (i.e. there's a bug somewhere or a particular C/C++ feature that's not yet supported). 

Here's how to fix that:
1. **[Join a Zig Community and ask for help](https://github.com/ziglang/zig/wiki/Community)**. People will be able to help you fix common mistakes. If this doesn't work, goto step 2.
2. **[Open an Issue on GitHub](https://github.com/ziglang/zig/issues)**. Make sure to explain in detail your setup and share the full error message you received. **We'll do our best to help you, especially if you did your due diligence with step 1**.  
  
<br>
*Now it's your turn to get out there and cross compile for great justice!*

**See also the followup post: https://zig.news/kristoff/building-sqlite-with-cgo-for-every-os-4cic**