How To Use Zig for Elixir NIFs
Tags: elixir
Published: 2023-10-31 14:44:38

Elixir excels at building scalable and maintainable applications. However, sometimes Elixir is not the best language to tackle specific tasks, and it can fall short in some areas, like direct system interaction.

Fortunately, Elixir offers NIFs (Native Implemented Functions): a path for integrating with other languages to improve these gaps. NIFs in Elixir act as an interface to call functions written in native languages like C or Rust, enabling developers to optimize performance-critical sections of their code.

NIFs can be written in many languages like Rust, Python, and Zig. For this tutorial, we will use Zig, due to its simplicity, speed, and safety.

Let's get going!

## What We'll Cover

We'll cover the following topics in this tutorial:

- Setting up Zig for Elixir NIF development.
- Understanding NIFs in Elixir.
- Writing a simple NIF in Zig.
- Integrating the NIF with an Elixir application.

## What Is Zig?

[Zig](https://ziglang.org/) is a general-purpose programming language designed for robustness, optimality, and maintainability.

It is known to provide excellent debugging and error-handling capabilities, making it suitable for various applications (including systems programming, embedded systems, and performance-critical applications).

## Setting up Zig for Elixir Development

Let's now look at how we can set up Zig for Elixir NIF development.

### Installing Zig

To get started with Zig, download the latest version of the Zig compiler from the [official Zig website](https://ziglang.org/download/). Follow the installation instructions for your specific operating system.

### Configuring Zig for Elixir NIF Development

To configure Zig for Elixir NIF development, you'll need to include the necessary libraries and dependencies.

Elixir NIFs rely on the Erlang NIF API provided by the `erl_nif.h` header file. Ensure you have the Erlang development package installed, and include the appropriate path to `erl_nif.h` in your build script or Zig build file.

Just add the following line to your `src/main.zig` file:

```zig
const erl = @cImport({
    @cInclude("erl_nif.h");
});
```

Importing this library will allow us to use the Erlang NIF API in our Zig code. It contains the native functions that allow our Zig code to interact with the Erlang VM. Later in this tutorial, we will see how to use this library to create our NIF.

### Verifying the Setup

To verify that Zig is correctly set up for Elixir NIF development, we will create a simple "Hello, World!" project in Zig.

Go ahead and run the following commands:

```bash
mkdir hello-world
cd hello-world
zig init-exe
```

If `zig` has been installed correctly, you should see the following output:

```bash
info: Created build.zig
info: Created src/main.zig
info: Next, try `zig build --help` or `zig build run`
```

Next, we'll run `zig build run` on our terminal to ensure we can compile and execute a Zig program. If successful, you should see the following output:

```bash
All your codebase are belong to us.
Run `zig build test` to run the tests.
```

Now that we are able to compile and run Zig code, we can move forward.

## Understanding NIFs in Elixir

### Why Use NIFs in Elixir?

NIFs can be called from Elixir code, providing a performance boost for computationally intensive tasks. While Elixir excels at concurrency and fault tolerance, there might be better choices for performance-sensitive tasks.

By writing NIFs in a lower-level language like Zig, you can leverage the language's speed and efficiency without sacrificing Elixir's maintainability and concurrency features.

### Advantages of Using Zig for NIF Development

Zig offers several advantages for NIF development:

- **Performance**: Zig compiles to efficient native code, providing significant performance improvements over interpreted languages like Elixir.
- **Safety**: Zig has strong static typing, compile-time checks, and a focus on error handling, reducing the likelihood of runtime errors.
- **Simplicity**: Zig's straightforward syntax and semantics make it easy to learn and use, especially for developers familiar with C or C++. Zig's standard library is also very comprehensive, providing a wide range of functionality for common tasks.

## Writing our First NIF

Let's start by setting up our Elixir project. Run the following command:

```bash
mix new example_nif
cd example_nif
```

This will create a basic Elixir project called `ExampleNif`. Next, let's initialize the Zig project inside our Elixir project with the following command:

```bash
zig init-lib
```

This will initialize a library inside our project that will create the following files:

- `build.zig` - which contains the code for compiling and building our zig library.
- `src/main.zig` - this is the main code of our library and will contain the logic that we will implement.

### Writing a Simple NIF in Zig

Our first step when working with Zig is to update the `build.zig` file with the following code:

```zig
const std = @import("std");
const Pkg = std.build.Pkg;

pub fn build(b: *std.build.Builder) void {
    const mode = b.standardReleaseOptions();

    const nif_step = b.step("example_lib", "Compiles erlang library");
    const example_lib = b.addSharedLibrary("example_nif", "./src/main.zig", .unversioned);
    example_lib.setBuildMode(mode);
    example_lib.setOutputDir("build");
    example_lib.addIncludePath("/usr/lib/erlang/usr/include/");

    example_lib.install();
    example_lib.linkLibC();
    nif_step.dependOn(&example_lib.step);
}
```

This will add the Erlang header path to the build environment and also link with the C library.

Next, we will update the `src/main.zig` code. For this example, we'll create a simple NIF that takes two integers and returns their sum. This function will be written in Zig and called from Elixir code.

```zig
const std = @import("std");
const erl = @cImport({
    @cInclude("erl_nif.h");
});

export fn nif_add(
    env: ?*erl.ErlNifEnv,
    argc: c_int,
    argv: [*c]const erl.ERL_NIF_TERM,
) erl.ERL_NIF_TERM {
    var a: i32 = undefined;
    var b: i32 = undefined;

    if ((argc != 2))
    {
        return erl.enif_make_badarg(env);
    }

    _ = erl.enif_get_int(env, argv[0], &a);
    _ = erl.enif_get_int(env, argv[1], &b);

    const result = a + b;
    return erl.enif_make_int(env, result);
}

const func_count = 1;

var funcs = [func_count]erl.ErlNifFunc{
    erl.ErlNifFunc{
        .name = "nif_add",
        .arity = 2,
        .fptr = nif_add,
        .flags = 0,
    },
};

var entry = erl.ErlNifEntry{
    .major = erl.ERL_NIF_MAJOR_VERSION,
    .minor = erl.ERL_NIF_MINOR_VERSION,
    .name = "Elixir.ExampleNif",
    .num_of_funcs = func_count,
    .funcs = &funcs,
    .load = null,
    .reload = null,
    .upgrade = null,
    .unload = null,
    .vm_variant = "beam.vanilla",
    .options = 1,
    .sizeof_ErlNifResourceTypeInit = @sizeOf(erl.ErlNifResourceTypeInit),
    .min_erts = "erts-10.4",
};

export fn nif_init() *erl.ErlNifEntry {
    return &entry;
}
```

While this code may look intimidating, it's actually quite simple. Let's break it down:

- We are first importing the `erl_nif.h` header file, which contains the definitions for the Erlang NIF API and the std library.
- Next, we define our first and only library function, `nim_add`, which takes three arguments:
  - `env` is a pointer to an `ErlNifEnv` struct, which contains the environment for the NIF. This struct is used to allocate memory, create Erlang terms, and more.
  - `argc` is the number of arguments passed to the NIF.
  - `argv` is an array of Erlang terms (the arguments passed to the NIF).
  - The function code itself is a very simple and naive implementation that adds two integers.
- After we have defined all our functions, we define a constant `func_count` (the number of functions in the NIF).
- We then define an array of `ErlNifFunc` structs, containing the name, arity, and function pointer for each function in the NIF.
- Finally, we define an `ErlNifEntry` struct, which contains the NIF's version, name, and functions. This struct is used to register the NIF with the Erlang VM.

It is important to highlight that the `num_of_funcs` field in the `ErlNifEntry` struct must match the number of functions in the `funcs` array. The `name` field must also match the module name and function in the `NIF_MOD_FUNCS` of the `build.zig` file.

### Compiling the Zig Code Into a NIF

To compile the Zig code into a NIF, use the zig build-lib command, specifying the appropriate target and output file. For example, on a Linux system:

```bash
zig build example_lib
```

This command will produce a shared library file: `libexample_nif.so` in the `build/` directory.

### Integrating the NIF with Elixir

To use the NIF in Elixir, we need to follow several steps:

- Load the shared library containing the NIF.
- Define a fallback function to handle cases where the NIF is not loaded.
- Call the NIF function from our Elixir code.

#### Loading the Shared Library

When using NIFs, Elixir needs to load the shared library containing the native code. To do this, we use the `:erlang.load_nif/2` function, which takes two arguments:

- The relative path to the shared library (`.so` on Linux, `.dll` on Windows, or `.dylib` on macOS).
- An optional integer value (usually 0) can be used for versioning purposes.

In our Elixir module, we'll define a `load_nif/0` private function that handles loading the shared library. We'll also use the `@on_load` module attribute to specify that `load_nif/0` should be called when the module is loaded.

#### Defining a Fallback Function

It's important to provide a fallback function that will be called if the NIF fails to load. This function should have the same name and arity as the NIF function and return an error tuple such as `{:error, reason}` or call the `:erlang.nif_error/1` BIF (Built-In Function) with an appropriate error reason.

In our example, we define an `add/2` fallback function that calls `:erlang.nif_error`(`:nif_not_loaded`).

#### Calling the NIF Function from Elixir Code

Once the shared library is loaded and the NIF function is linked to the Elixir module, you can call the NIF function just like any other Elixir function. In our example, we call `ExampleNif.add/2` to perform the addition using the NIF we wrote in Zig.

By following these steps, you can seamlessly integrate NIFs written in Zig into your Elixir applications, taking advantage of both the performance benefits of Zig and the maintainability and concurrency features of Elixir.

### Writing Elixir Code to Use the NIF

Create a new Elixir module called `ExampleNif`:

```elixir
defmodule ExampleNif do
  @on_load :load_nif

  def load_nif do
    :erlang.load_nif('./build/libexample_nif', 0)
  end

  def nif_add(_a, _b), do: :erlang.nif_error(:nif_not_loaded)

end
```

Let's review this. We:

- Create an Elixir module called `ExampleNif`.
- Specify that the `load_nif/0` function should be called when the module is loaded.
- Define a private function, `load_nif/0`, that loads the shared library containing the NIF.
- Define a fallback function (`nif_add/2`) that will be called if the NIF fails to load.

**Note:** _`:erlang.load_nif` loads and links the Zig compiled library to the Elixir module; and it also loads this function before the definition of our `nif_add/2` fallback function. This is why we need to define the fallback function `nif_add/2` with the same arity and name as the NIF function._

Because of Elixir's pattern-matching capabilities, the fallback function will handle calls to `nif_add/2` if the NIF is not loaded, and if the NIF is loaded, the NIF function will handle the call as defined in `load_nif/0`.

### Verifying That the NIF Works

To verify that the NIF is working as expected, create a simple test program in Elixir:

```elixir
defmodule ExampleNifTest do
  use ExUnit.Case

  test "add/2 NIF" do
    assert ExampleNif.nif_add(1, 2) == 3
    assert ExampleNif.nif_add(-5, 10) == 5
  end
end
```

Run the test suite using mix test. If the tests pass, it indicates that the NIF is functioning correctly, and our addition is done through our Zig compiled library rather than on Elixir code directly.

### Considerations

It is amazing that we can use code from another language through NIF. However, there are a few things you might want to be aware of, starting with the parameters we initially declared on our Zig library:

```zig
export fn nif_add(
    env: ?*erl.ErlNifEnv,
    argc: c_int,
    argv: [*c]const erl.ERL_NIF_TERM,
) erl.ERL_NIF_TERM {
    var a: i32 = undefined;
    var b: i32 = undefined;
...
    _ = erl.enif_get_int(env, argv[0], &a);
    _ = erl.enif_get_int(env, argv[1], &b);
```

We are parsing the first two arguments and casting them into an integer. What happens if we pass a float value to this function? Try it out:

```bash
iex(3)> ExampleNif.nif_add(-100.0,-11)
-1431655777
```

We get junk values, as there are no type checks or logic to handle float values in our Zig function. These can cause unexpected bugs and issues that developers need to be more vigilant to catch.

## Wrapping Up

In this post, we set up Zig for Elixir NIF development, wrote a simple NIF in Zig, and integrated the NIF with an Elixir application.

By leveraging the strengths of both Elixir and Zig, developers can create efficient, maintainable, and performant applications.

That said, NIFs should be used with caution and deliberately. Ensure you add the necessary tests and failsafes.

Happy coding!

**P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, [subscribe to our Elixir Alchemy newsletter and never miss a single post](/elixir-alchemy)!**