Amalgamating Nim programs
Tags: nim, c, amalgamation
Published: 2021-03-19 17:03:45

---
title: Amalgamating Nim programs
published: true
description: "Making a single C file out of a Nim program."
tags: [nim, c, amalgamation]
canonical_url: https://zen.su/posts/amalgamating-nim-programs/
---

## Intro

Over the last few years, there were a couple of threads on the Nim forum about the possibility of making a single self-contained (more-or-less) C file from a Nim program. Most of the time, those threads were about participating in a competition that doesn't have Nim or writing a C assignment in Nim.

Today I finally decided to try to do this myself and, to my surprise, succeeded.

## Starting out

Amalgamation is a technique that makes it possible to combine the whole project (usually in C or C++) into a few source files (usually one). It makes distributing libraries easier in some cases, or can even improve the performance, although usually, you can achieve similar results by enabling [Link-Time Optimization](https://en.wikipedia.org/wiki/Interprocedural_optimization).

The most popular library out there that can be amalgamated is probably [SQLite](https://www.sqlite.org/amalgamation.html).
In SQLite's case, this is achieved by carefully separating code into different files that still works when combined in a single file.

It is certainly possible to adjust the Nim's C backend to output C files that can be combined into a single one and compiled, but this requires certain effort. So I didn't go this route and started searching for a program that can amalgamate separate C files by itself, and found one - [C Intermediate Language](https://github.com/cil-project/cil).

The project is aimed at creating a simplified subset of C, but it has a feature [called "merger"](https://people.eecs.berkeley.edu/~necula/cil/merger.html) that we'll be using.

### Building CIL

First of all, we need to compile CIL itself. I chose to use [a maintained fork](https://github.com/goblint/cil) instead of the original so that it's more likely that it works :)

For building CIL, I just followed the [installation section in the README](https://github.com/goblint/cil#installation-for-the-version-in-this-repository). Here's the listing of what I did (might or might not work for you):

```sh
# Install OCaml on Arch
$ sudo pacman -Sy ocaml

# Clone the repo
$ git clone https://github.com/goblint/cil && cd cil

# Create a local opam environment ("switch"). This will take a while
$ opam switch create . 

# Add local opam environment to the current shell
# This command is specific to the fish shell. For others, you might need to modify it
$ eval (opam env)
# Configure the compilation, the prefix is the _opam directory
# in CWD (because we use `opam switch`)
# For POSIX-shells replace () with ``
$ ./configure --prefix=(opam config var prefix)

# Build CIL
$ make
# This make install fails for me, but it's needed for commands below
$ make install
# To fix the error in the previous step we need to remove the logwrites dir
$ rm -r _opam/lib/goblint-cli/logwrites/
# Finally install it into currentdir/_opam/bin/
$ make install

# Add it to our PATH (your directory will of course be different)
$ set PATH /home/dian/Stuff/cil/_opam/bin $PATH
```

### Patching nimbase.h

Now we're *almost* ready to use it for Nim programs! There's one thing left - we need to patch [nimbase.h](https://github.com/nim-lang/Nim/blob/devel/lib/nimbase.h) (it's a file that provides some generic defines for different compilers on different platforms). You can usually find nimbase.h in `your_nim_dist/lib/nimbase.h`.

```diff
diff --git a/lib/nimbase.h b/lib/nimbase.h
index cbd35605b..7d3620881 100644
--- a/lib/nimbase.h
+++ b/lib/nimbase.h
@@ -75,9 +75,9 @@ __AVR__
 #endif
 /* ------------------------------------------------------------------------- */
 
-#if defined(__GNUC__)
-#  define _GNU_SOURCE 1
-#endif
+//#if defined(__GNUC__)
+//#  define _GNU_SOURCE 1
+//#endif
 
 #if defined(__TINYC__)
 /*#  define __GNUC__ 3
@@ -195,7 +195,7 @@ __AVR__
 #  define N_LIB_EXPORT_VAR  __declspec(dllexport)
 #  define N_LIB_IMPORT  extern __declspec(dllimport)
 #else
-#  define N_LIB_PRIVATE __attribute__((visibility("hidden")))
+#  define N_LIB_PRIVATE
 #  if defined(__GNUC__)
 #    define N_CDECL(rettype, name) rettype name
 #    define N_STDCALL(rettype, name) rettype name
@@ -324,7 +324,7 @@ namespace USE_NIM_NAMESPACE {
 typedef unsigned char NIM_BOOL; // best effort
 #endif
 
-NIM_STATIC_ASSERT(sizeof(NIM_BOOL) == 1, ""); // check whether really needed
+//NIM_STATIC_ASSERT(sizeof(NIM_BOOL) == 1, ""); // check whether really needed
 
 #define NIM_TRUE true
 #define NIM_FALSE false
@@ -543,7 +543,7 @@ static inline void GCGuard (void *ptr) { asm volatile ("" :: "X" (ptr)); }
 #endif
 
 // Test to see if Nim and the C compiler agree on the size of a pointer.
-NIM_STATIC_ASSERT(sizeof(NI) == sizeof(void*) && NIM_INTBITS == sizeof(NI)*8, "");
+//NIM_STATIC_ASSERT(sizeof(NI) == sizeof(void*) && NIM_INTBITS == sizeof(NI)*8, "");
 
 #ifdef USE_NIM_NAMESPACE
 }
```

The main change here is that we don't want to define `_GNU_SOURCE` because CIL doesn't support most GNU-specific constructs. It also doesn't support the `NIM_STATIC_ASSERT` macro so we disable that as well, along with `N_LIB_PRIVATE`.

You can apply the patch with a simple `git apply /path/to/nimbase.diff` while in the Nim distribution directory.

## Hello, Amalgamation!
Now that we have all the necessary things prepared, we can test out our amalgamation setup.

First of all, since Nim expects a compiler binary to be an executable file, we need to create `cilly.sh` that will call `cilly` (CIL's main tool) with all needed arguments:
```sh
#!/usr/bin/env sh
cilly --noPrintLn --merge --keepmerged $@
```

- `--noPrintLn` specifies that we don't want #line directives.
- `--merge` for actually merging the C files.
- `--keepmerged` to keep the amalgamated C file after compilation.

Save it somewhere in your `$PATH`.


Now let's create our first amalgamated Nim program:

```nim
# hello.nim
echo "Hello, Amalgamation!"
```

We also need to specify that we want to use `cilly.sh` as our C compiler, so let's go ahead and create the configuration for our Nim file:
```nim
# hello.nim.cfg
gcc.exe="cilly.sh"
gcc.linkerexe="cilly.sh"
```

We could've provided those on the CLI, but it's better to create a separate config file.

And let's **finally** compile our Nim program and create an amalgamation:
```
nim c -d:danger hello.nim
```

There might be some warnings, but that's fine. After the compilation we'll have a new file called `hello_comb.c` in our directory, and that's the amalgamation that we wanted to get!

Now, if you check the line count of the file, you might be surprised - it's 6.6K lines long (at the time of writing).

Why is that? The main reason is that Nim treats C as a *backend* and makes many of its own types and functions.
That line count is not big - CIL merges **all** C files of your Nim program and system includes too, so the final size is relatively small. To improve it even more, you might want to consider using some of these options:

- `--gc:orc` - use [Nim's new GC](https://nim-lang.org/blog/2020/10/15/introduction-to-arc-orc-in-nim.html), it has a much smaller runtime. If you're sure that your program doesn't have cycles, use `--gc:arc` for an even smaller line count. 
- `--d:useMalloc` - generally not preferred, but this will make the Nim compiler use C's memory functions instead
of using its own allocator. 

After adding those two options, the resulting line count is down to 3.4K lines of code. You can check out the resulting C file [in this gist](https://gist.github.com/Yardanico/6fd4623d5e8853735dfa79ddb7f9b516). 

## Afterword
While this is an interesting matter, I think that amalgamations are rarely useful. It's really hard to make them portable, and they can lead to more weird bugs. That said, you can use them if you **really, really** need to have a single C file.

Some relevant discussions:
- [compile a nim single file to a single c file that only depends standard c header and lib](https://forum.nim-lang.org/t/7663)
- [Compile to a single C file?](https://forum.nim-lang.org/t/7300)
- [Generate a single self-contained C source file](https://github.com/nim-lang/Nim/issues/5056)

Thanks for reading!