Language Interoperability From the Ground Up
Tags: programming, linux, coding
Published: 2019-01-14 13:26:12

---
title: Language Interoperability From the Ground Up
published: true
tags: [programming, linux, coding]
canonical_url: http://samwho.co.uk/blog/2018/04/29/language-interoperability-from-the-ground-up/
---

How does a function in Ruby call a function in C? How does a function in Kotlin call a function in Java?

It’s no secret that you can call functions in one programming language from another. Not only is it possible, it can be done in multiple ways. But how does it _actually work_?

This post is going to be a bottom-up, in-depth look at interoperability. Each section will build upon the previous until we have an understanding of the mechanisms that make language interoperability possible.

# Who should read this post?

If you have been programming for a little while and have heard people talking about whether a library has “Java bindings” or “Ruby bindings.”

If you have been using a language like Kotlin for a while and called Java functions from Kotlin and wondered how on earth that works under the hood.

If you have a keen interest in the gory details of complicated systems, but want an explanation that doesn’t assume much and keeps the examples as simple and hands-on as possible.

Additionally, you will need to have the following programs installed on your computer and accessible from the command line if you would like to follow along with the examples: `nm`, `gcc`, `nasm`, `gdb`, `javac` `javap`, `kotlinc`,`kotlin`, `ruby`. All of the examples in this post were written and run on an Arch Linux installation, they’re not likely to work as shown here when run on a Mac or Windows machine.

# Why is language interoperability important?

The majority of language interoperability involves higher level languages calling libraries written in lower level languages. A Java library that offers you the ability to call in to the native OpenSSL libraries, for example. Or a Rust library that provides a more idiomatic Rust API in to the cURL library written in C.

Duplication is bad. When you’ve written a complicated library in one language, reimplementing it in another is just one more thing to forget to maintain.

Additionally, it’s a good idea for young languages to be able to make the most of existing work. If you create a new language and include a mechanism for using existing native libraries, you can immediately draw upon decades of hard work.

## The benefits of purity

Sometimes you might see a library advertised as a “pure Ruby” implementation, or a “pure Java” implementation of some other library. These libraries will be full reimplementations of their target technology.

For example, [this](https://github.com/dain/leveldb) is a pure Java implementation of[LevelDB](https://github.com/google/leveldb). This would have been a lot of work for the author, but the advantage is that people will be able to use LevelDB from Java without having to install the native LevelDB library on their system and package that up with their Java code for deployment.

While a pure reimplementation can be a lot of up-front effort and maintenance, they can be easier to use.

# Why start at the bottom and work up?

The key to interoperability is finding common ground. With computers, the common ground is a set of standards and conventions in the lower levels of how programs work that allow languages to speak to each other. Understanding these conventions is key to understanding when languages can interoperate, and when they can’t.

# Rock bottom: assembly language

```nasm
mov eax, 10
```

That there is a line of “assembly code.” It is a single “instruction,” and it consists of a “mnemonic”, `mov`, and two “operands,” `eax` and `10`.

This is the “move” instruction, and it instructs our CPU to move the value `10`in to the register `eax`. It is part of an instruction set called “x86,” which is used in 32-bit Intel and AMD CPUs[^1].

## What is a register?

Registers are small but fast bits of storage connected directly to your CPU. If you want to do something to some data, be it addition, subtraction, or some other operation, the data needs to first be loaded in to a register.

If you’re not used to writing code at this level, the concept of registers might seem silly. Why can’t we just store 10 in memory and operate on it there?

Because that isn’t physically possible. The CPU isn’t connected to memory directly. It is connected through a series of caches, and all memory access must go through the Memory Management Unit, or MMU. The MMU can’t process any operations, this happens in the Arithmetic Logic Unit, or ALU[^2], and the ALU is also directly connected to the CPU. It can only get at data if it is in a register.

![The MMU and friends](http://samwho.co.uk/images/mmu-and-friends.svg)

This is not to scale. In reality, there are around a kilobyte of registers, a few hundred kilobytes of L1 cache, a megabyte or two of L2 cache, low tens of megabytes of L3 cache, and then memory is often tens of gigabytes.

## Program flow

Doing mathematical operations is great and all, but for us to write code that does anything we need to be able to compare things and make decisions based on the outcome.

```nasm
mov     eax, 10
sub     eax, 5
mov     ebx, 5
test    eax, ebx
je      equal

notequal:
jmp     exit

equal:
; do something here

exit:
; end of program
```

We start this snippet with some moves and a subtract (`sub`). We introduce a new register, `ebx`. Then we do a `test`. The `test` instruction compares two values, they can either both be registers or one of them could be an “immediate” value, e.g. 10. The result of the `test` is stored in a special “flags” register that we don’t touch directly. Instead, we use a family of “jump” instructions that read the flags register and decide which instruction to run next.

The `je` instruction will jump to a point in our code, denoted by the “label”`equal`, if the result of the test was that both values are equal. Otherwise, the code falls through to the next assembly instruction, which will be whatever we decide to put below our `notequal` label. For now, we just do a `jmp`, which is an unconditional jump, to the end of our program at `exit`.

“Labels” are a new concept in this snippet. They aren’t instructions and the CPU doesn’t attempt to run them. Instead, they’re used to mark points in the code that can be jumped to, which is what we’ve used them for here. You can spot labels by looking for the trailing colon.

## Accessing memory

So far so good, but we’re only touching registers at the moment. Eventually we will run out of space in registers and have to fall back to using memory. How does that work in assembly?

```nasm
mov         eax, 0x7f00000
mov dword [eax], 123
```

The first bit should be familiar, we’re storing the hexidecimal value `0x7f00000`in to `eax`, but then we do something funky with the next instruction.

The square brackets around `[eax]` mean that we want to store a value in the memory address inside of `eax`. The `dword` keyword signifies that we want to store a 4-byte value (“double word”). We need the `dword` keyword in there because without it there’s no way to infer how large the value `123` should be in memory.

If you’re familiar with pointers in C, this is essentially how pointers work. You store the memory address you want to point to in a register and then access that memory by wrapping the register name in square brackets.

We can take this a little further:

```nasm
mov             eax, 0x7f00000
mov dword [eax + 1], 123
```

This will store the value `123` at address `0x7f00001`, one byte higher than before. The value of `eax` isn’t modified by doing this, it’s just letting us access the value at a register plus an offset. This is commonly seen in real-world assembly code, and we’ll see why later on.

## The stack

Fundamentals out of the way, this is our first recognisable concept from higher level languages. The stack is where languages typically store short-lived variables, but how does it work?

When your program starts, the value of a special register called `esp` is set to a memory location that represents the top of the stack space you are allowed to access, which is itself near the top of your program’s “address space.”

You can examine this phenomenon by writing a simple C program and running it in a debugger:

```c
int main() {
	return 42;
}
```

Compile with `gcc` and run using `gdb`, the “GNU Debugger”:

```
$ gcc main.c -o main -m32
$ gdb main
gdb> start
gdb> info register esp
esp	0xffffd468	0xffffd468
```

When we say `start` in the GDB prompt, we’re asking it to load our program in to memory, start running it, but pause it as soon as it starts. Then we inspect the contents of `esp` with `info register esp`.

### A brief detour into memory organisation

Why is the stack near the top of the address space? What even is an address space?

When your program gets loaded in to memory on a 32-bit system, it is given its own address space that is 4GB in size: 0x00000000 to 0xffffffff. This happens no matter how much physical RAM your machine has installed in it. Mapping this “virtual” address space is another one of the jobs that the MMU performs, and the details of it are beyond the scope of this post.

![Memory layout visualised](http://samwho.co.uk/images/memory-layout.svg)

This simplified view of memory shows that our program is loaded somewhere fairly low down in the address space, our stack is up near the top and grows down, then we have this mysterious place called the “heap” that lives just above our program. It’s not important for language interoperability, but the heap stores long-lived data, for example things you’ve allocated using`malloc()` in C or `new` in C++.

This allows for a fairly efficient use of the available space. In reality, the stack is limited in size to a handful of megabytes and exceeding that will cause your program to crash. The heap can grow all the way up to the base of the stack but no further. If it ever attempts to overlap with the stack, your`malloc()` calls will start to fail, which causes many programs to crash.

### Back to the stack

The x86 instruction set gives us some instructions for adding and removing values from the stack.

```nasm
push    1
push    2
push    3
pop     eax
pop     ebx
pop     ecx
```

At the end of this sequence of instructions, the value of `eax` will be 3,`ebx` will be 2 and `ecx` will be 1. Don’t trust me? We can verify this for ourselves with a couple of small modifications.

```nasm
global main
main:
        push    1
        push    2
        push    3
        pop     eax
        pop     ebx
        pop     ecx
        ret
```

Save this in to a file called `stack.s` and run the following commands:

```
$ nasm -f elf stack.s
$ gcc -m32 stack.o -o stack
```

If you don’t have `nasm` you’ll need to install it. It’s a type of program called an “assembler” and it can take assembly instructions and compile them down to machine code.

We now have an executable file in our working directory called “stack”. It’s a bonafide program you can run like any other program. The only modifications we had to make to it were giving it a `main` label, and making sure it correctly returns control back to the operating system with the `ret` instruction. We’ll explore `ret` in more detail later.

Running this program doesn’t really do anything. It will run, but exit silently[^3]. To see what’s going on inside of it, we will once again need a debugger.

```
$ gdb stack
gdb> break *&main+9
gdb> run
gdb> info registers eax ebx ecx
```

The output of this sequence of commands should verify what I said earlier about the contents of those registers. `gdb` allows us to load up a program, run it until a certain point (`break *&main+9` is us telling `gdb` to stop just before the `ret` instruction) and then examine the program state.

## So what do `push` and `pop` actually do?

The `push` and `pop` instructions are shorthand and can be expanded to the following:

```nasm
; push
sub     esp, 4
mov     [esp], operand

; pop
mov     operand, [esp]
add     esp, 4
```

All of which should be familiar to you from previous sections, and neatly demonstrates how the stack grown downwards and shrinks upwards as things are added and removed.

## Functions in assembly

Language interoperability is, at its most fundamental, the ability to call a function written in one language from a different language. So how do we define and call functions in assembly?

With the knowledge we have so far, you might be tempted to think that function calls look like this:

```nasm
main:
        jmp myfunction

myfunction:
        ; do things here
```

A label to denote where your function is in memory, and a jump instruction to call it. This approach has two critical problems: it doesn’t handle passing arguments to or returning values from the function and it doesn’t handle returning control to the caller when your function ends.

We could solve the first problem by putting arguments and return values on the stack:

```nasm
main:
        push    1
        push    2
        jmp     add
        pop     eax

add:
        pop     eax
        pop     ebx
        add     eax, ebx
        push    eax
```

This would work really well if only our `add` function were able to jump back to the caller when it was finished. At the moment, when `add` ends, the program ends. In an ideal world it would return back to just after the `jmp` in `main`.

What if we saved where we were when we called the function and jumped back to that location when the function was finished?

The `eip` register holds the location of the currently executing instruction. Using this knowledge, we could do this:

```nasm
main:
        push    1
        push    2
        push    eip
        jmp     add
        pop     eax

add:
        pop     edx	; store the return address for later
        pop     eax	; 2
        pop     ebx	; 1
        add     eax, ebx
        push    eax
        mov     eip, edx
```

We’re getting there. This approach has a couple of problems, though: we modify`esp` a lot more than we really have to and x86 doesn’t let you `mov` things in to `eip`.

Here’s what a compiler would _actually_ generate for our example above:

```nasm
main:
        push    ebp
        mov     ebp, esp
        push    2
        push    1
        call    add
        add     esp, 8
        pop     ebp
        ret

add:
        push    ebp
        mov     ebp, esp
        mov     edx, dword [ebp + 8]
        mov     eax, dword [ebp + 12]
        add     eax, edx
        pop     ebp
        ret
```

This is a lot to take in, so let’s go through it line by line.

We’re introducing a new special register: `ebp`. This is the “base pointer” register, and its purpose is to act as a pointer to the top of the stack at the moment a function is called. Every function starts with saving the old value of the base pointer on the stack, and then moving the new top of the stack in to`ebp`.

Next we do the familiar pushing of arguments on to the stack. At least we got that right. Then we use an instruction we haven’t seen before called `call`. `call` can be expanded to the following:

```nasm
; call
push    eip + 2
jmp     operand
```

With `eip + 2` meaning the instruction _after_ the `jmp` below it. It doesn’t matter what the value is, just think of it as pushing the address of the instruction after the `call` instruction on to the stack so we can refer to it later.

Then control is passed to `add`, which follows a similar pattern. The base pointer is saved, the stack pointer becomes the new base pointer, and then we get to see the base pointer in action.

```nasm
mov     edx, dword [ebp + 8]
mov     eax, dword [ebp + 12]
```

This code is pulling the two arguments to `add` off of the stack and in to registers so that we can operate on them. But why are they 8 bytes and 12 bytes away?

Remember we pushed the arguments on to the stack, and then `call` pushed the address to return to. This means that the first 4 bytes of stack are a return address, then then 8 bytes after that are our arguments. To get to the first argument, you need to move 8 bytes up from the stack pointer (because it grows downwards), and to get to the second argument you need to move 12 bytes up.

![Stack frame](http://samwho.co.uk/images/stack-frame.svg)

This has the added benefit of not requiring us to modify `esp` with every single`push` and `pop` instruction. It’s a small saving, but when you consider that this has to happen for every argument to every function called, it adds up.

When we’ve done what we need to do in our `add` function, we perform the steps we did at the start but in reverse order.

```nasm
pop     ebp
ret
```

The `ret` instruction is special because it allows us to set the value of `eip`. It pops the return value that `call` pushed for us and jumps to it, returning control of the program to the calling function.

The same thing happens in `main`, except there’s a subtle difference. The `add
esp, 8` is necessary to “free” the arguments to `add` we pushed on to the stack. If we don’t do this, the `pop ebp` will not correctly restore the base pointer and we’ll likely try to refer to memory we never intended to, crashing our program.

Lastly, you’ll notice that `add` doesn’t push its result back on to the stack when it’s done. It leaves the result in `eax`. This is because it’s conventional for a function’s return value to be stored in `eax`.

# Conventions

We’ve just done the deepest possible dive on how function calls work in x86, now let’s put names to each of the things we have learnt.

Saving the base pointer and moving the stack pointer prior to calling a function is called the **function prologue**.

Restoring the stack pointing and base pointer after a function call is called the **function epilogue**.

Those two concepts, along with returning values in `eax` and storing your function arguments on the stack make up what’s called a **calling convention** , and calling conventions are part of a larger concept known as an **application binary interface** , or **ABI**. Specifically, all of what I have described so far it part of the **System V ABI** , which is used by almost all Unix and Unix-like operating systems.

Before we start calling functions written in one language from functions written in another, there’s one last thing we need to be aware of.

# Object files

When you compile a C program, quite a lot of things happen under the hood. The most important concept to understand for language interoperability is “object files.”

If your program consists of 2 `.c` files, the first thing a compiler does is compile each of those `.c` files in to a `.o` file. There is typically a 1-to-1 mapping between`.c` files and `.o` files. The same is true of assembly, or `.s` files.

Object files, on Linux and lots of other Unix-like operating systems, are in a binary format called the “executable and linkable format,” or ELF for short. If you’re interested in the full range of data found inside of an ELF file, you can use the `readelf -a <elffile>` command to find out. The output is likely to be quite dizzying, though, and we’re only really interested in one of its features here.

## Symbol tables

To explain symbol tables, let’s split out our assembly from earlier in to two`.s` files:

`add.s`:

```nasm
add:
        push    ebp
        mov     ebp, esp
        mov     edx, dword [ebp + 8]
        mov     eax, dword [ebp + 12]
        add     eax, edx
        pop     ebp
        ret
```

`main.s`:

```nasm
main:
	push    ebp
	mov     ebp, esp
	push    2
	push    1
	call    add
	add     esp, 8
	pop     ebp
	ret
```

Assemble both of the files:

```
$ nasm -f elf add.s
$ nasm -f elf main.s
main.s:6: error symbol `add' undefined
```

Whoops. Our assembler isn’t happy about us calling an undefined function. This is sticky, because we want to define that function elsewhere and call it in `main.s`, but it seems here like the assembler doesn’t allow that.

The problem is that there is no `add` symbol defined in this file. If we want to tell the assembler that we intend to find this symbol in another file, we need to say so. Add this line to the top of `main.s`:

```nasm
extern add
```

And now it should assemble without complaint. Before we go further, have a look in your working directory. You should have two `.o` files: `main.o` and `add.o`. We can look at the contents of their symbol tables with a tool called `nm`:

```
$ nm add.o
00000000 t add

$ nm main.o
         U add
00000000 t main
```

The first column is the address of the symbol, the second column is the type of the symbol, and the third column is the name of the symbol. Notice that the symbol names match up with our label names. Also note that `main.o` has an`add` symbol of type `U`. `U` means “undefined”, which means we need to find a definition for it when we link these object files together later.

Both of our defined functions have a symbol type of `t`. This means that the symbol points to some code.

To create an executable out of these object files, we run:

```
$ gcc -m32 main.o add.o -o main
```

This will shout at you, claiming that it cannot find either `main` or `add`. What gives?

Unless we explicitly say so, the symbols in an object file cannot be used by other object files when the compiler links them together. To fix this, we need to add:

```nasm
global add
```

To the top of `add.s` and:

```nasm
global main
```

To the top of `main.s`. This allows the symbols to be linked and the result is that the compiler now takes our object files and creates an executable out of them without complaint.

```
$ nm main
```

This will produce a lot of output now, because the compiler has to link in a lot of administrative stuff, like the libc constructor and destructor handlers,` __libc_csu_init` and `__ libc_csu_fini` respectively. Don’t worry about them, the important thing is that both `main` and `add` are defined and the program runs without complaint.

![Assemble and link diagram](http://samwho.co.uk/images/assemble-link.svg)

# Calling a C function from C++

Let’s go up a level and look at some C and C++.

`main.cpp`:

```cpp
extern int inc(int i);

int main() {
	return inc(2);
}
```

The first line here is us telling the compiler to expect to find a function called`inc` in another file.

`inc.c`:

```c
int inc(int i) {
	return i + 1;
}
```

Here’s the set of steps we need to follow to compile them both separately and then link them together:

```
$ gcc -c main.cpp -o main.o
$ gcc -c inc.c -o inc.o
$ gcc inc.o main.o
```

Unfortunately, this fails with a seemingly unfathomable error:

```
main.o: In function `main'
main.cpp:(.text+0xa): undefined reference to `inc(int)'
collect2: error: ld return 1 exit status
```

But we supplied an object file with a definition of `inc(int)` in it, we made sure to tell the compiler to expect to find a function called `inc(int)`, why can’t it find it?

## Name mangling

Sadly C++ wasn’t able to provide all of the features it wanted to without diverging from the System V ABI a little bit. When you compile a function in C, that function gets given a symbol with the name you gave it so that others can call it by that name.

C++ does not do this by default. As well as the name you give it, C++ also tacks on information about the return type and argument types of that function. If the function is in a class, information about the class is also included. It does this to allow you to overload the name of a function, so you can define multiple variations of a function that takes different arguments. This is called **name mangling**.

As a result, when we compiled our `main.cpp` file, it was told to look for a function called `_Z3inci` instead of plain old `inc`. Our `inc.c` file provides a function called `inc`, and as such the two languages cannot interoperate without a little bit of help.

Fortunately, the problem is easily solved by adding 4 characters to our `main.cpp`:

```cpp
extern "C" int inc(int i);
```

This addition of `"C"` tells C++ to compile this file in search of a function with plain old C-style calling conventions, and this includes using the plain old C name of `inc`. Attempting to compile this code should now work as expected.

## Calling a C++ function from C

This relationship works similarly in the other direction. If we want to write a function in C++ but expose it in a way that a C program could call it, we would need to use `extern "C"` on that function:

```cpp
extern "C" int inc(int i) {
	return i + 1;
}
```

# What about Java?

What we’ve discussed up until now is the fundamental basis for how language interoperability works between _native_ languages, but what about a language that runs on a virtual machine, such as Java?

The process is a little more involved, but doable. The problem arises because Java runs on a thing called the Java Virtual Machine, or JVM, which acts as a layer of indirection between Java code and the machine on which it runs. Because of this, Java cannot link directly to native libraries in the same way that C and C++ can. We have to introduce a layer that translates between the native world and the JVM world.

Fortunately, the people behind Java gave this a lot of thought and they came up with the “Java Native Interface,” or JNI. It’s the accepted way to get Java code to talk to native code, and here’s how it works:

```java
public class Interop {
  static {
    System.loadLibrary("inc");
  }

  public static native int inc(int i);

  public static void main(String... args) {
    System.out.println(inc(2));
  }
}
```

Notice the use of the `native` keyword. This tells Java that the implementation for this function is defined in some native code somewhere. The`System.loadLibrary("inc")` line will search Java’s library path for a library called `libinc.so` and, when it finds it, we will be able to use the Java function `inc` to call our native code!

But how do we do that?

Step 1: Generate the JNI header file from our code.

```
$ javac -h . Interop.java
```

The `-h .` tells `javac` to generate a file called `Interop.h` in the current directory. This will define the function we have to to implement. The resulting file looks like this:

```c
/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class Interop */

#ifndef _Included_Interop
#define _Included_Interop
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class: Interop
 * Method: inc
 * Signature: (I)I
 */
JNIEXPORT jint JNICALL Java_Interop_inc
  (JNIEnv *, jclass, jint);

#ifdef __cplusplus
}
#endif
#endif
```

This looks a lot scarier than it is. The lines containing `_Included_Interop`are just “header guards” that make sure we can’t accidentally include this file twice and the `__cplusplus` bit checks if we’re compiling as C++ code and, if we are, wraps everything in an `extern "C"` block, which you’ll remember from earlier in this post.

The rest is the definition of the JNI function we have to implement:

```c
JNIEXPORT jint JNICALL Java_Interop_inc
  (JNIEnv *, jclass, jint);
```

It might not look like one, but this is indeed a function declaration. We implement it like so:

`inc-jni.c`:

```c
#include <jni.h>
#include "Interop.h"

JNIEXPORT jint JNICALL Java_Interop_inc
  (JNIEnv* env, jclass class, jint i) {
	return (jint)(i + 1);
}
```

The `jint` cast is to make sure our integer is the size that Java is expecting it to be. The `jni.h` include is required for all of the Java-specific things we’re seeing, such as `jint` and `jclass`.

To compile this we need to execute a pretty gnarly `gcc` call:

```
$ gcc -I"$JAVA_HOME\include" -I"$JAVA_HOME\include\linux" -fPIC -shared -o libinc.so inc-jni.c
```

The `-I` flags tell `gcc` where to find header files, which we need for the`#include <jni.h>` lines to work. The `-fPIC -shared` flags create a special type of object file called a “shared object.” What’s special about this type of object file is that it’s possible to, instead of compiling directly against it, load it in to your process at runtime. Shared object files are, by convention, named `lib<something>.so`.

Now we can run the Java code like so:

```
$ java -Djava.library.path=. Interop
3
```

Voila! We successfully called our native `inc` implementation from Java! How cool is that?

# What about Ruby?

Ruby has a similar approach to Java. It exposes an API called “Ruby native extensions” that you can hook in to to call native functions in Ruby. Given that we have explored Java’s way of doing this, and Ruby’s is not too dissimilar, I want to use Ruby to focus on a different and more convenient way of calling native code.

## ffi

`ffi` stands for “foreign function interface,” and it’s a Ruby gem that allows us to call functions in existing shared object files with very little setup. First, we install the ffi gem:

```
$ gem install ffi
```

Then we need to compile our original `inc.c` file in to a shared object:

```
$ gcc -fPIC -shared -o libinc.so inc.c
```

And then we can write the following short snippet of Ruby code and call our`inc` function:

```ruby
require 'ffi'

module Native
  extend FFI::Library
  ffi_lib './libinc.so'
  attach_function :inc, [:int], :int
end

puts Native.inc(2)
```

This method makes us jump through far fewer hoops than Java’s JNI does, and has the benefit of allowing us to call existing shared objects without modifying them. For example, you can call directly in to `libc.so`:

```ruby
require 'ffi'

module Libc
  extend FFI::Library
  ffi_lib 'c'
  attach_function :puts, [:string], :int
end

Libc.puts "Hello, libc!"
```

Notice that we only had to specify `'c'` as the library name. This is because Unix-like systems have standardised paths to find libraries in, often including`/lib` and `/usr/lib`. By default `ffi` will look for libraries with the naming scheme `lib<name>.so`. If you do `ls /usr/lib/libc.so` you should find a file exists at that path.

# Why doesn’t Java have FFI?

It does! There’s a library called [JNA](https://github.com/java-native-access/jna/blob/master/www/GettingStarted.md) that does the same job that Ruby’s FFI library does.

```ruby
import com.sun.jna.Library;
import com.sun.jna.Native;

class JNA {
  public interface Libc extends Library {
    Libc INSTANCE = (Libc)Native.loadLibrary("c", Libc.class);
    public int puts(String s);
  }

  public static void main(String... args) {
    Libc.INSTANCE.puts("Hello, libc!");
  }
}
```

We have to download the JNA library, which you can do so from [here](https://mvnrepository.com/artifact/net.java.dev.jna/jna/4.1.0). Then we compile and run the Java code including the JNA library:

```
$ javac -classpath jna-4.1.0.jar JNA.java
$ java -classpath jna-4.1.0.jar:. JNA
Hello, libc!
```

## Why would anyone use a language’s native interface if FFI is so much more convenient?

Lots of languages have some form of FFI library, and they’re very convenient for calling in to existing native libraries. The problem, though, is that it’s one-way communication. The library you’re calling in to can’t modify, for example, a Java object directly. It can’t call Java code. The only way to do that is to use Java’s JNI or Ruby’s native extensions, because they expose to you an API for doing that.

If you don’t need to have two-way communication between languages, though, and all you want to do is call existing native code, FFI is the way to go.

# Hold up, so how does Kotlin call functions written in Java?

Kotlin is a relatively new language that runs on the JVM.

_“Wait, doesn’t Java run on the JVM?”_

That’s true! But the JVM isn’t _just_ for Java, it’s a generic virtual machine that compilers can generate “machine code” for just like native machines. The JVM machine code is more commonly referred to as “bytecode,” because every instruction is one byte in length. Yes, this limits the instruction set to 256 instructions. Far fewer than the [2,034](https://stefanheule.com/blog/how-many-x86-64-instructions-are-there-anyway/) instructions you’ll find in x86.

In this respect, you could consider the JVM to be a “native” machine. That is the abstraction it aims to present to compilers and users. The only difference is that the native machine you’re running code on is being emulated by a piece of software called the JVM.

Let’s look at some example Java and Kotlin code.

`Hello.java`:

```java
public class Hello {
  public static void world() {
    System.out.println("Hello, world!");
  }
}
```

`Main.kt`:

```kotlin
fun main(args: Array<String>) {
  Hello.world()
}
```

Let’s compile and run this code.

```
$ javac Hello.java
$ kotlinc -classpath . Main.kt
$ kotlin MainKt
Hello, world!
```

And now we disassemble the code to see what’s going on under the hood:

```
$ javap -c MainKt.class
Compiled from "Main.kt"
public final class MainKt {
  public static final void main(java.lang.String[]);
    Code:
      0: aload_0
      1: ldc #9 // String args
      3: invokestatic #15 // Method kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull:(Ljava/lang/Object;Ljava/lang/String;)V
      6: invokestatic #21 // Method Hello.world:()V
      9: return
}
```

This looks a lot different to disassembled native code. The bit underneath the “Code:” heading is the actual bytecode that gets run. The numbers on the left hand side are the byte offset in to the code, the next column is the “opcode” and anything after the opcode are the operands.

_“But you said JVM bytecodes were one byte in length, why does the byte index on the left increment more than 1 byte per instruction?”_

The _opcodes_ are one byte in length. The operands make up the rest of the space. For example, the `invokestatic` opcode takes two additional bytes in operands: one byte to reference the class being called and another to reference the method on that class.

_“How does the number 21 reference our Hello.world() method?”_

A fundamental concept in JVM class files is the “constant pool.” It’s not output by default by `javap`, but we can ask for it:

```
$ javap -verbose MainKt.class
Classfile /tmp/MainKt.class
  Last modified May 7, 2018; size 735 bytes
  MD5 checksum f3ce23c2362512e852a5c91a1053c198
  Compiled from "Main.kt"
public final class MainKt
  minor version: 0
  major version: 50
  flags: ACC_PUBLIC, ACC_FINAL, ACC_SUPER
Constant pool:
  ...
  #16 = Utf8 Hello
  #17 = Class #16
  #18 = Utf8 world
  #19 = Utf8 ()V
  #20 = NameAndType #18:#19
  #21 = Methodref #17.#20
  ...
```

I’ve trimmed a lot of the output, but left the relevant entried in the constant pool. 16, 18 and 19 are the raw UTF-8 strings that give the names “Hello”, “world” and “()V”, which is a way of saying it’s a function that returns void. The `invokestatic` opcode specifically takes operands that are indexes in to the constant pool, and entries in the constant pool can reference other entries in the constant pool.

If we tried to compile `Main.kt` on its own, without adding Hello.class in to the classpath, we get an error very similar to the one we got when we tried to link an object files without all of the symbol definitions it required:

```
$ kotlinc Main.kt
Main.kt:2:2: error: unresolved reference: Hello
```

So because these languages, Java and Kotlin, both run using the same ABI and on the same instruction set, we are able to use that ABI’s calling conventions to make function calls across language boundaries.

# Conclusion

We’ve come a long way. From raw, native assembly code up to calling functions between languages running on the same virtual machine.

With the knowledge you have built up of calling conventions, instruction sets, object files and FFI libraries, you should now be well equipped to explore how languages not mentioned in this post would call functions written in other languages.

[^1]: Don’t worry too much about these words if they don’t mean anything to you. They’re just terms you may see in the wild and, when you do, you’ll know that the things in this post are what they are referring to.

[^2]: With exceptions including floating point calculations, which happen on the Floating Point Unit, or FPU.

[^3]: It’s not 100% silent. Check the exit status of the program when it finishes running. Why do you think it exits with the status it does?