New Game: A quest for graphics (Leaf-SDL #1)
Tags: leaf, graphics, coding, sdl
Published: 2018-02-28 09:38:34

---
title: "New Game: A quest for graphics (Leaf-SDL #1)"
published: true
description: A journey to get graphics into Leaf.
tags: leaf, graphics, coding, sdl
canonical_url: https://mortoray.com/2018/02/28/new-game-a-quest-for-graphics-leaf-sdl-1/
cover_image: https://mortoray.files.wordpress.com/2018/02/road-3186188_640.jpg
---

I've started a journey to make Leaf more visual. You may have seen some of my [videos about complexity](https://www.youtube.com/playlist?list=PLVBW3QM3BcGP0SGgomxBv1Te5dpdWrirn), complete with shaky mouse drawn lines. It's simply not good enough, especially if I want to show how tree rotations work. I want to make those better, but I want to do it in [Leaf](http://leaflang.org/).

Writing a full graphics solution in Leaf is premature;  I need to use an existing solution. Preferably one that is simple as possible, as integration isn't yet a strong point in the language.  Enter SDL - [Simple DirectMedia Layer](https://www.libsdl.org/).


## Define SDL_Init

The first step is initializing the SDL library. It seems simple but is a significant step when writing a language.

To call a foreign function we need to import it into Leaf, that looks like this:

```leaf
@import( "SDL_Init" ) multi sdl_init : ( flags : binary 32bit ) ->  ( : abi_int ) raw no_throw
```

The `@import( "SDL_Init")` code creates an import by name. The linker needs resolve this symbol.

The `multi` is just a Leaf thing, which would allow `sdl_init` to have multiple overloads. There are other options here, like `var` and `defn`, and it's possible that `multi` doesn't survive in the long-term.

The next bit, `( flags : binary 32bit ) ->  ( : abi_int ) raw no_throw` is the type of the function. The first part should be clear; it's a function that takes a 32-bit integer flags argument and returns an integer result. The `abi` in `abi_int` refers to the OS's application binary interface: the common definition of types on this system. It may not, and isn't, the same size as a Leaf `integer`.

`raw` is a bit harder to understand. In Leaf, function variables could represent a bound, unbound, or raw function. The binding refers to a context needed to call the function (think of a class function with a `this` variable). In Leaf code, you rarely think about this type, but when using external libraries, it's important. These will always be a `raw` type since they have no context or binding.

`no_throw` means that it doesn't produce any errors. At least in the sense of how Leaf reports errors: that `abi_int` return is nonetheless an error code. The term `no_throw` will likely be replaced with `no_error`, since `throw` is a term specific to exceptions, which Leaf doesn't use (at least not anymore).

## Call SDL_Init

We can now call this function:

```{.leaf}
var q = sdl_init( SDL_INIT_VIDEO )
std.println([ "sdl_init: ", q])
```

That SDL_INIT_VIDEO is a flag value from the library:

```
literal SDL_INIT_VIDEO =  0x0000_0020
```

Note there is no type on this literal: this is a feature of Leaf that constants can be rational numbers. When you use this value it will check that it can safely convert to the desired type. For simple flags, it's not that interesting.

So we're good to go. Just run it and... `SEGFAULT`.

```
#0  __strcmp_ssse3 () at ../sysdeps/x86_64/multiarch/../strcmp.S:173
#1  0x00007fffeff1a667 in llvm::cl::generic_parser_base::findOption(char const*) () from /home/src/leaf/sdl/dist/libLLVM-3.8.so
#2  0x00007ffff02f291f in llvm::RegisterPassParser<llvm::MachineSchedRegistry>::NotifyAdd(char const*, void* (*)(), char const*) () from /home/src/leaf/sdl/dist/libLLVM-3.8.so
#3  0x00007fffe3ab4754 in llvm::MachinePassRegistry::Add(llvm::MachinePassRegistryNode*) () from /usr/lib/x86_64-linux-gnu/libLLVM-5.0.so.1
#4  0x00007fffe36fe616 in ?? () from /usr/lib/x86_64-linux-gnu/libLLVM-5.0.so.1
#5  0x00007ffff7de76ba in call_init (l=<optimised out>, argc=argc@entry=4, argv=argv@entry=0x7fffffffdac8, env=env@entry=0x7fffffffdaf0) at dl-init.c:72
#6  0x00007ffff7de77cb in call_init (env=0x7fffffffdaf0, argv=0x7fffffffdac8, argc=4, l=<optimised out>) at dl-init.c:30
#7  _dl_init (main_map=main_map@entry=0x86b1b0, argc=4, argv=0x7fffffffdac8, env=0x7fffffffdaf0) at dl-init.c:120
#8  0x00007ffff7dec8e2 in dl_open_worker (a=a@entry=0x7fffffffc470) at dl-open.c:575
#9  0x00007ffff7de7564 in _dl_catch_error (objname=objname@entry=0x7fffffffc460, errstring=errstring@entry=0x7fffffffc468, mallocedp=mallocedp@entry=0x7fffffffc45f, 
    operate=operate@entry=0x7ffff7dec4d0 <dl_open_worker>, args=args@entry=0x7fffffffc470) at dl-error.c:187
#10 0x00007ffff7debda9 in _dl_open (file=0x7fffffffc6f0 "/usr/lib/x86_64-linux-gnu/dri/r600_dri.so", mode=-2147483390, caller_dlopen=0x7fffe9580a39, nsid=-2, argc=<optimised out>, 
    argv=<optimised out>, env=0x7fffffffdaf0) at dl-open.c:660
#11 0x00007ffff53c4f09 in dlopen_doit (a=a@entry=0x7fffffffc6a0) at dlopen.c:66
#12 0x00007ffff7de7564 in _dl_catch_error (objname=0x69d830, errstring=0x69d838, mallocedp=0x69d828, operate=0x7ffff53c4eb0 <dlopen_doit>, args=0x7fffffffc6a0) at dl-error.c:187
#13 0x00007ffff53c5571 in _dlerror_run (operate=operate@entry=0x7ffff53c4eb0 <dlopen_doit>, args=args@entry=0x7fffffffc6a0) at dlerror.c:163
#14 0x00007ffff53c4fa1 in __dlopen (file=<optimised out>, mode=<optimised out>) at dlopen.c:87
#15 0x00007fffe9580a39 in ?? () from /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1
#16 0x00007fffe95838f3 in ?? () from /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1
#17 0x00007fffe955ba54 in ?? () from /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1
#18 0x00007fffe9557c2b in ?? () from /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1
#19 0x00007fffe9558062 in glXQueryExtensionsString () from /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1
#20 0x00007fffef080caf in ?? () from /usr/lib/x86_64-linux-gnu/libSDL2.so
#21 0x00007fffef07337f in ?? () from /usr/lib/x86_64-linux-gnu/libSDL2.so
#22 0x00007fffef07528c in ?? () from /usr/lib/x86_64-linux-gnu/libSDL2.so
#23 0x00007fffef074f35 in ?? () from /usr/lib/x86_64-linux-gnu/libSDL2.so
#24 0x00007fffeefdc397 in ?? () from /usr/lib/x86_64-linux-gnu/libSDL2.so
#25 0x00007ffff7ff55b5 in _init_module_main_5 ()
#26 0x00007ffff7ff516f in _entry ()
#27 0x00007ffff7ff65dc in main ()
#28 0x00007ffff0d7f9bd in llvm::MCJIT::runFunction(llvm::Function*, llvm::ArrayRef<llvm::GenericValue>) () from /home/src/leaf/sdl/dist/libLLVM-3.8.so
#29 0x00007ffff4c843d9 in ir_llvm::gen::execute (this=this@entry=0x7fffffffcd90, m=..., sc="") at src/ir/llvm/gen.cpp:249
#30 0x00007ffff6bd9e90 in runner::execute (this=0x7fffffffd8b0, m=std::shared_ptr (count 3, weak 0) 0x6c9d40, args=std::vector of length 0, capacity 0) at src/runner/runner.cpp:339
#31 0x00007ffff6bda264 in runner::execute (this=this@entry=0x7fffffffd8b0, m=std::shared_ptr (count 3, weak 0) 0x6c9d40) at src/runner/runner.cpp:264
#32 0x0000000000413687 in main (argc=4, argv=0x7fffffffdac8) at src/bin/leaf.cpp:256
```

Uh oh! That's inside the LLVM execution engine. Notice how it isn't the loading of SDL that has failed, but a transitive library, through `libGL.so.1`, though `dri/r600_dri.so`. It's not the library that's failed either, but LLVM's runtime instrumentation.

I asked on the mailing list, but as of yet have no solution to this problem. Maybe an LLVM upgrade will help. Otherwise, I'll have to debug it myself.

Fortunately, Leaf doesn't have to run that way, it's a convenience for faster turnaround. We can compile programs the classic way instead.

```
leaf  --library SDL2 --exe /tmp/demo .
```

Of course, that failed: nothing is allowed to be simple!  Turns out I only ever implemented `--library` for the LLVM JIT engine. A small addition to the linker command-line and it's done.

Run `/tmp/demo` and `0` is printed on the screen.  `0` is a success value from `SDL_Init`. Great!


## SDL_Create_Window

I chose SDL instead of a GL library because I didn't want to deal with creating a native window. It's not that hard, but it's platform specific, and usually involves lots of defines, structures, and functions.

```leaf
typedef SDL_Window : abi_pack [
] 

literal SDL_WINDOW_SHOWN = 0x0000_0004

@import( "SDL_CreateWindow" ) multi sdl_create_window : ( title : raw_array｢abi_char｣,
    x : abi_int, y : abi_int, w : abi_int, h : abi_int, flags : binary 32bit ) -> ( : SDL_Window value_ptr ) raw no_throw
```

The `SDL_Window` is an opaque structure in the library (or maybe it isn't, but I don't need any parts of it). I nonetheless declare a type for safety reasons. The `value_ptr` on the return indicates a C-style pointer: these are considered "unsafe" references in Leaf, but necessary for calling libraries.

> Leaf does use `value_ptr` in many cases that will need to be safe.  I'm currently looking at how Rust does it's borrowing -- lifetime tracking will become part of Leaf. I'll be able to detect when use of `value_ptr` is invalid. These foreign functions will nonetheless remain unsafe.

```
var window = sdl_create_window( std.u8_encode("Leaf SDL Window"),  0, 0, 400, 300, SDL_WINDOW_SHOWN )
//TODO: null check on `window`
```

Strings in Leaf are, currently, just an `array｢char｣`, where `char` is a Unicode code point (a full 32-bit value). Calling an ABI API requires converting that to the ABI form. Here I'm just assuming that is `utf8`. Eventually, the standard library will need an `abi_encode` function to do it correctly (though utf-8 is the most common).

> Why did I put a `TODO` about null instead of just checking it?  It's a limitation now in Leaf that you can't inspect pointers directly. That `window` is an `SDL_Window`; that's its intrinsic type. Leaf primarily works on intrinsic types, but here I need a way to inspect the extrinsic part, the `value_ptr`.

Running this code gets us a window.

![empty window graphic](https://mortoray.files.wordpress.com/2018/02/sdl_window_empty.png)

You can see the title, but the contents seem wrong. It looks like it is transparent, just showing whatever was behind it when it started up. We haven't drawn anything, not even cleared it. It's an uninitialized area (though the capturing of what is already there seems like it might be a security issue).


## SDL_Get_Window_Surface

Next up is `SDL_Get_Window_Surface`, `SDL_Fill_Rect` and `SDL_Update_Window_Surface`. These are all just variations of what came before. I was able to get a gray color into the window.  But I didn't want gray, I wanted an RGB color.

For this we have the `SDL_Map_RGB` function, to map from RGB space to whatever colorspace the surface is using. It has this definition:

```leaf
typedef SDL_PixelFormat : abi_pack [
]
@import( "SDL_MapRGB" ) multi sdl_map_rgb : ( format : SDL_PixelFormat value_ptr, r : octet, b : octet, c : octet ) -> ( : binary 32bit ) raw no_throw
```

Hmm, it needs an `SDL_PixelFormat`. In the SDL examples, that comes from the surface.

```leaf
@import( "SDL_GetWindowSurface" ) multi sdl_get_window_surface : ( window : SDL_Window value_ptr ) -> ( : SDL_Surface value_ptr ) raw no_throw
```

Inside `SDL_Surface` is a property called `format`. To get `map_rgb` working I'll need to look into that structure -- no more opaque simplicity like `SDL_Window`.

```leaf
typedef SDL_Surface : abi_pack [
    _flags : binary 32bit,
    format : SDL_PixelFormat value_ptr,
    w : abi_int,
    h : abi_int,
    pixels : abi_ptr,
    userdata : abi_ptr,
    _locked : abi_int,
    _lock_data : abi_ptr,
    //more
]
```

Tuples are used in Leaf to represent any pure data type (those without functions, like classes). Note the use of `abi_pack` which says the ordering and alignment of the fields here must be compliant with the OS ABI. Leaf can reorder fields and have different alignment, so it's important to arrange this structure the same was as the native library.  _The `_` prefixed fields are those marked as internal by the docs_.

Before jumping into RGB mapping I could use the size fields to check if this is working:

```leaf
var surface = sdl_get_window_surface( window )
std.println(["surface ", surface.w, "x", surface.h])
```

It wasn't at first. I had forgotten the `_flags` field and was getting random values. Once fixed I could proceed to create a coloured rectangle.

```leaf
ignore sdl_fill_rect( surface, 0, sdl_map_rgb(surface.format, 0, 100, 0) )
ignore sdl_update_window_surface( window )
```

> `ignore` is required since I'm ignoring the return value from those functions. Leaf doesn't allow ignoring value by default. It's also bad form here: I should be checking those return values for errors.

![a green window](https://mortoray.files.wordpress.com/2018/02/sdl_window_green.png)

There we have it: a green window!

> Join in the fun by checking out the [SDL Branch](https://code.launchpad.net/~eda-qa/leaflang/sdl) of [Leaf](http://leaflang.org). The demo code is [on github](https://github.com/mortoray/leaf-algorithms/tree/master/sdl), though I can't promise what the current state is 🙈. Be sure to follow me on [Twitch](https://www.twitch.tv/mortoray) as well.