Managing Windows Within mruby Part final: full encapsulation of state
Tags: ruby, cpp, tutorial
Published: 2020-05-29 01:08:09

## The color is mauve

One trivial realization I had regarding the bike shed. Before I had the pattern of naming C functions bound to Ruby methods like `class_name_method_name_method`. While this makes sense I think it's better naming the functions like `ClassName_method_name` instead. This make it more visually distinct and easier searching for it. I've changed to that naming scheme here.

Also a little debugging trick when loading code externally into the Ruby VM with functions like `mrb_load_string`. After the function call you can check for failure by checking if the VM has an unhandled exception, like so

```c++
mrb_load_string(mrb, "foo");
if(mrb->exc) {
mrb_print_error(mrb);
}
```

Now that I've established that the bike shed is mauve today, lets establish where we're at and what we're doing to finish off this exploratory code. Right now we have an instance of the mruby VM running and can create Windows windows calling into the VM. The Ruby methods implemented in C use the Win32 API to create new windows. These are remarkable achievements and we still have some polish left. We want everything possible within the VM. Enumerated, here's the steps we'll take

1. Change the `create` method to a constructor
2. Add `add_window` and `remove_window` class methods on `W32Window`. These keep track of the number of open windows. Set `g_should_quit` after the last window closes.
3. In the new `W32Window` constructor register itself to the open windows using the new `W32Window.add_window` method.
4. Perform a three step procedure registering arbitrary data with a Windows window. We'll assign the `object_id` of the `W32Window` instances to the Windows window. This is the most complicated part of the program
5. Change the callback function for processing Windows messages to call `W32Window.remove_window` when the window closes. When there are no more windows left, the callback function requests the program quits.

## Creating a constructor method in C with mruby

This is straightforward. Just as if we're working entirely within Ruby we define an `initialize` instance method which the VM calls when we call `new`. Lets rename the original `w32_window_create_method` to `W32Window_initialize` and change `mrb_define_class_method` to `mrb_define_method` appropriately

``` c++
RClass *W32Window = mrb_define_class(mrb, "W32Window", mrb->object_class);
mrb_define_method(mrb, W32Window, "initialize",
                        W32Window_initialize, MRB_ARGS_REQ(2));
```

Then change `mrb_load_string` to create an instance instead of calling `create`

``` c++
mrb_load_string(mrb, "5.times { |n| W32Window.new \"Window #{n}\", width: n * 100, height: n * 100 }");
if(mrb->exc) {
  mrb_print_error(mrb);
}
```

Running this works exactly as before. The difference is we now have an instance backing each window. This allows us in the near future to match a Windows window with a Ruby object instance later.

The function prototypes for instance variable manipulation are in the `variable.h` header. Add that include at the top of the file. We're using the `mrb_iv_set` function, which is the equivalent of `instance_variable_set :@foo, 'bar'`. It's definition

``` c++
mrb_iv_set(mrb_state *mrb, mrb_value obj, mrb_sym sym, mrb_value v)
```

The only new thing here is the `mrb_sym` type. It's actually a typedef of `uint32_t`. Every symbol in Ruby is an index to a global hashed symbol table. The integer value is there result of the hash.

There are handy C functions for creating and finding symbols. The easiest one for our case is `mrb_intern_cstr`. The documentation implies that it creates a symbol. This function actually finds an existing symbol based upon the C string provided or creates a new symbol in the VM and returns a new symbol.

Putting it all together, right after extracting the arguments

``` c++
mrb_iv_set(state, self, mrb_intern_cstr(state, "@window_name"), window_name);
```

Lets validate that this works by changing up the `mrb_load_string` line a bit

``` c++
mrb_load_string(mrb, "1.upto 5 { |n| puts W32Window.new(\"Window #{n}\", width: n * 100, height: n * 100).inspect }");
```

![](https://dev-to-uploads.s3.amazonaws.com/i/5cl08qo59zf871w8pkdi.png)

One special thing about creating an instance. Currently we are returning `mrb_nil_value()`. While this is currently working the `new` method actually returns `nil` instead of the new instance. Change `return mrb_nil_value()` to `return self`. 

## Populating a Win32 window with a Ruby Object ID

Now we create instances of objects in the VM for each Win32 window we create. There is a disconnect between our instance and the window callback function. When Windows send a message to our window our callback function `main_window_callback` runs with the message passed into it. What we don't know is which Ruby object that window belongs to. We do know the HWND of that window. With that we could create a table in the VM of HWND objects to W32Window instances. However we have a shorter path available. Windows supports passing an arbitrary pointer upon creating the window. You can also set an arbitrary pointer on the window after it's finished creating. [Microsoft](https://docs.microsoft.com/en-us/windows/win32/learnwin32/managing-application-state-) has a detailed article explaining the procedure.

Notice the important distinction. [`CreateWindowEx`](https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-createwindowexa) initiates the window creation. We can pass an arbitrary pointer to `CreateWindowEx` as the last parameter, but we cannot set the arbitrary pointer on the window yet. For that, when Windows finishes constructing the window and just before presenting it, it sends the window the [`WM_CREATE`](https://docs.microsoft.com/en-us/windows/win32/winmsg/wm-create) message, with the `LPARAM` in the callback function set as the pointer we specified in `CreateWindowEx`. This means setting a pointer to our Ruby object on the window is a two step process. First, get the C pointer to `self` and pass it as the last parameter to `CreateWindowEx`. Then, in our callback function, intercept the `WM_CREATE` message. Cast the `LPARAM` back to the Ruby pointer, then use [`SetWindowLongPtr`](https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setwindowlongptra) to store the pointer along with the window so we can get it back later.

This is the gnarliest bit of code we'll ever get into so I want to be sure we're all understanding the different components involved. We cannot have full control over the window and events, Windows controls that. Windows notifies us when things happen through sending messages to our callback function. Our task is matching up the window messages with data inside the VM. In particular we want the VM to maintain a list of the open windows. When the list is empty, we want to notify Windows to terminate the program. Each time we create a window within the VM we add to the list. When Windows notifies us it destroyed a window we remove it from our list.

When Windows calls our callback function we're in the C. What we need now is a pattern in C for locating objects in the VM from outside the VM. The first intuition is get a C pointer to the live objects inside the VM. It's not recommended we keep a C pointer to VM objects. That pointer can become invalid for reasons outside our control resulting in a hard crash following a wild pointer. The object it points to can get garbage collected if we're not careful. The VM may move data around in memory, like the Ruby 2.7 heap compacting feature. What we do have that's guaranteed stable and unique for identifying objects within the VM is the `object_id` of every object. We'll use the `object_id`, a 64 bit unsigned integer.

### Keeping track of all open windows within Ruby

To get started lets add a bit of Ruby code. We'll have the W32Window class keep track of all of it's instances.

```c++
RClass *W32Window = mrb_define_class(mrb, "W32Window", mrb->object_class);
const char* ruby_str = R"(
class W32Window
  @open_windows = []

  def self.add_window obj
    @open_windows.push obj
  end

  def self.remove_window oid
    @open_windows.delete_if { |e| e.object_id == oid }
  end
end)";
mrb_load_string(mrb, ruby_str);
```

This sets us up for success later. Although iterating using Ruby code isn't as efficient as writing it in C this isn't called on a hot path.

In the `W32Window_initialize` function, lets add to the `@open_windows` upon creation using the method. We can call methods on Ruby objects from C using `mrb_funcall`.

```c++
mrb_value mrb_funcall(mrb_state *mrb, mrb_value val, const char *name, mrb_int argc, ...);
```

This is equivalent to `obj.send :method_name, arg1, arg2 .. ,argN`. The second parameter is the object we're sending the message to, third is a `const char *` of the name of the method, fourth is the number of method arguments we're sending, and then the `va_args` of the `mrb_values` as method parameters.

```c++
mrb_funcall(state, mrb_obj_value(mrb_obj_ptr(self)->c),
            "add_window", 1, self);
```

(There is a slightly more type-safe way using `mrb_funcall_argv` which takes an `mrb_symbol` for the method name instead of a literal C string)

The strange bit is `mrb_obj_value(mrb_obj_ptr(self)->c)`. What we're doing here is the Ruby equivalent of `self.class.add_window self`. The construction `mrb_obj_value(mrb_obj_ptr())` is like `self.class`. `mrb_obj_ptr` is a macro which takes any `mrb_value` type and casts it's `void * value.p` to an `RObject *`. (Recall that every `mrb_value` is a wrapper with a `void *`, and the `tt` member of the `mrb_value` informs how to interpret the `void *` With an `RObject *` we can get the objects `RClass *` by following the `->c` pointer. However, `mrb_funcall` doesn't want an `RClass *` or `RObject *`, it wants an `mrb_value`. So we have to convert the `RClass *` back to an `mrb_value` using the `mrb_obj_value` function.

Next we need a pointer to a bundle of data that we'll store with the window that we get back when Windows sends the window messages. We need two things, the `mrb_state` and the `object_id` of the `W32Window` instance tied to that window. Lets add that just underneath all the `#include`s

```c++
struct W32Window_cb_data {
  mrb_state* state;
  uint64_t object_id;
};
```

And then allocate a bit of memory on the heap to contain the data. I 100% would rather not allocate memory for such a situation. I did commit to working in Rome by working with a dynamic language with lots of small heap memory allocations, and will therefore work as the Romans do.

```c++
W32Window_cb_data *cb_data = reinterpret_cast<W32Window_cb_data *>
(malloc(sizeof(W32Window_cb_data)));
cb_data->state = state;
cb_data->object_id = mrb_obj_id(self);
```

Now finally pass that along to `CreateWindowEx` as the last parameter.

```c++
HWND main_window = CreateWindowEx(0, wclass.lpszClassName, RSTRING_PTR(window_name),
                                  main_window_style, CW_USEDEFAULT, CW_USEDEFAULT,
                                  mrb_fixnum(kw_values[0]),
                                  mrb_fixnum(kw_values[1]),
                                  0, 0,
                                  GetModuleHandle(nullptr),
                                  cb_data);
```

### Attaching the Ruby Object ID to the window

Great, that's the first part done. We allocated the information we need later and passed it along to the next step. The next step is extracting the pointer and saving it to the window's user data when Windows sends our window the `WM_CREATE` message. At the top of the `main_window_callback` we'll do that using `SetWindowLongPtr`

```c++
if(message == WM_CREATE) {
  CREATESTRUCT* create = reinterpret_cast<CREATESTRUCT *>(lparam);
  LONG_PTR objectid_ptr = reinterpret_cast<LONG_PTR>(create->lpCreateParams);
  SetWindowLongPtr(window, GWLP_USERDATA, objectid_ptr);
}
```

When windows sends the `WM_CREATE` messages it populates the `lparam` as a `CREATESTRUCT`. That type encapsulates all the parameters passed into `CreateWindowEx`. The member of the `CREATESTRUCT` we care about is `lpCreateParams`. That's the `W32Window_cb_data` we passed in into `CreateWindowEx`. We relay it one last time, setting the `GWLP_USERDATA` pointer to the `W32Window_cb_data` window pointer. We won't return after this condition, we'll let the rest of the function continue as normal.


### Deleting a Ruby object on window close

Now one last bit. We'll catch message sent to our window when Windows deletes the window. This is the `WM_DESTROY` message. What we'll do is extract the `GWLP_USERDATA` pointer from the closing window, cast it back to the `W32Window_cb_data` pointer, call `W32Window.remove_window` with the object_id. Then we'll check if there are less than 1 open windows. If so, we'll tell Windows to terminate the program just as before with `PostQuitMessage`. Since we used `malloc` to create a small struct on the heap containing `W32Window_cb_data`, we also have to `free` it to avoid a memory leak. The whole procedure in code, replacing `WM_CLOSE` with `WM_DESTROY`. There isn't anything here we haven't seen previously.

```c++
case(WM_DESTROY): {
  LONG_PTR instance_pointer = GetWindowLongPtr(window, GWLP_USERDATA);
  W32Window_cb_data *cb_data = reinterpret_cast<W32Window_cb_data*>(instance_pointer);
  RClass *klass = mrb_class_get(cb_data->state, "W32Window");
  assert(klass != nullptr);

  mrb_value oid;
  SET_INT_VALUE(oid, cb_data->object_id);
  mrb_funcall(cb_data->state, mrb_obj_value(klass), "remove_window", 1, oid);

  mrb_value open_windows = mrb_obj_iv_get(cb_data->state,
                                          (RObject *)klass,
                                          mrb_intern_cstr(cb_data->state, "@open_windows"));
  if(RARRAY_LEN(open_windows) < 1) {
    PostQuitMessage(0);
  }

  free(cb_data);
  break;
}
```

Compiling and running now, it works as expected! The program continues running until all the windows close.

![](https://dev-to-uploads.s3.amazonaws.com/i/d4ataxjk0a7hn6v3mgzd.gif)

This is a pretty large achievement. We integrated with Win32 while pushing most of the state of the program into the Ruby VM. Most of it. There's one last thing left, the main program loop is still in C. Lets put that into Ruby as well.

## Moving the main program loop to Ruby

Lets create an `Application` module with four static methods on it `start`, `loop`, `quit`and `process_windows_messages`. The first three are basic Ruby methods. The last one is a method we'll implement in C. We'll push our current `mrb_load_string` call creating `W32Windows` into `start`, call `process_windows_messages` in the `loop`, and then set a flag when we call `quit` that the program should terminate. 

First create the `Application` module in C so we have an easy reference to it later.

```c++
RClass *Application = mrb_define_module(mrb, "Application");
```

Adding to the current `ruby_str` string.


```ruby
module Application
  @is_running = true

  def self.start
    1.upto 5 do |n|
      W32Window.new("Window #{n}", width: n * 100, height: n * 100)
    end
    loop
  end;

  def self.loop
    while @is_running
      process_windows_messages
    end
  end

  def self.quit
    @is_running = false
  end
end
```

Define the `Application.process_windows_messages` in C, moving the `while(PeekMessage)` loop into it. When we receive the `WM_QUIT` message, we call the `Application.quit` method instead of setting `g_should_quit` instead.

```c++
mrb_value
Application_process_windows_messages(mrb_state* state, mrb_value self) {
  MSG message = {};
  while(PeekMessage(&message, nullptr, 0, 0, PM_REMOVE)) {
    switch (message.message) {
    case (WM_QUIT): {
      mrb_funcall(state, self, "quit", 0);
      break;
    }
    default: {
      TranslateMessage(&message);
      DispatchMessage(&message);
      break;
    }
    }
  }
  return mrb_nil_value();
}
```

Then add the `process_windows_messages` method on `Application` using `mrb_define_module_function`

```c++
mrb_define_module_function(mrb, Application, "process_windows_messages",
                            Application_process_windows_messages, MRB_ARGS_NONE());
```
Now we can delete the original loop and `g_should_quit` as these are entirely within the Ruby VM state.

The last thing we have to do is call `start` on `Application` to kick off the program loop written entirely in Ruby

```c++
mrb_funcall(mrb, mrb_obj_value(Application), "start", 0);
```

Compile and run again, it still works exactly as before. However, the amazing thing here is now the VM controls absolutely everything about the program. All of the program's functionality sets up the VM state and then kicks off the VM in a program loop. We implement Ruby methods in C whenever we must interface with something external to the VM. It's memory stable, though in it's current state it dominates the CPU. It runs the program loop as fast as possible. A quick though inelegant fix is add the `mruby-sleep` gem in the `build_config.rb` and rebuild mruby. This provides a `sleep` method for throttling the `while` loop.

## More polish

While this all works and we could wrap up our exploratory program there's one last little thing that bothers me. We call `RegisterClassEx` with the same information and window class name. While this doesn't crash the program it is an error. Lets fix this little error so that we create one window class on program start and use that instead.

Lets create a `W32Window.register_window_class` and call it within `Application.start`. We'll use the class name of `W32Window` as the Windows window class name. Move all of the code regarding registering a Windows window class into it.

```c++
mrb_value
W32Window_register_window_class(mrb_state *state, mrb_value self) {
  mrb_value klass_name = mrb_class_path(state, mrb_class_ptr(self));
  WNDCLASSEX wclass = {};
  wclass.cbSize = sizeof(WNDCLASSEX);
  wclass.lpszClassName = RSTRING_PTR(klass_name);
  wclass.lpfnWndProc = main_window_callback;
  wclass.hInstance = GetModuleHandle(nullptr);
  wclass.style = CS_OWNDC | CS_VREDRAW | CS_HREDRAW;
  RegisterClassEx(&wclass);

  return mrb_nil_value();
}
```

Only thing new here is `mrb_class_path` which returns an `mrb_value` containing the string name of the class. It's the same as calling `W32Window.name` within Ruby. Register this as a class method

```c++
mrb_define_class_method(mrb, W32Window, "register_window_class",
                        W32Window_register_window_class, MRB_ARGS_NONE());
```

Then within `W32Window_initialize` use the Ruby class name for registering the Windows window class

```c++
mrb_value klass_name = mrb_class_path(state, mrb_class(state, self));

CreateWindowEx(0, RSTRING_PTR(klass_name), RSTRING_PTR(window_name),
```

Lastly call `W32Window.register_window_class` at the top of the `Application.start` method. Compile and run and it works exactly as before, the only difference here is that it's more 'correct' as we're not repeatedly creating the same window class every time we create a new window.

## The end of a mini series

That's it for this exploratory code. I've proved to myself that it is possible encapsulating state entirely within the Ruby VM while working with a complicated external system in C. The size of the program by line count doubled. There is an increased mental overhead of passing data back and forth out to Windows and receiving it back in the message callback.

There are other approaches to this. We could keep some state in C like we did previously or keep tables of mapping from C to the Ruby VM in C and translate them ourselves when we need. Those approaches is how the current Ruby integrated environment I'm working on currently functions. I run into issues with this approach, where the right Ruby incantation can bring the whole program down in a hard to debug fashion. I've found that keeping as much within the Ruby VM as possible dramatically decreases complexity and increases the amount of debugging tools available especially as I add more functionality to the environment.

Two things I did not get a chance to touch on are `mrb_data` types, which allow you to set instance variables wrapping C structures, and the `MRB_TT_CPTR` class type encapsulating a C pointer.

I'm now taking the techniques I learned here and bringing them back to my _still unnamed_ project. It will take a while encapsulating it's current state into Ruby fully. A lot of uninteresting error prone grunt work not worth documenting.

The entire final exploratory program

```c++
#include <windows.h>
#include <stdio.h>
#include <assert.h>
#include "mruby.h"
#include "mruby/compile.h"
#include "mruby/string.h"
#include "mruby/variable.h"
#include "mruby/array.h"
#include "mruby/class.h"

struct W32Window_cb_data {
  mrb_state* state;
  uint64_t object_id;
};

LRESULT CALLBACK
main_window_callback(HWND window, UINT message, WPARAM wparam, LPARAM lparam) {
  if(message == WM_CREATE) {
    CREATESTRUCT* create = reinterpret_cast<CREATESTRUCT *>(lparam);
    LONG_PTR objectid_ptr = reinterpret_cast<LONG_PTR>(create->lpCreateParams);
    SetWindowLongPtr(window, GWLP_USERDATA, objectid_ptr);
  }

  LRESULT result = 0;
  switch(message) {
  case(WM_DESTROY): {
    LONG_PTR instance_pointer = GetWindowLongPtr(window, GWLP_USERDATA);
    W32Window_cb_data *cb_data = reinterpret_cast<W32Window_cb_data*>(instance_pointer);
    RClass *klass = mrb_class_get(cb_data->state, "W32Window");
    assert(klass != nullptr);

    mrb_value oid;
    SET_INT_VALUE(oid, cb_data->object_id);
    mrb_funcall(cb_data->state, mrb_obj_value(klass), "remove_window", 1, oid);

    mrb_value open_windows = mrb_obj_iv_get(cb_data->state,
                                            (RObject *)klass,
                                            mrb_intern_cstr(cb_data->state, "@open_windows"));
    if(RARRAY_LEN(open_windows) < 1) {
      PostQuitMessage(0);
    }
   
    free(cb_data);
    break;
  }
  default: {
    result = DefWindowProc(window, message, wparam, lparam);
  }
  }
  return result;
}

mrb_value
W32Window_register_window_class(mrb_state *state, mrb_value self) {
  mrb_value klass_name = mrb_class_path(state, mrb_class_ptr(self));
  WNDCLASSEX wclass = {};
  wclass.cbSize = sizeof(WNDCLASSEX);
  wclass.lpszClassName = RSTRING_PTR(klass_name);
  wclass.lpfnWndProc = main_window_callback;
  wclass.hInstance = GetModuleHandle(nullptr);
  wclass.style = CS_OWNDC | CS_VREDRAW | CS_HREDRAW;
  RegisterClassEx(&wclass);

  return mrb_nil_value();
}

mrb_value
W32Window_initialize(mrb_state* state, mrb_value self) {
  mrb_value window_name;
  mrb_int width;
  mrb_int height;
  const char* kw_names[2] = { "width", "height" };
  mrb_value kw_values[2] = {};
  const mrb_kwargs kwargs = { 2, kw_values, kw_names, 2, nullptr };
  mrb_get_args(state, "S:", &window_name, &kwargs);
  mrb_funcall(state, mrb_obj_value(mrb_obj_ptr(self)->c),
              "add_window", 1, self);
  mrb_iv_set(state, self, mrb_intern_cstr(state, "@window_name"), window_name);

  DWORD main_window_style = (WS_OVERLAPPEDWINDOW | WS_VISIBLE);

  W32Window_cb_data *cb_data = reinterpret_cast<W32Window_cb_data *>(malloc(sizeof(W32Window_cb_data)));
  cb_data->state = state;
  cb_data->object_id = mrb_obj_id(self);

  mrb_value klass_name = mrb_class_path(state, mrb_class(state, self));

  CreateWindowEx(0, RSTRING_PTR(klass_name), RSTRING_PTR(window_name),
                 main_window_style, CW_USEDEFAULT, CW_USEDEFAULT,
                 mrb_fixnum(kw_values[0]),
                 mrb_fixnum(kw_values[1]),
                 0, 0,
                 GetModuleHandle(nullptr),
                 cb_data);

  return self;
}

mrb_value
Application_process_windows_messages(mrb_state* state, mrb_value self) {
  MSG message = {};
  while(PeekMessage(&message, nullptr, 0, 0, PM_REMOVE)) {
    switch (message.message) {
    case (WM_QUIT): {
      mrb_funcall(state, self, "quit", 0);
      break;
    }
    default: {
      TranslateMessage(&message);
      DispatchMessage(&message);
      break;
    }
    }
  }

  return mrb_nil_value();
}

int main() {
  mrb_state *mrb = mrb_open();
  RClass *W32Window = mrb_define_class(mrb, "W32Window", mrb->object_class);
  RClass *Application = mrb_define_module(mrb, "Application");

  const char* ruby_str = R"(
class W32Window
  @open_windows = []

  def self.add_window obj
    @open_windows.push obj
  end

  def self.remove_window oid
    @open_windows.delete_if { |e| e.object_id == oid }
  end
end

module Application
  @is_running = true

  def self.start
    W32Window.register_window_class

    1.upto 5 do |n|
      W32Window.new("Window #{n}", width: n * 100, height: n * 100)
    end

    loop
  end;

  def self.loop
    while @is_running
      process_windows_messages
    end
  end

  def self.quit
    @is_running = false
  end
end
)";

  mrb_load_string(mrb, ruby_str);

  mrb_define_class_method(mrb, W32Window, "register_window_class",
                          W32Window_register_window_class, MRB_ARGS_NONE());

  mrb_define_method(mrb, W32Window, "initialize",
                    W32Window_initialize, MRB_ARGS_REQ(2));

  mrb_define_module_function(mrb, Application, "process_windows_messages",
                             Application_process_windows_messages, MRB_ARGS_NONE());

  mrb_funcall(mrb, mrb_obj_value(Application), "start", 0);

  mrb_close(mrb);

  return 0;
}
```