Managing a Windows window within mruby Part 1: Win32 window basics
Tags: ruby, windows
Published: 2020-05-21 20:16:48

My current project, a Smalltalk-style Ruby environment, is currently a C program wrapping the mruby VM. Most of the program state and run loop is in C, with the run loop calling the mruby VM once per GUI frame. It looks something like this, abbreviated

```c++
mrb_state mruby_state;

int main() {
mruby_state = mrb_open();
setup_windows();
setup_graphics();
while(!done) {
  process_window_messages();
  start_new_frame();
  mrb_load_string(mruby_state, "World.render");
  present_frame();
}
shutdown_graphics();
mrb_close(mruby_state);
return 0;
}
```

Methods inside the mruby VM call back out to C telling the GUI what to render next frame. Everything else is in C. This works okay though I'm running into issues regarding stability. Since control passes back and forth between the Ruby VM and C I have to be extra careful on the order of operations or I end up crashing the program. What I want to do is remove as much of the C program code as I can, instead pushing the program into the Ruby VM including managing the Windows state and the graphics setup. I'll implement these as Ruby methods written in C and using `mruby_data` types when necessary. An `mruby_data` type is a C type which can wrap a C structure of arbitrary data into a Ruby class.

We're going to create a throw away project of exploratory code that you can follow along with! This first part is creating a new window with Win32 that responds to close events using C. The second part we'll push the window management into the mruby VM like I want to achieve. By the end you'll learn some basics about using Win32 and the mruby C API.

First up on the list of conversions is opening and managing a Windows window within the mruby VM. I'll use the base Win32 C API for this. When working with Windows it's common practice for people to reach for a window management library like GLFW. I don't think that's necessary. The Win32 API for window management is refreshingly simple if a bit strange in modern times. It only takes a couple dozen lines of readable C code to create a basic window. The first version of the `CreateWindow` function, which instructs Windows to create a new window, first appeared with Windows 1.0 in 1984! We still use `CreateWindow` in the same way that programmers have for the last _36 years_.

_**note**_ (and shilling): significant portions of this post about Win32 come from a chapter in my in-progress book on low level graphics programming.

Project and Compiler Setup
===

You'll need at least the Visual Studio command line tools installed. We don't need, and I'll argue forever that you never need, the full Visual Studio IDE. You can use whatever editor you're comfortable with.  We'll use Microsoft's compiler, `cl`. It's theoretically possible to do all of this with just Clang on Windows without installing Microsoft's compiler. I haven't tested that much.

We'll need just two files, a `main.cpp` file and a `build.bat` file. Starting with a basic `main.cpp` file using your normal editor

```c++
// main.cpp
int main() {
  return 0;
}
```

Start a new `x64 Native Tools Commmand Prompt for VS 2019` from the Windows start menu. This sets up a new shell with the right environment variables for the `cl` compiler. You can test that it works by entering `cl`, which prints basic usage information.

`cd` in that shell window to where you created the `main.cpp` file. Compile it with `cl`. We'll turn on creating debug symbols, `/Zi`, use the latest C++ version the compiler supports, `/std:c++latest`, turn off RTTI (run-time type information), `/GR-`, disable the compiler splash screen, `/nologo`, and enable helpful static checks, `/analyze`.

```
cl /Zi /std:c++latest /GR- /nologo /analyze main.cpp
```

Run that and you'll have a new `main.exe` file along side the `main.cpp` file, plus other sundry intermediate compiler object and linkage files. Running the `main.exe` file doesn't do anything of course. We didn't do anything except immediately exit.

Once you tested a basic main program and the compiler flags work we need just two elements added for accessing the Win32 API. Add `#include <windows.h>` to the top of `main.cpp`. This header file defines all the types, function prototypes, and constants for the entire Win32 C API.

We also need to pass the linker some parameters as well. Add `/link` to the end of your compiler line, which tells `cl` everything following `/link` are linker options. We want to generate debug symbols from linked libraries with `/DEBUG`. Finally we need to link against the library containing the symbols for creating windows, `user32.lib`. We don't need to specify any library paths, the developer command prompt set up the Windows library paths for us. The full command line looks like

```
cl /Zi /std:c++latest /GR- /nologo /analyze main.cpp /link /debug user32.lib
```

To save us from having to find that line in our command line history or retyping it frequently, save that line to a new batch file. Common convention is naming the file `build.bat`.

That's all we need set up. We're ready to get to work.

## Opening a new window

We need just three components to create a new window; creating a callback function for processing events Windows sends to the window, creating a [`WNDCLASSEX`](https://docs.microsoft.com/en-us/windows/win32/api/winuser/ns-winuser-wndclassexa) structure defining how the window behaves, and then calling the Win32 [`CreateWindowEx`](https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-registerclassexa) function.

Starting with the callback function. We register this function when creating a new window. Windows calls this function for processing incoming messages such as the close window message, keyboard and mouse events, and the like. Let's create one that doesn't do anything at the moment and come back to it later. Right underneath `#include <windows.h>`

```c++
LRESULT CALLBACK 
main_window_callback(HWND window, UINT message, WPARAM wparam, LPARAM lparam) {
  LRESULT result = 0;
  return result;
}
```

The next step is filling out a `WNDCLASSEX` struct. The Windows API uses a
typical C programming pattern of filling out a struct and passing the address of that struct to a function. We treat the structs as parameters to functions. This is similar to passing an object to a function in JavaScript, and almost identical to using an interface in a function with TypeScript.

A `WNDCLASSEX` struct passed into the `RegisterClassEx` registers a _window
class_ within Windows. A window class is a reusable unit for repeatedly creating windows of identical functionality. That's not our use case so we can keep things simple. First create a new `WNDCLASSEX` struct zero-initialized, and then fill out some members we care about. In the `main` function

```c++
WNDCLASSEX wclass = {};
wclass.cbSize = sizeof(WNDCLASSEX);
wclass.lpszClassName = "mainWindowClass";
wclass.lpfnWndProc = main_window_callback;
wclass.hInstance = GetModuleHandle(nullptr);
wclass.style = CS_OWNDC | CS_VREDRAW | CS_HREDRAW;
RegisterClassEx(&wclass);
```

The verbosity of the struct member names likely feels alien to you. Microsoft created the Windows API in the early 1980s and used the Hungarian notation naming pattern. With Hungarian notation all variables and members have the data type in the name. This was for reducing type errors. At the time compilers and computers were a great deal slower and we did not have advanced IDEs which understood the types as you were programming. It was a high time cost treating something as the wrong type accidentally and waiting for the compiler to catch it. Hungarian notation fell out of favor and disappeared through the 90s as it's usefulness diminished.

Along the way the structs Windows uses for parameters picked up a common `cbSize` member. Set this to the `sizeof` the struct. This is for sanity checking and versioning of functions which use the structs. If you pass a struct to a Windows function and it doesn't recognize the size of it the function fails.

`lpszClassName` is straightforward. With Hungarian notation, `lpsz` says two things. `lp` means 'long pointer', historically 'far pointer'. Both of those terms are meaningless today. `sz` is 'string, zero terminated'. `lpsz` is just a standard C `char *`.

`lpfnWndProc` is also obvious. `lp` is again a long pointer, `fn` is function, so this is a standard function pointer to our callback function from earlier.

`hInstance` a handle to an identifier of a running program. Since we aren't doing anything interesting we call `GetModuleHandle` with a null parameter. This returns an `HINSTANCE` pointing to the currently running program.

The `style` member is an integer representing the window style and how it behaves. It uses another common C programming pattern of using one number representing multiple preferences at once as a bitmask representing multiple preferences at once as a bitmask with the numbers of the bitmask represented by `#define` constants or members of an enum.

That's all we need for registering a window class. We call `RegisterClassEx` with the address of the `wclass` struct.

Compiling at this point works, running it doesn't do anything. We still have to create the window using [`CreateWindowEx`](https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-createwindowexa)

```c++
DWORD main_window_style = (WS_OVERLAPPEDWINDOW | WS_VISIBLE);

HWND main_window = CreateWindowEx(0, wclass.lpszClassName, "Playground",
                                  main_window_style, CW_USEDEFAULT, CW_USEDEFAULT,
                                  1024, 768,
                                  0, 0, GetModuleHandle(nullptr), 0);
```

First we create a `DWORD`, another legacy type which is now `unsigned long`. Window styles are another bitmask. Here we state we want an overlapped window, which is a collection of styles giving a window a default look. We also want the window visible by default. 

Again we aren't doing anything interesting or advanced with our window so we'll leave most of the parameters at 0.

Compiling and running again still doesn't produce anything yet. We're missing one last piece. Windows does create the window successfully. Then it sends the `WM_CREATE` message to the callback function, `main_window_callback`. Since all we do in the callback is just return 0 we drop the message. If we don't pass the message back to Windows it doesn't make the window visible.

We won't need much for callback processing. For now all we have to do is send all the messages back to Windows using [`DefWindowProc`](https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-defwindowproca).

```c++
LRESULT CALLBACK
main_window_callback(HWND window, UINT message, WPARAM wparam, LPARAM lparam) {
  LRESULT result = 0;
  switch(message){
  default: {
    result = DefWindowProc(window, message, wparam, lparam);
  }
  }
  return result;
}
```

Compile, run and success! You'll see a brief flash of a window before the program exits. You can run the program in a debugger, halting just before returning from `main` or add a `system("pause");` right before returning and you'll see a new blank window of 1024 x 768 pixels.

## Processing window messages

Now we have to do a little bit more work for a fully functioning window, processing the messages from the callback function as they come in. We need an __evil__ global variable as a signal that the program is finished. Right after `include <windows.h>`, before defining the callback function add

```c++
bool g_should_quit = false;
```

Then right after calling `CreateWindowEx` create a while loop with that variable.

```
while(!g_should_quit) {
}
```

Compiling and running at this point you'll get a window and then after a few seconds Windows thinks the program is dead. We see a familiar (Not Responding) in the title bar and in Task Manager.

![](https://dev-to-uploads.s3.amazonaws.com/i/9sq73bo5v1jckmqj6gih.png)
![](https://dev-to-uploads.s3.amazonaws.com/i/oa20k6n5fc7w28voy3cb.png)

The program didn't crash. It's still running the `while` loop just fine. The program isn't responding to messages Windows sends it, so Windows thinks the program isn't functioning currently.

Easy fix now is processing the messages sent to the window. We can do that with the [`PeekMessage`](https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-peekmessagea) function which pulls messages sent to the window off of the message queue. Create an inner loop inside the existing `while` loop

```c++
while(!g_should_quit) {
  MSG message = {};
  while(PeekMessage(&message, nullptr, 0, 0, PM_REMOVE)) {

  }
}
```

`PeekMessage` returns `0` when there are no more messages. Otherwise, the loop runs once for each message in the queue. We aren't doing anything special with the messages or are doing multi threaded window management so we leave most of the defaults at `0`. The last parameter says remove the message from the queue when we process it.

Now lets fill out the rest of this final loop. The only message we care about at this point is the `WM_QUIT` message. Windows sends this message when the user clicks the close button on the window.

```c++
switch (message.message) {
case (WM_QUIT): {
  g_should_quit = true;
  break;
}
default: {
  TranslateMessage(&message);
  DispatchMessage(&message);
  break;
}
}
```

Here we switch on the message type represented by an integer. We capture the `WM_QUIT` message, set the state signal that we're quitting. For all other
messages we pass the message back to the default handlers using the Microsoft recommended `TranslateMessage` and `DispatchMessage` functions.

Running the program at this point we have an active window! The program loops over the message queue until the user closes the window with the close button. The window closes properly. Resizing is a bit strange. That's because we aren't redrawing the window when Windows tells us we have to redraw it with a `WM_PAINT` message. We don't actually care about that since we'll use something else for drawing the window contents.

However there's one last quirk to the Win32 API. You'll notice that when running the program from a terminal your program process quits yet the terminal does not return and you must terminate it with `Ctrl-C`. This is because your program did not notify Windows your program finished. To fix this we must call the function [`PostQuitMessage`](https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-postquitmessage) from the window callback. This is a simple addition. In `main_window_callback`, add a case for capturing the `WM_CLOSE` event.

```c++
case(WM_CLOSE): {
  PostQuitMessage(0);
  break;
}
```

And that's it! Here's the entire program for creating a functional basic Win32 window

```c++
#include <windows.h>

bool g_should_quit = false;

LRESULT CALLBACK
main_window_callback(HWND window, UINT message, WPARAM wparam, LPARAM lparam) {
  LRESULT result = 0;
  switch(message) {
  case(WM_CLOSE): {
    PostQuitMessage(0);
    break;
  }
  default: {
    result = DefWindowProc(window, message, wparam, lparam);
  }
  }
  return result;
}

int main() {
  WNDCLASSEX wclass = {};
  wclass.cbSize = sizeof(WNDCLASSEX);
  wclass.lpszClassName = "mainWindowClass";
  wclass.lpfnWndProc = main_window_callback;
  wclass.hInstance = GetModuleHandle(nullptr);
  wclass.style = CS_OWNDC | CS_VREDRAW | CS_HREDRAW;
  RegisterClassEx(&wclass);

  DWORD main_window_style = (WS_OVERLAPPEDWINDOW | WS_VISIBLE);

  HWND main_window = CreateWindowEx(0, wclass.lpszClassName, "Playground",
                                    main_window_style, CW_USEDEFAULT, CW_USEDEFAULT,
                                    1024, 768,
                                    0, 0, GetModuleHandle(nullptr), 0);

  while(!g_should_quit) {
    MSG message = {};
    while(PeekMessage(&message, nullptr, 0, 0, PM_REMOVE)) {
      switch (message.message) {
      case (WM_QUIT): {
        g_should_quit = true;
        break;
      }
      default: {
        TranslateMessage(&message);
        DispatchMessage(&message);
        break;
      }
      }
    }
  }

  return 0;
}
```

We can see plainly all the components necessary for creating a basic Win32 window. Now the next step is creating an mruby environment and setting it up so all of the management and message processing is within the mruby VM. We'll use this exploratory code project on figuring that out. Armed with that knowledge I can go back to the original mruby GUI project and refactor it with the window managment within the mruby VM.

One last polish thing. By default on a high resolution display, like a 4k monitor or high DPI laptop display, Windows will pixel double any window. This makes the window readable on a high DPI display but it makes the window blurry.  We can't see this since we aren't drawing anything into the window at the moment. Fixing this is a simple function call. Add

```c++
SetProcessDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2);
``` 

right before `CreateWindow`. This signals to Windows that the application is high DPI aware and handles it appropriately.

_**note**_ if you want to remove the console window from your program, add the linker options `/SUBSYSTEM:windows` and `/ENTRY:mainCRTStartup`. This signals to the linker assembling your program to use the `main` function of the program as the entry point and the main window controls the application.