Rusty Boy - Week 1: Of Timing and Modes
Tags: gameboy, emulation, rust
Published: 2020-06-22 03:37:35

*Hello there!*

It's been a week since my last report and, as promised, I'm back here with the news of the rusty lad. It's been an *interesting* week. Let's jump into it.

#### Monday

I started the week by looking into Dr. Mario and why it wouldn't boot. I haven't seen this one getting to the menu on Rusty-Boy since forever, so it was about time I took a look.
So I set a breakpoint after the Bootrom's execution and started stepping. I use BGB as my control emulator, since it's pretty accurate and it has a great debugger.


##### ___It's about Time___

After a while stepping, interrupts got enabled. This is where debugging starts to get tricky. My timing isn't great, especially when compared to BGB's, so I start triggering interrupts often while my control emulator just keeps running fine. My solution to this was to just set a breakpoint on the return address, and let the game run the interrupt routine. Doing this showed an interesting thing: the game would very frequently enter the Timer interrupt, and it'd spend a good chunk of time on it. Eventually, I hit Resume after entering an interrupt and the game never returned from it. That's where I decided to check something - what would happen if I disabled the Timer interrupt? Well, the game got to the menu just fine!


![Dr. Mario's Menu](https://dev-to-uploads.s3.amazonaws.com/i/0l9xr0rdhf9osl53mk00.png)<figcaption>Now that's something I haven't seen in a while!</figcaption>


But good things don't last for long, and a second or two later, the game goes into a white screen and never recovers. It enters a loop where it waits for a certain display mode switch to happen, but the LCD is disabled, so it won't happen (as far as I know at least). It's not the first time I've seen this issue, which makes it a bit more interesting, but I poked around for a while and couldn't find an issue, so I moved on for the time being. 


##### ___The Rogue Window - Part 1___

To finish the day, I took a look at the Window rendering. 
As I briefly mentioned last week, the window likes to render in the middle of the background in a few games. Apparently just pushing the window out of the screen was simpler than just, you know, disabling it. I'm not sure I really get why, but ok.

I started looking at the most obvious thing - checking if the coordinates were actually correct. My test subject for this was Bomberman. It uses the window in fixed locations during its intro sequences, and it actually showed the issue perfectly. 

![Broken Window Rendering](https://dev-to-uploads.s3.amazonaws.com/i/5uwyfkqworgkqy1j8p9a.png)<figcaption>The Window clearly is not supposed to be there!</figcaption>

So I loaded the game on both Rusty-Boy and BGB, paused it on the roughly the same spot, and compared both emu's window coordinates. And yeah, the values were indeed different! Progress, right? Nope. A few minutes later I remembered a kind of important detail - Rusty Boy shows coordinates in *decimal* values, while BGB shows all values in *hexadecimal*. So I changed the debugger's formatting so it'd show hex values, and now the values are the same. So much for that theory then, back to the drawing board.

#### Tuesday

This one is barely worth mentioning, since I spent most of the day trying to fix networking issues around the house. Still managed to work on the emu a bit.
I messed around with the Window's position a bit and came to the conclussion that the issue was definitely on the *blitting* stage. Blitting is basically the process of merging textures together (the very simplified explanation I work with). Since this is basically the most interesting thing that happened this day, I might as well explain a bit how I do rendering on Rusty Boy.

##### ___Rusty Rendering 101___

Rendering in Rusty Boy has been quite the rollercoaster, but currently it's done mostly using Textures.

The Background and the Window both have their own Textures, which are drawn line by line and then sent to the UI thread for displaying when the Vblank period is about to end.
Sprites and Tiles are cached. This cache is invalidated based on a hash of their memory sections. If it changes, then the cache gets nuked and regenerated. Tiles have 2 banks in memory, so they get also get one cache each.

The rendering process begins on the emulation thread, on the Video chip side of things. It first checks whether or not it should regenerate caches, and acts accordingly. Then, if it's on the correct mode, it draws a line of the background and the window. The lines are drawn using the tiles in the cache, and which tile to use is indicated by the data in the background maps - these contain a value pointing to each tile that should be drawn on screen. For the background, the lines usually have an offset - the scrolling values. This allows games to move the background horizontally or vertically, but it can also be changed when reaching a specific line using interrupt to generate [some interesting effects](https://www.youtube.com/watch?v=NxKRRTW5934). The results are then saved as an array of color data, and sent to the UI thread.

The UI thread will receive an object that contains 2 backgrounds (one scrolled for the screen, and a pure one), the window, and all the sprites. The actual textures are then generated and upscaled to the target resolution, so they can then be blitted into a final one which is showed on the Screen window. This is done for the background, the window (if it's enabled), and all sprites (if they should be displayed). The unscrolled Background and Window textures are also put into the Video Debugger - it can be useful to debug graphical issues (like the window being on top of the background when it shouldn't).

![debugger](https://dev-to-uploads.s3.amazonaws.com/i/y30toufogn1dfggsefyb.png)<figcaption>The background and the window, as shown on the Video Debugger</figcaption>

I think that basically sums up the current rendering process. Let's move on.

#### Wednesday

Oh boy, Wednesday was quite a ride.
I started the day by polishing code a bit here and there. I started with the Timer, since I haven't touched that in centuries. Nothing was fixed, but it looks better now at least. Then I moved to the Video file... Dear God why did I do that.

##### ___The Failed Cleanup Attempt___

My goal was basically to clean up interrupt requests and mode switching. This involves mostly 2 registers: LCD Status and Interrupts Flags.

This LCD Status (also known as LCD STAT or just STAT) stores the current mode of the Video chip, and when it should cause interrupts. The Interrupt flags one is the register that stores the currently requested interrupts. If, say, the timer wants to request an interrupt, that register is the place to go.

Now, when you change mode, you have to modify the first 2 bits of LCD STAT (should store 0-3 depending on the mode), and depending on other bits on the register, you may also have to request an interrupt. This whole process was a bit convoluted on Rusty-Boy, so I decided to get some fresh air and try to clean it up, maybe even fix something in the process, right? Yeah, no.

![Broken Bomberman](https://dev-to-uploads.s3.amazonaws.com/i/k4su4b9rm9mq9ss7kh7h.png)<figcaption>The cleanups broke everything</figcaption>

Games ranged from not running, to showing broken graphics, and even executing invalid opcodes! How?! I did have some typos on the initial re-implementation of some things, but even after they were fixed games were still broken.

I took the easy way out. I ran `git reset --hard` and kicked the cleanup to some random point in the future. I didn't want to deal with it for the time being. So I moved on to something else, which brings us to our next topic of the day.

##### ___The Input Hack___

I mentioned on the previous post that I had an input problem.

Basically, button presses are sent from the UI thread to the emulation thread for the CPU to handle. The issue is that both threads run at very different speeds, so inputs tend to get lost in translation. If your ROM only checks for button presses (like an input test for example), it tends to register an input press if you try hard enough. But games do other things, not just constantly check for inputs, so you need to *really* try to hopefully get one press registered. It's a really ugly situation.

Tl;Dr: You press a button, and RNGesus decides whether or not it'll get through.

As you can probably imagine, this makes testing a bit difficult, since I can't get past menus in most cases. So I decided to implement a workaround (or hack) after I come up with a better fix (famous last words).
This hack basically sends 4 input events per button press. Why 4 specifically? No idea, 4 seemed like a nice number in my mind. So with this hack in place, you can actually start playing stuff. Mostly Tetris, since it's the most reliable game so far. rex-run also works pretty nicely, it's a port of the dino game from Chrome, you know the one you get when you are offline? That one. It's a fun game that I have way more playtime than I'd like to admit.

![rex-run](https://dev-to-uploads.s3.amazonaws.com/i/vyw4s4235rz03hx8uefk.png)<figcaption>It's a bit hard to play at super speed</figcaption>

Let's continue.


#### Thursday

This was "Pokemon Red Debugging" day, which of course turned into "Hit your head against a Wall" day. Nothing is never simple here.

##### ___Investigating Pokemon Red___

Ever since I moved away from SDL for rendering and into OpenGL+Dear Imgui, Pokemon Red just loops forever after the bootrom.
My first instinct was trying to look into the commit to see if anything seemed bad, but this didn't go quite as planned. The commit is [kinda big](https://github.com/RainbowCookie32/rusty-boi/commit/87d2d7669b1d89a4d1852e805f09ad2a286e6aab), which made searching for differences a bit hard. I tried, but I couldn't spot any major differences that'd cause the game to get stuck like that. *What was going on?*. Well, time to get BGB running again then.

After stepping through instructions for a while, I got to a point where Rusty-Boy would acknowledge an interrupt, but never come back from it. A Vblank interrupt to be more precise. This didn't really answer many questions, but rather brought a few more to the table - it's not the first time it calls a vblank interrupt, why does it get stuck on that one specifically? I poked around some things a bit more, but couldn't find a reason so I just wrapped the day there and went to bed.

I don't really have a solid theory for that one, there might be some timing-related issues with interrupts, but I haven't stepped into the vblank routine yet. I'll probably take a look at a disassembly of the game first to have a better idea of what I'm looking at.


#### Friday

This day started a bit rough with me trying, again, to clean up mode switching and interrupts. As expected, it didn't go any better than the previous attempt, and ended up with the same solution. `git reset --hard`. Maybe next week.
So I moved on to something that I had some confidence on, Sprites.


##### ___Sprites___

Sprites are pretty simple overall, they are basically tiles with flags.
Each Sprite has an entry on a special region in memory, the OAM (Object Attribute Memory). The entries are made from 4 bytes:
* Y position
* X position
* Tile ID
* Flags (Priority, Flip on X or Y, palette to use)

There can only be 40 sprites on memory at the same time.

With the tile cache in place, the implementation was pretty painless. I added a Sprite struct that contained the screen coordinates, color data, and the flip flags for each sprite. And I cached them, same idea as the tile cache - have a hash of the memory region, regenerate the cache when the hash changes. Easy, right? Yeah, but not typo-proof.

![First Sprite](https://dev-to-uploads.s3.amazonaws.com/i/t87n3mhdl0o8smjldgpn.png)<figcaption>The first Sprite that got rendered</figcaption>

In case you missed it, it's a zero on the top left corner of the screen. Just that.
It didn't take me very long to notice what was wrong. I copy-pasted the code for hashing the tile banks and used it for sprites. But I never changed the memory region to hash! So if the tile cache wasn't invalidated, the sprites wouldn't either. Fixed that and...

![More Tetris Sprites](https://dev-to-uploads.s3.amazonaws.com/i/jyyrwhwp9sbgunlurv4m.png)<figcaption>It's a little confused, but it's got the spirit.</figcaption>

Wasn't looking too great, but it was getting there. This one took a bit longer to figure out.
The culprit for this one was the scaling. I was multiplying the *size* of the sprites by the scaling factor, but not the position. The X and Y coordinates were assuming, of course, a 160x144 screen - the Gameboy's screen. But after applying scaling, that's not the resolution we have. So after taking that into account, we have perfectly working sprites. Mostly.

![Evolution of Sprites](https://dev-to-uploads.s3.amazonaws.com/i/fw0g7a5qbq80gltvgupb.gif)<figcaption>Evolution of Sprites in Tetris</figcaption>

Looking pretty good so far, but I'm still missing some things. Most notably 8x16 sprites, priority, and transparency.
8x16 sprites are probably going to happen soon, the others I have to read up a bit on.


##### ___The Rogue Window - Part 2___

A few hours after fixing the sprite's position, I got an idea, a spark of hope: *what if all this time I just had to scale the window coordinates, just like the sprites?* Just as I finished that thought my toaster yeeted my toasts, really dramatic. So I rushed to my PC to test it, scared that the moment of enlightenment would go away, and indeed, that was all that was needed. The window was no longer over things it shouldn't.

![Fixedman](https://dev-to-uploads.s3.amazonaws.com/i/ef0cxm3nwlkfdeflbpzi.png)<figcaption>Bomberman's intro was finally fixed!</figcaption>

With the last window-related issue, I can probably say with a certain degree of certainty that the window is currently rendering perfectly.

---

So, you are all caught up now. This is where Rusty-Boy is right now. This was a bit of a frustrating week, with a lot of things not really going anywhere, but Sprites and fixing the window saved it really.

I'll probably try to figure out what's up with Dr. Mario's loop and Pokemon Red's interrupt. Hopefully it gets somewhere.

See ya in a week!

PS: I never linked the emu itself last week, so you can go check it out [here](https://github.com/RainbowCookie32/rusty-boi).