BUGOUT: Play Go against AI 💪 and Human Friends 👪
Tags: firstpost, gamedev, ai, kafka
Published: 2020-03-30 23:31:33

---
title: BUGOUT: Play Go against AI 💪 and Human Friends 👪
published: true
description: Programming for fun & skill
tags: firstpost, gamedev, ai, kafka 
---

BUGOUT provides a web interface to [KataGo](https://github.com/lightvector/KataGo), a leading, community-driven AI skilled in the [classic board game, Go](https://en.wikipedia.org/wiki/Go_(game)).  

BUGOUT also features a multiplayer option for playing against human friends, if you're lucky enough to have them.

BUGOUT's web interface is a [derivative](https://github.com/Terkwood/Sabaki) of [Sabaki](https://github.com/SabakiHQ/Sabaki), the much-loved desktop app which is already capable of hooking into KataGo, [Leela Zero](https://zero.sjeng.org/home), and [GNU Go](https://www.gnu.org/software/gnugo/).

## Source Code for the Project

The project is open-sourced, currently under MIT license. {% github Terkwood/BUGOUT no-readme %}

This write-up was originally posted at [terkwood.farm](https://terkwood.farm/tech/BUGOUT/index.html).





# Emotional Backdrop

I started playing GO sometime during high school.  Several of my friends also learned to play.  We would often play chess, for which I had a neat little carrying case with a simple game clock, a rollout board, and sturdy pieces.  I came into a possession of a GO board of standard dimensions, and used plastic pieces that were just heavy enough to feel like they mattered.

![That Good, Old Board](https://user-images.githubusercontent.com/38859656/77853799-16469d80-71b4-11ea-8f14-8ae52b84b81b.jpeg)

We would sometimes meet at a Chinese dive on West 10th Street in Indianapolis, consume delicious, sugary food, and play.

GO felt different than chess.  There was a sense of immense possibility.  There was time to think about the future.  Winning was hard.  Getting your skill up with a same-stage newbie was exhilarating, because you grew together.

Eventually I got my board signed by [VICTOR WOOTEN](https://www.victorwooten.com/) while attending a concert performance given by [Béla Fleck and the Flecktones](https://www.flecktones.com/) at the [Indiana Roof Ballroom](https://www.indianaroof.com/):  we had been playing some small rounds prior to the start of the concert.

It must have been... 1999?  Victor Wooten was super gracious, and inspired everyone to a wholesome life of technical mastery.  I was young.  I wore an undersized fedora.

The music was phenomenal, and Wooten's signature fueled my fire.

![That Good Wooten](https://user-images.githubusercontent.com/38859656/77853798-15157080-71b4-11ea-9a92-0fba25eb780d.jpeg)

In college I dreamt that one day I would wake to the sound of an obstreperous trumpet, having been mired in games and strenuous efforts to build some sort of ultimate strategy.  I dreamt that I would wake to a vision of clear dawn.

But the actual time I've spent playing GO in my life is tiny.  Life goes on, and its daily demands obscure our roots.

Still, the aesthetic delight of GO has been enjoyed by millions of individuals over the centuries, and it's this very beauty which has inspired my work over the last nine months.

GO is a game which helps me combat the lassitude of experience and boredom.

GO is not a game which must be won.  It's enough to face a player whose skill exceeds yours, and to learn.  When I play 9x9 against KataGo running on a 5W computer, I'm happy if I manage not to lose an embarrassing number of pieces.

_GO keeps me sharp_.



# Building the Project

BUGOUT started out as an excuse to implement something (anything) using [Kafka](https://kafka.apache.org/).  My previous two professional engagements had barely skirted opportunities to legitimately include a Kafka install:  one was simple enough to rely on HTTP microservices, [another had opted for Kinesis](https://github.com/WW-Digital/reactive-kinesis).

Fresh into successful semi-retirement, I didn't want to bother with finding a use case for my implementation.  I wanted what I couldn't have reasonably argued for in past, commercial settings.  I wanted to play with [Kafka Streams](https://kafka.apache.org/documentation/streams/)!

It turns out that mapping the [tiny domain model](https://github.com/Terkwood/BUGOUT/blob/unstable/judge/src/main/kotlin/Model.kt) of a GO game to kafka streams is... easy.  Initially BUGOUT was envisioned as a multiplayer GO board which could easily run in any modern browser and provide an enjoyable, boutique alternative to more popular servers and venues.  I wanted to play again with my old friends, just like when we were kids.  So the project [quickly incorporated a game lobby system](https://github.com/Terkwood/BUGOUT/issues/42) which allowed players to choose between joining a quick 19x19 game with the next person to visit the web page, or creating a nicely-formatted URL (`https://example.com/?join=nXblGBE7erWyocXYYpRN1YOzdD`) for their private game and sharing it with a friend.

![Choosing a Venue](https://user-images.githubusercontent.com/38859656/77851229-e42e3f00-71a5-11ea-8a91-93da91abf87a.png)

![Board Size Options](https://user-images.githubusercontent.com/38859656/77851228-e2647b80-71a5-11ea-9467-cf086d76fb8e.png)

![Turn Order Preference](https://user-images.githubusercontent.com/38859656/77851227-e1cbe500-71a5-11ea-8faa-264996a2257a.png)

![Sharing a Link to a Game](https://user-images.githubusercontent.com/38859656/77851226-e1334e80-71a5-11ea-834f-0c76a0e35080.png)

![Joining a Link to a Game](https://user-images.githubusercontent.com/38859656/77851225-ded0f480-71a5-11ea-92a6-bb23755df74c.png)

Finally, both players can be fairly assigned a side, and the game can begin.

![Your Color](https://user-images.githubusercontent.com/38859656/77851236-e7c1c600-71a5-11ea-94a3-750bbeab74df.png)

![Finally, A Game](https://user-images.githubusercontent.com/38859656/77851232-e55f6c00-71a5-11ea-827e-f0201c6d9f51.png)

It basically worked.  I spent time [making sure the websocket connection between the browser and the BUGOUT gateway server was solid](https://github.com/Terkwood/BUGOUT/issues/48).  I had to [guarantee that Kafka and all its dependent apps started up in an orderly fashion](https://github.com/Terkwood/BUGOUT/blob/dfb4a63be1b052bdb8dc448993d2feecfd91ce74/game-lobby/src/main/kotlin/Application.kt#L531).  And I enjoyed writing [functional-ish Kafka Streams code in Kotlin](https://github.com/Terkwood/BUGOUT/blob/dfb4a63be1b052bdb8dc448993d2feecfd91ce74/game-lobby/src/main/kotlin/Application.kt#L37):  from a cognitive perspective, it felt clean and tidy, even if the topology graphs quickly got out of hand.

![Game Lobby Message Processing: This is Getting Insane](https://raw.githubusercontent.com/Terkwood/BUGOUT/unstable/game-lobby/topo.jpg)



# Too Cheap for Pro Tier

It's pretty obvious to anyone that spends 30 minutes in the Kafka literature, that a production deployment of Kafka involves at least three machines.  I was running a single node, because my average user load was zero.  What's more, I was paying for AWS compute time using a personal credit card, so I've never been exactly fired up about running an individual host that comes anywhere near [the specs recommended by the vendor](https://docs.confluent.io/current/kafka/deployment.html).

Some additional reading turned up that [6GB of RAM is a reasonable minimum](https://www.infoq.com/articles/apache-kafka-best-practices-to-optimize-your-deployment/) for any single host:

> In most cases, Kafka can run optimally with 6 GB of RAM for heap space. For especially heavy production loads, use machines with 32 GB or more. Extra RAM will be used to bolster OS page cache and improve client throughput. While Kafka can run with less RAM, its ability to handle load is hampered when less memory is available.

Ever the miser, I settled on running a `t3.medium` compute host for my degenerate Kafka cluster with its single, lonely node.  4GB of RAM certainly worked well enough for my theoretically scalable backend -- at least as long as it didn't become popular.

Anyway, who cares about vendor reccomended minima or a realistic-looking deployment?  I was liberated from corporate hierarchy, profit motive, etc, and was ready to experience the raw freedom that I'd always dreamed about as a junior developer.  I was ready to _play with toys_.

The only problem was that running even a `t3.medium` around the clock [cost about $1/day](https://www.ec2instances.info/?filter=t3&cost_duration=daily).

🤑 That's expensive! 🤑

## Turning Kafka On and Off with Rust + AWS 

The windmills were ready to tilt.

[I spent a non-trivial amount of effort](https://github.com/Terkwood/BUGOUT/issues/75) using [rusoto](https://github.com/rusoto/rusoto) to control the startup and shutdown of my "very expensive" `t3.medium` instance.  Whenever the system detects that there are no users connected, a timer starts, and after a few minutes, [it shuts down](https://github.com/Terkwood/BUGOUT/tree/unstable/reaper) the `t3.medium` running Kafka, and all of the memory-hogging Kafka Streams apps supporting gameplay.

Whenever someone comes back to the website and wants to play a game, the system loyally [boots the Kafka host](https://github.com/Terkwood/BUGOUT/tree/unstable/bugle).

And that's great.  I can still afford to eat.  Working with rusoto was painless.  But the user experience of waiting for BUGOUT's Kafka backend to initialize is... abysmal.  It takes about 90 seconds for the EC2 instance and its various docker containers to start up.  That's 81 seconds longer than eternity.

## Starting on the Micro-Stack

Enough was enough.  I had no boss.  I had no users.  I had no chains.  My self-respect was questionable.

I would re-implement the backend [with redis streams](https://dev.to/pdambrauskas/event-sourcing-with-redis-45ha), so that my game could be online 24/7, using leftover CPU and RAM on the `t3.micro` instance that I was already paying for.

Why not?  Redis is awesome!  And by choosing `rust` for this portion of the impl, I knew I could keep my memory and CPU footprint low.

[The effort is ongoing](https://github.com/Terkwood/BUGOUT/issues/174). It will eventually be completed in the way that one completes a crocheted scarf.




# Welcome KataGo

After months of relatively steady operation, I came to the sobering conclusion that most of my close friends were busy with their own lives, and playing a synchronous session of GO with them, often across international time zones, was an exercise in scheduling prowess that surpasses even the most skilled project manager.

So I began to wonder about playing against an [AlphaZero-like](https://en.wikipedia.org/wiki/AlphaZero) AI through my browser.

If I can't play against my friends...

...why not _BUILD MY OWN FRIEND?_

## Kata Meets Nano

Thus was born the effort to integrate [KataGo](https://github.com/lightvector/KataGo), a community implementation of cutting edge Go AI that's gotten a nice little boost thanks to [hardware sponsorship from Jane Street](https://blog.janestreet.com/accelerating-self-play-learning-in-go/).

In fact, [you can already play against KataGo online](https://online-go.com/player/592684/).

That's fine. I like building things on my own -- or in this case, integrating other people's things on my own.  So I purchased an [NVIDIA Jetson Nano System-on-a-Chip](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-nano-developer-kit/) and set about linking it to my existing infrastructure.

{% github https://github.com/Terkwood/BUGOUT/issues/67 %}

The effort proceeded smoothly.  Soon I was suffering regular blows to my pride, delivered easily by a 128-core GPU that can subside on a mere 5W of power.  NVIDIA's pre-installed CUDA libs made compilation of KataGo relatively easy.  Hooking up the tiny SoC in my office to my dirt-cheap AWS `t3.micro` felt pleasantly conservative.

![Nvidia Jetson Nano Unboxed](https://user-images.githubusercontent.com/38859656/77859879-e611f580-71d9-11ea-898a-e2f928605cac.jpeg)





# A Happy Ending

There isn't much work left before the AI subsystem is complete.  It's already possible to play against KataGo through the web interface, if you're patient enough to run a read-eval-print-loop and put Kata's pieces on the board manually.

After just a few more hours of work, I'll be able to head to my website from anywhere in the world, and play a power-efficient little bot that's just right for my skill level.

And then, free from the lamentable desire for human interaction, I'll play against my own, new type of friend:  a friend born of silicon of and parallel arithmetic.

![Love the Log](https://user-images.githubusercontent.com/38859656/77859737-1a38e680-71d9-11ea-9a50-3ca25d7b9558.png)

![Losing peacefully to SkyNet](https://user-images.githubusercontent.com/38859656/77859794-61bf7280-71d9-11ea-8f00-d9eef8bc437b.png)