Early Heroku and App Platform infrastructure, with some thoughts about letting you know when your colony ship has been built
Tags: gamedev, heroku, digitalocean, devops
Published: 2021-05-28 12:24:22

Hello. In this part of my development blog I will describe the infrastructure I chose for the early development of my online persistent universe single/coop players space game, AGS9. I will try to describe the reasons for my choices and describe some technical stuff. This is what I'm going to implement next in my development process.

To ease deployment and not "fiddle around" with servers and infrastructure, I chose to run my game server under Heroku. Instead of choosing a real-time process that "runs in the background all the time" and calculates the next "tick" of the universe in some sort of gameloop, possibly using an ECS (Entity Component System) for tracking all the things that should be going on, I decided to go the old "classical web-app" way and have a relational database and a web server.

Even though the game "seems" real time, it doesn't have to be as responsive as a first-person-shooter, and I can have the network latency of using a relational database to answer questions from the client like: "what planets are in this solar system?" or "how many resources do I have?". The client can also ask the web server, or more appropriately game server, requests like "build me a nice colony ship". In this case, I intend to write a "build entry" in some table and then do something a bit strange.

The something that is a bit strange is this. I have a task/event system, in this case a Celery worker with a persistent Redis backend (also on Heroku), and I write a delayed task, 5 hours from now that will write to some queue (probably Redis again) that a colony ship was built. I need to check the database before I publish this event, because the user might have chosen to cancel the build, and I don't have an easy or reliable way to remove the "ship built" event from the queue.

The game server, i.e. the web process on Heroku, listens to the queue I mentioned. Upon receiving the successful "ship built" event, I notify all connected WebSocket clients of the event so they can shows a nice message to the player. The ship is also added to the fleet table for the player (probably by the worker though, not the web process).

This is a bit strange, but I prefer to have an event-based system instead of some sort of "DB-queue", that queries a "next builds" table for rows with an "end time" that is before Now, and acts accordingly. I would hate having such loop run in some 1-second cron, possibly not doing a lot each iteration.

The persistence of the Redis server should keep the delayed tasks after a restart or failure of Redis, so I hope I will be fine in this case.

I do need to read a bit and think of a case where I'll have to scale the Redis servers while keeping persistence between them. That's an interesting point.

A few words on the client side.

Currently, in the current development stage, I host two static websites on the DigitalOcean App Platform. The first is a statically-exported Next JS app that authenticates the player with Auth0. The other website is a Phaser-based "game-only" app that connects to my Heroku server with a WebSocket. There's a small "handshake" after the WebSocket connection is established, where the Auth0 access token is passed to the server for validation.

The reason for separating the two, is that the first site is built quickly to just give the most basic authentication and managing games functionality, and after the game is more mature I could replace this "portal" or "lobby website" with a well-polished drop-in replacement I will build, without changing the code for the Phaser game client.

I had to come up with a way to pass over the token from the Next JS site to the Phaser app. I came up with having a page under the Next app that loads the other game app in a full-viewport iframe and uses postMessage to pass the token. Passing the token via a query parameter for example is bad for various reasons, and I didn't want to keep the token in a global window.something variable, to defend the token (over-engineered security :)).

These websites are deployed with the App Platform of DigitalOcean, which alongside Heroku gives me some sort of rudimentary CI/CD system that lets me focus on app-development and not deployment and cloud infrastructure.

A quick tip with Next JS static sites and the App Platform is to add `trailingSlash: true` to your next config, so a route like mydomain.com/play is not a 404 with the App Platform's current routing. The exported play.html page is not accessible on such route, but if you have a play/index.html it works. See this: https://www.digitalocean.com/community/questions/nextjs-export-redirect-path-without-extension-to-path-with-html?answer=65145

I hope this is not too boring. In the next part I intend to write about the first 'test game' I am going to develop and how I built it.