LGTM Devlog 5: Technical Decisions - Serverless Architecture
Tags: devjournal, github, serverless, python
Published: 2020-12-28 19:24:55

Next in our technical decisions we need to make, is how and where do we host this thing? We've established a few things that colour our decision:

* We need to receive GitHub webhooks as we need to be notified about forks happening
* We need to poll the GitHub Notifications API to get information about when comments and other things happen
* We need to run our game code somewhere, and it's probably going to be in Python
* We have to track user task completion, so a database is needed

I've had a lot more thoughts as well that I haven't written about, but have technical implementation implications:
* GitHub API has usage quotas, and can fail if we try to do too many things at once. This means we need our code to handle retries.
* For realism, we might want to schedule things to happen on a timer. In fact, it's not crucial for anything to happen in real-time at all. When a player submits a result, we only need it to respond within a few tens of seconds. This vastly simplifies the requirements

# Google Firebase
I already have a strong preference for Google Cloud, mostly due to familiarity. 
It _feels_ like Google Firebase will work very well: it has cloud functions, which can receive the webhooks, if not run our entire game stack; Firestore is an appropriate database to store our game data; and furthermore the hosting would be great to run the website (which shows stats and stuff), and we may as well make use of the Auth service for managing user accounts (even though the registration won't be a traditional email/password affair).

For the Task Queue requirement, it would seem that Google's Cloud Tasks would be up for the job, and appears to work quite well with Firebase, and is a required pre-requisite to allow scheduled tasks (which is likely to be needed to control polling of the notifications API)

Our tech stack is, therefore:
* Firebase Functions (running Python)
* Firebase Auth
* Firestore
* Cloud Tasks

All of this together means I'd be able to build the game all on Serverless!

# Architecture
With these things in mind, the architecture looks something like this:

![LGTM architecture diagram](https://dev-to-uploads.s3.amazonaws.com/i/fq4ya7a3q1qlf05dzfam.png)

The reason for splitting apart the game backend into microservices is so the work is nicely chunked up, and also gives us a lot of flexibility in attaching others services. For example, I could one day add an extra set of webhook receivers and dispatchers for Gitlab:

![Architecture diagram with GitHub and GitLab](https://dev-to-uploads.s3.amazonaws.com/i/z4ql6uuy5puxtz4d7hjg.png)

And also, these diagrams aren't the full picture, I can already imagine a lot of other things that would need to be added - the database for player's game data and accounts; internationalization; and the web frotnend for the player to look at their stats and details and do other things to their account that we can't through the git/github interface; and the oauth to allow players to log into the web interface. The list goes on, and the full diagram might be even more complicated than this one:

![Full architecture maybe](https://dev-to-uploads.s3.amazonaws.com/i/5ol0wep2frjapfgo76i9.png) 
   
I think this is a good place to leave the technical and architecture decisions. We can start working with it in mind that we'll be putting things onto Firebase