I became tired of making manual deployment mistakes and this happened
Tags: unity3d, python, deployment, automation
Published: 2022-05-14 15:58:24

---
title: I became tired of making manual deployment mistakes and this happened
published: true
description: Sometimes the problem you're dealing with is the best coding project.
tags: unity3d, python, deployment, automation
canonical_url: https://www.clpsplug.com/en/2022/05/15/automation.html
//cover_image: https://direct_url_to_image.jpg
---

This is the story of how I decided to automate the download process of Unity Cloud Build artifacts.

{% github Clpsplug/ucb-webhook-receiver %}

## A rather big set of binaries

Let me set the scene for you.

I have a game published on Steam - [Twin Horizons](https://store.steampowered.com/app/1352730/). It's a music game, and it's made with Unity.

The repository I save the code employs the [git-flow](https://danielkummer.github.io/git-flow-cheatsheet/index.html) model, and whenever there is a commit on the `base` branch, it kicks builds on the Unity Cloud Build (thank _goodness_ they _finally_ supported IL2CPP for Windows.)  

There are four targets, or types of the game executable, to be built. They are:
* macOS version, Production Build
* macOS version, Developer Build w/ development tools
* Windows version, Production Build
* Windows version, Developer Build w/ development tools

After they are built, I have to: 
1. download them 
1. place them on the specific location on my filesystem and
1. add an Addressable bundle to all of them.

This is what I need to do so that the tools provided by Steam can see the new build.

## The problem

Several month ago, I received a message from one of my tester that his installation broke on the latest update. *Ah s#!+, here we go again.*
  
I gathered the logs. I tested on Unity. I even tested the built app. No luck. Heck, it was not even supposed to occur with the build he received.

...or so I thought.

Turns out, when I placed the executables, **I made a mistake and placed the production version in the development version directory.**


![FACEPALM](https://upload.wikimedia.org/wikipedia/commons/3/3b/Paris_Tuileries_Garden_Facepalm_statue.jpg)
> Alex E. Proimos, [CC BY 2.0](https://creativecommons.org/licenses/by/2.0), via Wikimedia Commons


Thank _goodness_ it was not the other way around.

...Come to think of it, I had a similar mess-ups in the past. I *did* avoid the catastrophe at that time because I at least had a temporary branch to test the game I have uploaded. 

## THAT'S IT! NO MORE HANDICRAFT!

(╯°□°)╯︵ ┻━┻

So I launched an investigation. I knew Unity Cloud Build support webhooks. So somehow, I had to receive them and automatically download them.

**Where** do I host the receiver, though?

I have a rented server, but it is for serving webpages. If I had done what I'm trying to do, eventually they will shut my account down for using too much of their processing capability. Remember, I have to run the uploader provided by the publisher on the machine that has the game.
I do have an VPS account, but they tend to be on the expensive side. I'm a broke-ass university student; Unless I want to set up a VPN or something to get a static IP, I want to avoid that.

At this point I thought, "it's kind of risky, but *maybe* I can **try hosting it myself from my old PC and somehow expose it to the Internet.**"  

That's when I found [Loophole](https://loophole.cloud/), a free tunneling service. If I can host an HTTP server that can receive the webhook from UCB, I can kick off automated binary downloads.

{% github loophole/cli %}

Now that the hosting method is decided, I had to learn how to listen to HTTP requests and run scripts in background simultaneously.

## How do I make the server?

I wanted to make this as lightweight as possible. So using apache2 sounded very exaggerated to me; that is a full-featured web server. 
Jenkins was not my option for a similar reason; if I was building the game on my machine, it would've made sense to use it. But the only goal here is to download a game and place it in the correct place. 
I just wanted something lightweight that the only thing it does is to listen to a port, and worked on the language I'm comfortable with.

Since I wanted to make something in Python besides my research projects, I decided to code it in Python.

Few web searches later I found [CherryPy](https://cherrypy.dev/), a lightweight web server in Python. It turns out it can safely spawn background tasks while it serves the webhook. Perfect.

## What to download?

This is an example of the "Buiid Successful" payload - most of the information is redacted, but you get the idea:

{% gist https://gist.github.com/Clpsplug/c62af18ca0428a173584c3a4b99900db %}

The `artifacts` array has several other elements for you to download, including debug symbols. I omitted all but the one whose `primary` key is `true`. That is our game.

You can also generate secret key to check the payload - see the `x-unitycloudbuild-signature` header.

## The result

Several filesystem f*ck-ups later, here's my invention:

{% github Clpsplug/ucb-webhook-receiver %}

This project receives webhooks from UCB, downloads the artifact from them, deploys them to the output folder while adding static accompaniments, should there be any.

Now I can be confident I will always deploy the correct binary! Also it was a good exercise to learn how filesystems are treated in Python.

Side note: this project can be deployed on Docker! If you cannot risk your main filesystem (as you should not,) I reckon this is the safest way to run this project.

Side side note: this code is not tested outside of my environment, so it may crash on you - if you experience any, issue reports are always welcome!