How to quickly setup an infra [Part 1]
Tags: docker, devops
Published: 2024-03-20 17:37:05

This is the first part of a serie where I would like to present a simple way to setup an infra for either

- an ambitious personal project
- a small SME/a startup in it's infancy. 

## Why this serie?

Good question! 😁 You could go with the heroku-like approach, or the hyperscaler one. I am sure there are tons of other tutorials out there to help you do so. 

But the approach that I am going to present is ultra cheap (only 1 VM needed) and has some pedagogical merits 😊.

Plus in an era where [companies](https://www.infoworld.com/article/3712861/why-companies-are-leaving-the-cloud.html) are more and more [moving away from the cloud](https://world.hey.com/dhh/why-we-re-leaving-the-cloud-654b47e0), you never know, it might proves useful 😋.

## What are we aiming at?

By the end of this serie, you should have a stack looking like so


![What this serie is aiming at: self-managed infra for small SME/startup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/orhxgwsrwu6rn8yktsf7.png)

This stack includes:

- <a href=https://doc.traefik.io/traefik/ class="external-link" target="_blank">Traefik</a> as a <a href=https://en.wikipedia.org/wiki/Reverse_proxy class="external-link" target="_blank">reverse proxy</a>
-  <a href=https://www.postgresql.org/ class="external-link" target="_blank">PostgreSQL</a> as a sql db and <a href=https://www.pgadmin.org/ class="external-link" target="_blank">pgAdmin</a> as its admin interface
- <a href=https://www.bookstackapp.com/ class="external-link" target="_blank">Bookstack</a> and its associated db for documenting your product
- <a href=https://www.keycloak.org/ class="external-link" target="_blank">keycloak</a> and  <a href=https://doc.traefik.io/traefik/middlewares/http/forwardauth/ class="external-link" target="_blank">traefik forward-auth</a>    to protect apps that cannot implement authentication without keycloak
- <a href=https://min.io/ class="external-link" target="_blank">minio</a> back and front to stored/exchange data
- <a href=https://github.com/mcuadros/ofelia class="external-link" target="_blank">Ofelia</a> as a job scheduler
- <a href=https://github.com/pypiserver/pypiserver class="external-link" target="_blank">Private pypi</a> to store your precious private python libraries between collegues.
- <a href=https://dozzle.dev/ class="external-link" target="_blank">dozzle</a> for log aggregation
- <a href=https://github.com/louislam/uptime-kuma class="external-link" target="_blank">uptime-kuma</a> for monitoring and alerting

I'll talk (python) web app setup in another subsequent serie 😊.

## Where to start? 

Well, thanks to the work we did at [EcoAct](https://eco-act.com/), we got you covered:

- This [documentation](https://ecodev-doc.lcabox.com/cookiecutters/infra/) page and the ones related are the content I am going to present in this serie. This post corresponds to [this page](https://ecodev-doc.lcabox.com/cookiecutters/infra/setup/setup_ubuntu_vm/).
- The associated code can be found in [this](https://github.com/FR-PAR-ECOACT/ecodev-infra) open source (MIT licence) repo.

## What are we going to learn today? 

Setting up a VM with one simple script! Yes I know, it surely could be done with [ansible](https://www.ansible.com/), but fur such a simple use case it seems overkill (plus one has to install ansible before being able to use it, by contrast with bash 😂).

TL DR: clone the repo, launch the setup script. The end! 😂 Now the details.

## Actual setup

There are countless blog posts to help you decide which cloud provider is the best for you, and I won't add to this literature here 😊. 

 I therefore just assume you have a Virtual Machine (VM) Cloud setup with ssh access. 

Here I provide a simple <a href=https://www.freecodecamp.org/news/bash-scripting-tutorial-linux-shell-script-and-command-line-for-beginners/ class="external-link" target="_blank">bash script</a> to setup a freshly bought VM, assuming an <a href=https://ubuntu.com/ class="external-link" target="_blank">Ubuntu</a> Operating System (this OS choice is important for the firewall setup).

### Launching the setup script

After having accessed with ssh to your VM, just `git clone` [ecodev-infra](https://github.com/FR-PAR-ECOACT/ecodev-infra). Then in the main folder, launch 
```
make setup-vm <YOURUSER> 
```
Where `<YOURUSER>` is the username you used for connecting to the VM. The bash script executed will

- Install docker and related components
- Add `<YOURUSER>` to the `sudo` and `docker` group of the VM (might need to disconnect and reconnect **at the end of the setup** for this to take effect) 
- Setup <a href=https://doc.ubuntu.org/ufw class="external-link" target="_blank">ufw</a> (uncomplicated firewall) to work with docker (more on that below) 
- block all but 22, 80 (tcp only) and 443 (tcp only) ports.
- Setup a decent `history` (<a href=https://www.digitalocean.com/community/tutorials/how-to-use-bash-history-commands-and-expansions-on-a-linux-vps class="external-link" target="_blank">english resource on the topic</a>) memory size.

### 🚨 **Technical topic incoming** 🚨 

Why going into all this trouble with ufw?

As explained <a href=https://www.baeldung.com/linux/docker-container-published-port-ignoring-ufw-rules class="external-link" target="_blank">here</a> way better than I could, the standard **ufw setup won't forbid access to docker exposed ports on your VM, even ports explicitely blocked with an ufw rule!!**.
    
This has to do with technicalities related to  <a href=https://help.ubuntu.com/community/IptablesHowTo class="external-link" target="_blank">iptables</a>  explained in the linked posts. 
    
To make ufw work with docker, one **has** to use this (awesome!) <a href=https://github.com/chaifeng/ufw-docker class="external-link" target="_blank">solution</a>. This is the reason of the `sudo cp after.rules /etc/ufw/after.rules` line in the `setup.sh` script.


You might be as curious as I was and wondering why there is no Open Source docker container responsible for dealing with firewall issues. 

To the best of my knowledge, this has to do with the fact that firewell related stuff is really low level and needs to live close to the VM kernel. <a href=https://www.reddit.com/r/docker/comments/cmuxcs/how_do_i_deploy_a_firewall_in_a_container/ class="external-link" target="_blank">One example</a> out of the numerous conversations on the topic. 

### The script in its whole splendor

If you're too curious 😋 to wait to go inspect the [EcoAct infra repo](https://github.com/FR-PAR-ECOACT/ecodev-infra) (though I still highly encourage you to do so), here is the srcipt 

```
#!/bin/bash

# This is a VM setup for ubuntu only OS
# Provide as argument your username (in order to add you to docker and sudo group)

echo "Installing docker"
sudo apt-get update
sudo apt-get install ca-certificates curl gnupg make
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo   "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
       "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" |   sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

echo "Add access rights for current user" $1
sudo usermod -a -G docker $1
sudo usermod -a -G sudo  $1

echo "Setting up ufw to be docker compatible"
sudo ufw enable

echo "Edit ufw rules to be docker compatible"
# ref: https://github.com/chaifeng/ufw-docker?tab=readme-ov-file#solving-ufw-and-docker-issues
sudo cp after.rules /etc/ufw/after.rules

sudo ufw reload
echo "Setup ufw: block all but 22, 80 (tcp only) and 443 (tcp only) ports"
sudo ufw default deny
sudo ufw allow 22
sudo ufw route allow proto tcp from any to any port 80
sudo ufw route allow proto tcp from any to any port 443
echo "Turning ufw log"
# ufw is known to be glutton in terms of memory
sudo ufw logging off

echo "Restarting docker"

sudo systemctl restart docker

echo "Setup decent history size"

echo "export HISTSIZE=100000" >> .bashrc
echo "export HISTFILESIZE=100000" >> .bashrc
echo "export HISTCONTROL=ignoreboth:erasedups" >> .bashrc

echo "All done!"
```

## I like what I read!

That's awesome! If you want to encourage me to continue the serie, do not hesitate to ⭐ [ecodev-infra](https://github.com/FR-PAR-ECOACT/ecodev-infra), best way to aknowledge the hard work we put at [EcoAct](https://eco-act.com/) to create this repo. 

## What's the (hopefully no so the distant) future?

We would like to do a full pythonic equivalent of [Tiangolo's work](https://github.com/tiangolo/full-stack-fastapi-template). Of course Tiangolo repo is **the** place to start if you're new to modern python, but the work we hope to build will rely on [dash](https://dash.plotly.com/) for the frontend part, to have the full web stack within python! Plus the infra bricks are a little more numerous here, since we want to do more than just we app development (remember: build a full-fledged infra stack for a small structure) 

The first brick is [there](https://github.com/FR-PAR-ECOACT/ecodev-core) and the associated documentation [there](https://ecodev-doc.lcabox.com/libraries/core/). But one step at a time! Let's first setup the infra, then I'll talk python! 🥰