But What is Container Orchestration?
By: Zain Raza
Tags: Containers, Distributed Systems, Docker, DevOps, Django

Like any great orchestra, great software architectures often rely on multiple individual pieces coming together in harmony. Photo by Vince Veras on Unsplash

Container Orchestration — Your Ticket to Distributed Computing

Like any great orchestra, great software architectures often rely upon having multiple pieces coming together in harmony. Just as the conductor’s baton guides the musicians in an orchestra, a system that leverages distributed computing will have each of its pieces share expectations for how they will communicate, and share the work of running our application.

Container orchestration simply refers to how you coordinate the different pieces of your application — in the context of this blog, the Docker containers — to work together in harmony as one distributed system. When done right, it can deliver amazing performances for end users.

When I first started learning about container orchestration, it was harder than trying to find tickets for the Harry Potter Concert Series during the COVID-19 lockdown in Boston— but never fear! It’s easier than it sounds, I promise.

The Problem with Docker Compose

Docker Compose is a tool for running multi-container applications on Docker. Image source: Docker Compose GitHub Repository

Many software developers who use Docker will know that in their local environment, Docker Compose is a great way to run multi-container applications — only problem is that it only works locally (for now). So, how will you take your amazing product out from behind the curtain of your local machine, and out in front of your intended users?

Learning Objectives

I recently encountered this exact problem while deploying Conscious Consumer, a web application I built with a Django back-end and PostgreSQL database. I’ll share the approach I eventually used in this blog post, and by the end I intend to show you:

How to Configure Django Settings for Deployment on CapRover How to Write a Shell Script to Help Setup your Container Environment How to Connect Multiple Apps on CapRover for your Project

By the end of the blog I hope to show you an example of what I mean, when I was talking about distributed computing earlier!

PREREQUISITE: In this blog, I assume you already know how Docker works for single container apps, and you have a solid understanding of Django. But if you don’t yet understand — no problem! I recommend reading the explanation on Docker’s website to learn more first, and then you should be all set to come back and continue with this reading. Resources on the Django web framework will be linked at the bottom, since it is a much broader topic.

How To Deploy a Django-PostgreSQL Application on CapRover (Example)

PREREQUISITES:

Buy a Website Domain Name, from a service such as Namecheap or Name.com. We’ll be using a virtual private server (VPS) to host our site, since it’s more practical than having to worry about maintaining our own physical server. I’ll be using DigitalOcean for this example, since they specifically offer CapRover instances. Set up your Personal PaaS on CapRover, using this tutorial.

The Steps

Step 1: Create a separate Django settings file for CapRover

Make sure the values that are placed in the DATABASES setting match what will we enter into the CapRover PostgreSQL instance (coming next in the tutorial)!

As a good rule of thumb, I like to have separate settings files in my Django project, so that it makes it easier to separate concerns over what value a certain setting should have in a given environment. On the left, you can see what the file structure of the Conscious Consumer GitHub repo looks like when viewed in VSCode.

The most important thing to focus on in your CapRover-specific settings module is the DATABASES setting. We’ll come back to this soon, but suffice to say whatever values are coming from your .env are important, because they need to match the values entered in the PostgreSQL database you create on CapRover later!

Step 2: Dockerize the Django project

Dockerfile for the Django Project

In the next step I added a Dockerfile to the repository, in the same directory as the requirements.txt and manage.py (you don’t have to though, it’s just important to keep in mind when copying your files into the container image).

As the comments show, the Dockerfile then goes ahead with installing requirements, copying the Django files into the container image. The container image is just a fancy tarball that holds everything needed to support your application in other environments — in this case, CapRover!

Note that you must have a .env file in your repository, if you are following along with the approach here. Even though this file doesn’t appear on the GitHub repository, it’s super important — because this is the file your container will look at in order to set values for the following database environment variables, as shown in the settings file in the step above:

POSTGRES_DB

POSTGRES_USER

POSTGRES_PASSWORD

POSTGRES_HOST

Step 3: Create an entrypoint.sh script for the Docker container

This file basically tells Docker a bunch of commands to run as soon as the container is built.

In the next step, go ahead and make an entrypoint.sh script. This file basically tells Docker a bunch of commands to run as soon as the container is built.

Here, I used the script to basically setup the Django server. It sets the DJANGO_SETTINGS_MODULE setting to use our CapRover settings from Step 1, to run the database migrations (in future steps we’ll need to make sure that the database is already running when this command executes). Then, it gathers all the static files. If the application had any data migrations or fixtures, we could do that here as well — but since the web application doesn’t have any persistent data, I just went ahead and added the command to run Django server!

Note that you don’t really necessarily need this file — you could technically do all of this in the Dockerfile, using the RUN commands. However, using this approach you will avoid adding extra layers to the Docker image (1 line of code = 1 layer after all), which in effect improves performance!

Step 4: Create a Captain-Definition file

This little JSON is all you need to be ready to deploy on CapRover.

If you made it this far — great! Our Dockerfile has already done most of the work in standardizing the environment we need to run this Django project.

That means that with as little as just 4 lines of JSON code, the captain-definition file gives us all we need to now deploy on CapRover!

Step 5: Create the PostgreSQL app on CapRover

As stated earlier, we are ready to deploy the Django project on CapRover — except if we want it to actually work, we’ll need to create a PostgreSQL instance on our account first!

Fortunately, you can do this easily using the PostgreSQL template provided by CapRover. Please watch this short 1-minute video tutorial to see how:

The specific environment variables you need to add to the PostgreSQL instance are shown below:

CAPROVER=True

CR_SECRET_KEY=YOUR_KEY

CR_HOSTS=demo-django.YOUR_CAPROVER_DOMAIN,CUSTOM_DOMAIN

CR_DB_NAME=XXX

CR_DB_USER=XXX

CR_DB_PASSWORD=XXX

CR_DB_HOST=srv-captain--demo-db

CR_DB_PORT=5432

When settings these variables, the TL;DR is basically to make sure there’s consistency between what values are on here, and what’s on the CapRover settings module in your Django project:

Make sure CR_SECRET_KEY is equal to your Django SECRET_KEY

And that you include your website domain in CR_HOSTS

CR_DB_NAME, CR_DB_USER, and CR_DB_PASSWORD should all be the same as what you setup when you made the PostgreSQL instance (which in turn should equal the values for the equivalent keys you have in the Django DATABASES setting).

CR_DB_HOST from CapRover and DATABASES[“HOST”] must be equal.

Step 6: Create the Django app on CapRover

If you would like to use the web dashboard, you can go to the Apps tab of CapRover, and use the “Create New App” form at the top of the page.

Almost there! Now all you will need to create a new app on your CapRover account for the Django application. If you would like to use the web dashboard, you can go to the Apps tab of CapRover, and use the “Create New App” form at the top of the page. Leave the “Has Persistent Data” box unchecked, as it does not apply to the web app (since our database will hold persistent data for us already).

Alternatively, you could do all this via the Terminal — however I do recommend CapRover’s dashboard, as you will need to use it anyway in order to enable HTTPS on the site we make.

The “Enable HTTPS” button can be found on the page you see after clicking on one of your CapRover apps.

Step 7: Deploy your Django Project!

If you are using the web dashboard approach mentioned above, you will need to upload a Tarfile of the Django project to your app on CapRover.

To make a Tarfile on a Unix-based machine (i.e. something that runs macOS or a Linux kernel), we can use the “tar” command via Terminal. The syntax of this command is as follows:

tar -czf {{target.tar}} {{path/to/directory being zipped}}

For this example, we simply need to navigate to whichever directory holds the captain-definition file, and enter something like the follow (obviously, you can name the tarfile being created anything you want):

tar -cvf conscious.tar ./

We can now take this file, and upload it to our CapRover app in the “Deployment” tab.

We can now take this file, and upload it to our CapRover app in the “Deployment” tab.

All you will need to do is to drag the Tarfile (with the .tar extension) into the appropiate section of this page.

All you will need to do is to drag the Tarfile (with the .tar extension) into the appropiate section of this page.

If everything works, your build logs will something similar to the following!

Build Logs on CapRover

The log messages for the application will look similar to the following (note this output matches the commands we put in entrypoint.sh)!

App Logs on CapRover

Ta-da! You will now be able to see your application on its provided URL address.

Review Time!

Architecture of our Orchestrated Docker containers on CapRover

You may be asking, “hey that’s cool, but how do I even understand what’s happening?”

Great question! Let’s use the diagram to the left as a tool to review some big-picture concepts:

At the lowest level of this stack, we have the VPS — this is the physical machine hosting our site. On the software side, we basically created two Docker containers, also called services, for the two different pieces of our application — Django, which programs our server and provides the MVC pattern, and PostgreSQL, which gives us a relational database. Through configuring the port numbers and providing consistent environment variables for both containers, we gave them shared expectations about how to “communicate” with each other. The web application knows how to send requests to the database over HTTP, and in response PostgreSQL can send back the appropriate data — thus, our container orchestra is a distributed system! Both containers run their own web or database server, and if either one goes down it doesn’t necessarily bring down the entire network.

Further Reading

Take a step back and appreciate your big brains, since container orchestration, distributed systems, and DevOps in general are huge topics! When you’re ready to keep moving forward, here are some resources to feed your curiosity: