Deploy app using FastApI, Docker and AWS | Towards Data Science

You can visit the final app on rdok.net. The code is open-source on GitHub.This is mostly a procedural article. The purpose is to describe the full process of creating an app without bombarding the reader with code snippets.
What’s the app about? The app is called Random Dose of Knowledge and looks like this
There are 4 buttons, each one from a specific subreddit. When the user clicks on a button, a random post of this subreddit along with the top comments is displayed. I suppose you are familiar with Reddit, but it doesn’t matter if not. Just think of the app as a place where the user can keep clicking buttons to learn cool facts. At the bottom of the page, there exists an interactive plot that displays the cumulative count of the button clicks overtime per button.
Potential reader 1: Interested only in the process. Would like to see in a very abstract and fast way the steps to create such a project.
If you are reader 1, don't read content in blocks like this, don’t read code snippets.
Estimated time: 5 minutes.
Potential reader 2: Interested in specific parts that he doesn’t know. How to use HTML with FastAPI? How to dockerize the project? How to deploy on EC2?
If you are reader 2, do a quick scroll of the article to find what you want to learn and/or control+f your keywords.
Estimated time: Varies.
Potential reader 3: Interested in the process and code. Would like to recreate this app for himself.
If you are reader 3, clone the GitHub code. Read everything in the article. Read it again. Reproduce it.
Estimated time: at most 8 hours.
Frameworks/Software/Technologies/Platforms used:
Git
Miniconda
FastAPI
SQLAlchemy
HTML, CSS
Jinja2
Reddit API
Python (pandas, Plotly)
Docker
AWS EC2, ECR
First, create a folder that will host your application. Mine is called random-dose-of-knowledge. Download anaconda or miniconda to create a virtual environment. Go inside the folder, open a terminal, and type:
git init. # initialize git repositoryconda create -n rdok python=3.8 # create a virt. env. called rdokconda activate rdok # activate it
Create a file called requirements.txt and inside it, write:
fastapiuvicornSQLAlchemypandasplotlyaiofilesrequestsJinja2python-multipart
In terminal type: pip install -r requirements.txt
We now have git and a virtual environment with all the packages we need.
In this section I give the file structure, explain the files, and we take a peek at the most important parts of some files. The complete files are on GitHub.
random-dose-of-knowledge/    |-- .gitignore    |-- Dockerfile    |-- requirements.txt    |-- app/    |-- __init__.py    |-- main.py        |-- db/        |    |-- __init__.py        |    |-- database.py        |    |-- random-knowledge.db        |-- models/        |    |-- __init__.py        |    |-- sql_table_entities.py        |-- utils/        |    |-- __init__.py        |    |-- reddit_api.py        |-- templates/        |    |-- base.html        |    |-- index.html        |-- static/        |    |-- css/        |    |    |-- style.css        |    |-- images/        |    |    |-- favicon.png
Contains all the files and folder that will not be tracked by git. Here it is
Includes commands to assemble a docker image. More on in later
Include this empty file in directories to make python see them as packages
Our database
Script to create a DB connection using SQLAlchemy
Includes definitions of the tables of our database
There exists a one-to-many relationship between the tables ‘posts’ and ‘comments’, as a post can have multiple comments.
Below are the essential parts of the main.py for inserting data in our tables. Give extra attention to line 25.
On line 4 with subreddit: str = Form('todayilearned') , we define a parameter called subreddit of type str . Its value will be the form output of the button click. For example, if the user clicks on the ‘Science’ button, the output of the form will be ‘science’. <form action="/" method="post"> <input type="hidden" name="subreddit" value="science" /> <button type="submit"> Science </button></form>
The default value, as can be seen on line 4 is ‘todayilearned’.
The file where Python work is done. Define functions to connect to the Reddit API, get requests, pre-process data extracted from Reddit posts and comments. Also, define the function that creates the plot of the app.
How to use Reddit API. Go here. Scroll to the bottom. Click ‘create another app’. Insert a name, choose script, leave description and about url empty, enter a website (doesn’t matter which--can be your personal website, github or linkedin) to the redirect uri. Your CLIENT_ID is the ‘personal use script’. Your SECRET KEY is the ‘secret’. In order to connect you also need your REDDIT_USERNAME and REDDIT_PASSWORD. Save the latter in a .txt file
These files are in the templates folder which needs to be defined in main.py (see below main.py script line 2)It is possible to create a full website with FastAPI and not just return json files. My app is a one-page website. The base.html is the base file that is included in all the other pages of the website (in my case, just the index.html). The index.html is the landing page of the app.
FastAPI + Jinja2 + HTML. To return the results of a FastAPI function to an HTML page:1. define response_class in the decorator of the function@app.get('/', response_class=HTMLResponse)
2. define a parameter in the function with type Requestdef my_function(request: Request):
3. return a TemplateResponse specifying the landing .html page, the request parameter and the variables to pass
return templates.TemplateResponse('index.html', { 'request': request, 'variable1': var_1, 'variable2': var_2})
4. use the passed variables in HTML using Jinja2For example if var_1 was a dataframe we can use it like this:{% for i, row in var_1.iterros() %} <div class="row"> {{row['body']}} </div>{% endfor %}
The script to call and run the application. The below code includes the minimum essentials (except imports) of the main.py file
Line 12 runs the application called ‘app’ as defined in line 1 of the file ‘main’. The app will be available locally on localhost:8000
At this point the app is ready and you can run it locally: python main.py
Install docker.
Refer to the file structure to understand the paths. Create a Dockerfile:
# install python in the containerFROM python:3.8.8-slim # copy the local requirements.txt file to the # /app/requirements.txt in the container# (the /app dir will be created)COPY ./requirements.txt /app/requirements.txt# install the packages from the requirements.txt file in the containerRUN pip install -r /app/requirements.txt# expose the port that uvicorn will run the appEXPOSE 8000# copy the local app/ folder to the /app fodler in the containerCOPY app/ /app# set the working directory in the container to be the /appWORKDIR /app# execute the command python main.py (in the WORKDIR) to start the appCMD ["python", "main.py"]
Build a docker image from the Dockerfile. Go to the folder where the Dockerfile is located. Type:
docker build -t rdok:1.1 .
Don’t forget the dot! The name of the image is rdok and the tag is 1.1
View the image:
docker images
Create a container from the image:
docker run --name rdok -d -p 80:8000 rdok:1.1
with name, we specify the name of the containerwith -d, the container will run in detached modewith -p 80:8000, we bind the local port 80 to the docker port 8000rdok:1.1 is the name of the image
Bonus. We can connect the local app folder with the docker app folder. Changes in the docker app will be copied in the local folder and vise versa. The container will also be persistent. If we stop the container or even delete it, the changes will stay. To do this add the following to the docker run command:
-v absolute/path/to/the/local/app:/app
On the left side is the absolute path on our local machine where our /app/ folder resides. On the right part is the /app/ folder we previously created (in the Dockerfile) in the container.
Check the running container:
docker ps
The app is running! Go to localhost:80 or simple localhost.
Make an account on AWS and connect. Follow the steps in the image to create an EC2 instance. This will be our server that will host our app:
In a terminal navigate to the .pem key and type:
chmod 400 my_key_pair.pem
If you are on a windows machine do this instead.
Continue by connecting to your instance. In your terminal (you are inside a folder with the .pem key) run the last command on the below image.
You are inside your instance now. Run the following commands to install Docker in the instance:
sudo yum update -ysudo yum install dockersudo service docker startsudo usermod -a -G docker ec2-usersudo dockerd
Reboot the instance now and follow the same steps as described above to connect once again.
Type:
systemctl start dockersudo service docker start
Docker is installed!
Before proceeding, download AWS CLI, and configure by authenticating with your AWS credentials.
After authenticating, go to your ECR, select the repository and click ‘view push commands’.
A window will open and guide you on how to push your LOCAL docker image to the ECR instance.
The commands are to be run on your local machine:
aw ecr get-login-pa... Authenticates the connection between our local machine and ECR.docker build.. Build the image. We ALREADY built a docker image of our app in the Docker section.docker tag.. It copies and renames the local image. Type docker imagesto see it.docker push.. Push the copied image to the ECR repository.
aw ecr get-login-pa... Authenticates the connection between our local machine and ECR.
docker build.. Build the image. We ALREADY built a docker image of our app in the Docker section.
docker tag.. It copies and renames the local image. Type docker imagesto see it.
docker push.. Push the copied image to the ECR repository.
Our image now rests in our ECR repository. Let’s pull it to our EC2 repository.
Go to your terminal that is inside your EC2 instance.
We first have to authenticate the connection between EC2 and ECR. Run the same aw ecr get-login-pa... command as you did in your local machine and then do docker pull name-of-the-copied-image.
Now our image is inside our server! Type docker images to see it.
Let’s run it:
docker run --name my-app -d -p 80:8000 IMAGE_NAME:TAG
Go to the AWS website on your EC2 dashboard, click the repository, and go to the ‘Security’ tab. Click on the link under the ‘Security groups’. Next, click ‘Edit inbound rules’. Click ‘Add rule’. Type: HTTP, Protocol TCP, Port Range 80, Source Custom 0.0.0.0/0 and save.
Your rules should look like this:
To visit your app go to Public IPv4 address, but write http instead of https.
Congratulations! You reached the end of this article. I hope you learned as many things as I did with this project.
As a final note, I share again the GitHub link for this project.
You can connect with me on LinkedIn.
See you soon!