Deploying Django Application on AWS with Terraform - Part 1
Tags: devops, aws, python
Published: 2024-04-01 05:25:32

Hi everyone, I am back with another project. This time we will be working on a CICD pipeline, deploying our application to our AWS account. In this project we will use Terraform to deploy our Django application to our AWS account.

Below are the tools and services that we will use and their categories.

https://gist.github.com/apotitech/eefebc53ce4c154f180defdc758d385f

Before we get into today's hands on writeup, let us start with the different parts of our project from start to finish.

### Parts of Project

1. [Minimal Working Setup](#part-1-minimal-working-setup)
2. [Connecting PostgreSQL RDS](#part-2-connecting-postgresql-rds)
3. [GitLab CI/CD](#part-3-gitlab-cicd)
4. [Namecheap Domain + SSL](#part-4-namecheap-domain-ssl)
5. [Celery and SQS](#part-5-celery-and-sqs)
6. [Connecting to Amazon S3](#part-6-connecting-to-amazon-s3)
7. [ECS Autoscaling](#part-7-ecs-autoscaling)

We are starting with the first part today

## Minimal Working Setup

In this part, we will go through first - the setup of our AWS account, then we will create our Terraform project, and lastly we will define resources for our web application. At the end, we will deploy our Django application on our AWS ECS. We will access our app using our Load Balancer URL.

![scrnli3252024121301 AMgif](https://miro.medium.com/v2/resize:fit:720/format:webp/1*gSy-S5RkI_G7JZ0TR-c4Ow.png)

## Creating Django project

Let’s start with our Django application. Create a new folder and initialize a default Django project.

```bash
$ mkdir django-aws && cd django-aws
$ mkdir django-aws-backend && cd django-aws-backend
$ git init --initial-branch=main
$ python3.10 -m venv venv
$ . ./venv/bin/activate
(venv) $ pip install Django==3.2.13
(venv) $ django-admin startproject django_aws .
(venv) $ ./manage.py migrate
(venv) $ ./manage.py runserver
```

Now that our Django server is setup, let's check our Django greeting page at [http://127.0.0.1:8000](http://127.0.0.1:8000/ "http://127.0.0.1:8000"), ensure that Django is running, and kill the development server.

![1](https://miro.medium.com/v2/resize:fit:720/format:webp/1*TEx96-8c2q5fiY5bv-5WPQ.png)

Next, we are going to dockerize our application. First, we will add a `requirements.txt` file to the Django project:

```python
Django==3.2.13
```

For testing purposes, enable debug mode and allow all hosts in our `settings.py`

```python
DEBUG = True

ALLOWED_HOSTS = ['*']
```

To containerize our app, we need to have a dockerfile. So next, we add a `Dockerfile` in our working directory:

```dockerfile
FROM python:3.10-slim-buster

# Open http port
EXPOSE 8000

ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWRITEBYTECODE 1
ENV DEBIAN_FRONTEND noninteractive

# Install pip and gunicorn web server
RUN pip install --no-cache-dir --upgrade pip
RUN pip install gunicorn==20.1.0

# Install requirements.txt
COPY requirements.txt /
RUN pip install --no-cache-dir -r /requirements.txt

# Moving application files
WORKDIR /app
COPY . /app
```

Now let us go ahead and build and run our docker container locally.

```bash
$ docker build . -t django-aws-backend
$ docker run -p 8000:8000 django-aws-backend gunicorn -b 0.0.0.0:8000 django_aws.wsgi:application
```

Let's go to pur localhost port [http://127.0.0.1:8000](http://127.0.0.1:8000/ "http://127.0.0.1:8000") page and verify that we have successfully built and run the docker image with a Django application. You should see exactly the same greeting page as for the `runserver` command.

There may be some files that we do not want to push to our git repo. For that. let’s add a `.gitignore` file:

```gitignore
*.sqlite3
.idea
.env
venv
.DS_Store
__pycache__
static
media
```

Next, we will continue with our git steps. Now we commit our changes:

```bash
$ git add .
$ git commit -m "initial commit"
```

Good job making it this far.

For now, we are done with the Django part. In the next steps, we deploy our application on AWS. But first, we need to configure our AWS account.

We need to create credentials for AWS CLI and Terraform. So we’ll create a new user with administration access to the AWS account. This user will be able to create and change resources on your AWS account.

First, we will go to the [IAM](https://console.aws.amazon.com/iam/home "IAM") service, select the “Users” tab, and click “Add Users”.

![eewr](https://miro.medium.com/v2/resize:fit:700/1*vCpaRiNkSs5-HwuxyFr7Tw.png)

Enter Username and choose the ‘Access key — Programmatic access’ option. This option means that your user will have ‘Access key’ to use AWS API. Also, this user won’t be able to sing in the AWS web console.

![rrft](https://miro.medium.com/v2/resize:fit:700/1*TeGa5toCd388YE5GhjGRrw.png)

Select the “**_Attach existing policies directly_**” tab and select “**AdministratorAccess**.” Then click next and skip the “Add tags” step.

![ccxsd](https://miro.medium.com/v2/resize:fit:700/1*STRH0UO9iM-JOW7knJyo6Q.png)

Review user details and click “**_Create user_**.”

![ggter](https://miro.medium.com/v2/resize:fit:700/1*-gcZo9jg3cRTulBK7P-FAg.png)

Yay, we have successfully created our user!

Now we need to save your **Access key ID** and **Secret access key** in some safe place. Be aware of committing these keys in public repositories or other public places. Anybody who owns these keys can manage your AWS account.

![ffrrer](https://miro.medium.com/v2/resize:fit:700/1*nbqUJ3zj__YidLyT15wggg.png)

Now we can configure AWS CLI and check our credentials. We will use the `us-east-2` region in this guide. Feel free to change it.

Now we can configure AWS CLI and check our credentials. We will use the `us-east-2` region in this guide. Feel free to change it.

```bash
$ aws configure
AWS Access Key ID [None]: AKU832EUBFEFWICT
AWS Secret Access Key [None]: 5HZMEFi4ff4F4DEi24HYEsOPDNE8DYWTzCx
Default region name [us-east-2]: us-east-2
Default output format [table]: table
$ aws sts get-caller-identity
-----------------------------------------------------
|                 GetCallerIdentity                 |
+---------+-----------------------------------------+
|  Account|  947134793474                           |  <- AWS_ACCOUNT_ID
|  Arn    |  arn:aws:iam::947134793474:user/admin   |
|  UserId |  AIDJEFFEIUFBFUR245EPV                  |
+---------+-----------------------------------------+
```

Remember your `AWS_ACCOUNT_ID`. We'll use it in the next steps.

Now we are all set up to create Terraform project!

## Creating Terraform Project

Let’s create a new folder `django-aws/django-aws-infrastructure` for our Terraform project.

```bash
cd .. 
mkdir django-aws-infrastructure && cd django-aws-infrastructure  
git init --initial-branch=main
```

Let us add our `provider.tf`:

```hcl
provider "aws" {
  region = var.region
}
```

Here, we defined our [AWS](https://registry.terraform.io/providers/hashicorp/aws/latest/docs "AWS") provider. We use [Terraform variable](https://www.terraform.io/language/values/variables "Terraform variable") for specifying an AWS region. Let’s define `region` and `project_name` variables in the `variables.tf` file:

```hcl
variable "region" {
  description = "The AWS region to create resources in."
  default     = "us-east-2"
}

variable "project_name" {
  description = "Project name to use in resource names"
  default     = "django-aws"
}
```

Now, we will run `terraform init` to create a new Terraform working directory, download the AWS provider, and everything else that will be needed.

Now we are ready to create resources for our infrastructure.

### Resources we will use

Here is the plan of the services we wull use to configure our project

https://gist.github.com/apotitech/6c1ac26ee528eacb7dbeae7635253058

#### Elastic Container Repository, ECR

```hcl
resource "aws_ecr_repository" "backend" {
  name                 = "${var.project_name}-backend"
  image_tag_mutability = "MUTABLE"
}
```

Next, we will run `terraform plan`. We'll see that Terraform is going to create an ECR repository.

```bash
Terraform will perform the following actions:
  # aws_ecr_repository.backend will be created
  + resource "aws_ecr_repository" "backend" {
      ...
    }
Plan: 1 to add, 0 to change, 0 to destroy.
```

If the plan looks good, we will go ahead and run `terraform apply`. We will be prompted to accept or refuse the plan. Type `yes` to confirm changes.

```bash
aws_ecr_repository.backend: Creating...
aws_ecr_repository.backend: Creation complete after 1s [id=django-aws-backend]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
```

Great, our repository is created. Now, let’s push our Django image to this new registry. Before we do that, we will need to build an image with tag, below is mine `${AWS_ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/django-aws-backend:latest`, authorize in the ECR, and push an image:

In my own case, my account ID is `94713479347` and I will use the `us-east-2` region

```bash
$ cd ../django-aws-backend
$ docker build . -t 947134793474.dkr.ecr.us-east-2.amazonaws.com/django-aws-backend:latest
$ aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin 947134793474.dkr.ecr.us-east-2.amazonaws.com
$ docker push 947134793474.dkr.ecr.us-east-2.amazonaws.com/django-aws-backend:latest
```

#### Network

Now, let’s create a network for our application. First we create our variables.tf file. Add this block to the `variables.tf` file:

```hcl
variable "availability_zones" {
  description = "Availability zones"
  default     = ["us-east-2a", "us-east-2c"]
}
```

Next, we will create a `network.tf` file with the following content:

https://gist.github.com/apotitech/ba6d140d902cef8e7038118829335d7e

Below are the resources that we will be creating. We’ve defined the following resources with our code:

- [Virtual Private Cloud](https://aws.amazon.com/vpc/).
- [Public and Private subnets](https://docs.aws.amazon.com/vpc/latest/userguide/configure-subnets.html) in different [Availability zones](https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/RegionsAndAZs.html)
- [Internet Gateway](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Internet_Gateway.html) for internet access for our public subnets.
- [NAT Gateway](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html) for internet access for our private subnets.

Next we will apply what we just coded. Run `terraform apply` command to apply changes on AWS.

### Load Balancer

We continue our infrastructure building. Next we will create our load balancer `load_balancer.tf` file with the following content:

https://gist.github.com/apotitech/da3f7ba10640b6c1809c2894c7525d62

Let us look at the resources that the code below will create:

- [Application Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html)
- [LB Listener](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-listeners.html) to receive incoming HTTP requests.
- [LB Target group](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-target-groups.html) to route requests to the Django application.
- [Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) to control incoming traffic to load balancer.

Before we proceed, we want to know the load balancer URL. For that we need our code to output it in our terminal. For that we will add a `outputs.tf` file with the following code and run `terraform apply` to create load balancer and see its hostname.

```hcl
output "prod_lb_domain" {
  value = aws_lb.prod.dns_name
}
```

We will see our ALB domain in output.

```bash
Outputs:
prod_lb_hostname = "prod-57218461274.us-east-2.elb.amazonaws.com"
```

We can now check our this domain in our browser. It should respond with `503 Service Temporarily Unavailable` error because there are no targets associated with the target groups we created yet.

In the next step, we'll deploy the Django application that will be accessible by this URL.

### Application

Last but not least, we’ll create the application using the [ECS Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). For this, we will add a `ecs.tf` file with following content:

https://gist.github.com/apotitech/bfb372e92d2b88fee935ee39a8f68ed7

Also, we will add the `ecs_prod_backend_retention_days` variable to the `variables.tf` file:

```hcl
variable "ecs_prod_backend_retention_days" {
  description = "Retention period for backend logs"
  default     = 30
}
```

then we add a container definition in a new file `templates/backend_container.json.tpl` and run `terraform apply`.

```json
[
  {
    "name": "${name}",
    "image": "${image}",
    "essential": true,
    "links": [],
    "portMappings": [
      {
        "containerPort": 8000,
        "hostPort": 8000,
        "protocol": "tcp"
      }
    ],
    "command": ${jsonencode(command)},
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {
        "awslogs-group": "${log_group}",
        "awslogs-region": "${region}",
        "awslogs-stream-prefix": "${log_stream}"
      }
    }
  }
]
```

Our code will be creating:

- [ECS Cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html)
- [ECS Task Definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html)
- [ECS Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) to run tasks with the specified definition in the ECS cluster
- [IAM Policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html) to allow tasks access to resources.
- [Cloudwatch Log](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html) group and stream for log collection.

Now, go to the our [ECS AWS Console](https://us-east-2.console.aws.amazon.com/ecs/home) and look at our running service and tasks.

![img](https://miro.medium.com/v2/resize:fit:720/format:webp/1*3NvXTuELvOvVS1SgAg9CTQ.png)

![ffr](https://miro.medium.com/v2/resize:fit:720/format:webp/1*83jt5PsBeD4CQpilznc3eA.png)

![ccd](https://miro.medium.com/v2/resize:fit:720/format:webp/1*7QbdvV62zvSs6YQmDyblnQ.png)

Now we will go ahead and Check the Load Balancer domain in a browser again, to ensure that our setup works. Now we will see the Django’s starting page.

![ccf](https://miro.medium.com/v2/resize:fit:720/format:webp/1*CDJDFFPIpQp_tvJflmiuNA.png)

Awesoome, great job making it this far, our setup is working. It’s time to commit our changes in the `django-aws-infrastructure` repo. We will add another file, a `.gitignore` file

Great job making it this far. Now we will go ahead and apply all of our terraform configuration `terraform apply` . We will st

```hcl
output "prod_lb_domain" {
  value = aws_lb.prod.dns_name
}
```

We will see our ALB domain in output.

```bash
Outputs:
prod_lb_hostname = "prod-57218461274.us-east-2.elb.amazonaws.com"
```

We can now check our this domain in our browser. It should respond with `503 Service Temporarily Unavailable` error because there are no targets associated with the target groups we created yet.

https://gist.github.com/apotitech/5f4afbe7826322f8cf6de112d150cd8f

Our code is all ready to go. Now we will save, commit our changes and push our changes:

```bash
$ git add .
$ git commit -m "initialize infrastructure"
```

## Congratulations

Yay! We have now deployed our Django web application with ECS Service + Fargate on AWS. But now it works with [SQLite](https://www.sqlite.org/index.html) file database. This file will be recreated on every service restart. So, our app cannot persist any data for now. [In the next article we’ll connect](https://medium.com/@softwaresennin/deploying-django-application-on-aws-with-terraform-connecting-postgresql-rds-bcd9c29d6276) Django to [AWS RDS PostgreSQL](https://aws.amazon.com/rds/postgresql/).

If you need technical consulting on your project or have any questions or suggestions, please comment below or connect with me directly on [LinkedIn](https://www.linkedin.com/in/lionel-tchami/).

Do not forget the 👏**❤️** and share if you like this content!

> Thank you for joining me, and best of luck with your AWS endeavors!