Deploying Django Application on AWS with Terraform. Connecting PostgreSQL RDS
Tags: aws, django, postgres, terraform
Published: 2022-07-29 09:50:00


--- series: Django + AWS ---


In the previous [part of this guide](https://dev.to/daiquiri_team/deploying-django-application-on-aws-with-terraform-minimal-working-setup-587g), we deployed the Django web application on [AWS ECS](https://aws.amazon.com/ecs/). 

In this part, we'll create a [PostgreSQL RDS](https://aws.amazon.com/rds/postgresql/) instance on AWS, connect it to the Django ECS task and enable access to [Django Admin](https://docs.djangoproject.com/en/3.2/ref/contrib/admin/). We choose PostgreSQL because it supports complex SQL queries, [MVCC](https://www.postgresql.org/docs/current/mvcc.html), good performance for concurrent queries, and has a large community. As AWS says:

> PostgreSQL has become the preferred open source relational database for many enterprise developers and start-ups, powering leading business and mobile applications. Amazon RDS makes it easy to set up, operate, and scale PostgreSQL deployments in the cloud. 

### Add PostgreSQL to Django project

Go to the `django-aws-backend` folder and activate virtual environment: `cd ../django-aws-backend && . ./venv/bin/activate`.

First, let's set up PostgreSQL via Docker for local development. Create `docker-compose.yml` with the following content:

```yaml
version: "3.1"

services:
  postgres:
    image: "postgres:14"
    ports:
      - "5433:5432"
    environment:
      POSTGRES_PASSWORD: "postgres"
      POSTGRES_DB: "django_aws"
```

Then run `docker-compose up -d` to start a container. Now, let's connect Django to PostgreSQL. 

> We use a non-standard 5433 port to avoid potential conflict with your local PostgreSQL

Next, we need to add pip packages: [psycopg2-binary](https://pypi.org/project/psycopg2-binary/) for working with PostgreSQL and [django-environ](https://django-environ.readthedocs.io/en/latest/) to retrieve PostgreSQL connection string from environment. Also, we'll add the [WhiteNoise](http://whitenoise.evans.io/en/stable/) package to serve static files for Django Admin.  

Add to the `requirements.txt` file these packages and run `pip install -r requirements.txt`:

```requirements.txt
django-environ==0.8.1
psycopg2-binary==2.9.3
whitenoise==6.1.0
```

Now, change `settings.py`. First, let's load env variables. We try to read the `.env` file in the project root if it exists.

```python
import environ

BASE_DIR = Path(__file__).resolve().parent.parent

env = environ.Env()
env_path = BASE_DIR / ".env"
if env_path.is_file():
    environ.Env.read_env(env_file=str(env_path))
```

Now we can use environment variables in `settings.py`. Let's provide `DATABASES` from env:

```python
DATABASES = {
    'default': env.db(default="postgresql://postgres:postgres@127.0.0.1:5433/django_aws")
}
```

We've set a default value for the `DATABASE_URL` to allow running the Django project locally without specifying this variable in the `.env` file. 

Third, add the `WhiteNoiseMiddleware` to serve static files and specify the `STATIC_ROOT` variable.

```python
MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    "whitenoise.middleware.WhiteNoiseMiddleware",
    ...
]

...

STATIC_URL = '/static/'
STATIC_ROOT = BASE_DIR / "static"
```

Also, add the `RUN ./manage.py collectstatic --noinput` line to the bottom of `Dockerfile` to collect static files in the static folder.

Apply migrations, create a superuser and start a web server:

```shell
(venv) $ python manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions
...
(venv) $ python manage.py createsuperuser
Username: admin
Email address: 
Password: 
Password (again): 
Superuser created 
(venv) $ python manage.py runserver
...
Starting development server at http://127.0.0.1:8000/
```

Go to [http://127.0.0.1:8000/admin/](http://127.0.0.1:8000/admin/) and sign in with your user.

![Django Admin](https://s3.eu-central-1.wasabisys.com/daiquiri-external-content/blog/eugen1j/django-aws/img/django_admin_1.png)

Finally, let's commit changes, build and push the docker image.

```shell
$ git add . 
$ git commit -m "add postgresql, environ and static files serving"
$ docker build . -t 947134793474.dkr.ecr.us-east-2.amazonaws.com/django-aws-backend:latest
$ docker push 947134793474.dkr.ecr.us-east-2.amazonaws.com/django-aws-backend:latest
```

We are done with the Django part. Next, we create a PostgreSQL instance on AWS and connect it to the ECS task.

### Creating RDS

AWS provides the [Relational Database Service](https://aws.amazon.com/rds/postgresql/) to run a [PostgreSQL](https://www.postgresql.org/). Now, we'll describe the RDS setup with Terraform. Go to the Terraform folder `../django-aws-infrastructure` and add a `rds.tf` file:

```terraform
resource "aws_db_subnet_group" "prod" {
  name       = "prod"
  subnet_ids = [aws_subnet.prod_private_1.id, aws_subnet.prod_private_2.id]
}

resource "aws_db_instance" "prod" {
  identifier              = "prod"
  db_name                 = var.prod_rds_db_name
  username                = var.prod_rds_username
  password                = var.prod_rds_password
  port                    = "5432"
  engine                  = "postgres"
  engine_version          = "14.2"
  instance_class          = var.prod_rds_instance_class
  allocated_storage       = "20"
  storage_encrypted       = false
  vpc_security_group_ids  = [aws_security_group.rds_prod.id]
  db_subnet_group_name    = aws_db_subnet_group.prod.name
  multi_az                = false
  storage_type            = "gp2"
  publicly_accessible     = false
  backup_retention_period = 5
  skip_final_snapshot     = true
}

# RDS Security Group (traffic ECS -> RDS)
resource "aws_security_group" "rds_prod" {
  name        = "rds-prod"
  vpc_id      = aws_vpc.prod.id

  ingress {
    protocol        = "tcp"
    from_port       = "5432"
    to_port         = "5432"
    security_groups = [aws_security_group.prod_ecs_backend.id]
  }

  egress {
    protocol    = "-1"
    from_port   = 0
    to_port     = 0
    cidr_blocks = ["0.0.0.0/0"]
  }
}
```

Here we create an RDS instance, a DB subnet group, and a Security Group to allow incoming traffic from ECS 5432 port only. We'll specify `db_name`, `username`, `password`, and `instance_class` in the `variables.tf`: 

```terraform
# rds

variable "prod_rds_db_name" {
  description = "RDS database name"
  default     = "django_aws"
}
variable "prod_rds_username" {
  description = "RDS database username"
  default     = "django_aws"
}
variable "prod_rds_password" {
  description = "postgres password for production DB"
}
variable "prod_rds_instance_class" {
  description = "RDS instance type"
  default     = "db.t4g.micro"
}
```

Notice that we haven't provided a default value for the `prod_rds_password` variable to prevent committing the database password to the repository. Terraform will ask you for a variable value if you try to apply these changes.

```shell
$ terraform apply             
var.prod_rds_password
  postgres password for production DB

  Enter a value: 
```

It's not convenient and error-prone to type a password every time. Gladly, Terraform can read variable values from the environment. Create `.env` file with `TF_VAR_prod_rds_password=YOUR_PASSWORD` variable. Interrupt password prompting, run `export $(cat .env | xargs)` to load `.env` variable, and rerun `terraform apply`. Now Terraform picks the password from the `.env` file and creates an RDS PostgreSQL instance. Check it in the [AWS RDS console](https://console.aws.amazon.com/rds#databases:).

![RDS Instance](https://s3.eu-central-1.wasabisys.com/daiquiri-external-content/blog/eugen1j/django-aws/img/rds_instance.png)

### Connecting to ECS

Let's connect the RDS instance to ECS tasks.

Add to `backend_container.json.tpl` environment var `DATABASE_URL`

```json
[
  {
    ...
    "environment": [
      {
        "name": "DATABASE_URL",
        "value": "postgresql://${rds_username}:${rds_password}@${rds_hostname}:5432/${rds_db_name}"
      }
    ],
    ...
  }
]
```

Add RDS variables to the `prod_backend_web` task definition in the `ecs.tf` file:

```terraform
resource "aws_ecs_task_definition" "prod_backend_web" {
  ...
  
  container_definitions = templatefile(
    "templates/backend_container.json.tpl",
    {
      ...
      
      rds_db_name  = var.prod_rds_db_name
      rds_username = var.prod_rds_username
      rds_password = var.prod_rds_password
      rds_hostname = aws_db_instance.prod.address
    },
  )
  ...
}
```

Let's apply changes and update the ECS service with the new task definition. Run `terraform apply`, stop current task via web console and wait for the new task to arise. 

![Stopping ECS task](https://s3.eu-central-1.wasabisys.com/daiquiri-external-content/blog/eugen1j/django-aws/img/ecs_services_stop_tasks.png)

Now, go to the admin URL on the load balancer hostname and try to log in with random credentials. You should get the `relation "auth_user" does not exist` error. This error means that the Django application successfully connected to PostgreSQL, but no migrations was run.

![Django no migrations error](https://s3.eu-central-1.wasabisys.com/daiquiri-external-content/blog/eugen1j/django-aws/img/django_no_migrations_error.png)

### Running migrations

Now, we need to run migrations and create a superuser. But how can we do it? Our infrastructure has no EC2 instances to connect via SSH and run this command. The solution is [ECS Exec](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-exec.html). 

> With Amazon ECS Exec, you can directly interact with containers without needing to first interact with the host container operating system, open inbound ports, or manage SSH keys. 

First, we need to install the Session Manager plugin. For macOS, you can use `brew install session-manager-plugin`. For other platforms, check this [link](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html).

Next, we need to provide an IAM policy to the `prod_backend_task` role and enable `execute_command` for the ECS service. Add this code to the `ecs.tf` and apply changes:

```terraform
resource "aws_ecs_service" "prod_backend_web" {
  ...
  enable_execute_command             = true
  ...
}

resource "aws_iam_role" "prod_backend_task" {
  ...
          
  inline_policy {
    name = "prod-backend-task-ssmmessages"
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Action   = [
            "ssmmessages:CreateControlChannel",
            "ssmmessages:CreateDataChannel",
            "ssmmessages:OpenControlChannel",
            "ssmmessages:OpenDataChannel",
          ]
          Effect   = "Allow"
          Resource = "*"
        },
      ]
    })
  }
}
```

Stop the old ECS task in web console and wait for the new task.

Then let's retrieve the new `task_id` via CLI and run `aws esc execute-command`. Create a new file `touch backend_web_shell.sh && chmod 777 backend_web_shell.sh` with the content:

```shell
#!/bin/bash

TASK_ID=$(aws ecs list-tasks --cluster prod --service-name prod-backend-web  --query 'taskArns[0]' --output text  | awk '{split($0,a,"/"); print a[3]}')
aws ecs execute-command --task $TASK_ID --command "bash" --interactive --cluster prod --region us-east-2
```

Run `./backend_web_shell.sh` to shell into task. If you have some troubles, please check [Amazon ECS Exec Checker](https://github.com/aws-containers/amazon-ecs-exec-checker). This script checks that your CLI environment and ECS cluster/task are ready for ECS Exec.

Run migrations and create a superuser from the task's console:

```shell
$ ./manage.py migrate
$ ./manage.py createsuperuser 
Username: admin
Email address: 
Password: 
Password (again): 
Superuser created 
```

After this, check your deployment admin URL and try to sign in with provided credentials. 

![Successful admin deployment](https://s3.eu-central-1.wasabisys.com/daiquiri-external-content/blog/eugen1j/django-aws/img/django_admin_2.png)

Congratulations! We've successfully connected PostgreSQL to the ECS service. Now you can commit changes in the Terraform project and move to the next part. 

In the next part we'll set up [CI/CD](https://docs.gitlab.com/ee/ci/introduction/index.html#continuous-delivery) with GitLab.

You can find the source code of backend and infrastructure projects [here](https://gitlab.com/django-aws/django-aws-backend) and [here](https://gitlab.com/django-aws/django-aws-infrastructure).

If you need technical consulting on your project, check out our [website](https://daiquiri.team/services/technical-consulting?utm_medium=referral&utm_source=dev_to&utm_campaign=django_aws_2) or connect with me directly on [LinkedIn](https://www.linkedin.com/in/yevhen-bondar/).