AWS App Performance
Tags: aws, elasticache, python, redis
Published: 2021-06-24 05:26:59

## TL;DR 

I completed another [#CloudGuruChallenge]
(https://acloudguru.com/blog/engineering/cloudguruchallenge-improve-application-performance-using-amazon-elasticache). This month we were challenged to improve a python app's performance by implementing ElastiCache for Redis. 

I thought this challenge was really interesting because application performance is such a big topic in the industry. I've seen Python and Redis used multiple times already. 

Unfortunately due to the nature of the services required for this challenge I was unable to take a serverless approach. That also means that I would have already tore down the resources by the time you're reading this. Don't worry though I took some pictures for us to enjoy! 

### Database Response

![Database Response](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b2e5q2u4kie121t6smc5.PNG)

### Cache Response

![Cache Response](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mhxn7p7lhr9akvm8xbf1.PNG)

If you'd like to checkout the code or connect on LinkedIn then see below. 

**GitHub**
https://github.com/wheelerswebservices/cgc-aws-app-performance

**LinkedIn**
https://www.linkedin.com/in/wheelerswebservices/

In addition to the requirements I decided to go above and beyond. If that interests you then keep reading!

---

## Architecture

I decided to manage the architecture for the application and the network separately since the costs vary drastically between both among other reasons. 

### Network Architecture
![Network Architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mszx4fwy18jeir3mueao.png)
* VPC
* 4 Public Subnets
* 4 Private Subnets
* Public Route Table
* Private Route Table
* Internet Gateway
* NAT Gateway
* Public Security Group (HTTP and SSH)
* Private Security Group (PostgreSQL and Redis)

### Application Architecture
![Application Architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tv0sbdk8fi0grcrnvm9h.png)
* EC2 Instance
* EC2 KeyPair
* ElasticCache Redis Instance
* ElasticCache Subnet Group
* RDS PostgreSQL Instance
* RDS Subnet Group

It's true that this configuration is probably overkill for this simple application. However, I wanted to push myself to design for a production scenario. 

Here are some next steps that would improve this architecture that I deemed out of scope. 

**HTTPS Ingress**
* update public security group to permit HTTPS instead of HTTP
* setup NGINX to use HTTPS

**Failover NAT Gateway**
* setup failover NAT Gateway in another AZ
* update private route table to use one or both NAT Gateways

**Multiple Application Servers w/ Load Balancer**
* setup the app to run as part of an ASG instead
* setup an ALB/NLB in front of the ASG to load balance traffic

---

## Terraform

Much like in previous projects I decided to use Terraform here for Infrastructure as Code (IaC). It seems that every time I use Terraform I find something new that I like about it. 

Below are a list of new found features that I recently learned about and used on this project. 

### locals block

This feature allows users to define local variables inside their templates.

https://www.terraform.io/docs/language/values/locals.html

```terraform
locals {
  availability_zones = [
    "us-east-1a",
    "us-east-1b",
    "us-east-1c",
    "us-east-1d"
  ]
  cidr_anywhere_block = "0.0.0.0/0"
  ...
}

local.cidr_anywhere_block
```

### file function

This feature allows you to easily pass a file into the resource declaration. That allowed me to bootstrap my EC2 instance while managing that script outside of the template itself. No crazy formatting required here!

https://www.terraform.io/docs/language/functions/file.html

```terraform
resource "aws_instance" "server" {
  ...
  user_data = file("user-data.sh")
}
```

### random_shuffle provider

This feature allows you to shuffle a list of values and return x number of values back. I thought this was far more clever than simply providing a single value and more flexible too!

https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/shuffle

```terraform
variable "subnet" {
  description = "subnet ids"
  type = object({
    private = list(string)
    public  = list(string)
  })
}

subnet = {
  ...
  public  = [
    "subnet-01b36f97fa8490e43",
    "subnet-0a4272a81af160127",
    "subnet-03939599dd92f0f07",
    "subnet-0b6f2c6a35b31e4d5"
  ]
}

resource "random_shuffle" "public_subnet" {
  input        = var.subnet.public
  result_count = 1
}

subnet_id = random_shuffle.public_subnet.result[0]
```

### count meta-argument

This feature allows you to provide looping behavior to your templates so you don't need to repeat yourself. 

https://www.terraform.io/docs/language/meta-arguments/count.html

```terraform
resource "aws_subnet" "public" {
  count = length(local.cidr_public_blocks)
  cidr_block        = local.cidr_public_blocks[count.index]
  ...
}
```

[GitHub (Terraform)](https://github.com/wheelerswebservices/cgc-aws-app-performance/tree/main/terraform)

---

## Flask, Gunicorn, and Styles 

I changed the default application that was provided by David Thomas to use some new found knowledge on related topics.

> https://github.com/ACloudGuru/elastic-cache-challenge

### Jinja Expressions
This allowed me to put logic into my HTML templates similar to `ngIf` for Angular. I mainly did this because when testing I didn't want to see empty sections drawn on the screen. Now they will be hidden if the values are not passed into the `render_template` function. 

```jinja
{% if db_host %}
<div class="section">
    <span class="section-title">
        Database Host
    </span>
    <span>{{ db_host }}</span>
</div>
{% endif %}
```
Furthermore, this allowed me to add an error display on the screen that is hidden if there is no error. Since I know there is a risk with displaying some error logs to the user I modified the code to only pass the actual error if the `DEBUG` variable is set appropriately on the server.

```python
debug = os.environ.get('DEBUG')
debug = debug.lower() in ['1', 't', 'true', 'y', 'yes'] if debug is not None else False

...
try:
    config = ini('postgres')
    db_host = config['host']
    db_version = select_version()

except Exception as e:
    print('index() error:', e)
    error = e if debug else "we're currently experiencing technical difficulties"

return render_template(
    html,
    error=error,
    db_host=db_host,
    db_version=db_version
)
```

![Error Message](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/alkk0pfqhrn6n7eoigcj.PNG)

### Jinja Inheritance
This allowed me to create a base template and reuse that in other templates, if I had other templates, so I could keep a consistent style app-wide with low maintenance. 

```jinja
<head>
  <meta charset="UTF-8" />
  <link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}" />
  <title>CloudGuruChallenge_21.06</title>
</head>
<body>
  {% block content %}
  {% endblock %}
</body>
```

```jinja
{% extends 'base.html' %}
{% block content %}
...
<div class="section">
    <span class="section-title">
        Elapsed Time
    </span>
    <span>{{ g.request_time() }}</span>
</div>
{% endblock %}
```
### Gunicorn Server

I had quickly learned that the built-in flask server is not intended to work in production. Although, it may have been fine for this simple project I wanted to go the extra mile to setup gunicorn instead.

Once installed I just ran the app as a daemon in my bootstrap script. e.g. `gunicorn -D app:app`

### Styling

I couldn't stand the bare HTML page and decided to use some CSS to spruce it up a tad. In addition to styling I wanted to leverage flask's `static` assets and `url_for` as well to solidify that knowledge.

`<link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}" />`

[GitHub (Python)](https://github.com/wheelerswebservices/cgc-aws-app-performance/tree/main/app)

---

## Conclusion

Overall I really enjoyed this challenge. I can't wait to see what could possibly be next. Regardless I will be up the task!

I'd love to connect on LinkedIn and hear your feedback. If you completed the challenge as well I would like to see your work.

Cheers!

**GitHub**
https://github.com/wheelerswebservices/cgc-aws-app-performance

**LinkedIn**
https://www.linkedin.com/in/wheelerswebservices/