AWS ECS container health check
Tags: aws, ecs, docker, terraform
Published: 2022-11-25 06:57:48

## What is Container health check?

Container health check is a feature of Docker. It checks container is healthy or not(unhealthy). You don't have to use it, but it provides container status and is easy to check container status when entering `docker ps` command.

```sh
$ docker ps
CONTAINER ID   IMAGE     COMMAND                  CREATED          STATUS                    PORTS                    NAMES
1b5c86ccfa6d   redis     "docker-entrypoint.s…"   13 seconds ago   Up 13 seconds (healthy)   0.0.0.0:6379->6379/tcp   redis
```

You can see health info from `STATUS` or use `docker inspect`

```sh
$ docker inspect redis | jq ".[0].State.Health"
{
  "Status": "healthy",
  "FailingStreak": 0,
  "Log": [
    {
      "Start": "2022-11-25T06:35:47.907103919Z",
      "End": "2022-11-25T06:35:47.973231669Z",
      "ExitCode": 0,
      "Output": "PONG\n"
    }
  ]
}
```

`ExitCode` is the result of the command in `HEALTHCHECK` instruction. If `ExitCode` is non-zero, container is unhealthy. I'll explain it in detail below.

### How to use health check

It's straightforward. Use `HEALTHCHECK` instruction in Dockerfile.
- HEALTHCHECK [OPTIONS] CMD command 

Details in [Dockerfile Docs](https://docs.docker.com/engine/reference/builder/#healthcheck).

```Dockerfile
FROM redis:7-alpine

HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
    CMD redis-cli ping
```
`redis-cli ping` does a health check. If you want to check the HTTP server, you can write
- `CMD-SHELL curl -f http://localhost:80/ || exit 1`

instead. `CMD-SHELL` is similar to `CMD`, but the former uses the container's default shell.

**CAUTION**: You should install `curl` in the container when using `curl` command. If not, the container will be unhealthy on ECS even if there's no problem in the local environment. Redis in the above has `redis-cli` command already, so it's okay.

The options that can appear before CMD are:

- `--interval=DURATION` default is `30s`
- `--timeout=DURATION` default is `30s`
- `--start-period=DURATION` default is `0s`
- `--retries=N` default is `3`

## ECS container health check

Okay, it's done, right? Unfortunately, ECS doesn't support container health check from Docker for monitoring.

[AWS Docs](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_HealthCheck.html) says:
> The Amazon ECS container agent only monitors and reports on the health checks specified in the task definition. Amazon ECS does not monitor Docker health checks that are embedded in a container image and not specified in the container definition. Health check parameters that are specified in a container definition override any Docker health checks that exist in the container image.

You have to put it into container definitions in task definition. Below is an incomplete ECS terraform example, so see only how it works.

```terraform
# Three containers will be deployed: "app", "worker", "redis"
# Only "redis" has `healthCheck`
resource "aws_ecs_task_definition" "this" {
  family                   = "example"
  requires_compatibilities = ["FARGATE"]
  network_mode             = "awsvpc"
  cpu                      = 1024
  memory                   = 2048

  # other settings are omitted..

  container_definitions = jsonencode([
    {
      name      = "redis"
      image     = "redis:7-alpine"
      cpu       = 256
      memory    = 512
      essential = true

      portMappings = [{
        containerPort = 6379
        hostPort      = 6379
      }]

      healthCheck = {
        command     = ["CMD", "redis-cli", "ping"]
        interval    = 30
        retries     = 3
        startPeriod = 30
        timeout     = 5
      }
    },
    {
      name      = "app"
      image     = "django:latest"
      # other settings are omitted..
    },
    {
      name      = "worker"
      image     = "celery:latest"
      # other settings are omitted..
    }
  ])
}
```

After deploying the container, you can see `redis` container is Healthy, unlike the others are Unknown on AWS console.


![container status](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8qw382o6ct539wkmytiw.png)