ECS on FargateでRedash(v10.1.0)を構築してみた

# はじめに
Redashとは、データソースに接続・クエリ結果の抽出ができ、目的に沿ってデータやクエリ結果をダッシュボード上で自由に可視化することができるBIツール。
データソースは広くサポートしている。

参考-> <a href="https://redash.io/help/data-sources/querying/supported-data-sources">Supported Data Sources</a> 

AWS周りでは、下記サービスがサポートされている。

- Amazon Athena
- Amazon CloudWatch
- Amazon CloudWatch Logs Insights
- Amazon DynamoDB
- Amazon Redshift


# アーキテクチャ
下記コンポーネントを使い構築する（コード管理はterraformを使用）
- Redash (Web)Server/Worker -> ECS on Fargate (image: <a href="https://hub.docker.com/layers/redash/redash/10.1.0.b50633/images/sha256-f3e8d95656255d9684051604a586dfd50035fa1e297e68379dc1b557f1885c0f?context=explore">redash/redash:10.1.0.b50633</a> )
- Redis -> ECS on Fargate (iamge: redis:3-alpine) 
- PostgreSQL -> RDS
- Data Source -> Athena

![architect.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/521220/9b0125f3-8bda-8110-6519-35160b973087.png)

# 構築手順
## SecurityGroup
- ELB: Redash使用予定端末のセグメントからの接続(HTTP:80)のみ許可
（今回とりあえずということでCertificationの設定を省いたが、最終的にはHTTPSで接続させる）
- ECS on Fargate: ELBのSecurityGroupIDからの接続(TCP:5000)のみ許可
- RDS: ECS on Fargate-ServiceのSecurityGroupからの接続(TCP:5432)のみ許可

## RDSを構築
### 1. インスタンスを作成
```rb
resource "aws_db_instance" "redash-test-rds" {
    allocated_storage                     = 20
    availability_zone                     = "us-east-1a"
    engine                                = "postgres"
    engine_version                        = "13.7"
    instance_class                        = "db.t3.micro"
    username                              = "postgres"
    username                              = "postgres"
    skip_final_snapshot                   = true
    identifier                            = "redash-test-rds"
    port                                  = 5432
    vpc_security_group_ids                = [
        "sg-XXXXX",
    ]
    db_subnet_group_name                  = "default"
}
```

### 2. redash user作成
- 初期設定として、redash userを作成しておく
```
psql -h test-redash-db.XXXXX.us-east-1.rds.amazonaws.com -U postgres -p 5432
Password for user postgres: *****
postgres=> CREATE USER redash WITH PASSWORD redash123 CREATEDB; # username=redash, password=redash123で設定する場合
```
### 3. redash database作成
- 初期設定として、redash databaseを作成しておく
```
postgres=> CREATE DATABASE redash;
postgres=> GRANT ALL ON ALL TABLES IN SCHEMA public TO redash;
```

## Secrets Managerを構築
- 環境変数[REDASH_COOKIE_SECRET, REDASH_SECRET_KEY, REDASH_DATABASE_URL]は機密情報を含むため、予めSecretを作成しておく
### 1. Secretを作成
```rb
resource "aws_secretsmanager_secret" "redash-test-secretmanager" {
    name                           = "redash-test-secret"
    }
```

### 2. Secretの値を作成
![secret.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/521220/7a6e5d2a-4e83-bfdd-6b76-8d0843d0329c.png)

### 参照するSecretの値として下記3つを登録
#### REDASH_COOKIE_SECRET
Redash起動に伴い、ユーザ認証、Cookie署名、セッション情報保存などを暗号化する際に必要になる。
ランダム値を設定する必要があり、下記いずれかのコマンドによりキーの値を作成する。
```
$ python -c 'import secrets; print(secrets.token_hex())'
```
```
$ pwgen -1s 64
```
#### REDASH_SECRET_KEY
Redash起動に伴い、各種設定を暗号化する際に必要になる。
明示的に設定しない場合、REDASH_COOKIE_SECRETを代わりに使用する仕様にはなっているが、セキュリティを確保するために、REDASH_SECRET_KEYも設定することが推奨されている。
REDASH_COOKIE_SECRET同様のやり方で、ランダム値を設定する必要がある。

参考-> <a href="https://redash.io/help/open-source/admin-guide/secrets">Secret Keys in Redash</a> 

#### REDASH_DATABASE_URL
下記の文法でRDSのURLを設定する
```
postgresql://${PostgresUsername}:${PostgresPassword}@postgres:5432/${PostgresDBName}
```
```
ex) 
・ユーザ名: redash
・パスワード: redash123
・DB名: redash
-> postgresql://redash:redash123@postgres:5432/redash
```

参考-> <a href="https://redash.io/help/open-source/admin-guide/env-vars-settings">Environment Variables Settings</a> 

## ALBを構築
- TargetGroupのリスナーに設定には、Redash Web ServerがListenしているHTTP/5000を設定　★1
- health check用のAPIエンドポイントとして用意されている/ping を使う ★2
参考-> <a href="https://redash.io/help/open-source/dev-guide/setup">Developer Installation Guide</a> 


```rb
resource "aws_lb" "redash-test-alb" {
    name                       = "redash-test-ELB"
    internal                   = false
    load_balancer_type         = "application"
    security_groups            = [
        "sg-XXXXX",
    ]
    subnets                    = [
        "subnet-XXXXX",
        "subnet-XXXXX",
        "subnet-XXXXX",
    ]
}

resource "aws_lb_target_group" "redash-test-tg" {
    name                          = "redash-test-TG"
    port                          = 5000   # ★1
    protocol                      = "HTTP" # ★1
    target_type                   = "ip"   # ★1
    vpc_id                        = "vpc-XXXXX"

    health_check {
        enabled             = true
        healthy_threshold   = 5
        interval            = 30
        path                = "/ping"        # ★2
        port                = "traffic-port" # ★2
        protocol            = "HTTP"         # ★2
        timeout             = 5
        unhealthy_threshold = 2
    }
}

resource "aws_lb_listener" "redash-test-listener" {
    load_balancer_arn = "arn:aws:elasticloadbalancing:us-east-1:XXXXX:loadbalancer/app/redash-test-ELB/XXXXX"
    port              = 80
    protocol          = "HTTP"

    default_action {
        order            = 1
        target_group_arn = "arn:aws:elasticloadbalancing:us-east-1:XXXXX:targetgroup/redash-test-TG/XXXXX"
        type             = "forward"
    }
}
```

## ECS on Fargateを構築
### 1. Serviceと紐づけるTask定義を作成
- Redash Web Server, Redash Worker, Redis の3つのContainer分を作成する
- nameをContainer毎に設定（任意の値でOK)　★1
- commandをContainer毎に適切に設定 (Redashの<a href="https://github.com/getredash/redash/blob/master/bin/docker-entrypoint">/app/bin/docker-entrypoint</a>ファイルに記載の関数名＝command名になる )　★2
- <a href="https://github.com/getredash/redash/blob/master/docker-compose.yml">docker-compose.yml</a>を参考に、Redash Web ServerとRedisはportMappingsを設定する　★3
- Redisのimageは<a href="https://github.com/getredash/redash/blob/master/docker-compose.yml">docker-compose.yml</a>に記載のimage名を設定　★4
- <a href="https://redash.io/help/open-source/dev-guide/setup">Developer Installation Guide</a>を参考に、SecretManagerで設定したもの＋PYTHONUNBUFFERED, PYTHONUNBUFFEREDの環境変数を設定
```rb
resource "aws_ecs_task_definition" "redash-test-ecs-task-definition" {
    family                   = "redash-test-Task"
    cpu                      = "4096"
    execution_role_arn       = "arn:aws:iam::XXXXX:role/ecsTaskExecutionRole"
    memory                   = "16384"
    network_mode             = "awsvpc"
    requires_compatibilities = [
        "FARGATE",
    ]
    runtime_platform {
        operating_system_family = "LINUX"
    }
    container_definitions    = jsonencode(
        [
            ### 以下RedisContainerのTask定義 ###
            {
                name             = "redis"  #★1
                image            = "redis:3-alpine"  #★4
                cpu              = 0
                essential        = true
                portMappings     = [
                    {
                        containerPort = 6379　 #★3
                        hostPort      = 6379　 #★3
                        protocol      = "tcp"　#★3
                    },
                ]
                environment      = [
                    {
                        name  = "PYTHONUNBUFFERED"
                        value = "0"
                    },
                    {
                        name  = "REDASH_LOG_LEVEL"
                        value = "INFO"
                    },
                ]
                logConfiguration = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/ecs/redash-test-Task"
                        awslogs-region        = "us-east-1"
                        awslogs-stream-prefix = "ecs"
                    }
                }
                secrets          = [
                    {
                        name      = "REDASH_COOKIE_SECRET"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_COOKIE_SECRET::"
                    },
                    {
                        name      = "REDASH_DATABASE_URL"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_DATABASE_URL::"
                    },
                    {
                        name      = "REDASH_SECRET_KEY"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_SECRET_KEY::"
                    },
                ]
            },
            ### 以下Redash Web ServerのTask定義 ###
            {
                name             = "server"　#★1
                image            = "redash/redash:10.1.0.b50633"
                cpu              = 0
                #essential        = true
                portMappings     = [
                    {
                        containerPort = 5000   #★3
                        hostPort      = 5000   #★3
                        protocol      = "tcp"  #★3
                    },
                ]
                command          = [
                    "server",  #★2
                ]
                environment      = [
                    {
                        name  = "PYTHONUNBUFFERED"
                        value = "0"
                    },
                    {
                        name  = "REDASH_LOG_LEVEL"
                        value = "INFO"
                    },
                ]
                logConfiguration = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/ecs/redash-test-Task"
                        awslogs-region        = "us-east-1"
                        awslogs-stream-prefix = "ecs"
                    }
                }
                secrets          = [
                    {
                        name      = "REDASH_COOKIE_SECRET"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_COOKIE_SECRET::"
                    },
                    {
                        name      = "REDASH_DATABASE_URL"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_DATABASE_URL::"
                    },
                    {
                        name      = "REDASH_SECRET_KEY"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_SECRET_KEY::"                    },
                ]

            },
            ### 以下Redash Workerのタスク定義 ###
            {
                name             = "worker"　#★1
                image            = "redash/redash:10.1.0.b50633"
                cpu              = 0
                essential        = true
                portMappings     = []
                command          = [
                    "worker",  #★2
                ]
                environment      = [
                    {
                        name  = "PYTHONUNBUFFERED"
                        value = "0"
                    },
                    {
                        name  = "REDASH_LOG_LEVEL"
                        value = "INFO"
                    },
                ]
                logConfiguration = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/ecs/redash-test-Task"
                        awslogs-region        = "us-east-1"
                        awslogs-stream-prefix = "ecs"
                    }
                }
                secrets          = [
                    {
                        name      = "REDASH_COOKIE_SECRET"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_COOKIE_SECRET::"
                    },
                    {
                        name      = "REDASH_DATABASE_URL"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_DATABASE_URL::"
                    },
                    {
                        name      = "REDASH_SECRET_KEY"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_SECRET_KEY::"
                    },
                ]
            },
        ]
    )
}

```

### 2. 初期設定用のTask定義(createdb)を作成
- Redash Web Server、create_dbの2つのContainer分を作成する
- nameをContainer毎に設定（任意の値でOK)
- commandをContainer毎に適切に設定 (Redashの<a href="https://github.com/getredash/redash/blob/master/bin/docker-entrypoint">/app/bin/docker-entrypoint</a>ファイルに記載の関数名＝command名になる )
- <a href="https://github.com/getredash/redash/blob/master/docker-compose.yml">docker-compose.yml</a>を参考に、Redash Web ServerはportMappingsを設定する

```rb
resource "aws_ecs_task_definition" "redash-test-ecs-task-definition-createdb" {
    family                   = "redash-test-Task-createdb"
    cpu                      = "4096"
    execution_role_arn       = "arn:aws:iam::XXXXX:role/ecsTaskExecutionRole"
    memory                   = "16384"
    network_mode             = "awsvpc"
    requires_compatibilities = [
        "FARGATE",
    ]
    runtime_platform {
        operating_system_family = "LINUX"
    }
    container_definitions    = jsonencode(
        [
            ### 以下Redash Web ServerのTask定義 ###
            {
                name             = "server"　#★1
                image            = "redash/redash:10.1.0.b50633"
                cpu              = 0
                #essential        = true
                portMappings     = [
                    {
                        containerPort = 5000   #★3
                        hostPort      = 5000   #★3
                        protocol      = "tcp"  #★3
                    },
                ]
                command          = [
                    "server",  #★2
                ]
                environment      = [
                    {
                        name  = "PYTHONUNBUFFERED"
                        value = "0"
                    },
                    {
                        name  = "REDASH_LOG_LEVEL"
                        value = "INFO"
                    },
                ]
                logConfiguration = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/ecs/redash-test-Task"
                        awslogs-region        = "us-east-1"
                        awslogs-stream-prefix = "ecs"
                    }
                }
                secrets          = [
                    {
                        name      = "REDASH_COOKIE_SECRET"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_COOKIE_SECRET::"
                    },
                    {
                        name      = "REDASH_DATABASE_URL"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_DATABASE_URL::"
                    },
                    {
                        name      = "REDASH_SECRET_KEY"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_SECRET_KEY::"                    },
                ]

            },
            ### 以下Redash Createdbのタスク定義 ###
            {
                name             = "create_db"　#★1
                image            = "redash/redash:10.1.0.b50633"
                cpu              = 0
                essential        = true
                portMappings     = []
                command          = [
                    "create_db",  　#★2
                ]
                environment      = [
                    {
                        name  = "PYTHONUNBUFFERED"
                        value = "0"
                    },
                    {
                        name  = "REDASH_LOG_LEVEL"
                        value = "INFO"
                    },
                ]
                logConfiguration = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/ecs/redash-test-Task"
                        awslogs-region        = "us-east-1"
                        awslogs-stream-prefix = "ecs"
                    }
                }
                secrets          = [
                    {
                        name      = "REDASH_COOKIE_SECRET"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_COOKIE_SECRET::"
                    },
                    {
                        name      = "REDASH_DATABASE_URL"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_DATABASE_URL::"
                    },
                    {
                        name      = "REDASH_SECRET_KEY"
                        valueFrom = "arn:aws:secretsmanager:us-east-1:XXXXX:secret:redash-test-secret-XXXX:REDASH_SECRET_KEY::"
                    },
                ]
            },
        ]
    )
}

```
### 3. ClusterとServiceを作成

```rb
resource "aws_ecs_cluster" "redash-test-ecs-cluster" {
    capacity_providers = [
        "FARGATE",
        "FARGATE_SPOT",
    ]
    name               = "redash-test-ecs-ECSCluster"

    setting {
        name  = "containerInsights"
        value = "disabled"
    }
}

resource "aws_ecs_service" "redash-test-ecs-service" {
    name                               = "redash-test-ecs-ECSService"
    cluster                            = "arn:aws:ecs:us-east-1:XXXXX:cluster/redash-test-ecs-ECSCluster"
    desired_count                      = 1
    enable_ecs_managed_tags            = true
    iam_role                           = "AWSServiceRoleForECS"
    launch_type                        = "FARGATE"
    task_definition                    = "redash-test-Task:1"

    load_balancer {
        container_name   = "server"
        container_port   = 5000
        target_group_arn = "arn:aws:elasticloadbalancing:us-east-1:XXXXX:targetgroup/redash-test-TG/XXXXX"
    }

    network_configuration {
        assign_public_ip = true
        security_groups  = [
            "sg-XXXXX",
        ]
        subnets          = [
            "subnet-XXXXX",
        ]
    }
}
```

# Container起動
### 1. 初期設定のため、redash-test-Task-createdbを起動
- 最初の1回だけ、初期設定のために手動実行する。
- コンテナが起動し、数分で終了する。
- CloudWatch ロググループからエラーが出ていないことを確認する。

### 2. redash-test-ecs-serviceを起動
- CloudWatch ロググループからエラーが出ていないことを確認する。
- http://[ELBのDNS名] にアクセスし、RedashのSetUP画面が出ることを確認する

# おわりに
初のECS on Fargateということもあり、いろいろと躓いたので、お役に立てたらうれしいです。