AWS FargateでRailsアプリとSidekiqを動かすやり方

FargateでRailsコンテナと別にSidekiqを動かしたかったので試してみました。
その手順を書いていきます。
なお、RailsにSidekiqを導入するやり方については、本記事では紹介しません。


## 前提

RailsアプリをFargateで動かすまでにやり方については以下の記事に書いております。

https://qiita.com/hatsu/items/22e11e94a0a981d78efa

https://qiita.com/hatsu/items/a70d5c49a39561836751

### Sidekiqの設定確認

```ruby
Sidekiq.configure_server do |config|
  config.redis = { url: "redis://#{ENV.fetch("REDIS_URL", "localhost:6379")}", namespace: "sidekiq" }
end

Sidekiq.configure_client do |config|
  config.redis = { url: "redis://#{ENV.fetch("REDIS_URL", "localhost:6379")}", namespace: "sidekiq" }
end
```
この `REDIS_URL`は環境変数で定義出来るようにしています。


Jobは以下のようなものを設置しています。

```ruby
class Batch::TestJob
  include Sidekiq::Worker
  sidekiq_options queue: "default"

  def perform
    logger.info "========Batch::TestJob==========="
  end
end
```

Jobはこれで動きます。

```ruby
Batch::TestJob.perform_async
```

### ElastiCacheの導入

Redisを用意するため、ElastiCacheを導入します。

#### ElastiCache用セキュリティグループの作成

AWS管理画面からセキュリティグループの作成を行います。

セキュリティグループの入力する値は以下です。 

|フォーム | 値 |
| -- | -- |
|セキュリティグループ名|fargate-test-redis|
|説明|fargate-test-redis|
|VPC|fargate-test|
|インバウンドルール タイプ|カスタムTCP|
|インバウンドルール ポート範囲|6379|
|インバウンドルール ソース|fargate-test|

イメージ
![VPC_Management_Console.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107345/aa124293-430d-f4a7-fbbe-9878d3dbc2c8.png)

入力が完了したら、作成して完了です。


#### ElastiCacheの作成

AWS管理画面の ElastiCache ページから「今すぐ始める」を押します。
![ElastiCache_Management_Console.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107345/ca8cff51-0def-587c-82f9-9df3a07b12d6.png)


|フォーム | 値 |
| -- | -- |
|クラスターエンジン|redis|
|ロケーション|Amazon クラウド|
|名前|fargate-test|
|説明|fargate-test-redis|
|エンジンバージョンの互換性|6.x(最新)|
|ポート|6379|
|パラメータグループ|default.redis6.x|
|ノードのタイプ|cache.t3.micro１|
|パラメータグループ|default.redis6.x|
|レプリケーション数|1|
|マルチAZ|チェック|
|サブネットグループ|新規作成|
|サブネットグループ名|fargate-test-redis|
|サブネットグループ 説明|fargate-test-redis||
|サブネットグループ VPC|fargate-testのもの|
|サブネットグループ サブネット|プライベートサブネット2つ(ap-northeast-1a,ap-northeast-1c)|
|アベイラビリティーゾーンの配置|指定なし|
|セキュリティグループ|先程作ったfaragate-test-redisのセキュリティグループ|


イメージ

![ElastiCache_Management_Console.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107345/b9ef4e72-3a4d-50ef-146d-4c73e63f3381.png)
![ElastiCache_Management_Console.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107345/4a6cc34e-c207-da73-d097-6b4832bbfa99.png)
![ElastiCache_Management_Console.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107345/f7655845-743f-ee3d-de25-fd0ed273df4d.png)
![ElastiCache_Management_Console.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107345/7fd3a2ea-caa9-9d69-8ba9-62cfa699ebb7.png)

上記のように入力が完了したら、「作成」ボタンを押して作成を行います。

### SSMパラメータストアにREDIS_URLを登録

ElastiCacheの作成が完了したら、ElastiCacheの「プライマリエンドポイント」をパラメータストアに登録します。(クラスタモードを有効にしている場合は「設定エンドポイント」を登録)
System Managerの画面からパラメータストアを選択します。

![AWS_Systems_Manager.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107345/47c8826a-b63f-fa62-089c-200ef1f3ee6c.png)

パラメータストアの作成を行います。
![AWS_Systems_Manager_-_Parameter_Store.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107345/2d311cab-efc6-cc27-1e15-29147052b773.png)

プライマリエンドポイント
![ElastiCache_Management_Console.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107345/fdeeec50-da3c-e2ee-35a6-4c8826672da0.png)

入力する内容は以下です。

|名前|タイプ|KMS キーソース|KMS キー ID|値|
| ---- | ---- |---| ---- | ---- |
|/fargate-test/redis-url|安全な文字列|現在のアカウント|alias/aws/ssm|ElasitCacheのプライマリエンドポイント |

イメージ
![AWS_Systems_Manager_-_Parameter_Store.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107345/9cf185ff-6f15-7a94-6d57-67b216171b79.png)
![AWS_Systems_Manager_-_Parameter_Store.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107345/4903bdc9-c975-642b-aec8-e2bb8a3c8801.png)

## 自動デプロイでSidekiqコンテナを動かす

### アプリケーションに書いたtask-definition.jsonを編集

https://qiita.com/hatsu/items/a70d5c49a39561836751

↑こちらの記事で自動デプロイの設定がすでに済んでいる前提で進めます。
deploy/task-definition.jsonの編集


<details><summary>元のtask-definition.json</summary><div>

```json
{
  "ipcMode": null,
  "executionRoleArn": "arn:aws:iam::318288222771:role/ecsTaskExecutionRole",
  "containerDefinitions": [
    {
      "dnsSearchDomains": null,
      "environmentFiles": null,
      "logConfiguration": {
        "logDriver": "awslogs",
        "secretOptions": null,
        "options": {
          "awslogs-group": "/ecs/fargate-test",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "ecs"
        }
      },
      "entryPoint": [],
      "portMappings": [
        {
          "hostPort": 80,
          "protocol": "tcp",
          "containerPort": 80
        }
      ],
      "command": [
        "/app/entrypoint.sh"
      ],
      "linuxParameters": null,
      "cpu": 0,
      "environment": [
        {
          "name": "RAILS_ENV",
          "value": "production"
        },
        {
          "name": "RAILS_LOG_TO_STDOUT",
          "value": "true"
        },
        {
          "name": "RAILS_SERVE_STATIC_FILES",
          "value": "true"
        }
      ],
      "resourceRequirements": null,
      "ulimits": null,
      "dnsServers": null,
      "mountPoints": [],
      "workingDirectory": null,
      "secrets": [
        {
          "valueFrom": "/fargate-test/db-host",
          "name": "DB_HOST"
        },
        {
          "valueFrom": "/fargate-test/db-password",
          "name": "DB_PASSWORD"
        },
        {
          "valueFrom": "/fargate-test/db-username",
          "name": "DB_USERNAME"
        },
        {
          "valueFrom": "/fargate-test/rails-master-key",
          "name": "RAILS_MASTER_KEY"
        }
      ],
      "dockerSecurityOptions": null,
      "memory": null,
      "memoryReservation": null,
      "volumesFrom": [],
      "stopTimeout": null,
      "image": "318288222771.dkr.ecr.ap-northeast-1.amazonaws.com/fargate-test:latest",
      "startTimeout": null,
      "firelensConfiguration": null,
      "dependsOn": null,
      "disableNetworking": null,
      "interactive": null,
      "healthCheck": null,
      "essential": true,
      "links": null,
      "hostname": null,
      "extraHosts": null,
      "pseudoTerminal": null,
      "user": null,
      "readonlyRootFilesystem": null,
      "dockerLabels": null,
      "systemControls": null,
      "privileged": null,
      "name": "fargate-test"
    }
  ],
  "placementConstraints": [],
  "memory": "512",
  "taskRoleArn": "arn:aws:iam::318288222771:role/ecsTaskExecutionRole",
  "compatibilities": [
    "EC2",
    "FARGATE"
  ],
  "taskDefinitionArn": "arn:aws:ecs:ap-northeast-1:318288222771:task-definition/fargate-test:5",
  "family": "fargate-test",
  "requiresAttributes": [
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.execution-role-awslogs"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.ecr-auth"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.task-iam-role"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.execution-role-ecr-pull"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.secrets.ssm.environment-variables"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.task-eni"
    }
  ],
  "pidMode": null,
  "requiresCompatibilities": [
    "FARGATE"
  ],
  "networkMode": "awsvpc",
  "cpu": "256",
  "revision": 5,
  "status": "ACTIVE",
  "inferenceAccelerators": null,
  "proxyConfiguration": null,
  "volumes": []
}
```
</div></details>


#### 編集後のtask-definition.json

やることは、`"containerDefinitions"`にSidekiqのコンテナも作ることです。

```json
{
  "ipcMode": null,
  "executionRoleArn": "arn:aws:iam::#####:role/ecsTaskExecutionRole",
  "containerDefinitions": [
    {
      "dnsSearchDomains": null,
      "environmentFiles": null,
      "logConfiguration": {
        "logDriver": "awslogs",
        "secretOptions": null,
        "options": {
          "awslogs-group": "/ecs/fargate-test",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "ecs"
        }
      },
      "entryPoint": [],
      "portMappings": [
        {
          "hostPort": 80,
          "protocol": "tcp",
          "containerPort": 80
        }
      ],
      "command": [
        "/app/entrypoint.sh"
      ],
      "linuxParameters": null,
      "cpu": 0,
      "environment": [
        {
          "name": "RAILS_ENV",
          "value": "production"
        },
        {
          "name": "RAILS_LOG_TO_STDOUT",
          "value": "true"
        },
        {
          "name": "RAILS_SERVE_STATIC_FILES",
          "value": "true"
        }
      ],
      "resourceRequirements": null,
      "ulimits": null,
      "dnsServers": null,
      "mountPoints": [],
      "workingDirectory": null,
      "secrets": [
        {
          "valueFrom": "/fargate-test/db-host",
          "name": "DB_HOST"
        },
        {
          "valueFrom": "/fargate-test/db-password",
          "name": "DB_PASSWORD"
        },
        {
          "valueFrom": "/fargate-test/db-username",
          "name": "DB_USERNAME"
        },
        {
          "valueFrom": "/fargate-test/rails-master-key",
          "name": "RAILS_MASTER_KEY"
        },
        {
          // パラメータストアからREDIS_URLを追加
          "valueFrom": "/fargate-test/redis-url",
          "name": "REDIS_URL"
        }
      ],
      "dockerSecurityOptions": null,
      "memory": null,
      "memoryReservation": null,
      "volumesFrom": [],
      "stopTimeout": null,
      "startTimeout": null,
      "firelensConfiguration": null,
      "dependsOn": null,
      "disableNetworking": null,
      "interactive": null,
      "healthCheck": null,
      "essential": true,
      "links": null,
      "hostname": null,
      "extraHosts": null,
      "pseudoTerminal": null,
      "user": null,
      "readonlyRootFilesystem": null,
      "dockerLabels": null,
      "systemControls": null,
      "privileged": null,
      "name": "fargate-test"
    },
    // ↓ここから追加
    {
      "dnsSearchDomains": null,
      "environmentFiles": null,
      "logConfiguration": {
        "logDriver": "awslogs",
        "secretOptions": null,
        "options": {
          "awslogs-group": "/ecs/fargate-test",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "ecs"
        }
      },
      "entryPoint": [],
      // portMappingsの記述は削除
      // Sidekiqを動かすコマンドに変更
      "command": [
        "bundle", "exec", "sidekiq", "-C", "config/sidekiq.yml"
      ],
      "linuxParameters": null,
      "cpu": 0,
      "environment": [
        {
          "name": "RAILS_ENV",
          "value": "production"
        },
        {
          "name": "RAILS_LOG_TO_STDOUT",
          "value": "true"
        },
        {
          "name": "RAILS_SERVE_STATIC_FILES",
          "value": "true"
        }
      ],
      "resourceRequirements": null,
      "ulimits": null,
      "dnsServers": null,
      "mountPoints": [],
      "workingDirectory": null,
      "secrets": [
        {
          "valueFrom": "/fargate-test/db-host",
          "name": "DB_HOST"
        },
        {
          "valueFrom": "/fargate-test/db-password",
          "name": "DB_PASSWORD"
        },
        {
          "valueFrom": "/fargate-test/db-username",
          "name": "DB_USERNAME"
        },
        {
          "valueFrom": "/fargate-test/rails-master-key",
          "name": "RAILS_MASTER_KEY"
        },
        {
          // パラメータストアからREDIS_URLを読み込む
          "valueFrom": "/fargate-test/redis-url",
          "name": "REDIS_URL"
        }
      ],
      "dockerSecurityOptions": null,
      "memory": null,
      "memoryReservation": null,
      "volumesFrom": [],
      "stopTimeout": null,
      "image": "#####.dkr.ecr.ap-northeast-1.amazonaws.com/fargate-test:latest",
      "startTimeout": null,
      "firelensConfiguration": null,
      "dependsOn": null,
      "disableNetworking": null,
      "interactive": null,
      "healthCheck": null,
      "essential": true,
      "links": null,
      "hostname": null,
      "extraHosts": null,
      "pseudoTerminal": null,
      "user": null,
      "readonlyRootFilesystem": null,
      "dockerLabels": null,
      "systemControls": null,
      "privileged": null,
      // コンテナ名をfargate-test-sidekiqにする
      "name": "fargate-test-sidekiq"
    }
   // ↑ここまで追加
  ],
  "placementConstraints": [],
  "memory": "512",
  "taskRoleArn": "arn:aws:iam::#####:role/ecsTaskExecutionRole",
  "compatibilities": [
    "EC2",
    "FARGATE"
  ],
  "taskDefinitionArn": "arn:aws:ecs:ap-northeast-1:#####:task-definition/fargate-test:5",
  "family": "fargate-test",
  "requiresAttributes": [
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.execution-role-awslogs"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.ecr-auth"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.task-iam-role"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.execution-role-ecr-pull"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.secrets.ssm.environment-variables"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.task-eni"
    }
  ],
  "pidMode": null,
  "requiresCompatibilities": [
    "FARGATE"
  ],
  "networkMode": "awsvpc",
  "cpu": "256",
  "revision": 5,
  "status": "ACTIVE",
  "inferenceAccelerators": null,
  "proxyConfiguration": null,
  "volumes": []
}
```

### GithubActionsの編集

<details><summary>元の.github/workflows/deploy.yml</summary><div>

```yml
on:
  pull_request:
  push:
    branches:
      - master

name: Deploy to Amazon ECS

jobs:
  deploy:
    name: Deploy
    runs-on: ubuntu-18.04
    environment: production

    steps:
    - name: Checkout
      uses: actions/checkout@v2

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: ap-northeast-1

    - name: Login to Amazon ECR
      id: login-ecr
      uses: aws-actions/amazon-ecr-login@v1

    - name: Build, tag, and push image to Amazon ECR
      id: build-image
      env:
        DOCKER_BUILDKIT: 1
        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
        ECR_REPOSITORY: fargate-test #ECRのリポジトリ名
        IMAGE_TAG: ${{ github.sha }}
      run: |
        docker build \
          -f docker/production/Dockerfile \
          --cache-from=$ECR_REGISTRY/$ECR_REPOSITORY:latest \
          --build-arg BUILDKIT_INLINE_CACHE=1 \
          -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG \
          -t $ECR_REGISTRY/$ECR_REPOSITORY:latest .
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest
        echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"

    - name: 【Rails】 Fill in the new image ID in the Amazon ECS task definition
      id: task-def-with-rails
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: deploy/task-definition.json
        container-name: fargate-test
        image: ${{ steps.build-image.outputs.image }}

    - name: Deploy Amazon ECS task definition
      uses: aws-actions/amazon-ecs-deploy-task-definition@v1
      with:
        task-definition: ${{ steps.task-def-with-rails.outputs.task-definition }} # 上のid: task-def-with-railsを利用する
        service: fargate-test
        cluster: fargate-test
        wait-for-service-stability: true
        codedeploy-appspec: deploy/appspec.yml
        codedeploy-application: AppECS-fargate-test-fargate-test
        codedeploy-deployment-group: DgpECS-fargate-test-fargate-test
```
</div></details>

#### 編集後のdeploy.yml

元の.github/workflows/deploy.ymlを編集します。

```yml
on:
  pull_request:
  push:
    branches:
      - master

name: Deploy to Amazon ECS

jobs:
  deploy:
    name: Deploy
    runs-on: ubuntu-18.04
    environment: production

    steps:
    - name: Checkout
      uses: actions/checkout@v2

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: ap-northeast-1

    - name: Login to Amazon ECR
      id: login-ecr
      uses: aws-actions/amazon-ecr-login@v1

    - name: Build, tag, and push image to Amazon ECR
      id: build-image
      env:
        DOCKER_BUILDKIT: 1
        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
        ECR_REPOSITORY: fargate-test #ECRのリポジトリ名
        IMAGE_TAG: ${{ github.sha }}
      run: |
        docker build \
          -f docker/production/Dockerfile \
          --cache-from=$ECR_REGISTRY/$ECR_REPOSITORY:latest \
          --build-arg BUILDKIT_INLINE_CACHE=1 \
          -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG \
          -t $ECR_REGISTRY/$ECR_REPOSITORY:latest .
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest
        echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"

    - name: 【Rails】 Fill in the new image ID in the Amazon ECS task definition
      id: task-def-with-rails
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: deploy/task-definition.json
        container-name: fargate-test
        image: ${{ steps.build-image.outputs.image }}

    # ↓ここから追加
    - name: 【Sidekiq】 Fill in the new image ID in the Amazon ECS task definition
      id: task-def-with-sidekiq
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: ${{ steps.task-def-with-rails.outputs.task-definition }} # 上のid: task-def-with-railsを利用する
        container-name: fargate-test-sidekiq # コンテナ名を記述
        image: ${{ steps.build-image.outputs.image }}
    # ↑ここまで追加

    - name: Deploy Amazon ECS task definition
      uses: aws-actions/amazon-ecs-deploy-task-definition@v1
      with:
        # ↓sidekiqコンテナの設定を追加したtask定義を読み込むように編集
        task-definition: ${{ steps.task-def-with-sidekiq.outputs.task-definition }} # 上のid: task-def-with-sidekiqを利用する
        service: fargate-test
        cluster: fargate-test
        wait-for-service-stability: true
        codedeploy-appspec: deploy/appspec.yml
        codedeploy-application: AppECS-fargate-test-fargate-test
        codedeploy-deployment-group: DgpECS-fargate-test-fargate-test
```


## これで完成
これで完成です。
Mainブランチにマージしてデプロイを行うと、Sidekiqも動くようになっているかと思います。

![Amazon_ECS_と_見積もりフォーマット_numbers.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107345/1686a140-e08c-dbf7-0bb1-7d788190889d.png)


![Sidekiq.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107345/4ebe2c96-0e16-b57b-4e9d-0f9c81616b96.png)