ECS Scheduled TasksをTerraformとSpring Bootで作る

ECS Scheduled Tasksは公式ドキュメントの内容が少ない上に、マネジメントコンソールでの手順しか書いていなかったので苦戦しました。なんとか動かせたので共有します。

# 環境
- macOS 14.5 (Intel版)
- Terraform 1.7.5
- Terraform AWS Provider 5.49.0
- Spring Boot 3.2.5
- JDK 21
- OrbStack 1.5.1

# アプリ
## ソースコード
```java:BatchApplication.java
@SpringBootApplication
public class EcsTestBatchApplication {

    private static final Logger logger = LoggerFactory.getLogger(EcsTestBatchApplication.class);

    public static void main(String[] args) {
        String param1 = System.getenv("PARAM1");
        String param2 = System.getenv("PARAM2");
        logger.info("バッチ開始 param1 = {}, param2 = {}", param1, param2);
        SpringApplication.run(EcsTestBatchApplication.class, args);
        logger.info("バッチ終了 param1 = {}, param2 = {}", param1, param2);
    }

}
```

## コンテナイメージにビルド
```shell
mvn clean spring-boot:build-image
```

## ECRにプッシュ
> TerraformでECR作成後に行います。

```shell
TAG=0.0.1-SNAPSHOT
aws ecr get-login-password --region ap-northeast-1 | docker login --username AWS --password-stdin (AWSアカウントID).dkr.ecr.ap-northeast-1.amazonaws.com
docker tag "ecs-test-batch:$TAG" "(AWSアカウントID).dkr.ecr.ap-northeast-1.amazonaws.com/test-batch:$TAG"
docker push "(AWSアカウントID).dkr.ecr.ap-northeast-1.amazonaws.com/test-batch:$TAG"
```

# Terraform
## main.tf
```terraform:main.tf
terraform {
  required_version = ">= 1.2.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "5.49.0"
    }
  }
  backend "local" {
    path = "terraform.tfstate"
  }
}

provider "aws" {
  profile = "プロファイル名"
  region  = "ap-northeast-1"
}
```

## ECR
```terraform:ecr.tf
resource "aws_ecr_repository" "test_batch" {
  name                 = "test-batch"
  image_tag_mutability = "IMMUTABLE"

  image_scanning_configuration {
    scan_on_push = true
  }
}
```

## VPC
```terraform:vpc.tf
# VPC
resource "aws_vpc" "test" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name = "test-vpc"
  }
}

# ap-northeast-1aのサブネット
resource "aws_subnet" "subnet_1" {
  vpc_id            = aws_vpc.test.id
  cidr_block        = "10.0.1.0/24"
  availability_zone = "ap-northeast-1a"

  tags = {
    Name = "subnet-1"
  }
}

# ap-northeast-1cのサブネット
resource "aws_subnet" "subnet_2" {
  vpc_id            = aws_vpc.test.id
  cidr_block        = "10.0.2.0/24"
  availability_zone = "ap-northeast-1c"

  tags = {
    Name = "subnet-2"
  }
}
```

## AWSサービスへのVPCエンドポイント
```terraform:aws_service_vpc_endpoint.tf
# AWSサービスに対するVPCエンドポイントのセキュリティグループ
resource "aws_security_group" "aws_service_vpc_endpoint" {
  name        = "aws-service-vpc-endpoint-sg"
  description = "Allow all access from this VPC to port 443"
  vpc_id      = aws_vpc.test.id
  tags = {
    Name = "aws-service-vpc-endpoint-sg"
  }
}

# aws-service-vpc-endpoint-sgに対する443ポートを許可
resource "aws_vpc_security_group_ingress_rule" "aws_service_vpc_endpoint_443" {
  from_port         = 443
  to_port           = 443
  ip_protocol       = "tcp"
  security_group_id = aws_security_group.aws_service_vpc_endpoint.id
  cidr_ipv4         = aws_vpc.test.cidr_block
}

# aws-service-vpc-endpoint-sgからの全egressを許可
resource "aws_vpc_security_group_egress_rule" "aws_service_vpc_endpoint" {
  ip_protocol       = "-1"
  security_group_id = aws_security_group.aws_service_vpc_endpoint.id
  cidr_ipv4         = "0.0.0.0/0"
}

# SSMへのVPCエンドポイントを作成
# 踏み台サーバーにSession Manager接続するのに必要
resource "aws_vpc_endpoint" "ssm" {
  vpc_id              = aws_vpc.test.id
  service_name        = "com.amazonaws.ap-northeast-1.ssm"
  vpc_endpoint_type   = "Interface"
  private_dns_enabled = true
  security_group_ids = [
    aws_security_group.aws_service_vpc_endpoint.id,
  ]
  subnet_ids = [
    aws_subnet.subnet_1.id,
    aws_subnet.subnet_2.id,
  ]
  tags = {
    Name = "ssm-endpoint"
  }
}

# SSM-MESSAGESへのVPCエンドポイントを作成
# 踏み台サーバーにSession Manager接続するのに必要
resource "aws_vpc_endpoint" "ssmmessages" {
  vpc_id              = aws_vpc.test.id
  service_name        = "com.amazonaws.ap-northeast-1.ssmmessages"
  vpc_endpoint_type   = "Interface"
  private_dns_enabled = true
  security_group_ids = [
    aws_security_group.aws_service_vpc_endpoint.id,
  ]
  subnet_ids = [
    aws_subnet.subnet_1.id,
    aws_subnet.subnet_2.id,
  ]
  tags = {
    Name = "ssm-messages-endpoint"
  }
}

# EC2-MESSAGESへのVPCエンドポイントを作成
# 踏み台サーバーにSession Manager接続するのに必要
resource "aws_vpc_endpoint" "ec2messages" {
  vpc_id              = aws_vpc.test.id
  service_name        = "com.amazonaws.ap-northeast-1.ec2messages"
  vpc_endpoint_type   = "Interface"
  private_dns_enabled = true
  security_group_ids = [
    aws_security_group.aws_service_vpc_endpoint.id,
  ]
  subnet_ids = [
    aws_subnet.subnet_1.id,
    aws_subnet.subnet_2.id,
  ]
  tags = {
    Name = "ec2-messages-endpoint"
  }
}

# ECSからECRのAPIを実行する際に必要
resource "aws_vpc_endpoint" "ecr_api" {
  vpc_id              = aws_vpc.test.id
  service_name        = "com.amazonaws.ap-northeast-1.ecr.api"
  vpc_endpoint_type   = "Interface"
  private_dns_enabled = true
  security_group_ids = [
    aws_security_group.aws_service_vpc_endpoint.id,
  ]
  subnet_ids = [
    aws_subnet.subnet_1.id,
    aws_subnet.subnet_2.id,
  ]
  tags = {
    Name = "ecr-api-endpoint"
  }
}

# ECSからdockerコマンドを実行する時のエンドポイントになるので必要
resource "aws_vpc_endpoint" "ecr_dkr" {
  vpc_id              = aws_vpc.test.id
  service_name        = "com.amazonaws.ap-northeast-1.ecr.dkr"
  vpc_endpoint_type   = "Interface"
  private_dns_enabled = true
  security_group_ids = [
    aws_security_group.aws_service_vpc_endpoint.id,
  ]
  subnet_ids = [
    aws_subnet.subnet_1.id,
    aws_subnet.subnet_2.id,
  ]
  tags = {
    Name = "ecr-dkr-endpoint"
  }
}

# ECRのイメージがS3に置かれているので必要
resource "aws_vpc_endpoint" "s3" {
  vpc_id       = aws_vpc.test.id
  service_name = "com.amazonaws.ap-northeast-1.s3"
  route_table_ids = [
    aws_vpc.test.main_route_table_id
  ]
  tags = {
    Name = "s3-endpoint"
  }
}

# ECSタスクからCloudWatch Logsにログを出力する際に必要
resource "aws_vpc_endpoint" "cw_logs" {
  vpc_id              = aws_vpc.test.id
  service_name        = "com.amazonaws.ap-northeast-1.logs"
  vpc_endpoint_type   = "Interface"
  private_dns_enabled = true
  security_group_ids = [
    aws_security_group.aws_service_vpc_endpoint.id,
  ]
  subnet_ids = [
    aws_subnet.subnet_1.id,
    aws_subnet.subnet_2.id,
  ]
  tags = {
    Name = "cloudwatch-logs-endpoint"
  }
}

# ECSタスクがSecrets Managerからシークレットを取得する際に必要
resource "aws_vpc_endpoint" "secrets_manager" {
  vpc_id              = aws_vpc.test.id
  service_name        = "com.amazonaws.ap-northeast-1.secretsmanager"
  vpc_endpoint_type   = "Interface"
  private_dns_enabled = true
  security_group_ids = [
    aws_security_group.aws_service_vpc_endpoint.id,
  ]
  subnet_ids = [
    aws_subnet.subnet_1.id,
    aws_subnet.subnet_2.id,
  ]
  tags = {
    Name = "secrets-manager-endpoint"
  }
}
```

## ECSクラスター
```terraform:ecs_cluster.tf
# ECSクラスタの作成
resource "aws_ecs_cluster" "test" {
  name = "test-cluster"

  setting {
    name  = "containerInsights"
    value = "enabled"
  }
}
```

## ECSタスクのIAMロール
```terraform:ecs_batch_task.tf
# ECSタスクロール
resource "aws_iam_role" "batch" {
  name = "ecs-batch-task-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Sid    = ""
        Effect = "Allow"
        Principal = {
          Service = "ecs-tasks.amazonaws.com"
        }
        Action = "sts:AssumeRole"
      }
    ]
  })
}

# タスクにECS ExecできるようにするためのIAMポリシー
resource "aws_iam_policy" "ecs_exec" {
  name        = "ecs-exec-policy"
  path        = "/"
  description = "ECS Exec"
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      # タスクにECS Execできるようにする
      # https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html#ecs-exec-required-iam-permissions
      {
        Effect = "Allow"
        Action = [
          "ssmmessages:CreateControlChannel",
          "ssmmessages:CreateDataChannel",
          "ssmmessages:OpenControlChannel",
          "ssmmessages:OpenDataChannel",
        ]
        Resource = [
          "*",
        ]
      },
    ]
  })
}

resource "aws_iam_role_policy_attachment" "ecs_exec_batch" {
  policy_arn = aws_iam_policy.ecs_exec.arn
  role       = aws_iam_role.batch.name
}
```

## ECSのタスク実行ロール
```terraform:ecs_task_execution_role.tf
# タスク起動用IAMロールの定義
resource "aws_iam_role" "ecs_task_execution" {
  name = "ecs-task-execution-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Sid    = ""
        Effect = "Allow"
        Principal = {
          Service = "ecs-tasks.amazonaws.com"
        }
        Action = "sts:AssumeRole"
      }
    ]
  })
}

# タスク起動用IAMロールでECRからイメージをプルできるようにする
resource "aws_iam_role_policy_attachment" "ecs_task_execution" {
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy" # ECRからイメージをプルできるようになる
  role       = aws_iam_role.ecs_task_execution.name
}

# タスク起動時に
# - Secrets Managerにアクセスして、シークレット値を環境変数に設定できるようにする
resource "aws_iam_role_policy" "ecs_task_execution" {
  name = "ecs-execution-additional-policy"
  role = aws_iam_role.ecs_task_execution.name
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      # Secrets Managerへのアクセス
      {
        Effect = "Allow"
        Action = [
          "secretsmanager:GetSecretValue",
        ]
        Resource = [
          data.aws_secretsmanager_secret.ecs_batch.arn,
        ]
      },
    ]
  })
}
```

## Secrets Manager
👇のようなJSONを保存します。

```json:Secrets_Managerのシークレット
{
  "param1": "value1",
  "param2": "value2"
}
```

Data Sourceで参照します。

```terraform:secrets_manager.tf
data "aws_secretsmanager_secret" "ecs_batch" {
  name = "test/ecs-batch"
}
```

## タスク定義
```terraform:ecs_batch_task_definition.tf
# タスク定義の作成
resource "aws_ecs_task_definition" "batch" {
  family                   = "test-batch"
  network_mode             = "awsvpc" # Fargateの場合は"awsvpc"一択
  requires_compatibilities = ["FARGATE"]
  execution_role_arn       = aws_iam_role.ecs_task_execution.arn
  task_role_arn            = aws_iam_role.batch.arn
  cpu                      = 1024 # GuardDuty Agentのコンテナもタスクに含まれるため、少し大きめにする
  memory                   = 2048 # GuardDuty Agentのコンテナもタスクに含まれるため、少し大きめにする

  container_definitions = jsonencode([
    {
      name      = local.web_app_container_name
      image     = "${aws_ecr_repository.test_batch.repository_url}:0.0.1"
      cpu       = 512
      memory    = 1024
      essential = true
      logConfiguration = {
        logDriver = "awslogs"
        options = {
          awslogs-region        = "ap-northeast-1"
          awslogs-group         = "任意のロググループ名"
          awslogs-stream-prefix = "ログストリームのプレフィックス"
        }
      }
      # ログのタイムスタンプが日本時間で出るようにする
      environment = [
        {
          name  = "TZ"
          value = "Asia/Tokyo"
        },
      ]
      # Secrets Managerからシークレット値を取得して環境変数に設定
      secrets = [
        {
          name      = "PARAM1"                                                  # 環境変数名
          valueFrom = "${data.aws_secretsmanager_secret.ecs_batch.arn}:param1::" # Secrets ManagerのARN
        },
        {
          name      = "PARAM2"
          valueFrom = "${data.aws_secretsmanager_secret.ecs_batch.arn}:param2::"
        },
      ]
    },
  ])

  runtime_platform {
    operating_system_family = "LINUX"
    cpu_architecture        = "X86_64"
  }
}
```

## EventBridge SchedulerのIAMロール
SchedulerがECS Scheduled Taskを実行します。

> EventBridgeでもできますが、EventBridge Schedulerの方がcronで日本時間を指定可能など、色々と便利です。（[Blackbelt資料](https://pages.awscloud.com/rs/112-TZM-766/images/AWS-Black-Belt_2023_Amazon-EventBridge-Scheduler_0930_v1.pdf)のP.9参照）

```terraform:scheduler_role.tf
resource "aws_iam_role" "ecs_batch_scheduler" {
  name = "ecs-batch-scheduler-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Sid    = ""
        Effect = "Allow"
        Principal = {
          Service = "scheduler.amazonaws.com" # EventBridge SchedulerがこのIAMロールを使用する
        }
        Action = "sts:AssumeRole"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "ecs_batch_scheduler" {
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceEventsRole"
  role       = aws_iam_role.ecs_batch_scheduler.name
}
```

## EventBridge Schedulerの作成
```terraform:scheduler.tf
resource "aws_security_group" "batch" {
  name   = "ecs-batch-sg"
  vpc_id = aws_vpc.test.id
}

resource "aws_vpc_security_group_egress_rule" "batch" {
  security_group_id = aws_security_group.batch.id
  ip_protocol       = "-1"
  cidr_ipv4         = "0.0.0.0/0"
}

resource "aws_scheduler_schedule" "ecs_batch" {
  name                         = "run-ecs-test-batch"
  schedule_expression_timezone = "Asia/Tokyo"         # 日本のタイムゾーンを指定
  schedule_expression          = "cron(30 9 * * ? *)" # 日本時間の毎朝9:30に実行

  flexible_time_window {
    mode                      = "FLEXIBLE"
    maximum_window_in_minutes = 1
  }

  target {
    arn      = aws_ecs_cluster.test.arn             # ECSクラスターのARN
    role_arn = aws_iam_role.ecs_batch_scheduler.arn # ECSバッチスケジューラーのIAMロールのARN

    ecs_parameters {
      task_definition_arn    = aws_ecs_task_definition.batch.arn # ECSタスク定義のARN
      enable_execute_command = true
      launch_type            = "FARGATE"
      task_count             = 1 # 動かすタスクの数

      network_configuration {
        assign_public_ip = false
        security_groups = [
          aws_security_group.batch.id
        ]
        subnets = [
          aws_subnet.subnet_1.id,
          aws_subnet.subnet_2.id,
        ]
      }
    }
  }
}
```

# 動作確認
CloudWatch Logsでログが出ているか確認してください。動かない場合は👇の記事を参照してください。

https://repost.aws/ja/knowledge-center/ecs-scheduled-task-issues

# 参考資料
- [ECSのAWS公式ドキュメント](https://docs.aws.amazon.com/ja_jp/AmazonECS/latest/developerguide/tasks-scheduled-eventbridge-scheduler.html)
- [EvemtBridge SchedulerのBlackbelt資料](https://pages.awscloud.com/rs/112-TZM-766/images/AWS-Black-Belt_2023_Amazon-EventBridge-Scheduler_0930_v1.pdf)
- [Terraform AWS Providerの公式ドキュメント](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/scheduler_schedule)