awscliでlocalstackにecs/fargateを作ってバッチを動かす

中身は↓と同じ
https://zenn.dev/notfound_pan/articles/e6b742eeb83afa

# 注意事項

僕はpro課金勢です
無料版では動くかは確認してません

# 前の記事
https://qiita.com/kk-tn/items/a0b6d1c32feea91d8fd0

# やりたいこと

- localstackでecs/fargateを動かしたい
    - いつも通りawsは課金が怖くて使えないから

# 今のところの感想

かなりいろんなところで詰まった

localstackでどこまでできるのかわからない

トラブったときに自分が悪いのかlocalstackの制限、仕様なのか判断がつかない

# 今回作りたいもの

AWSでつくるならこんな感じ

- 開発環境からawscliでECR、ECS、S3を作る
- 開発環境でECSで動かすことになるdockerのimageを作る
- 作ったイメージはdocker cliでECRにpush
- 全部できたらawscliでECSのタスクをrunする

・・・というものをlocalstackで作る

```mermaid
flowchart LR

%%外部要素
subgraph DEV[開発環境]
	subgraph Docker[Docker]
	end
end

%%グループとサービス
subgraph AWS[AWS]
	subgraph ECS[ECS]
		CONTINER("CONTINER")
	end
ECR("ECR")
S3[("S3")]
end

%%サービス同士の関係
DEV --> |create aws cli|S3
DEV --> |create aws cli|ECR
DEV --> |create aws cli|ECS
DEV --> |push image|ECR 
DEV --> |run task|CONTINER
ECR --> |pull image|ECS
CONTINER --> |put item|S3

%%グループのスタイル
classDef SAWS fill:none,color:#345,stroke:#345
class AWS SAWS

classDef SDEV fill:none,color:#345,stroke:#345
class DEV SDEV

classDef SECS fill:#e83,color:#fff,stroke:none
class ECS SECS

%%サービスのスタイル
classDef SOR fill:#aaa,color:#fff,stroke:#fff
class USER SOR

classDef SECR fill:#e83,color:#fff,stroke:none
class ECR SECR

classDef SST fill:#493,color:#fff,stroke:#fff
class S3 SST
```

localstackで作るとこんな構成・・・といっていいのか

dood構成なこともあって思ったように図が書けない

まあ、だいたいこんな感じってことで

```mermaid
flowchart LR

%%外部要素
subgraph Docker[Docker]
	subgraph DEV[開発コンテナ]
	end
	subgraph LOCALSTACK[LOCALSTACK]
		ECR("ECR")
		S3[("S3")]
		ECS[("ECS")]
	end
	CONTINER("CONTINER")
end

%%グループとサービス

%%サービス同士の関係
DEV --> |create aws cli|S3
DEV --> |create aws cli|ECR
DEV --> |create aws cli|ECS
DEV --> |push image|ECR
ECR --> |pull image|ECS
DEV --> |run task|ECS
ECS--> |run task|CONTINER

CONTINER --> |put item|S3

%%グループのスタイル
classDef SAWS fill:none,color:#345,stroke:#345
class AWS SAWS

classDef SDEV fill:none,color:#345,stroke:#345
class DEV SDEV

classDef SECS fill:#e83,color:#fff,stroke:none
class ECS SECS

%%サービスのスタイル
classDef SOR fill:#aaa,color:#fff,stroke:#fff
class USER SOR

classDef SECR fill:#e83,color:#fff,stroke:none
class ECR SECR

classDef SST fill:#493,color:#fff,stroke:#fff
class S3 SST
```

# フォルダ構成

こんな感じ

```bash
satoshi@Ubuntu:/workspace/projects$ tree localstack -La 4
localstack
├── dev_lambnda_python3.9.16_Dockerfile
├── docker-compose.yml
├── localstack_with_dockercli
└── mount
    ├── dev
    │   ├── home
    │   │   └── .aws
    │   └── work
    └── localstack
```

mount以下はホストとコンテナで共有するvolume

.awsはaws configureした後にできるファイルたちを残したいからコンテナ内の~/.awsを保持してもらう。

## 各ファイル

### dev_lambnda_python3.9.16_Dockerfile

```docker
FROM ubuntu:jammy

#base
ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update
RUN apt install -y apt-transport-https ca-certificates curl software-properties-common \
	&& curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - \
	&& add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu jammy stable"

RUN apt-get update
RUN apt-get install -y apt-utils sudo locales tzdata \
	iputils-ping net-tools dnsutils jq tzdata curl gcc make tree \
	g++ git unzip groff-base \
	build-essential libtool texinfo dpkg-dev pkg-config \
	gfortran ccache qt5-qmake \
	zlib1g-dev libffi-dev libbz2-dev libreadline-dev libssl-dev libsqlite3-dev liblzma-dev tk-dev docker-ce-cli \
	&& apt-get clean \
	&& rm -rf /var/lib/apt/lists/*

RUN localedef -f UTF-8 -i ja_JP ja_JP.UTF-8
ENV LANG ja_JP.UTF-8
ENV LC_ALL ja_JP.UTF-8
ENV LANGUAGE ja_JP:ja
ENV TZ Asia/Tokyo

RUN cd /tmp \ 
	&& curl -O https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tgz \
	&& rm -rf Python-3.9.16 \
	&& tar -xvzof Python-3.9.16.tgz \
	&& cd ./Python-3.9.16 \
	&& ./configure --enable-optimizations \
	&& make \
	&& sudo make install

RUN python3 -m pip install -U pip setuptools \
	&& pip install --upgrade pip \
	&& pip install pipenv

RUN groupadd -g 1000 dev-lambda-user \
	&& useradd -u 1000 -g 1000 -m dev-lambda-user \
    && useradd -g 1000 -G sudo -m -s /bin/bash iganari \
	&& echo "dev-lambda-user:passwd" | chpasswd \
	&& echo "root:passwd" | chpasswd
RUN echo 'Defaults visiblepw'             >> /etc/sudoers
RUN echo 'dev-lambda-user ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers

RUN cd /tmp \
	&& curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" \
	&& unzip awscliv2.zip \
	&& sudo ./aws/install \
	&& rm -fr aws/ awscliv2.zip \
	&& pip install awscli-local \
	&& pip install aws-sam-cli \
	&& pip install aws-sam-cli-local

USER dev-lambda-user

SHELL [ "/bin/bash", "-c" ]
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash

RUN . /home/dev-lambda-user/.nvm/nvm.sh \
    && nvm install --lts \
    && nvm use --lts \
	&& nvm cache clear \
    && node -v && npm -v \
    && npm i -g yarn wscat

SHELL [ "/bin/bash", "-c" ]
RUN echo -e 'export PATH=/home/dev-lambda-user/.local/bin:$PATH' >> /home/dev-lambda-user/.bashrc \
	&& echo -e 'export PIPENV_VENV_IN_PROJECT=true' >> /home/dev-lambda-user/.bashrc \
	&& source /home/dev-lambda-user/.bashrc

RUN echo -e "\
export NVM_DIR=\"\$HOME/.nvm\"\n\
[ -s \"\$NVM_DIR/nvm.sh\" ] && \. \"\$NVM_DIR/nvm.sh\"\n\
" >> /home/dev-lambda-user/.bashrc \
	&& source /home/dev-lambda-user/.bashrc

RUN echo -e "complete -C '/usr/local/bin/aws_completer' aws" >> /home/dev-lambda-user/.bashrc \
	&& echo -e "complete -C '/usr/local/bin/aws_completer' awslocal" >> /home/dev-lambda-user/.bashrc \
	&& source /home/dev-lambda-user/.bashrc
	
CMD ["bash"]
```

ecrにpushしたいからdocker cliをイメージに追加している

### localstack_with_dockercli

```docker
FROM localstack/localstack-pro:1.4.0

#base
ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update
RUN apt install -y apt-transport-https ca-certificates curl software-properties-common \
	&& curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
RUN echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian \
  $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null

RUN apt-get update
RUN apt-get install -y libc6 docker-ce-cli iputils-ping net-tools dnsutils tree \
	&& apt-get clean \
	&& rm -rf /var/lib/apt/lists/*
```

### docker-compose.yml

```docker
version: "3.7"

services:
  localstack:
    container_name: "localstack_a04"
    build:
      context: .
      dockerfile: localstack_with_dockercli
    user: "${UID}:${GID}"
    ports:
      - "127.0.0.1:4566:4566"            # LocalStack Gateway
      - "127.0.0.1:4510-4559:4510-4559"  # external services port range
      - "127.0.0.1:53:53"                # DNS config (only required for Pro)
      - "127.0.0.1:53:53/udp"            # DNS config (only required for Pro)
      - "127.0.0.1:443:443"              # LocalStack HTTPS Gateway (only required for Pro
      - 8000:8080
    environment:
      - SERVICES=${SERVICES- }
      - DEBUG=1
      # - LEGACY_DOCKER_CLIENT=1
      - DATA_DIR=${DATA_DIR- }
      - LAMBDA_EXECUTOR=${LAMBDA_EXECUTOR- }
      - LAMBDA_DOCKER_NETWORK=localstack.internal
      - LOCALSTACK_API_KEY=${LOCALSTACK_API_KEY- }  # only required for Pro
      - HOST_TMP_FOLDER=${TMPDIR:-/tmp/}localstack
      - DOCKER_HOST=unix:///var/run/docker.sock
      - LAMBDA_EXECUTOR=docker-reuse
      - HOSTNAME_EXTERNAL=localstack
    volumes:
      - "/workspace/projects/localstack/mount/localstack/var/lib/localstack:/var/lib/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"
    networks:
      - container-link
  devlambda:
    build:
      context: .
      dockerfile: dev_lambnda_python3.9.16_Dockerfile
    container_name: "localstack_dev_a03"
    user: "dev-lambda-user:dev-lambda-user"
    environment:
      - LOCALSTACK_HOSTNAME=localstack
      - LOCALSTACK_HOST=localstack
      - DEFAULT_REGION=ap-northeast-1
    volumes:
      - "/workspace/projects/localstack/mount/dev/home/.aws:/home/dev-lambda-user/.aws"
      - "/workspace/projects/localstack/mount/dev/work:/work"
      - "/var/run/docker.sock:/var/run/docker.sock"
    tty: true
    working_dir: /work
    networks:
      - container-link
networks:
  container-link:
    name: localstack.internal
```

### ポイント

```bash
      - 4510:4510
```

ecrが4510ポートで開くので足しとく

doodで動かすのでホスト側から4510につながらないと後段でのecrへのpushができなくなる

```tsx
      - LAMBDA_DOCKER_NETWORK=localstack.internal
```

これが結構はまった

localstackはecs/fargateだけサポートされており、コンテナ実行時にはlocalstackとは別のコンテナを起動してそこでアプリを動かすらしい

localstack、開発コンテナともにコンテナで動かしているのでさらにそこからecs/fargate用にコンテナを動かすときには以下のどちらかでやることになる

- dind（docker in docker）
- dood（docker-outside-of-docker）

dindはdockerをPrivilegedで動かす必要があるので避けたい

[Dockerコンテナを特権モードで実行することが危険な理由](https://www.trendmicro.com/ja_jp/research/20/a/why-it-is-dangerous-to-run-Docker-containers-in-privileged-mode.html)

doodはコンテナ内からホストのdockerを動かすイメージ

そのために

```tsx
environment:
・・・
      - DOCKER_HOST=unix:///var/run/docker.sock
・・・
    volumes:
      - "/workspace/projects/localstack/mount/localstack/var/lib/localstack:/var/lib/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"
```

この辺でコンテナ内のdocker cliがみるdocker.sockをホストのものにつないでいる

結局のところdoodでdocker cliを叩いてもホスト側のdocker cliを叩いているのと同じこと

で、先のdocker-compose.ymlでlocalstackを起動するとlocalstackと開発用のコンテナはlocalstack.internalに属する

一方、ecs run-taskで実行するとデフォルトではfargate相当のコンテナはdockerのデフォルトのbridgeにぶち込まれる

このため、ecsで動かしているコンテナ内でlocalstack.internalにいるlocalstackのコンテナにあるs3につなごうとするとネットワーク的につながらない

これはダメかとあきらめかけたら公式ドキュメントに以下の記述が

[https://docs.localstack.cloud/references/configuration/#lambda-new--asf](https://docs.localstack.cloud/references/configuration/#lambda-new--asf)
![](https://storage.googleapis.com/zenn-user-upload/9ae6cefffd45-20230325.png)

LAMBDA_DOCKER_NETWORK環境変数で指定したdocker networkの設定はecsのコンテナにも効くらしい

いや、lambdaセクションに書かれても気が付かないって・・・

# ecsで動かすDockerイメージを作る

冒頭のフォルダ構成でいうと

```bash
/workspace/projects/localstack$ 
```

上記パスでdocker compose upするとlocalstackと開発用コンテナが動く

## 開発環境の準備

### vscodeで開発用のコンテナにつなぐ
![](https://storage.googleapis.com/zenn-user-upload/f3174ff9b250-20230325.png)
### /workを開く
![](https://storage.googleapis.com/zenn-user-upload/bb3b7becc6e0-20230325.png)

docker-compose.ymlでホストの/workspace/projects/localstack/mount/dev/workとつなげているので開発コンテナ内ではそこが/workでみえる

初期状態ではworkの下は空

```docker
      - "/workspace/projects/localstack/mount/dev/work:/work"
```

以降はvscodeのターミナルで実行

### aws configureでlocalstackにつなぐときのprofileを作る

適当でOK

```bash
dev-lambda-user@6ebb0a4f7fd1:/work$ aws configure --profile localstack
AWS Access Key ID [None]: dummy
AWS Secret Access Key [None]: dummy
Default region name [None]: ap-northeast-1
Default output format [None]: json
```

### プロジェクトフォルダの作成とpipenvの仮想環境の作成

プロジェクトのフォルダを掘る

ecs-batchにした

その中にpythonの仮想環境を開発コンテナ内につくる

```bash
dev-lambda-user@f220fa55d305:/work$ mkdir ecs-batch
dev-lambda-user@f220fa55d305:/work$ cd ecs-batch/
dev-lambda-user@f220fa55d305:/work/ecs-batch$ pipenv --python 3.9.16
Creating a virtualenv for this project...
Pipfile: /work/ecs-batch/Pipfile
Using /usr/local/bin/python3.9 (3.9.16) to create virtualenv...
⠹ Creating virtual environment...created virtual environment CPython3.9.16.final.0-64 in 645ms
  creator CPython3Posix(dest=/work/ecs-batch/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/dev-lambda-user/.local/share/virtualenv)
    added seed packages: pip==23.0.1, setuptools==67.4.0, wheel==0.38.4
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /work/ecs-batch/.venv
Creating a Pipfile for this project...
```

仮想環境に入る

```bash
dev-lambda-user@f220fa55d305:/work/ecs-batch$ pipenv shell
Launching subshell in virtual environment...
 . /work/ecs-batch/.venv/bin/activate
```

## ecsで動かすDockerイメージを作る

### ecsでうごかすプログラムを作る

boto3とpytzを入れる

```bash
(ecs-batch) dev-lambda-user@f220fa55d305:/work/ecs-batch$ pipenv install boto3 pytz
Installing boto3...
Resolving boto3...
Installing...
Adding boto3 to Pipfile's [packages] ...
✔ Installation Succeeded
Installing pytz...
Resolving pytz...
Installing...
Adding pytz to Pipfile's [packages] ...
✔ Installation Succeeded
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [dev-packages] dependencies...
Updated Pipfile.lock (cbe17059447d77e590767cf847fd02fd2dd19b8e3612e8660e439e6766392a33)!
Installing dependencies from Pipfile.lock (392a33)...
```

### bat01.py

ecsで動かすプログラムがこれ

s3にファイルを置くだけ

/ecs-batchの下に作っておく

```python
# -*- coding: utf-8 -*-
import boto3
import sys
import datetime
import pytz

now = datetime.datetime.now(pytz.timezone('Asia/Tokyo'))
endpoint_url = 'http://localstack:4566/'
s3 = boto3.resource("s3", region_name='ap-northeast-1',endpoint_url=endpoint_url, aws_access_key_id='dummy',aws_secret_access_key='dummy')
bucket = s3.Bucket('my-bucket')
src_object_body = bucket.put_object(Key='key01',Body= now.isoformat())

print (src_object_body)
```

requirements.txtも作っておく

```python
(ecs-batch) dev-lambda-user@f220fa55d305:/work/ecs-batch$ pipenv requirements > requirements.txt
```

### Dockerイメージを作る

ecsでうごかすdockerイメージを作る

/ecs-batchの下にDockerfileを置いておく

Dockerfileは以下

「/」にソースをコピーしちゃってるが動けばいいから深く考えない

### Dockerfile

```docker
FROM python:3.9
USER root

RUN apt-get update && apt-get install -y --no-install-recommends \
	curl \
	unzip

RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
RUN unzip awscliv2.zip
RUN ./aws/install

RUN pip install --upgrade pip
RUN pip install --upgrade setuptools

COPY ./requirements.txt ./
COPY ./bat01.py ./

RUN pip install --no-cache-dir -r requirements.txt

ENTRYPOINT ["python3", "bat01.py"]
```

requirements.txtとbat01.pyはイメージの中にCOPYしてpip installで依存関係も入れる

### Dockerイメージのビルド

ecrに入れるイメージを開発環境側のコンテナ内でbuild

dood構成なのでできたimageはホスト側にできる

```bash
dev-lambda-user@f220fa55d305:/work/ecs-batch$ sudo docker build --no-cache -t localhost:4510/my-batch .
[+] Building 31.5s (15/15) FINISHED                                                                                                                                                                                                 
 => [internal] load build definition from Dockerfile                                                                                                                                                                           0.0s
 => => transferring dockerfile: 32B                                                                                                                                                                                            0.0s
 => [internal] load .dockerignore                                                                                                                                                                                              0.0s
 => => transferring context: 2B                                                                                                                                                                                                0.0s
 => [internal] load metadata for docker.io/library/python:3.9                                                                                                                                                                  0.0s
 => CACHED [ 1/10] FROM docker.io/library/python:3.9                                                                                                                                                                           0.0s
 => [internal] load build context                                                                                                                                                                                              0.0s
 => => transferring context: 66B                                                                                                                                                                                               0.0s
 => [ 2/10] RUN apt-get update && apt-get install -y --no-install-recommends  curl  unzip                                                                                                                                      4.6s
 => [ 3/10] RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"                                                                                                                              6.1s
 => [ 4/10] RUN unzip awscliv2.zip                                                                                                                                                                                             3.8s 
 => [ 5/10] RUN ./aws/install                                                                                                                                                                                                  2.7s
 => [ 6/10] RUN pip install --upgrade pip                                                                                                                                                                                      4.4s
 => [ 7/10] RUN pip install --upgrade setuptools                                                                                                                                                                               2.1s
 => [ 8/10] COPY ./requirements.txt ./                                                                                                                                                                                         0.1s
 => [ 9/10] COPY ./bat01.py ./                                                                                                                                                                                                 0.1s
 => [10/10] RUN pip install --no-cache-dir -r requirements.txt                                                                                                                                                                 6.0s
 => exporting to image                                                                                                                                                                                                         1.3s
 => => exporting layers                                                                                                                                                                                                        1.3s
 => => writing image sha256:e4a4f9f937a4832f1cbec28aee7113c5845f2107d6efe13710c2388a77b1f20d                                                                                                                                   0.0s
 => => naming to localhost:4510/my-batch
```

注意したいのが-t localhost:4510/my-batchにしているところ

前述したようにコンテナ内でdockerコマンドを打ってもホストのdockerを動かすことになるのでlocalhost:4510にしておく

そうしないとホストからecrが見えない

ホスト側でdocker psするとホスト側にイメージができているのが見える

```bash
satoshi@Ubuntu:~$ docker images|grep localhost:4510/my-batch
localhost:4510/my-batch                       latest             e4a4f9f937a4   48 seconds ago       1.48GB
```

# 作ったイメージをecrにpush

## ecrにリポジトリを作っておく

```bash
dev-lambda-user@f220fa55d305:/work/ecs-batch$ awslocal ecr create-repository --repository-name my-batch
{
    "repository": {
        "repositoryArn": "arn:aws:ecr:ap-northeast-1:000000000000:repository/my-batch",
        "registryId": "000000000000",
        "repositoryName": "my-batch",
        "repositoryUri": "localstack:4510/my-batch",
        "createdAt": "2023-03-24T14:54:40+00:00",
        "imageTagMutability": "MUTABLE",
        "imageScanningConfiguration": {
            "scanOnPush": false
        },
        "encryptionConfiguration": {
            "encryptionType": "AES256"
        }
    }
}
```

## ecrにイメージをpush

```bash
dev-lambda-user@f220fa55d305:/work/ecs-batch$ sudo docker push localhost:4510/my-batch
Using default tag: latest
The push refers to repository [localhost:4510/my-batch]
e0aca79f1c6a: Preparing 
8e7fb646ab35: Preparing 
3ebdd45f5fb9: Preparing 
8fd18d301a62: Preparing 
c76dae780f3b: Preparing 
59b28b9c6ebc: Preparing 
86b6e3833850: Pushed 
8091b198121e: Pushed 
6fe128a522be: Pushed 
26f5bb747440: Pushed 
2383c46b6fbe: Pushed 
7739e9ebed9a: Pushed 
248397b6b856: Pushed 
fa1175420e6f: Pushed 
bb2453e12947: Pushed 
7354e83da007: Pushed 
c284f546974c: Pushed 
4efcd4003c84: Pushed 
latest: digest: sha256:be79ff1ffc4ffca359f015a7f5162f10d2169ceec0339a7909f78011f879dddf size: 4114
```

# ecsクラスタとtask definitionを作る

## ecsのクラスタを作る

```bash
dev-lambda-user@f220fa55d305:/work/ecs-batch$ awslocal ecs create-cluster --cluster-name ecs-fargate-batch
{
    "cluster": {
        "clusterArn": "arn:aws:ecs:ap-northeast-1:000000000000:cluster/ecs-fargate-batch",
        "clusterName": "ecs-fargate-batch",
        "status": "ACTIVE",
        "registeredContainerInstancesCount": 0,
        "runningTasksCount": 0,
        "pendingTasksCount": 0,
        "activeServicesCount": 0
    }
}
```

できたクラスタをみる

```bash
dev-lambda-user@f220fa55d305:/work/ecs-batch$ awslocal ecs list-clusters
{
    "clusterArns": [
        "arn:aws:ecs:ap-northeast-1:000000000000:cluster/ecs-fargate-batch"
    ]
}
```

## task definitionを作る

こいつがecsで動かすタスク（コンテナ）のひな型になる

/ecs-batch/task-def.jsonに以下を書く

### task-def.json

```json
{
    "family": "fargate-task-Definition", 
    "networkMode": "awsvpc", 
    "requiresCompatibilities": [ "FARGATE" ], 
    "cpu": "256", 
    "memory": "512",
    "containerDefinitions": [
        {
            "name": "fargate-batch-app", 
            "image": "localhost:4510/my-batch:latest", 
            "essential": true
       }
    ]
}
```

task-def.jsonの内容でtask definitionを作る

```bash
dev-lambda-user@f220fa55d305:/work/ecs-batch$ awslocal ecs register-task-definition --cli-input-json file://task-def.json
{
    "taskDefinition": {
        "taskDefinitionArn": "arn:aws:ecs:ap-northeast-1:000000000000:task-definition/fargate-task-Definition:1",
        "containerDefinitions": [
            {
                "name": "fargate-batch-app",
                "image": "localhost:4510/my-batch:latest",
                "cpu": 0,
                "portMappings": [],
                "essential": true,
                "environment": [],
                "mountPoints": [],
                "volumesFrom": []
            }
        ],
        "family": "fargate-task-Definition",
        "networkMode": "awsvpc",
        "revision": 1,
        "volumes": [],
        "status": "ACTIVE",
        "placementConstraints": [],
        "compatibilities": [
            "EC2",
            "FARGATE"
        ],
        "requiresCompatibilities": [
            "FARGATE"
        ],
        "cpu": "256",
        "memory": "512"
    }
}
```

task definitionみる

```bash
dev-lambda-user@f220fa55d305:/work/ecs-batch$ awslocal ecs list-task-definitions | jq .
{
  "taskDefinitionArns": [
    "arn:aws:ecs:ap-northeast-1:000000000000:task-definition/fargate-task-Definition:1"
  ]
}
```

# タスクを実行する

アプリはs3のバケットに書き込むだけなのでバケットを先に作っておく

```bash
(ecs-batch) dev-lambda-user@f220fa55d305:/work/ecs-batch$ awslocal s3 mb s3://my-bucket
make_bucket: my-bucke
```

run-taskで実行するのだけど、subnetとsgが必要なのでデフォルトのものをlocalstackから取得してそれをrun-taskに渡す

subnetとsgは作るのめんどくさいしlocalstackはその辺適当っぽいのでありものを使う

以下スクリプトを/ecs-batch/run_task.shを作って以下を書く

run-taskにはクラスタ名、task-definition、SubnetId、securityGroupのIDが必要

run-task後に以下でタスクの終了を待つ

```bash
awslocal ecs wait tasks-stopped --tasks $TASK_ARN --cluster $CLUSTER_ARN
```

```bash
#!/bin/bash
set -eu
TARGET_SUBNET_ID=`awslocal ec2 describe-subnets --output json \
    |jq -r '.Subnets[] | select(.CidrBlock == "172.31.16.0/20") | .SubnetId'`

echo $TARGET_SUBNET_ID

TARGET_SG_ID=`awslocal ec2 describe-security-groups --output=json \
    |jq -r '.SecurityGroups[] | select(.GroupName == "default") | .GroupId'`

echo $TARGET_SG_ID

RUN_TASK_RESULT=`awslocal ecs run-task \
    --cluster ecs-fargate-batch \
    --task-definition fargate-task-Definition \
    --launch-type FARGATE \
    --network-configuration "awsvpcConfiguration={subnets=[$TARGET_SUBNET_ID],securityGroups=[$TARGET_SG_ID],assignPublicIp=ENABLED}"`
echo $RUN_TASK_RESULT

CLUSTER_ARN=`awslocal ecs list-clusters | jq -r .clusterArns[0]`
echo $CLUSTER_ARN

TASK_ARN=`echo $RUN_TASK_RESULT | jq -r '.tasks[0].taskArn'`
echo $TASK_ARN

awslocal ecs wait tasks-stopped --tasks $TASK_ARN --cluster $CLUSTER_ARN
```

```bash
dev-lambda-user@f220fa55d305:/work/ecs-batch$ ./run_task.sh 
subnet-13089d2d
sg-c5dd45db2fc6d9a37
{ "tasks": [ { "attachments": [ { "id": "cddc14a8-df33-4b6c-8c8b-b32ad3bec21a", "type": "ElasticNetworkInterface", "status": "ATTACHED", "details": [ { "name": "subnetId", "value": "subnet-13089d2d" }, { "name": "networkInterfaceId", "value": "eni-1b3218dd" }, { "name": "macAddress", "value": "02:00:00:17902x:7102x:8502x" }, { "name": "privateDnsName", "value": "ip-10-203-101-56.ec2.internal" }, { "name": "privateIPv4Address", "value": "10.203.101.56" } ] } ], "clusterArn": "arn:aws:ecs:ap-northeast-1:000000000000:cluster/ecs-fargate-batch", "containers": [ { "containerArn": "arn:aws:ecs:ap-northeast-1:000000000000:container/ecs-fargate-batch/4662ad27-34c6-4ee3-9b33-6ba6e984c08f/bb46b4aa", "taskArn": "arn:aws:ecs:ap-northeast-1:000000000000:task-definition/fargate-task-Definition:1", "name": "fargate-batch-app", "image": "localhost:4510/my-batch:latest", "lastStatus": "RUNNING", "networkBindings": [], "networkInterfaces": [], "healthStatus": "UNKNOWN", "cpu": "0", "memory": "32" } ], "desiredStatus": "RUNNING", "lastStatus": "RUNNING", "launchType": "FARGATE", "overrides": {}, "startedBy": "", "stoppedReason": "", "tags": [], "taskArn": "arn:aws:ecs:ap-northeast-1:000000000000:task/ecs-fargate-batch/4662ad27-34c6-4ee3-9b33-6ba6e984c08f", "taskDefinitionArn": "arn:aws:ecs:ap-northeast-1:000000000000:task-definition/fargate-task-Definition:1" } ], "failures": [] }
arn:aws:ecs:ap-northeast-1:000000000000:cluster/ecs-fargate-batch
arn:aws:ecs:ap-northeast-1:000000000000:task/ecs-fargate-batch/4662ad27-34c6-4ee3-9b33-6ba6e984c08f
dev-lambda-user@f220fa55d305:/work/ecs-batch$
```

s3のバケットにファイルができてればOK
![](https://storage.googleapis.com/zenn-user-upload/f4c76b6156c8-20230325.png)