[AWS Experiment] 8 - AWS ECS Deep Dive (Part.2)
Tags: aws, webdev
Published: 2022-07-26 02:04:00

# 1. Useful Resources & Links

- [Docker Image](https://hub.docker.com/repository/docker/exit21sb/ecs-demo)
- [GitHub Repository for Scripts](https://github.com/ben9543/ecs-demo)

# 2. Experiment Implementation

#### Docker Image Explanation

I created a custom Docker image for this tutorial. I briefly mentioned its specs below.

- Base Image: `python-alpine`

- Image Name: `exit21sb/ecs-demo:taskrole-test`

- Python Scripts

  - `script_session.py`: Start the job. Download all necessary images.
  - `script_session_process_images.py`: Process the images.
  - `script_session_move_images.py`: Move the images to `/old` directory.

- Libraries: 

  - boto3 (AWS SDK)

#### Uploading sample images to the S3 Bucket

- Create a bucket and create two folders which are `/old` and `/new`

- The application will download all files in the `/new`, then process the images, and will move the images to the `/old` folder.

- Upload some images to the `/new` folder.


<figure>

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v15tvqipuvt4v9q6827b.png)
<figcaption>Uploading images - 1</figcaption>
</figure>


<figure>

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5ds9rekjjxti2ehr0nho.png)
<figcaption>Uploading images - 2</figcaption>
</figure>


- Make sure the bucket is not public. It won't affect following the tutorial, but just for your sake.

- We don't need to worry about the Bucket Policy since we are going to use `S3FullAccess` permission.

#### Task Role

- Go to IAM console and create `DemoRoleForECSandS3` 

- ***Add Elastic Container Service (ECS) as Trusted Entity of the role.***

<figure>
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rzy9bm6i4pywndnlcvt7.png)
<figcaption>Task Role</figcaption>
</figure>

- Add `AmazonS3FullAccess` permission (AWS managed policy).

Perfect! We are going to attach the role to the ECS Task later. Attaching the role to tasks is very convenient since we now don't need to worry about how the application inside of ECS container authenticates to AWS services.

Not we can move on to the Task Definition part.

#### Task Definition

<figure>
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4t8l6yp5403pgymb2g55.png)
<figcaption>Task Definition - 1</figcaption>
</figure>
<figure>
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l5e40c6rs5h2810uqxi3.png)
<figcaption>Task Definition - 2</figcaption>
</figure>

- Name: Any
- Image URI: `exit21sb/ecs-demo:taskrole-test`
- Port Mappings: None
- Environment Variables (required):
  - `BUCKET_NAME`
  - `REGION`
- ***App Environment: Fargate***
- ***Operating system/Architecture: Linux/ARM64 (important)***
- CPU: 1 vCPU (does not matter)
- Memory: 3GB (does not matter)
- Task Role: `DemoRoleForECSandS3` (the one we created beforehand)
- ... and leave all optional settings as default

Great! Now we are ready to run the ECS container. 

#### Run the Task

- Go to ECS console and create a cluster.
- Go to `Tasks` tab and click the "Run new task" button. Service and Tasks are a little bit different. For this tutorial, our ECS container will not last for too long time, we are sticking to Task option.
- Specify the Launch type. Choose `FARGATE` option.
- Application type is `Task`.
- Leave desired tasks option as 1.
- Choose the Family and Revision for the Task Definition we created beforehand.
- Launch the Task

#### Task Running Process

The application will sleep for 5 mins because of the script `script_session_process_images.py`. I put sleep function just to demo the behavior. Feel free to update the script and deploy new Docker image if you want to use the application for the real-world project.

After 5 minutes, all the images in `/new` folder will be moved to `/old` folder. This can take a little bit of time.

#### Launched Screen

<figure>
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8bxks1jg39hznrp6ejyr.png)
<figcaption>Task Launched - 1</figcaption>
</figure>

<figure>
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8b7flx7cnbdcuor0nvmv.png)
<figcaption>Task Launched - 2</figcaption>
</figure>

<figure>
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7i5xkr4yhmnb1pei04vr.png)
<figcaption>Task Launched - 3</figcaption>
</figure>

#### S3 Screen

<figure>
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c6n7itz4ysp49hu1ur6o.png)
<figcaption>No images in /new folder anymore</figcaption>
</figure>

<figure>
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1lczwujenhvxupfzpv5j.png)
<figcaption>/old folder</figcaption>
</figure>

#### Cloud Watch Logs

<figure>

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vzyn8h1cdw5yjuyng1ec.png)
<figcaption></figcaption>
</figure>

Notice: The "Image processing is done!" log printed out earlier than the time when actual job gets done. I think this is because of the way how Fargate works. It seems like Fargate runs another threads to process the rest of the jobs while the current thread is sleeping (not really sure if my explanation is right).


# 3. Conclusion

Using ECS can be very easy with Fargate and Task role. Try use ECS for you batch jobs that take lots of time to process!