Debugging Node.js applications running on ECS Fargate
Tags: aws, node, firstpost, serverless
Published: 2021-08-14 09:03:38

AWS Fargate is a serverless compute engine for containers. It removes the need to provision and manage servers. To know more on Fargate, you can visit [this page](https://aws.amazon.com/fargate/?nc=sn&loc=1).
In this blog post, we will see how to debug a Node.js application running on ECS Fargate using the new ECS Exec feature. Once this is done, you'll be able to debug your application using Chrome DevTools. I used this to debug memory leaks in a Node.js application.

## Prerequisites
* SSH and Google Chrome installed (to create SSH tunnel).
* A SSH key pair (public key and private key). I used PuTTYgen to generate and save these keys.
* AWS CLI configured.
* AWS Systems Manager Session Manager plugin installed.
* Node.js application running in ECS Fargate.
* ECS service's security group has ports 22 and 9229 open in inbound rules.

We'll use SSH tunnel to establish connection between the Fargate server and the local system.

## AWS Configurations
Considering that you have a task already running in ECS Fargate, the next step is to enable debugging for the application.

### Enable Debugging Mode
When enabled, the Node.js process listens for a debugging client on port 9229. To enable this, you have to add `--inspect` when running the application. Example: ```bash
node server.js --inspect
```

### Enable ECS Exec Feature
Open the ECS Task role in IAM and add below policy to the role:
```json
{
   "Version": "2012-10-17",
   "Statement": [
       {
       "Effect": "Allow",
       "Action": [
            "ssmmessages:CreateControlChannel",
            "ssmmessages:CreateDataChannel",
            "ssmmessages:OpenControlChannel",
            "ssmmessages:OpenDataChannel"
       ],
      "Resource": "*"
      }
   ]
}
```
Next, we run the AWS CLI command:
```bash
aws ecs update-service --service <service-name> --cluster <cluster-name> \
    --region <region> \
    --enable-execute-command --force-new-deployment
```
Replace the content inside angular brackets with appropriate values.

Once the above command is executed, a new task will start with ECS Exec enabled. If you receive any error when running the above command, please check the error response you received and try to resolve it. You can find troubleshooting tips from AWS docs.

### Install requirements in the ECS task
Now we can ECS Exec into the running task and configure SSH. Use the below command to exec into the ECS task:
```bash
aws ecs execute-command --cluster <cluster-name> \
    --task <task-id> \
    --container <container-name-to-execute-the-command-on> \
    --interactive \
    --command "/bin/sh"
```
Now you should be getting an interactive session in the Fargate server. Next, run:
```bash
apt-get update
apt-get install openssh-server -y
service ssh start
```
Now you'll have SSH server running in the ECS task.
We'll have to do a few more steps to get this working right. Create a file called `docker-entrypoint.sh` and paste below code to it:
```bash
#!/bin/sh
SSH_PUBLIC_KEY="<ssh-public-key>"
if [ -z "$SSH_PUBLIC_KEY" ]; then
  echo "Need your SSH public key"
  exit 1
fi
# Create a folder to store user's SSH keys if it does not exist.
USER_SSH_KEYS_FOLDER=~/.ssh
[ ! -d "$USER_SSH_KEYS_FOLDER" ] && mkdir -p $USER_SSH_KEYS_FOLDER

# Copy contents from the `SSH_PUBLIC_KEY` variable
# to the `${USER_SSH_KEYS_FOLDER}/authorized_keys` file.
echo $SSH_PUBLIC_KEY > ${USER_SSH_KEYS_FOLDER}/authorized_keys

# Clear the `SSH_PUBLIC_KEY` variable.
unset SSH_PUBLIC_KEY

# Start the SSH daemon.
/usr/sbin/sshd -D
echo "Success"
```
Replace `<ssh-public-key>` with the SSH public key you generated earlier. Now save the file.
So far so good. Next, run the script:
```bash
chmod +x docker-entrypoint.sh 
./docker-entrypoint.sh 
```
Now we come back to our local system.

## Local System Configurations
Go to the directory where the SSH private key is stored and run:```bash
chmod 700 ssh
```
To test the SSH connection, you can run:
```bash
ssh root@<ecs-task-ip> -i ssh
```
Finally run the command to create the SSH tunnel:
```bash
ssh -nNT -L 9229:127.0.0.1:9229 root@<ecs-task-ip> -i ssh
```
That should do it!
Now open Chrome and enter `chrome://inspect` in the URL. DevTools page will appear. Notice the "Remote Target" section. Below it, you should be able to find the Fargate application for debugging.

Thanks for reading!

## References
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-exec.html

If you like this post, please consider [buying me a coffee](https://www.buymeacoffee.com/amalj).