Send Memory Utilization Metrics to CloudWatch
Tags: aws, cloudwatch, metrics
Published: 2021-01-06 16:18:33

---
title: Send Memory Utilization Metrics to CloudWatch
published: true
description: 
tags: 
- aws
- cloudwatch
- metrics
//cover_image: https://direct_url_to_image.jpg
---

Below reviews 2 ways to collect extra metrics from an ec2 instance and send to cloudwatch. The first is a procedural, _quick n dirty_ way. The second is the aws preferred way using the cloudwatch agent & agent configuration.

This advice is not production ready but just to get your feet wet.

## Quick 'n Dirty

This is a setup for Ubuntu but pretty much everything should transfer to RHEL based. The idea is to have a cron job execute a script that checks free memory then use aws-cli to write to cloudwatch. You can extend by generating additional variables and doing more `put-metric-data` calls.

1. Setup a role with CloudWatch permissions and attach it to your instance.
2. Install AWS CLI
3. Script, i placed this at `~/mem.sh` for testing. We retrieve and inject the token because we're using [IMDSv2 to protect against SSRF](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html#configuring-instance-metadata-options).
    ```bash
    #!/usr/bin/env bash
    readonly TOKEN=$(curl -X PUT -H "X-aws-ec2-metadata-token-ttl-seconds: 30" "http://169.254.169.254/latest/api/token")
    USEDMEMORY=$(free -m | awk 'NR==2{printf "%.2f\t", ($3/$2)*100 }')
    INSTANCE_ID=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id)
    REGION=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" -s http://169.254.169.254/latest/dynamic/instance-identity/document | grep '\"region\"' | cut -d\" -f4)

    aws cloudwatch put-metric-data --metric-name memory-usage --dimensions Instance=$INSTANCE_ID --namespace "Custom" --value $USEDMEMORY --region $REGION
```
4. Create Cron job: `echo '*/5 * * * * ubuntu /home/ubuntu/mem.sh' | sudo tee /etc/cron.d/cw_mem`


## AWS Preferred Method

AWS publishes a tool, the CloudWatch Agent, which can run as a daemon and publish metrics for you. This requires a configuration file as well as systemd scaffolding. If you install via SSM the systemd files come free and only require minor tweaking.

1. IAM Instance Role:
    - Cloudwatch Permissions
    - ec2:DescribeTags 
2. [Install Cloudwatch Agent (prefer SSM)](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/download-CloudWatch-Agent-on-EC2-Instance-SSM-first.html)
2. Install collectd `sudo apt-get update && sudo apt-get install collectd`
3. Populate a configuration file for cloudwatch agent, [example](https://gist.github.com/csereno/deac72776418173c4f3169b3f34c1246). I located my file to `/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json`
4. start service: 
    - manually: `sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a start -c  /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json`
    - Alternatively you can update the unit file located at `/etc/systemd/system/amazon-cloudwatch-agent.service` 

Thanks to @danquack for helping me adjust my `curl` calls so I can enforce and comply with IMDSv2