Monitor Linux Memory Metrics in AWS CloudWatch
Tags: aws, devops, cloud, monitoring
Published: 2022-09-14 12:50:13


### How to install the CloudWatch agent in Ubuntu 20.04 and how to get memory metrics in the CloudWatch console


In AWS, for the EC2 instance, we won't get memory metrics by default in CloudWatch. So one way to get this done is with the CloudWatch agent. By getting the memory metrics in AWS CloudWatch we can set up an Alarm to trigger notifications or any action. In this post, we see how the CloudWatch agent is installed in Ubuntu 20.04 and how to get memory metrics in the CloudWatch console.

### Step 1: Create an IAM role

As we use CloudWatch we need to authenticate to push metrics. If you are planning to implement this in an on-premise Ubuntu server, we can do this with IAM users, with programmatic access. As our instance is in EC2 we create an IAM role with the following steps.
>  Note : if you already have an IAM role attached to instance then just attach **CloudWatchAgentServerPolicy** policy to that role

**Step 1.1**: Sign in to the AWS Management Console and open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/).

**Step 1.2**: In the navigation pane of the IAM console, choose **Roles**, and then choose to **Create role**.

**Step 1.3**: **For Select trusted entity**, choose AWS service.

![Selecting trusted entity and use case](https://cdn-images-1.medium.com/max/2222/1*IsBryefcekQqpSJHnh9pTA.png)

**Step 1.4**: Choose the use case as **EC2**. Then, choose **Next**.

**Step 1.5**: In **Permission policies** search for **CloudWatchAgentServerPolicy** and select that, then click **Next**

![Attaching permission policies](https://cdn-images-1.medium.com/max/2510/1*aJVqADnZlB3io19mf1RGwA.png)

**Step 1.6**: Give a name for the role created here we provide the name as EC2CloudWatchAgentRole. Below that we can review the things we created and, then click **Create role**

![Providing a name for the role](https://cdn-images-1.medium.com/max/2040/1*QENg00hjL01mCwF0hI_R-A.png)

Now we have created our IAM role.

### Step 2: Launch an EC2 with Ubuntu 20.04 ami

If you already have an EC2 launched you can skip this step. If you are doing this on an on-premise instance you can skip this step(See [this](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-cloudwatch-agent-configuration-file-wizard.html)).

**Step 2.1:** Provide any name of the instance.

**Step 2.2:** Select Ubuntu Server 20.04 LTS AMI

![Selecting Ubuntu 20.04 LTS AMI while creating an instance](https://cdn-images-1.medium.com/max/2000/1*CdpIJXcE_p71hp1P4SUwuA.png)

**Step 2.3:** Choose t3a.micro or any instance type.

**Step 2.4:** Choose a key pair or create a new one if you don’t have access to existing key pairs.

**Step 2.5:** Keep Network settings default if you are new to VPC or customise as required

**Step 2.6:** Allocate more space if required. Suppose it's for testing purposes 8GB is enough.

**Step 2.7:** Once reviewed you can click on **launch instance.**

Now we have created the instance.

### Step 3: Attach the IAM role to the instance

Now we attach the IAM role created in Step 1

**Step 3.1:** Select instance click **Actions** then from **Security** select **Modify IAM role**

![Attaching role to instance](https://cdn-images-1.medium.com/max/2228/1*8SxSJWaN_T5AD1CUv-jqag.png)

**Step 3.2:** Now from the drop-down you can select the role we created earlier

![Selecting role](https://cdn-images-1.medium.com/max/2000/1*qDuFUeYZ_BFMajZjIsi-cw.png)

Select the role and click **Save**

**Step 4: Let's install and configure CloudWatch Agent**

Let's run this as root so we could avoid sudo in every command.

    sudo su -

![Becoming root ](https://cdn-images-1.medium.com/max/2000/1*Lo7E2KNOA47Ey5p1ciqTcg.png)

**Step 4.1:** Download the CloudWatch agent:

    wget [https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb](https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb)

![Downloading clouwatch agent](https://cdn-images-1.medium.com/max/2728/1*eUcjI35fY2tzuRfKjvM7kw.png)

**Step 4.2:** Install the package:

    dpkg -i -E ./amazon-cloudwatch-agent.deb

![Installing cloudwatch agent](https://cdn-images-1.medium.com/max/2728/1*Q19hKkw1IDfKWasAs6yQQA.png)

This will create a user **cwagent**, group with relevant permissions and installs the CloudWatch agent

Now lets packages

     apt-get update
    

**Step 4.3:** Create the CloudWatch Agent Configuration File 
We could do this in two ways:
1) Create this **config.json** file directly
2) Create the CloudWatch agent configuration file with the wizard

For automating purposes I would suggest the First option. If you choose the second option, the wizard would create the **config.json** for you, which also can be modified.

 1. **Creating config.json file directly**

Create a file named **config.json** in this path **/opt/aws/amazon-cloudwatch-agent/bin/config.json** and paste this JSON there

    {
     "agent": {
      "metrics_collection_interval": 60,
      "run_as_user": "cwagent"
     },
     "metrics": {
      "aggregation_dimensions": [
       [
        "InstanceId"
       ]
      ],
      "metrics_collected": {
       "mem": {
        "measurement": [
         "mem_used_percent"
        ],
        "metrics_collection_interval": 60
       }
      }
     }
    }
>  NOTE : This policy is sending only memory metrics in every 60s. we have other intervals as 1s, 10s, 30s, and 60s. This metrics is fetched as cwagent

2. **Create the CloudWatch agent configuration file with the wizard**

    /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

The questions and options are as follows

* On which OS are you planning to use the agent?

1. Linux
2. Windows
3. Darwin
default choice: [1]:

* Trying to fetch the default region based on ec2 metadata…
Are you using EC2 or On-Premises hosts?

1. EC2
2. On-Premises
default choice: [1]:

* Which user are you planning to run the agent?

1. root
2. cwagent
3. others
default choice: [1]:

* Do you want to turn on StatsD daemon?

1. yes
2. no
default choice: [1]:

* Do you want to monitor metrics from CollectD? WARNING: CollectD must be installed or the Agent will fail to start

1. yes
2. no
default choice: [1]:

* Do you want to monitor any host metrics? e.g. CPU, memory, etc.

1. yes
2. no
default choice: [1]:

* Do you want to monitor CPU metrics per core?

1. yes
2. no
default choice: [1]:

* Do you want to add ec2 dimensions (ImageId, InstanceId, InstanceType, AutoScalingGroupName) into all of your metrics if the info is available?

1. yes
2. no
default choice: [1]:

* Do you want to aggregate ec2 dimensions (InstanceId)?

1. yes
2. no
default choice: [1]:

* Would you like to collect your metrics at high resolution (sub-minute resolution)? This enables sub-minute resolution for all metrics, but you can customize for specific metrics in the output JSON file.

1. 1s
2. 10s
3. 30s
4. 60s
default choice: [4]:

* Which default metrics config do you want?

1. Basic
2. Standard
3. Advanced
4. None
default choice: [1]:

After answering this series of questions it will create a **config.json** at the same path as above.

**Step 4.4:** Check the status of the agent

To check the status of the CloudWatch agent.

    /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status

![Status before starting cwagent](https://cdn-images-1.medium.com/max/2696/1*sN7I9cbPdlposEviRfs-Tg.png)

**Step 4.5:** To start the CloudWatch agent

    /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json

![Response of starting cwagent](https://cdn-images-1.medium.com/max/2716/1*iy1S3uE5PiFph390wF17iA.png)

You can check the status again to verify its running

![Response of status check after starting cwagent](https://cdn-images-1.medium.com/max/2722/1*OwteAWF--l7MDaPoYIGl3g.png)

### **Step 5: Let's verify memory metrics are arriving in the CloudWatch console**

Now to check in the AWS console, you can go to CloudWatch console, then metrics -> custom metrics -> host

![CloudWatch console](https://cdn-images-1.medium.com/max/2720/1*G9dZkNmC2sy2CULflse8NA.png)

That’s how we monitor Ubuntu memory utilisation with CloudWatch. Even though we used Ubuntu 20.04 this should also work in other Debian-based OS. In case you don’t find metrics in the CloudWatch console double-check the role and its access.

### References

* [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/download-cloudwatch-agent-commandline.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/download-cloudwatch-agent-commandline.html)

* [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-iam-roles-for-cloudwatch-agent-commandline.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-iam-roles-for-cloudwatch-agent-commandline.html)

* [https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-cloudwatch-agent-configuration-file-wizard.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-cloudwatch-agent-configuration-file-wizard.html)