Custom Cloudwatch Metrics using the CDK
Tags: aws, python, devops
Published: 2020-08-14 15:04:33

Over the past few months, I have been working to create an internal dashboard using AWS CloudWatch for my summer internship.

After trying to find a way to easily deploy the dashboard I came across the [AWS Cloud Development Kit.](https://docs.aws.amazon.com/cdk/latest/guide/home.html) This allowed me to create a Stack via a language I am familiar with (Python), rather than using CloudFormation. 

This is by no means a tutorial with best practices on creating a stack with the CDK, but hopefully someone can find it helpful for getting started.

Thanks to [Simon-Pierre Gingras](https://medium.com/poka-techblog/cloudwatch-dashboards-as-code-the-right-way-using-aws-cdk-1453309c5481) for his post that helped me through custom metrics and the CDK. 

***

###Installing the CDK
I will link to the AWS Docs for installing the CDK: [Installing CDK Docs]()

***
### Let's get started!
Once we have the CDK installed we need to create a directory for our project to live in, once created we need to change directory into it and create the cdk project. 
```
mkdir my-project
cd my-project
cdk init app --language python
```
This creates all the code/files necessary for the CDK. It also creates a python venv which we will need to start and install the `requirements.txt` onto.
```
source .env/bin/activate
python -m pip install -r requirements.txt
```
***
#### Creating the Dashboard
Now we have the project created we can get into creating a basic dashboard and include some AWS widgets for now. 

Firstly we will need to import a few things from `aws_cdk`
I will explain what each is used for as we go along. 
```python
from aws_cdk import(
  aws_lambda as _lambda,
  aws_cloudwatch as cw,
  core
)
from aws_cdk.aws_cloudwatch
import TextWidget, SingleValueWidget, GraphWidget
```

Now we can create the dashboard and add some widgets. To do Under the `__init__` function in the `my_project_name_stack.py` file we will add the following code:
```python
cw.Dashboard(
  self,
  "dashboard_name",
  widgets = [
    TextWidget(
      markdown = 'Title widget'
      width = 24,
      height = 2
    )
  ]
)
```
This code will create a dashboard with the name of `dashboard_name` and will display a single text widget with the markdown specified. We have also added width and height attributes which will tell the dashboard how to display the widget.

We will add a more useful AWS default metric under the previous widget using the `SingleValueWidget` function:
```python
SingleValueWidget(
  metrics = [cw.Metric(
    metric_name = 'ResourceCount',
    namespace = 'AWS/Usage',
    dimensions = dict(
      Type = 'Resource',
      Resource = 'vCPU',
      Service = 'EC2',
      Class = 'Standard/OnDemand'
    )
  )],
  width = 6,
  height = 3,
)
```
Add the code above to `metrics=[]`. Following up each separate widget with a `,` as the `metrics=[]` is expecting an list of widgets and uses the list to format the dashboard which we will look at shortly. 
***
#### Deploy the stack
Now we have some basic widgets in the stack, we can try to deploy the dashboard to AWS using the CLI. 
- Firstly make sure you have valid AWS credentials in the `.aws/credentials` file within the project directory.
- To test we don't have any errors we can use `cdk synth` to convert our CDK code into a cloudformation change set (This is what AWS cloudformation will receive when building the stack) 
- Finally we can deploy the stack to AWS using the `cdk deploy` command this will create the change set and send it to AWS CloudFormation which will create the dashboard with our widgets for us. (This can take a few minutes)
- Once deployed, have a look for our new dashboard within the CloudWatch page on the AWS Console. If everything is working we can take down the stack again using `cdk destroy` this will delete all the resources from the stack so we wont get charged for them. 

***

#### Custom Metrics time!
Now we have a basic dashboard working from the CDK, we can start to add some custom metrics. To do this we will need to create a lambda function using boto3 to access cloudwatch. We will also need to add the lamdba into our `XXXX_stack.py` file as it will need to be deployed at the same time as our dashboard. 

Firstly we need to add a lambda folder into the project, and create a file within called `custom_metric.py` within the new file add the following code:


```python
import boto3

metric_data = <The value you want the metric to display: int>
metric = {
    'MetricName': 'Instance Metrics',
    'Dimensions': [
        {
            'Name': 'MetricName',
            'Value': 'CustomNumberMetric',
        },
    ],
    'Unit': 'None',
    'Value': metric_data
}
```
- `MetricName` the name of the metric within the namespace.
- `Name` The name of the metric column.
- `Value` the value of the metric itself.

Now we need to add the code to submit the metric to cloudwatch.

```python
cloudwatch = boto3.client('cloudwatch', <RegionName>)
cloudwatch.put_metric_data(
    Namespace='CustomMetrics',
    MetricData=metric
)
```
***
####Let's add the lambda function to the CDK!

Now we have our basic custom metric created we can add it into our stack. To do this we need to add a few other items. The CDK will try its best to create IAM roles for the lambda functions to give the access they need to function. In this tutorial we will add our own roles so our lambda can access CloudWatch. 

#####Creating the IAM role:
- Firstly we need to add another import statement `aws_iam as iam` inside the `from aws_cdk import()` import statement
- Next we create the role with the following code 
```python
custom_metric_role = iam.Role(
            self, 'custom_metrics_role',
            assumed_by=iam.ServicePrincipal("lambda.amazonaws.com"),
         managed_policies=[
         iam.ManagedPolicy.from_managed_policy_arn(
             scope=self,
             id='ExeRole',                 
         managed_policy_arn='arn:aws:iam::aws:policy/service-role/AWSLambdaKinesisExecutionRole'
        )
    ]
)

custom_metric_role.add_to_policy(
    statement=iam.PolicyStatement(
        actions=[
            "cloudwatch:PutMetricData"
        ],
        resources=[
            "*"
        ]
     )
)
```
Note: This probably isnt the best way to create a role (I am still learning ahah!)

#####Adding the Lambda Function to the stack
Now we have our role we can add the lambda function into the stack.
- Once again we need to add another import statement. Add `aws_lambda as _lambda` to the `from aws_cdk import()`
- Now we add the code below for the lambda function
```python
_lambda.Function(
  self, "custom_metric_lambda",
  runtime = _lambda.Runtime.PYTHON_3_7,
  code = _lambda.Code.from_asset('lambda'),
  handler = 'custom_metric.lambda_handler',
  role = custom_metric_role,
  timeout = core.Duration.seconds(10)
)
``` 

####Add the metric to the dashboard
- Add the following code inside the `metric=[]` argument of the cloudwatch dashboard function. 
```python
SingleValueWidget(
  metrics = [cw.Metric(
    metric_name = 'Instance Metrics',
    namespace = 'CustomMetrics',
    dimensions = dict(
      MetricName = 'CustomMetric',
    )
  )],
  width = 6,
  height = 3
)
```
That's all the coding finished! 
***
####Final Deployment! 
Now all we have to do is deploy the stack.
- In the terminal use `cdk diff` this command will run our code and will show any errors. It will also show you the differences in the code base compared to the last deployment.
- If that all passes we can deploy using `cdk deploy`
***

#### Thank You! 
I am by no means an expert in the topic but found the documentation really awkward to get your head around so hopefully this 'tutorial' helps. 

***

##### Notes
Any comments would be greatly appreciated, I have a lot more to learn about the CDK and would love to create a sever-less app using this stack and the AWS SAM model.

This is also my very first Dev.to post, would love suggestions of good topics to look at as a student dev.