Monitor and protect sensitive log data with CloudWatch and Terraform
Tags: aws, cloudwatch, dataprotection, terraform
Published: 2022-12-02 11:12:00

New feature for CloudWatch Logs has been announced on the first day of this years AWS Re:Invent - you can now set up a policy for each CloudWatch logs group, which will allow you to mask and audit sensitive log data. 

In this post I will write about the details of this functionality and walkthrough how to implement it in the AWS Console and using Terraform.


## How does it work?

This new CloudWatch feature will allow you to:
- **mask** any information that you deem sensitive
- **audit** it by setting a destination for it - this could be S3 bucket, Kinesis Data Firehose or/and CloudWatch Logs.

After Data Protection policy is attached to a log group any logs added to that log group will be masked, and only users with `logs:Unmask` permissions will be able to view them.

Making sure that any sensitive data is protected is 
![How does masked data looks like in the CloudWatch console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9uo1enq8hxc99eyeo2z6.png)

> Important to note that any sensitive information in the log groups created prior to setting up sensitive data policy will not be detected. 

## What types of data you can identify?

AWS gives yo ua choice of 100 data types to choose from, such as 

- **Credentials**, such as AWS Access Keys or Private keys
- **Personally Identifiable information**, such as national identification numbers, Social Security Number , drivers license numbers, passport number, addresses, Electoral roll number taxpayers number etc
- **Protected Health Information (PHI)** such as health insurance numbers, NHS numbers, health insurance information
- **Financial information** - bank account numbers, credit card numbers and security codes
- **Device Identifiers** such as IP adress

You can find the whole list of the types along with their ARN identifiers [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/protect-sensitive-log-data-types.html).

## How to create sensitive data policy in the AWS Console

In your AWS console, go to CloudWatch, select Log groups, choose log group you would like to create the policy for, and next select _Data protection_ section, and click on _Create policy_.

![Choose Log group and select Data protection](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xg91vysxan34zhdg4l56.png)

Now you will be able to select Data identifiers you want to mask and audit - for the purpose of this tutorial, we will choose three: Address, BankAccountNumber-GB and BankAccountNumber-US. Next we can select audit destination, here we have selected an existing _sensitive-data-audit-example-bucket_, however this is optional,
![Creating data protection policy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cdoqvlcqy6f55pr9yke7.png)

You also have ability to view how your policy looks like in JSON format. This is the example policy that we have just created:

```json
{
  "Name": "data-protection-policy",
  "Description": "",
  "Version": "2021-06-01",
  "Statement": [
    {
      "Sid": "audit-policy",
      "DataIdentifier": [
        "arn:aws:dataprotection::aws:data-identifier/Address",
        "arn:aws:dataprotection::aws:data-identifier/BankAccountNumber-GB",
        "arn:aws:dataprotection::aws:data-identifier/BankAccountNumber-US"
      ],
      "Operation": {
        "Audit": {
          "FindingsDestination": {
            "S3": {
              "Bucket": "sensitive-data-audit-example-bucket"
            }
          }
        }
      }
    },
    {
      "Sid": "redact-policy",
      "DataIdentifier": [
        "arn:aws:dataprotection::aws:data-identifier/Address",
        "arn:aws:dataprotection::aws:data-identifier/BankAccountNumber-GB",
        "arn:aws:dataprotection::aws:data-identifier/BankAccountNumber-US"
      ],
      "Operation": {
        "Deidentify": {
          "MaskConfig": {}
        }
      }
    }
  ]
}

```
Next click on _Activate data protection_, and your newly created policy will mask and audit any logs added to the log group **after** the policy was created.


## Create data protection policy for CloudWatch with Terraform

Below you can see example of how to add a Data Protection policy resource in Terraform. 

Here we will be setting up similar policy to the one created before - adding Data policy to an existing log_group_example policywith s3 bucket as audit destination, Address and BankAccount data identifiers.

```jsx

resource "aws_cloudwatch_log_group" "log_group_example" {
  name = "log_group_example"
}

resource "aws_s3_bucket" "sensitive_data_audit-example_bucket" {
  bucket = "sensitive-data-audit-example-bucket"
}

resource "aws_cloudwatch_log_data_protection_policy" "log_group_example" {
  log_group_name = aws_cloudwatch_log_group.log_group_example.name

  policy_document = jsonencode({
    Name    = "data_protection_policy"
    Version = "2021-06-01"

    Statement = [
      {
        Sid = "Audit"
        DataIdentifier = ["arn:aws:dataprotection::aws:data-identifier/Address",
          "arn:aws:dataprotection::aws:data-identifier/BankAccountNumber-US",
        "arn:aws:dataprotection::aws:data-identifier/BankAccountNumber-GB"]
        Operation = {
          Audit = {
            FindingsDestination = {
              S3 = {
                Bucket = aws_s3_bucket.sensitive_data_audit-example_bucket.bucket
              }
            }
          }
        }
      },
      {
        Sid = "Redact"
        DataIdentifier = ["arn:aws:dataprotection::aws:data-identifier/Address",
          "arn:aws:dataprotection::aws:data-identifier/BankAccountNumber-US",
        "arn:aws:dataprotection::aws:data-identifier/BankAccountNumber-GB"]
        Operation = {
          Deidentify = {
            MaskConfig = {}
          }
        }
      }
    ]
  })
}

```


You can read more about CloudWatch Data Protection policy in the [Amazon](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/mask-sensitive-log-data.html)