Using Terraform and AWS CloudFormation to Enforce Your AWS Tags
Tags: aws
Published: 2020-08-27 18:06:57

---
title: Using Terraform and AWS CloudFormation to Enforce Your AWS Tags
published: True
description: You'll learn how to use Terraform and AWS CloudFormation to better enforce your AWS tagging strategy, deploy AWS tags and ensure your tags are being used across all your AWS resources. 
tags: #aws
cover_image: https://www.cloudforecast.io/blog/assets/media/tagspart2.jpg
---
Once <a href="https://dev.to/cloudforecast/aws-tagging-best-practices-guide-part-1-of-3-3f85" target="_blank">you have adopted an AWS tagging strategy</a>, you’ll need to make sure that all your existing AWS resources and any new ones you create abide by it. Consistency is the key - if you don’t proactively enforce your AWS tagging strategy, you’ll always be playing catch up and chasing down team members to make sure they add the right tags to their resources.

While you can apply AWS tags to your resources manually using the <a href="https://docs.aws.amazon.com/cli/latest/reference/resourcegroupstaggingapi/tag-resources.html" target="_blank">AWS CLI</a> or <a href="https://docs.aws.amazon.com/ARG/latest/userguide/tag-editor.html" target="_blank">AWS Tag Editor</a>, you’ll probably find this cumbersome and error-prone at scale. A better approach is to automatically apply AWS tags to your resources and use rules to enforce their consistent usage.

Depending on the tool you use to maintain your infrastructure on AWS, your method of proactively enforcing AWS tags on new resources may vary. In this guide, I’ll highlight two tools: Terraform and AWS CloudFormation. You’ll see how to use each to create and update AWS cost allocation tags on your resources and then enforce the proper use of specific tags for new resources. By proactively enforcing your AWS tagging strategy, you’ll minimize your time spent auditing and correcting improper AWS tags and force developers to learn best AWS tagging best practices for your environment.

## Using Terraform for AWS Tags

The first infrastructure management tool I’ll cover is [Terraform](https://www.terraform.io/). Terraform works across a variety of cloud hosting providers to help you provision and maintain your AWS resources. With Terraform, you can define your servers, databases, and networks in code and apply your changes programmatically to your AWS account.

If you’re new to Terraform, they have a well-documented [Getting Started guide](https://learn.hashicorp.com/terraform/getting-started/intro) and several [AWS template examples on GitHub](https://github.com/terraform-providers/terraform-provider-aws/tree/master/examples). In this section, I’ll show you some snippets from a demo Terraform project and module that [is available on GitHub](https://github.com/CloudForecast/cf-terraform-demo). You'll learn the following in this Terraform AWS tags:

1. Tag a New AWS EC2 Instance with Terraform
2. Using Terraform to Update Existing AWS Tags
3. Enforce AWS Tags with Terraform

### Tag a New AWS EC2 Instance with Terraform

If you want to deploy an EC2 instance with AWS Tags using Terraform, your configuration might include something like this:

    resource "aws_instance" "cart" {
      connection {
        type = "ssh"
        user = "ubuntu"
        host = self.public_ip
        private_key = file(var.private_key_path)
      }
    
      instance_type = "t2.micro"
    
      ami = var.aws_amis[var.aws_region]
    
      key_name = aws_key_pair.auth.key_name
    
      vpc_security_group_ids = [aws_security_group.default.id]
    
      subnet_id = aws_subnet.default.id
    
      provisioner "remote-exec" {
        inline = [
          "sudo apt-get -y update",
          "sudo apt-get -y install nginx",
          "sudo service nginx start",
        ]
      }
      tags = {
        contact = "j-mark"
        env = "dev"
        service = "cart"
      }
    }

The above example includes three AWS cost allocation tags: `contact`, `env`, and `service` with values described as strings. When you [apply this configuration](https://www.terraform.io/docs/commands/apply.html). Terraform will connect to AWS and deploy an EC2 instance having the AWS tags you specified.

![](https://paper-attachments.dropbox.com/s_11675783F6270CC3362B9903B770D88B278C2B2D779D11551760688B8EA1DFC4_1596383259429_cf-2020-08-02-a.png)

### Using Terraform to Update Existing AWS Tags

Terraform makes it easy to update already existing resources with AWS tags in reversible and consistent ways. If you’re using AWS tags to keep track of a resource’s contact (e.g.: `j-mark` in the above example), you’re likely to need to update the AWS tag when the team member leaves or changes roles.

To update the AWS tags on your resource, simply update the corresponding tags in your Terraform configuration. The new tags will overwrite any previous tags assigned to the resource, including tags added outside of Terraform.

For example, to change the `contact` cost allocation tag on the EC2 instance above, you might update the `tags` block above with the following:

    tags = {
      contact = "l-duke"
      env = "dev"
      service = "cart"
    }

When you apply this configuration, the AWS tags will be automatically updated in the AWS console:

![](https://paper-attachments.dropbox.com/s_11675783F6270CC3362B9903B770D88B278C2B2D779D11551760688B8EA1DFC4_1596383277121_cf-2020-08-02-b.png)

If you keep your Terraform configuration files in version control - which is probably a good idea - you will be able to see how tags have changed over time. You can also review changes using the same code review process that your application code goes through to help you catch mistakes in the execution of your tagging strategy.

### Enforce AWS Tags with Terraform

As your infrastructure grows, a code review process likely won’t be enough to prevent improper AWS tagging. Fortunately, you can enforce AWS tag names and values using [variables](https://www.terraform.io/docs/configuration/variables.html) and custom validation rules in Terraform.

In the examples above, the `tags` list was hard-coded into the EC2 instance definition. A more scalable pattern would be to break your EC2 instance template into its own [module](https://www.terraform.io/docs/configuration/modules.html) and use a `tags` variable. You can then write a [custom validation rule](https://www.terraform.io/docs/configuration/variables.html#custom-validation-rules) to check that the tags comply with your strategy.

For example, if you want to check that:

1. The user specifies at least one tag
2. The `contact` tag is either `j-mark` or `l-duke`
3. The `env` tag is set
4. The `service` tag is either `cart` or `search`

You might create a module with a variable specified like this:

    variable "tags" {
      description = "The tags for this resource."
      validation {
        condition = length(var.tags) > 0 && contains(["j-mark", "l-duke"], var.tags.contact) && var.tags.env != null && contains(["cart", "search", "cart:search"], var.tags.service)
        error_message = "Invalid resource tags applied."
    
      }
    }

Now when you run `terraform plan` with a missing or invalid tag, you’ll get an error:

    Error: Invalid value for variable
    ...
    Invalid resource tags applied.

Your rules can be as complex as [Terraform’s Configuration Language](https://www.terraform.io/docs/configuration/index.html) allows, so functions like `regex()`, `substr()`, and `distinct()` are all available. That said, there are some caveats to this approach.

First, custom variable validation is an experimental feature in Terraform. [Experimental features](https://www.terraform.io/docs/configuration/terraform.html#experimental-language-features) are subject to change, meaning that you might need to pay attention to Terraform update mores closely. To enable `variable_validation`, add the following to your `terraform` block:

    terraform {
      experiments = [variable_validation]
    }

Second, Terraform’s variable validation only happens during the `terraform plan` phase of your infrastructure’s lifecycle. It can’t prevent users from accidentally changing your tags directly in the AWS console, and it’s only as good as the validation rules you write. If you start using a new resource but forget to add validation rules, you might end up with lots of resources that don’t adhere to your tagging strategy.

Another option for paid Terraform Cloud customers is [Sentinel](https://www.terraform.io/docs/cloud/sentinel/index.html) , which allows you to create custom policies for your resources. I won’t cover this method here, but Terraform has created an [example policy](https://www.terraform.io/docs/cloud/sentinel/examples.html) to show you how to enforce mandatory AWS tags.

## Using AWS CloudFormation for Tags

Similar to Terraform, AWS [CloudFormation](https://aws.amazon.com/cloudformation/) lets you provision AWS resources based on configuration files. Unlike Terraform, CloudFormation is part of Amazon’s offerings, so it won’t necessarily help you if you want to use another infrastructure provider. The approach to tagging your resources in CloudFormation is similar to that used by Terraform, but as you’ll see, the configuration format is different.

If you’re new to AWS CloudFormation, [Amazon’s official walkthrough](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/GettingStarted.Walkthrough.html) will help you get started deploying some basic templates. In this section, I’ll show you some snippets from a demo AWS CloudFormation template which is [also available on GitHub](https://github.com/CloudForecast/cf-cloudformation-demo). You'll learn the following in this Terraform AWS tags section:

1. AWS CloudFormation Template to Deploy Tags
2. Using CloudFormation to Update AWS Tags
3. CloudFormation Template to Enforce AWS Tags

### AWS CloudFormation Template to Deploy Tags

AWS CloudFormation is designed to make it easy to create AWS resources with a single template file. Using a CloudFormation template, every resource that can be deployed with an AWS tag.

For example, to create a new EC2 instance with the same three AWS tags used in the Terraform example above, add an array of `Tags` to the resource’s `Properties` block:

    "Resources" : {
      "WebServerInstance": {  
        "Type": "AWS::EC2::Instance",
        "Metadata" : {...},
        "Properties": {
          "Tags" : [
           {
              "Key" : "contact",
              "Value" : "j-mark"
           },
           {
              "Key" : "env",
              "Value" : "dev"
           },
           {
              "Key" : "service",
              "Value" : "cart"
           }
          ],
          ...       
        }
      },
      ...         
    },

Using [AWS CLI](https://aws.amazon.com/cli/), you can [deploy this CloudFormation template as a new stack](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudformation/create-stack.html). This will ensure your template is valid and create the specified resources with their tags on AWS:

    aws cloudformation create-stack --template-body file://path/to/your/template.json --stack-name=<YOUR_STACK_NAME>

If you have lots of similar resources in your template, you can deploy AWS tags to all the resources in the stack at once using the `--tags`  flag with the `create-stack` or `update-stack` commands:

    # Creating a stack with tags
    aws cloudformation create-stack --template-body file://path/to/your/template.json --stack-name=<YOUR_STACK_NAME> --tags="Key=env,Value=dev"
    
    # Updating a stack with tags
    aws cloudformation update-stack --template-body file://path/to/your/template.json --stack-name=<YOUR_STACK_NAME> --tags="Key=env,Value=dev"

### Using CloudFormation to Update AWS Tags

If you want to change the contact on your EC2 instance created above, simply change the `Tags` section of your template file and use the `[update-stack](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudformation/update-stack.html)` [command](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudformation/update-stack.html) to deploy your changes.

    "Tags" : [
     {
        "Key" : "contact",
        "Value" : "l-duke"
     },
      ...
    ],

AWS CloudFormation behaves the same way that Terraform does when you update tags outside your template file. Any tags set manually will be overridden by the `update-stack` command, so be sure that everyone on your team deploy's tags through CloudFormation.

### CloudFormation Template to Enforce AWS Tags

AWS provides [Organization Tag Policies](https://aws.amazon.com/about-aws/whats-new/2019/11/aws-launches-tag-policies/) and [Config Managed Rules](https://docs.aws.amazon.com/config/latest/developerguide/evaluate-config_use-managed-rules.html) to help you _find_ improperly tagged resources, but neither of these tools prevents you from _creating_ resources with missing or invalid tags. One way to proactively enforce your tagging strategy is by using the [CloudFormation linter](https://github.com/aws-cloudformation/cfn-python-lint).

`cfn-lint` is a command-line tool that will make sure your AWS CloudFormation template is correctly formatted. It checks the formatting of your JSON or YAML file, proper typing of your inputs, and [a few hundred other best practices](https://github.com/aws-cloudformation/cfn-python-lint/blob/master/docs/rules.md). While the presence of specific tags isn’t checked by default, you can write a custom rule to do so in Python.

For example, if you want to ensure that your CloudFormation web servers follow the same rules as the Terraform example above and have:

1. At least one AWS tag
2. The `contact` tag set to either `j-mark` or `l-duke`
3. The `env` tag set
4. The `service` tag set to `cart` or `search`

You can create a new rule called `TagsRequired.py`:

    from cfnlint.rules import CloudFormationLintRule
    from cfnlint.rules import RuleMatch
    
    
    class TagsRequired(CloudFormationLintRule):
        id = 'E9000'
        shortdesc = 'Tags are properly set'
        description = 'Check all Tag rules for WebServerInstaces'
    
        def match(self, cfn):
            matches = []
            approved_contacts = ['j-mark', 'l-duke']
            valid_services = ['cart', 'search']
            web_servers = [x for x in cfn.search_deep_keys('WebServerInstance') if x[0] == 'Resources']
            
            for web_server in web_servers:
                tags = web_server[-1]['Properties']['Tags']
    
                if not tags:
                    message = "All resources must have at least one tag"
                    matches.append(RuleMatch(web_server, message.format()))
    
                if not next((x for x in tags if x.get('Key') == 'env'), None):
                    message = "All resources must have an 'env' tag"
                    matches.append(RuleMatch(web_server, message.format()))
    
                for tag in tags:
                    if tag.get('Key') == 'contact' and tag.get('Value') not in approved_contacts:
                        message = "The contact must be an approved contact"
                        matches.append(RuleMatch(web_server, message.format()))
    
                    if tag.get('Key') == 'service' and tag.get('Value') not in valid_services:
                        message = "The service must be a valid service"
                        matches.append(RuleMatch(web_server, message.format()))
    
            return matches

When you run `cfn-lint`, include your custom rule:

    cfn-lint template.json -a ./path/to/custom/rules

If your CloudFormation template is missing any tags, you’ll see an error:

    E9000 Missing Tag contact at Resources/WebServerInstance/Properties/Tags
    template.json:169:9

Using linting to validate your AWS CloudFormation rules is a great way to enforce your AWS tags proactively. If you’re storing your CloudFormation templates in version control, you can run `cfn-lint` using [pre-commit hooks](https://github.com/aws-cloudformation/cfn-python-lint#pre-commit) or by making it part of your continuous integration workflow.

Because these rules are written in Python, they can be as complex as you need them to be, but they have drawbacks as well. Like Terraform’s custom variable validation, linting rules won’t tell you about existing problems in resources that aren’t managed by CloudFormation, so they work best when combined with a reactive tag audit and adjustment strategy.

## Conclusion

Properly tagged resources will help you [predict and control your costs](https://www.cloudforecast.io/blog/how-tagging-resources-can-reduce-your-aws-bill/), but your tagging strategy can’t just be reactive. Having proactively enforced patterns will require an up-front investment, but will save you time and money in the long-run.

Once you’ve adopted a tagging strategy and proactive enforcement method, the last piece of the puzzle is catching up when you fall behind. In the final part of this guide, you’ll see how to audit and find mistagged resources to ensure your tagging strategy continues to succeed in the future.

If you’re interested in getting help with your tagging, reach out to our CTO at [francois@cloudforecast.io](mailto:francois@cloudforecast.io) to receive a free tagging compliance report.

_This was originally posted on our blog on 8/27/2020: <https://www.cloudforecast.io/blog/aws-tagging-best-practices-guide-part-2/>_