Terraform module for custom Azure policies
Tags: terraform, azure, test, cloud
Published: 2020-12-02 15:46:14

Terraform modules are used to create reusable components that include groups of resources meant to be deployed together. This is a natural fit for custom Azure policies and initiatives since it allows organizations to implement all those definitions in a centralized component.

This post goes through an example that shows how to implement and test a Terraform module that defines custom Azure policies and initiatives. The full version of the code can be found in the following repo: {% github beandrad/terraform-azurerm-policy-sample no-readme%}

# Module structure

Hashicorp published a set of [guidelines on module definitions](https://www.terraform.io/docs/modules/index.html) stating some naming conventions and the general module structure.

According to those guidelines, module repository names follow the format `terraform-<PROVIDER>-<NAME>`, where `PROVIDER` in our case is AzureRM and `NAME` is a label describing the infrastructure provided. In our case, the Terraform module is called `terraform-azurerm-policy-sample`.

The module structure follows the standard practices recommended by Hashicorp; note, however, that some of the paths defined are specific to this module.
- [`main.tf`](https://github.com/beandrad/terraform-azurerm-policy-sample/blob/main/main.tf). Defines the configuration requirements. It may also configure other resources.

- [`variables.tf`](https://github.com/beandrad/terraform-azurerm-policy-sample/blob/main/variables.tf). Declares the module input variables.

- [`outputs.tf`](https://github.com/beandrad/terraform-azurerm-policy-sample/blob/main/outputs.tf). Declares the module outputs; in this case, the initiative IDs.

- [`/policies`](https://github.com/beandrad/terraform-azurerm-policy-sample/tree/main/policies). (Module specific) Includes the definitions of custom Azure policies. Each policy has its own folder, where the file `policy-rule.json` has the definition and `policy-parameters.json`, the parameters if applicable.

- `<label>-initiative.tf`. (Module specific) Configures the definition of policies and initiatives in Azure. Policies are grouped into initiatives based on the resources they affect and/or by industry-level standards (such as the CIS Hardening Guidelines).

- [`initiative-parameters.tf`](https://github.com/beandrad/terraform-azurerm-policy-sample/blob/main/initiative-parameters.json). (Module specific) Declares the input parameters of all the initiatives defined in the module.

- [`/tests`](https://github.com/beandrad/terraform-azurerm-policy-sample/tree/main/tests). Implements the module acceptance tests.

# Definition of policies and initiatives

This module defines custom policies and initiatives under a management group (`definition_management_group` in `variables.tf`) or current subscription if the management group name is not defined.

Custom policy definitions are created using the `azurerm_policy_definition` resource and built-in policies are imported using the `azurerm_policy_definition` data resource. Both resources are included in the corresponding initiatives Terraform configuration files; unless they are shared across initiatives, in which case they are defined in the `main.tf` file.

It is important to note that policy data resources should be imported using its policy `name` (as opposed to the `displayName`). The reason is that the `displayName` is not unique and it may change, whereas the `name` is unique and it remains the same until the policy is deleted. In the configuration, the `displayName` appears commented out as it describes the policy being imported.

# Acceptance tests

The acceptance tests deploy resources to Azure to check whether the defined initiatives actually work: non-compliant resources cannot be deployed whereas compliant ones are allowed to be deployed.

The following conventions were followed when testing the policy module:

- Tests check that the initiative is working as expected, as opposed to testing individual policies. The reason is that this module only outputs initiatives, all the policies are linked from an initiative.

- Only the behavior of custom policies is tested; built-in policies are expected to work.

Tests are implemented using the [Go testing framework](https://golang.org/pkg/testing/) together with the [Terratest module](https://terratest.gruntwork.io/docs/). This configuration allows calling the Terraform configuration from the Go tests.

The lifecycle of the tests is as follows:

- Setup: load the policy module to define policies and initiatives and assign initiatives in Azure ([`tests/terraform/main.tf`](https://github.com/beandrad/terraform-azurerm-policy-sample/blob/main/tests/terraform/main.tf)).
- Run: try to create compliant resources (for example, [`tests/terraform/resource-location-allow`](https://github.com/beandrad/terraform-azurerm-policy-sample/tree/main/tests/terraform/resource-location-allow)) and non-compliant resources (for example, [`tests/terraform/resource-location-audit`](https://github.com/beandrad/terraform-azurerm-policy-sample/tree/main/tests/terraform/resource-location-audit)).
- Assert: check whether the policy has been correctly applied using the returned error from the Terraform apply for `deny` effect or the policy state for `audit` effect.
- Teardown: delete test resources from Azure.

Running this kind of tests is slow, in particular, those checking effects other than `deny`. There are two main components that cause this delay: the first one is policy definition and assignment, and the second one is policy evaluation (which, as stated above, is required to check the `audit` effect).

In order to speed up the tests, test cases are run in parallel using the [`Parallel() function`](https://golang.org/pkg/testing/#T.Parallel).

Finally, regarding the teardown, it is important to note that it is done in two different steps: one for the resources provisioned during the setup and the other for the resources deployed by each of the test cases. In the first case, the [Cleanup function](https://godoc.org/testing#T.Cleanup) is used; defer wouldn't work since deferred functions are run before parallel subtests are executed. On the other hand, resources created by the test cases are destroyed in a deferred function.

# Final thoughts

As stated in the beginning, the aim of this post is to provide a baseline example of an Azure policy module; however, it is important to point out that quite a few opinionated design decisions have been made and, therefore, this example shouldn't be taken as the only correct way of implementing an Azure policy module.

Having said that, I hope this post helps those devs out there looking for some guidance on how to implement Terraform modules, define Azure custom policies and initiatives, and test those definitions.