Exploring the awesome Bicep Test Framework 🧪
Tags: bicep, azurebicep, azure, devops
Published: 2023-10-22 09:37:39

---
title: Exploring the awesome Bicep Test Framework 🧪
published: true
date: 2023-10-09 07:21:00 UTC
tags: Bicep,AzureBicep,Azure,bicep,devops
canonical_url: https://rios.engineer/exploring-the-bicep-test-framework-%f0%9f%a7%aa/
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5rwara6l5hguming291j.png
---

## Introduction

Bicep test framework is still in early developmental phase being an **experimental** feature. It was first presented by Bicep team on the August community call and peaked the interest from the community, myself included.

The goal of the test framework is a focus on client-side tests without the need to send off to ARM, this will help validate earlier in the CI/CD lifecycle and bring more confidence come the time of actual deployment.

Sounds great and I’m a huge fan of this direction.

All the content and files are available to lab in my Bicep learn repository, [Bicepify!](https://github.com/riosengineer/Bicepify)

## How to enable the Bicep test framework?

Test framework is still in experimental stages! Proceed with caution. Bugs, changes at any moment, all the usual warnings.

Adding the two experimental flags in your `bicepconfig.json` file is required. This sits in your root directory, you can find out more from the Microsoft documentation [here](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/bicep-config).

<svg xmlns="http://www.w3.org/2000/svg" width="54" height="14" viewbox="0 0 54 14"><g fill="none" fill-rule="evenodd" transform="translate(1 1)"><circle cx="6" cy="6" r="6" fill="#FF5F56" stroke="#E0443E" stroke-width=".5"></circle><circle cx="26" cy="6" r="6" fill="#FFBD2E" stroke="#DEA123" stroke-width=".5"></circle><circle cx="46" cy="6" r="6" fill="#27C93F" stroke="#1AAB29" stroke-width=".5"></circle></g></svg><svg xmlns="http://www.w3.org/2000/svg" style="width:24px;height:24px" fill="none" viewbox="0 0 24 24" stroke="currentColor" stroke-width="2"><path stroke-linecap="round" stroke-linejoin="round" d="M4.5 12.75l6 6 9-13.5"></path><path stroke-linecap="round" stroke-linejoin="round" d="M16.5 8.25V6a2.25 2.25 0 00-2.25-2.25H6A2.25 2.25 0 003.75 6v8.25A2.25 2.25 0 006 16.5h2.25m8.25-8.25H18a2.25 2.25 0 012.25 2.25V18A2.25 2.25 0 0118 20.25h-7.5A2.25 2.25 0 018.25 18v-1.5m8.25-8.25h-6a2.25 2.25 0 00-2.25 2.25v6"></path></svg>
```
{
    "experimentalFeaturesEnabled": {
        "testFramework": true,
        "assertions": true
  }
}
```
JSON

Make sure your Bicep version is updated (Azure CLI):

```
az bicep upgrade
az bicep --version
```

Also make sure your Bicep VSCode extension is up to date.

I had issues with Azure CLI and had to manually install the Bicep CLI to get the ‘test’ part of the cmdlet to work for me at the time of writing: [Set up Bicep development and deployment environments – Azure Resource Manager | Microsoft Learn](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/install#install-manually)

## Assertions

Assertion statements are Boolean expressions that help validate the expectations of the file, which we can then reference to a corresponding `tests`, file for example:

<svg xmlns="http://www.w3.org/2000/svg" width="54" height="14" viewbox="0 0 54 14"><g fill="none" fill-rule="evenodd" transform="translate(1 1)"><circle cx="6" cy="6" r="6" fill="#FF5F56" stroke="#E0443E" stroke-width=".5"></circle><circle cx="26" cy="6" r="6" fill="#FFBD2E" stroke="#DEA123" stroke-width=".5"></circle><circle cx="46" cy="6" r="6" fill="#27C93F" stroke="#1AAB29" stroke-width=".5"></circle></g></svg><svg xmlns="http://www.w3.org/2000/svg" style="width:24px;height:24px" fill="none" viewbox="0 0 24 24" stroke="currentColor" stroke-width="2"><path stroke-linecap="round" stroke-linejoin="round" d="M4.5 12.75l6 6 9-13.5"></path><path stroke-linecap="round" stroke-linejoin="round" d="M16.5 8.25V6a2.25 2.25 0 00-2.25-2.25H6A2.25 2.25 0 003.75 6v8.25A2.25 2.25 0 006 16.5h2.25m8.25-8.25H18a2.25 2.25 0 012.25 2.25V18A2.25 2.25 0 0118 20.25h-7.5A2.25 2.25 0 018.25 18v-1.5m8.25-8.25h-6a2.25 2.25 0 00-2.25 2.25v6"></path></svg>
```
// Rios Engineer - Bicep Test Framework
targetScope = 'resourceGroup'

param location string
param kind string

// Storage Account
module storage 'br/public:storage/storage-account:3.0.1' = {
  name: 'riosengineerlongchar001'
  params:{
    kind: kind
    location: location
  }
}

// Test location is valid for UK regions
assert locationUk = contains(location, 'uk')

// SKU assert
assert sku = contains(kind, 'StorageV2')
```
BICEP

We’re assessing that the `storage` code block returns a `true` or `false` values for each of our assertions listed above. In this example, we are testing that the region contains ‘uk’ (so either uksouth or ukwest regions), and that the **kind** returns a true value for `StorageV2`.

**Assert statement syntax:**

- `assert` keyword
- Name of your assertion statement, in our case `locationUk`
- Must be Boolean for this to work – as mentioned above 

**bicepparam file example:**

<svg xmlns="http://www.w3.org/2000/svg" width="54" height="14" viewbox="0 0 54 14"><g fill="none" fill-rule="evenodd" transform="translate(1 1)"><circle cx="6" cy="6" r="6" fill="#FF5F56" stroke="#E0443E" stroke-width=".5"></circle><circle cx="26" cy="6" r="6" fill="#FFBD2E" stroke="#DEA123" stroke-width=".5"></circle><circle cx="46" cy="6" r="6" fill="#27C93F" stroke="#1AAB29" stroke-width=".5"></circle></g></svg><svg xmlns="http://www.w3.org/2000/svg" style="width:24px;height:24px" fill="none" viewbox="0 0 24 24" stroke="currentColor" stroke-width="2"><path stroke-linecap="round" stroke-linejoin="round" d="M4.5 12.75l6 6 9-13.5"></path><path stroke-linecap="round" stroke-linejoin="round" d="M16.5 8.25V6a2.25 2.25 0 00-2.25-2.25H6A2.25 2.25 0 003.75 6v8.25A2.25 2.25 0 006 16.5h2.25m8.25-8.25H18a2.25 2.25 0 012.25 2.25V18A2.25 2.25 0 0118 20.25h-7.5A2.25 2.25 0 018.25 18v-1.5m8.25-8.25h-6a2.25 2.25 0 00-2.25 2.25v6"></path></svg>
```
using './main.bicep'

param location = 'northeurope'
param kind = 'StorageV2'
```
BICEP

## Test File

In my observations tests & assertions are case sensitive.

Now we’ve got some assertions defined we can run some tests against these to check their values represent what is intended.

Creating a `tests.bicep` file like the below to run the tests against:

<svg xmlns="http://www.w3.org/2000/svg" width="54" height="14" viewbox="0 0 54 14"><g fill="none" fill-rule="evenodd" transform="translate(1 1)"><circle cx="6" cy="6" r="6" fill="#FF5F56" stroke="#E0443E" stroke-width=".5"></circle><circle cx="26" cy="6" r="6" fill="#FFBD2E" stroke="#DEA123" stroke-width=".5"></circle><circle cx="46" cy="6" r="6" fill="#27C93F" stroke="#1AAB29" stroke-width=".5"></circle></g></svg><svg xmlns="http://www.w3.org/2000/svg" style="width:24px;height:24px" fill="none" viewbox="0 0 24 24" stroke="currentColor" stroke-width="2"><path stroke-linecap="round" stroke-linejoin="round" d="M4.5 12.75l6 6 9-13.5"></path><path stroke-linecap="round" stroke-linejoin="round" d="M16.5 8.25V6a2.25 2.25 0 00-2.25-2.25H6A2.25 2.25 0 003.75 6v8.25A2.25 2.25 0 006 16.5h2.25m8.25-8.25H18a2.25 2.25 0 012.25 2.25V18A2.25 2.25 0 0118 20.25h-7.5A2.25 2.25 0 018.25 18v-1.5m8.25-8.25h-6a2.25 2.25 0 00-2.25 2.25v6"></path></svg>
```
// Rios Engineer - Bicep Test Framework
// Pass test 
test mainTest 'main.bicep' = {
  params: {
    kind: 'StorageV2'
    location: 'uksouth'
  }
}
// Fail test 
test mainTestFail 'main.bicep' = {
  params: {
    kind: 'BlobStorage'
    location: 'northeurope'
  }
}
```
BICEP

**Test syntax:**

- `test` keyword
- Name of the test code block (e.g. mainTest in our example)
- Reference the file to run tests against 
- Parameters object mocking from the main Bicep file

## Running the test

We should have files like so now:

```
.
└── bicep/
    ├── main.bicep
    ├── main.bicepparams
    └── tests.bicep
```

Which means we can now run the tests cmdlet to review if it passes or fails the tests file we want to validate with.

```
bicep test .\filename.bicep
```

<svg xmlns="http://www.w3.org/2000/svg" width="54" height="14" viewbox="0 0 54 14"><g fill="none" fill-rule="evenodd" transform="translate(1 1)"><circle cx="6" cy="6" r="6" fill="#FF5F56" stroke="#E0443E" stroke-width=".5"></circle><circle cx="26" cy="6" r="6" fill="#FFBD2E" stroke="#DEA123" stroke-width=".5"></circle><circle cx="46" cy="6" r="6" fill="#27C93F" stroke="#1AAB29" stroke-width=".5"></circle></g></svg><svg xmlns="http://www.w3.org/2000/svg" style="width:24px;height:24px" fill="none" viewbox="0 0 24 24" stroke="currentColor" stroke-width="2"><path stroke-linecap="round" stroke-linejoin="round" d="M4.5 12.75l6 6 9-13.5"></path><path stroke-linecap="round" stroke-linejoin="round" d="M16.5 8.25V6a2.25 2.25 0 00-2.25-2.25H6A2.25 2.25 0 003.75 6v8.25A2.25 2.25 0 006 16.5h2.25m8.25-8.25H18a2.25 2.25 0 012.25 2.25V18A2.25 2.25 0 0118 20.25h-7.5A2.25 2.25 0 018.25 18v-1.5m8.25-8.25h-6a2.25 2.25 0 00-2.25 2.25v6"></path></svg>
```
 ~ bicep test .\tests.bicep
WARNING: The following experimental Bicep features have been enabled: TestFramework. Experimental features should be enabled for testing purposes only. Do not enable these settings for any production usage, or you may be unexpectedly broken at any time!
[✓] Evaluation mainTest Passed!
[✗] Evaluation mainTestFail Failed at 2 / 3 assertions!
        [✗] Assertion locationUk failed!
        [✗] Assertion sku failed!
Evaluation Summary: Failure!
Total: 2 - Success: 1 - Skipped: 0 - Failed: 1
```
BICEP

We can see the first test block passed successfully as the parameters being passed through all match the true values we want to test for.

However, as we can see the second test block fails because this test block looks for a different SKU type, which the parameter is not matching. The assertion returns a false value and this test fails. Awesome right?

### Some limitations & observations

- You must pass in a parameter object, you cannot reference an existing parameter file currently
- Test blocks must be in a separate file from the .bicep template you want to test
- Test result output summary is not standardised
- strings are case sensitive, e.g. StorageV2 would pass, but storagev2 would not if you don’t match your param case sensitivity throughout

## CICD Unit Testing

With the introduction to this new test framework we can now start to look at how this can be integrated to CI/CD pipeline flows, as another linting tool in the arsenal to validate code pre-deployment.

I hope with time the team provide support for some Sarif output so we can have some nice Azure DevOps reporting on the tests output for reviewing.

## Conclusion

This is a fantastic step forward for Bicep, shifting left some unit testing and giving us another useful linting tool. However, as it’s still in experimental I expect things to change and more features to be added (I believe the file will move to .biceptest instead).

Yes it does potentially overlap what-if + Azure Policy but I think it’s a good way to help bring another useful validation & linting tool to the table for teams to validate their code during development and pre-deployment.

I think it’ll also further increase adoption of Bicep with more and more useful features like this.

Check out my blog on testing Bicep modules here: [https://rios.engineer/bicep-modules-with-psrule-testing-documentation-ci-pipeline-examples/](https://rios.engineer/bicep-modules-with-psrule-testing-documentation-ci-pipeline-examples/)

The post [Exploring the awesome Bicep Test Framework 🧪](https://rios.engineer/exploring-the-bicep-test-framework-%f0%9f%a7%aa/) appeared first on [Rios Engineer](https://rios.engineer).