Azure Pipelines secret management with Key Vault
Published: 2022-04-23 05:51:03

# Summary
In this article, I would like to introduce an example to input secrets as variables for the IaC pipeline in Azure Pipelines and then keep the secrets solely in Azure Key Vault securely. Once you deploy the Azure resources, you do not need to keep the secrets as Pipeline Variable any more. This makes your secret management easier and more effective.

Sample code: [api-management-sample](https://github.com/koheikawata/api-management-sample)

# TOC
- [Context](#context)
- [IaC pipeline](#iac-pipeline)
  - [Sequence](#sequence)
  - [Key Vault stage](#key-vault-stage)
  - [Bicep stage](#bicep-stage)

# Context <a name="context"></a>

Azure Pipeline Library allow you to use Variable Group to input secrets to the IaC pipeline. You can [Link secrets from an Azure key vault](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups?view=azure-devops&tabs=yaml#link-secrets-from-an-azure-key-vault) to secure them. However, obviously you need a Key Vault in the first run that stores secrets, and it does not work for setting secrets in for the first time run of IaC pipeline.

**This does not work**
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f0q30xho3uea8wvbpiv1.png)

Another option is [Azure Pipeline Secret Variable](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#secret-variables) where you can securely set secrets for the IaC pipeline first run. However, when you run the IaC pipeline the second time or later for the purpose of infrastructure configuration, you still need the same secret in the Pipeline Secret Variable. This is not what I want to do because you have to keep the secrets in two locations, the Key Vault and Pipeline Secret Variable for the system operation. The most secure and effective secret management is to store secrets solely in the Key Vault for the operation.

**Keeping secrets in two locations is not what I want to do**

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rcz9ug68a148pano8i8q.png)
 
What I want to do is to set secrets in Pipeline Secret Variable for the first time run of IaC deployment, and to remove the secrets from Pipeline Secret Variable and store them only in the Key Vault. The IaC pipeline faces errors without secret inputs from Pipeline Secret Variable for the second run. What I need to make this work for the second run is to download the existing secrets from the Key Vault and set them as Azure Pipeline variables.

**This is what I want to do**

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b5gkpqi461t6dmxzpfm0.png)

# IaC pipeline <a name="iac-pipeline"></a>

I am trying to explain how to make this work with the sample code [api-management-sample](https://github.com/koheikawata/api-management-sample). For the sample code, you can refer to those articles.

- [Azure API Management authentication - Part.1](https://dev.to/koheikawata/azure-api-management-authentication-part1-1198)
- [Azure API Management authentication - Part.2](https://dev.to/koheikawata/azure-api-management-authentication-part2-pd9)

## Sequence <a name="sequence"></a>

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0pygjzsko61r6br7qr0b.png)

## Key Vault stage <a name="key-vault-stage"></a>

- Two secret inputs of `BASIC_AUTH_PASS` and `CLIENT_SECRET` have to be set as Azure DevOps Pipeline Secret Variable.
- In Key Vault stage, it uses for each statement for the repeatable tasks with two secret variables.
- Parameters for the repeatable tasks have `name`, `secret`, `kvsecretname`.
- `secret` is important to set as parameter so the pipeline can dynamically use the secret value in the powershell script.
- In the powershell script, `if ( '${{ PipelineSecret.secret }}' -like '*${{ PipelineSecret.name }}*' )` sees if secrets exist in Pipeline Secret Variable. This syntax is tricky, but if Pipeline Secret Variable does not exist, `PipelineSecret.secret` becomes `$(BASIC_AUTH_PASS)` or `$(CLIENT_SECRET)`. That is why `like '*${{ PipelineSecret.name }}*'` syntax is used.
- `Write-Host "##vso[task.setvariable variable=$env:SecretVariableName;issecret=true;isOutput=true]$secretValue"` sets secrets as variables in this pipeline, which can be used in the later stage. `issecret=true` makes the variable as secure string, and `isOutput=true` allows to be used in the later stage.

[iac_pipeline_subscription.yml](https://github.com/koheikawata/api-management-sample/blob/main/pipelines/iac_pipeline_subscription.yml)
```
parameters:
- name: PipelineSecrets
  type: object
  default:
    - name: BASIC_AUTH_PASS
      secret: $(BASIC_AUTH_PASS)
      kvsecretname: $(KVSECRET_NAME_BASIC_PASS)
    - name: CLIENT_SECRET
      secret: $(CLIENT_SECRET)
      kvsecretname: $(KVSECRET_NAME_CLIENTSECRET)

stages:
- stage: KeyVault
  jobs:
  - ${{ each PipelineSecret in parameters.PipelineSecrets }}:
    - job: ${{ PipelineSecret.name }}
      variables:
        - name: SecretVariableName
          value: ${{ PipelineSecret.name }}
      steps:
      - task: AzurePowerShell@5
        name: ${{ PipelineSecret.name }}
        displayName: Set ${{ PipelineSecret.name }} variable
        env:
          SecretValue: ${{ PipelineSecret.secret }}
        inputs:
          azureSubscription: $(AZURE_SVC_NAME)
          azurePowerShellVersion: latestVersion
          ScriptType: InlineScript
          Inline: |
            if ( '${{ PipelineSecret.secret }}' -like '*${{ PipelineSecret.name }}*' ){
              $secretValue = Get-AzKeyVaultSecret -VaultName $env:KeyVaultName -Name '${{ PipelineSecret.kvsecretname }}' -AsPlainText
              Write-Host "##vso[task.setvariable variable=$env:SecretVariableName;issecret=true;isOutput=true]$secretValue"
              Write-Host "Input variable $env:SecretVariableName does not exists. $secretValue is set for the later stage."
            }
            else{
              Write-Host "##vso[task.setvariable variable=$env:SecretVariableName;issecret=true;isOutput=true]'${{ PipelineSecret.secret }}'"
              Write-Host "Input variable $env:SecretVariableName already exists. Use this variable for the later stage."
            }
```


## Bicep stage <a name="bicep-stage"></a>

- This stage needs `value: $[ stageDependencies.KeyVault.BASIC_AUTH_PASS.outputs['BASIC_AUTH_PASS.BASIC_AUTH_PASS'] ]` to use variables set in the previous stage. You can refer the syntax to [Use output variables from tasks](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#use-output-variables-from-tasks)
- Secret variables should be set with `env:`. The syntax is described in [Set secret variables](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#secret-variables)

[iac_pipeline_subscription.yml](https://github.com/koheikawata/api-management-sample/blob/main/pipelines/iac_pipeline_subscription.yml)
```
- stage: Deploy
  jobs:
  - job: AzureResourceGroupDeployment
    timeoutInMinutes: 0
    variables:
      - name: BasicAuthPass
        value: $[ stageDependencies.KeyVault.BASIC_AUTH_PASS.outputs['BASIC_AUTH_PASS.BASIC_AUTH_PASS'] ]
      - name: ClientSecret
        value: $[ stageDependencies.KeyVault.CLIENT_SECRET.outputs['CLIENT_SECRET.CLIENT_SECRET'] ]
      - name: CertThumbprint
        value: $[ stageDependencies.KeyVault.Certificate.outputs['TaskCertificate.CertThumbprint'] ]
    steps:
    - task: AzureCLI@2
      displayName: Deploy Azure resources
      env:
        BasicAuthPass: $(BasicAuthPass)
        ClientSecret: $(ClientSecret)
        CertThumbprint: $(CertThumbprint)
      inputs:
        azureSubscription: $(AZURE_SVC_NAME)
        scriptType: ps
        scriptLocation: inlineScript
        inlineScript: |
          az group create --name $(ResourceGroupName) --location $(LOCATION)
          az deployment group create --resource-group $(ResourceGroupName) --template-file $(BicepFilePath) `
            --parameters $(BicepParameterFilePath) `
            base_name=$(BASE_NAME) `
            environment_symbol=$(ENVIRONMENT_SYMBOL) `
            aad_objectid_svc=$(AAD_OBJECTID_SVC) `
            aad_appid_client=$(AAD_APPID_CLIENT) `
            aad_appid_backend=$(AAD_APPID_BACKEND) `
            aad_tenantid=$(AAD_TENANTID) `
            basic_auth_user=$(BASIC_AUTH_USER) `
            kvcert_name_api=$(KVCERT_NAME_API) `
            kvsecret_name_basic_pass=$(KVSECRET_NAME_BASIC_PASS) `
            kvsecret_name_cert_thumbprint=$(KVSECRET_NAME_CERT_THUMBPRINT) `
            kvsecret_name_clientsecret=$(KVSECRET_NAME_CLIENTSECRET) `
            kvsecret_name_subscription_key=$(KVSECRET_NAME_SUBSCRIPTION_KEY) `
            apim_api_name_ad=$(APIM_API_NAME_AD) `
            apim_api_path_ad=$(APIM_API_PASH_AD) `
            apim_api_name_basic=$(APIM_API_NAME_BASIC) `
            apim_api_path_basic=$(APIM_API_PASH_BASIC) `
            apim_api_name_cert=$(APIM_API_NAME_CERT) `
            apim_api_path_cert=$(APIM_API_PASH_CERT) `
            basic_auth_pass=$(BasicAuthPass) `
            client_secret=$(ClientSecret) `
            cert_thumbprint=$(CertThumbprint)
```