Building and Deploying C# Azure Functions using Multi-Stage Pipelines in Azure DevOps | by Will Velida | Towards Data Science

As part of my personal development, I’ve created a personal health platform that uses various different microservices (Built using Azure Functions) that extract data from my Fitbit account and store them in an Azure Cosmos DB database. I have other microservices that pass messages between different services via Azure Service Bus.
For this project, I use Azure DevOps to build my artifacts, run my unit tests and deploy my microservices to Azure. The great thing about DevOps is that we can do all of this with in the YAML pipeline.
Yes I said YAML. Honestly, I don’t know what the fuss is all about 😂
In a previous post, I talked about how we can deploy NuGet packages to a private feed in Azure Artifacts using YAML pipelines. If you haven’t read that post yet, you can check it out below!
Publishing NuGet packages to a private Azure Artifacts feed with YAML Build files
In this article, we will turn our attention to building and deploying C# Azure Functions using a single build file.
We’ve got quite a bit to cover, so I’ll break down my YAML file and talk about each stage in the following order:
Triggering a Build 👷‍♂️👷‍♀️
Using User-Defined Variables in our pipelines 👨‍🔬👩‍🔬
Defining Stages 💻
Building our project 🔨
Running our tests 🧪
Getting code coverage 🧾
Producing a Build Artifact 🏠
Using Secrets from Key Vault 🔑
Deploying our Function to Azure ⚡
Running our build pipeline 🚀
In our build pipelines, we can use triggers to run our pipelines. We can use different types of triggers, ranging from triggering a build when we push our code to a defined branch, to scheduled triggers based on a CRON schedule.
For this project, I’m just triggering my build every time I push to the main branch. To do so, I define this in my YAML file like so:
trigger:  - main
Now if I wanted to do this properly, we could define multiple branch names to trigger builds from like so:
trigger:  - feature/*  - bugfix/*
This would trigger the builds when we push code to either our feature or bugfix branch. When we raise Pull Requests in DevOps, we can create policies on our PR to only be eligible to merge back into our main branch provided that the build was successful.
This helps us increase our confidence that the code we are merging (and deploying) is of high quality, particularly if we are running tests as part of our build pipeline.
To enable build validation on your branch policies, check out this documentation
We can use user-defined variables in our YAML files to use throughout our pipeline. Doing this gives us the benefit of re-using common values across various stages and tasks in our pipeline, and it allows us to track changes to them over time using version control.
We can define variables within out file with the following scope:
At the root level, making it available to all jobs in our pipeline.
At the stage level, making it available to a specific stage in our pipeline.
At the job level, making it available to a specific job in our pipeline.
In my case, I’m going to be re-using these variables throughout my pipeline, so I’m defining them at the root level.
variables:  buildConfiguration: 'Release'  vmImageName: 'vs2017-win2016'  functionAppName: 'famyhealthfitbitbody'  azureSubscription: '<azureSubscription-id-or-name>'  workingDirectory: '$(System.DefaultWorkingDirectory)/MyHealth.Fitbit.Body'  projectName: 'MyHealth.Fitbit.Body'
We’re not just limited to user-defined variables in YAML pipelines! We can use System and Environment variables as well. To learn more about defining variables in YAML pipelines, check out this article
Within our YAML pipelines, we can organize our jobs into Stages. These divide our pipeline into logical....well, stages that perform certain tasks, such as ‘build the app’, ‘deploy to production’.
In this example, I just have a stage and deploy stage. Every YAML pipeline has at least one stage even if we don’t define it. As I have done in my file, we can also run stages depending on whether the previous stage has succeeded or not, like so:
stages:  - stage: Build    displayName: Build Stage    jobs:      - job: Build        displayName: Build MyHealth.Fitbit.Body        pool:          vmImage: $(vmImageName)
Using YAML pipelines, we can define multiple stages of our pipeline and store it as code. Check out the documentation if you want to learn more about how Stages work in YAML pipelines.
Let’s turn our attention to our Function. First up, I want to restore and build my .csproj files. In our YAML pipelines, we can se .NET Core tasks to run dotnet commands against our artifacts. To build my project, I’ll restore and build.
I’m using a custom NuGet feed hosted on Azure Artifacts, so I will need to point to this feed when restoring.
For the build task, I want to build a release package, so I’ll pass in the buildConfiguration variable as an argument when running the build command.
For both commands, I’m going to point to all .csproj files in my repository.
steps:- task: DotNetCoreCLI@2  displayName: Restore  inputs:   command: 'restore'   feedsToUse: 'select'   vstsFeed: '<feed-id>'   projects: '**/*.csproj'- task: DotNetCoreCLI@2  displayName: Build  inputs:   command: 'build'   projects: '**/*.csproj'   arguments: --configuration $(buildConfiguration)
Alongside my function code, I have a project for my Unit Tests. As part of my build, I want to run my unit tests to make sure that my changes haven’t broken any of our tests and that they still pass. If any of my tests fail, I want the build to fail.
To run our tests, we can use a .NET Core task that runs the test command and point the task at our Unit Test .csproj file.
I also want to collect code coverage metrics with Coverlet. Coverlet is .NET tool that allows us to get unit test coverage for line, branch and method coverage. In our arguments parameters, I’m telling my task to run Coverlet to get the code coverage and then publish the test results to the */TestResults/Coverage folder on my build agent:
- task: DotNetCoreCLI@2  displayName: Run Unit Tests  inputs:   command: 'test'   projects: '**/*UnitTests/*.csproj'   arguments: '--configuration $(buildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/'   publishTestResults: true
Once we’ve generated our coverage report, we can publish it to see the results. I’ve got two tasks within my pipeline to achieve this.
First, I’m using a bash script task to install the report generator onto the Build agent. Once that’s installed, I’m creating the xml report into my TestResults/Coverage folder on the agent. We can also specify the report type that we want to generate, but for this tutorial I’m just generating an inline HTML report.
If you want to use a task for this instead of running an inline script, you can try installing the following task onto your DevOps organisation.
Once we have generated a report, I’ve got a second task that publishes the code coverage report.
- script: |    dotnet tool install -g dotnet-reportgenerator-globaltool    reportgenerator -reports:$(Build.SourcesDirectory)/TestResults/Coverage/coverage.cobertura.xml -targetdir:$(Build.SourcesDirectory)/CodeCoverage -reporttypes:HtmlInline_AzurePipelines;Cobertura    displayName: Create Code coverage report- task: PublishCodeCoverageResults@1  displayName: 'Publish Code Coverage'  inputs:    codeCoverageTool: Cobertura    summaryFileLocation: '$(Build.SourcesDirectory)/**/coverage.cobertura.xml'    reportDirectory: '$(Build.SourcesDirectory)/TestResults/Coverage/'
When our build finishes, we can review our code coverage report within DevOps! It should look similar to this:
We’ve built our project and our tests pass! We now need to produce a build artifact that we will deploy as our Function.
Starting with publish, all we need here is a .NET Core task that runs the publish command, publishes our project as a ‘Release’ package, zip it and then publish it to our Artifact Staging Directory on our build agent.
We then use a Publish Build Artifact task to publish an artifact called ‘drop’ into our staging directory.
- task: DotNetCoreCLI@2  displayName: Publish  inputs:    command: 'publish'    publishWebProjects: false    projects: '**/*.csproj'    arguments: '--configuration $(buildConfiguration) --output $(build.artifactstagingdirectory)'    zipAfterPublish: True- task: PublishBuildArtifacts@1  displayName: 'Publish Artifact'  inputs:    PathtoPublish: '$(build.artifactstagingdirectory)'
Just a mini-recap before we move onto deployment. In our Build Stage we have:
Built our .NET Projects
Run our Unit Tests to ensure our build is a quality build
Generate and publish a code test coverage report.
Publish a build artifact to deploy to Azure.
With that achieved, we are now ready to move onto our deployment stage!
For our deployment stage, we’ll need to create a new stage in our YAML file.
In this case, I’ve added a condition and dependsOn property to our Stage YAML block. In the YAML file, I’m saying that the ‘Deploy’ stage depends on our ‘Build’ stage and that the ‘Deploy’ stage will only run on the condition that the ‘Build’ Stage has succeeded.
- stage: Deploy    displayName: 'Deploy MyHealth.Fitbit.Body'    dependsOn: Build    condition: succeeded()    jobs:      - deployment: Deploy        displayName: Deploy MyHealth.Fitbit.Body        environment: Production        pool:          vmImage: $(vmImageName)
When we use multiple stages in a pipeline, they will run in the order that we define them in the YAML file. We can add dependencies to our stages using the dependsOn parameter.
We can control which stages run by specifying the condition of that Stage. In my case, I’m stating that the ‘Deploy’ stage will only run provided that the ‘Build’ stage has run successfully.
To learn more about how conditions work in YAML pipelines, check out the following documentation.
In my function, I’m doing performing a variety of tasks such as sending messages to Service Bus Queues and Topics, calling APIs using Access tokens etc. etc.
In order to make these operations, I need to use connection strings and secrets that I’m storing in a Azure Key Vault. Because these are secrets and I don’t want to expose them, I need to use a Azure Key Vault task within my pipeline.
For this purpose, I can use the Azure Key Vault task to download the secrets I need.
As a pre-requisite for this task, I need to create a Azure Resource Manager service connection that’s linked to my Azure Subscription and I need a Key Vault that contains my secrets.
Once I have this setup, I can use the following snippet in my pipeline:
steps:  - task: AzureKeyVault@1    inputs:      azureSubscription: '$(azureSubscription)'      KeyVaultName: '<key-vault-name>'      SecretsFilter: '*'      RunAsPreJob: false
Here, we are defining the subscription that contains our Key Vault with our secrets, and we are retrieving all of the secrets in our Key Vault. We could filter these out to use a comma-separated list of the secret names we want to download, but for now I’m just downloading everything.
We’re finally ready to deploy our Function! ⚡
For this I’m going to use a Azure App Service Deploy Task
We can use this task to deploy a variety of different app service types, but in my YAML snippet, I’m specifying functionApp as our type.
As a pre-req for this task, you’ll need an App Service instance to deploy your code to.
I’m also passing the package that we built earlier as our package, using the $functionAppName variable for the name of our Function and then passing in key-values for our App Settings. I can pass in the secrets I need for my Function since we downloaded them in our Key Vault task.
- task: AzureRmWebAppDeployment@4  displayName: 'Deploy Azure Function'  inputs:    azureSubscription: '$(azureSubscription)'    appType: functionApp    WebAppName: $(functionAppName)                       package: '$(Pipeline.Workspace)/drop/MyHealth.Fitbit.Body.zip'    appSettings: '-FUNCTIONS_WORKER_RUNTIME "dotnet" -FUNCTIONS_EXTENSION_VERSION "~3" -KeyVaultName "<key-vault-name>" -AccessTokenName "<secret-name>" -ServiceBusConnectionString "$(<secret-value>)" -APPINSIGHTS_INSTRUMENTATIONKEY "<some-key>" -ExceptionQueue "myhealthexceptionqueue" -BodyTopic "myhealthbodytopic" -WEBSITE_TIME_ZONE "New Zealand Standard Time"'    enableCustomDeployment: true    RemoveAdditionalFilesFlag: true
Before we finish, here’s my full YAML build pipeline:
trigger:  - mainvariables:  buildConfiguration: 'Release'  vmImageName: 'vs2017-win2016'  functionAppName: 'famyhealthfitbitbody'  azureSubscription: '<azureSubscription-id-or-name>'  workingDirectory: '$(System.DefaultWorkingDirectory)/MyHealth.Fitbit.Body'  projectName: 'MyHealth.Fitbit.Body'stages:  - stage: Build    displayName: Build Stage    jobs:      - job: Build        displayName: Build MyHealth.Fitbit.Body        pool:          vmImage: $(vmImageName)        steps:          - task: DotNetCoreCLI@2            displayName: Restore            inputs:              command: 'restore'              feedsToUse: 'select'              vstsFeed: '<artifact-feed-id>'              projects: '**/*.csproj'          - task: DotNetCoreCLI@2            displayName: Build            inputs:              command: 'build'              projects: '**/*.csproj'              arguments: --configuration $(buildConfiguration)          - task: DotNetCoreCLI@2            displayName: Run Unit Tests            inputs:              command: 'test'              projects: '**/*UnitTests/*.csproj'              arguments: '--configuration $(buildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/'              publishTestResults: true          - script: |              dotnet tool install -g dotnet-reportgenerator-globaltool              reportgenerator -reports:$(Build.SourcesDirectory)/TestResults/Coverage/coverage.cobertura.xml -targetdir:$(Build.SourcesDirectory)/CodeCoverage -reporttypes:HtmlInline_AzurePipelines;Cobertura            displayName: Create Code coverage report          - task: PublishCodeCoverageResults@1            displayName: 'Publish Code Coverage'            inputs:              codeCoverageTool: Cobertura              summaryFileLocation: '$(Build.SourcesDirectory)/**/coverage.cobertura.xml'              reportDirectory: '$(Build.SourcesDirectory)/TestResults/Coverage/'          - task: DotNetCoreCLI@2            displayName: Publish            inputs:              command: 'publish'              publishWebProjects: false              projects: '**/*.csproj'              arguments: '--configuration $(buildConfiguration) --output $(build.artifactstagingdirectory)'              zipAfterPublish: True          - task: PublishBuildArtifacts@1            displayName: 'Publish Artifact'            inputs:              PathtoPublish: '$(build.artifactstagingdirectory)'  - stage: Deploy    displayName: 'Deploy MyHealth.Fitbit.Body'    dependsOn: Build    condition: succeeded()    jobs:      - deployment: Deploy        displayName: Deploy MyHealth.Fitbit.Body        environment: Production        pool:          vmImage: $(vmImageName)        strategy:          runOnce:            deploy:              steps:                - task: AzureKeyVault@1                  inputs:                    azureSubscription: '$(azureSubscription)'                    KeyVaultName: '<key-vault-name>'                    SecretsFilter: '*'                    RunAsPreJob: false                - task: AzureRmWebAppDeployment@4                  displayName: 'Deploy Azure Function'                  inputs:                    azureSubscription: '$(azureSubscription)'                    appType: functionApp                    WebAppName: $(functionAppName)                                       package: '$(Pipeline.Workspace)/drop/MyHealth.Fitbit.Body.zip'                    appSettings: '-FUNCTIONS_WORKER_RUNTIME "dotnet" -FUNCTIONS_EXTENSION_VERSION "~3" -KeyVaultName "<key-vault-name>" -AccessTokenName "<secret-name>" -ServiceBusConnectionString "$(<secret-value>)" -APPINSIGHTS_INSTRUMENTATIONKEY "<some-key>" -ExceptionQueue "myhealthexceptionqueue" -BodyTopic "myhealthbodytopic" -WEBSITE_TIME_ZONE "New Zealand Standard Time"'                    enableCustomDeployment: true                    RemoveAdditionalFilesFlag: true
We’ve successfully created a build pipeline in YAML that will be triggered every time we push to our main branch. We can view the status of our build in Azure DevOps, which looks like this:
As you can see at the bottom of the image, we can see the result of each stage within our pipeline. If a particular stage had failed, we could use this to re-run a particular stage instead of having to push our code again to main.
Multi-Stage Build pipelines are awesome for defining our build and release process all within the same file. By using source control, we can track changes to our build process over time. We can also do useful things like generate and publish code coverage reports, download secrets from key vault and deploy apps all within one file!
Hopefully you found this article useful! As always, if you have any questions, feel free to comment below or ask me on Twitter!
Happy Coding! 💻👨‍💻👩‍💻