Building a CI/CD pipeline for your Azure project with GitHub Actions from scratch
Tags: github, azure
Published: 2021-02-17 09:21:22

---
title: Building a CI/CD pipeline for your Azure project with GitHub Actions from scratch
published: true
description: 
tags: github, azure
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/ngrraihlorwtpsb9l31h.jpg
---

> Cover photo by [Quinten de Graaf][1] on [Unsplash][2]

So you have decided to host your web application on Azure. Now it's time to take care of the infrastructure. Let's open the Azure portal, create all the resources and publish your application with Visual Studio.

But wait! You're going to add some resources and re-create the same infrastructure for different environments. Manual work all over, time-consuming, and prone to error. Wouldn't it be great to have infrastructure as code and let the CI/CD system continuously deploy it together with your web application?

In this blog post, we will use Azure Resource Manager templates and GitHub actions to achieve all of that.

Before we start, please be aware that this blog post isn't a theoretical introduction to ARM templates nor GitHub actions. Our goal is to set up a CI/CD pipeline from scratch to avoid right-click publishing from Visual Studio and manually create Azure resources. **We're good people, and good people don't let friends right-click publishing their projects ;-)**

If you're interested in Azure infrastructure as code in general, definitely check out other solutions as well like Terraform, Pulumi, Farmer and Bicep.

# Tools
The following tools will help us to set up our pipeline:
* Visual Studio Code
 * [Azure Resource Manager (ARM) Tools extension][10] (snippets, syntax highlighting, Azure schema completion and validation)
 * [GitHub Actions Extension][11] (syntax highlighting and snippets)
* Windows Terminal
* Azure CLI
* Git

# Prerequisites
To set up a CI/CD pipeline for our Azure project, we need a GitHub repository, an active Azure subscription and a web application.

In the following example, we use a plain web application created by the dotnet CLI:
```
dotnet new webapp -o HelloWorld
```

With that, our basic folder structure in our GitHub repository looks like the following:
```
│   .gitignore
│
└───src
    │
    └───web
        │   HelloWorld.csproj
        │   ...
```
The last prerequisite is an active Azure subscription. We can use the Azure CLI to log in to our account:
```
az login
```
After a successful login we can see our subscription details in the command prompt:
``` json
{
  "environmentName": "AzureCloud",
  "homeTenantId": "xxx-xxx-xxx-xxx-xxx",
  "id": "xxx-xxx-xxx-xxx-xxx",
  "isDefault": true,
  "managedByTenants": [],
  "name": "Azure subscription 1",
  "state": "Enabled",
  "tenantId": "xxx-xxx-xxx-xxx-xxx",
  "user": {
    "name": "manuel.sidler@xxx.xxx",
    "type": "user"
  }
}
```

Now we're ready to go!

# Azure infrastructure
First, we need to create an Azure resource group. It's a container to group different Azure resources together. We can use the Azure CLI to make the group:
```
az group create --location "West Europe" --name rg-dev-blog
```
Please notice the abbreviation _rg-_ in the name parameter. It's important to use some conventions when we name Azure resources. In case we don't have any specific ones defined in our company, we should stick to the abbreviations documented by [Microsoft][4]. If the command ran successful, we get a response with some details about the created resource group:
``` json
{
  "id": "/subscriptions/xxx-xxx-xxx-xxx-xxx/resourceGroups/rg-dev-blog",
  "location": "westeurope",
  "managedBy": null,
  "name": "rg-dev-blog",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}
```

For our web application, we're going to set up three Azure resources:
* Application Insights
* App Service Plan
* App Service

Application Insights is optional, but it's always a good idea to have it for logging, events, and other stuff. For more information, visit the official [documentation from Microsoft][5].

The App Service Plan is kind of a server farm, which will host our App Service (the dotnet web application in this case). An App Service has to be assigned to an App Service Plan. If you're coming from a Windows Server world, you can think of an App Service Plan as an IIS and an app service as a site.

### ARM Template

Instead of creating these three resources in the Azure portal, we define it in an ARM template JSON file. The question now is where to store it. As we use infrastructure **as code** here obviously, we should save it in our source directory:
```
│   ...
│
└───src
    │
    └───azure
    |   |   azuredeploy.json
    |
    └───web
        │   ...

```
Thanks to the _Azure Resource Manager (ARM) Tools_ extension for VS Code, we now can type `arm!` in the empty file. This snippet will create a skeleton of the ARM template:
``` json
{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {},
    "functions": [],
    "variables": {},
    "resources": [],
    "outputs": {}
}
```
Now let's start with the [Application Insights resource][6]:
``` json
{
    ...
    "variables": {
        "appInsightsName": "appi-dev-blog"
    },
    "resources": [
        {
            "name": "[variables('appInsightsName')]",
            "type": "Microsoft.Insights/components",
            "apiVersion": "2015-05-01",
            "location": "[resourceGroup().location]",
            "kind": "web",
            "properties": {
                "application_Type": "web"
            }
        }
    ],
    ...
}
```
There are two things to notice here:
* Use variables for resource names, because we usually reference them in several places inside an ARM template
* To avoid additional traffic cost and performance issues, host your Azure resources in the same location. We can use the `resourceGroup().location` function to refer to the resource group location

Next, we take care of the [App Service Plan][7]:
``` json
{
    ...
    "parameters": {
        "appServicePlanSku": {
            "type": "string"
        }
    },
    ...
    "variables": {
        ...
        "appServicePlanName": "plan-dev-blog"
    },
    "resources": [
        ...
        {
            "name": "[variables('appServicePlanName')]",
            "type": "Microsoft.Web/serverfarms",
            "apiVersion": "2018-02-01",
            "location": "[resourceGroup().location]",
            "sku": {
                "name": "[parameters('appServicePlanSku')]"
            },
            "properties": {
                
            }
        }
    ],
    ...
}
```
Please notice the `sku` option, which basically defines the CPU and memory of our machine. This option will probably vary in different environments. We introduce a new parameter to pass it from outside into the template. We'll take care of that in a later step.

Last but not least, here's the template definition for our [Azure App Service][8]:
``` json
{
    ...
    "variables": {
        ...
        "appServiceName": "app-dev-blog"
    },
    "resources": [
        ...
        {
            "name": "[variables('appServiceName')]",
            "type": "Microsoft.Web/sites",
            "apiVersion": "2018-11-01",
            "location": "[resourceGroup().location]",
            "dependsOn": [
                "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]",
                "[resourceId('Microsoft.Insights/components', variables('appInsightsName'))]"
            ],
            "kind": "app",
            "properties": {
                "serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]",
                "siteConfig": {
                    "appSettings": [
                        {
                            "name": "APPLICATIONINSIGHTS_CONNECTION_STRING",
                            "value": "[concat('InstrumentationKey=',reference(resourceId('Microsoft.Insights/components', variables('appInsightsName')), '2015-05-01').InstrumentationKey)]"
                        },
                        {
                            "name": "ApplicationInsightsAgent_EXTENSION_VERSION",
                            "value": "~2"
                        },
                        {
                            "name": "APPINSIGHTS_PROFILERFEATURE_VERSION",
                            "value": "1.0.0"
                        },
                        {
                            "name": "APPINSIGHTS_SNAPSHOTFEATURE_VERSION",
                            "value": "1.0.0"
                        },
                        {
                            "name": "DiagnosticServices_EXTENSION_VERSION",
                            "value": "~3"
                        },
                        {
                            "name": "InstrumentationEngine_EXTENSION_VERSION",
                            "value": "~1"
                        },
                        {
                            "name": "SnapshotDebugger_EXTENSION_VERSION",
                            "value": "~1"
                        },
                        {
                            "name": "XDT_MicrosoftApplicationInsights_BaseExtensions",
                            "value": "~1"
                        },
                        {
                            "name": "XDT_MicrosoftApplicationInsights_Mode",
                            "value": "recommended"
                        },
                        {
                            "name": "XDT_MicrosoftApplicationInsights_PreemptSdk",
                            "value": "disabled"
                        }
                    ]
                }
            }
        }
    ],
    ...
}
```
A couple of things to mention here:
* With `dependsOn` we're able to define dependencies between resources in an ARM template. When deploying the template, the Azure Resource Manager will take care of the dependency graph and create the Azure resources in the correct order
* While `dependsOn` defines the relationship inside the template, `serverFarmId` sets the actual reference to the App Service Plan instance
* We add several app settings entries to enable and configure the Application Insights options

### ARM Template Parameters

Our ARM template is now complete. Or is it? Remember the `sku` parameter for the App Service Plan? Right! We have to create an additional parameters file:
```
│   ...
│
└───src
    │
    └───azure
    |   |   azuredeploy.json
    |   |   azuredeploy.parameters.json
    |
    └───web
        │   ...

```
In a real project, we would create a parameters file per environment. But in our example here one is enough. The content of the file is actually quite simple. We can again use a snippet (`armp!`) to create the skeleton:
``` json
{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
    }
}
```
Now we just have to define our `sku` parameter (visit the official [documentation][9] for all available plans):
``` json
{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "appServicePlanSku": {
            "value": "F1"
        }
    }
}
```

### Azure Active Directory Service Principal

There's one thing left to do before we take care of the actual CI/CD pipeline. We need an Azure Active Directory service principal which we have to use for deploying the ARM template. Let's go back to the Azure CLI and do that:
```
az ad sp create-for-rbac --name sp-dev-blog --role contributor --scopes /subscriptions/xxx-xxx-xxx-xxx-xxx/resourceGroups/rg-dev-blog --sdk-auth
```
When working in Azure, we should always stick with the principle of least privilege. That's why we assign the contributor role just to the scope of our resource group.

By setting the `--sdk-auth` parameter, we get back a JSON response which we have to save as a GitHub repository secret:
``` json
{
  "clientId": "xxx-xxx-xxx-xxx-xxx",
  "clientSecret": "xxx",
  "subscriptionId": "xxx-xxx-xxx-xxx-xxx",
  "tenantId": "xxx-xxx-xxx-xxx-xxx",
  "activeDirectoryEndpointUrl": "https://login.microsoftonline.com",
  "resourceManagerEndpointUrl": "https://management.azure.com/",
  "activeDirectoryGraphResourceId": "https://graph.windows.net/",
  "sqlManagementEndpointUrl": "https://management.core.windows.net:8443/",
  "galleryEndpointUrl": "https://gallery.azure.com/",
  "managementEndpointUrl": "https://management.core.windows.net/"
}
```
To do that, we open our GitHub repository, switch to _Settings_, click _Secrets_ and add the secret there:
![GitHub secrets](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k510b3pz0ypf2d0kyjrc.png)

We're done now with defining our Azure infrastructure as code. Perfect!

# GitHub actions for Azure
GitHub Action workflows are defined in `.yml` files and stored under _.github/workflows_:
```
│   ...
│
│───.github
│   │
│   └───workflows
│       |   azure.yml
│       |   web.yml
|
└───src
    │
    └───azure
    |   |   ...
    |
    └───web
        │   ...
```

So let's create those folders and the `.yml` files and start with `azure.yml`. First, we have to give the workflow a name and define a trigger:
``` yml
name: Azure

on:
  push:
    paths:
      - 'src/azure/**'
      - '.github/workflows/azure.yml'
    branches:
      - '**'
```
Since we have our web application and Azure infrastructure as code in the same repository, we don't want to trigger this workflow if something inside the web application changes. That's why we configure a path filter here.

Next, we define some environment variables which we later can reuse in our jobs:
``` yml
...

env:
  AZURE_RESOURCE_GROUP: 'rg-dev-blog'
  TEMPLATE_FILE: 'src/azure/azuredeploy.json'
  PARAMETERS_FILE: 'src/azure/azuredeploy.parameters.json'
```
Now we have to define the continuous integration and deployment jobs. For continuous integration, we'd like to execute the following steps:
1. Checkout source code
2. Log in to Azure CLI
3. Validate the ARM template
4. Execute a `what-if` command to see, what actually  would happen in case of a deployment

And here's the result in our `azure.yml` file:
``` yml
...

jobs:
  CI:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: login to Azure
        uses: Azure/login@v1
        with:
          creds: ${{ secrets.AZURE_DEPLOYMENT_CREDENTIALS }}

      - name: validate template
        uses: Azure/cli@v1.0.0
        with:
          inlineScript: 'az deployment group validate --resource-group ${{ env.AZURE_RESOURCE_GROUP}} --template-file ${{ env.TEMPLATE_FILE }} --parameters ${{ env.PARAMETERS_FILE }}'

      - name: what-if
        uses: Azure/cli@v1.0.0
        with:
          inlineScript: 'az deployment group what-if --resource-group ${{ env.AZURE_RESOURCE_GROUP}} --template-file ${{ env.TEMPLATE_FILE }} --parameters ${{ env.PARAMETERS_FILE }}'
```
Notice the reference to our service principal secrets (`${{ secrets.AZURE_DEPLOYMENT_CREDENTIALS }}`) when login to the Azure CLI.

The continuous deployment job should behave a little bit different. First, it has only to run if the CI job was successful and if we're on the _main_ branch. Second, there's no need to validate the template and rerun the _what-if_ command. We can simply deploy the template:
``` yml
jobs:
  CI:
    ...

  CD:
    needs: [CI]
    if: success() && (github.ref == 'refs/heads/main')

    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: login to Azure
        uses: Azure/login@v1
        with:
          creds: ${{ secrets.AZURE_DEPLOYMENT_CREDENTIALS }}

      - name: deploy
        uses: Azure/cli@v1.0.0
        with:
          inlineScript: 'az deployment group create --resource-group ${{ env.AZURE_RESOURCE_GROUP}} --template-file ${{ env.TEMPLATE_FILE }} --parameters ${{ env.PARAMETERS_FILE }}'
```
That's it! If we now commit our workflow file, the actions get triggered. Since we're committing directly to the _main_ branch, the CD action runs and executes our ARM template deployment. To see this in action, we can switch to GitHub and click on the _Actions_ menu:
![GitHub Action in progress](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1g7jvkoabxd71cvhwuhn.png)
![GitHub Action CI](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z4o1w15ukcrbwgtloamq.png)

To verify that all Azure resources have been created successfully, we can run the following Azure CLI command:
```
az resource list --query "[?resourceGroup=='rg-dev-blog'].{ name: name, resourceType: type }"
```
The command returns a list of our deployed resources:
``` json
[
  {
    "name": "appi-dev-blog",
    "resourceType": "Microsoft.Insights/components"
  },
  {
    "name": "plan-dev-blog",
    "resourceType": "Microsoft.Web/serverFarms"
  },
  {
    "name": "app-dev-blog",
    "resourceType": "Microsoft.Web/sites"
  }
]
```

# GitHub actions for web
One thing left to complete the CI/CD pipeline for our Azure project: a GitHub workflow file for the web application. So let's open the empty `web.yml` file and define our action. Again, we first start with the name and the trigger:
``` yml
name: Web

on:
  push:
    paths:
      - 'src/web/**'
      - '.github/workflows/web.yml'
    branches:
      - '**'
```
Like in the workflow for our Azure infrastructure, this workflow should only get triggered by changes at the web application or the workflow file itself. We continue with some environment variables:
``` yml
...

env:
  BUILD_CONFIGURATION: Release
  PUBLISH_OUTPUT: web_publish_output
  AZURE_WEBAPP_NAME: app-dev-blog
  WEB_PROJECT: ./src/web/HelloWorld.csproj
```
The CI job will actually be quite simple:
1. Check out source code
2. Set up .NET Core 5
3. Compile the project

Of course, if we had some unit tests, we would also execute them here:
``` yml
...

jobs:
  CI:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2

    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 5.0.100

    - name: build web
      run: dotnet build ${{ env.WEB_PROJECT }}
```

As in our previous workflow, the CD job should only run if the CI job was successful and if we're on the _main_ branch. Again we check out the source code and set up .NET Core 5. After that, we use the dotnet CLI to publish the web application and deploy it via the Azure CLI:
``` yml
jobs:
  CI:
    ...
        
  CD:
    needs: [CI]
    if: success() && (github.ref == 'refs/heads/main')
    
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v2
        
      - name: Setup .NET Core
        uses: actions/setup-dotnet@v1
        with:
          dotnet-version: 5.0.100

      - name: publish web
        run: dotnet publish ${{ env.WEB_PROJECT }} -c ${{ env.BUILD_CONFIGURATION }} -o ${{ env.PUBLISH_OUTPUT }}
          
      - name: login to azure
        uses: Azure/login@v1
        with:
          creds: ${{ secrets.AZURE_DEPLOYMENT_CREDENTIALS }}
      
      - name: deploy to azure
        uses: Azure/webapps-deploy@v2
        with:
          app-name: ${{ env.AZURE_WEBAPP_NAME }}
          package: './${{ env.PUBLISH_OUTPUT }}'
```
After a commit to the _main_ branch, the CI and CD jobs ran successfully, and the web application got deployed:
![GitHub action web](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vcbh59a4ppik2o3nah90.png)
![Web application](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qju87ni8xfmo4hw7yj20.png)

#Conclusion
As we saw in this blog post, it's not that hard to create a simple CI/CD pipeline. Of course, in a real-world project, the pipeline's and ARM templates' size and complexity will grow. That's exactly why it's important to start with them right at the beginning of the project. It's so much harder to set up a CI/CD pipeline when a project already got to a larger size and complexity.

You can find the complete code samples in [this][3] GitHub repository.

[1]: https://unsplash.com/@quinten149?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText
[2]: https://unsplash.com/s/photos/pipeline?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText
[3]: https://github.com/manuelsidler/azure-github-ci-cd
[4]: https://docs.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview
[5]: https://docs.microsoft.com/en-us/azure/cloud-adoption-framework/ready/azure-best-practices/resource-abbreviations
[6]: https://docs.microsoft.com/en-us/azure/templates/microsoft.insights/components
[7]: https://docs.microsoft.com/en-us/azure/templates/microsoft.web/serverfarms
[8]: https://docs.microsoft.com/en-us/azure/templates/microsoft.web/sites
[9]: https://azure.microsoft.com/en-us/pricing/details/app-service/windows/
[10]: https://marketplace.visualstudio.com/items?itemName=msazurermtools.azurerm-vscode-tools
[11]: https://github.com/formulahendry/vscode-github-actions