Configure Azure Diagnostic Settings with Azure Policies
Tags: azure, devops, arm, policy
Published: 2020-09-14 18:15:23

<link rel="canonical" href="https://markusmeyer.hashnode.dev/configure-azure-diagnostic-settings-with-azure-policies"/>

# Configure Azure Diagnostic Settings with Azure Policies

## Table of Contents
1 [Objective](#objective )  
2 [Diagnostic Settings in Azure Portal](#diagnosticsettings)  
3 [Policy](#policy)  
3.1 [Policy Rule](#policyrule)  
3.2 [Policy Set Definitions](#policysetdefinitions)  
4 [Permissions](#permissions)  
5 [Deployment](#deployment)  
6 [Policy in Azure Portal](#policyinazureportal)  

## 1 Objective <a id="objective "></a>

Azure resources had to be configured with diagnostic settings.
To apply these settings the general approach with [Azure Policies](https://docs.microsoft.com/en-us/azure/governance/policy/overview "Azure Policies") had the most advantages:

- Management with ARM Templates
- Check for Compliance
- CI/CD Integration

[Azure Diagnostic Settings](https://docs.microsoft.com/en-us/azure/azure-monitor/platform/diagnostic-settings "Azure Diagnostic Settings") can be configured in several ways:

- Azure Portal
- PowerShell
- Azure CLI
- REST API
- [Azure Policy](https://docs.microsoft.com/en-us/azure/governance/policy/overview "Azure Policy")

## 2 Diagnostic Settings in Azure Portal <a id="diagnosticsettings"></a>

The screenshots below display the diagnostic settings (logs and metrics) for a Cosmos DB account.

![Diagnostic Settings](https://user-images.githubusercontent.com/8818198/92326777-f5348380-f054-11ea-82fd-9e02d62e7da9.png "Diagnostic Settings")

![Diagnostic Settings](https://user-images.githubusercontent.com/8818198/92326764-dcc46900-f054-11ea-9373-2f13bfb14b70.png "Diagnostic Settings")

## 3 Policy <a id="policy"></a>

A [policy](https://docs.microsoft.com/en-us/azure/governance/policy/concepts/definition-structure "Azure Policy definition structure") contains different elements.


### 3.1 Policy Rule <a id="policyrule"></a>

Each resource type needs a [Policy Rule](https://docs.microsoft.com/en-us/azure/governance/policy/concepts/definition-structure "Azure Policy definition structure").

I my case, I configured Policies for following resource types:
 - Microsoft.KeyVault/vaults
 - Microsoft.EventGrid/topics
 - Microsoft.EventGrid/eventSubscriptions
 - Microsoft.Web/sites
 - Microsoft.ServiceBus/namespaces
 - Microsoft.Network/networkSecurityGroups
 - Microsoft.Network/applicationGateways

#### 3.1.1 Example policy rule type

*Extract from policyRule:*
```json
          "policyRule": {
  "if": {
    "field": "type",
    "equals": "Microsoft.DocumentDB/databaseAccounts"
  },
  "then": {
    "effect": "[[parameters('effect')]",
    "details": {
      "type": "Microsoft.Insights/diagnosticSettings",
      "name": "[[parameters('profileName')]",
      "existenceCondition": {
        "allOf": [
          {
            "field": "Microsoft.Insights/diagnosticSettings/logs.enabled",
            "equals": "[[parameters('logsEnabled')]"
          },
          {
            "field": "Microsoft.Insights/diagnosticSettings/metrics.enabled",
            "equals": "[[parameters('metricsEnabled')]"
          },
          {
            "field": "Microsoft.Insights/diagnosticSettings/storageAccountId",
            "equals": "[[parameters('storageExists')]"
          }
        ]
      },
      "roleDefinitionIds": [
        "/providers/microsoft.authorization/roleDefinitions/749f88d5-cbae-40b8-bcfc-e573ddc772fa",
        "/providers/microsoft.authorization/roleDefinitions/92aaf0da-9dab-42b6-94a3-d43ce8d16293"
      ]
    }
  }
}

```

##### 3.1.1.1 existenceCondition

Based on the parameter values, the diagnostic settings for logging, metric and storage logging will be checked.

##### 3.1.1.2 roleDefinitionIds

[Azure built-in roles](https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles "Azure built-in roles") are required to set proper permissions:

- [Monitoring Contributor](https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#monitoring-contributor "Monitoring Contributor")
    - 749f88d5-cbae-40b8-bcfc-e573ddc772fa

- [Log Analytics Contributor](https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#log-analytics-contributor "Log Analytics Contributor")
    - 92aaf0da-9dab-42b6-94a3-d43ce8d16293

#### 3.1.2 

The deployment property contains the logs and metrics.

*Extract from policyRule:*

```json
{
  "policyRule": {
    "if": {
    },
    "then": {
      "deployment": {
        "properties": {
          "mode": "incremental",
          "template": {
            "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
            "contentVersion": "1.0.0.0",
            "parameters": {
              "resourceName": {
                "type": "string"
              },
              "location": {
                "type": "string"
              },
              "logAnalytics": {
                "type": "string"
              },
              "archiveStorage": {
                "type": "string"
              },
              "metricsEnabled": {
                "type": "string"
              },
              "logsEnabled": {
                "type": "string"
              },
              "storageExists": {
                "type": "string"
              },
              "profileName": {
                "type": "string"
              }
            },
            "variables": {},
            "resources": [
              {
                "type": "Microsoft.DocumentDB/databaseAccounts/providers/diagnosticSettings",
                "apiVersion": "2017-05-01-preview",
                "name": "[[concat(parameters('resourceName'), '/', 'Microsoft.Insights/', parameters('profileName'))]",
                "location": "[[parameters('location')]",
                "dependsOn": [],
                "properties": {
                  "workspaceId": "[[parameters('logAnalytics')]",
                  "storageAccountId": "[[parameters('archiveStorage')]",
                  "metrics": [
                    {
                      "category": "Requests",
                      "enabled": true,
                      "retentionPolicy": {
                        "days": 0,
                        "enabled": false
                      },
                      "timeGrain": null
                    }
                  ],
                  "logs": [
                    {
                      "category": "DataPlaneRequests",
                      "enabled": true
                    },
                    {
                      "category": "QueryRuntimeStatistics",
                      "enabled": true
                    },
                    {
                      "category": "PartitionKeyStatistics",
                      "enabled": true
                    },
                    {
                      "category": "PartitionKeyRUConsumption",
                      "enabled": true
                    },
                    {
                      "category": "ControlPlaneRequests",
                      "enabled": true
                    },
                    {
                      "category": "GremlinRequests",
                      "enabled": true
                    }
                  ]
                }
              }
            ],
            "outputs": {}
          },
          "parameters": {
            "location": {
              "value": "[[field('location')]"
            },
            "resourceName": {
              "value": "[[field('name')]"
            },
            "logAnalytics": {
              "value": "[[parameters('logAnalytics')]"
            },
            "archiveStorage": {
              "value": "[[parameters('archiveStorage')]"
            },
            "metricsEnabled": {
              "value": "[[parameters('metricsEnabled')]"
            },
            "logsEnabled": {
              "value": "[[parameters('logsEnabled')]"
            },
            "storageExists": {
              "value": "[[parameters('storageExists')]"
            },
            "profileName": {
              "value": "[[parameters('profileName')]"
            }
          }
        }
      }
    }
  }
}

```

### 3.2 Policy Set Definitions<a id="policysetdefinitions"></a>

An [Azure Policy Initiative](https://docs.microsoft.com/en-us/azure/governance/policy/concepts/initiative-definition-structure "Azure Policy Initiative") is a set of policies.

![Initiative - Policies](https://user-images.githubusercontent.com/8818198/92326814-32007a80-f055-11ea-8412-64fc345f06f9.png "Initiative - Policies")



*Extract with the Policy Set Definition:*

```json
{
  "type": "Microsoft.Authorization/policySetDefinitions",
  "name": "[variables('diagnosticSettingsInitiativeName')]",
  "apiVersion": "2019-09-01",
  "dependsOn": [
  ],
  "properties": {
    "displayName": "MarkusMeyer - Apply diagnostic settings for applicable resources - Log Analytics",
    "description": "This initiative configures application Azure resources to forward diagnostic logs and metrics to an Azure Log Analytics workspace.",
    "metadata": {
      "category": "Monitoring"
    },
    "parameters": {
      "logAnalytics": {
        "type": "string",
        "metadata": {
          "displayName": "Log Analytics workspace",
          "description": "Select the Log Analytics workspace from dropdown list",
          "strongType": "omsWorkspace"
        }
      }
    },
    "policyDefinitions": [
      {
        "policyDefinitionId": "[resourceId('Microsoft.Authorization/policyDefinitions/', variables('kvLogPolicyName'))]",
        "parameters": {
          "logAnalytics": {
            "value": "[[parameters('logAnalytics')]"
          }
        }
      },
      {
        "policyDefinitionId": "[resourceId('Microsoft.Authorization/policyDefinitions/', variables('sbLogPolicyName'))]",
        "parameters": {
          "logAnalytics": {
            "value": "[[parameters('logAnalytics')]"
          }
        }
      },
      {
        "policyDefinitionId": "[resourceId('Microsoft.Authorization/policyDefinitions/', variables('cosmosLogPolicyName'))]",
        "parameters": {
          "logAnalytics": {
            "value": "[[parameters('logAnalytics')]"
          }
        }
      },
      {
        "policyDefinitionId": "[resourceId('Microsoft.Authorization/policyDefinitions/', variables('funcLogPolicyName'))]",
        "parameters": {
          "logAnalytics": {
            "value": "[[parameters('logAnalytics')]"
          }
        }
      },
      {
        "policyDefinitionId": "[resourceId('Microsoft.Authorization/policyDefinitions/', variables('evgTopicLogPolicyName'))]",
        "parameters": {
          "logAnalytics": {
            "value": "[[parameters('logAnalytics')]"
          }
        }
      }
    ]
  }
}

```
The complete template can be found in my [GitHub repository](https://github.com/MarkusMeyer13/Azure-ARM-Diagnostic-Policies "https://github.com/MarkusMeyer13/Azure-ARM-Diagnostic-Policies").

## 4 Permissions<a id="permissions"></a>

Following [permissions](https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#resource-policy-contributor) are required to deploy the policies:


|||
|-|-|
| Actions | Description |
| */read | Read resources of all types, except secrets. |
| Microsoft.Authorization/policyassignments/* | Create and manage policy assignments |
| Microsoft.Authorization/policydefinitions/* | Create and manage policy definitions |
| Microsoft.Authorization/policyexemptions/* | |
| Microsoft.Authorization/policysetdefinitions/* | Create and manage policy sets |
| Microsoft.PolicyInsights/* ||
| Microsoft.Support/* | Create and update a support ticket |


## 5 Deployment <a id="deployment"></a>

Ths policies will be [deployed on subscription level](https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/deploy-to-subscription "Create resource groups and resources at the subscription level")

Policies can also be deployed to:
- [Management group](https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/deploy-to-management-group "Create resources at the management group level")
- [Tenant](https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/deploy-to-tenant "Create resources at the tenant level")



### 5.1 Azure CLI

```azurecli-interactive
az deployment sub create  --location westeurope --template-file .\Policy.json --parameters "@Policy.parameters.json"
```

### 5.2 Powershell

```powershell
New-AzDeployment -Name "diagPolicies" -location "West Europe" -TemplateFile .\Policy.json  -verbose -TemplateParameterFile .\Policy.parameters.json
```

[![Deploy To Azure](https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/1-CONTRIBUTION-GUIDE/images/deploytoazure.svg?sanitize=true)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FMarkusMeyer13%2FAzure-ARM-Diagnostic-Policies%2Fmaster%2FPolicy.json)

## 6 Policy in Azure Portal<a id="policyinazureportal"></a>

The deployed policies will be display in [Azure Portal](https://portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyMenuBlade/Overview "Azure Portal").

![Deploy To Azure](https://user-images.githubusercontent.com/8818198/92326826-3fb60000-f055-11ea-9491-1cc74aa673a1.png "Deployed Policy")