Virtual Network architecture 3 - Key Vault Private Endpoint
Published: 2022-08-29 14:38:00

# Summary

This article is Part.3 of virtual network architecture series. I will share the details of the private endpoint with Key Vault in [api-management-vnet](https://github.com/koheikawata/api-management-vnet). 

- [Virtual Network architecture 1 - Do I need virtual network?](https://dev.to/koheikawata/virtual-network-architecture-1-do-i-need-virtual-network-1nhk)
- [Virtual Network architecture 2 - Deployment pipelines](https://dev.to/koheikawata/virtual-network-architecture-2-deployment-pipelines-39p5)
- [Virtual Network architecture 3 - Key Vault Private Endpoint](https://dev.to/koheikawata/virtual-network-architecture-3-key-vault-private-endpoint-43nj)
- [Virtual Network architecture 4 - SQL Database Private Endpoit](https://dev.to/koheikawata/virtual-network-architecture-4-sql-database-private-endpoit-49o2)
- [Virtual Network architecture 5 - App Service Private Endpoint](https://dev.to/koheikawata/virtual-network-architecture-5-app-service-private-endpoint-5eao)
- [Virtual Network architecture 6 - Service Bus Private Endpoint](https://dev.to/koheikawata/virtual-network-architecture-6-service-bus-private-endpoint-oa)
- [Virtual Network architecture 7 - Self-hosted agent](https://dev.to/koheikawata/virtual-network-architecture-7-self-hosted-agent-32j4)

# TOC
- [Private Endpoint configuration](#configuration)
- [Access to Key Vault](#access)
- [Application Gateway SSL certificate](#ssl)

# Private Endpoint configuration <a name="configuration"></a>

- **Private Endpoint**: Deploy the private endpoint and connect it to Key Vault and Key Vault's subnet in [PrivateEndpoint.bicep](https://github.com/koheikawata/api-management-vnet/blob/main/src/bicep/vnet/modules/PrivateEndpoint.bicep).

```
KeyVaultId:existingKv.id
VirtualNetwork2SubnetIdKv:existingVnet2.properties.subnets[3].id

properties: {
  privateLinkServiceConnections: [
    {
      name: pe_name_kv
      properties: {
        privateLinkServiceId: KeyVaultId
        groupIds: [
          'vault'
        ]
      }
    }
  ]
  subnet: {
    id: VirtualNetwork2SubnetIdKv
    properties: {
      privateEndpointNetworkPolicies: 'Enabled'
    }
  }
}
```

- **Private DNS**: Deploy Private DNS with the DNS name `privatelink.vaultcore.azure.net` in [PrivateDns2.bicep](https://github.com/koheikawata/api-management-vnet/blob/main/src/bicep/vnet/modules/PrivateDns2.bicep).

```
var pdns_name_kv = 'privatelink.vaultcore.azure.net'

resource PrivateDnsKv 'Microsoft.Network/privateDnsZones@2020-06-01' = {
  name: pdns_name_kv
  location: 'global'
}
```
- **Virtual network link**: Link the deployed Private DNS to the virtual network where the Key Vault subnet exists in [PrivateDns2.bicep](https://github.com/koheikawata/api-management-vnet/blob/main/src/bicep/vnet/modules/PrivateDns2.bicep).

```
VirtualNetwork2Id:existingVnet2.id

resource VnetLinkKv 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2020-06-01' = {
  name: '${PrivateDnsKv.name}/${PrivateDnsKv.name}-linkc'
  location: 'global'
  properties: {
    registrationEnabled: false
    virtualNetwork: {
      id: VirtualNetwork2Id
    }
  }
}
```

- **Private DNS A record**: Create a DNS A record and set the IP address from the deployed private endpoint in [PrivateDns2.bicep](https://github.com/koheikawata/api-management-vnet/blob/main/src/bicep/vnet/modules/PrivateDns2.bicep).

```
output PrivateEndpointKvIpAddress string = PrivateEndpointKv.properties.customDnsConfigs[0].ipAddresses[0]

resource PrivateDnsAKv 'Microsoft.Network/privateDnsZones/A@2020-06-01' = {
  name: '${PrivateDnsKv.name}/${KeyVaultName}'
  properties: {
    ttl: 3600
    aRecords: [
      {
        ipv4Address: PrivateEndpointKvIpAddress
      }
    ]
  }
}
```

# Access to Key Vault <a name="access"></a>

- **App Service** extracts credentials from the Key Vault with Key Vault reference through the Private Endpoint. The outbound from App Service is through the v-net integration. The Key Vault reference is defined in [AppService.bicep](https://github.com/koheikawata/api-management-vnet/blob/main/src/bicep/vnet/modules/AppService.bicep).

```
resource AppServiceConfig 'Microsoft.Web/sites/config@2021-02-01' = {
  properties: {
    'Sql:ConnectionString': '@Microsoft.KeyVault(VaultName=${kv_name};SecretName=${kvsecret_name_sqlcs})'
    'Servicebus:ConnectionString': '@Microsoft.KeyVault(VaultName=${kv_name};SecretName=${kvsecret_name_sbcs})'
  }
}
```

- **Functions** extracts credentials from the Key Vault in the same way as the App Service.The Key Vault reference is defined in [FunctionApp.bicep](https://github.com/koheikawata/api-management-vnet/blob/main/src/bicep/vnet/modules/FunctionApp.bicep).

```
resource FuncConfig 'Microsoft.Web/sites/config@2021-02-01' = {
  properties: {
    'ServiceBusConnectionString': '@Microsoft.KeyVault(VaultName=${kv_name};SecretName=${kvsecret_name_sbcs})'
    'SqlConnectionString': '@Microsoft.KeyVault(VaultName=${kv_name};SecretName=${kvsecret_name_sqlcs})'
  }
}
```

- **Container Instance** running the self-hosted agent extracts Azure Active Directory client app secret from the Key Vault to get the Azure Active Directory OAuth2.0 token for the integration test in [integration-test-bash.yml](https://github.com/koheikawata/api-management-vnet/blob/main/pipelines/templates/integration-test-bash.yml).

```
clientSecret=$(az keyvault secret show --vault-name $(KEYVAULT_NAME) --name $(KVSECRET_NAME_AADCLIENT) --query value -o tsv)
```

- **Application Gateway** implements TLS termination for the backend pool, which means the API Management, with the certificate stored in the Key Vault. Application Gateway is not capable of accessing Key Vault through Private Endpoint but through Service Endpoint.

```
var agw_kv_secret_id = 'https://${kv_name}${environment().suffixes.keyvaultDns}/secrets/${kvcert_name_agw}'

sslCertificates: [
  {
    name: agw_ssl_certificate_name
    properties: {
      keyVaultSecretId: agw_kv_secret_id
    }
  }
]

httpListeners: [
  {
    properties: {
      sslCertificate: {
        id: resourceId('Microsoft.Network/applicationGateways/sslCertificates', agw_name, agw_ssl_certificate_name)
      }
    }
  }
]
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g71pxjlpaxdbbve03bm7.png)

# Application Gateway SSL certificate <a name="ssl"></a>

The problem is that Application Gateway cannot access Key Vault through Private Endpoint to extract SSL certificate in the configuration above, as of August 2022. This is discussed in the forum [Application Gateway: Integration with Key Vault does not work #33157](https://github.com/MicrosoftDocs/azure-docs/issues/33157). In sum, Application Gateway has to access Key Vault's public IP through [Service Endpoint](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-service-endpoints-overview), not through private IP.

- **Service Endpoint**: To enable Service Endpoint for this case, you have to add the configuration when deploying the virtual network subnet for the Key Vault. You can see [VirtualNetwork.bicep](https://github.com/koheikawata/api-management-vnet/blob/main/src/bicep/vnet/modules/VirtualNetwork.bicep) and find the configuration below. The Service Endpoint configuration should be implemented on the virtual network subnet where the service that wants to access Key Vault, which means Application Gateway subnet.

```
resource VirtualNetwork1 'Microsoft.Network/virtualNetworks@2021-03-01' = {
  properties: {
    subnets: [
      {
        name: snet_name_1_agw
        properties: {
          addressPrefix: snet_prefix_1_agw
          networkSecurityGroup: {
            id: Nsg1AgwId
          }
          serviceEndpoints: [
            {
              service: 'Microsoft.KeyVault'
              locations: [
                '*'
              ]
            }
          ]
        }
      }
    ]
  }
}
```

- **Public network access** should be enabled because Application Gateway reaches out to the Key Vault through the public IP, but you can limit the access only from the Application Gateway subnet in [KeyVault2.bicep](https://github.com/koheikawata/api-management-vnet/blob/main/src/bicep/vnet/modules/KeyVault2.bicep). `bypass: 'AzureServices'` enables all Azure services to access the Key Vault through its public IP, `virtualNetworkRules: [{id: VirtualNetwork1SubnetIdAgw}]` allows access from Application Gateway subnet, and `defaultAction: 'Deny'` shut down all the access except Azure services and the Application Gateway virtual network subnet. All the access other than Application Gateway SSL certificate are through the Private Endpoint.

```
resource KeyVault 'Microsoft.KeyVault/vaults@2021-10-01' = {
  properties: {
    publicNetworkAccess: 'Enabled'
    networkAcls: {
      bypass: 'AzureServices'
      defaultAction: 'Deny'
      virtualNetworkRules: [
        {
          id: VirtualNetwork1SubnetIdAgw
        }
      ]
    }
  }
}
```