Use Azure Application Gateway private link configuration for an internal API Management
Tags: azure, network, privatelink, api
Published: 2022-02-26 16:48:16

---
title: Use Azure Application Gateway private link configuration for an internal API Management
published: true
description: Linking API Management in internal virtual network mode with an Application Gateway private link configuration to another virtual network.
tags: #azure #network #privatelink #api
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/voa3lsfj9rga8uxae0z4.png
---

## TL;DR

When operating **Azure API Management** in an internal virtual network, already integrated with **Azure Application Gateway**, an upcoming feature `AllowApplicationGatewayPrivateLink` allows you to connect this configuration to another virtual network using [Private Link and Private Endpoint](#solution-elements).

## Motivation

In [a post I made beginning of February 2022](https://dev.to/kaiwalter/preliminary-private-linking-an-azure-container-app-environment-3cnf) I linked a virtual network with limited address space - like in a corporate / ExpressRoute / SD-WAN connected scenario - to a Container Apps environment:

![hub/spoke Container Apps configuration with private link](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ruketol7klgmhth52u2e.png)

That setup works as long as ingress only goes from resources in Hub network to Spoke network. But if **Container Apps** needs to call (as in my target scenario) **Azure API Management** - calling from Spoke network into Hub network - that solution is not sufficient.

Looking for options I had [this post](https://dev.to/pwd9000/access-internal-apim-securely-with-private-link-service-313o) which [Marcel.L](https://dev.to/pwd9000) pointed me to back then, where he forwards calls to API Management in another virtual network using a Private Endpoint/Link and Virtual Machine Scale Set combination.

However, as pointed out in my earlier post, my endgame should be to **replace** [**Azure Service Fabric IaaS** container hosting](https://dev.to/kaiwalter/azure-functions-on-service-fabric-40n8)  **with** a higher level abstraction, **PaaS** like **Container Apps**, hence I did not necessarily want to add just another **IaaS** in the process - even one with a lower complexity.

Searching for alternatives I checked on **private linking** capabilities of **Azure API Management** itself. However this cannot be used and mixed when it is already operated in external or internal virtual network mode - [see preview limitations](https://azure.microsoft.com/en-us/blog/secure-your-apis-with-private-link-support-for-azure-api-management/). Hence no option for me. 

> "As with container compute I need my API Management with 100% ingress and egress within a virtual network"

Researching on private IP address options on **Azure Application Gateway** I stumbled over a [**private link feature** in Azure CLI](https://docs.microsoft.com/en-us/cli/azure/network/application-gateway/private-link?view=azure-cli-latest) which - lacking tangible documentation - I exploited here and converted to **Bicep**

## Solution Elements<a name="solution-elements"></a>

To achieve this configuration

![public and private ingress to API Management](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4b5i7ln4k7on0nou1cwu.png)

following solution elements [additionally to my earlier post](https://dev.to/kaiwalter/preliminary-private-linking-an-azure-container-app-environment-3cnf#solution-elements) are required:

- a **Private Link Configuration** on the **Application Gateway**; still to subnet within Hub virtual network
- a **Private Endpoint** in Spoke virtual network
- a **Private DNS Zone** linked to Spoke virtual network so that **Container Apps** resolve to **Private Endpoint IP address**

> complete configuration for all snippets shown in this post can be found in [this repo / tag](https://github.com/KaiWalter/container-apps-experimental/tree/apim-private-link/ca-bicep); I'll state the `filenames` so that snippets can be found more easily

## Prerequisites

- Azure CLI
- Bicep
- Linux shell / bash / ...

## Stage 1 - get preview feature working (as of Feb'2022)

When first trying to deploy Application Gateway with specifying `privateLinkConfiguration`, I was rewarded with a nice error code `SubscriptionNotRegisteredForFeature` and a message like `Subscription /subscriptions/... is not registered for feature PrivateLinkConfigurations required to carry out the requested operation`. Finding no suitable documentation I tried to find the feature switch myself with

```
az feature list --namespace Microsoft.Network -o table | grep Priv
```

I then activated what seemed to make sense for my case:

```
az feature register --name AllowApplicationGatewayPrivateLink --namespace Microsoft.Network
az feature register --name AllowAppGwPublicAndPrivateIpOnSamePort --namespace Microsoft.Network
az provider register -n Microsoft.Network
```

> **DISCLAIMER**: The above steps I figured out being unaware that this is (as of Feb'22) a private preview.
> The official process: _For inquiries about private preview features, please send your subscription id, region, and customer name to appgwpreview@microsoft.com for enrollment._
> This will entitle you officially to the preview and you will get proper documentation and sample code.

## Stage 2 - configuration Application Gateway

To add private link, an entry in `privateLinkConfigurations` section is required. It needs to be put in the same virtual network but a different subnet as the gateway itself (once Application Gateway is deployed to a subnet, it only allows for other Application Gateways to share this subnet, but no other resources).

That `privateLinkConfiguration` is then to be referenced on the `frontendIPConfiguration`. In my sample I share it with the public IP, but I guess, this also can be separated to allow various forwarding rules depending from where ingress is coming from.

`appgw-priv.bicep`

```
...
var fipName = 'appgw-frontend'
...
frontendIPConfigurations: [
  {
    name: fipName
    properties: {
      publicIPAddress: {
        id: pip.id
      }
      privateIPAllocationMethod:'Dynamic'
      privateLinkConfiguration:{
        id: resourceId('Microsoft.Network/applicationGateways/privateLinkConfigurations', appGwName, 'private')
      }
    }
  }
]
privateLinkConfigurations: [
  {
    name: 'private'
    properties: {
      ipConfigurations: [
        {
          name: 'private-ip'
          properties: {
            primary: true
            privateIPAllocationMethod:'Dynamic'
            subnet: {
              id: subnetJumpHubId
            }
          }
        }
      ]
    }
  }
]
...
```

> for simplification in my sample I am using HTTP on port 8080; in production this will be replaced by a proper HTTPS, FQDNs and certificates

Based on private link configuration in Hub network now a private endpoint can be added in Spoke network:

`appgw-priv.bicep`

```
resource pep 'Microsoft.Network/privateEndpoints@2021-05-01' = {
  name: 'pep-priv-gateway'
  location: location
  properties: {
    subnet: {
      id: subnetJumpSpokeId
    }
    privateLinkServiceConnections: [
      {
        properties: {
          privateLinkServiceId: appgw.id
          groupIds: [
            fipName
          ]
        }
        name: 'pep-priv-gateway'
      }
    ]
  }
}
```

`fipName`

**Important to highlight:** the name of the `groupId` added to the Private Endpoint has to match the name of the `frontendIPConfiguration` of the Application Gateway.

## Stage 3 - private DNS zone

Spoke network needs a private DNS zone so that traffic towards the **Private Endpoint** is resolved correctly:

`appgw-priv-dns.bicep`

```
param vnetSpokeId string
param apiName string
param pepIp string

var privateDNSZoneName = 'internal-api.net'

resource privateDnsZone 'Microsoft.Network/privateDnsZones@2020-06-01' = {
  name: privateDNSZoneName
  location: 'global'
}

resource privateDnsZoneLink 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2020-06-01' = {
  parent: privateDnsZone
  name: '${privateDnsZone.name}-link'
  location: 'global'
  properties: {
    registrationEnabled: false
    virtualNetwork: {
      id: vnetSpokeId
    }
  }
}

resource privateDnsZoneEntry 'Microsoft.Network/privateDnsZones/A@2020-06-01' = {
  name: apiName
  parent: privateDnsZone
  properties: {
    aRecords: [
      {
        ipv4Address: pepIp
      }
    ]
    ttl: 3600
  }
}
```

I use my own DNS zone `internal-api.net` here as - unlike other Azure private link capable resources - Application Gateway does (and I guess will) not have its [own private endpoint DNS zones](https://docs.microsoft.com/en-us/azure/private-link/private-endpoint-dns#azure-services-dns-zone-configuration). 

## Stage 4 - deploy API Management, Application Gateway and DNS zone

I had to split deployment of API Management & Application Gateway, return and extract private IP address information from private endpoint and then continue with private DNS deployment. I was not able (or just too lazy) to wire this up within one Bicep file.

`deploy-stage-2.sh`

```
...
az deployment group create --resource-group $RESOURCE_GROUP \
    --template-file apim.bicep \
    --parameters apimName=$APIMNAME \
    appInsightsName=$APPINSIGHTNAME \
    logAnalyticsWorkspaceName=$LOGANALYTICSNAME \
    fapp1Fqdn=$fapp1Fqdn \
    fapp2Fqdn=$fapp2Fqdn

VNET_SPOKE_ID=`az network vnet list --resource-group ${RESOURCE_GROUP} --query "[?contains(name,'spoke')].id" -o tsv`
PEP_NIC_ID=`az network private-endpoint list -g $RESOURCE_GROUP --query "[?name=='pep-priv-gateway'].networkInterfaces[0].id" -o tsv`
PEP_IP=`az network nic show --ids $PEP_NIC_ID --query ipConfigurations[0].privateIpAddress -o tsv`

az deployment group create --resource-group $RESOURCE_GROUP \
    --template-file appgw-priv-dns.bicep \
    --parameters "{\"pepIp\": {\"value\": \"$PEP_IP\"},\"vnetSpokeId\": {\"value\": \"$VNET_SPOKE_ID\"},\"apiName\": {\"value\": \"$APIMNAME\"}}"
```

> along with API Management instance I deploy an API to test calls to Function Apps hosted in Container Apps environment; `appgw-priv.bicep` shown above is a module of `apim.bicep`, hence deployed in the first block

## Stage 5 - testing

To check, that I have not introduced regressions I first test access to Function Apps over public IP of Application Gateway:

`test-api.sh`

```
...
APIMID=`az apim show -n $APIMNAME -g $RESOURCE_GROUP --query id -o tsv`
APIMURL=`az apim show -n $APIMNAME -g $RESOURCE_GROUP --query gatewayUrl -o tsv`
GWURL=`az network public-ip show -n ${APIMNAME}-priv-gateway-pip -g $RESOURCE_GROUP --query dnsSettings.fqdn -o tsv`
GWPORT=`az network application-gateway show -n ${APIMNAME}-gateway -g $RESOURCE_GROUP --query frontendPorts[0].port -o tsv`
SUBKEY=`az rest --method post  --uri ${APIMID}/subscriptions/test-subscription/listSecrets?api-version=2021-08-01 --query primaryKey -o tsv`

curl -s ${GWURL}:${GWPORT}/test/fapp1?subscription-key=$SUBKEY
printf '\n'
curl -s ${GWURL}:${GWPORT}/test/fapp2?subscription-key=$SUBKEY
printf '\n'
```

To make more thorough tests of the Functions Apps in combination with calls to API Management I have to hop on a jump VM in Spoke network:

`test-fapps.sh`

```
...
IP=$(az vm list-ip-addresses -g $RESOURCE_GROUP --query "[?contains(virtualMachine.name, 'hub')].virtualMachine.network.publicIpAddresses[0].ipAddress" -o tsv)

declare -a apps=("fapp1" "fapp2")

for app in "${apps[@]}"
do
    echo "$app"
    fqdn=$(az containerapp show -n $app -g $RESOURCE_GROUP --query configuration.ingress.fqdn -o tsv --only-show-errors)
    ssh ca@$IP curl -s https://$fqdn/api/health
    echo " <<-- check APIM internal status"
    ssh ca@$IP curl -s https://$fqdn/api/apim-status
    echo " <<-- check $app APIM health"
    ssh ca@$IP curl -s https://$fqdn/api/apim-internal-status
    echo " <<-- check $app APIM internal status"
done
```

where 

- `curl -s https://$fqdn/api/health` checks Function App only with a call to a HTTP trigger
- `curl -s https://$fqdn/api/apim-status` and `curl -s https://$fqdn/api/apim-internal-status` call HTTP triggers which themselves forward calls to API Management instance (using the `internal-api.net` domain)

```
[FunctionName("Apim-Status")]
public static async Task<IActionResult> ApimStatus([HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "apim-status")] HttpRequest req)
  => new OkObjectResult(await httpClient.GetAsync("http://ca-kw.internal-api.net:8080/status-0123456789abcdef"));

[FunctionName("Apim-Internal-Status")]
public static async Task<IActionResult> ApimInternalStatus([HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "apim-internal-status")] HttpRequest req)
  => new OkObjectResult(await httpClient.GetAsync("http://ca-kw.internal-api.net:8080/internal-status-0123456789abcdef"));
```

## Public IP vs SKU vs Private Link Configuration

For my scenario a public IP would not be required. Removing it is not supported for `Standard_v2` SKU:

_Application Gateway /subscriptions/.../ca-kw-priv-gateway does not support Application Gateway without Public IP for the selected SKU tier Standard_v2. Supported SKU tiers are Standard,WAF._

On the other hand `PrivateLinkConfigurations` is only supported for `Standard_v2`:

_Application Gateway /subscriptions/.../ca-kw-priv-gateway does not support PrivateLinkConfigurations for the selected SKU tier Standard. Supported SKU tiers are Standard_v2,WAF_v2._

So for the moment I'd keep the public IP and will check again at a later stage of the preview.

## Conclusion

With this solution a major roadblock for me is out of the way and I do not need to operate my own compute resources for network traffic forwarding. It still requires a some polishing and hardening before I can use it even close to our production resources.

A positive effect for me is that with reconfiguration of the existing Application Gateway and a very low footprint of IP addresses consumed in my corporate virtual network, I can even operate the existing environment with **Service Fabric** in coexistence with the new **Container Apps** environment, which reduces migration risks tremendously.