Test Azure Functions scaling on Container Apps with Azure Load Testing
Tags: azure, containerapps, functions, testing
Published: 2022-01-30 13:58:02

---
title: Test Azure Functions scaling on Container Apps with Azure Load Testing
published: true
description: This post shows how to create a workload with Azure Functions in Container Apps - one HTTP ingress and one Service Bus queue ingress - and then test the behavior in Azure Load Testing.
tags: #azure #containerapps #functions #testing
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7ole6jcqvlmp81wu9z6l.png
---

## motivation

I am currently evaluating, whether and how I can transfer this [Azure Functions containers on Service Fabric workload](https://dev.to/kaiwalter/azure-functions-on-service-fabric-40n8) to [Azure Container Apps](https://aka.ms/containerapps).

Primary reason to use Service Fabric back then was to be able to host Azure Functions within our own virtual networks - inbound and outbound - a feature that is now also available with regular Azure Functions (Premium) / App Service hosting. However this type of hosting has other limitations - e.g. the amount of outbound IP addresses required - which does not fit into our scenario.

Container Apps seem to become a valid hosting option for our scenario and our requirements once [BYO/bring-your-own virtual network feature](https://github.com/microsoft/azure-container-apps/issues/3) is available.

> "you could do Kubernetes" ... yeah, but there would be no operational savings & gain for us, compared to Service Fabric; I want the real deal: **I just don't care about the cluster/compute resource**

While waiting I want to explore some of the other capabilities which might come in handy later - starting with [auto scaling](https://docs.microsoft.com/en-us/azure/container-apps/scale-app).

## setup

With [the steps described in the accompanying repository](https://github.com/KaiWalter/container-apps-experimental#test-functions-scaling) these resources are created - using [Pulumi Native](https://www.pulumi.com/registry/packages/azure-native/) this set of resources is created:

> the step - change `ca-pulumi-dotnet/Program.cs` to `FunctionAppStack.cs` - before deploying is important in case to test any of the other Container Apps workload like Dapr; to get the exact working version it makes sense to checkout the [tagged version](https://github.com/KaiWalter/container-apps-experimental/tree/ca-functions-scaling)

- a **Container Registry** to store the images of the 2 Function Apps
- a **Container App Environment** to host 2 **Container App**s : **fapp1** = HTTP ingress onto Service Bus, **fapp2** = Service Bus queue ingress
- a **Service Bus Namespace** to process queued ingress
- a **Load Testing** resource to execute the actual load test
- an **Application Insights** instance to evaluate some of the scaling behavior

What is created with **Pulumi** here can also be achieved with **Azure CLI**, **ARM** or **Bicep** - or a combination. I just wanted the whole stack including Docker image creation in one concise script.

## special ingredients

Being new to **Pulumi** IaC these things needed some more exploring and work on my end:

- hooking the private **Container Registry** credentials into the **Container App** deployment
- hooking **Application Insights** in the **Function App**s
- creating the **Load Test** service including an assignment of **Load Test Owner** role to whomever is executing the deployment

### considering Functions host Id

A few days after original creation of this post [I remembered](https://dev.to/kaiwalter/azure-functions-on-service-fabric-40n8#singleton) that when hosting Function Apps in containers "on your own" / outside App Services and you want to have **proper singleton and scaling behavior**, you need to 

- actively set `hostId` to the same value on all container instances for the same Function App with environment variable `AzureFunctionsWebHost__hostId`
- connect to a **storage account** where Functions runtime keeps all synchronization information with environment variable `AzureWebJobsStorage`

```csharp
    new EnvironmentVarArgs
    {
        Name = "AzureWebJobsStorage",
        SecretRef = "storageconnection",
    },
    new EnvironmentVarArgs
    {
        Name = "AzureFunctionsWebHost__hostId",
        Value = Guid.NewGuid().ToString().Replace("-", ""),
    }
```

> I conducted the test again after this addition, however it had no noticeable influence on scaling and test results.

## load test creation and execution

I created a simple **JMeter** test plan and added it to the repository as `ca-pulumi-dotnet/loadtest.jmx` which then allows inserting `loadtesturlfapp1` Pulumi output value as parameter `ingress_url` when creating the test:

![creating the test and setting URL parameter](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/27v1tbiw4f9c0zx96ohy.png)

Once created the test plan can be executed. It is configured to submit 10000 requests and then stop:

![test plan completed execution](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zis1t8xp6s5fau7nnypv.png)

> an error **Non HTTP response code: java.net.UnknownHostException** indicates, that the URL passed into the `ingress_url` parameter is not correct

## test results

A simple Application Insights query cannot show the effective scaling of the **Container App Environment** but it reveals, that both Function Apps (both in pink color) dwell ~ 15-30 seconds with their minimum single instances and then kick into full gear and start scaling to handle the increased load:

```
requests
| where timestamp between ( todatetime("2022-01-30T12:45:00") .. todatetime("2022-01-30T12:55:00"))
| summarize count() by bin(timestamp,15s), cloud_RoleInstance
| render columnchart 
```

![telemetry showing distribution of scaled instances](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7ole6jcqvlmp81wu9z6l.png)

From **12:49:00 PM** one can see, that HTTP Ingress is finished and the **fapp2** instances process the queued ingress in a balanced manner.

Additionally I crafted this Azure Monitor query to see whether the same amount of instances was started and stopped:

```
ContainerAppConsoleLogs_CL 
| where ContainerImage_s contains "fapp"
| where TimeGenerated between( todatetime("2022-01-30T12:45:00") .. todatetime("2022-01-30T13:15:00") )
| where Log_s contains "Application is shutting down..." or Log_s startswith "Application started."
| parse Log_s with * "Application" eventName "." *
| summarize count() by eventName, bin(TimeGenerated, 5m)
| render columnchart 
```

> `Application started` is logged twice, I only take the entry where the message starts in the beginning without leading blanks.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w3zgvv7sk56s9py1i2lk.png)

## conclusion

To me the simplicity of 
- hosting
- scaling
- load testing

in this sample is overwhelming. I am sure there will be some quirks waiting for our team along the way. I will keep you posted here.

## similar posts

- [Moving Azure Functions from AKS to Container Apps](https://dev.to/christle/moving-azure-functions-from-aks-to-container-apps-k60)