F# App Stub for AKS hosting (with WebJobs but without Azure Functions fluff)
Tags: fsharp, azure
Published: 2021-06-01 17:01:58

**TLDR;** Use a standard .NET 5 host which gives you full control (incl. upgradability!) and still allows for WebJobs syntactic sugar, instead of the fluffy and "magical" Azure Functions host. 

## Introduction

In a [previous post](https://dev.to/deyanp/use-azure-kubernetes-service-aks-traefik-instead-of-azure-functions-hosting-azure-api-management-1gkg) I listed a number of reasons why you'd better migrate away from Azure Functions and host your .NET (F# recommended!) apps in AKS. In this post I will try to give you as much code as you need to be able to start with this alternative approach in a matter of minutes instead of hours/days.

I will create a fully-blown stub including all the functionality you may need - WebJobs (EventHubTrigger, QueueTrigger, TimerTrigger), Web Server, custom Background Services, App Insights integration, Console Log Formatting, so it will look as close to Azure Functions as possible, but with full control. 

Note: [Azure Functions support for .NET 5](https://docs.microsoft.com/en-us/azure/azure-functions/dotnet-isolated-process-guide) (the so-called Isolated Host) has actually taken a similar approach, but there is still too much fluff/magic for my liking ..

## Design Approach/Principles


1. Mimic Azure Functions setup - including the usage of WebJobs (used by Azure Functions themselves), Application Insights integration, even the same or slightly better console log formatting!
2. "Close to the metal" or as basic as possible - just use Asp.Net Core, no framework like Saturn, Giraffe etc. required for just a few PUT/POST/PATCH/GET operations.
3. Utilize standard HostBuilder functionality like running additional [HostedServices](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-5.0&tabs=visual-studio#ihostedservice-interface) or [BackgroundServices](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-5.0&tabs=visual-studio#backgroundservice-base-class).

## Program.fs

If you don't have prior knowledge about HostBuilder then in a nutshell it is just a configurable way of spawning several threads/application services, e.g. a web server, some background services used by the WebJobs SDK for listening to Azure Storage Queues or Event Hubs for example, some other custom background processes of yours (e.g. for watching a MongoDB collection) or even some startup code which runs when the whole application starts (e.g. for filling an in-memory cache from a db table).

The main bootstrapping code is in the main function and may look like this:

```fsharp
[<EntryPoint>]
let main argv =
    let builder =
        Host.CreateDefaultBuilder(argv)
        |> Framework.Hosting.HostBuilder.configureLogging
        |> Framework.Hosting.HostBuilder.configureAppInsights
        |> configureWorker1
        |> configureWorker2
        |> Framework.Hosting.HostBuilder.configureWebHost configureEndpoints
        |> configureWebJobs
    
    use tokenSource = new CancellationTokenSource()    
    use host = builder.Build()
    host.RunAsync(tokenSource.Token) |> Async.AwaitTask |> Async.RunSynchronously
    
    0 // return an integer exit code
```
As you see from above, we are configuring logging, App Insights integration, then 2 background workers (IHostedServices), the web server with 3 "HttpTriggers", and web jobs which actually means EventHubTriggers, QueueTriggers, TimerTriggers etc.

## WebJobs Configuration
WebJobs configuration is extremely straightforward: 

```fsharp
let configureWebJobs (builder:IHostBuilder) = 
    builder.ConfigureWebJobs(fun b ->
        b.AddAzureStorageCoreServices() |> ignore
        b.AddEventHubs() |> ignore
        b.AddAzureStorageQueues() |> ignore
        b.AddTimers() |> ignore)
```
Additional services can be configured as well, e.g. `b.AddSignalR() |> ignore`, etc.

## Web Server Configuration
Standard Kestrel configuration ... basic mapping of GET/PUT/POST/PATCH to functions, with manual injection of dependencies (e.g. logger) as a function parameter (even partial application not required):

```fsharp
let configureEndpoints (app:IApplicationBuilder) (endpoints:IEndpointRouteBuilder) =
    let hostedServices = Framework.BackgroundService.Hosted.findAll app.ApplicationServices

    let worker1 = Framework.BackgroundService.Hosted.find hostedServices "Worker1"
    endpoints.MapGet("api/v1/background-services/worker1/status", fun context ->
        worker1.GetProcessingStatusHttp()
        |> MappedHttpResponse.toHttpResponse context
        :> Task) |> ignore

    let worker2 = Framework.BackgroundService.Hosted.find hostedServices "Worker2"
    endpoints.MapGet("api/v1/background-services/worker2/status", fun context ->
        worker2.GetProcessingStatusHttp()
        |> MappedHttpResponse.toHttpResponse context
        :> Task) |> ignore
        
    endpoints.MapGet("api/v1/webFunction1", fun context ->
        let logger = app.ApplicationServices.GetService<ILogger<Object>>()
        
        WebApi.webFunction1 logger context.Request
        |> Async.StartAsTask
        |> Task.bind (MappedHttpResponse.toHttpResponse context)
        :> Task) |> ignore

let configureWebHost configureEndpoints (builder:IHostBuilder) : IHostBuilder =
    builder.ConfigureServices(fun context services ->
        if context.HostingEnvironment.IsDevelopment() then
            services.AddCors(fun options ->
                options.AddDefaultPolicy(fun b ->
                    b
                        .AllowCredentials()
                        .WithOrigins("http://localhost:8080")
                        .WithHeaders("authorization","content-type","if-match","etag","content-disposition","x-requested-with")
                        .WithExposedHeaders("authorization","content-type","if-match","etag","content-disposition")
                        .WithMethods("GET","POST","PUT","PATCH")
                    |> ignore
                )
            ) |> ignore
        ) |> ignore
    
    builder.ConfigureWebHostDefaults(fun b ->
        b.Configure(fun context app ->
            app.UseRouting() |> ignore
            
            if context.HostingEnvironment.IsDevelopment() then        
                app.UseCors() |> ignore

            app.UseEndpoints (fun endpoints -> configureEndpoints app endpoints) |> ignore
            
        ) |> ignore
    )

```
Bonus: As you see above there are 2 status web functions which report the status of the threads started by the background services (see next section).

## Custom Background Services Configuration
You can do anything in a HostedService/BackgroundService, even get rid of the WebJobs SDK and simply listen to event hubs using EventHubProcessor for example by yourself (I have always regarded XxxTrigger as a non-mandatory syntactic sugar ...): 

```fsharp
let startedEvent1 = new ManualResetEvent(false) 

let configureWorker1 (builder:IHostBuilder) : IHostBuilder =
    builder.ConfigureServices(fun context services ->
        services.AddSingleton<IHostedService>(
            fun serviceProvider -> 
                let logger = serviceProvider.GetService<ILogger<Object>>()
                Framework.BackgroundService.Hosted.create
                    "Worker1"
                    (BackgroundServices.worker1 logger startedEvent1) :> IHostedService) |> ignore
        )

let configureWorker2 (builder:IHostBuilder) : IHostBuilder =
    builder.ConfigureServices(fun context services ->
        services.AddSingleton<IHostedService>(
            fun serviceProvider -> 
                let logger = serviceProvider.GetService<ILogger<Object>>()
                Framework.BackgroundService.Hosted.create
                    "Worker2"
                    (BackgroundServices.worker2 logger startedEvent1) :> IHostedService) |> ignore
        )
```

## Logging Configuration, including Console Log Formatting

To simulate the Azure Functions logger formatting I had to unfortunately implement a full-blown ConsoleFormatter ... but also made the log output a bit more-concise on the way ;)

```fsharp
let configureLogging (builder:IHostBuilder) : IHostBuilder =
    builder.ConfigureLogging(fun context b ->
//        b.ClearProviders() |> ignore  // commented out because otherwise Application Insights does not collect logger.Information ...
        b.AddConsole(fun options -> options.FormatterName <- "CustomConsoleFormatter")
            .AddConsoleFormatter<Framework.Logging.Console.CustomConsoleFormatter, Framework.Logging.Console.CustomConsoleFormatterOptions>(fun options ->
                options.Excludes <- Framework.Logging.Console.defaultExcludes
                options.SingleLinePerCategory <- ["Microsoft.Azure.WebJobs.Hosting.OptionsLoggingService", false; "Host.Triggers.Timer", false] |> dict
                options.SingleLine <- true
                options.ColorBehavior <- LoggerColorBehavior.Enabled)
        |> ignore
    )
```

## Application Insights Integration
Application Insights works here as well, note that I am injecting a custom initializer and processor to suppress some http status codes etc. 

```fsharp
let configureAppInsights (builder:IHostBuilder) : IHostBuilder =    
    builder.ConfigureServices(fun context services ->
        let aiOptions = ApplicationInsightsServiceOptions(EnableAdaptiveSampling = false)   // disable sampling (turned on by default, see https://docs.microsoft.com/en-us/azure/azure-monitor/app/sampling#brief-summary)
        services.AddApplicationInsightsTelemetry(aiOptions) |> ignore
        services.AddSingleton<ITelemetryInitializer>(
            Framework.Logging.Telemetry.Configuration.CustomTelemetryInitializer(
                Environment.GetEnvironmentVariable("WEBSITE_SITE_NAME", true),
                Framework.Logging.Telemetry.Configuration.defaultSuppressedStatusCodes))
        |> ignore

        services.AddSingleton<Framework.Logging.Telemetry.Configuration.CustomTelemetryProcessorConfig>(Framework.Logging.Telemetry.Configuration.defaultCustomTelemetryProcessorConfig) |> ignore
        services.AddApplicationInsightsTelemetryProcessor<Framework.Logging.Telemetry.Configuration.CustomTelemetryProcessor>() |> ignore
    )
```

## Custom Functions

And the last bit are the user-defined functions (="Azure Functions): 
```fsharp


module BackgroundServices =
    let worker1 (logger:ILogger) (startedEvent: ManualResetEvent) (token:CancellationToken) : Async<unit> =
        async {
            startedEvent.Set() |> ignore
            while true do
                do logger.LogInformation("Hello from worker1")
                Thread.Sleep(5000)
        }        
    let worker2 (logger:ILogger) (startedEvent: ManualResetEvent) (token:CancellationToken) : Async<unit> =
        async {
            startedEvent.Set() |> ignore
            while true do
                do logger.LogInformation("Hello from worker2")
                Thread.Sleep(5000)
        }        

module WebApi =
    let webFunction1 (logger:ILogger) (req: HttpRequest) : Async<MappedHttpResponse> =
        async {
            do logger.LogInformation("webFunction1")
            return {
                StatusCode = 200
                Content = "Some response" |> MappedHttpResponseContent.Json
                Headers = List.empty
            }
        }

type WebJobs(telemetryConfiguration: TelemetryConfiguration) =
    let telemetryClient = TelemetryClient(telemetryConfiguration)

    [<FunctionName("HandleEventHubMessage")>]
    member _.HandleEventHubMessage
        ([<EventHubTrigger("", Connection = "EventHubConnectionString", ConsumerGroup = "test-cg")>]    // path to event hub is in the connection string
        msg: EventData,
        enqueuedTimeUtc: DateTime,
        sequenceNumber: Int64,
        offset: string,
        logger: ILogger)
        =
        async {
            do logger.LogInformation($"HandleEventHubMessage: {Encoding.UTF8.GetString(msg.Body.ToArray())}")
        } |> Async.StartAsTask

    [<FunctionName("HandleQueueMessage")>]
    member _.HandleQueueMessage
        ([<QueueTrigger("test-queue", Connection = "StorageQueueConnectionString")>] msg: string,
        logger:ILogger)
        =
        async {
            do logger.LogInformation($"HandleQueueMessage: {msg}")
        } |> Async.StartAsTask
        
    [<FunctionName("HandleTimerEvent")>]
    member this.HandleTimerEvent
        ([<TimerTrigger("0 0 0 1 * *", RunOnStartup = true)>] timer: TimerInfo,
        logger: ILogger)
        =
        async {
            do logger.LogInformation("HandleTimerEvent")
        } |> Async.StartAsTask
```
As you see, not much difference compared to "real" Azure Functions ...

## Migration from Azure Functions
1. Add Program.fs if not already existing, and copy over the host builder pipeline (including the required WebJobs AddXxxx services)
2. Configure WebJobs (no change required to corresponding XxxTriggers)
3. Implement configureEndpoints, and remove HttpTrigger attributes from the http-based azure functions (no change required for the non-http azure functions, attributes remain)

## Conclusion
With greater control comes bigger responsibility, however you can see from the above that the required code is not only relatively easy to grasp and maintain, but it offers much more! For example, background services can be injected nicely, and a lot of additional HostBuilder functionality can be used - there are tons of Q&As and documentations online. Last but not least, after migrating from Azure Functions v3 to the above "custom .NET host" we experienced a [50% drop in memory consumption of the AKS pods](https://github.com/Azure/Azure-Functions/issues/1810) which I believe is attributable to the removed Azure Functions bloat, so that was another bonus for us. 

P.S. Working project can be found at https://github.com/deyanp/FSharpAKSAppStub.