C#: Azure Open AI and Function Calling
Tags: azure, openai, functioncalling
Published: 2023-07-28 08:12:00

As announced [in this post](https://techcommunity.microsoft.com/t5/azure-ai-services-blog/function-calling-is-now-available-in-azure-openai-service/ba-p/3879241), Azure Open AI now supports Function Calling feature.

I don't explain what it is, but I share my experiment result and C# code.

## Prerequisites

- Azure Subscription and Open AI account
- Deploy model that supports Function Calling, e.g. gpt-35-turbo-16k

## Test scenario

I wonder if LLM can chain functions if needed. How it behaves when it has multiple functions, etc. So, I tested.

Firstly, I added two functions.

``GetWeatherFunction.cs``
```csharp
public class GetWeatherFunction
{
    static public string Name = "get_current_weather";

    // Return the function metadata
    static public FunctionDefinition GetFunctionDefinition()
    {
        return new FunctionDefinition()
        {
            Name = Name,
            Description = "Get the current weather in a given location",
            Parameters = BinaryData.FromObjectAsJson(
            new
            {
                Type = "object",
                Properties = new
                {
                    Location = new
                    {
                        Type = "string",
                        Description = "The city and state, e.g. San Francisco, CA",
                    },
                    Unit = new
                    {
                        Type = "string",
                        Enum = new[] { "Celsius", "Fahrenheit" },
                    }
                },
                Required = new[] { "location" },
            },
            new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }),
        };
    }

    // The function implementation. It always returns 31 for now.
    static public Weather GetWeather(string location, string unit)
    {
        return new Weather() { Temperature = 31, Unit = unit };
    }
}

// Argument for the function
public class WeatherInput
{
    public string Location { get; set; } = string.Empty;
    public string Unit { get; set; } = "Celsius";
}

// Return type
public class Weather
{
    public int Temperature { get; set; }
    public string Unit { get; set; } = "Celsius";
}
```
``GetCapitalFunction.cs``
```csharp
public class GetCapitalFunction
{
    static public string Name = "get_capital";

    // Return the function metadata
    static public FunctionDefinition GetFunctionDefinition()
    {
        return new FunctionDefinition()
        {
            Name = Name,
            Description = "Get the capital of the location",
            Parameters = BinaryData.FromObjectAsJson(
            new
            {
                Type = "object",
                Properties = new
                {
                    Location = new
                    {
                        Type = "string",
                        Description = "The city, state or country, e.g. San Francisco, CA",
                    }
                },
                Required = new[] { "location" },
            },
            new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }),
        };
    }

    // The function implementation. It always return Tokyo for now.
    static public string GetCapital(string location)
    {
        return "Tokyo";
    }
}

// Argument for the function
public class CapitalInput
{
    public string Location { get; set; } = string.Empty;
}
```
Then I register them in Program.cs

```csharp
Uri openAIUri = new("https://<your account>.openai.azure.com/");
string openAIApiKey = "<your key>";
string model = "gpt-35-turbo-16k";

// Instantiate OpenAIClient for Azure Open AI.
OpenAIClient client = new(openAIUri, new AzureKeyCredential(openAIApiKey));

ChatCompletionsOptions chatCompletionsOptions = new();
ChatCompletions response;
ChatChoice responseChoice;

// Add function definitions
FunctionDefinition getWeatherFuntionDefinition = GetWeatherFunction.GetFunctionDefinition();
FunctionDefinition getCapitalFuntionDefinition = GetCapitalFunction.GetFunctionDefinition();
chatCompletionsOptions.Functions.Add(getWeatherFuntionDefinition);
chatCompletionsOptions.Functions.Add(getCapitalFuntionDefinition);
```
I set user question like below.

```csharp
string question = "What's the weather in the capital city of Japan?";
chatCompletionsOptions.Messages.Add(new(ChatRole.User, question));
```

Then I call the Completion in a loop to see the finish reason is function or stop.

- If the finish reason is function call, then 
     - Get arguments value
     - Call the function
- Register responses and results to ``chatCompletionsOptions.Messages``.
- Call LLM again with the history.

```csharp
while (responseChoice.FinishReason == CompletionsFinishReason.FunctionCall)
{
    // Add message as a history.
    chatCompletionsOptions.Messages.Add(responseChoice.Message);

    if (responseChoice.Message.FunctionCall.Name == GetWeatherFunction.Name)
    {
        string unvalidatedArguments = responseChoice.Message.FunctionCall.Arguments;
        WeatherInput input = JsonSerializer.Deserialize<WeatherInput>(unvalidatedArguments,
                new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase })!;
        var functionResultData = GetWeatherFunction.GetWeather(input.Location, input.Unit);
        var functionResponseMessage = new ChatMessage(
            ChatRole.Function,
            JsonSerializer.Serialize(
                functionResultData,
                new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }));
        functionResponseMessage.Name = GetWeatherFunction.Name;
        chatCompletionsOptions.Messages.Add(functionResponseMessage);
    }
    else if (responseChoice.Message.FunctionCall.Name == GetCapitalFunction.Name)
    {
        string unvalidatedArguments = responseChoice.Message.FunctionCall.Arguments;
        CapitalInput input = JsonSerializer.Deserialize<CapitalInput>(unvalidatedArguments,
                new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase })!;
        var functionResultData = GetCapitalFunction.GetCapital(input.Location);
        var functionResponseMessage = new ChatMessage(
            ChatRole.Function,
            JsonSerializer.Serialize(
                functionResultData,
                new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }));
        functionResponseMessage.Name = GetCapitalFunction.Name;
        chatCompletionsOptions.Messages.Add(functionResponseMessage);
    }

    // Call LLM again to generate the response.
    response =
        await client.GetChatCompletionsAsync(
            model,
            chatCompletionsOptions);

    responseChoice = response.Choices[0];
}

Console.WriteLine(responseChoice.Message.Content);
```

## Result

1. I sent ``"What's the weather in the capital city of Japan?"`` to LLM.
1. LLM returns ``CompletionsFinishReason.FunctionCall`` to use ``get_capital``.
1. Send back the message to LLM again with function result (Tokyo).
1. LLM returns ``CompletionsFinishReason.FunctionCall`` to use ``get_current_weather``.
1. Send back the message to LLM again with function result (31 degree.)
1. LLM returns final response saying: ``"The current weather in the capital city of Japan, Tokyo, is 31 degrees Celsius."``.

## Conclusion

I see that LLM can chain functions depending on the user input. This behavior is similar with Semantic Kernel Planner, so I will compare them when I have time.