Repository: promptfoo/promptfoo
Issue: Testing LLM Response After Function Call(s)
Root cause: Documentation Deficiency

First of all, congrats on this project! I'm rather new to Promptfoo, but already really like it.

I've been experimenting with the [function-tools-callback example](https://github.com/promptfoo/promptfoo/blob/main/examples%2Ffunction-tools-callback%2Fpromptfooconfig.js) and am trying to figure out the best way to test the final LLM response, after function call results provided by the callback have been processed by the LLM.

Currently, it seems like the tests primarily focus on whether the correct function is called, and perhaps the arguments passed to it, or the result of the function call callback. However, I'd like to evaluate the LLM's reasoning and output after it has received the function's results. For example, if the function returns weather data, I want to test if the LLM correctly summarizes or acts upon that information in its final response to the user.

Is there a recommended approach or example for how to achieve this kind of testing within Promptfoo?

Thank you very much for your help!

Thanks @planger! Glad you're enjoying Promptfoo. Which provider are you currently using? That would help me provide more specific guidance for your use case.


You've hit on an important topic regarding function tool testing. When testing LLMs with function calls, there are typically two things you want to evaluate:

1. Did the LLM correctly identify when to call a function (and with what arguments)? 
2. After receiving the function results, did the LLM reason about the results correctly to produce a good final response?

You're looking to focus on that second part - testing the LLM's final reasoning after it receives function results. Here are a few approaches you can use:

### 1. For OpenAI Assistants 

The OpenAI Assistants provider already tracks the entire conversation flow, including function calls and the final LLM response. You can directly test the complete output, which includes both function calls and the final reasoning:

```yaml
providers:
  - openai:assistant:your_assistant_id
```

The output format will include both function calls and the final response:
```
[Call function get_weather with arguments {"location":"Seattle"}]

[Function output: {"temperature": 72, "conditions": "sunny"}]

[Assistant] It's currently 72°F and sunny in Seattle. Great weather for...
```

You can test against this entire output, or use a transform to extract just the final response part.

### 2. Using Transforms to Extract the Final Response

For any provider (including the function-tools-callback example), you can use a transform to extract just the final LLM response after function execution:

```yaml
tests:
  - vars:
      message: "What's the weather in Seattle?"
    assert:
      - type: contains
        value: "sunny"
        transform: |
          // Extract just the final LLM response after function calls
          const parts = output.split('\n\n');
          return parts[parts.length - 1];  // Return just the last part
```

### 3. Custom Provider with Two-Stage Evaluation

For more complex scenarios, you can create a [custom JavaScript provider](https://promptfoo.dev/docs/providers/custom-api/) that explicitly:
1. Makes the initial LLM call
2. Processes function calls
3. Makes a second LLM call with the function results
4. Returns the final response for testing

Example structure:
```javascript
// customWeatherProvider.js
module.exports = class WeatherProvider {
  async callApi(prompt) {
    // 1. First LLM call to detect intent
    const initialResponse = await callLLM(prompt);
    
    // 2. Process any function calls
    let functionResults = null;
    if (hasFunctionCall(initialResponse)) {
      functionResults = await executeFunction(initialResponse.functionCall);
    }
    
    // 3. Final LLM call with function results
    const finalResponse = await callLLMWithResults(prompt, functionResults);
    
    // 4. Return just the final response for testing
    return { output: finalResponse };
  }
};
```

For more detailed examples, check out:
- [Function Tool Callbacks Example](https://github.com/promptfoo/promptfoo/tree/main/examples/function-tools-callback)
- [OpenAI Assistants Evaluation Guide](https://promptfoo.dev/docs/guides/evaluate-openai-assistants)
- [Custom Provider Documentation](https://promptfoo.dev/docs/providers/custom-api/)
- [Using Transforms](https://promptfoo.dev/docs/configuration/guide/#transforming-outputs)



@mldangelo, thank you very much for your fast response!

I'm trying approach 2 "Using Transforms to Extract the Final Response" that you recommended. However, the `output` only contains the result of the tool call for me, but not the LLM's response after processing that result.

Here's my minimal example:

```ts
module.exports = /** @type {import('promptfoo').TestSuiteConfig} */ ({
    prompts: [
      'Please sum {{a}} and {{b}} with the function `addNumbers` and give me the resulting number in Spanish.'
    ],
    providers: [
      {
        id: 'openai:gpt-4o',
        config: {
          model: 'gpt-4o',
          temperature: 0,
          tools: [
            {
              type: 'function',
              function: {
                name: 'addNumbers',
                description: 'Add two numbers together',
                parameters: {
                  type: 'object',
                  properties: {
                    a: { type: 'number' },
                    b: { type: 'number' },
                  },
                  required: ['a', 'b'],
                },
              },
            },
          ],
          tool_choice: 'auto',
          functionToolCallbacks: {
            addNumbers: (parametersJsonString) => {
              const { a, b } = JSON.parse(parametersJsonString);
              return JSON.stringify(a + b);
            },
          },
        },
      },
    ],
    tests: [
      {
        vars: { a: 5, b: 6 },
        assert: [
          {
            type: 'contains',
            transform: 'file://transform.js:actualResponse',
            value: 'once',
          },
        ],
      },
    ],
  });
```

Next to the `promptfooconfig.ts` I have a `transform.js` with the following content:

```js
module.exports = {
    actualResponse: (output, context) => {
        console.log('Output', output);
        console.log('Context', context);
        const parts = output.split('\n\n');
        return parts[parts.length - 1];
    },
  };
```

When I print the `output` variable to the console, I see that it only contains `11`:
```
Output 11
Context {
  vars: { a: 5, b: 6 },
  prompt: {
    label: 'Please sum 5 and 6 with the function `addNumbers` and give me the resulting number in Spanish.'
  }
}
```

When running with `--verbose`, I can see the OpenAI chat completion API response contains the function call but no content:
```
OpenAI chat completions API response: {
  "choices": [{
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "type": "function",
        "function": {
          "name": "addNumbers",
          "arguments": "{\"a\":5,\"b\":6}"
        }
      }]
    },
    "finish_reason": "tool_calls"
  }]
}
```

Am I doing something wrong? It seems like promptfoo is only returning the tool call result (11) and not completing the second part of the LLM response where the LLM would respond in Spanish as requested in the prompt.

Thank you very much in advance!

@mldangelo I'd like to follow up on my message above. Do you have any feedback how we can define assertions on the LLM response after tool calls with registered tool callbacks?

This would help us a lot. Thank you!

@planger did you make progress here? Currently doing an eval with this same need.

I _think_ forking the provider is the right option here but @mldangelo it would be really useful to be able to set config at the provider level to enable N turns, i.e. set a function call depth allowable, outside of the assistants provider implementation.

The eval I'm writing in https://github.com/flexpa/llm-fhir-eval needs this in order to be able to prove that some benchmarks are improved by N turns of tool use

@jdjkelly I didn't look at it for a few weeks, but as far as I remember, I was able to evaluate the LLM response after a tool call by specifying the prompt with raw function messages explicitly, like below. At least this worked with OpenAI models:

```json
{
  "prompt": [
    {
      "role": "system",
      "content": "<systemMessage>"
    },
    ...
    {
      "role": "function",
      "name": "<functionName>",
      "content": "<toolresult>"
    }
  ]
}
```

I guess, it can also be implemented with a custom provider.

My main use case was having tests that validate whether the right tool calls happened, and evaluate the response after the tool calls happened. With the approach above, I had to split those into separate tests, but it did the job.

However, on the long run more native support in Promptfoo would be much appreciated.

@planger @jdjkelly - thanks for the discussion here and for sharing your workarounds.

To summarize the current approaches that work:

1. **Raw function messages in prompt** - as @planger found, you can structure your prompt with explicit function message roles to test the LLM's response after receiving tool results
2. **Custom provider** - wrap the LLM calls to control the full conversation flow and return whatever output you want to assert against
3. **OpenAI Assistants provider** - returns the full conversation including tool calls and final response

Closing this since the original question has been answered. The feedback about wanting more native support for multi-turn tool call testing is noted - if someone wants to open a focused feature request for something like a `toolCallDepth` config option or similar, that would be a reasonable enhancement to track separately.