Repository: promptfoo/promptfoo
Issue: [Discussion Request] Tool calling as a more reliable grading output mechanic
Root cause: Unimplemented Feature Gap

I tried to search if this has already come up because I thought this is a rather intuitive use case for function / tool calling since we require an exact output JSON and not free form text, but I haven't found anything.

Am I overlooking something? I see how implementing it for the different providers is not as easy as sending a prompt string, but I am getting "Could not extract JSON from llm-rubric response" which, according to the source code, happens when the `extractJsonObjects` fails. I can see why; LLMs are not really made for outputting exact coherent JSON formats from prompts. 

I am confused as to why not more people have this issue, so, maybe I am overlooking something here.

If this is something that would be useful, I'd be open to contributing after talking through.

<!-- linear-linkback -->
<p><a href="https://linear.app/promptfooo/issue/ENG-1194/discussion-request-tool-calling-as-a-more-reliable-grading-output">ENG-1194</a></p>

Hey, thanks for raising this! Which model/provider are you using for grading?

Interestingly, we actually do use tool calling for our default Anthropic llm-rubric provider - it forces a `grade_output` tool call with a `{pass, score, reason}` schema. We've found this works well with Claude models specifically.

For OpenAI and other providers, we use a prompt-based approach that asks the model to output JSON, then extract it from the response. We've generally found that forcing strict output schemas can degrade LLM-as-judge quality for some models, but it varies by provider.

If you're hitting consistent JSON extraction failures, we'd love to understand your use case better:
- Which model/provider are you using?
- Are you using a custom rubric prompt?

Let us know and we can dig in!

Hi, I tested a few different models with openrouter, maybe that was an issue where you just don't use tool calling? I implemented my own provider quick and dirty that sends an openrouter request with my selected model and a tool requiring the pass, score and reason schema.

So if you already use toolcalling for openrouter, than I don't know why it failed to extract JSON error message appeared. If for that provider you don't, maybe we should add that?

Hey @xdivby0! Thanks for the follow-up and for digging into this.

You're right - OpenRouter (and OpenAI) don't use tool calling for grading, which is why you're hitting the JSON extraction issues. As mentioned, Anthropic has a special `AnthropicLlmRubricProvider` that forces a `grade_output` tool call with a strict `{pass, score, reason}` schema, which is why it's more reliable.

**Good news**: you can work around this today without any changes to promptfoo by using a custom Python grader:

```yaml
# promptfooconfig.yaml
defaultTest:
  assert:
    - type: python
      value: file://graders/tool_grader.py
```

```python
# graders/tool_grader.py
import json
import os
import openai

def get_assert(output, context):
    client = openai.OpenAI(
        base_url="https://openrouter.ai/api/v1",
        api_key=os.environ.get("OPENROUTER_API_KEY")
    )

    rubric = context.get("test", {}).get("metadata", {}).get("rubric", "Is the output accurate?")

    response = client.chat.completions.create(
        model="anthropic/claude-sonnet-4",  # or any model that supports tools
        messages=[
            {"role": "system", "content": "Grade the output based on the rubric."},
            {"role": "user", "content": f"Rubric: {rubric}\n\nOutput:\n{output}"}
        ],
        tools=[{
            "type": "function",
            "function": {
                "name": "grade_output",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "pass": {"type": "boolean"},
                        "score": {"type": "number"},
                        "reason": {"type": "string"}
                    },
                    "required": ["pass", "score", "reason"]
                }
            }
        }],
        tool_choice={"type": "function", "function": {"name": "grade_output"}}
    )

    return json.loads(response.choices[0].message.tool_calls[0].function.arguments)
```

This gives you full control over the grading call and works with any OpenRouter model that supports tool calling.

That said, native tool calling support for OpenAI/OpenRouter grading would definitely be a nice improvement. If you're still interested in contributing, we'd welcome a PR that adds an `OpenAiLlmRubricProvider` similar to the Anthropic one - happy to help guide the implementation!

Closing this for now since there's a working solution, but feel free to reopen if you run into any issues with the custom grader approach or want to discuss a PR. 🙏