Repository: promptfoo/promptfoo
Issue: Azure model o1-mini returns error max_tokens is not supported
Root cause: Configuration Error

When I use the azure:chat:o1-mini model with Promptfoo v0.111.0, I get the following error:

> API response error: unsupported_parameter
> Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.

I tried adding parameters via passthrough, but Promptfoo keeps sending max_tokens, but I would like an official solution or a declarative way from .yaml.

**Example .yaml**

> providers:
>   - id: azure:chat:o1-mini
>     config:
>       apiKey: "xxxx"
>       apiBaseUrl: "xxxx"
>       deploymentName: "o1-mini"
>       apiVersion: "2024-02-01"
> 
>   - id: azure:chat:gpt-4o
>     config:
>       apiKey: "xxxxx"
>       apiBaseUrl: "xxxxx"
>       deploymentName: "gpt-4o"
>       apiVersion: "2024-02-01"

Set *isReasoningModel: true* in the config for each provider
https://www.promptfoo.dev/docs/providers/azure/#configuration-1 :)

On Tue, Apr 22, 2025 at 8:07 AM mariana-pineda ***@***.***>
wrote:

> When I use the azure:chat:o1-mini model with Promptfoo v0.111.0, I get the
> following error:
>
> API response error: unsupported_parameter
> Unsupported parameter: 'max_tokens' is not supported with this model. Use
> 'max_completion_tokens' instead.
>
> I tried adding parameters via passthrough, but Promptfoo keeps sending
> max_tokens, but I would like an official solution or a declarative way from
> .yaml.
>
> *Example .yaml*
>
> providers:
>
>    -
>
>    id: azure:chat:o1-mini
>    config:
>    apiKey: "xxxx"
>    apiBaseUrl: "xxxx"
>    deploymentName: "o1-mini"
>    apiVersion: "2024-02-01"
>    -
>
>    id: azure:chat:gpt-4o
>    config:
>    apiKey: "xxxxx"
>    apiBaseUrl: "xxxxx"
>    deploymentName: "gpt-4o"
>    apiVersion: "2024-02-01"
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/promptfoo/promptfoo/issues/3781>, or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AACLYJRKTMPSW724B5ILD4322ZLLVAVCNFSM6AAAAAB3T63H2CVHI2DSMVQWIX3LMV43ASLTON2WKOZTGAYTCMZXG42TCNI>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
> *mariana-pineda* created an issue (promptfoo/promptfoo#3781)
> <https://github.com/promptfoo/promptfoo/issues/3781>
>
> When I use the azure:chat:o1-mini model with Promptfoo v0.111.0, I get the
> following error:
>
> API response error: unsupported_parameter
> Unsupported parameter: 'max_tokens' is not supported with this model. Use
> 'max_completion_tokens' instead.
>
> I tried adding parameters via passthrough, but Promptfoo keeps sending
> max_tokens, but I would like an official solution or a declarative way from
> .yaml.
>
> *Example .yaml*
>
> providers:
>
>    -
>
>    id: azure:chat:o1-mini
>    config:
>    apiKey: "xxxx"
>    apiBaseUrl: "xxxx"
>    deploymentName: "o1-mini"
>    apiVersion: "2024-02-01"
>    -
>
>    id: azure:chat:gpt-4o
>    config:
>    apiKey: "xxxxx"
>    apiBaseUrl: "xxxxx"
>    deploymentName: "gpt-4o"
>    apiVersion: "2024-02-01"
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/promptfoo/promptfoo/issues/3781>, or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AACLYJRKTMPSW724B5ILD4322ZLLVAVCNFSM6AAAAAB3T63H2CVHI2DSMVQWIX3LMV43ASLTON2WKOZTGAYTCMZXG42TCNI>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>


I have added that parameter but now I have the following error. I also tested adding reasoning_effort: 'medium', and the error persists. Btw, I use the promptfoo eval --no-cache command to clear the cache. Any idea of ​​the problem?

> API response error: unknown_parameter Unknown parameter: 'reasoning_effort'.

**Example**

> providers:
>   - id: azure:chat:o1-mini
>     config:
>       apiKey: "xxxx"
>       apiBaseUrl: "xxxx"
>       deploymentName: "o1-mini"
>       apiVersion: "2024-02-01"
>       isReasoningModel: true

Hi @mariana-pineda, the original issue with `max_tokens` has been addressed, and the follow-up `reasoning_effort` error was due to using an old API version.

**Root cause of both errors:**
1. Azure o1 models require `max_completion_tokens` instead of `max_tokens` → Fixed by setting `isReasoningModel: true`
2. When `isReasoningModel` is true, promptfoo sends `reasoning_effort` parameter, but your API version `2024-02-01` didn't support it

**Solution:** Update to a recent API version. The current default is `2024-12-01-preview`, and the docs recommend `2025-04-01-preview` or later for reasoning models:

```yaml
providers:
  - id: azure:chat:o1-mini
    config:
      apiKey: "xxxx"
      apiBaseUrl: "xxxx"
      deploymentName: "o1-mini"
      apiVersion: "2025-04-01-preview"  # Required for reasoning models
      isReasoningModel: true
```

If you don't specify `apiVersion`, promptfoo now defaults to `2024-12-01-preview` which should work. The `isReasoningModel` flag is also auto-detected for deployments with names containing `o1`, `o3`, etc.

See the updated [Azure provider documentation](https://www.promptfoo.dev/docs/providers/azure/#reasoning-models) for complete guidance.