Repository: promptfoo/promptfoo
Issue: [not sure if bug] - The output is truncated even if token limit not reached
Root cause: Configuration Error

**Describe the bug**
In the UI when expanding the output, the result that is shown is always truncated, even if the maximum of tokens not reached.

My prompt is to parse a TXT file and extract its data into a JSON array. I see how the beginning of the records from the .txt file are correctly extracted into JSON, but there are many different small an tiny deviations in the .txt records, so I need to test the correctness of the parsing mechanism and to see every single row of JSON array in the UI output. Currently it's truncated. Thanks

**To Reproduce**
Steps to reproduce the behavior, including example Promptfoo configurations if applicable:

Use as input quite a long text.
Use a prompt that parses this text
See in the web, UI that the output does not correspond to the input in terms of length

**Expected behavior**
The expectation is that if the original input file has 200 lines in it, the output result result in the UI would also reflect these 200 lines unless the tokens for the output are strictly limited to the smaller number than the input file

**Screenshots**
![image](https://github.com/user-attachments/assets/43a534ad-a62b-4858-9aa3-32bab04bab95)


**System information:**
MAC OS and Chrom as browser

- Promptfoo version:
Version 0.103.3



@kuatroka try clicking table settings, then setting the text length to unlimited. See below

<img width="826" alt="Screenshot 2025-01-03 at 1 10 36 PM" src="https://github.com/user-attachments/assets/f5f4805e-f32a-460e-9006-2125c8a57f3d" />


I should have mentioned that I had already enabled that option, so no luck with this one

![promtptfoo_short_output](https://github.com/user-attachments/assets/13e65dd9-0a89-4227-95ee-b25bef0b6bb7)


and set up the `maxOutputTokens` too

```
providers:
  - id: openrouter:deepseek/deepseek-chat
    config:
      temperature: 0.3
      maxOutputTokens: 128000
```

It looks like the correct config param is `max_tokens`, not `maxOutputTokens`: https://openrouter.ai/docs/requests.  Can you give that a try please?

I tried and this parameter does seem to work, but the output still is not respected, at least not everywhere. I think it might be that the config file does not respect the selection of providers. Let me explain:

I tried an experiment, I ran the prompt with the same context and the same prompt for **deepseek V3** model where you've got about four different providers and therefore four different output token length. I have also tried the model called **minstral- 8B**. In this case, there is only one single provider - **Ministral**.

The output from **Ministral** is indeed as it should be and the length is as it should be, but with **deepseek V3** it's not and I think it might be because the config file does not respect the provider selection. Thanks

providers for **deepseek**
![image](https://github.com/user-attachments/assets/dc1a73fe-2503-48d7-bac1-bd3b18a6a56d)

providers for **ministral**
![image](https://github.com/user-attachments/assets/ebb7b708-d711-4581-9585-aebe93dfee37)


**provider portion of my config file**

```
providers:
  - id: openrouter:deepseek/deepseek-chat
    config:
      temperature: 0.3
      max_tokens: 123000
      provider:
        - order: Fireworks
        - ignore: [DeepSeek, DeepInfra, Together]

  - id: openrouter:mistralai/ministral-8b
    config:
      temperature: 0.3
      # maxOutputTokens: 126000
      max_tokens: 123000
      provider:
        - order: Mistral

```

**result of promptfoo output for both models**
![promtptfoo_short_output2](https://github.com/user-attachments/assets/7f64a5a3-0c64-4cca-8e8b-b35471d712bc)


Ah yes, that is exactly the issue, the [`ProviderPreferences`](https://openrouter.ai/docs/requests) selection is not being forwarded to OpenRouter.  Thank you for identifying the issue.  Should be something we can fix in the next release.

#2620 should fix

Hi @kuatroka! We've just released [0.103.9](https://github.com/promptfoo/promptfoo/releases/tag/0.103.9), which includes a fix to correctly forward provider selection settings to OpenRouter. Please update and let us know if it addresses your truncated output issue. If you have any more questions or run into any other problems, feel free to reopen or start a new issue.

Thank @mldangelo but it doesn't seem like the issue is solved yet and it should be reopen. 

my settings was
```
providers:
  - id: openrouter:deepseek/deepseek-chat
    config:
      temperature: 0.2
      max_tokens: 123000
      provider: {
          "allow_fallbacks": False,
          # "data_collection": "deny",
          "order": ["Fireworks", "DeepInfra"],
          "ignore": ["DeepSeek", "Together"],
        }
```

and the output was still  cut short to 
```
...
{"cik": "0001126328", "cusip": "0150080", "title_of_class": "COM", "rdate": "20050930", "fdate": "20051114", "name_of_issuer": "TULLOW OIL PLC", "value": 12595, "shares": 2740443},
{"cik": "0001126328", "cusip": "89989610", "title_of_class": "COM", "rdate": "20050930", "fdate": "20051114", "name_of_issuer": "TUP

Tokens: 7,355 (4,450+2,905)
Latency: 173,520 ms
...
```

I tested it with **ministral-8B**, it does return full data as it should be. 

To debug it better, is there a way to see what provider was used to generate the output?