Repository: modelscope/evalscope
Issue: evalscope perf    --max-tokens这个参数设置后看结果的输出token不到4096
Root cause: Configuration Error

模型是ollma运行的

进行测试时候为什么设置了--max-tokens=4096，输出token没到4096，这个参数生效吗

evalscope perf     --url "http://127.0.0.1:11434/v1/chat/completions"     --parallel 1     --model deepseek-r1:7b       --number 10     --api openai    --max-tokens=4096   --dataset longalpaca     --stream

2025-04-16 17:36:47,036 - evalscope - INFO - 
Benchmarking summary:
+-----------------------------------+----------------------------------------------------------+
| Key                               | Value                                                    |
+===================================+==========================================================+
| Time taken for tests (s)          | 193.794                                                  |
+-----------------------------------+----------------------------------------------------------+
| Number of concurrency             | 1                                                        |
+-----------------------------------+----------------------------------------------------------+
| Total requests                    | 10                                                       |
+-----------------------------------+----------------------------------------------------------+
| Succeed requests                  | 10                                                       |
+-----------------------------------+----------------------------------------------------------+
| Failed requests                   | 0                                                        |
+-----------------------------------+----------------------------------------------------------+
| Throughput(average tokens/s)      | 47.649                                                   |
+-----------------------------------+----------------------------------------------------------+
| Average QPS                       | 0.052                                                    |
+-----------------------------------+----------------------------------------------------------+
| Average latency (s)               | 19.379                                                   |
+-----------------------------------+----------------------------------------------------------+
| Average time to first token (s)   | 1.245                                                    |
+-----------------------------------+----------------------------------------------------------+
| Average time per output token (s) | 0.021                                                    |
+-----------------------------------+----------------------------------------------------------+
| Average input tokens per request  | 2048.0                                                   |
+-----------------------------------+----------------------------------------------------------+
| Average output tokens per request | 923.4                                                    |
+-----------------------------------+----------------------------------------------------------+
| Average package latency (s)       | 0.02                                                     |
+-----------------------------------+----------------------------------------------------------+
| Average package per request       | 922.4                                                    |
+-----------------------------------+----------------------------------------------------------+
| Expected number of requests       | 10                                                       |
+-----------------------------------+----------------------------------------------------------+
| Result DB path                    | outputs/20250416_173325/deepseek-r1_7b/benchmark_data.db |
+-----------------------------------+----------------------------------------------------------+
2025-04-16 17:36:47,044 - evalscope - INFO - 
Percentile results:
+------------+----------+----------+-------------+--------------+---------------+----------------------+
| Percentile | TTFT (s) | TPOT (s) | Latency (s) | Input tokens | Output tokens | Throughput(tokens/s) |
+------------+----------+----------+-------------+--------------+---------------+----------------------+
|    10%     |  1.2207  |  0.0194  |   14.8311   |     2048     |      698      |       46.9297        |
|    25%     |  1.2302  |  0.0195  |   16.2797   |     2048     |      764      |       47.0634        |
|    50%     |  1.2509  |  0.0196  |   19.6566   |     2048     |      937      |        47.681        |
|    66%     |  1.2511  |  0.0197  |   19.7144   |     2048     |      940      |       47.7507        |
|    75%     |  1.254   |  0.0197  |   21.1096   |     2048     |     1008      |       47.8778        |
|    80%     |  1.2584  |  0.0197  |   23.0743   |     2048     |     1108      |       48.0189        |
|    90%     |  1.2754  |  0.0198  |   28.6623   |     2048     |     1397      |        48.74         |
|    95%     |  1.2754  |   0.02   |   28.6623   |     2048     |     1397      |        48.74         |
|    98%     |  1.2754  |  0.0201  |   28.6623   |     2048     |     1397      |        48.74         |
|    99%     |  1.2754  |  0.0203  |   28.6623   |     2048     |     1397      |        48.74         |
+------------+----------+----------+-------------+--------------+---------------+----------------------+

重新编辑了下格式，这个--max-token设置了4096，但是output token上至于1k多
```
 Benchmarking summary:
+-----------------------------------+----------------------------------------------------------+
| Key                               | Value                                                    |
+===================================+==========================================================+
| Time taken for tests (s)          | 193.794                                                  |
+-----------------------------------+----------------------------------------------------------+
| Number of concurrency             | 1                                                        |
+-----------------------------------+----------------------------------------------------------+
| Total requests                    | 10                                                       |
+-----------------------------------+----------------------------------------------------------+
| Succeed requests                  | 10                                                       |
+-----------------------------------+----------------------------------------------------------+
| Failed requests                   | 0                                                        |
+-----------------------------------+----------------------------------------------------------+
| Throughput(average tokens/s)      | 47.649                                                   |
+-----------------------------------+----------------------------------------------------------+
| Average QPS                       | 0.052                                                    |
+-----------------------------------+----------------------------------------------------------+
| Average latency (s)               | 19.379                                                   |
+-----------------------------------+----------------------------------------------------------+
| Average time to first token (s)   | 1.245                                                    |
+-----------------------------------+----------------------------------------------------------+
| Average time per output token (s) | 0.021                                                    |
+-----------------------------------+----------------------------------------------------------+
| Average input tokens per request  | 2048.0                                                   |
+-----------------------------------+----------------------------------------------------------+
| Average output tokens per request | 923.4                                                    |
+-----------------------------------+----------------------------------------------------------+
| Average package latency (s)       | 0.02                                                     |
+-----------------------------------+----------------------------------------------------------+
| Average package per request       | 922.4                                                    |
+-----------------------------------+----------------------------------------------------------+
| Expected number of requests       | 10                                                       |
+-----------------------------------+----------------------------------------------------------+
| Result DB path                    | outputs/20250416_173325/deepseek-r1_7b/benchmark_data.db |
+-----------------------------------+----------------------------------------------------------+
2025-04-16 17:36:47,044 - evalscope - INFO - 
Percentile results:
+------------+----------+----------+-------------+--------------+---------------+----------------------+
| Percentile | TTFT (s) | TPOT (s) | Latency (s) | Input tokens | Output tokens | Throughput(tokens/s) |
+------------+----------+----------+-------------+--------------+---------------+----------------------+
|    10%     |  1.2207  |  0.0194  |   14.8311   |     2048     |      698      |       46.9297        |
|    25%     |  1.2302  |  0.0195  |   16.2797   |     2048     |      764      |       47.0634        |
|    50%     |  1.2509  |  0.0196  |   19.6566   |     2048     |      937      |        47.681        |
|    66%     |  1.2511  |  0.0197  |   19.7144   |     2048     |      940      |       47.7507        |
|    75%     |  1.254   |  0.0197  |   21.1096   |     2048     |     1008      |       47.8778        |
|    80%     |  1.2584  |  0.0197  |   23.0743   |     2048     |     1108      |       48.0189        |
|    90%     |  1.2754  |  0.0198  |   28.6623   |     2048     |     1397      |        48.74         |
|    95%     |  1.2754  |   0.02   |   28.6623   |     2048     |     1397      |        48.74         |
|    98%     |  1.2754  |  0.0201  |   28.6623   |     2048     |     1397      |        48.74         |
|    99%     |  1.2754  |  0.0203  |   28.6623   |     2048     |     1397      |        48.74         |
+------------+----------+----------+-------------+--------------+---------------+----------------------+``
```

我是vllm部署的服务来评测，`--max-tokens` 设置不会生效甚至会和`--min_tokens`冲突

查看了下是0.13.2版本实际在请求体里传递的是`max_completion_tokens`，这个会被vllm忽略

所以我的解决方案是修改下下面的文件：

https://github.com/modelscope/evalscope/blob/main/evalscope/perf/plugin/api/openai_api.py

把73行改成max_tokens（不过这个问题官方0.14.0已经修复了）



@ekochen 应该是ollama的服务不支持max-tokens这个参数

> 感谢你的反馈！我们将关闭此问题。如果您有任何疑问，请随时重新打开它。如果EvalScope对您有所帮助，欢迎给我们点个STAR以示支持，谢谢！