Repository: modelscope/evalscope
Issue: 版本bug
Root cause: Documentation Deficiency

## 问题描述 / Issue Description
最新版本和v0.13.0版本测试结果不一致。在输入和输出相差不大的条件下，Average time to first token (s)测试结果差异极大。

请简要描述您遇到的问题。 / Please briefly describe the issue you encountered.

用例1：
evalscope perf \
--model deepseek_r1 \
--url http://127.0.0.1/:9112/v1/chat/completions \
--api openai \
--dataset random \
--min-tokens 800 \
--max-tokens 800 \
--prefix-length 16 \
--min-prompt-length 16 \
--max-prompt-length 16 \
--tokenizer-path /root/models/DeepSeek-R1-Distill-Qwen-1.5B \
--number 200 \
--parallel 10
测试结果
+-----------------------------------+-------------------------------------------------------+
| Key                               | Value                                                 |
+===================================+=======================================================+
| Time taken for tests (s)          | 157.144                                               |
+-----------------------------------+-------------------------------------------------------+
| Number of concurrency             | 10                                                    |
+-----------------------------------+-------------------------------------------------------+
| Total requests                    | 200                                                   |
+-----------------------------------+-------------------------------------------------------+
| Succeed requests                  | 200                                                   |
+-----------------------------------+-------------------------------------------------------+
| Failed requests                   | 0                                                     |
+-----------------------------------+-------------------------------------------------------+
| Throughput(average tokens/s)      | 1018.173                                              |
+-----------------------------------+-------------------------------------------------------+
| Average QPS                       | 1.273                                                 |
+-----------------------------------+-------------------------------------------------------+
| Average latency (s)               | 7.856                                                 |
+-----------------------------------+-------------------------------------------------------+
| Average time to first token (s)   | 7.856                                                 |
+-----------------------------------+-------------------------------------------------------+
| Average time per output token (s) | 0.01                                                  |
+-----------------------------------+-------------------------------------------------------+
| Average input tokens per request  | 33.5                                                  |
+-----------------------------------+-------------------------------------------------------+
| Average output tokens per request | 800.0                                                 |
+-----------------------------------+-------------------------------------------------------+
| Average package latency (s)       | 7.856                                                 |
+-----------------------------------+-------------------------------------------------------+
| Average package per request       | 1.0                                                   |
+-----------------------------------+-------------------------------------------------------+
| Expected number of requests       | 200                                                   |
+-----------------------------------+-------------------------------------------------------+
| Result DB path                    | outputs/20250320_152727/deepseek_r1/benchmark_data.db |
+-----------------------------------+-------------------------------------------------------+

用例2
evalscope perf \
--url "http://127.0.0.1/:9112/v1/chat/completions" \
--model deepseek_r1 \
--api openai \
--dataset openqa \
--stream \
--number 200 \
--parallel 10
测试结果
+-----------------------------------+-------------------------------------------------------+
| Key                               | Value                                                 |
+===================================+=======================================================+
| Time taken for tests (s)          | 183.77                                                |
+-----------------------------------+-------------------------------------------------------+
| Number of concurrency             | 10                                                    |
+-----------------------------------+-------------------------------------------------------+
| Total requests                    | 200                                                   |
+-----------------------------------+-------------------------------------------------------+
| Succeed requests                  | 200                                                   |
+-----------------------------------+-------------------------------------------------------+
| Failed requests                   | 0                                                     |
+-----------------------------------+-------------------------------------------------------+
| Throughput(average tokens/s)      | 987.453                                               |
+-----------------------------------+-------------------------------------------------------+
| Average QPS                       | 1.088                                                 |
+-----------------------------------+-------------------------------------------------------+
| Average latency (s)               | 8.999                                                 |
+-----------------------------------+-------------------------------------------------------+
| Average time to first token (s)   | 0.026                                                 |
+-----------------------------------+-------------------------------------------------------+
| Average time per output token (s) | 0.01                                                  |
+-----------------------------------+-------------------------------------------------------+
| Average input tokens per request  | 26.345                                                |
+-----------------------------------+-------------------------------------------------------+
| Average output tokens per request | 907.32                                                |
+-----------------------------------+-------------------------------------------------------+
| Average package latency (s)       | 0.01                                                  |
+-----------------------------------+-------------------------------------------------------+
| Average package per request       | 907.32                                                |
+-----------------------------------+-------------------------------------------------------+
| Expected number of requests       | 200                                                   |
+-----------------------------------+-------------------------------------------------------+
| Result DB path                    | outputs/20250320_153136/deepseek_r1/benchmark_data.db |

## 使用的工具 / Tools Used
- [ ] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [x] Perf / 模型推理压测工具
- [ ] Arena  /竞技场模式

## 运行环境 / Runtime Environment

- 操作系统 / Operating System:
  - [ ] Windows
  - [ ] macOS
  - [x ] Ubuntu

- Python版本 / Python Version:
  - [x] 3.12



## 其他信息 / Additional Information

如果有其他相关信息，请在此处提供。 / If there is any other relevant information, please provide it here.


你用例1没使用stream，用例2使用了。没使用stream 情况下 TTFT = Latency

> 感谢你的反馈！我们将关闭此问题。如果您有任何疑问，请随时重新打开它。如果EvalScope对您有所帮助，欢迎给我们点个STAR以示支持，谢谢！