Repository: modelscope/evalscope
Issue: TTFT测试结果不对劲，我完成50个请求的总时间才30多秒，TTFT也是30多秒，什么情况
Root cause: Documentation Deficiency

## 问题描述 / Issue Description
TTFT测试结果不对劲，我完成50个请求的总时间才30多秒，TTFT也是30多秒，什么情况

请简要描述您遇到的问题。 / Please briefly describe the issue you encountered.

## 使用的工具 / Tools Used
- [ ] Perf / 模型推理压测工具



## 执行的代码或指令 / Code or Commands Executed

evalscope perf  --parallel 50  --url http://10.0.018:290/v1/chat/completions  --model qwen  --log-every-n-query 5  --connect-timeout 6000  --read-timeout 6000  --api openai  --prompt '写一个科幻小说，不少于2000字，请开始你的表演'  -n 50

## 错误日志 / Error Log

Processing: 50it [00:32,  1.56it/s]
2025-03-21 10:38:49,471 - evalscope - INFO - 
Benchmarking summary:
+-----------------------------------+------------------------------------------------+
| Key                               | Value                                          |
+===================================+================================================+
| Time taken for tests (s)          | 32.078                                         |
+-----------------------------------+------------------------------------------------+
| Number of concurrency             | 50                                             |
+-----------------------------------+------------------------------------------------+
| Total requests                    | 50                                             |
+-----------------------------------+------------------------------------------------+
| Succeed requests                  | 50                                             |
+-----------------------------------+------------------------------------------------+
| Failed requests                   | 0                                              |
+-----------------------------------+------------------------------------------------+
| Throughput(average tokens/s)      | 1517.456                                       |
+-----------------------------------+------------------------------------------------+
| Average QPS                       | 1.559                                          |
+-----------------------------------+------------------------------------------------+
| Average latency (s)               | 30.117                                         |
+-----------------------------------+------------------------------------------------+
| Average time to first token (s)   | 30.117                                         |
+-----------------------------------+------------------------------------------------+
| Average time per output token (s) | 0.031                                          |
+-----------------------------------+------------------------------------------------+
| Average input tokens per request  | 44.0                                           |
+-----------------------------------+------------------------------------------------+
| Average output tokens per request | 973.54                                         |
+-----------------------------------+------------------------------------------------+
| Average package latency (s)       | 30.117                                         |
+-----------------------------------+------------------------------------------------+
| Average package per request       | 1.0                                            |
+-----------------------------------+------------------------------------------------+
| Expected number of requests       | 50                                             |
+-----------------------------------+------------------------------------------------+
| Result DB path                    | outputs/20250321_103817/qwen/benchmark_data.db |
+-----------------------------------+------------------------------------------------+
2025-03-21 10:38:49,472 - evalscope - INFO - 
Percentile results:
+------------+----------+----------+-------------+--------------+---------------+----------------------+
| Percentile | TTFT (s) | TPOT (s) | Latency (s) | Input tokens | Output tokens | Throughput(tokens/s) |
+------------+----------+----------+-------------+--------------+---------------+----------------------+
|    10%     | 28.7033  |   nan    |   28.7033   |      44      |      921      |       32.0822        |
|    25%     | 28.7061  |   nan    |   28.7061   |      44      |      921      |       32.0838        |
|    50%     | 28.7082  |   nan    |   28.7082   |      44      |      921      |       32.1314        |
|    66%     |  31.693  |   nan    |   31.693    |      44      |     1032      |       32.5598        |
|    75%     | 31.6936  |   nan    |   31.6936   |      44      |     1032      |       32.5616        |
|    80%     | 31.6949  |   nan    |   31.6949   |      44      |     1032      |       32.5618        |
|    90%     | 31.6957  |   nan    |   31.6957   |      44      |     1032      |        32.563        |
|    95%     | 31.6961  |   nan    |   31.6961   |      44      |     1032      |       32.5636        |
|    98%     |  31.744  |   nan    |   31.744    |      44      |     1032      |       32.5641        |
|    99%     |  31.744  |   nan    |   31.744    |      44      |     1032      |       32.5641        |
+------------+----------+----------+-------------+--------------+---------------+----------------------+


- 操作系统 / Operating System:
  - [ ] Ubuntu

- Python版本 / Python Version:
  - [ ] 3.10



要准确统计Time to First Token (TTFT)指标，需要在请求中包含`--stream`参数，否则TTFT将与Latency相同。

> 要准确统计Time to First Token (TTFT)指标，需要在请求中包含`--stream`参数，否则TTFT将与Latency相同。

感谢