Repository: modelscope/evalscope
Issue: 性能测试中TPOT指标计算问题
Root cause: Algorithmic Error

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [ ] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [ ] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [ ] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题

## 问题描述
参考 #585 

```
请简要描述您遇到的问题。
在计算TPOT的公式中：
'Average time per output token (s)': round(self.avg_time_per_token, default_ndigits)，

其中，self.avg_time_per_token = self.n_time_per_output_token / self.n_succeed_queries，
self.n_time_per_output_token += benchmark_data.time_per_output_token，
self.time_per_output_token = self.n_chunks_time / self.completion_tokens

最后一个公式中计算单条请求平均TPOT时使用的分子n_chunks_time是每个请求中去掉首尾chunk的时间，而分母completion_tokens是全部的token数，我的疑问是：似乎分母使用n_chunks更合适？这样分子分母都是不包含首尾chunk的。

```

如果一个 chunk 内不是一个 token 的话，那么这个算法就不对了。

应该除以 （self.completion_tokens - 第一个trunck包含的token个数） 。完整表述如下：

在计算TPOT的公式中：
'Average time per output token (s)': round(self.avg_time_per_token, default_ndigits)，

其中，self.avg_time_per_token = self.n_time_per_output_token / self.n_succeed_queries，
self.n_time_per_output_token += benchmark_data.time_per_output_token，
self.time_per_output_token = self.n_chunks_time / （self.completion_tokens -  第一个trunck包含的token个数）

另外，Percentile results的结果是从SQLite数据库取得数据，这里的tpot的算法和summary_result的tpot的算法不一样。也是不合适的，2边应该一致

## EvalScope 版本（必填）
v0.17.0

## 使用的工具
- [ ] Perf / 模型推理压测工具



一个chunk中的token数一般都是一个，v0.17.1版本已修复该问题，计算公式改为：https://github.com/modelscope/evalscope/blob/7ff9c55132bc8127e3a51b2b55e00a703ebd72ff/evalscope/perf/utils/benchmark_util.py#L36-L37

MTP的话，一个chunk 可以包含多个token
```
root@LongTeng-prod:/vllm-workspace# curl -X 'POST' 'https://api.modelarts-maas.com/v1/chat/completions' -H 'accept: application/json' -H 'Content-Type: application/json' -H 'Authorization: Bearer xxxxxx' -d '{
    "model": "deepseek-r1-250528",
    "messages": [{"role":"user", "content":"Which number is larger, 9.11 or 9.8"}],
    "max_tokens": 30,
    "stream": "True",
    "stream_options": {"include_usage": "True"}
}'
data:{"id": "910a224ce4154d479f43196c1a341c90", "object": "chat.completion.chunk", "created": 1753064906, "model": "deepseek-r1-250528", "choices": [{"index": 0, "delta": {"content": null, "reasoning_content": "I"}, "logprobs": null, "finish_reason": null}], "usage": {"prompt_tokens": 23, "total_tokens": 24, "completion_tokens": 1, "prompt_tokens_details": null, "total_delay": 452}, "system_fingerprint": null}

data:{"id": "910a224ce4154d479f43196c1a341c90", "object": "chat.completion.chunk", "created": 1753064906, "model": "deepseek-r1-250528", "choices": [{"index": 0, "delta": {"content": null, "reasoning_content": " need to"}, "logprobs": null, "finish_reason": null}], "usage": {"prompt_tokens": 23, "total_tokens": 26, "completion_tokens": 3, "prompt_tokens_details": null, "total_delay": 559}, "system_fingerprint": null}

data:{"id": "910a224ce4154d479f43196c1a341c90", "object": "chat.completion.chunk", "created": 1753064906, "model": "deepseek-r1-250528", "choices": [{"index": 0, "delta": {"content": null, "reasoning_content": " determine which number is larger between 9"}, "logprobs": null, "finish_reason": null}], "usage": {"prompt_tokens": 23, "total_tokens": 34, "completion_tokens": 11, "prompt_tokens_details": null, "total_delay": 698}, "system_fingerprint": null}

data:{"id": "910a224ce4154d479f43196c1a341c90", "object": "chat.completion.chunk", "created": 1753064906, "model": "deepseek-r1-250528", "choices": [{"index": 0, "delta": {"content": null, "reasoning_content": ".11 and 9.8."}, "logprobs": null, "finish_reason": null}], "usage": {"prompt_tokens": 23, "total_tokens": 42, "completion_tokens": 19, "prompt_tokens_details": null, "total_delay": 837}, "system_fingerprint": null}

data:{"id": "910a224ce4154d479f43196c1a341c90", "object": "chat.completion.chunk", "created": 1753064906, "model": "deepseek-r1-250528", "choices": [{"index": 0, "delta": {"content": null, "reasoning_content": " Both are decimals, so I should"}, "logprobs": null, "finish_reason": null}], "usage": {"prompt_tokens": 23, "total_tokens": 49, "completion_tokens": 26, "prompt_tokens_details": null, "total_delay": 976}, "system_fingerprint": null}

data:{"id": "910a224ce4154d479f43196c1a341c90", "object": "chat.completion.chunk", "created": 1753064906, "model": "deepseek-r1-250528", "choices": [{"index": 0, "delta": {"content": null, "reasoning_content": " compare them digit by"}, "logprobs": null, "finish_reason": null}], "usage": {"prompt_tokens": 23, "total_tokens": 53, "completion_tokens": 30, "prompt_tokens_details": null, "total_delay": 1115}, "system_fingerprint": null}

data:{"id": "910a224ce4154d479f43196c1a341c90", "object": "chat.completion.chunk", "created": 1753064906, "model": "deepseek-r1-250528", "choices": [{"index": 0, "delta": {"content": "", "reasoning_content": null}, "logprobs": null, "finish_reason": "length"}], "usage": {"prompt_tokens": 23, "total_tokens": 53, "completion_tokens": 30, "prompt_tokens_details": null, "total_delay": 1115, "ttft": 451.29, "tpot": 22.88}, "system_fingerprint": null}

data:[DONE]
```

MTP这种算特殊情况，后续需额外支持

v1.1.1 版本之后已修复该问题，即使是MTP指标计算也修正了，使用的是真实的completion_tokens，而不是chunk数量

> 感谢你的反馈！我们将关闭此问题。如果您有任何疑问，请随时重新打开它。如果EvalScope对您有所帮助，欢迎给我们点个STAR以示支持，谢谢！