Repository: modelscope/evalscope
Issue: 模型推理性能压测时提示 ZeroDivisionError: float division by zero
Root cause: Validation Gap

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [x] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [x] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [x] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题。

有看到有人提了一个相同报错信息的issue（[加了tokenizer-path参数，报错ZeroDivisionError: float division by zero #612](https://github.com/modelscope/evalscope/issues/612)），但是他的场景可能和我的不一样，并且在他的issue中没有看到明确的解决方案。

## 问题描述

对模型进行推理性能测试时，频繁出现 `ZeroDivisionError: float division by zero` 报错，详细日志请见下面的错误日志


## EvalScope 版本（必填）
0.16.1

## 使用的工具
- [ ] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [x] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令

```python
from evalscope.perf.main import run_perf_benchmark
from evalscope.perf.arguments import Arguments

task_cfg = Arguments(
    parallel=[1],
    number=[50],
    model=model,
    url=url,
    tokenizer_path="/deepseek-r1/DeepSeek-R1-bf16-hfd-w8a8",
    api='openai',
    dataset='random',
    min_tokens=2048,
    max_tokens=2048,
    min_prompt_length=2048,
    max_prompt_length=2048,
)

result = run_perf_benchmark(task_cfg)
```

## 错误日志

```
Processing:  58%|█████▊    | 29/50 [07:23<04:04, 11.63s/it]2025-07-03 04:09:27,218 - evalscope - INFO - {
  "Time taken for tests (s)": 460.9237,
  "Number of concurrency": 1,
  "Total requests": 30,
  "Succeed requests": 30,
  "Failed requests": 0,
  "Output token throughput (tok/s)": 13.909,
  "Total token throughput (tok/s)": 82.0852,
  "Request throughput (req/s)": 0.0651,
  "Average latency (s)": 15.3633,
  "Average time to first token (s)": 0.3347,
  "Average time per output token (s)": 0.0707,
  "Average input tokens per request": 1047.4667,
  "Average output tokens per request": 213.7,
  "Average package latency (s)": 0.071,
  "Average package per request": 211.7
}
Processing:  70%|███████   | 35/50 [09:13<03:57, 15.82s/it]
2025-07-03 04:11:00,053 - evalscope - ERROR - Exception in async function 'statistic_benchmark_metric': float division by zero
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py", line 17, in async_wrapper
    return await func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/benchmark.py", line 127, in statistic_benchmark_metric
    metrics.update_metrics(benchmark_data, api_plugin)
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/benchmark_util.py", line 113, in update_metrics
    benchmark_data._calculate_query_stream_metric()
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/benchmark_util.py", line 41, in _calculate_query_stream_metric
    self.time_per_output_token = self.n_chunks_time / self.n_chunks
ZeroDivisionError: float division by zero
2025-07-03 04:11:00,242 - asyncio - ERROR - Task exception was never retrieved
future: <Task finished name='Task-8' coro=<statistic_benchmark_metric() done, defined at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:14> exception=SystemExit(1)>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py", line 17, in async_wrapper
    return await func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/benchmark.py", line 127, in statistic_benchmark_metric
    metrics.update_metrics(benchmark_data, api_plugin)
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/benchmark_util.py", line 113, in update_metrics
    benchmark_data._calculate_query_stream_metric()
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/benchmark_util.py", line 41, in _calculate_query_stream_metric
    self.time_per_output_token = self.n_chunks_time / self.n_chunks
ZeroDivisionError: float division by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/benchmark/llm-benchmark.py", line 464, in mindie_bench
    result = evalscope_bench(bench_conf)
  File "/data/benchmark/llm-benchmark.py", line 398, in evalscope_bench
    result = run_perf_benchmark(task_cfg)
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/main.py", line 94, in run_perf_benchmark
    return run_one_benchmark(args, output_path=output_path)
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/main.py", line 41, in run_one_benchmark
    return loop.run_until_complete(benchmark(args))
  File "/usr/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
    self._run_once()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 1909, in _run_once
    handle._run()
  File "/usr/lib/python3.10/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py", line 20, in async_wrapper
    sys.exit(1)
SystemExit: 1
2025-07-03 04:11:00,244 - asyncio - ERROR - Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0xfffd513794e0>
2025-07-03 04:11:00,247 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-45' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[BaseSelectorEventLoop._sock_write_done(7, handle=<Handle BaseS...4.65', 1025))>)(), Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
2025-07-03 04:11:00,247 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-46' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
2025-07-03 04:11:00,247 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-47' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
2025-07-03 04:11:00,247 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-48' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
2025-07-03 04:11:00,247 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-49' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
2025-07-03 04:11:00,247 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-50' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
2025-07-03 04:11:00,247 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-51' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
2025-07-03 04:11:00,247 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-52' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
2025-07-03 04:11:00,247 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-53' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
2025-07-03 04:11:00,248 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-54' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
2025-07-03 04:11:00,248 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-55' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
2025-07-03 04:11:00,248 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-56' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
2025-07-03 04:11:00,248 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-57' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
2025-07-03 04:11:00,248 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-58' coro=<send_request() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:17> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /usr/lib/python3.10/asyncio/tasks.py:720]>
Exception ignored in: <coroutine object send_request at 0xfffd513a5000>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/benchmark.py", line 76, in send_request
    async with semaphore:
  File "/usr/lib/python3.10/asyncio/locks.py", line 20, in __aexit__
    self.release()
  File "/usr/lib/python3.10/asyncio/locks.py", line 410, in release
    self._wake_up_next()
  File "/usr/lib/python3.10/asyncio/locks.py", line 420, in _wake_up_next
    fut.set_result(True)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 753, in call_soon
    self._check_closed()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
2025-07-03 04:11:00,249 - evalscope - ERROR - Exception in async function 'send_request': coroutine ignored GeneratorExit
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py", line 17, in async_wrapper
    return await func(*args, **kwargs)
RuntimeError: coroutine ignored GeneratorExit
Exception ignored in: <coroutine object send_request at 0xfffd513a4270>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py", line 20, in async_wrapper
    sys.exit(1)
SystemExit: 1
    
```

## 运行环境

操作系统：Ubuntu 22.04.5 LTS
Python版本：3.10.12
显卡：昇腾910B * 8 * 2 （910B 单卡64G显存，共2台8卡服务器）
模型推理引擎：MindIE RC1 （开启了 MTP）
模型：DeepSeek-R1-bf16-hfd-w8a8 671B  (https://modelers.cn/models/State_Cloud/DeepSeek-R1-bf16-hfd-w8a8)

## 其他信息

使用 MindIE RC1 版本部署的模型，直接使用 evalscope 对模型提供的接口进行测试，没有在模型外有任何封装


已经指定了min-tokens，不会出现空的content。可能是模型服务返回格式和标准openai格式不一致，请求错误码不一样，导致失败的请求被解析为成功的请求。

使用在main分支更新后的evalscope测试后，没有出现过同样的报错。
查看最新的 Release 看到这个 fix 已经合并到 [v0.17.0](https://github.com/modelscope/evalscope/releases/tag/v0.17.0) 发布。
谢谢