Repository: modelscope/evalscope
Issue: 加了tokenizer-path参数，报错ZeroDivisionError: float division by zero
Root cause: Validation Gap

## 问题描述

对自定义封装的openai格式接口进行压测，接口没有返回useage，使用--dataset random并且加上了--tokenizer-path路径报错ZeroDivisionError: float division by zero

## EvalScope 版本（必填）
v0.16.0

## 使用的工具
- [ ] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [x] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令

```shell
evalscope perf \
  --parallel 1 5 \
  --number 5 10 \
  --model Qwen3-14B \
  --url http://xxx.xxx.xxx.xxx/v1/chat/completions \
  --api openai \
  --prompt '帮我写首诗' \
  --query-template '{"messages": [{"role": "user","content": "%p"}], "stream": true}' \
  --max-tokens 400 \
  --prefix-length 500 \
  --min-prompt-length 100 \
  --max-prompt-length 512 \
  --dataset random \
  --tokenizer-path Qwen/Qwen3-14B \
  --debug
```

## 错误日志

2025-05-30 18:32:09,281 - evalscope - http_client.py - _handle_stream - 48 - DEBUG - Response recevied: data: {"id":"123","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant","content":null},"finish_reason":"stop"}],"answer_type":"理解回答","created":1748601129}
2025-05-30 18:32:09,281 - evalscope - http_client.py - _handle_stream - 48 - DEBUG - Response recevied: data: [DONE]
Processing:   0%|                                                                                                                                               | 0/5 [00:01<?, ?it/s]
2025-05-30 18:32:09,282 - evalscope - handler.py - async_wrapper - 19 - ERROR - Exception in async function 'statistic_benchmark_metric': float division by zero
Traceback (most recent call last):
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/handler.py", line 17, in async_wrapper
    return await func(*args, **kwargs)
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/benchmark.py", line 131, in statistic_benchmark_metric
    metrics.update_metrics(benchmark_data, api_plugin)
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/benchmark_util.py", line 113, in update_metrics
    benchmark_data._calculate_query_stream_metric()
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/benchmark_util.py", line 41, in _calculate_query_stream_metric
    self.time_per_output_token = self.n_chunks_time / self.completion_tokens
ZeroDivisionError: float division by zero
2025-05-30 18:32:09,392 - asyncio - ERROR - Task exception was never retrieved
future: <Task finished name='Task-9' coro=<statistic_benchmark_metric() done, defined at /Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/handler.py:14> exception=SystemExit(1)>
Traceback (most recent call last):
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/handler.py", line 17, in async_wrapper
    return await func(*args, **kwargs)
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/benchmark.py", line 131, in statistic_benchmark_metric
    metrics.update_metrics(benchmark_data, api_plugin)
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/benchmark_util.py", line 113, in update_metrics
    benchmark_data._calculate_query_stream_metric()
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/benchmark_util.py", line 41, in _calculate_query_stream_metric
    self.time_per_output_token = self.n_chunks_time / self.completion_tokens
ZeroDivisionError: float division by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
    self.run_forever()
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
    self._run_once()
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/base_events.py", line 1909, in _run_once
    handle._run()
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/handler.py", line 20, in async_wrapper
    sys.exit(1)
SystemExit: 1
2025-05-30 18:32:09,396 - evalscope - handler.py - async_wrapper - 19 - ERROR - Exception in async function 'send_request': Event loop is closed
Traceback (most recent call last):
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/benchmark.py", line 82, in send_request
    async with client:
GeneratorExit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/handler.py", line 17, in async_wrapper
    return await func(*args, **kwargs)
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/benchmark.py", line 80, in send_request
    async with semaphore:
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/locks.py", line 20, in __aexit__
    self.release()
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/locks.py", line 410, in release
    self._wake_up_next()
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/locks.py", line 420, in _wake_up_next
    fut.set_result(True)
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/base_events.py", line 753, in call_soon
    self._check_closed()
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object send_request at 0x12b5abd10>
Traceback (most recent call last):
  File "/Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/handler.py", line 20, in async_wrapper
    sys.exit(1)
SystemExit: 1
2025-05-30 18:32:09,397 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-10' coro=<send_request() done, defined at /Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/handler.py:14> wait_for=<Task pending name='Task-176' coro=<_wait_for_close() running at /Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/aiohttp/connector.py:137> wait_for=<_GatheringFuture finished result=[None]> cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/tasks.py:720]>
2025-05-30 18:32:09,397 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-11' coro=<send_request() done, defined at /Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/handler.py:14> wait_for=<Future finished result=True> cb=[gather.<locals>._done_callback() at /Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/tasks.py:720]>
2025-05-30 18:32:09,397 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-12' coro=<send_request() done, defined at /Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/handler.py:14> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/tasks.py:720]>
2025-05-30 18:32:09,398 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-13' coro=<send_request() done, defined at /Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/handler.py:14> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/tasks.py:720]>
2025-05-30 18:32:09,398 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-14' coro=<send_request() done, defined at /Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/handler.py:14> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/asyncio/tasks.py:720]>
2025-05-30 18:32:09,398 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-176' coro=<_wait_for_close() done, defined at /Users/kerr/.local/share/mamba/envs/evalscope/lib/python3.10/site-packages/aiohttp/connector.py:135> wait_for=<_GatheringFuture finished result=[None]> cb=[Task.task_wakeup()]>


## 运行环境

- 操作系统：
- Python版本：3.10.17




看起来是解析usage方法返回的completion_tokens为0，导致了报错。请问：

1. 控制台中有正确打印模型的输出吗？
2. 自定义封装的openai接口格式是否正确？

> 看起来是解析usage方法返回的completion_tokens为0，导致了报错。请问：
> 
> 1. 控制台中有正确打印模型的输出吗？
> 2. 自定义封装的openai接口格式是否正确？

1、使用evalscope perf命令压测自定义接口可以控制台可以正常流式输出。
2、自定义接口只保留了openai接口中的id、object、created、choices字段，同时在流式输出结束加了[DONE]的输出标识。
控制台接口输出日志类似：
```
2025-06-03 14:13:13,979 - evalscope - http_client.py - _handle_stream - 48 - DEBUG - Response recevied: data: {"id":"123","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant","content":"xxx"},"finish_reason":"stop"}],"created":1748931193}
2025-06-03 14:13:13,979 - evalscope - http_client.py - _handle_stream - 48 - DEBUG - Response recevied: data: [DONE]
Processing:   0%|                                                                                                                                               | 0/5 [00:02<?, ?it/s]
2025-06-03 14:13:13,980 - evalscope - handler.py - async_wrapper - 19 - ERROR - Exception in async function 'statistic_benchmark_metric': float division by zero
```


我这边无法复现你的问题，因为没有你的自定义api接口
1. 你可以尝试拉main分支代码安装，然后debug一下
2. 或者自定义api解析逻辑：https://evalscope.readthedocs.io/zh-cn/latest/user_guides/stress_test/custom.html#api

> 我这边无法复现你的问题，因为没有你的自定义api接口
> 
> 1. 你可以尝试拉main分支代码安装，然后debug一下
> 2. 或者自定义api解析逻辑：https://evalscope.readthedocs.io/zh-cn/latest/user_guides/stress_test/custom.html#api

请问是不是不管是否流式输出，接口必须要有类似下面的字段压测才不会报错？
`"usage":{"prompt_tokens":1,"total_tokens":17,"completion_tokens":16}}`

> > 我这边无法复现你的问题，因为没有你的自定义api接口
> > 
> > 1. 你可以尝试拉main分支代码安装，然后debug一下
> > 2. 或者自定义api解析逻辑：https://evalscope.readthedocs.io/zh-cn/latest/user_guides/stress_test/custom.html#api
> 
> 请问是不是不管是否流式输出，接口必须要有类似下面的字段压测才不会报错？ `"usage":{"prompt_tokens":1,"total_tokens":17,"completion_tokens":16}}`

不是的，在没有usage时，框架会自动解析出content并用tokenizer计算token数量，但接口返回的结构得和openai api结构一致

> > > 我这边无法复现你的问题，因为没有你的自定义api接口
> > > 
> > > 1. 你可以尝试拉main分支代码安装，然后debug一下
> > > 2. 或者自定义api解析逻辑：https://evalscope.readthedocs.io/zh-cn/latest/user_guides/stress_test/custom.html#api
> > 
> > 
> > 请问是不是不管是否流式输出，接口必须要有类似下面的字段压测才不会报错？ `"usage":{"prompt_tokens":1,"total_tokens":17,"completion_tokens":16}}`
> 
> 不是的，在没有usage时，框架会自动解析出content并用tokenizer计算token数量，但接口返回的结构得和openai api结构一致

好的，感谢解答。