Repository: modelscope/evalscope
Issue: 压测多模态模型多并发有问题
Root cause: Resource Mishandling

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [x] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [x] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [x] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题

## 问题描述

压测多模态模型，1并发是正常的，多并发时报错，2并发/5并发都会出错

## EvalScope 版本（必填）
v0.16.0

## 使用的工具
- [ ] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [x] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令

evalscope perf \
  --parallel 1 10 50 100 200 300 \
  --number 10 20 150 300 600 900 \
  --model Qwen2.5-VL-7B-Instruct \
  --url xxxxxx/v1/chat/completions \
  --api openai \
  --api-key xxxxx \
  --dataset flickr8k \
  --prefix-length 0 \
  --max-tokens 1024 \
  --min-tokens 1024 \
  --prefix-length 0 \
  --min-prompt-length 1024 \
  --max-prompt-length 1024 \
  --extra-args '{"ignore_eos": true}'

## 错误日志

Traceback (most recent call last):
  File "/opt/conda/envs/py_mllm/lib/python3.10/site-packages/evalscope/perf/utils/handler.py", line 17, in async_wrapper
    return await func(*args, **kwargs)
  File "/opt/conda/envs/py_mllm/lib/python3.10/site-packages/evalscope/perf/benchmark.py", line 131, in statistic_benchmark_metric
    metrics.update_metrics(benchmark_data, api_plugin)
  File "/opt/conda/envs/py_mllm/lib/python3.10/site-packages/evalscope/perf/utils/benchmark_util.py", line 109, in update_metrics
    benchmark_data._calculate_tokens(api_plugin)
  File "/opt/conda/envs/py_mllm/lib/python3.10/site-packages/evalscope/perf/utils/benchmark_util.py", line 45, in _calculate_tokens
    api_plugin.parse_responses(self.response_messages, request=self.request)
  File "/opt/conda/envs/py_mllm/lib/python3.10/site-packages/evalscope/perf/plugin/api/openai_api.py", line 121, in parse_responses
    self.__process_no_object(js, delta_contents)
  File "/opt/conda/envs/py_mllm/lib/python3.10/site-packages/evalscope/perf/plugin/api/openai_api.py", line 144, in __process_no_object
    for choice in js['choices']:



2025-06-11 20:34:57,678 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-6234' coro=<TCPConnector._resolve_host_with_throttle() done, defined at /opt/conda/envs/py_mllm/lib/python3.10/site-packages/aiohttp/connector.py:1005> wait_for=<Future cancelled> cb=[set.discard(), shield.<locals>._inner_done_callback() at /opt/conda/envs/py_mllm/lib/python3.10/asyncio/tasks.py:847]>
2025-06-11 20:34:57,678 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-6236' coro=<TCPConnector._resolve_host_with_throttle() done, defined at /opt/conda/envs/py_mllm/lib/python3.10/site-packages/aiohttp/connector.py:1005> wait_for=<Future cancelled> cb=[set.discard(), shield.<locals>._inner_done_callback() at /opt/conda/envs/py_mllm/lib/python3.10/asyncio/tasks.py:847]>
2025-06-11 20:34:57,678 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-6237' coro=<TCPConnector._resolve_host_with_throttle() done, defined at /opt/conda/envs/py_mllm/lib/python3.10/site-packages/aiohttp/connector.py:1005> wait_for=<Future cancelled> cb=[set.discard(), shield.<locals>._inner_done_callback() at /opt/conda/envs/py_mllm/lib/python3.10/asyncio/tasks.py:847]>

## 运行环境

- 操作系统：ubuntu 22
- Python版本：python3.10

## 其他信息

如果有其他相关信息，请在此处提供。


看报错是测试的模型服务返回不是标准openai api格式，解析出错了

> 看报错是测试的模型服务返回不是标准openai api格式，解析出错了

服务是vllm部署的，是标准openai格式的，单并发是可以的，完成了测试，多并发的时候就出问题了

我这边无法复现这个问题

![Image](https://github.com/user-attachments/assets/62922262-ab33-4136-b28d-32613ce6259e)

> 感谢你的反馈！我们将关闭此问题。如果您有任何疑问，请随时重新打开它。如果EvalScope对您有所帮助，欢迎给我们点个STAR以示支持，谢谢！

遇到了相同的问题，使用sglang部署模型，单并发可以压测，多并发出现相同错误。请问你后来有解决这个问题吗？