Repository: modelscope/evalscope
Issue: perf on Qwen2.5-VL with flickr8k got error
Root cause: Interface Contract Mismatch

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [x] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [x] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [x] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题

## 问题描述
使用 evalscope 的perf功能压测qwen2.5-vl-3b时报如下错误，使用的是modelscope自动下载的flickr8k数据集，backend是本地基于vllm部署的模型
`CUDA_VISIBLE_DEVICES=0 vllm serve /root/workspace/models/qwen2.5-vl-3b/ \
  --served-model-name chain-vl
  --host 0.0.0.0 \
  --port 5000 \
  --trust-remote-code \
  --max-model-len 4096 \
  --limit-mm-per-prompt '{"image":5}'`


## EvalScope 版本（必填）
v0.17.1

## 使用的工具
- [] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [x] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令
`evalscope perf \
  --parallel 1 4 \
  --model chain-vl \
  --url http://xxx.com/v1/chat/completions \
  --api openai \
  --dataset flickr8k \
  --prefix-length 64 \
  --number 10 40 \
  --tokenizer-path /opt/apps/benchmark/models/Qwen2.5-VL/ \
  --debug \
  --swanlab-api-key 'local' \
  --name 'name_of_swanlab_log'`
请提供您执行的主要代码或指令。

## 错误日志

```
2025-07-28 08:55:07,151 - evalscope - main.py - run_one_benchmark - 31 - INFO - Starting benchmark with args: 
2025-07-28 08:55:07,151 - evalscope - main.py - run_one_benchmark - 32 - INFO - {
    "model": "chain-vl",
    "model_id": "chain-vl",
    "attn_implementation": null,
    "api": "openai",
    "tokenizer_path": "/opt/apps/benchmark/models/Qwen2.5-VL/",
    "port": 8877,
    "url": "http://ide-6f66a2537f-vlm.zjai-bj-kubeone.shizhuang-inc.com/v1/chat/completions",
    "headers": {},
    "connect_timeout": 600,
    "read_timeout": 600,
    "api_key": null,
    "no_test_connection": false,
    "number": 10,
    "parallel": 1,
    "rate": -1,
    "sleep_interval": 5,
    "log_every_n_query": 10,
    "debug": true,
    "wandb_api_key": null,
    "swanlab_api_key": "local",
    "name": "name_of_swanlab_log",
    "outputs_dir": "outputs/20250728_085505/name_of_swanlab_log/parallel_1_number_10",
    "max_prompt_length": 9223372036854775807,
    "min_prompt_length": 0,
    "prefix_length": 64,
    "prompt": null,
    "query_template": null,
    "apply_chat_template": true,
    "image_width": 224,
    "image_height": 224,
    "image_format": "RGB",
    "image_num": 1,
    "dataset": "flickr8k",
    "dataset_path": null,
    "frequency_penalty": null,
    "repetition_penalty": null,
    "logprobs": null,
    "max_tokens": 2048,
    "min_tokens": null,
    "n_choices": null,
    "seed": 0,
    "stop": null,
    "stop_token_ids": null,
    "stream": true,
    "temperature": 0.0,
    "top_p": null,
    "top_k": null,
    "extra_args": {}
}
2025-07-28 08:55:10,610 - evalscope - http_client.py - on_request_start - 65 - DEBUG - Starting request: <TraceRequestStartParams(method='POST', url=URL('http://ide-6f66a2537f-vlm.zjai-bj-kubeone.shizhuang-inc.com/v1/chat/completions'), headers=<CIMultiDict('Content-Type': 'application/json', 'user-agent': 'modelscope_bench')>)>
2025-07-28 08:55:10,614 - evalscope - http_client.py - on_request_chunk_sent - 77 - DEBUG - Request sent: <method='POST',  url=URL('http://ide-6f66a2537f-vlm.zjai-bj-kubeone.shizhuang-inc.com/v1/chat/completions'), truncated_chunk='{"messages": [{"role": "user", "content": "hello"}], "model": "chain-vl", "max_tokens": 2048, "seed": 0, "stream": true, "stream_options": {"include_usage": true}, "temperature": 0.0}'>
2025-07-28 08:55:11,318 - evalscope - http_client.py - test_connection - 110 - INFO - Test connection successful.
2025-07-28 08:55:12,725 - modelscope - ERROR - Couldn't find file at https://www.modelscope.cn/api/v1/datasets/clip-benchmark/wds_flickr8k/repo?Source=SDK&Revision=master&FilePath=README.md&View=False
2025-07-28 08:55:13,944 - modelscope - ERROR - Couldn't find file at https://www.modelscope.cn/api/v1/datasets/clip-benchmark/wds_flickr8k/repo?Source=SDK&Revision=master&FilePath=README.md&View=False
Downloading data: 100%|████████████████████████████████████████████████████████████████████████| 212M/212M [00:51<00:00, 4.12MB/s]
Downloading data: 100%|████████████████████████████████████████████████████████████████████████| 213M/213M [00:12<00:00, 16.8MB/s]
Downloading data: 100%|████████████████████████████████████████████████████████████████████████| 213M/213M [01:25<00:00, 2.49MB/s]
Downloading data: 100%|████████████████████████████████████████████████████████████████████████| 213M/213M [00:12<00:00, 16.6MB/s]
Downloading data: 100%|██████████████████████████████████████████████████████████████████████| 27.5M/27.5M [00:09<00:00, 2.97MB/s]
Downloading data: 100%|██████████████████████████████████████████████████████████████████████| 28.6M/28.6M [00:09<00:00, 2.94MB/s]
Downloading data: 100%|██████████████████████████████████████████████████████████████████████| 28.6M/28.6M [00:02<00:00, 13.4MB/s]
Downloading data: 100%|██████████████████████████████████████████████████████████████████████| 27.8M/27.8M [00:02<00:00, 11.5MB/s]
Downloading data: 100%|██████████████████████████████████████████████████████████████████████| 27.9M/27.9M [00:06<00:00, 4.58MB/s]
Generating train split: 6000 examples [00:03, 1560.13 examples/s]
Generating test split: 1000 examples [00:00, 2184.53 examples/s]
2025-07-28 08:58:41,130 - evalscope - handler.py - async_wrapper - 19 - ERROR - Exception in async function 'benchmark': DatasetPluginBase.create_message() got an unexpected keyword argument 'image_url'
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py", line 17, in async_wrapper
    return await func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/benchmark.py", line 196, in benchmark
    async for request in get_requests(args, api_plugin):
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/benchmark.py", line 70, in get_requests
    async for request in generator:
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/benchmark.py", line 43, in generate_requests_from_dataset
    for messages in message_generator.build_messages():
  File "/usr/local/lib/python3.10/dist-packages/evalscope/perf/plugin/datasets/flickr8k.py", line 27, in build_messages
    message = self.create_message(text=text, image_url=f'data:image/jpeg;base64,{base64_image}')
TypeError: DatasetPluginBase.create_message() got an unexpected keyword argument 'image_url'
swanlab: Experiment name_of_swanlab_log has completed
swanlab: 🌟 Run `swanlab watch /root/workspace/eval-scope/outputs` to view SwanLab Experiment Dashboard locally
2025-07-28 08:58:41,442 - asyncio - ERROR - Task was destroyed but it is pending!
task: <Task pending name='Task-9' coro=<statistic_benchmark_metric() running at /usr/local/lib/python3.10/dist-packages/evalscope/perf/utils/handler.py:14>>
sys:1: RuntimeWarning: coroutine 'statistic_benchmark_metric' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
```

## 运行环境

- 操作系统：ubuntu 22.04
- Python版本：python 3.10

## 其他信息

如果有其他相关信息，请在此处提供。


回退了0.16.0版本，可以跑通，感觉是0.17.1的改动

main分支已修复这个问题

> 感谢你的反馈！我们将关闭此问题。如果您有任何疑问，请随时重新打开它。如果EvalScope对您有所帮助，欢迎给我们点个STAR以示支持，谢谢！