Repository: modelscope/evalscope
Issue: evalscope perf使用longalpaca和默认的open_qa数据集均报错Dataset is empty!
Root cause: Validation Gap

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [x] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [x ] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [ x] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题

## 问题描述
perf 指定多种数据集无法执行测试，报错： evalscope - ERROR - Exception in async function 'benchmark': Dataset is empty!

## EvalScope 版本（必填）
v1.0.0 latest

## 使用的工具
- [ ] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [x ] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令
evalscope perf \
  --parallel 1 10 50 100 200 \
  --number 10 20 100 200 400 \
  --model /xx/qwen2.5-7b-instruct-gptq-int4/ \
  --url http://127.0.0.1:8000/v1/completions \
  --api openai \
  --dataset longalpaca \
  --max-tokens 1024 \
  --min-tokens 1024 \
  --prefix-length 0 \
  --min-prompt-length 1024 \
  --max-prompt-length 1024 \
  --tokenizer-path /xx/qwen2.5-7b-instruct-gptq-int4/ \
  --extra-args '{"ignore_eos": true}'

## 错误日志
2025-09-03 10:20:34 - evalscope - INFO: Save the result to: outputs/20250903_102034/
2025-09-03 10:20:34 - evalscope - INFO: Starting benchmark with args:
2025-09-03 10:20:34 - evalscope - INFO: {
    "model": "/sfmodel/qwen2.5-7b-instruct-gptq-int4/",
    "model_id": "",
    "attn_implementation": null,
    "api": "openai",
    "tokenizer_path": "/sfmodel/qwen2.5-7b-instruct-gptq-int4",
    "port": 8877,
    "url": "http://127.0.0.1:8000/v1/completions",
    "headers": {},
    "connect_timeout": 600,
    "read_timeout": 600,
    "api_key": null,
    "no_test_connection": false,
    "number": 10,
    "parallel": 1,
    "rate": -1,
    "sleep_interval": 5,
    "log_every_n_query": 10,
    "debug": false,
    "wandb_api_key": null,
    "swanlab_api_key": null,
    "name": null,
    "outputs_dir": "outputs/20250903_102034/parallel_1_number_10",
    "max_prompt_length": 1024,
    "min_prompt_length": 1024,
    "prefix_length": 0,
    "prompt": null,
    "query_template": null,
    "apply_chat_template": false,
    "image_width": 224,
    "image_height": 224,
    "image_format": "RGB",
    "image_num": 1,
    "dataset": "longalpaca",
    "dataset_path": null,
    "frequency_penalty": null,
    "repetition_penalty": null,
    "logprobs": null,
    "max_tokens": 1024,
    "min_tokens": 1024,
    "n_choices": null,
    "seed": 0,
    "stop": null,
    "stop_token_ids": null,
    "stream": true,
    "temperature": 0.0,
    "top_p": null,
    "top_k": null,
    "extra_args": {
        "ignore_eos": true
    }
}
2025-09-03 10:20:37 - evalscope - INFO: Test connection successful.
Downloading [README.md]: 23.6kB [00:00, 509kB/s]
2025-09-03 10:20:43,844 - modelscope - INFO - storing https://www.modelscope.cn/api/v1/datasets/AI-ModelScope/LongAlpaca-12k/repo?Source=SDK&Revision=master&FilePath=README.md&View=False in cache at /root/.cache/modelscope/hub/datasets/48c94f45985d5f4a20d22d9fd0f45a0c5fac43efc070a5d084c8dabba8d739ff
2025-09-03 10:20:43,846 - modelscope - INFO - creating metadata file for /root/.cache/modelscope/hub/datasets/48c94f45985d5f4a20d22d9fd0f45a0c5fac43efc070a5d084c8dabba8d739ff
Downloading data: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 490M/490M [02:47<00:00, 2.93MB/s]
2025-09-03 10:23:43,928 - modelscope - INFO - storing https://www.modelscope.cn/api/v1/datasets/AI-ModelScope/LongAlpaca-12k/repo?Source=SDK&Revision=master&FilePath=LongAlpaca-12k_.csv in cache at /root/.cache/modelscope/hub/datasets/downloads/2128f41478a5e98299a55d38118eeeac850c57b77432c6972a04173b2ebff54b
2025-09-03 10:23:43,930 - modelscope - INFO - creating metadata file for /root/.cache/modelscope/hub/datasets/downloads/2128f41478a5e98299a55d38118eeeac850c57b77432c6972a04173b2ebff54b
Generating train split: 12000 examples [00:05, 2291.25 examples/s]
2025-09-03 10:23:50 - evalscope - ERROR: Exception in async function 'benchmark': Dataset is empty!
Traceback (most recent call last):
  File "/sfmodel/lqw/annaconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/handler.py", line 17, in async_wrapper
    return await func(*args, **kwargs)
  File "/sfmodel/lqw/annaconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/benchmark.py", line 197, in benchmark
    async for request in get_requests(args, api_plugin):
  File "/sfmodel/lqw/annaconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/benchmark.py", line 70, in get_requests
    async for request in generator:
  File "/sfmodel/lqw/annaconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/benchmark.py", line 49, in generate_requests_from_dataset
    raise Exception('Dataset is empty!')
Exception: Dataset is empty!
2025-09-03 10:23:51,167 - asyncio - ERROR: Task was destroyed but it is pending!
task: <Task pending name='Task-8' coro=<statistic_benchmark_metric() running at /sfmodel/lqw/annaconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/perf/utils/handler.py:14>>
sys:1: RuntimeWarning: coroutine 'statistic_benchmark_metric' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback



## 运行环境

- 操作系统：Ubuntu22.04
- Python版本：3.10

## 其他信息
不指定dataset时，默认使用openqa数据集，也报错相同问题，降级到0.17，问题同样出现


命令中设置了prompt长度，找不到对应长度的prompt导致数据集为空，去掉这两个参数
```
--min-prompt-length 1024
--max-prompt-length 1024
```

原来如此! 已解决~ 非常感谢