Repository: modelscope/evalscope
Issue: 本地离线加载数据集和Qwen2.5-VL评测mathvision问题
Root cause: Documentation Deficiency

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [✅] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [✅] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [✅] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题

## 问题描述
您好！
在本地离线加载数据集和Qwen2.5-VL评测mathvision时，遇到如下问题
```
ValueError: Unrecognized configuration class <class 'transformers.models.qwen2_5_vl.configuration_qwen2_5_vl.Qwen2_5_VLConfig'> for this kind of AutoModel: AutoModelForCausalLM.
Model type should be one of AriaTextConfig, BambaConfig, BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig, CamembertConfig, LlamaConfig, CodeGenConfig, CohereConfig, Cohere2Config, CpmAntConfig, CTRLConfig, Data2VecTextConfig, DbrxConfig, DeepseekV3Config, DiffLlamaConfig, ElectraConfig, Emu3Config, ErnieConfig, FalconConfig, FalconMambaConfig, FuyuConfig, GemmaConfig, Gemma2Config, Gemma3Config, Gemma3TextConfig, GitConfig, GlmConfig, GotOcr2Config, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTNeoXJapaneseConfig, GPTJConfig, GraniteConfig, GraniteMoeConfig, GraniteMoeSharedConfig, HeliumConfig, JambaConfig, JetMoeConfig, LlamaConfig, Llama4Config, Llama4TextConfig, MambaConfig, Mamba2Config, MarianConfig, MBartConfig, MegaConfig, MegatronBertConfig, MistralConfig, MixtralConfig, MllamaConfig, MoshiConfig, MptConfig, MusicgenConfig, MusicgenMelodyConfig, MvpConfig, NemotronConfig, OlmoConfig, Olmo2Config, OlmoeConfig, OpenLlamaConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PersimmonConfig, PhiConfig, Phi3Config, Phi4MultimodalConfig, PhimoeConfig, PLBartConfig, ProphetNetConfig, QDQBertConfig, Qwen2Config, Qwen2MoeConfig, Qwen3Config, Qwen3MoeConfig, RecurrentGemmaConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, RwkvConfig, Speech2Text2Config, StableLmConfig, Starcoder2Config, TransfoXLConfig, TrOCRConfig, WhisperConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig, ZambaConfig, Zamba2Config.
```
且更换了qwen2.5加载方式后显示"只支持文本输入"

我的启动命令是
```
evalscope eval \
 --model /data1/yyy25/datasets/Qwen2.5-VL-3B-Instruct \
 --datasets math_vision \
 --dataset-args '{"math_vision": {"local_path": "/data1/yyy25/datasets/mathvision_for_eval"}}' \

```

请问快速开始能不能出一个评测VL的demo呢？目前所有demo都是文本模态的。感谢！

## EvalScope 版本（必填）
v0.xx.x

## 使用的工具
- [ ] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [ ] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令

请提供您执行的主要代码或指令。

## 错误日志

请粘贴完整的错误日志或控制台输出。

## 运行环境

- 操作系统：
- Python版本：

## 其他信息

如果有其他相关信息，请在此处提供。


作者您好，换了一种写法之后，我先用vllm启动judge：
```
# 
export CUDA_VISIBLE_DEVICES=1
python -m vllm.entrypoints.openai.api_server \
    --model /data1/yyy25/datasets/Qwen2.5-VL-3B-Instruct \
    --port 8801 \
    --served-model-name local_judge_model
```
，然后运行评测
```
task_cfg_dict = {
    'eval_backend': 'VLMEvalKit',
    'eval_type': 'openai_api', 
    'api_key':'EMPTY',
    'api_url':'http://127.0.0.1:8801/v1',
    'eval_config': {
        'data': ['MathVision'],
        'limit': 5,
        'mode': 'all',
        'model': [
            # 1. 你的主测试模型（本地权重加载）
            {
                'name': 'Qwen2.5-VL-7B-Instruct',
                'model_path': '/data1/yyy25/datasets/Qwen2.5-VL-7B-Instruct',
            }
        ],
        'reuse': False,
        'work_dir': 'outputs',
        # 核心修改：将 judge 设置为上面定义的 API 模型名称
    }
}

from evalscope.run import run_task
from evalscope.summarizer import Summarizer


def run_eval():
    # 选项 1: python 字典
    task_cfg = task_cfg_dict

    # 选项 2: yaml 配置文件
    # task_cfg = 'eval_openai_api.yaml'

    run_task(task_cfg=task_cfg)

    print('>> Start to get the report with summarizer ...')
    report_list = Summarizer.get_report_from_cfg(task_cfg)
    print(f'\n>> The report list: {report_list}')


run_eval()
```

评测后仍然报错
```
`torch_dtype` is deprecated! Use `dtype` instead!
You are attempting to use Flash Attention 2 without specifying a torch dtype. This might lead to unexpected behaviour
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:03<00:00,  1.29it/s]
Infer Qwen2.5-VL-7B-Instruct/MathVision, Rank 0/1: 100%|█████████████████████████████████████████████████████████████████████████████████| 5/5 [00:28<00:00,  5.79s/it]
{'nproc': 16, 'verbose': False, 'retry': 3, 'model': 'gpt-4o-mini'}
ANTLR runtime and generated code versions disagree: 4.9.3!=4.7.2
ANTLR runtime and generated code versions disagree: 4.9.3!=4.7.2
ANTLR runtime and generated code versions disagree: 4.9.3!=4.7.2
ANTLR runtime and generated code versions disagree: 4.9.3!=4.7.2
2025-12-29 22:42:43,953 - root - INFO: Did not detect the .env file at /home/yyy/miniconda3/envs/azr/lib/python3.10/site-packages/.env, don't load. 
Using API Base: https://api.openai.com/v1/chat/completions; API Key: 
<class 'KeyError'>: 'choices'
{
    "error": {
        "message": "You didn't provide an API key. You need to provide your API key in an Authorization header using Bearer auth (i.e. Authorization: Bearer YOUR_KEY), or as the password field (with blank username) if you're accessing the API from your browser and are prompted for a username and password. You can obtain an API key from https://platform.openai.com/account/api-keys.",
        "type": "invalid_request_error",
        "param": null,
        "code": null
    }
}

<class 'KeyError'>: 'choices'
{
    "error": {
        "message": "You didn't provide an API key. You need to provide your API key in an Authorization header using Bearer auth (i.e. Authorization: Bearer YOUR_KEY), or as the password field (with blank username) if you're accessing the API from your browser and are prompted for a username and password. You can obtain an API key from https://platform.openai.com/account/api-keys.",
        "type": "invalid_request_error",
        "param": null,
        "code": null
    }
}
```

多模态模型不支持直接transformer推理，需要使用vllm启动模型服务来推理，参考使用文档：https://evalscope.readthedocs.io/zh-cn/latest/get_started/basic_usage.html#api