Repository: modelscope/evalscope
Issue: opencompass测评结果为空
Root cause: Documentation Deficiency

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [x] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [x] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [x] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题

## 问题描述
执行python example_eval_openai_api.py之后中间也没有报错，但是打印内容为空。

## EvalScope 版本（必填）
v0.15.1

## 使用的工具
- [x] Opencompass backend

## 执行的代码或指令

python example_eval_openai_api.py
内容如下：
from evalscope.run import run_task
from evalscope.summarizer import Summarizer
task_cfg_dict = dict(
    eval_backend='OpenCompass',
    eval_config={
        'datasets': ['ceval'],
        'models': [
            {'path': '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B', 
            'openai_api_base': 'http://127.0.0.1:8877/v1/chat/completions', 
            'is_chat': True,
            'batch_size': 16},
        ],
        'work_dir': 'outputs/DeepSeek-R1-Distill-Qwen-7B_eval_result',
        'limit': 100,
        },
    )
    
def run_eval():
    # 选项 1: python 字典
    task_cfg = task_cfg_dict

    # 选项 2: yaml 配置文件
    # task_cfg = 'eval_openai_api.yaml'

    # 选项 3: json 配置文件
    # task_cfg = 'eval_openai_api.json'

    run_task(task_cfg=task_cfg)

    print('>> Start to get the report with summarizer ...')
    report_list = Summarizer.get_report_from_cfg(task_cfg)
    print(f'\n>> The report list: {report_list}')

run_eval()
## 错误日志

python example_eval_openai_api.py
2025-05-19 15:45:23,453 - evalscope - INFO - Args: Task config is provided with dictionary type.
2025-05-19 15:45:26,106 - evalscope - INFO - Check the OpenCompass environment: OK
2025-05-19 15:45:26,108 - evalscope - INFO - Dump task config to ./outputs/20250519_154523/configs/task_config_a8e419.yaml
2025-05-19 15:45:26,109 - evalscope - INFO - {
    "model": "DummyCustomModel",
    "model_id": "dummy-model",
    "model_args": {
        "revision": "master",
        "precision": "torch.float16"
    },
    "model_task": "text_generation",
    "template_type": null,
    "chat_template": null,
    "datasets": [],
    "dataset_args": {},
    "dataset_dir": "/mnt/workspace/.cache/modelscope/hub/datasets",
    "dataset_hub": "modelscope",
    "generation_config": {
        "max_length": 2048,
        "max_new_tokens": 512,
        "do_sample": false,
        "top_k": 50,
        "top_p": 1.0,
        "temperature": 1.0
    },
    "eval_type": "custom",
    "eval_backend": "OpenCompass",
    "eval_config": {
        "datasets": [
            "ceval"
        ],
        "models": [
            {
                "path": "/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B",
                "openai_api_base": "http://127.0.0.1:8877/v1/chat/completions",
                "is_chat": true,
                "batch_size": 16
            }
        ],
        "work_dir": "outputs/DeepSeek-R1-Distill-Qwen-7B_eval_result",
        "limit": 100,
        "time_str": "20250519_154523"
    },
    "stage": "all",
    "limit": null,
    "eval_batch_size": 1,
    "mem_cache": false,
    "use_cache": null,
    "work_dir": "./outputs/20250519_154523",
    "outputs": null,
    "debug": false,
    "dry_run": false,
    "seed": 42,
    "api_url": null,
    "api_key": "EMPTY",
    "timeout": null,
    "stream": false,
    "judge_strategy": "auto",
    "judge_worker_num": 1,
    "judge_model_args": {}
}
[2025-05-19 15:45:28,380] [INFO] [real_accelerator.py:239:get_accelerator] Setting ds_accelerator to cuda (auto detect)
INFO 05-19 15:45:29 [__init__.py:239] Automatically detected platform cuda.
2025-05-19 15:45:31,648 - evalscope - INFO - *** Run task with config: /tmp/tmpcl4_wihd.py 
05/19 15:45:31 - OpenCompass - INFO - Current exp folder: outputs/DeepSeek-R1-Distill-Qwen-7B_eval_result/20250519_154523
05/19 15:45:33 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored.
05/19 15:45:33 - OpenCompass - INFO - Partitioned into 52 tasks.
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-computer_network] on CPU                                                                                            
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-operating_system] on CPU                                                                                            
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-computer_architecture] on CPU                                                                                       
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-college_programming] on CPU                                                                                         
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-college_physics] on CPU                                                                                             
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-college_chemistry] on CPU                                                                                           
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-advanced_mathematics] on CPU                                                                                        
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-probability_and_statistics] on CPU                                                                                  
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-discrete_mathematics] on CPU                                                                                        
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-electrical_engineer] on CPU                                                                                         
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-metrology_engineer] on CPU                                                                                          
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-high_school_mathematics] on CPU                                                                                     
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-high_school_physics] on CPU                                                                                         
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-high_school_chemistry] on CPU                                                                                       
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-high_school_biology] on CPU                                                                                         
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-middle_school_mathematics] on CPU                                                                                   
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-middle_school_biology] on CPU                                                                                       
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-middle_school_physics] on CPU                                                                                       
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-middle_school_chemistry] on CPU                                                                                     
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-veterinary_medicine] on CPU                                                                                         
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-college_economics] on CPU                                                                                           
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-business_administration] on CPU                                                                                     
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-marxism] on CPU                                                                                                     
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-mao_zedong_thought] on CPU                                                                                          
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-education_science] on CPU                                                                                           
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-teacher_qualification] on CPU                                                                                       
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-high_school_politics] on CPU                                                                                        
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-high_school_geography] on CPU                                                                                       
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-middle_school_politics] on CPU                                                                                      
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-middle_school_geography] on CPU                                                                                     
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-modern_chinese_history] on CPU                                                                                      
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-ideological_and_moral_cultivation] on CPU                                                                           
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-logic] on CPU                                                                                                       
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-law] on CPU                                                                                                         
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-chinese_language_and_literature] on CPU                                                                             
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-art_studies] on CPU                                                                                                 
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-professional_tour_guide] on CPU                                                                                     
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-legal_professional] on CPU                                                                                          
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-high_school_chinese] on CPU                                                                                         
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-high_school_history] on CPU                                                                                         
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-middle_school_history] on CPU                                                                                       
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-civil_servant] on CPU                                                                                               
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-sports_science] on CPU                                                                                              
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-plant_protection] on CPU                                                                                            
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-basic_medicine] on CPU                                                                                              
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-clinical_medicine] on CPU                                                                                           
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-urban_and_rural_planner] on CPU                                                                                     
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-accountant] on CPU                                                                                                  
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-fire_engineer] on CPU                                                                                               
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-environmental_impact_assessment_engineer] on CPU                                                                    
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-tax_accountant] on CPU                                                                                              
launch OpenICLInfer[/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B/ceval-physician] on CPU                                                                                                   
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 52/52 [09:19<00:00, 10.77s/it]
05/19 15:54:53 - OpenCompass - INFO - Partitioned into 0 tasks.
0it [00:00, ?it/s]
05/19 15:54:53 - OpenCompass - WARNING - unknown result format: {'0': {'origin_prompt': '以下是中国关于计算机网络考试的单项选择题，请选出其中的正确答案。\n下列设备属于资源子网的是____。\nA. 计算机软件\nB. 网桥\nC. 交换机\nD. 路由器\n答案: \nA\n以下是中国关于计算机网络考试的单项选择题，请选出其中的正确答案。\n滑动窗口的作用是____。\nA. 流量控制\nB. 拥塞控制\nC. 路由控制\nD. 差错控制\n答案: \nA\n以下是中国关于计算机网络考试的单项选择题，请选出其中的正确答案。\n在OSI参考模型中，直接为会话层提供服务的是____。\nA. 应用层\nB. 表示层\nC. 传输层\nD. 网络层\n答案: \nC\n以下是中国关于计算机网络考试的单项选择题，请选出其中的正确答案。\n协议是指在____之间进行通信的规则或约定。\nA. 同一结点的上下层\nB. 不同结点\nC. 相邻实体\nD. 不同结点对等实体\n答案: \nD\n以下是中国关于计算机网络考试的单项选择题，请选出其中的正确答案。\n主机甲与主机乙之间XXXXXXXXXXXXXXXX要解答这个关于骨骼肌中Ca离子结合位点的问题。首先，我得回忆一下课本上的相关知识。骨骼肌细胞中的钙离子结合位点应该和肌钙蛋白有关。\n\n我记得肌钙蛋白分为两个亚单位，一个是I，一个是C。肌钙蛋白I负责与钙离子结合，而肌钙蛋白C则与肌球蛋白结合，维持肌丝的结构。所以，当骨骼肌收缩时，钙离子会与肌钙蛋白I结合，促进肌丝的收缩。\n\n现在看选项，选项C是肌钙蛋白亚单位I，选项D是肌钙蛋白亚单位C。问题问的是结合位点，所以应该是选项C。其他选项，比如肌纤蛋白和原肌凝蛋白，这些是肌膜上的蛋白，与Ca结合无关，所以排除A和B。选项D是肌钙蛋白C，不是结合Ca的位点，所以也不对。\n\n综上所述，正确答案应该是选项C。\n</think>\n\n骨骼肌中Ca离子的结合位点是肌钙蛋白亚单位I，因此正确答案是选项C。\n\n答案：C', 'gold': 'D'}}, continue
dataset                                         version    metric    mode    /nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B
----------------------------------------------  ---------  --------  ------  ---------------------------------------------------
ceval-computer_network                          -          -         -       -
ceval-operating_system                          -          -         -       -
ceval-computer_architecture                     -          -         -       -
ceval-college_programming                       -          -         -       -
ceval-college_physics                           -          -         -       -
ceval-college_chemistry                         -          -         -       -
ceval-advanced_mathematics                      -          -         -       -
ceval-probability_and_statistics                -          -         -       -
ceval-discrete_mathematics                      -          -         -       -
ceval-electrical_engineer                       -          -         -       -
ceval-metrology_engineer                        -          -         -       -
ceval-high_school_mathematics                   -          -         -       -
ceval-high_school_physics                       -          -         -       -
ceval-high_school_chemistry                     -          -         -       -
ceval-high_school_biology                       -          -         -       -
ceval-middle_school_mathematics                 -          -         -       -
ceval-middle_school_biology                     -          -         -       -
ceval-middle_school_physics                     -          -         -       -
ceval-middle_school_chemistry                   -          -         -       -
ceval-veterinary_medicine                       -          -         -       -
ceval-college_economics                         -          -         -       -
ceval-business_administration                   -          -         -       -
ceval-marxism                                   -          -         -       -
ceval-mao_zedong_thought                        -          -         -       -
ceval-education_science                         -          -         -       -
ceval-teacher_qualification                     -          -         -       -
ceval-high_school_politics                      -          -         -       -
ceval-high_school_geography                     -          -         -       -
ceval-middle_school_politics                    -          -         -       -
ceval-middle_school_geography                   -          -         -       -
ceval-modern_chinese_history                    -          -         -       -
ceval-ideological_and_moral_cultivation         -          -         -       -
ceval-logic                                     -          -         -       -
ceval-law                                       -          -         -       -
ceval-chinese_language_and_literature           -          -         -       -
ceval-art_studies                               -          -         -       -
ceval-professional_tour_guide                   -          -         -       -
ceval-legal_professional                        -          -         -       -
ceval-high_school_chinese                       -          -         -       -
ceval-high_school_history                       -          -         -       -
ceval-middle_school_history                     -          -         -       -
ceval-civil_servant                             -          -         -       -
ceval-sports_science                            -          -         -       -
ceval-plant_protection                          -          -         -       -
ceval-basic_medicine                            -          -         -       -
ceval-clinical_medicine                         -          -         -       -
ceval-urban_and_rural_planner                   -          -         -       -
ceval-accountant                                -          -         -       -
ceval-fire_engineer                             -          -         -       -
ceval-environmental_impact_assessment_engineer  -          -         -       -
ceval-tax_accountant                            -          -         -       -
ceval-physician                                 -          -         -       -
05/19 15:54:54 - OpenCompass - INFO - write summary to /outputs/DeepSeek-R1-Distill-Qwen-7B_eval_result/20250519_154523/summary/summary_20250519_154523.txt
05/19 15:54:54 - OpenCompass - INFO - write csv to /outputs/DeepSeek-R1-Distill-Qwen-7B_eval_result/20250519_154523/summary/summary_20250519_154523.csv


The markdown format results is as below:

| dataset | version | metric | mode | /nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B |
|----- | ----- | ----- | ----- | -----|
| ceval-computer_network | - | - | - | - |
| ceval-operating_system | - | - | - | - |
| ceval-computer_architecture | - | - | - | - |
| ceval-college_programming | - | - | - | - |
| ceval-college_physics | - | - | - | - |
| ceval-college_chemistry | - | - | - | - |
| ceval-advanced_mathematics | - | - | - | - |
| ceval-probability_and_statistics | - | - | - | - |
| ceval-discrete_mathematics | - | - | - | - |
| ceval-electrical_engineer | - | - | - | - |
| ceval-metrology_engineer | - | - | - | - |
| ceval-high_school_mathematics | - | - | - | - |
| ceval-high_school_physics | - | - | - | - |
| ceval-high_school_chemistry | - | - | - | - |
| ceval-high_school_biology | - | - | - | - |
| ceval-middle_school_mathematics | - | - | - | - |
| ceval-middle_school_biology | - | - | - | - |
| ceval-middle_school_physics | - | - | - | - |
| ceval-middle_school_chemistry | - | - | - | - |
| ceval-veterinary_medicine | - | - | - | - |
| ceval-college_economics | - | - | - | - |
| ceval-business_administration | - | - | - | - |
| ceval-marxism | - | - | - | - |
| ceval-mao_zedong_thought | - | - | - | - |
| ceval-education_science | - | - | - | - |
| ceval-teacher_qualification | - | - | - | - |
| ceval-high_school_politics | - | - | - | - |
| ceval-high_school_geography | - | - | - | - |
| ceval-middle_school_politics | - | - | - | - |
| ceval-middle_school_geography | - | - | - | - |
| ceval-modern_chinese_history | - | - | - | - |
| ceval-ideological_and_moral_cultivation | - | - | - | - |
| ceval-logic | - | - | - | - |
| ceval-law | - | - | - | - |
| ceval-chinese_language_and_literature | - | - | - | - |
| ceval-art_studies | - | - | - | - |
| ceval-professional_tour_guide | - | - | - | - |
| ceval-legal_professional | - | - | - | - |
| ceval-high_school_chinese | - | - | - | - |
| ceval-high_school_history | - | - | - | - |
| ceval-middle_school_history | - | - | - | - |
| ceval-civil_servant | - | - | - | - |
| ceval-sports_science | - | - | - | - |
| ceval-plant_protection | - | - | - | - |
| ceval-basic_medicine | - | - | - | - |
| ceval-clinical_medicine | - | - | - | - |
| ceval-urban_and_rural_planner | - | - | - | - |
| ceval-accountant | - | - | - | - |
| ceval-fire_engineer | - | - | - | - |
| ceval-environmental_impact_assessment_engineer | - | - | - | - |
| ceval-tax_accountant | - | - | - | - |
| ceval-physician | - | - | - | - |

05/19 15:54:54 - OpenCompass - INFO - write markdown summary to /outputs/DeepSeek-R1-Distill-Qwen-7B_eval_result/20250519_154523/summary/summary_20250519_154523.md
>> Start to get the report with summarizer ...
2025-05-19 15:54:54,424 - evalscope - INFO - Args: Task config is provided with dictionary type.
2025-05-19 15:54:54,424 - evalscope - INFO - **Loading task cfg for summarizer: {
    "model": "DummyCustomModel",
    "model_id": "dummy-model",
    "model_args": {
        "revision": "master",
        "precision": "torch.float16"
    },
    "model_task": "text_generation",
    "template_type": null,
    "chat_template": null,
    "datasets": [],
    "dataset_args": {},
    "dataset_dir": "/mnt/workspace/.cache/modelscope/hub/datasets",
    "dataset_hub": "modelscope",
    "generation_config": {
        "max_length": 2048,
        "max_new_tokens": 512,
        "do_sample": false,
        "top_k": 50,
        "top_p": 1.0,
        "temperature": 1.0
    },
    "eval_type": "custom",
    "eval_backend": "OpenCompass",
    "eval_config": {
        "datasets": [
            "ceval"
        ],
        "models": [
            {
                "path": "/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B",
                "openai_api_base": "http://127.0.0.1:8877/v1/chat/completions",
                "is_chat": true,
                "batch_size": 16,
                "abbr": "/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B"
            }
        ],
        "work_dir": "outputs/DeepSeek-R1-Distill-Qwen-7B_eval_result",
        "limit": 100,
        "time_str": "20250519_154523"
    },
    "stage": "all",
    "limit": null,
    "eval_batch_size": 1,
    "mem_cache": false,
    "use_cache": null,
    "work_dir": "./outputs",
    "outputs": null,
    "debug": false,
    "dry_run": false,
    "seed": 42,
    "api_url": null,
    "api_key": "EMPTY",
    "timeout": null,
    "stream": false,
    "judge_strategy": "auto",
    "judge_worker_num": 1,
    "judge_model_args": {}
}

>> The report list: [{'dataset': 'ceval-computer_network', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-operating_system', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-computer_architecture', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-college_programming', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-college_physics', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-college_chemistry', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-advanced_mathematics', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-probability_and_statistics', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-discrete_mathematics', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-electrical_engineer', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-metrology_engineer', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-high_school_mathematics', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-high_school_physics', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-high_school_chemistry', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-high_school_biology', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-middle_school_mathematics', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-middle_school_biology', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-middle_school_physics', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-middle_school_chemistry', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-veterinary_medicine', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-college_economics', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-business_administration', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-marxism', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-mao_zedong_thought', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-education_science', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-teacher_qualification', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-high_school_politics', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-high_school_geography', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-middle_school_politics', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-middle_school_geography', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-modern_chinese_history', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-ideological_and_moral_cultivation', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-logic', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-law', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-chinese_language_and_literature', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-art_studies', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-professional_tour_guide', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-legal_professional', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-high_school_chinese', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-high_school_history', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-middle_school_history', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-civil_servant', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-sports_science', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-plant_protection', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-basic_medicine', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-clinical_medicine', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-urban_and_rural_planner', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-accountant', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-fire_engineer', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-environmental_impact_assessment_engineer', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-tax_accountant', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}, {'dataset': 'ceval-physician', 'version': '-', 'metric': '-', 'mode': '-', '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B': '-'}]

## 运行环境

- 操作系统：linux
- Python版本：3.11.11
ms-opencompass                    0.1.6
ms_swift                          3.3.0.post1
## 其他信息

如果有其他相关信息，请在此处提供。


看一下 outputs/DeepSeek-R1-Distill-Qwen-7B_eval_result 下面的内容，有模型的prediction结果吗

> 看一下 outputs/DeepSeek-R1-Distill-Qwen-7B_eval_result 下面的内容，有模型的prediction结果吗

![Image](https://github.com/user-attachments/assets/b5ee79fc-b5df-4ab0-b091-309ab600dd97)
是空的

prediction下面有结果吗，我这复现不了你的问题：

![Image](https://github.com/user-attachments/assets/71618874-f778-4ef1-8eed-d80c209d5afd)

> 预测下面有结果吗，我这复现不了你的问题：
> 
> ![Image](https://github.com/user-attachments/assets/71618874-f778-4ef1-8eed-d80c209d5afd)

![Image](https://github.com/user-attachments/assets/46d7955d-507c-41e8-94df-930d5547c69f)
我的没有predictions这个文件夹

看一下log输出，有报错吗

![Image](https://github.com/user-attachments/assets/88c45606-4f42-436a-9f8b-0e0138e80549)

![Image](https://github.com/user-attachments/assets/20a11646-724c-4531-97e2-302ccda32191)
是这个吗？


应该是`'path': '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B'` 导致的问题，这里能设置模型名称吗，不要使用绝对路径

> 应该是`'path': '/nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B'`导致的问题，这里可以设置模型名称吗，不要使用绝对路径

我是用vllm部署的模型。我之前试过只写模型名字，但是会报错。

vllm可以设置served_model_name，你可以看看具体的文档说明

> vllm可以设置served_model_name，你可以看看具体的文档说明

![Image](https://github.com/user-attachments/assets/b3d64274-aa24-429c-af6f-feafdb7eed3b)
文档里是这样写的，但是不知道model id是啥。
我的vllm部署命令是：
VLLM_USE_MODELSCOPE=True CUDA_VISIBLE_DEVICES=4 python -m vllm.entrypoints.openai.api_server --model /nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B --port 8877
运行脚本配置：
eval_config={
        'datasets': ['mmlu'],
        'models': [
            {'path': 'DeepSeek-R1-Distill-Qwen-7B', 
            'openai_api_base': 'http://127.0.0.1:8877/v1/chat/completions', 
            'is_chat': True,
            'batch_size': 16},
        ],
        'work_dir': 'outputs/DeepSeek-R1-Distill-Qwen-7B_eval_result',
        'limit': 100,
        },
运行时报错：

![Image](https://github.com/user-attachments/assets/ae57245e-25e6-48b1-a3ac-8cc8da22449c)

![Image](https://github.com/user-attachments/assets/33767963-4c9f-4c61-b900-eb5754198033)


> vllm可以设置served_model_name，你可以看看具体的文档说明

我还打印了一下model id，结果就是模型路径

![Image](https://github.com/user-attachments/assets/c004bd59-669e-4824-afbf-c273dcd5286c)

这样写命令：
```bash
VLLM_USE_MODELSCOPE=True CUDA_VISIBLE_DEVICES=4 python -m vllm.entrypoints.openai.api_server --model /nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B --port 8877 --served-model-name DeepSeek-R1-Distill-Qwen-7B
```
就可以用DeepSeek-R1-Distill-Qwen-7B作为model id了

> 这样写命令：
> 
> VLLM_USE_MODELSCOPE=True CUDA_VISIBLE_DEVICES=4 python -m vllm.entrypoints.openai.api_server --model /nfs/largemodel/model/DeepSeek-R1-Distill-Qwen-7B --port 8877 --served-model-name DeepSeek-R1-Distill-Qwen-7B
> 就可以用DeepSeek-R1-Distill-Qwen-7B作为model id了

已解决，谢谢

> 感谢你的反馈！我们将关闭此问题。如果您有任何疑问，请随时重新打开它。如果EvalScope对您有所帮助，欢迎给我们点个STAR以示支持，谢谢！