Repository: open-compass/opencompass
Issue: 修改配置文件，无法测试本地模型，没有报错信息[Bug] 
Root cause: Validation Gap

### 先决条件

- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。

### 问题类型

我正在使用官方支持的任务/模型/数据集进行评估。

### 环境

‘’‘
python run.py configs\eval_chatglm6b.py
’‘’

### 重现问题 - 代码/配置示例

配置文件如下所示。
‘’‘
from mmengine.config import read_base

with read_base():
    from .datasets.siqa.siqa_gen import siqa_datasets

datasets = [*siqa_datasets]

from opencompass.models import HuggingFaceCausalLM


models = [
    dict(
        type=HuggingFaceCausalLM,
        abbr='chatglm6b',    # 运行完结果展示的名称
        path="D:\\code\\ChatGLM-6B-main\\THUDM\\chatglm2-6",    # 模型路径
        tokenizer_path='D:\\code\\ChatGLM-6B-main\\THUDM\\chatglm2-6',    # 分词器路径
        model_kwargs=dict(
            device_map='auto',
            trust_remote_code=True
        ),
        tokenizer_kwargs=dict(
            padding_side='left',
            truncation_side='left',
            trust_remote_code=True,
            use_fast=False
        ),
        pad_token_id=151643,
        max_out_len=100,
        max_seq_len=2048,
        batch_size=16,
        #run_cfg=dict(num_gpus=8, num_procs=8),    # 我使用A800 开启8张卡评测，就算使用1，也默认使用多卡
    )
]
’‘’

### 重现问题 - 命令或脚本

‘’‘
python run.py configs\eval_chatglm6b.py
’‘’

### 重现问题 - 错误信息

.out 文件没有错误信息
![image](https://github.com/open-compass/opencompass/assets/37373361/babbb3b2-4430-48d4-be49-2444bcdb8e26)


### 其他信息

_No response_

麻烦帮忙看下，本地已有模型，修改了 path 和 tokenizer_path，但是运行不起来。

plz show the content of "D:\code\ChatGLM-6B-main\THUDM\chatglm2-6".

![image](https://github.com/open-compass/opencompass/assets/7881589/c516eca0-78e8-427f-991e-300797fcfce7)
Please check the log carefully.

> ![image](https://private-user-images.githubusercontent.com/7881589/296649932-c516eca0-78e8-427f-991e-300797fcfce7.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDUyOTc3OTYsIm5iZiI6MTcwNTI5NzQ5NiwicGF0aCI6Ii83ODgxNTg5LzI5NjY0OTkzMi1jNTE2ZWNhMC03OGU4LTQyN2YtOTkxZS0zMDA3OTdmY2ZjZTcucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDExNSUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDAxMTVUMDU0NDU2WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MDRjZDRhODllNzNmYTlhYzQwMjJiNDE3ZDg4ZTdmMTlhNTEwZGUzNGRmMjcwN2E1OGEwNmYzYWRlMjkzMmM4NSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.7V0c8A3q4ZlrJfbU2SVo0Hr35TWqwgJzjI3szyJ-Mzg) Please check the log carefully.

这个文件内容为空。

> plz show the content of "D:\code\ChatGLM-6B-main\THUDM\chatglm2-6".

![image](https://github.com/open-compass/opencompass/assets/37373361/03804907-6102-4b5f-b1f7-3ae0c6e29ee1)
b
ChatGLM2-6B本地运行是可以的。

你可以在运行的时候，加上 --debug，看下报错信息。比如python run.py ChatGLM2-6B.py --debug

> --debug

debug 模式下并没有发现错误，但是最后没有分数，debug 内容如下：
(opencompass0112) D:\code\opencompass>python run.py configs\eval_chatglm6b.py --debug
01/15 15:04:24 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored.
01/15 15:04:24 - OpenCompass - DEBUG - Modules of opencompass's partitioner registry have been automatically imported from opencompass.partitioners
01/15 15:04:24 - OpenCompass - DEBUG - Get class `SizePartitioner` from "partitioner" registry in "opencompass"
01/15 15:04:24 - OpenCompass - DEBUG - An `SizePartitioner` instance is built from registry, and its implementation can be found in opencompass.partitioners.size
01/15 15:04:24 - OpenCompass - DEBUG - Key eval.runner.task.judge_cfg not found in config, ignored.
01/15 15:04:24 - OpenCompass - DEBUG - Key eval.runner.task.dump_details not found in config, ignored.
01/15 15:04:24 - OpenCompass - DEBUG - Additional config: {}
01/15 15:04:24 - OpenCompass - INFO - Partitioned into 1 tasks.
01/15 15:04:24 - OpenCompass - DEBUG - Task 0: [chatglm6b/ceval-college_economics,chatglm6b/ceval-accountant,chatglm6b/ceval-tax_accountant,chatglm6b/ceval-physician,chatglm6b/ceval-civil_servant,chatglm6b/ceval-urban_and_rural_planner,chatglm6b/ceval-teacher_qualification,chatglm6b/ceval-college_programming,chatglm6b/ceval-electrical_engineer,chatglm6b/ceval-business_administration,chatglm6b/ceval-art_studies,chatglm6b/ceval-fire_engineer,chatglm6b/ceval-environmental_impact_assessment_engineer,chatglm6b/ceval-education_science,chatglm6b/ceval-professional_tour_guide,chatglm6b/ceval-college_chemistry,chatglm6b/ceval-metrology_engineer,chatglm6b/ceval-mao_zedong_thought,chatglm6b/ceval-law,chatglm6b/ceval-veterinary_medicine,chatglm6b/ceval-modern_chinese_history,chatglm6b/ceval-chinese_language_and_literature,chatglm6b/ceval-legal_professional,chatglm6b/ceval-logic,chatglm6b/ceval-middle_school_history,chatglm6b/ceval-plant_protection,chatglm6b/ceval-clinical_medicine,chatglm6b/ceval-computer_architecture,chatglm6b/ceval-middle_school_biology,chatglm6b/ceval-middle_school_politics,chatglm6b/ceval-middle_school_chemistry,chatglm6b/ceval-high_school_history,chatglm6b/ceval-computer_network,chatglm6b/ceval-operating_system,chatglm6b/ceval-college_physics,chatglm6b/ceval-advanced_mathematics,chatglm6b/ceval-high_school_physics,chatglm6b/ceval-high_school_chemistry,chatglm6b/ceval-high_school_biology,chatglm6b/ceval-middle_school_mathematics,chatglm6b/ceval-middle_school_physics,chatglm6b/ceval-marxism,chatglm6b/ceval-high_school_politics,chatglm6b/ceval-high_school_geography,chatglm6b/ceval-ideological_and_moral_cultivation,chatglm6b/ceval-high_school_chinese,chatglm6b/ceval-sports_science,chatglm6b/ceval-basic_medicine,chatglm6b/ceval-probability_and_statistics,chatglm6b/ceval-high_school_mathematics,chatglm6b/ceval-discrete_mathematics,chatglm6b/ceval-middle_school_geography]
01/15 15:04:24 - OpenCompass - DEBUG - Modules of opencompass's runner registry have been automatically imported from opencompass.runners
01/15 15:04:24 - OpenCompass - DEBUG - Get class `LocalRunner` from "runner" registry in "opencompass"
01/15 15:04:24 - OpenCompass - DEBUG - An `LocalRunner` instance is built from registry, and its implementation can be found in opencompass.runners.local
01/15 15:04:24 - OpenCompass - DEBUG - Modules of opencompass's task registry have been automatically imported from opencompass.tasks
01/15 15:04:24 - OpenCompass - DEBUG - Get class `OpenICLInferTask` from "task" registry in "opencompass"
01/15 15:04:24 - OpenCompass - DEBUG - An `OpenICLInferTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_infer
01/15 15:04:28 - OpenCompass - DEBUG - Get class `NaivePartitioner` from "partitioner" registry in "opencompass"
01/15 15:04:28 - OpenCompass - DEBUG - An `NaivePartitioner` instance is built from registry, and its implementation can be found in opencompass.partitioners.naive
01/15 15:04:28 - OpenCompass - DEBUG - Key eval.runner.task.judge_cfg not found in config, ignored.
01/15 15:04:28 - OpenCompass - DEBUG - Key eval.runner.task.dump_details not found in config, ignored.
01/15 15:04:28 - OpenCompass - DEBUG - Additional config: {'eval': {'runner': {'task': {}}}}
01/15 15:04:28 - OpenCompass - INFO - Partitioned into 52 tasks.
01/15 15:04:28 - OpenCompass - DEBUG - Task 0: [chatglm6b/ceval-computer_network]
01/15 15:04:28 - OpenCompass - DEBUG - Task 1: [chatglm6b/ceval-operating_system]
01/15 15:04:28 - OpenCompass - DEBUG - Task 2: [chatglm6b/ceval-computer_architecture]
01/15 15:04:28 - OpenCompass - DEBUG - Task 3: [chatglm6b/ceval-college_programming]
01/15 15:04:28 - OpenCompass - DEBUG - Task 4: [chatglm6b/ceval-college_physics]
01/15 15:04:28 - OpenCompass - DEBUG - Task 5: [chatglm6b/ceval-college_chemistry]
01/15 15:04:28 - OpenCompass - DEBUG - Task 6: [chatglm6b/ceval-advanced_mathematics]
01/15 15:04:28 - OpenCompass - DEBUG - Task 7: [chatglm6b/ceval-probability_and_statistics]
01/15 15:04:28 - OpenCompass - DEBUG - Task 8: [chatglm6b/ceval-discrete_mathematics]
01/15 15:04:28 - OpenCompass - DEBUG - Task 9: [chatglm6b/ceval-electrical_engineer]
01/15 15:04:28 - OpenCompass - DEBUG - Task 10: [chatglm6b/ceval-metrology_engineer]
01/15 15:04:28 - OpenCompass - DEBUG - Task 11: [chatglm6b/ceval-high_school_mathematics]
01/15 15:04:28 - OpenCompass - DEBUG - Task 12: [chatglm6b/ceval-high_school_physics]
01/15 15:04:28 - OpenCompass - DEBUG - Task 13: [chatglm6b/ceval-high_school_chemistry]
01/15 15:04:28 - OpenCompass - DEBUG - Task 14: [chatglm6b/ceval-high_school_biology]
01/15 15:04:28 - OpenCompass - DEBUG - Task 15: [chatglm6b/ceval-middle_school_mathematics]
01/15 15:04:28 - OpenCompass - DEBUG - Task 16: [chatglm6b/ceval-middle_school_biology]
01/15 15:04:28 - OpenCompass - DEBUG - Task 17: [chatglm6b/ceval-middle_school_physics]
01/15 15:04:28 - OpenCompass - DEBUG - Task 18: [chatglm6b/ceval-middle_school_chemistry]
01/15 15:04:28 - OpenCompass - DEBUG - Task 19: [chatglm6b/ceval-veterinary_medicine]
01/15 15:04:28 - OpenCompass - DEBUG - Task 20: [chatglm6b/ceval-college_economics]
01/15 15:04:28 - OpenCompass - DEBUG - Task 21: [chatglm6b/ceval-business_administration]
01/15 15:04:28 - OpenCompass - DEBUG - Task 22: [chatglm6b/ceval-marxism]
01/15 15:04:28 - OpenCompass - DEBUG - Task 23: [chatglm6b/ceval-mao_zedong_thought]
01/15 15:04:28 - OpenCompass - DEBUG - Task 24: [chatglm6b/ceval-education_science]
01/15 15:04:28 - OpenCompass - DEBUG - Task 25: [chatglm6b/ceval-teacher_qualification]
01/15 15:04:28 - OpenCompass - DEBUG - Task 26: [chatglm6b/ceval-high_school_politics]
01/15 15:04:28 - OpenCompass - DEBUG - Task 27: [chatglm6b/ceval-high_school_geography]
01/15 15:04:28 - OpenCompass - DEBUG - Task 28: [chatglm6b/ceval-middle_school_politics]
01/15 15:04:28 - OpenCompass - DEBUG - Task 29: [chatglm6b/ceval-middle_school_geography]
01/15 15:04:28 - OpenCompass - DEBUG - Task 30: [chatglm6b/ceval-modern_chinese_history]
01/15 15:04:28 - OpenCompass - DEBUG - Task 31: [chatglm6b/ceval-ideological_and_moral_cultivation]
01/15 15:04:28 - OpenCompass - DEBUG - Task 32: [chatglm6b/ceval-logic]
01/15 15:04:28 - OpenCompass - DEBUG - Task 33: [chatglm6b/ceval-law]
01/15 15:04:28 - OpenCompass - DEBUG - Task 34: [chatglm6b/ceval-chinese_language_and_literature]
01/15 15:04:28 - OpenCompass - DEBUG - Task 35: [chatglm6b/ceval-art_studies]
01/15 15:04:28 - OpenCompass - DEBUG - Task 36: [chatglm6b/ceval-professional_tour_guide]
01/15 15:04:28 - OpenCompass - DEBUG - Task 37: [chatglm6b/ceval-legal_professional]
01/15 15:04:28 - OpenCompass - DEBUG - Task 38: [chatglm6b/ceval-high_school_chinese]
01/15 15:04:28 - OpenCompass - DEBUG - Task 39: [chatglm6b/ceval-high_school_history]
01/15 15:04:28 - OpenCompass - DEBUG - Task 40: [chatglm6b/ceval-middle_school_history]
01/15 15:04:28 - OpenCompass - DEBUG - Task 41: [chatglm6b/ceval-civil_servant]
01/15 15:04:28 - OpenCompass - DEBUG - Task 42: [chatglm6b/ceval-sports_science]
01/15 15:04:28 - OpenCompass - DEBUG - Task 43: [chatglm6b/ceval-plant_protection]
01/15 15:04:28 - OpenCompass - DEBUG - Task 44: [chatglm6b/ceval-basic_medicine]
01/15 15:04:28 - OpenCompass - DEBUG - Task 45: [chatglm6b/ceval-clinical_medicine]
01/15 15:04:28 - OpenCompass - DEBUG - Task 46: [chatglm6b/ceval-urban_and_rural_planner]
01/15 15:04:28 - OpenCompass - DEBUG - Task 47: [chatglm6b/ceval-accountant]
01/15 15:04:28 - OpenCompass - DEBUG - Task 48: [chatglm6b/ceval-fire_engineer]
01/15 15:04:28 - OpenCompass - DEBUG - Task 49: [chatglm6b/ceval-environmental_impact_assessment_engineer]
01/15 15:04:28 - OpenCompass - DEBUG - Task 50: [chatglm6b/ceval-tax_accountant]
01/15 15:04:28 - OpenCompass - DEBUG - Task 51: [chatglm6b/ceval-physician]
01/15 15:04:28 - OpenCompass - DEBUG - Get class `LocalRunner` from "runner" registry in "opencompass"
01/15 15:04:28 - OpenCompass - DEBUG - An `LocalRunner` instance is built from registry, and its implementation can be found in opencompass.runners.local
01/15 15:04:28 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:28 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:04:31 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:31 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:04:33 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:33 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:04:35 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:35 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:04:38 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:38 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:04:40 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:40 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:04:42 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:42 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:04:45 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:45 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:04:47 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:47 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:04:49 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:49 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:04:51 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:51 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:04:54 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:54 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:04:56 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:56 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:04:58 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:04:58 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:01 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:01 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:03 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:03 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:05 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:05 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:07 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:07 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:10 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:10 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:12 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:12 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:14 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:14 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:17 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:17 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:19 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:19 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:21 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:21 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:24 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:24 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:26 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:26 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:28 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:28 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:31 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:31 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:33 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:33 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:35 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:35 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:37 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:37 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:40 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:40 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:42 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:42 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:45 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:45 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:47 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:47 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:50 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:50 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:52 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:52 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:54 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:54 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:57 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:57 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:05:59 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:05:59 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:06:01 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:06:01 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:06:04 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:06:04 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:06:06 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:06:06 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:06:09 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:06:09 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:06:11 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:06:11 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:06:13 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:06:13 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:06:16 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:06:16 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:06:18 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:06:18 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:06:21 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:06:21 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:06:23 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:06:23 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:06:25 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:06:25 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:06:28 - OpenCompass - DEBUG - Get class `OpenICLEvalTask` from "task" registry in "opencompass"
01/15 15:06:28 - OpenCompass - DEBUG - An `OpenICLEvalTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_eval
01/15 15:06:30 - OpenCompass - DEBUG - An `DefaultSummarizer` instance is built from registry, and its implementation can be found in opencompass.summarizers.default
dataset                                         version    metric    mode    chatglm6b
----------------------------------------------  ---------  --------  ------  -----------
ceval-computer_network                          -          -         -       -
ceval-operating_system                          -          -         -       -
ceval-computer_architecture                     -          -         -       -
ceval-college_programming                       -          -         -       -
ceval-college_physics                           -          -         -       -
ceval-college_chemistry                         -          -         -       -
ceval-advanced_mathematics                      -          -         -       -
ceval-probability_and_statistics                -          -         -       -
ceval-discrete_mathematics                      -          -         -       -
ceval-electrical_engineer                       -          -         -       -
ceval-metrology_engineer                        -          -         -       -
ceval-high_school_mathematics                   -          -         -       -
ceval-high_school_physics                       -          -         -       -
ceval-high_school_chemistry                     -          -         -       -
ceval-high_school_biology                       -          -         -       -
ceval-middle_school_mathematics                 -          -         -       -
ceval-middle_school_biology                     -          -         -       -
ceval-middle_school_physics                     -          -         -       -
ceval-middle_school_chemistry                   -          -         -       -
ceval-veterinary_medicine                       -          -         -       -
ceval-college_economics                         -          -         -       -
ceval-business_administration                   -          -         -       -
ceval-marxism                                   -          -         -       -
ceval-mao_zedong_thought                        -          -         -       -
ceval-education_science                         -          -         -       -
ceval-teacher_qualification                     -          -         -       -
ceval-high_school_politics                      -          -         -       -
ceval-high_school_geography                     -          -         -       -
ceval-middle_school_politics                    -          -         -       -
ceval-middle_school_geography                   -          -         -       -
ceval-modern_chinese_history                    -          -         -       -
ceval-ideological_and_moral_cultivation         -          -         -       -
ceval-logic                                     -          -         -       -
ceval-law                                       -          -         -       -
ceval-chinese_language_and_literature           -          -         -       -
ceval-art_studies                               -          -         -       -
ceval-professional_tour_guide                   -          -         -       -
ceval-legal_professional                        -          -         -       -
ceval-high_school_chinese                       -          -         -       -
ceval-high_school_history                       -          -         -       -
ceval-middle_school_history                     -          -         -       -
ceval-civil_servant                             -          -         -       -
ceval-sports_science                            -          -         -       -
ceval-plant_protection                          -          -         -       -
ceval-basic_medicine                            -          -         -       -
ceval-clinical_medicine                         -          -         -       -
ceval-urban_and_rural_planner                   -          -         -       -
ceval-accountant                                -          -         -       -
ceval-fire_engineer                             -          -         -       -
ceval-environmental_impact_assessment_engineer  -          -         -       -
ceval-tax_accountant                            -          -         -       -
ceval-physician                                 -          -         -       -
01/15 15:06:30 - OpenCompass - INFO - write summary to D:\code\opencompass\outputs\default\20240115_150424\summary\summary_20240115_150424.txt
01/15 15:06:30 - OpenCompass - INFO - write csv to D:\code\opencompass\outputs\default\20240115_150424\summary\summary_20240115_150424.csv

(opencompass0112) D:\code\opencompass>

为什么 datasets = [*siqa_datasets]，log是ceval? 提供一个完整的config文件吧



> 为什么 datasets = [*siqa_datasets]，log是ceval? 提供一个完整的config文件吧

from mmengine.config import read_base

with read_base():
    from .datasets.ceval.ceval_gen import ceval_datasets

datasets = [*ceval_datasets]

from opencompass.models import HuggingFaceCausalLM


models = [
    dict(
        type=HuggingFaceCausalLM,
        abbr='chatglm6b',    # 运行完结果展示的名称
        path='D:\\code\\ChatGLM-6B-main\\THUDM\\chatglm2-6b',    # 模型路径
        tokenizer_path='D:\\code\\ChatGLM-6B-main\\THUDM\\chatglm2-6b',    # 分词器路径
        model_kwargs=dict(
            device_map='auto',
            trust_remote_code=True
        ),
        tokenizer_kwargs=dict(
            padding_side='left',
            truncation_side='left',
            trust_remote_code=True,
            use_fast=False
        ),
        pad_token_id=151643,
        max_out_len=100,
        max_seq_len=2048,
        batch_size=16,
        run_cfg=dict(num_gpus=0),    
        #run_cfg=dict(num_gpus=8, num_procs=8),    
    )
]

why use num_gpus=0? Please try num_gpus=1