Repository: open-compass/opencompass
Issue: [Bug] 无法测评openai接口格式部署的模型
Root cause: Validation Gap

### Prerequisite

- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I'm evaluating with the officially supported tasks/models/datasets.

### Environment

{'CUDA available': False,
 'GCC': 'n/a',
 'MMEngine': '0.10.3',
 'MUSA available': False,
 'OpenCV': '4.9.0',
 'PyTorch': '2.1.2',
 'PyTorch compiling details': 'PyTorch built with:\n'
                              '  - C++ Version: 199711\n'
                              '  - MSVC 192930151\n'
                              '  - Intel(R) Math Kernel Library Version '
                              '2020.0.2 Product Build 20200624 for Intel(R) 64 '
                              'architecture applications\n'
                              '  - Intel(R) MKL-DNN v3.1.1 (Git Hash '
                              '64f6bcbcbab628e96f33a62c3e975f8535a7bde4)\n'
                              '  - OpenMP 2019\n'
                              '  - LAPACK is enabled (usually provided by '
                              'MKL)\n'
                              '  - CPU capability usage: AVX2\n'
                              '  - Build settings: BLAS_INFO=mkl, '
                              'BUILD_TYPE=Release, '
                              'CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, '
                              'CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /bigobj '
                              '/FS -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO '
                              '-DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER '
                              '-DUSE_FBGEMM -DUSE_XNNPACK '
                              '-DSYMBOLICATE_MOBILE_DEBUG_HANDLE /utf-8 '
                              '/wd4624 /wd4068 /wd4067 /wd4267 /wd4661 /wd4717 '
                              '/wd4244 /wd4804 /wd4273, LAPACK_INFO=mkl, '
                              'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, '
                              'PERF_WITH_AVX512=1, '
                              'TORCH_DISABLE_GPU_ASSERTS=OFF, '
                              'TORCH_VERSION=2.1.2, USE_CUDA=0, USE_CUDNN=OFF, '
                              'USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, '
                              'USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, '
                              'USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, '
                              'USE_OPENMP=ON, USE_ROCM=OFF, \n',
 'Python': '3.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:27:34) '
           '[MSC v.1937 64 bit (AMD64)]',
 'TorchVision': '0.16.2',
 'numpy_random_seed': 2147483648,
 'opencompass': '0.2.2+',
 'sys.platform': 'win32'}

### Reproduces the problem - code/configuration sample

from mmengine.config import read_base
from opencompass.models import OpenAI
from opencompass.partitioners import NaivePartitioner
from opencompass.runners import LocalRunner
from opencompass.tasks import OpenICLInferTask

with read_base():
    # from .datasets.collections.chat_medium import datasets
    from .summarizers.medium import summarizer
    #from .datasets.MedBench.medbench_gen import medbench_datasets
    from .datasets.ceval.ceval_gen import ceval_datasets


datasets = [
    *ceval_datasets,
]




api_meta_template = dict(
    round=[
            dict(role='HUMAN', api_role='HUMAN'),
            dict(role='BOT', api_role='BOT', generate=True),
    ],
)

models = [
    dict(
        abbr='qwen-chat-14b',
        type=OpenAI, 
        path='qwen-chat-14b',
        key='None',  # The key will be obtained from $OPENAI_API_KEY, but you can write down your key here as well
        openai_api_base='https://172.22.227.18:9997/v1/chat/completions',
        max_seq_len=2048, 
        meta_template=api_meta_template,
        query_per_second=1,
        max_out_len=2048,
        batch_size=1,
        
        
        #run_cfg=dict(num_gpus=0),
        ),
]

infer = dict(
    partitioner=dict(type=NaivePartitioner),
    runner=dict(
        type=LocalRunner,
        max_num_workers=4,
        task=dict(type=OpenICLInferTask)),
)


### Reproduces the problem - command or script

python run.py configs/xinference.py -w outputs/demo

### Reproduces the problem - error message

outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-marxism.out
02/18 16:33:13 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-college_economics] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-college_economics.out
02/18 16:33:13 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-business_administration] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-business_administration.out
launch OpenICLInfer[qwen-chat-14b/ceval-teacher_qualification] on CPU 
launch OpenICLInfer[qwen-chat-14b/ceval-high_school_politics] on CPU 
launch OpenICLInfer[qwen-chat-14b/ceval-high_school_geography] on CPU 
 46%|█████████████████████████████████████████████████████████▋                                                                   | 24/52 [01:39<02:07,  4.56s/it]02/18 16:33:29 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-education_science] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-education_science.out
launch OpenICLInfer[qwen-chat-14b/ceval-middle_school_politics] on CPU
 48%|████████████████████████████████████████████████████████████                                                                 | 25/52 [01:55<02:11,  4.88s/it0 
2/18 16:33:29 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-teacher_qualification] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-teacher_qualification.out
                                                                                                                                                                  02/18 16:33:29 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-high_school_politics] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-high_school_politics.out
launch OpenICLInfer[qwen-chat-14b/ceval-middle_school_geography] on CPU 
 50%|██████████████████████████████████████████████████████████████▌                                                              | 26/52 [01:55<02:06,  4.88s/it]02/18 16:33:29 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-high_school_geography] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-high_school_geography.out
launch OpenICLInfer[qwen-chat-14b/ceval-modern_chinese_history] on CPU 
launch OpenICLInfer[qwen-chat-14b/ceval-ideological_and_moral_cultivation] on CPU 
 54%|███████████████████████████████████████████████████████████████████▎                                                         | 28/52 [02:10<01:27,  3.63s/it]02/18 16:33:47 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-ideological_and_moral_cultivation] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-ideological_and_moral_cultivation.out
launch OpenICLInfer[qwen-chat-14b/ceval-logic] on CPU
 56%|█████████████████████████████████████████████████████████████████████▋                                                       | 29/52 [02:13<01:57,  5.11s/it0 
2/18 16:33:48 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-middle_school_geography] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-middle_school_geography.out
02/18 16:33:48 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-middle_school_politics] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-middle_school_politics.out
                                                                                                                                                                  02/18 16:33:48 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-modern_chinese_history] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-modern_chinese_history.out
launch OpenICLInfer[qwen-chat-14b/ceval-law] on CPU 
launch OpenICLInfer[qwen-chat-14b/ceval-chinese_language_and_literature] on CPU 
launch OpenICLInfer[qwen-chat-14b/ceval-art_studies] on CPU 
 62%|████████████████████████████████████████████████████████████████████████████▉                                                | 32/52 [02:14<01:04,  3.24s/it]02/18 16:34:04 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-law] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-law.out
02/18 16:34:04 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-art_studies] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-art_studies.out
launch OpenICLInfer[qwen-chat-14b/ceval-professional_tour_guide] on CPU
02/18 16:34:04 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-logic] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-logic.out
launch OpenICLInfer[qwen-chat-14b/ceval-legal_professional] on CPU 
 65%|█████████████████████████████████████████████████████████████████████████████████▋                                           | 34/52 [02:30<01:21,  4.55s/it]02/18 16:34:04 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-chinese_language_and_literature] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-chinese_language_and_literature.out
launch OpenICLInfer[qwen-chat-14b/ceval-high_school_chinese] on CPU 
launch OpenICLInfer[qwen-chat-14b/ceval-high_school_history] on CPU 
 69%|██████████████████████████████████████████████████████████████████████████████████████▌                                      | 36/52 [02:40<01:01,  3.86s/it]02/18 16:34:25 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-high_school_history] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-high_school_history.out
launch OpenICLInfer[qwen-chat-14b/ceval-middle_school_history] on CPU
 71%|████████████████████████████████████████████████████████████████████████████████████████▉                                    | 37/52 [02:51<01:28,  5.87s/it0 
2/18 16:34:25 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-professional_tour_guide] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-professional_tour_guide.out
02/18 16:34:25 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-legal_professional] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-legal_professional.out
02/18 16:34:25 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-high_school_chinese] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-high_school_chinese.out
launch OpenICLInfer[qwen-chat-14b/ceval-civil_servant] on CPU 
launch OpenICLInfer[qwen-chat-14b/ceval-sports_science] on CPU 
launch OpenICLInfer[qwen-chat-14b/ceval-plant_protection] on CPU 
 77%|████████████████████████████████████████████████████████████████████████████████████████████████▏                            | 40/52 [03:11<00:49,  4.09s/it]02/18 16:34:46 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-sports_science] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-sports_science.out
launch OpenICLInfer[qwen-chat-14b/ceval-basic_medicine] on CPU
 79%|██████████████████████████████████████████████████████████████████████████████████████████████████▌                          | 41/52 [03:12<01:06,  6.02s/it0 
2/18 16:34:46 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-plant_protection] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-plant_protection.out
02/18 16:34:46 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-middle_school_history] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-middle_school_history.out
02/18 16:34:46 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-civil_servant] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-civil_servant.out
launch OpenICLInfer[qwen-chat-14b/ceval-clinical_medicine] on CPU 
launch OpenICLInfer[qwen-chat-14b/ceval-urban_and_rural_planner] on CPU 
launch OpenICLInfer[qwen-chat-14b/ceval-accountant] on CPU 
 85%|█████████████████████████████████████████████████████████████████████████████████████████████████████████▊                   | 44/52 [03:31<00:29,  3.66s/it]02/18 16:35:08 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-accountant] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-accountant.out
launch OpenICLInfer[qwen-chat-14b/ceval-fire_engineer] on CPU
 87%|████████████████████████████████████████████████████████████████████████████████████████████████████████████▏                | 45/52 [03:33<00:45,  6.49s/it0 
2/18 16:35:08 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-clinical_medicine] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-clinical_medicine.out
02/18 16:35:08 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-basic_medicine] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-basic_medicine.out
                                                                                                                                                                  02/18 16:35:08 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-urban_and_rural_planner] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-urban_and_rural_planner.out
launch OpenICLInfer[qwen-chat-14b/ceval-environmental_impact_assessment_engineer] on CPU 
launch OpenICLInfer[qwen-chat-14b/ceval-tax_accountant] on CPU 
launch OpenICLInfer[qwen-chat-14b/ceval-physician] on CPU 
 92%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍         | 48/52 [03:51<00:10,  2.71s/it]02/18 16:35:28 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-physician] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-physician.out
 94%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊       | 49/52 [03:54<00:16,  5.54s/it]02/18 16:35:28 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-environmental_impact_assessment_engineer] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-environmental_impact_assessment_engineer.out
02/18 16:35:28 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-fire_engineer] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-fire_engineer.out
02/18 16:35:28 - OpenCompass - WARNING - task OpenICLInfer[qwen-chat-14b/ceval-tax_accountant] fail, see
outputs/demo\20240218_163133\logs/infer\qwen-chat-14b\ceval-tax_accountant.out
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 52/52 [03:54<00:00,  4.51s/it]
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-computer_network] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-operating_system] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-computer_architecture] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-college_programming] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-college_physics] failed with 
code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-college_chemistry] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-advanced_mathematics] failed 
with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-probability_and_statistics] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-discrete_mathematics] failed 
with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-electrical_engineer] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-metrology_engineer] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-high_school_mathematics] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-high_school_physics] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-high_school_chemistry] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-high_school_biology] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-middle_school_mathematics] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-middle_school_biology] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-middle_school_physics] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-middle_school_chemistry] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-veterinary_medicine] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-college_economics] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-business_administration] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-marxism] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-mao_zedong_thought] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-education_science] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-teacher_qualification] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-high_school_politics] failed 
with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-high_school_geography] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-middle_school_politics] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-middle_school_geography] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-modern_chinese_history] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-ideological_and_moral_cultivation] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-logic] failed with code 9009 
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-law] failed with code 9009   
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-chinese_language_and_literature] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-art_studies] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-professional_tour_guide] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-legal_professional] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-high_school_chinese] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-high_school_history] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-middle_school_history] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-civil_servant] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-sports_science] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-plant_protection] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-basic_medicine] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-clinical_medicine] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-urban_and_rural_planner] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-accountant] failed with code 
9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-fire_engineer] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-environmental_impact_assessment_engineer] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-tax_accountant] failed with code 9009
02/18 16:35:28 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLInfer[qwen-chat-14b/ceval-physician] failed with code 9009
02/18 16:35:28 - OpenCompass - INFO - Partitioned into 52 tasks.
launch OpenICLEval[qwen-chat-14b/ceval-computer_network] on CPU
launch OpenICLEval[qwen-chat-14b/ceval-operating_system] on CPU
launch OpenICLEval[qwen-chat-14b/ceval-computer_architecture] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-college_programming] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-college_physics] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-college_chemistry] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-advanced_mathematics] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-probability_and_statistics] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-discrete_mathematics] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-electrical_engineer] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-metrology_engineer] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-high_school_mathematics] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-high_school_physics] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-high_school_chemistry] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-high_school_biology] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-middle_school_mathematics] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-middle_school_biology] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-middle_school_physics] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-middle_school_chemistry] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-veterinary_medicine] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-college_economics] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-business_administration] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-marxism] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-mao_zedong_thought] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-education_science] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-teacher_qualification] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-high_school_politics] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-high_school_geography] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-middle_school_politics] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-middle_school_geography] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-modern_chinese_history] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-ideological_and_moral_cultivation] on CPU 
  0%|                                                                                                                                      | 0/52 [00:37<?, ?it/s]02/18 16:38:11 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-computer_network] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-computer_network.out
launch OpenICLEval[qwen-chat-14b/ceval-logic] on CPU 
  2%|██▎                                                                                                                        | 1/52 [02:42<2:18:03, 162.42s/it]02/18 16:38:13 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-electrical_engineer] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-electrical_engineer.out
launch OpenICLEval[qwen-chat-14b/ceval-law] on CPU 
  4%|████▊                                                                                                                         | 2/52 [02:44<56:45, 68.10s/it]02/18 16:38:14 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-college_physics] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-college_physics.out
launch OpenICLEval[qwen-chat-14b/ceval-chinese_language_and_literature] on CPU
  6%|███████▎                                                                                                                      | 3/52 [02:45<30:40, 37.56s/it]02/18 16:38:14 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-discrete_mathematics] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-discrete_mathematics.out
launch OpenICLEval[qwen-chat-14b/ceval-art_studies] on CPU 
  8%|█████████▋                                                                                                                    | 4/52 [02:46<18:18, 22.89s/it]02/18 16:38:16 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-operating_system] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-operating_system.out
launch OpenICLEval[qwen-chat-14b/ceval-professional_tour_guide] on CPU
 10%|████████████                                                                                                                  | 5/52 [02:47<11:47, 15.05s/it]02/18 16:38:16 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-college_chemistry] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-college_chemistry.out
02/18 16:38:17 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-business_administration] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-business_administration.out
launch OpenICLEval[qwen-chat-14b/ceval-legal_professional] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-high_school_chinese] on CPU 
 13%|████████████████▉                                                                                                             | 7/52 [02:49<05:21,  7.14s/it]02/18 16:38:19 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-college_economics] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-college_economics.out
launch OpenICLEval[qwen-chat-14b/ceval-high_school_history] on CPU 
 15%|███████████████████▍                                                                                                          | 8/52 [02:50<03:50,  5.24s/it]02/18 16:38:19 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-middle_school_biology] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-middle_school_biology.out
02/18 16:38:20 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-high_school_physics] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-high_school_physics.out
                                                                                                                                                                  02/18 16:38:20 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-teacher_qualification] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-teacher_qualification.out
launch OpenICLEval[qwen-chat-14b/ceval-middle_school_history] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-civil_servant] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-sports_science] on CPU 
 21%|██████████████████████████▍                                                                                                  | 11/52 [02:53<01:26,  2.11s/it]02/18 16:38:22 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-high_school_chemistry] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-high_school_chemistry.out
launch OpenICLEval[qwen-chat-14b/ceval-plant_protection] on CPU 
 23%|████████████████████████████▊                                                                                                | 12/52 [02:55<01:23,  2.09s/it]02/18 16:38:26 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-education_science] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-education_science.out
 25%|███████████████████████████████▎                                                                                             | 13/52 [02:58<01:44,  2.68s/it]02/18 16:38:28 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-middle_school_physics] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-middle_school_physics.out
launch OpenICLEval[qwen-chat-14b/ceval-basic_medicine] on CPU 
 27%|█████████████████████████████████▋                                                                                           | 14/52 [03:00<01:17,  2.04s/it]02/18 16:38:29 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-middle_school_geography] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-middle_school_geography.out
launch OpenICLEval[qwen-chat-14b/ceval-clinical_medicine] on CPU 
 29%|████████████████████████████████████                                                                                         | 15/52 [03:03<01:01,  1.65s/it]02/18 16:38:32 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-middle_school_politics] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-middle_school_politics.out
launch OpenICLEval[qwen-chat-14b/ceval-urban_and_rural_planner] on CPU 
launch OpenICLEval[qwen-chat-14b/ceval-accountant] on CPU 
 31%|██████████████████████████████████████▍                                                                                      | 16/52 [03:07<01:17,  2.16s/it]02/18 16:39:06 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-computer_architecture] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-computer_architecture.out
launch OpenICLEval[qwen-chat-14b/ceval-fire_engineer] on CPU 
 33%|████████████████████████████████████████▊                                                                                    | 17/52 [03:40<05:59, 10.27s/it]02/18 16:39:22 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-college_programming] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-college_programming.out
launch OpenICLEval[qwen-chat-14b/ceval-environmental_impact_assessment_engineer] on CPU 
 35%|███████████████████████████████████████████▎                                                                                 | 18/52 [03:54<06:34, 11.59s/it]02/18 16:39:45 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-middle_school_chemistry] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-middle_school_chemistry.out
launch OpenICLEval[qwen-chat-14b/ceval-tax_accountant] on CPU 
 37%|█████████████████████████████████████████████▋                                                                               | 19/52 [04:17<08:07, 14.77s/it]02/18 16:39:52 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-advanced_mathematics] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-advanced_mathematics.out
launch OpenICLEval[qwen-chat-14b/ceval-physician] on CPU 
 38%|████████████████████████████████████████████████                                                                             | 20/52 [04:24<06:38, 12.47s/it]02/18 16:39:53 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-high_school_biology] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-high_school_biology.out
02/18 16:39:53 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-middle_school_mathematics] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-middle_school_mathematics.out
 40%|██████████████████████████████████████████████████▍                                                                          | 21/52 [04:25<04:48,  9.32s/it]02/18 16:39:54 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-high_school_mathematics] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-high_school_mathematics.out
 44%|███████████████████████████████████████████████████████▎                                                                     | 23/52 [04:26<02:21,  4.89s/it]02/18 16:39:58 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-metrology_engineer] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-metrology_engineer.out
 46%|█████████████████████████████████████████████████████████▋                                                                   | 24/52 [04:30<02:14,  4.79s/it]02/18 16:40:01 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-marxism] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-marxism.out
 48%|████████████████████████████████████████████████████████████                                                                 | 25/52 [04:32<01:46,  3.95s/it]02/18 16:40:03 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-high_school_politics] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-high_school_politics.out
02/18 16:40:04 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-probability_and_statistics] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-probability_and_statistics.out
 52%|████████████████████████████████████████████████████████████████▉                                                            | 27/52 [04:35<01:03,  2.55s/it]02/18 16:40:04 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-mao_zedong_thought] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-mao_zedong_thought.out
02/18 16:40:04 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-veterinary_medicine] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-veterinary_medicine.out
 56%|█████████████████████████████████████████████████████████████████████▋                                                       | 29/52 [04:36<00:33,  1.46s/it]02/18 16:40:05 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-high_school_geography] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-high_school_geography.out
 58%|████████████████████████████████████████████████████████████████████████                                                     | 30/52 [04:37<00:27,  1.27s/it]02/18 16:40:06 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-ideological_and_moral_cultivation] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-ideological_and_moral_cultivation.out
02/18 16:40:06 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-modern_chinese_history] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-modern_chinese_history.out
 62%|████████████████████████████████████████████████████████████████████████████▉                                                | 32/52 [04:38<00:14,  1.40it/s]02/18 16:40:17 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-high_school_history] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-high_school_history.out
 63%|███████████████████████████████████████████████████████████████████████████████▎                                             | 33/52 [04:48<00:59,  3.14s/it]02/18 16:40:19 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-logic] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-logic.out
 65%|█████████████████████████████████████████████████████████████████████████████████▋                                           | 34/52 [04:50<00:50,  2.83s/it]02/18 16:40:19 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-civil_servant] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-civil_servant.out
 67%|████████████████████████████████████████████████████████████████████████████████████▏                                        | 35/52 [04:50<00:36,  2.15s/it]02/18 16:40:19 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-professional_tour_guide] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-professional_tour_guide.out
02/18 16:40:19 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-law] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-law.out
 71%|████████████████████████████████████████████████████████████████████████████████████████▉                                    | 37/52 [04:50<00:18,  1.27s/it]02/18 16:40:20 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-high_school_chinese] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-high_school_chinese.out
 73%|███████████████████████████████████████████████████████████████████████████████████████████▎                                 | 38/52 [04:51<00:14,  1.02s/it]02/18 16:40:20 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-art_studies] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-art_studies.out
 75%|█████████████████████████████████████████████████████████████████████████████████████████████▊                               | 39/52 [04:51<00:10,  1.26it/s]02/18 16:40:20 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-chinese_language_and_literature] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-chinese_language_and_literature.out
02/18 16:40:20 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-middle_school_history] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-middle_school_history.out
 77%|████████████████████████████████████████████████████████████████████████████████████████████████▏                            | 40/52 [04:51<00:07,  1.52it/s]02/18 16:40:21 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-basic_medicine] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-basic_medicine.out
02/18 16:40:21 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-sports_science] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-sports_science.out
 81%|████████████████████████████████████████████████████████████████████████████████████████████████████▉                        | 42/52 [04:52<00:06,  1.63it/s]02/18 16:40:21 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-clinical_medicine] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-clinical_medicine.out
02/18 16:40:21 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-legal_professional] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-legal_professional.out
02/18 16:40:21 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-accountant] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-accountant.out
 88%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████▌              | 46/52 [04:53<00:01,  3.04it/s]02/18 16:40:22 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-plant_protection] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-plant_protection.out
02/18 16:40:22 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-urban_and_rural_planner] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-urban_and_rural_planner.out
 92%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍         | 48/52 [04:54<00:01,  2.68it/s]02/18 16:40:24 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-environmental_impact_assessment_engineer] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-environmental_impact_assessment_engineer.out
 94%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊       | 49/52 [04:55<00:01,  2.04it/s]02/18 16:40:24 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-fire_engineer] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-fire_engineer.out
02/18 16:40:24 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-tax_accountant] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-tax_accountant.out
02/18 16:40:24 - OpenCompass - WARNING - task OpenICLEval[qwen-chat-14b/ceval-physician] fail, see
outputs/demo\20240218_163133\logs/eval\qwen-chat-14b\ceval-physician.out
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 52/52 [04:55<00:00,  5.68s/it] 
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-computer_network] failed with 
code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-operating_system] failed with 
code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-computer_architecture] failed 
with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-college_programming] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-college_physics] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-college_chemistry] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-advanced_mathematics] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-probability_and_statistics] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-discrete_mathematics] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-electrical_engineer] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-metrology_engineer] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-high_school_mathematics] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-high_school_physics] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-high_school_chemistry] failed 
with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-high_school_biology] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-middle_school_mathematics] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-middle_school_biology] failed 
with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-middle_school_physics] failed 
with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-middle_school_chemistry] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-veterinary_medicine] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-college_economics] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-business_administration] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-marxism] failed with code 900902/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-mao_zedong_thought] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-education_science] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-teacher_qualification] failed 
with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-high_school_politics] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-high_school_geography] failed 
with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-middle_school_politics] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-middle_school_geography] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-modern_chinese_history] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-ideological_and_moral_cultivation] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-logic] failed with code 9009  
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-law] failed with code 9009    
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-chinese_language_and_literature] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-art_studies] failed with code 
9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-professional_tour_guide] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-legal_professional] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-high_school_chinese] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-high_school_history] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-middle_school_history] failed 
with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-civil_servant] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-sports_science] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-plant_protection] failed with 
code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-basic_medicine] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-clinical_medicine] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-urban_and_rural_planner] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-accountant] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-fire_engineer] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-environmental_impact_assessment_engineer] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-tax_accountant] failed with code 9009
02/18 16:40:24 - OpenCompass - ERROR - D:\opencompass\opencompass\runners\base.py - summarize - 63 - OpenICLEval[qwen-chat-14b/ceval-physician] failed with code 9009
dataset                                 version    metric    mode    qwen-chat-14b
--------------------------------------  ---------  --------  ------  ---------------
--------- 考试 Exam ---------           -          -         -       -
ceval                                   -          -         -       -
agieval                                 -          -         -       -
mmlu                                    -          -         -       -
GaokaoBench                             -          -         -       -
ARC-c                                   -          -         -       -
--------- 语言 Language ---------       -          -         -       -
WiC                                     -          -         -       -
summedits                               -          -         -       -
chid-dev                                -          -         -       -
afqmc-dev                               -          -         -       -
bustm-dev                               -          -         -       -
cluewsc-dev                             -          -         -       -
WSC                                     -          -         -       -
winogrande                              -          -         -       -
flores_100                              -          -         -       -
--------- 知识 Knowledge ---------      -          -         -       -
BoolQ                                   -          -         -       -
commonsense_qa                          -          -         -       -
nq                                      -          -         -       -
triviaqa                                -          -         -       -
--------- 推理 Reasoning ---------      -          -         -       -
cmnli                                   -          -         -       -
ocnli                                   -          -         -       -
ocnli_fc-dev                            -          -         -       -
AX_b                                    -          -         -       -
AX_g                                    -          -         -       -
CB                                      -          -         -       -
RTE                                     -          -         -       -
story_cloze                             -          -         -       -
COPA                                    -          -         -       -
ReCoRD                                  -          -         -       -
hellaswag                               -          -         -       -
piqa                                    -          -         -       -
siqa                                    -          -         -       -
strategyqa                              -          -         -       -
math                                    -          -         -       -
gsm8k                                   -          -         -       -
TheoremQA                               -          -         -       -
openai_humaneval                        -          -         -       -
mbpp                                    -          -         -       -
bbh                                     -          -         -       -
--------- 理解 Understanding ---------  -          -         -       -
C3                                      -          -         -       -
CMRC_dev                                -          -         -       -
DRCD_dev                                -          -         -       -
MultiRC                                 -          -         -       -
race-middle                             -          -         -       -
race-high                               -          -         -       -
openbookqa_fact                         -          -         -       -
csl_dev                                 -          -         -       -
lcsts                                   -          -         -       -
Xsum                                    -          -         -       -
eprstmt-dev                             -          -         -       -
lambada                                 -          -         -       -
tnews-dev                               -          -         -       -
02/18 16:40:24 - OpenCompass - INFO - write summary to D:\opencompass\outputs\demo\20240218_163133\summary\summary_20240218_163133.txt
02/18 16:40:24 - OpenCompass - INFO - write csv to D:\opencompass\outputs\demo\20240218_163133\summary\summary_20240218_163133.csv

### Other information

用xinference部署的qwen-14b测试，xinference支持openai接口格式。
.out文件都是空的，启用debug模式的话没有错误提示
![image](https://github.com/open-compass/opencompass/assets/50295520/8c1e8106-e89c-4d29-9a10-a6a74510462e)
![image](https://github.com/open-compass/opencompass/assets/50295520/ca0521ee-62ca-4866-a5bd-1de7d494edbc)

postman如上图是可以调通模型接口的

Please check the `failed with code 9009`

What does `failed with code 9009` mean and how can i solve it? I cant get any info about it.

Maybe you can use `--debug` for more error information

At first i used --debug in windows and got no error info, then i tried in linux and got the following output
```
02/19 14:04:58 - OpenCompass - DEBUG - Task 40: [qwen-chat-14b/ceval-middle_school_history]
02/19 14:04:58 - OpenCompass - DEBUG - Task 41: [qwen-chat-14b/ceval-civil_servant]
02/19 14:04:58 - OpenCompass - DEBUG - Task 42: [qwen-chat-14b/ceval-sports_science]
02/19 14:04:58 - OpenCompass - DEBUG - Task 43: [qwen-chat-14b/ceval-plant_protection]
02/19 14:04:58 - OpenCompass - DEBUG - Task 44: [qwen-chat-14b/ceval-basic_medicine]
02/19 14:04:58 - OpenCompass - DEBUG - Task 45: [qwen-chat-14b/ceval-clinical_medicine]
02/19 14:04:58 - OpenCompass - DEBUG - Task 46: [qwen-chat-14b/ceval-urban_and_rural_planner]
02/19 14:04:58 - OpenCompass - DEBUG - Task 47: [qwen-chat-14b/ceval-accountant]
02/19 14:04:58 - OpenCompass - DEBUG - Task 48: [qwen-chat-14b/ceval-fire_engineer]
02/19 14:04:58 - OpenCompass - DEBUG - Task 49: [qwen-chat-14b/ceval-environmental_impact_assessment_engineer]
02/19 14:04:58 - OpenCompass - DEBUG - Task 50: [qwen-chat-14b/ceval-tax_accountant]
02/19 14:04:58 - OpenCompass - DEBUG - Task 51: [qwen-chat-14b/ceval-physician]
02/19 14:04:58 - OpenCompass - DEBUG - Modules of opencompass's runner registry have been automatically imported from opencompass.runners
02/19 14:04:58 - OpenCompass - DEBUG - Get class `LocalRunner` from "runner" registry in "opencompass"
02/19 14:04:58 - OpenCompass - DEBUG - An `LocalRunner` instance is built from registry, and its implementation can be found in opencompass.runners.local
02/19 14:04:58 - OpenCompass - DEBUG - Modules of opencompass's task registry have been automatically imported from opencompass.tasks
02/19 14:04:58 - OpenCompass - DEBUG - Get class `OpenICLInferTask` from "task" registry in "opencompass"
02/19 14:04:58 - OpenCompass - DEBUG - An `OpenICLInferTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_infer
02/19 14:05:04 - OpenCompass - INFO - Task [qwen-chat-14b/ceval-computer_network]
02/19 14:05:06 - OpenCompass - INFO - Start inferencing [qwen-chat-14b/ceval-computer_network]
100%|████████████████████████████████████████████████████████████████████████████████████| 19/19 [00:00<00:00, 524288.00it/s]
Traceback (most recent call last):
  File "/home/jingtianran/nlp/opencompass/opencompass/tasks/openicl_infer.py", line 153, in <module>
    inferencer.run()
  File "/home/jingtianran/nlp/opencompass/opencompass/tasks/openicl_infer.py", line 81, in run
    self._inference()
  File "/home/jingtianran/nlp/opencompass/opencompass/tasks/openicl_infer.py", line 131, in _inference
    inferencer.inference(retriever,
  File "/home/jingtianran/nlp/opencompass/opencompass/openicl/icl_inferencer/icl_gen_inferencer.py", line 97, in inference
    prompt_list = self.get_generation_prompt_list_from_retriever_indices(
  File "/home/jingtianran/nlp/opencompass/opencompass/openicl/icl_inferencer/icl_gen_inferencer.py", line 201, in get_generation_prompt_list_from_retriever_indices
    prompt_token_num = self.model.get_token_len_from_template(
  File "/home/jingtianran/nlp/opencompass/opencompass/models/base.py", line 189, in get_token_len_from_template
    token_lens = [self.get_token_len(prompt) for prompt in prompts]
  File "/home/jingtianran/nlp/opencompass/opencompass/models/base.py", line 189, in <listcomp>
    token_lens = [self.get_token_len(prompt) for prompt in prompts]
  File "/home/jingtianran/nlp/opencompass/opencompass/models/openai_api.py", line 267, in get_token_len
    enc = self.tiktoken.encoding_for_model(self.path)
  File "/home/jingtianran/anaconda3/envs/opencompass/lib/python3.10/site-packages/tiktoken/model.py", line 97, in encoding_for_model
    return get_encoding(encoding_name_for_model(model_name))
  File "/home/jingtianran/anaconda3/envs/opencompass/lib/python3.10/site-packages/tiktoken/model.py", line 84, in encoding_name_for_model
    raise KeyError(
KeyError: 'Could not automatically map qwen-chat-14b to a tokeniser. Please use `tiktoken.get_encoding` to explicitly get the tokeniser you expect.'
02/19 14:05:07 - OpenCompass - DEBUG - Get class `OpenICLInferTask` from "task" registry in "opencompass"
02/19 14:05:07 - OpenCompass - DEBUG - An `OpenICLInferTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_infer
02/19 14:05:12 - OpenCompass - INFO - Task [qwen-chat-14b/ceval-operating_system]
02/19 14:05:14 - OpenCompass - INFO - Start inferencing [qwen-chat-14b/ceval-operating_system]
100%|████████████████████████████████████████████████████████████████████████████████████| 19/19 [00:00<00:00, 510844.72it/s]
Traceback (most recent call last):
  File "/home/jingtianran/nlp/opencompass/opencompass/tasks/openicl_infer.py", line 153, in <module>
    inferencer.run()
  File "/home/jingtianran/nlp/opencompass/opencompass/tasks/openicl_infer.py", line 81, in run
    self._inference()
  File "/home/jingtianran/nlp/opencompass/opencompass/tasks/openicl_infer.py", line 131, in _inference
    inferencer.inference(retriever,
  File "/home/jingtianran/nlp/opencompass/opencompass/openicl/icl_inferencer/icl_gen_inferencer.py", line 97, in inference
    prompt_list = self.get_generation_prompt_list_from_retriever_indices(
  File "/home/jingtianran/nlp/opencompass/opencompass/openicl/icl_inferencer/icl_gen_inferencer.py", line 201, in get_generation_prompt_list_from_retriever_indices
    prompt_token_num = self.model.get_token_len_from_template(
  File "/home/jingtianran/nlp/opencompass/opencompass/models/base.py", line 189, in get_token_len_from_template
    token_lens = [self.get_token_len(prompt) for prompt in prompts]
  File "/home/jingtianran/nlp/opencompass/opencompass/models/base.py", line 189, in <listcomp>
    token_lens = [self.get_token_len(prompt) for prompt in prompts]
  File "/home/jingtianran/nlp/opencompass/opencompass/models/openai_api.py", line 267, in get_token_len
    enc = self.tiktoken.encoding_for_model(self.path)
  File "/home/jingtianran/anaconda3/envs/opencompass/lib/python3.10/site-packages/tiktoken/model.py", line 97, in encoding_for_model
    return get_encoding(encoding_name_for_model(model_name))
  File "/home/jingtianran/anaconda3/envs/opencompass/lib/python3.10/site-packages/tiktoken/model.py", line 84, in encoding_name_for_model
    raise KeyError(
KeyError: 'Could not automatically map qwen-chat-14b to a tokeniser. Please use `tiktoken.get_encoding` to explicitly get the tokeniser you expect.'
02/19 14:05:15 - OpenCompass - DEBUG - Get class `OpenICLInferTask` from "task" registry in "opencompass"
02/19 14:05:15 - OpenCompass - DEBUG - An `OpenICLInferTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_infer

```

ok i got it, i modified "opencompass/opencompass/models/openai_api.py", line 267, in get_token_len
`enc = self.tiktoken.encoding_for_model(self.path)`
to
`enc = self.tiktoken.get_encoding("cl100k_base")`
and it works


> ok i got it, i modified "opencompass/opencompass/models/openai_api.py", line 267, in get_token_len `enc = self.tiktoken.encoding_for_model(self.path)` to `enc = self.tiktoken.get_encoding("cl100k_base")` and it works

你这个等于用的是 cl100k_base的分词器吧。如果我们要是用开源的模型的话，是不是得把这个函数改成对应的tokenizer 来计算得到的token数啊。


为什么用url还要指定tokeniser嘞，本地还要加载模型的吗
KeyError: 'Could not automatically map http://model/v1/chat/completions to a tokeniser. Please use tiktoken.get_encoding to explicitly get the tokeniser you expect.'

各位佬，模型是openBuddy要怎么写配置文件啊
KeyError: 'Could not automatically map http://modelApi:8336/v1/chat/completions to a tokeniser. Please use tiktoken.get_encoding to explicitly get the tokeniser you expect.'
```python
from mmengine.config import read_base
from opencompass.models import OpenAI
from opencompass.partitioners import NaivePartitioner
from opencompass.runners.local_api import LocalAPIRunner

from opencompass.tasks import OpenICLInferTask
from stutil import JsonUtil
model_token_obj={
    "token": "xxxxxxxxxx",
    "url": "http://diadiahoda:3131"
}

print("model_token_obj",model_token_obj)


with read_base():
    from ..summarizers.medium import summarizer
    from ..datasets.cmmlu.cmmlu_gen_c13365 import cmmlu_datasets
    

datasets = [
    *cmmlu_datasets,
]

url_base=model_token_obj.get('url')

models = [
    dict(
        type=OpenAI,
        openai_api_base=rf"{url_base}/v1/chat/completions",
        # abbr="mymodel",
        # path="mymodel",
        abbr="OpenBuddy",
        path=rf"{url_base}/v1/chat/completions",
        key=model_token_obj.get('token'),  # please give you key
        run_cfg=dict(num_gpus=0),
        query_per_second=1,
        batch_size=1,
    ),
]

infer = dict(
    partitioner=dict(type=NaivePartitioner),
    runner=dict(
        type=LocalAPIRunner,
        max_num_workers=1,
        concurrent_users=1,
        task=dict(type=OpenICLInferTask)),
)

work_dir = "outputs/api_mymodel/"
```

> 各位佬，模型是openBuddy要怎么写配置文件啊 KeyError: 'Could not automatically map http://modelApi:8336/v1/chat/completions to a tokeniser. Please use tiktoken.get_encoding to explicitly get the tokeniser you expect.'
> 
> ```python
> from mmengine.config import read_base
> from opencompass.models import OpenAI
> from opencompass.partitioners import NaivePartitioner
> from opencompass.runners.local_api import LocalAPIRunner
> 
> from opencompass.tasks import OpenICLInferTask
> from stutil import JsonUtil
> model_token_obj={
>     "token": "xxxxxxxxxx",
>     "url": "http://diadiahoda:3131"
> }
> 
> print("model_token_obj",model_token_obj)
> 
> 
> with read_base():
>     from ..summarizers.medium import summarizer
>     from ..datasets.cmmlu.cmmlu_gen_c13365 import cmmlu_datasets
>     
> 
> datasets = [
>     *cmmlu_datasets,
> ]
> 
> url_base=model_token_obj.get('url')
> 
> models = [
>     dict(
>         type=OpenAI,
>         openai_api_base=rf"{url_base}/v1/chat/completions",
>         # abbr="mymodel",
>         # path="mymodel",
>         abbr="OpenBuddy",
>         path=rf"{url_base}/v1/chat/completions",
>         key=model_token_obj.get('token'),  # please give you key
>         run_cfg=dict(num_gpus=0),
>         query_per_second=1,
>         batch_size=1,
>     ),
> ]
> 
> infer = dict(
>     partitioner=dict(type=NaivePartitioner),
>     runner=dict(
>         type=LocalAPIRunner,
>         max_num_workers=1,
>         concurrent_users=1,
>         task=dict(type=OpenICLInferTask)),
> )
> 
> work_dir = "outputs/api_mymodel/"
> ```

没记错的话，你应该到那个openai的类里面，有一个 get_tokens 的函数，你把这个函数，改成你自己对应模型的计算tokens的方法就可以。他这个函数主要影响 你输出的max_length。
另外，如果你不想改这个函数，好像就需要把算tokens的函数指定一下就行。就是楼上说的 把
`enc = self.tiktoken.encoding_for_model(self.path)`
改成
`enc = self.tiktoken.get_encoding("cl100k_base")`