Repository: modelscope/evalscope
Issue: 测试 WMT2024++ 数据集时提示报错
Root cause: Validation Gap


## 问题描述

测试 WMT2024++ 数据集时提示：
```
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] COMET batch calculation failed: 'list' object has no attribute 'get'
2025-12-09 20:26:39 - evalscope - WARNING: [WMT24PPAdapter] BERTScore batch calculation failed: 'list' object has no attribute 'get'
```

## EvalScope 版本（必填）
1.3.0


- 操作系统：
- Python版本：3.10




问题已修复，使用main分支进行测试

> 问题已修复，使用main分支进行测试

大佬，还有问题：

<img width="791" height="431" alt="Image" src="https://github.com/user-attachments/assets/8a7224d3-7d04-4236-bc2b-3d4333aad689" />


测试数据：
`
  "wmt24pp": {
    "dataset_id":"/data/fffan/0_experiment/10_大模型测试/3_大模型效果测试/data/MultiLingual/WMT2024++/wmt24pp",
    "prompt_template":"Translate the following {source_language} sentence into {target_language}:\\n\\n{source_language}: {source_text}\\n{target_language}:",
    "metric_list": ["bert_score", "bleu", "comet"],
    "few_shot_num": 0,
    "few_shot_random": false,
    "subset_list": ["en-ar_eg", "en-ar_sa", "en-bg_bg", "en-bn_in", "en-ca_es", "en-cs_cz", "en-da_dk", "en-de_de", "en-el_gr", "en-es_mx", "en-et_ee", "en-fa_ir", "en-fi_fi", "en-fil_ph", "en-fr_ca",
          "en-fr_fr", "en-gu_in", "en-he_il", "en-hi_in", "en-hr_hr", "en-hu_hu", "en-id_id", "en-is_is", "en-it_it", "en-ja_jp", "en-kn_in", "en-ko_kr", "en-lt_lt", "en-lv_lv", "en-ml_in",
          "en-mr_in", "en-nl_nl", "en-no_no", "en-pa_in", "en-pl_pl", "en-pt_br", "en-pt_pt", "en-ro_ro", "en-ru_ru", "en-sk_sk", "en-sl_si", "en-sr_rs", "en-sv_se", "en-sw_ke", "en-sw_tz",
          "en-ta_in", "en-te_in", "en-th_th", "en-tr_tr", "en-uk_ua", "en-ur_pk", "en-vi_vn", "en-zh_cn", "en-zh_tw", "en-zu_za"]
  }
}

`



完整运行代码：
```
from evalscope import TaskConfig, run_task
from evalscope.constants import EvalType
def evaluate_test(self,task_name, data_name,data_info):
    print("####  任务：", task_name)
    print("####  数据集：", data_name)
    print("####  数据集详情：", data_info)
    print("####  测试batch数：", self.eval_batch_size)
    save_path = os.path.join(self.outpath, task_name)
    print("save_path: ",save_path)
    start_time = time.time()
    task_cfg = TaskConfig(
        model=self.model_name,
        api_url=self.model_url,
        eval_type=EvalType.SERVICE,
        datasets=[
            data_name,
        ],
        dataset_args={
            data_name: data_info
        },
        eval_batch_size=self.eval_batch_size,
        generation_config={
            'timeout': self.time_limit,
            'stream': self.stream
        },
        work_dir=save_path,
        #limit=10,  # 设置为100条数据进行测试
    )

    run_task(task_cfg=task_cfg)
    end_time = time.time()
    print(f"###  任务 {data_name}  耗时:{end_time - start_time}")

```




去掉 metric_list，这个会在框架内容进行处理