Repository: modelscope/evalscope
Issue: DeepSeek-R1-0528-Qwen3_8b 评测hle数据集时出现错误
Root cause: Documentation Deficiency

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [ ] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [ ] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [ ] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题

## 问题描述

请简要描述您遇到的问题。

## EvalScope 版本（必填）
v0.17.1

## 使用的工具
- [√] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [ ] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令

请提供您执行的主要代码或指令。

## 错误日志
2025-08-14 11:04:07,158 - evalscope - WARNING - Output type generation is not supported for service evaluation. Using server model adapter instead.
2025-08-14 11:04:07,461 - evalscope - INFO - Dump task config to llm_results/test_results/acc/DeepSeek-R1-0528-Qwen3_8b-int8-gptq-dynamic-vacc/B2S_Y/20250814_110406/configs/task_config_ae6955.yaml
2025-08-14 11:04:07,465 - evalscope - INFO - {
    "model": "DeepSeek-R1-0528-Qwen3_8b-int8-gptq-dynamic-vacc",
    "model_id": "DeepSeek-R1-0528-Qwen3_8b-int8-gptq-dynamic-vacc",
    "model_args": {},
    "model_task": "text_generation",
    "template_type": null,
    "chat_template": null,
    "datasets": [
        "hle"
    ],
    "dataset_args": {
        "hle": {
            "name": "hle",
            "dataset_id": "datasets/llm_datasets/hle/data/",
            "model_adapter": "generation",
            "output_types": [
                "generation"
            ],
            "subset_list": [
                "Math"
            ],
            "metric_list": [
                "AverageAccuracy"
            ],
            "few_shot_num": 0,
            "few_shot_random": false,
            "train_split": null,
            "eval_split": "test",
            "prompt_template": "{query}\n\nPlease reason step by step, and put your final answer within \\boxed{{}}.",
            "system_prompt": null,
            "query_template": null,
            "pretty_name": "Humanity's-Last-Exam",
            "description": "Humanity's Last Exam (HLE) is a language model benchmark consisting of 2,500 questions across a broad range of subjects. It was created jointly by the Center for AI Safety and Scale AI. The benchmark classifies the questions into the following broad subjects: mathematics (41%), physics (9%), biology/medicine (11%), humanities/social science (9%), computer science/artificial intelligence (10%), engineering (4%), chemistry (7%), and other (9%). Around 14% of the questions require the ability to understand both text and images, i.e., multi-modality. 24% of the questions are multiple-choice; the rest are short-answer, exact-match questions.",
            "tags": [
                "Knowledge",
                "QA"
            ],
            "filters": {
                "remove_until": "</think>"
            },
            "extra_params": {}
        }
    },
    "dataset_dir": "/root/.cache/modelscope/hub/datasets",
    "dataset_hub": "modelscope",
    "generation_config": {
        "max_tokens": 32768,
        "temperature": 0.6,
        "top_p": 0.95,
        "top_k": 20,
        "n": 1
    },
    "eval_type": "service",
    "eval_backend": "Native",
    "eval_config": null,
    "stage": "all",
    "limit": 1,
    "eval_batch_size": 8,
    "mem_cache": false,
    "use_cache": null,
    "work_dir": "llm_results/test_results/acc/DeepSeek-R1-0528-Qwen3_8b-int8-gptq-dynamic-vacc/B2S_Y/20250814_110406",
    "outputs": null,
    "ignore_errors": false,
    "debug": false,
    "dry_run": false,
    "seed": 42,
    "api_url": "http://0.0.0.0:9900/v1",
    "api_key": "token-abc123",
    "timeout": 100000.0,
    "stream": true,
    "judge_strategy": "auto",
    "judge_worker_num": 1,
    "judge_model_args": {},
    "analysis_report": false
}
2025-08-14 11:04:07,466 - evalscope - INFO - Start evaluating on dataset datasets/llm_datasets/hle/data/
2025-08-14 11:04:07,466 - evalscope - INFO - Loading dataset from local disk: datasets/llm_datasets/hle/data/
2025-08-14 11:04:07,719 - evalscope - INFO - Loading dataset: dataset_name: datasets/llm_datasets/hle/data/ > subsets: ['default']
2025-08-14 11:04:18,564 - evalscope - INFO - Use settings: > few_shot_num: 0, > few_shot_split: None, > target_eval_split: test
2025-08-14 11:23:58,294 - evalscope - INFO - Dump predictions to llm_results/test_results/acc/DeepSeek-R1-0528-Qwen3_8b-int8-gptq-dynamic-vacc/B2S_Y/20250814_110406/predictions/DeepSeek-R1-0528-Qwen3_8b-int8-gptq-dynamic-vacc/hle_Math.jsonl.
2025-08-14 11:23:58,823 - evalscope - ERROR - Error when calling remote API: Error code: 401 - {'errors': {'message': 'Authentication failed, please make sure that a valid ModelScope token is supplied.', 'request_id': '73f8fe50-fd90-4ead-8cac-46991f004df9'}}
2025-08-14 11:23:58,823 - evalscope - ERROR - Error occurred during Qwen/Qwen3-235B-A22B@https://api-inference.modelscope.cn/v1/ LLM judge evaluation: Error code: 401 - {'errors': {'message': 'Authentication failed, please make sure that a valid ModelScope token is supplied.', 'request_id': '73f8fe50-fd90-4ead-8cac-46991f004df9'}}
2025-08-14 11:23:58,824 - evalscope - WARNING - No match found for pattern '(A|B)' in response: 
2025-08-14 11:23:58,841 - evalscope - INFO - 
datasets/llm_datasets/hle/data/ report table:
+--------------------------------------------------+-----------+-----------------+----------+-------+---------+---------+
| Model                                            | Dataset   | Metric          | Subset   |   Num |   Score | Cat.0   |
+==================================================+===========+=================+==========+=======+=========+=========+
| DeepSeek-R1-0528-Qwen3_8b-int8-gptq-dynamic-vacc | hle       | AverageAccuracy | Math     |     1 |       0 | default |
+--------------------------------------------------+-----------+-----------------+----------+-------+---------+---------+ 

2025-08-14 11:23:58,841 - evalscope - INFO - Skipping report analysis (`analysis_report=False`).
2025-08-14 11:23:58,841 - evalscope - INFO - Dump report to: llm_results/test_results/acc/DeepSeek-R1-0528-Qwen3_8b-int8-gptq-dynamic-vacc/B2S_Y/20250814_110406/reports/DeepSeek-R1-0528-Qwen3_8b-int8-gptq-dynamic-vacc/hle.json 

2025-08-14 11:23:58,841 - evalscope - INFO - Evaluation finished on datasets/llm_datasets/hle/data/
2025-08-14 11:23:58,845 - evalscope - INFO - Overall report table: 
+--------------------------------------------------+-----------+-----------------+----------+-------+---------+---------+
| Model                                            | Dataset   | Metric          | Subset   |   Num |   Score | Cat.0   |
+==================================================+===========+=================+==========+=======+=========+=========+
| DeepSeek-R1-0528-Qwen3_8b-int8-gptq-dynamic-vacc | hle       | AverageAccuracy | Math     |     1 |       0 | default |
+--------------------------------------------------+-----------+-----------------+----------+-------+---------+---------+ 

2025-08-14 11:23:58,845 - evalscope - INFO - Finished evaluation for DeepSeek-R1-0528-Qwen3_8b-int8-gptq-dynamic-vacc on ['hle']
2025-08-14 11:23:58,845 - evalscope - INFO - Output directory: llm_results/test_results/acc/DeepSeek-R1-0528-Qwen3_8b-int8-gptq-dynamic-vacc/B2S_Y/20250814_110406
请粘贴完整的错误日志或控制台输出。

## 运行环境

- 操作系统：
- Python版本：

## 其他信息

如果有其他相关信息，请在此处提供。


hle需要配置judge model进行评测，请参考：https://evalscope.readthedocs.io/zh-cn/latest/get_started/basic_usage.html#id9

> 感谢你的反馈！我们将关闭此问题。如果您有任何疑问，请随时重新打开它。如果EvalScope对您有所帮助，欢迎给我们点个STAR以示支持，谢谢！