Repository: modelscope/evalscope
Issue: evalscope - ERROR - Error when calling remote API: Connection error
Root cause: Documentation Deficiency

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [ ] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [ ] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [ ] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题

## 问题描述

2025-05-26 08:46:06,216 - evalscope - ERROR - Error when calling remote API: Connection error. 2025-05-26 08:46:06,217 - evalscope - ERROR - Error during LLM evaluation: Connection error. 2025-05-26 08:47:06,290 - openai._base_client - INFO - Retrying request to /chat/completions in 0.388478 seconds 2025-05-26 08:48:06,747 - openai._base_client - INFO - Retrying request to /chat/completions in 0.978265 seconds 2025-05-26 08:49:07,797 - evalscope - ERROR - Error when calling remote API: Connection error. 2025-05-26 08:49:07,797 - evalscope - ERROR - Error during LLM evaluation: Connection error.

## EvalScope 版本（必填）
v2.0.0

## 使用的工具
- [ ] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [ ] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令

evalscope eval --model /data/home/zengxiangxi/project/lvyouLLM/Fine_tuning/merged_qwen3_0.6b_lora_tourism3   --datasets alpaca_eval  --limit 100

## 错误日志

(evalscope) zengxiangxi@xianshitest3day3:~$ evalscope eval --model /data/home/zengxiangxi/project/lvyouLLM/Fine_tunin3_0.6b_lora_tourism3   --datasets alpaca_eval  --limit 100
2025-05-26 08:15:45,289 - evalscope - INFO - Args: Task config is provided with CommandLine type.
2025-05-26 08:15:47,154 - evalscope - INFO - Loading model /data/home/zengxiangxi/project/lvyouLLM/Fine_tuning/mergelora_tourism3 ...
--2025-05-26 08:15:52--  https://modelscope-open.oss-cn-hangzhou.aliyuncs.com/open_data/nltk_data/punkt_tab.zip
Resolving modelscope-open.oss-cn-hangzhou.aliyuncs.com (modelscope-open.oss-cn-hangzhou.aliyuncs.com)... 39.173.44.1
Connecting to modelscope-open.oss-cn-hangzhou.aliyuncs.com (modelscope-open.oss-cn-hangzhou.aliyuncs.com)|39.173.44.onnected.
ERROR: cannot verify modelscope-open.oss-cn-hangzhou.aliyuncs.com's certificate, issued by ‘CN=GlobalSign GCC R3 OV =GlobalSign nv-sa,C=BE’:
  Unable to locally verify the issuer's authority.
To connect to modelscope-open.oss-cn-hangzhou.aliyuncs.com insecurely, use `--no-check-certificate'.
unzip:  cannot find or open /data/home/zengxiangxi/nltk_data/tokenizers/punkt_tab.zip, /data/home/zengxiangxi/nltk_ds/punkt_tab.zip.zip or /data/home/zengxiangxi/nltk_data/tokenizers/punkt_tab.zip.ZIP.
2025-05-26 08:15:52,855 - evalscope - WARNING - Got local model dir: /data/home/zengxiangxi/project/lvyouLLM/Fine_tuwen3_0.6b_lora_tourism3
2025-05-26 08:15:52,855 - evalscope - INFO - Updating generation config ...
2025-05-26 08:15:53,036 - evalscope - INFO - Dump task config to ./outputs/20250526_081545/configs/task_config_fffb4
2025-05-26 08:15:53,039 - evalscope - INFO - {
    "model": "/data/home/zengxiangxi/project/lvyouLLM/Fine_tuning/merged_qwen3_0.6b_lora_tourism3",
    "model_id": "merged_qwen3_0.6b_lora_tourism3",
    "model_args": {
        "revision": "master",
        "precision": "torch.float16"
    },
    "model_task": "text_generation",
    "template_type": null,
    "chat_template": null,
    "datasets": [
        "alpaca_eval"
    ],
    "dataset_args": {
        "alpaca_eval": {
            "name": "alpaca_eval",
            "dataset_id": "AI-ModelScope/alpaca_eval",
            "model_adapter": "generation",
            "output_types": [
                "generation"
            ],
            "subset_list": [
                "alpaca_eval_gpt4_baseline"
            ],
            "metric_list": [
                "winrate"
            ],
            "few_shot_num": 0,
            "few_shot_random": false,
            "train_split": null,
            "eval_split": "eval",
            "prompt_template": null,
            "system_prompt": null,
            "query_template": null,
            "pretty_name": "AlpacaEval2.0",
            "filters": null,
            "extra_params": {}
        }
    },
    "dataset_dir": "/data/home/zengxiangxi/.cache/modelscope/hub/datasets",
    "dataset_hub": "modelscope",
    "generation_config": {
        "max_length": 2048,
        "max_new_tokens": 512,
        "do_sample": false,
        "top_k": 50,
        "top_p": 1.0,
        "temperature": 1.0
    },
    "eval_type": "checkpoint",
    "eval_backend": "Native",
    "eval_config": null,
    "stage": "all",
    "limit": 100,
    "eval_batch_size": 1,
    "mem_cache": false,
    "use_cache": null,
    "work_dir": "./outputs/20250526_081545",
    "outputs": null,
    "ignore_errors": false,
    "debug": false,
    "dry_run": false,
    "seed": 42,
    "api_url": null,
    "api_key": "EMPTY",
    "timeout": null,
    "stream": false,
    "judge_strategy": "auto",
    "judge_worker_num": 1,
    "judge_model_args": {}
}
2025-05-26 08:15:53,039 - evalscope - INFO - **** Start evaluating on dataset AI-ModelScope/alpaca_eval ****
2025-05-26 08:15:53,039 - evalscope - INFO - Loading dataset from hub: AI-ModelScope/alpaca_eval
2025-05-26 08:15:53,281 - evalscope - INFO - Loading dataset: dataset_name: AI-ModelScope/alpaca_eval > subsets: ['at4_baseline']
2025-05-26 08:15:59,242 - evalscope - INFO - Use settings: > few_shot_num: 0, > few_shot_split: None, > target_eval_
Predicting(alpaca_eval_gpt4_baseline): 100%|███████████████████████████████████████████████████| 100/100 [21:02<00:0
2025-05-26 08:37:01,600 - evalscope - INFO - Dump predictions to ./outputs/20250526_081545/predictions/merged_qwen3_rism3/alpaca_eval_alpaca_eval_gpt4_baseline.jsonl.
Reviewing(alpaca_eval_gpt4_baseline):   0%|                                                              | 0/100 [002025-05-26 08:38:01,675 - openai._base_client - INFO - Retrying request to /chat/completions in 0.420072 seconds
2025-05-26 08:39:02,110 - openai._base_client - INFO - Retrying request to /chat/completions in 0.993747 seconds
2025-05-26 08:40:03,171 - evalscope - ERROR - Error when calling remote API: Connection error.
2025-05-26 08:40:03,172 - evalscope - ERROR - Error during LLM evaluation: Connection error.
Reviewing(alpaca_eval_gpt4_baseline):   0%|                                                              | 0/100 [03
2025-05-26 08:41:03,244 - openai._base_client - INFO - Retrying request to /chat/completions in 0.465621 seconds
2025-05-26 08:42:03,779 - openai._base_client - INFO - Retrying request to /chat/completions in 0.944197 seconds
2025-05-26 08:43:04,795 - evalscope - ERROR - Error when calling remote API: Connection error.
2025-05-26 08:43:04,796 - evalscope - ERROR - Error during LLM evaluation: Connection error.
2025-05-26 08:44:04,863 - openai._base_client - INFO - Retrying request to /chat/completions in 0.407941 seconds
2025-05-26 08:45:05,330 - openai._base_client - INFO - Retrying request to /chat/completions in 0.830825 seconds
2025-05-26 08:46:06,216 - evalscope - ERROR - Error when calling remote API: Connection error.
2025-05-26 08:46:06,217 - evalscope - ERROR - Error during LLM evaluation: Connection error.
2025-05-26 08:47:06,290 - openai._base_client - INFO - Retrying request to /chat/completions in 0.388478 seconds
2025-05-26 08:48:06,747 - openai._base_client - INFO - Retrying request to /chat/completions in 0.978265 seconds
2025-05-26 08:49:07,797 - evalscope - ERROR - Error when calling remote API: Connection error.
2025-05-26 08:49:07,797 - evalscope - ERROR - Error during LLM evaluation: Connection error.
2025-05-26 08:50:07,855 - openai._base_client - INFO - Retrying request to /chat/completions in 0.447260 seconds
2025-05-26 08:51:08,374 - openai._base_client - INFO - Retrying request to /chat/completions in 0.992551 seconds
2025-05-26 08:52:09,437 - evalscope - ERROR - Error when calling remote API: Connection error.
2025-05-26 08:52:09,437 - evalscope - ERROR - Error during LLM evaluation: Connection error.


## 运行环境

- 操作系统：
- Python版本：3.10.16

## 其他信息

如果有其他相关信息，请在此处提供。


alpaca_eval 需要指定Judge Model进行评测，默认模型调用的是openai的模型，没有配置的话就会报错。这个文档里有说明：https://evalscope.readthedocs.io/zh-cn/latest/get_started/supported_dataset.html#llm



> alpaca_eval 需要指定Judge Model进行评测，默认模型调用的是openai的模型，没有配置的话就会报错。这个文档里有说明：https://evalscope.readthedocs.io/zh-cn/latest/get_started/supported_dataset.html#llm

好的，非常感谢