Repository: modelscope/evalscope
Issue: tau-bench评估api-key 报错
Root cause: Documentation Deficiency

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [✅] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [✅] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [✅] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题

## 问题描述
`
请简要描述您遇到的问题。
评估 tau bench 按照官方设置报错 api key 无效
## EvalScope 版本（必填）
v0.xx.x
uv pip show evalscope
Name: evalscope
Version: 0.17.0
Location: /workspace/eval_res/evalscope/.venv/lib/python3.10/site-packages
Requires: accelerate, datasets, immutabledict, jieba, jsonlines, langdetect, latex2sympy2-extended, matplotlib, modelscope, nltk, openai, pandas, pillow, pyarrow, pyyaml, requests, rouge-chinese, rouge-score, sacrebleu, scikit-learn, seaborn, sympy, tabulate, torch, tqdm, transformers, word2number
Required-by:

## 使用的工具
- [ ] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [✅] RAGEval backend
- [ ] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令
 uv run python api_tau_bench_ori.py 
请提供您执行的主要代码或指令。
```python
from evalscope import TaskConfig, run_task

task_cfg = TaskConfig(
    model='/workspace/models/Qwen3-8B',
    api_url='http://127.0.0.1:8080/v1/chat/completions',
    api_key='EMPTY',
    eval_type='service',
    datasets=['tau_bench'],
    dataset_args={"tau_bench":{"local_path":"/workspace/eval_res/evalscope/eval_datasets/data/tau-bench"}},
    eval_batch_size=128,
    generation_config={
        'max_new_tokens': 1024,          'temperature': 0.7, 
    }
)

run_task(task_cfg=task_cfg)
```
## 错误日志
uv run python api_tau_bench_ori.py 
2025-07-22 18:24:19,595 - evalscope - INFO - Args: Task config is provided with TaskConfig type.
2025-07-22 18:24:24,232 - evalscope - INFO - Dump task config to ./outputs/20250722_182419/configs/task_config_a43aa8.yaml
2025-07-22 18:24:24,235 - evalscope - INFO - {
    "model": "/workspace/models/Qwen3-8B",
    "model_id": "Qwen3-8B",
    "model_args": {},
    "model_task": "text_generation",
    "template_type": null,
    "chat_template": null,
    "datasets": [
        "tau_bench"
    ],
    "dataset_args": {
        "tau_bench": {
            "name": "tau_bench",
            "dataset_id": "/workspace/eval_res/evalscope/eval_datasets/data/tau-bench",
            "model_adapter": "tau_bench_server",
            "output_types": [
                "generation"
            ],
            "subset_list": [
                "airline",
                "retail"
            ],
            "metric_list": [
                "Pass^1"
            ],
            "few_shot_num": 0,
            "few_shot_random": false,
            "train_split": null,
            "eval_split": "test",
            "prompt_template": null,
            "system_prompt": null,
            "query_template": null,
            "pretty_name": "τ-bench",
            "description": "A benchmark emulating dynamic conversations between a user (simulated by language models) and a language agent provided with domain-specific API tools and policy guidelines. Please install it with `pip install git+https://github.com/sierra-research/tau-bench` before evaluating and set a user model. ",
            "tags": [
                "Reasoning",
                "Agent",
                "Function Calling"
            ],
            "filters": null,
            "extra_params": {
                "user_model": "qwen-plus",
                "api_key": "EMPTY",
                "api_base": "https://dashscope.aliyuncs.com/compatible-mode/v1",
                "generation_config": {
                    "temperature": 0.7,
                    "max_new_tokens": 1024
                }
            }
        }
    },
    "dataset_dir": "/mnt/workspace/.cache/modelscope/hub/datasets",
    "dataset_hub": "modelscope",
    "generation_config": {
        "max_new_tokens": 1024,
        "temperature": 0.7
    },
    "eval_type": "service",
    "eval_backend": "Native",
    "eval_config": null,
    "stage": "all",
    "limit": null,
    "eval_batch_size": 128,
    "mem_cache": false,
    "use_cache": null,
    "work_dir": "./outputs/20250722_182419",
    "outputs": null,
    "ignore_errors": false,
    "debug": false,
    "dry_run": false,
    "seed": 42,
    "api_url": "http://0.0.0.0:8080/v1",
    "api_key": "EMPTY",
    "timeout": null,
    "stream": false,
    "judge_strategy": "auto",
    "judge_worker_num": 1,
    "judge_model_args": {},
    "analysis_report": false
}
2025-07-22 18:24:24,235 - evalscope - INFO - Start evaluating on dataset /workspace/eval_res/evalscope/eval_datasets/data/tau-bench
2025-07-22 18:24:24,235 - evalscope - INFO - Loading TauBench environment: airline
2025-07-22 18:24:25,730 - evalscope - ERROR - Error when calling remote API: Error code: 401 - {'error': {'message': 'Incorrect API key provided. ', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}, 'request_id': 'bdf1cc1b-22eb-9f26-aad9-2e43aa4313e4'}
Traceback (most recent call last):
  File "/workspace/eval_res/evalscope/api_tau_bench_ori.py", line 17, in <module>
    run_task(task_cfg=task_cfg)
  File "/workspace/eval_res/evalscope/evalscope/run.py", line 31, in run_task
    return run_single_task(task_cfg, run_time)
  File "/workspace/eval_res/evalscope/evalscope/run.py", line 44, in run_single_task
    result = evaluate_model(task_cfg, outputs)
  File "/workspace/eval_res/evalscope/evalscope/run.py", line 131, in evaluate_model
    res_dict = evaluator.eval()
  File "/workspace/eval_res/evalscope/evalscope/evaluator/evaluator.py", line 457, in eval
    prompts = self.load_dataset()
  File "/workspace/eval_res/evalscope/evalscope/evaluator/evaluator.py", line 74, in load_dataset
    dataset = self.data_adapter.load(
  File "/workspace/eval_res/evalscope/evalscope/benchmarks/tau_bench/tau_bench_adapter.py", line 84, in load
    env = get_env(
  File "/workspace/eval_res/evalscope/.venv/lib/python3.10/site-packages/tau_bench/envs/__init__.py", line 29, in get_env
    return MockAirlineDomainEnv(
  File "/workspace/eval_res/evalscope/.venv/lib/python3.10/site-packages/tau_bench/envs/airline/env.py", line 26, in __init__
    super().__init__(
  File "/workspace/eval_res/evalscope/.venv/lib/python3.10/site-packages/tau_bench/envs/base.py", line 73, in __init__
    self.user = load_user(
  File "/workspace/eval_res/evalscope/.venv/lib/python3.10/site-packages/tau_bench/envs/user.py", line 334, in load_user
    return LLMUserSimulationEnv(model=model, provider=provider)
  File "/workspace/eval_res/evalscope/.venv/lib/python3.10/site-packages/tau_bench/envs/user.py", line 44, in __init__
    self.reset()
  File "/workspace/eval_res/evalscope/.venv/lib/python3.10/site-packages/tau_bench/envs/user.py", line 78, in reset
    return self.generate_next_message(self.messages)
  File "/workspace/eval_res/evalscope/evalscope/benchmarks/tau_bench/tau_bench_adapter.py", line 67, in new_generate_next_message
    res = user_server.send_request(request_json)
  File "/workspace/eval_res/evalscope/evalscope/models/adapters/server_adapter.py", line 144, in send_request
    raise e
  File "/workspace/eval_res/evalscope/evalscope/models/adapters/server_adapter.py", line 136, in send_request
    response = self.client.chat.completions.create(**parsed_request)
  File "/workspace/eval_res/evalscope/.venv/lib/python3.10/site-packages/openai/_utils/_utils.py", line 287, in wrapper
    return func(*args, **kwargs)
  File "/workspace/eval_res/evalscope/.venv/lib/python3.10/site-packages/openai/resources/chat/completions/completions.py", line 1087, in create
    return self._post(
  File "/workspace/eval_res/evalscope/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 1256, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
  File "/workspace/eval_res/evalscope/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 1044, in request
    raise self._make_status_error_from_response(err.response) from None
openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided. ', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}, 'request_id': 'bdf1cc1b-22eb-9f26-aad9-2e43aa4313e4'}
请粘贴完整的错误日志或控制台输出。

## 运行环境

- 操作系统：linux
- Python版本：3.10

## 其他信息

如果有其他相关信息，请在此处提供。


请按照这里的说明，配置user model ：https://evalscope.readthedocs.io/zh-cn/latest/get_started/supported_dataset/llm.html#bench

```python
dataset_args={
    'tau_bench': {
        'extra_params': {
            'user_model': 'qwen-plus',
            'api_key': env.get('DASHSCOPE_API_KEY'),
            'api_base': 'https://dashscope.aliyuncs.com/compatible-mode/v1',
        }
    },
},
```

> 请按照这里的说明，配置user model ：https://evalscope.readthedocs.io/zh-cn/latest/get_started/supported_dataset/llm.html#bench
> 
> dataset_args={
>     'tau_bench': {
>         'extra_params': {
>             'user_model': 'qwen-plus',
>             'api_key': env.get('DASHSCOPE_API_KEY'),
>             'api_base': 'https://dashscope.aliyuncs.com/compatible-mode/v1',
>         }
>     },
> },

请问 gpt4 能评估吗？还是一定得 qwen plus？

> > 请按照这里的说明，配置user model ：https://evalscope.readthedocs.io/zh-cn/latest/get_started/supported_dataset/llm.html#bench
> > dataset_args={
> > 'tau_bench': {
> > 'extra_params': {
> > 'user_model': 'qwen-plus',
> > 'api_key': env.get('DASHSCOPE_API_KEY'),
> > 'api_base': 'https://dashscope.aliyuncs.com/compatible-mode/v1',
> > }
> > },
> > },
> 
> 请问 gpt4 能评估吗？还是一定得 qwen plus？

可以，只要支持openai api格式都行

> 感谢你的反馈！我们将关闭此问题。如果您有任何疑问，请随时重新打开它。如果EvalScope对您有所帮助，欢迎给我们点个STAR以示支持，谢谢！