Repository: modelscope/evalscope
Issue: evalscope速度基准测试（本地vLLM推理）
Root cause: Validation Gap

## 问题描述

使用Qwen2.5-0.5B-Instruct模型，evalscope速度基准测试（本地vLLM推理）报`Cannot connect to host 127.0.0.1:8877 ssl:default`错误

## EvalScope 版本（必填）
v0.16.0

## 使用的工具
- [ ] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [✓] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令
```
> CUDA_VISIBLE_DEVICES=0 evalscope perf \
 --parallel 1 \
 --number 1 \
 --model Qwen/Qwen2.5-0.5B-Instruct \
 --log-every-n-query 5 \
 --connect-timeout 6000 \
 --read-timeout 6000 \
 --max-tokens 2048 \
 --min-tokens 2048 \
 --api local_vllm \
 --dataset speed_benchmark_short
```
## 错误日志

```
2025-05-20 10:57:36,518 - evalscope - main.py - run_one_benchmark - 29 - INFO - Starting benchmark with args: 
2025-05-20 10:57:36,518 - evalscope - main.py - run_one_benchmark - 30 - INFO - {
    "model": "Qwen/Qwen2.5-0.5B-Instruct",
    "model_id": "Qwen2.5-0.5B-Instruct",
    "attn_implementation": null,
    "api": "local_vllm",
    "tokenizer_path": null,
    "port": 8877,
    "url": "http://127.0.0.1:8877/v1/completions",
    "headers": {},
    "connect_timeout": 6000,
    "read_timeout": 6000,
    "api_key": null,
    "no_test_connection": false,
    "number": 1,
    "parallel": 1,
    "rate": -1,
    "log_every_n_query": 10,
    "debug": true,
    "wandb_api_key": null,
    "swanlab_api_key": null,
    "name": null,
    "outputs_dir": "./outputs/20250520_105736/Qwen2.5-0.5B-Instruct",
    "max_prompt_length": 131072,
    "min_prompt_length": 0,
    "prefix_length": 0,
    "prompt": null,
    "query_template": null,
    "apply_chat_template": false,
    "dataset": "speed_benchmark",
    "dataset_path": null,
    "frequency_penalty": null,
    "logprobs": null,
    "max_tokens": 2048,
    "min_tokens": 2048,
    "n_choices": null,
    "seed": 0,
    "stop": [],
    "stop_token_ids": [],
    "stream": true,
    "temperature": 0.0,
    "top_p": null,
    "top_k": null,
    "extra_args": null
}
2025-05-20 10:57:36,520 - evalscope - http_client.py - on_request_start - 112 - DEBUG - Starting request: <TraceRequestStartParams(method='POST', url=URL('http://127.0.0.1:8877/v1/completions'), headers=<CIMultiDict('Content-Type': 'application/json', 'user-agent': 'modelscope_bench')>)>
2025-05-20 10:57:36,521 - evalscope - http_client.py - post - 107 - ERROR - Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
2025-05-20 10:57:36,521 - evalscope - http_client.py - test_connection - 168 - WARNING - Retrying...  <None> Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
2025-05-20 10:57:46,532 - evalscope - http_client.py - on_request_start - 112 - DEBUG - Starting request: <TraceRequestStartParams(method='POST', url=URL('http://127.0.0.1:8877/v1/completions'), headers=<CIMultiDict('Content-Type': 'application/json', 'user-agent': 'modelscope_bench')>)>
2025-05-20 10:57:46,532 - evalscope - http_client.py - post - 107 - ERROR - Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
2025-05-20 10:57:46,533 - evalscope - http_client.py - test_connection - 168 - WARNING - Retrying...  <None> Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
2025-05-20 10:57:56,543 - evalscope - http_client.py - on_request_start - 112 - DEBUG - Starting request: <TraceRequestStartParams(method='POST', url=URL('http://127.0.0.1:8877/v1/completions'), headers=<CIMultiDict('Content-Type': 'application/json', 'user-agent': 'modelscope_bench')>)>
2025-05-20 10:57:56,544 - evalscope - http_client.py - post - 107 - ERROR - Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
2025-05-20 10:57:56,544 - evalscope - http_client.py - test_connection - 168 - WARNING - Retrying...  <None> Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
2025-05-20 10:58:06,555 - evalscope - http_client.py - on_request_start - 112 - DEBUG - Starting request: <TraceRequestStartParams(method='POST', url=URL('http://127.0.0.1:8877/v1/completions'), headers=<CIMultiDict('Content-Type': 'application/json', 'user-agent': 'modelscope_bench')>)>
2025-05-20 10:58:06,556 - evalscope - http_client.py - post - 107 - ERROR - Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
2025-05-20 10:58:06,556 - evalscope - http_client.py - test_connection - 168 - WARNING - Retrying...  <None> Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
2025-05-20 10:58:16,567 - evalscope - http_client.py - on_request_start - 112 - DEBUG - Starting request: <TraceRequestStartParams(method='POST', url=URL('http://127.0.0.1:8877/v1/completions'), headers=<CIMultiDict('Content-Type': 'application/json', 'user-agent': 'modelscope_bench')>)>
2025-05-20 10:58:16,568 - evalscope - http_client.py - post - 107 - ERROR - Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
2025-05-20 10:58:16,568 - evalscope - http_client.py - test_connection - 168 - WARNING - Retrying...  <None> Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
2025-05-20 10:58:26,578 - evalscope - http_client.py - on_request_start - 112 - DEBUG - Starting request: <TraceRequestStartParams(method='POST', url=URL('http://127.0.0.1:8877/v1/completions'), headers=<CIMultiDict('Content-Type': 'application/json', 'user-agent': 'modelscope_bench')>)>
2025-05-20 10:58:26,579 - evalscope - http_client.py - post - 107 - ERROR - Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
2025-05-20 10:58:26,580 - evalscope - http_client.py - test_connection - 168 - WARNING - Retrying...  <None> Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
2025-05-20 10:58:36,590 - evalscope - http_client.py - on_request_start - 112 - DEBUG - Starting request: <TraceRequestStartParams(method='POST', url=URL('http://127.0.0.1:8877/v1/completions'), headers=<CIMultiDict('Content-Type': 'application/json', 'user-agent': 'modelscope_bench')>)>
2025-05-20 10:58:36,591 - evalscope - http_client.py - post - 107 - ERROR - Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
2025-05-20 10:58:36,592 - evalscope - http_client.py - test_connection - 168 - WARNING - Retrying...  <None> Cannot connect to host 127.0.0.1:8877 ssl:default [Connect call failed ('127.0.0.1', 8877)]
```

## 运行环境

- 操作系统：Ubuntu
- Python版本：3.10.6

## 其他信息

vllm版本：0.8.5.post1


多等待一会儿呢，会自动从本地vllm拉起服务

👌等待了一会儿就可以正常拉起服务了

> 感谢你的反馈！我们将关闭此问题。如果您有任何疑问，请随时重新打开它。如果EvalScope对您有所帮助，欢迎给我们点个STAR以示支持，谢谢！