Repository: modelscope/evalscope
Issue: 使用evalscope perf测试 chat接口服务出错
Root cause: Validation Gap

## 问题描述 ： 使用evalscope perf测试 chat接口服务出错

请简要描述您遇到的问题： 使用sglang部署的qwen2.5-32b-instruct-awq模型，然后使用evalscope进行性能测试，报错，报错信息如下：
```shell
2025-03-26 19:07:50,600 - evalscope - http_client.py - test_connection - 169 - WARNING - Retrying...  <500> Internal Server Error
2025-03-26 19:08:00,612 - evalscope - http_client.py - on_request_start - 113 - DEBUG - Starting request: <TraceRequestStartParams(method='POST', url=URL('http://localhost:8102/v1/chat/completions'), headers=<CIMultiDict('Content-Type': 'application/json', 'user-agent': 'modelscope_bench')>)>
2025-03-26 19:08:00,614 - evalscope - http_client.py - on_request_chunk_sent - 125 - DEBUG - Request sent: <method='POST',  url=URL('http://localhost:8102/v1/chat/completions'), truncated_chunk='{"messages": [{"role": "user", "content": "hello"}], "model": "Qwen2.5-32B-Instruct-AWQ", "max_tokens": 10, "stream": null}'>
2025-03-26 19:08:00,617 - evalscope - http_client.py - on_response_chunk_received - 137 - DEBUG - Request received: <method='POST',  url=URL('http://localhost:8102/v1/chat/completions'), truncated_chunk='Internal Server Error'>
2025-03-26 19:08:00,617 - evalscope - http_client.py - test_connection - 169 - WARNING - Retrying...  <500> Internal Server Error
2025-03-26 19:08:10,627 - evalscope - http_client.py - on_request_start - 113 - DEBUG - Starting request: <TraceRequestStartParams(method='POST', url=URL('http://localhost:8102/v1/chat/completions'), headers=<CIMultiDict('Content-Type': 'application/json', 'user-agent': 'modelscope_bench')>)>
2025-03-26 19:08:10,630 - evalscope - http_client.py - on_request_chunk_sent - 125 - DEBUG - Request sent: <method='POST',  url=URL('http://localhost:8102/v1/chat/completions'), truncated_chunk='{"messages": [{"role": "user", "content": "hello"}], "model": "Qwen2.5-32B-Instruct-AWQ", "max_tokens": 10, "stream": null}'>
2025-03-26 19:08:10,633 - evalscope - http_client.py - on_response_chunk_received - 137 - DEBUG - Request received: <method='POST',  url=URL('http://localhost:8102/v1/chat/completions'), truncated_chunk='Internal Server Error'>
2025-03-26 19:08:10,634 - evalscope - http_client.py - test_connection - 169 - WARNING - Retrying...  <500> Internal Server Error
```
```shell
# sglang版本
 pip install "sglang[all]>=0.4.4.post1" --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer-python
# 部署方式如下
python -m sglang.launch_server --model-path Qwen/Qwen2.5-32B-Instruct-AWQ --trust-remote-code --tp 2 --mem-fraction-static 0.9 --host 0.0.0.0 --port 8102 --enable-p2p-check --served-model-name Qwen2.5-32B-Instruct-AWQ
# evalscope测试如下
evalscope perf --parallel 100 --url http://localhost/v1/chat/completions --model Qwen2.5-32B-Instruct-AWQ --log-every-n-query 128 --connect-timeout 600 --read-timeout 600 --api openai --prompt '写一个科幻小说，不少于200字' -n 2048 --number 512 --debug

# 但是使用上述测试命令对vllm部署的同一个模型，没有问题

sglang服务端报错如下：
[2025-03-26 19:09:10] INFO:     127.0.0.1:34864 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
[2025-03-26 19:09:10] ERROR:    Exception in ASGI application

validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for ChatCompletionRequest
Input should be a valid boolean [type=bool_type, input_value=None, input_type=NoneType]
    For further information visit https://errors.pydantic.dev/2.10/v/bool_type
```
## EvalScope 版本 / Version (必须)
v0.13.0、v0.13.1 均测试过

## 使用的工具 / Tools Used
- [ ] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [X] Perf / 模型推理压测工具
- [ ] Arena  /竞技场模式


## 执行的代码或指令 / Code or Commands Executed

请提供您执行的主要代码或指令。 / Please provide the main code or commands you executed. 例如 / For example:

```bash
# sglang版本
 pip install "sglang[all]>=0.4.4.post1" --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer-python
# 部署方式如下
python -m sglang.launch_server --model-path Qwen/Qwen2.5-32B-Instruct-AWQ --trust-remote-code --tp 2 --mem-fraction-static 0.9 --host 0.0.0.0 --port 8102 --enable-p2p-check --served-model-name Qwen2.5-32B-Instruct-AWQ
# evalscope测试如下
evalscope perf --parallel 100 --url http://localhost/v1/chat/completions --model Qwen2.5-32B-Instruct-AWQ --log-every-n-query 128 --connect-timeout 600 --read-timeout 600 --api openai --prompt '写一个科幻小说，不少于200字' -n 2048 --number 512 --debug

# 但是使用上述测试命令对vllm部署的同一个模型，没有问题
```
```python
# 直接执行请求没有问题
import subprocess, json

curl_command = f"""
curl -s http://localhost:8102/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{{"model": "Qwen2.5-32B-Instruct-AWQ", "messages": [{{"role": "user", "content": "写一个科幻小说，不少于200字"}}]}}'
"""

response = json.loads(subprocess.check_output(curl_command, shell=True))
print(response)

# 结果如下
{'id': '47441f117fac46f1beb0a7b8498993a2', 'object': 'chat.completion', 'created': 1742988532, 'model': 'Qwen2.5-32B-Instruct-AWQ', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': '在遥远的未来，地球资源枯竭，人类不得不寻找新的居住地。经过多年的探索，人类发现了一颗名为艾尔达的星球，它拥有丰富的资源和适宜的环境。一支探险队被派往艾尔达，队长是一名名叫李明的年轻科学家。\n\n李明和他的团队在艾尔达上发现了一种未知的生物，这种生物似乎拥有控制时间的能力。在与这些生物的交流中，李明发现它们的名字叫做“时间守护者”，这些守护者能够帮助人类解决资源短缺的问题，但条件是人类必须学会与自然和谐共生。\n\n在接下来的几年里，李明带领他的团队与时间守护者合作，学习如何利用他们的能力来改善人类的生活。人类开始意识到，真正的财富不是资源的多少，而是与自然和谐共存的能力。艾尔达星球成为了人类新的家园，而时间守护者与人类之间的友谊，也成为了宇宙中一段佳话。', 'reasoning_content': None, 'tool_calls': None}, 'logprobs': None, 'finish_reason': 'stop', 'matched_stop': 151645}], 'usage': {'prompt_tokens': 39, 'total_tokens': 237, 'completion_tokens': 198, 'prompt_tokens_details': None}}
```

## 错误日志 / Error Log

请粘贴完整的错误日志或控制台输出。 / Please paste the full error log or console output. 例如 / For example:

```
# 客户端测
2025-03-26 19:07:50,600 - evalscope - http_client.py - test_connection - 169 - WARNING - Retrying...  <500> Internal Server Error
2025-03-26 19:08:00,612 - evalscope - http_client.py - on_request_start - 113 - DEBUG - Starting request: <TraceRequestStartParams(method='POST', url=URL('http://localhost:8102/v1/chat/completions'), headers=<CIMultiDict('Content-Type': 'application/json', 'user-agent': 'modelscope_bench')>)>
2025-03-26 19:08:00,614 - evalscope - http_client.py - on_request_chunk_sent - 125 - DEBUG - Request sent: <method='POST',  url=URL('http://localhost:8102/v1/chat/completions'), truncated_chunk='{"messages": [{"role": "user", "content": "hello"}], "model": "Qwen2.5-32B-Instruct-AWQ", "max_tokens": 10, "stream": null}'>
2025-03-26 19:08:00,617 - evalscope - http_client.py - on_response_chunk_received - 137 - DEBUG - Request received: <method='POST',  url=URL('http://localhost:8102/v1/chat/completions'), truncated_chunk='Internal Server Error'>
2025-03-26 19:08:00,617 - evalscope - http_client.py - test_connection - 169 - WARNING - Retrying...  <500> Internal Server Error
2025-03-26 19:08:10,627 - evalscope - http_client.py - on_request_start - 113 - DEBUG - Starting request: <TraceRequestStartParams(method='POST', url=URL('http://localhost:8102/v1/chat/completions'), headers=<CIMultiDict('Content-Type': 'application/json', 'user-agent': 'modelscope_bench')>)>
2025-03-26 19:08:10,630 - evalscope - http_client.py - on_request_chunk_sent - 125 - DEBUG - Request sent: <method='POST',  url=URL('http://localhost:8102/v1/chat/completions'), truncated_chunk='{"messages": [{"role": "user", "content": "hello"}], "model": "Qwen2.5-32B-Instruct-AWQ", "max_tokens": 10, "stream": null}'>
2025-03-26 19:08:10,633 - evalscope - http_client.py - on_response_chunk_received - 137 - DEBUG - Request received: <method='POST',  url=URL('http://localhost:8102/v1/chat/completions'), truncated_chunk='Internal Server Error'>
2025-03-26 19:08:10,634 - evalscope - http_client.py - test_connection - 169 - WARNING - Retrying...  <500> Internal Server Error

# sglang服务侧
# sglang服务端报错如下：
[2025-03-26 19:09:10] INFO:     127.0.0.1:34864 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
[2025-03-26 19:09:10] ERROR:    Exception in ASGI application

validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for ChatCompletionRequest
Input should be a valid boolean [type=bool_type, input_value=None, input_type=NoneType]
    For further information visit https://errors.pydantic.dev/2.10/v/bool_type
```

## 运行环境 / Runtime Environment

- 操作系统 / Operating System:
  - [ ] Windows
  - [ ] macOS
  - [X] Ubuntu

- Python版本 / Python Version:
  - [ ] 3.11
  - [X] 3.10
  - [ ] 3.9


## 其他信息 / Additional Information

如果有其他相关信息，请在此处提供。 / If there is any other relevant information, please provide it here.


这个问题应该修复了，使用main分支代码试试