Repository: modelscope/evalscope
Issue: 使用evalscope perf测试deepseek v3.2模型，vllm收到请求后报错。
Root cause: Configuration Error

evalscope perf \
  --parallel 16 \
  --number 16 \
  --model deepseek_v32 \
  --url https://x.x.x.x**/v1/completions** \
  --api openai \
  --log-every-n-query 5 \
 --connect-timeout 6000 \
 --read-timeout 6000 \
 --max-tokens 2048 \
  --min-tokens 2048 \
  --api-key $INF_API_KEY \
 --prompt '<｜begin▁of▁sentence｜><｜User｜>hello<｜Assistant｜></think>Hello! I am DeepSeek.<｜end▁of▁sentence｜><｜User｜>写一个科幻小说，不少于2000字，请开始你的表演<｜Assistant｜><think>'

vllm收到请求后报错，请求路径由/v1/completions解析为/chat/completions：
2025-12-04 15:47:24.585  (APIServer pid=158) INFO:     10.250.145.10:53780 - "POST **/chat/completions** HTTP/1.1" 404 Not Found
2025-12-04 15:47:24.491  (APIServer pid=158) INFO:     10.250.145.12:51896 - "POST **/chat/completions** HTTP/1.1" 404 Not Found
2025-12-04 15:47:24.490  (APIServer pid=158) INFO:     10.250.145.10:53784 - "POST **/chat/completions** HTTP/1.1" 404 Not Found

使用curl发送请求成功：
curl https:/x.x.x.x/v1/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $INF_API_KEY" \
  -d '{
    "model": "deepseek_v32",
    "prompt": "<｜begin▁of▁sentence｜><｜User｜>hello<｜Assistant｜></think>Hello! I am DeepSeek.<｜end▁of▁sentence｜><｜User｜>自我介绍<｜Assistant｜><think>",
    "max_tokens": 8192,
    "temperature": 1.0,
    "top_p": 0.95,
    "stop": ["<｜end▁of▁sentence｜>"]
  }'



{"id":"cmpl-a1d73efa-8538-4a9a-beff-a1a731c70ee0","object":"text_completion","created":1764833900,"model":"deepseek_v32","choices":[{"index":0,"text":"哦，用户让我自我介绍。之前对话历史只有简单的问候，所以需要全面但简洁地说明身份和功能。\n\n可以用公司背景和核心特点开头，再列举关键功能，最后用友好语气结尾。需要涵盖文本处理、文件支持、上下文长度和免费属性这些重点，避免技术细节堆砌。\n\n加上表情符号能让语气更亲切，但思考中不需要考虑表情符号，那是最终回复的格式问题。</think>你好！我是DeepSeek，一个由深度求索公司创造的AI助手😊\n\n让我简单介绍一下自己：\n\n**基本信息：**\n- 我是一个纯文本AI模型，擅长理解和生成自然语言\n- 知识截止到2024年7月，拥有128K的上下文长度\n- 完全免费使用，没有任何收费计划\n\n**我能为你做什么：**\n- 回答各种问题，进行深度对话\n- 协助写作、翻译、编程等任务\n- 分析和处理你上传的文件（支持图像、txt、pdf、ppt、word、excel等格式）\n- 从你提供的网页链接中读取信息进行分析\n\n**使用方式：**\n- 可以通过网页版直接使用\n- 也可以在官方应用商店下载App\n\n**特别提醒：**\n我没有语音功能，但可以帮你处理文字信息。如果需要最新信息，记得在Web/App上点开联网搜索功能哦！\n\n有什么具体问题或需要帮助的地方吗？我很乐意为你提供帮助！✨","logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null,"prompt_logprobs":null,"prompt_token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":18,"total_tokens":312,"completion_tokens":294,"prompt_tokens_details":null},"kv_transfer_params":null}(evalscope-zcl) [root:/]$ 


DeepSeek-V3.2 在其聊天模板中相比之前的版本引入了重大更新。主要变化涉及工具调用格式的修订以及“使用工具思考”功能的引入。

为了帮助社区理解和适应这个新模板，我们提供了一个专门的 encoding 文件夹，其中包含 Python 脚本和测试用例，演示如何将 OpenAI 兼容格式的消息编码为模型的输入字符串，以及如何解析模型的文本输出。

下面是一个简短的例子：

import transformers
# encoding/encoding_dsv32.py
from encoding_dsv32 import encode_messages, parse_message_from_completion_text

tokenizer = transformers.AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V3.2")

messages = [
    {"role": "user", "content": "hello"},
    {"role": "assistant", "content": "Hello! I am DeepSeek.", "reasoning_content": "thinking..."},
    {"role": "user", "content": "1+1=?"}
]
encode_config = dict(thinking_mode="thinking", drop_thinking=True, add_default_bos_token=True)

# messages -> string
prompt = encode_messages(messages, **encode_config)
# Output: "<｜begin▁of▁sentence｜><｜User｜>hello<｜Assistant｜></think>Hello! I am DeepSeek.<｜end▁of▁sentence｜><｜User｜>1+1=?<｜Assistant｜><think>"

# string -> tokens
tokens = tokenizer.encode(prompt)
# Output: [0, 128803, 33310, 128804, 128799, 19923, 3, 342, 1030, 22651, 4374, 1465, 16, 1, 128803, 19, 13, 19, 127252, 128804, 128798]

贴一下输出的log和使用的evalscope版本

> 贴一下输出的log和使用的evalscope版本

evalscope  1.1.1

(evalscope-zcl) [root:/]$ evalscope perf   --parallel 16   --number 16   --model deepseek_v32   --url https://hmg8kemkdqj9cojbjcmgppghgbbmmje8.openapi-sj.sii.edu.cn/v1/completions   --api openai   --log-every-n-query 5  --connect-timeout 6000  --read-timeout 6000  --max-tokens 2048   --min-tokens 2048      --api-key $INF_API_KEY  --prompt '<｜begin▁of▁sentence｜><｜User｜>hello<｜Assistant｜></think>Hello! I am DeepSeek.<｜end▁of▁sentence｜><｜User｜>写一个科幻小说，不少于2000字，请开始你的表演<｜Assistant｜><think>'
2025-12-05 11:38:15 - evalscope - INFO: Save the result to: outputs/20251205_113815/deepseek_v32
2025-12-05 11:38:15 - evalscope - INFO: Starting benchmark with args: 
2025-12-05 11:38:15 - evalscope - INFO: {
    "model": "deepseek_v32",
    "model_id": "deepseek_v32",
    "attn_implementation": null,
    "api": "openai",
    "tokenizer_path": null,
    "port": 8877,
    "url": "https://hmg8kemkdqj9cojbjcmgppghgbbmmje8.openapi-sj.sii.edu.cn/v1/completions",
    "headers": {
        "Authorization": "Bearer xxx"
    },
    "connect_timeout": 6000,
    "read_timeout": 6000,
    "api_key": "xxx",
    "no_test_connection": false,
    "number": 16,
    "parallel": 16,
    "rate": -1,
    "sleep_interval": 5,
    "db_commit_interval": 1000,
    "queue_size_multiplier": 5,
    "in_flight_task_multiplier": 2,
    "log_every_n_query": 5,
    "debug": false,
    "visualizer": null,
    "wandb_api_key": null,
    "swanlab_api_key": null,
    "name": null,
    "outputs_dir": "outputs/20251205_113815/deepseek_v32",
    "max_prompt_length": 9223372036854775807,
    "min_prompt_length": 0,
    "prefix_length": 0,
    "prompt": "<｜begin▁of▁sentence｜><｜User｜>hello<｜Assistant｜></think>Hello! I am DeepSeek.<｜end▁of▁sentence｜><｜User｜>写一个科幻小说，不少于2000字，请开始你的表演<｜Assistant｜><think>",
    "query_template": null,
    "apply_chat_template": false,
    "image_width": 224,
    "image_height": 224,
    "image_format": "RGB",
    "image_num": 1,
    "image_patch_size": 28,
    "dataset": "openqa",
    "dataset_path": null,
    "frequency_penalty": null,
    "repetition_penalty": null,
    "logprobs": null,
    "max_tokens": 2048,
    "min_tokens": 2048,
    "n_choices": null,
    "seed": null,
    "stop": null,
    "stop_token_ids": null,
    "stream": true,
    "temperature": 0.0,
    "top_p": null,
    "top_k": null,
    "extra_args": {}
}





2025-12-05 11:40:46 - evalscope - INFO: Test connection successful.
2025-12-05 11:40:46 - evalscope - INFO: Save the data base to: outputs/20251205_113815/deepseek_v32/benchmark_data.db
Processing:   0%|                                                                                                                                                              | 0/16 [00:00<?, ?it/s]2025-12-05 11:40:46 - evalscope - INFO: {
  "Time taken for tests (s)": 0.0,
  "Number of concurrency": 16,
  "Total requests": 5,
  "Succeed requests": 0,
  "Failed requests": 5,
  "Output token throughput (tok/s)": -1,
  "Total token throughput (tok/s)": -1,
  "Request throughput (req/s)": -1,
  "Average latency (s)": -1,
  "Average time to first token (s)": -1,
  "Average time per output token (s)": -1,
  "Average inter-token latency (s)": -1,
  "Average input tokens per request": -1,
  "Average output tokens per request": -1
}
2025-12-05 11:40:46 - evalscope - INFO: {
  "Time taken for tests (s)": 0.0,
  "Number of concurrency": 16,
  "Total requests": 10,
  "Succeed requests": 0,
  "Failed requests": 10,
  "Output token throughput (tok/s)": -1,
  "Total token throughput (tok/s)": -1,
  "Request throughput (req/s)": -1,
  "Average latency (s)": -1,
  "Average time to first token (s)": -1,
  "Average time per output token (s)": -1,
  "Average inter-token latency (s)": -1,
  "Average input tokens per request": -1,
  "Average output tokens per request": -1
}
2025-12-05 11:40:46 - evalscope - INFO: {
  "Time taken for tests (s)": 0.0,
  "Number of concurrency": 16,
  "Total requests": 15,
  "Succeed requests": 0,
  "Failed requests": 15,
  "Output token throughput (tok/s)": -1,
  "Total token throughput (tok/s)": -1,
  "Request throughput (req/s)": -1,
  "Average latency (s)": -1,
  "Average time to first token (s)": -1,
  "Average time per output token (s)": -1,
  "Average inter-token latency (s)": -1,
  "Average input tokens per request": -1,
  "Average output tokens per request": -1
}
Processing: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 97.63it/s]
2025-12-05 11:40:46 - evalscope - INFO: 
Benchmarking summary:
+-----------------------------------+---------+
| Key                               |   Value |
+===================================+=========+
| Time taken for tests (s)          |       0 |
+-----------------------------------+---------+
| Number of concurrency             |      16 |
+-----------------------------------+---------+
| Total requests                    |      16 |
+-----------------------------------+---------+
| Succeed requests                  |       0 |
+-----------------------------------+---------+
| Failed requests                   |      16 |
+-----------------------------------+---------+
| Output token throughput (tok/s)   |      -1 |
+-----------------------------------+---------+
| Total token throughput (tok/s)    |      -1 |
+-----------------------------------+---------+
| Request throughput (req/s)        |      -1 |
+-----------------------------------+---------+
| Average latency (s)               |      -1 |
+-----------------------------------+---------+
| Average time to first token (s)   |      -1 |
+-----------------------------------+---------+
| Average time per output token (s) |      -1 |
+-----------------------------------+---------+
| Average inter-token latency (s)   |      -1 |
+-----------------------------------+---------+
| Average input tokens per request  |      -1 |
+-----------------------------------+---------+
| Average output tokens per request |      -1 |
+-----------------------------------+---------+
2025-12-05 11:40:46 - evalscope - INFO: 
Percentile results:
+-------------+----------+---------+----------+-------------+--------------+---------------+----------------+---------------+
| Percentiles | TTFT (s) | ITL (s) | TPOT (s) | Latency (s) | Input tokens | Output tokens | Output (tok/s) | Total (tok/s) |
+-------------+----------+---------+----------+-------------+--------------+---------------+----------------+---------------+
|     10%     |   nan    |   nan   |   nan    |     nan     |     nan      |      nan      |      nan       |      nan      |
|     25%     |   nan    |   nan   |   nan    |     nan     |     nan      |      nan      |      nan       |      nan      |
|     50%     |   nan    |   nan   |   nan    |     nan     |     nan      |      nan      |      nan       |      nan      |
|     66%     |   nan    |   nan   |   nan    |     nan     |     nan      |      nan      |      nan       |      nan      |
|     75%     |   nan    |   nan   |   nan    |     nan     |     nan      |      nan      |      nan       |      nan      |
|     80%     |   nan    |   nan   |   nan    |     nan     |     nan      |      nan      |      nan       |      nan      |
|     90%     |   nan    |   nan   |   nan    |     nan     |     nan      |      nan      |      nan       |      nan      |
|     95%     |   nan    |   nan   |   nan    |     nan     |     nan      |      nan      |      nan       |      nan      |
|     98%     |   nan    |   nan   |   nan    |     nan     |     nan      |      nan      |      nan       |      nan      |
|     99%     |   nan    |   nan   |   nan    |     nan     |     nan      |      nan      |      nan       |      nan      |
+-------------+----------+---------+----------+-------------+--------------+---------------+----------------+---------------+
2025-12-05 11:40:46 - evalscope - INFO: Save the summary to: outputs/20251205_113815/deepseek_v32
(evalscope-zcl) [root:/]$ 

请更新evalscope到1.3.0再试试