Repository: modelscope/evalscope
Issue: R1模型parse_content_with_reasoning失败(humaneval分数很低)
Root cause: Validation Gap

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [x] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [x] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [x] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题

## 问题描述

H20上面测试`Deepseek R1` humaneval数据集分数偏低，只有44.51，正常应该能达到80+

```bash
2025-10-13 08:40:30 - evalscope - INFO: Overall report table: 
+------------------+-----------+----------+------------------+-------+---------+---------+
| Model            | Dataset   | Metric   | Subset           |   Num |   Score | Cat.0   |
+==================+===========+==========+==================+=======+=========+=========+
| DeepSeek-R1      | humaneval | pass@1   | openai_humaneval |   164 |  0.4451 | default |
+------------------+-----------+----------+------------------+-------+---------+---------+ 
```

找了其中一个case：
```json
{
    "index": 28,
    "input": "**User**: \nRead the following function signature and docstring, and fully implement the function described. Your response should only contain the code for this function.\nfrom typing import List\n\n\ndef concatenate(strings: List[str]) -> str:\n    \"\"\" Concatenate list of strings into a single string\n    >>> concatenate([])\n    ''\n    >>> concatenate(['a', 'b', 'c'])\n    'abc'\n    \"\"\"\n",
    "target": "    return ''.join(strings)\n",
    "sample_score": {
        "score": {
            "value": {
                "pass": false
            },
            "extracted_prediction": "Okay, I need to implement the concatenate function that takes a list of strings and returns them all joined together. Let's see. The docstring examples show that when the input is an empty list, it returns an empty string. And when there are elements like ['a','b','c'], it returns 'abc'. So the straightforward way here is to use the join method in Python.\n\nWait, right. The join method takes all elements in an iterable and concatenates them. So for a list of strings, ''.join(list) would combine them all into one string. But what if the list is empty? Well, joining an empty list would just give an empty string, which matches the first example. So the function can be as simple as returning the joined result of the input list.\n\nSo the code would be something like:\n\ndef concatenate(strings: List[str]) -> str:\n    return ''.join(strings)\n\nThat's it. Let me test the examples. For the empty list, ''.join([]) is indeed ''. For ['a','b','c'], joining gives 'abc'. So this should work. I don't see any edge cases that would break this. Even if the list has empty strings, they would be included as per normal joining behavior. But the problem statement doesn't mention handling those, so the function as written should satisfy the requirements.\n</think>\n\ndef concatenate(strings: List[str]) -> str:\n    return ''.join(strings)",
            "prediction": "Okay, I need to implement the concatenate function that takes a list of strings and returns them all joined together. Let's see. The docstring examples show that when the input is an empty list, it returns an empty string. And when there are elements like ['a','b','c'], it returns 'abc'. So the straightforward way here is to use the join method in Python.\n\nWait, right. The join method takes all elements in an iterable and concatenates them. So for a list of strings, ''.join(list) would combine them all into one string. But what if the list is empty? Well, joining an empty list would just give an empty string, which matches the first example. So the function can be as simple as returning the joined result of the input list.\n\nSo the code would be something like:\n\ndef concatenate(strings: List[str]) -> str:\n    return ''.join(strings)\n\nThat's it. Let me test the examples. For the empty list, ''.join([]) is indeed ''. For ['a','b','c'], joining gives 'abc'. So this should work. I don't see any edge cases that would break this. Even if the list has empty strings, they would be included as per normal joining behavior. But the problem statement doesn't mention handling those, so the function as written should satisfy the requirements.\n</think>\n\ndef concatenate(strings: List[str]) -> str:\n    return ''.join(strings)",
            "explanation": null,
            "metadata": {
                "task_id": "HumanEval/28",
                "timeout": 4,
                "execution_result": {
                    "task_id": "HumanEval/28",
                    "passed": false,
                    "result": "failed: unterminated string literal (detected at line 11) (<string>, line 11)",
                    "completion_id": null
                }
            },
            "main_score_name": "pass"
        },
        "sample_id": 28,
        "group_id": 28,
        "sample_metadata": {
            "task_id": "HumanEval/28",
            "entry_point": "concatenate",
            "prompt": "from typing import List\n\n\ndef concatenate(strings: List[str]) -> str:\n    \"\"\" Concatenate list of strings into a single string\n    >>> concatenate([])\n    ''\n    >>> concatenate(['a', 'b', 'c'])\n    'abc'\n    \"\"\"\n",
            "test": "\n\nMETADATA = {\n    'author': 'jt',\n    'dataset': 'test'\n}\n\n\ndef check(candidate):\n    assert candidate([]) == ''\n    assert candidate(['x', 'y', 'z']) == 'xyz'\n    assert candidate(['x', 'y', 'z', 'w', 'k']) == 'xyzwk'\n"
        }
    }
}
```

发现提取的pred结果包含think的内容

查看当前[reasoning parsing](https://github.com/modelscope/evalscope/blob/3a30bc4e9864485ab785a61f481c51b724629bcf/evalscope/api/messages/utils.py#L7)的逻辑，发现是要提取`<think>``</think>`对之间的内容，但是对于`Deepseek R1`模型而言，他的chat template里面包含了`<think>`，因此模型输出不带有`<think>`，导致reasoning parsing失败。

参考：
- https://github.com/deepseek-ai/DeepSeek-R1/issues/352#issuecomment-2647127637
- https://huggingface.co/deepseek-ai/DeepSeek-R1/blob/main/tokenizer_config.json


## EvalScope 版本（必填）
当前最新代码，commit id: `975ca2eb659e828b017f55137a3129f2bca44acb`

## 使用的工具
- [x] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [ ] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令

请提供您执行的主要代码或指令。

## 错误日志

请粘贴完整的错误日志或控制台输出。

## 运行环境

- 操作系统：`Ubuntu 20:04`
- Python版本：`Python3.10`

## 其他信息

如果有其他相关信息，请在此处提供。


可以额外添加下面的参数再试试：
```
--dataset-args '{
  "humaneval": {
    "filters": {
      "remove_until": "</think>"
    }
  },
}'
```

> 可以额外添加下面的参数再试试：
> 
> ```
> --dataset-args '{
>   "humaneval": {
>     "filters": {
>       "remove_until": "</think>"
>     }
>   },
> }'
> ```

这个方法可以，分数正常了：
```bash
+------------------+-----------+----------+------------------+-------+---------+---------+
| Model            | Dataset   | Metric   | Subset           |   Num |   Score | Cat.0   |
+==================+===========+==========+==================+=======+=========+=========+
| DeepSeek-R1      | humaneval | pass@1   | openai_humaneval |   164 |  0.9756 | default |
+------------------+-----------+----------+------------------+-------+---------+---------+
```