Repository: vllm-project/vllm
Pull request #14821: [Bugfix][Frontend] Eliminate regex based check in reasoning full generator
Author: login: gaocegege
Labels: name: frontend
name: ready
description: ONLY add when PR is ready to merge/full CI is needed
Created: 2025-03-14T14:18:02Z | Merged: 2025-03-28T11:20:35Z

FIX https://github.com/vllm-project/vllm/issues/14761

This PR:

- Removes buggy regex-based parser in reasoning full generation and introduces a string-compare approach.
- Adds more test cases for deepseek reasoning



Diff:
diff --git a/tests/reasoning/test_deepseekr1_reasoning_parser.py b/tests/reasoning/test_deepseekr1_reasoning_parser.py
index 7b6af183a86a..1b669c8fd2fb 100644
--- a/tests/reasoning/test_deepseekr1_reasoning_parser.py
+++ b/tests/reasoning/test_deepseekr1_reasoning_parser.py
@@ -90,6 +90,40 @@ def deepseek_r1_qwen_tokenizer():
     "content": "This is the rest",
     "is_reasoning_end": True,
 }
+THINK_NO_END = {
+    "output": "<think>This is a reasoning section",
+    "reasoning_content": "This is a reasoning section",
+    "content": None,
+    "is_reasoning_end": False,
+}
+EMPTY = {
+    "output": "",
+    "reasoning_content": "",
+    "content": None,
+    "is_reasoning_end": False,
+}
+EMPTY_STREAMING = {
+    "output": "",
+    "reasoning_content": None,
+    "content": None,
+    "is_reasoning_end": False,
+}
+NEW_LINE = {
+    "output": "\n<think>This is a reasoning section</think>\nThis is the rest",
+    "reasoning_content": "This is a reasoning section",
+    "content": "\nThis is the rest",
+    "is_reasoning_end": True,
+}
+# Streaming cannot handle new lines at the beginning of the output
+# because we need to support <think>...</think> and </think>...
+# We cannot know if the text before <think> is reasoning content
+# or not.
+NEW_LINE_STREAMING = {
+    "output": "\n<think>This is a reasoning section</think>\nThis is the rest",
+    "reasoning_content": "\nThis is a reasoning section",
+    "content": "\nThis is the rest",
+    "is_reasoning_end": True,
+}
 
 TEST_CASES = [
     pytest.param(
@@ -182,6 +216,36 @@ def deepseek_r1_qwen_tokenizer():
         SHORTEST_REASONING_WITH_THINK,
         id="shortest_with_think_streaming",
     ),
+    pytest.param(
+        False,
+        THINK_NO_END,
+        id="think_no_end",
+    ),
+    pytest.param(
+        True,
+        THINK_NO_END,
+        id="think_no_end_streaming",
+    ),
+    pytest.param(
+        False,
+        EMPTY,
+        id="empty",
+    ),
+    pytest.param(
+        True,
+        EMPTY_STREAMING,
+        id="empty_streaming",
+    ),
+    pytest.param(
+        False,
+        NEW_LINE,
+        id="new_line",
+    ),
+    pytest.param(
+        True,
+        NEW_LINE_STREAMING,
+        id="new_line_streaming",
+    ),
 ]
 
 
diff --git a/vllm/reasoning/deepseek_r1_reasoning_parser.py b/vllm/reasoning/deepseek_r1_reasoning_parser.py
index 73be6d4d1ab1..1c283c092a28 100644
--- a/vllm/reasoning/deepseek_r1_reasoning_parser.py
+++ b/vllm/reasoning/deepseek_r1_reasoning_parser.py
@@ -1,6 +1,5 @@
 # SPDX-License-Identifier: Apache-2.0
 
-import re
 from collections.abc import Sequence
 from typing import Optional, Union
 
@@ -32,9 +31,6 @@ class DeepSeekR1ReasoningParser(ReasoningParser):
     def __init__(self, tokenizer: PreTrainedTokenizerBase):
         super().__init__(tokenizer)
 
-        self.reasoning_regex = re.compile(
-            rf"{self.start_token}(.*?){self.end_token}", re.DOTALL)
-
         if not self.model_tokenizer:
             raise ValueError(
                 "The model tokenizer must be passed to the ReasoningParser "
@@ -143,23 +139,34 @@ def extract_reasoning_content_streaming(
     def extract_reasoning_content(
             self, model_output: str, request: ChatCompletionRequest
     ) -> tuple[Optional[str], Optional[str]]:
+        """
+        Extract reasoning content from the model output.
+
+        For text <think>abc</think>xyz:
+        - 'abc' goes to reasoning_content
+        - 'xyz' goes to content
+
+        Returns:
+            tuple[Optional[str], Optional[str]]: reasoning content and content
+        """
+
+        # Check if the start token is present in the model output, remove it
+        # if it is present.
+        model_output_parts = model_output.partition(self.start_token)
+        model_output = model_output_parts[2] if model_output_parts[
+            1] else model_output_parts[0]
+
         # DeepSeek R1 doesn't generate <think> now.
         # Thus we assume the reasoning content is always at the start.
         # Ref https://huggingface.co/deepseek-ai/DeepSeek-R1/commit/8a58a132790c9935686eb97f042afa8013451c9f
         if self.end_token not in model_output:
             return model_output, None
         else:
-            # Add a start token if it's missing to keep compatibility.
-            if self.start_token not in model_output:
-                model_output = f"{self.start_token}{model_output}"
-            # Use a regex to find the reasoning content
-            reasoning_content = self.reasoning_regex.findall(model_output)[0]
-
-            end_index = len(
-                f"{self.start_token}{reasoning_content}{self.end_token}")
-            final_output = model_output[end_index:]
-
-            if len(final_output) == 0:
-                return reasoning_content, None
-
-            return reasoning_content, final_output
+            reasoning_content, _, content = model_output.partition(
+                self.end_token)
+            # If the end token is not found, return the model output as is.
+            # It should not happen since we already checked for the presence
+            # of the end token.
+            # If generation stops right after end-of-think, return null content
+            final_content = content or None
+            return reasoning_content, final_content


Reviews:
user: login: gaocegege
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-15T01:37:36Z
user: login: cjackal
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-15T16:35:16Z
user: login: cjackal
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-15T16:48:53Z
user: login: aarnphm
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-15T17:34:52Z
user: login: gaocegege
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-16T02:45:38Z
user: login: gaocegege
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-16T03:06:28Z
user: login: aarnphm
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-16T03:09:43Z
user: login: gaocegege
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-16T07:01:58Z
user: login: gaocegege
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-16T07:03:17Z
user: login: aarnphm
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-16T11:00:02Z
user: login: aarnphm
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-03-16T11:00:37Z
user: login: gaocegege
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-16T11:36:56Z
user: login: cjackal
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-16T12:19:50Z
user: login: DarkLight1337
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-28T08:32:42Z

Review comments:
diff_hunk: @@ -124,25 +120,42 @@ def extract_reasoning_content_streaming(
     def extract_reasoning_content(
             self, model_output: str, request: ChatCompletionRequest
     ) -> tuple[Optional[str], Optional[str]]:
+        """
+        Extract reasoning content from the model output.
+
+        For text <think>abc</think>xyz:
+        - 'abc' goes to reasoning_content
+        - 'xyz' goes to content
+
+        Returns:
+            tuple[Optional[str], Optional[str]]: reasoning content and content
+        """
+
+        # Check if the start token is present in the model output, remove it
path: vllm/entrypoints/openai/reasoning_parsers/deepseek_r1_reasoning_parser.py
user: login: gaocegege
body: There may be a bug about regex, according to the issue, but I have not reproduced it.

But I found a bug with old chat templates: `<think>This is a reasoning section` will result `reasoning_content: <think>This is a reasoning section`.

This PR fixed it and added more test cases

created_at: 2025-03-15T01:37:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -124,25 +120,42 @@ def extract_reasoning_content_streaming(
     def extract_reasoning_content(
             self, model_output: str, request: ChatCompletionRequest
     ) -> tuple[Optional[str], Optional[str]]:
+        """
+        Extract reasoning content from the model output.
+
+        For text <think>abc</think>xyz:
+        - 'abc' goes to reasoning_content
+        - 'xyz' goes to content
+
+        Returns:
+            tuple[Optional[str], Optional[str]]: reasoning content and content
+        """
+
+        # Check if the start token is present in the model output, remove it
+        # if it is present.
+        start_token_index = model_output.find(self.think_start_token)
+        if start_token_index != -1:
+            model_output = model_output[start_token_index +
+                                        len(self.think_start_token):]
path: vllm/entrypoints/openai/reasoning_parsers/deepseek_r1_reasoning_parser.py
user: login: cjackal
body: Nit: we can use `str.partition` for performance.
```suggestion
        model_output_parts = model_output.partition(self.think_start_token)
        model_output = model_output_parts[2] if model_output_parts[1] else model_output_parts[0]
```
created_at: 2025-03-15T16:35:16Z
author_association: CONTRIBUTOR
diff_hunk: @@ -124,25 +120,42 @@ def extract_reasoning_content_streaming(
     def extract_reasoning_content(
             self, model_output: str, request: ChatCompletionRequest
     ) -> tuple[Optional[str], Optional[str]]:
+        """
+        Extract reasoning content from the model output.
+
+        For text <think>abc</think>xyz:
+        - 'abc' goes to reasoning_content
+        - 'xyz' goes to content
+
+        Returns:
+            tuple[Optional[str], Optional[str]]: reasoning content and content
+        """
+
+        # Check if the start token is present in the model output, remove it
+        # if it is present.
+        start_token_index = model_output.find(self.think_start_token)
+        if start_token_index != -1:
+            model_output = model_output[start_token_index +
+                                        len(self.think_start_token):]
 
         # DeepSeek R1 doesn't generate <think> now.
         # Thus we assume the reasoning content is always at the start.
         # Ref https://huggingface.co/deepseek-ai/DeepSeek-R1/commit/8a58a132790c9935686eb97f042afa8013451c9f
         if self.think_end_token not in model_output:
             return model_output, None
         else:
-            # Add a start token if it's missing to keep compatibility.
-            if self.think_start_token not in model_output:
-                model_output = f"{self.think_start_token}{model_output}"
-            # Use a regex to find the reasoning content
-            reasoning_content = self.reasoning_regex.findall(model_output)[0]
-
-            end_index = len(
-                f"{self.think_start_token}{reasoning_content}{self.think_end_token}"
-            )
-            final_output = model_output[end_index:]
-
-            if len(final_output) == 0:
+            # Find the end token index in the model output.
+            end_token_index = model_output.find(self.think_end_token)
+            # If the end token is not found, return the model output as is.
+            # It should not happen since we already checked for the presence
+            # of the end token.
+            if end_token_index == -1:
+                return model_output, None
+            # Extract the reasoning content before the end token.
+            reasoning_content = model_output[:end_token_index]
+            # Extract the content after the end token.
+            content = model_output[end_token_index +
+                                   len(self.think_end_token):]
+            if len(content) == 0:
                 return reasoning_content, None
path: vllm/entrypoints/openai/reasoning_parsers/deepseek_r1_reasoning_parser.py
user: login: cjackal
body: Ditto, now the logic is simpler than the above as reasoning content is always at the first half.
```suggestion
            reasoning_content, think_end_token, content = model_output.partition(self.think_end_token)
            # If the end token is not found, return the model output as is.
            # It should not happen since we already checked for the presence
            # of the end token.
            # If generation stops right after end-of-think, return null content
            content = content or None
```
created_at: 2025-03-15T16:48:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -124,25 +120,42 @@ def extract_reasoning_content_streaming(
     def extract_reasoning_content(
             self, model_output: str, request: ChatCompletionRequest
     ) -> tuple[Optional[str], Optional[str]]:
+        """
+        Extract reasoning content from the model output.
+
+        For text <think>abc</think>xyz:
+        - 'abc' goes to reasoning_content
+        - 'xyz' goes to content
+
+        Returns:
+            tuple[Optional[str], Optional[str]]: reasoning content and content
+        """
+
+        # Check if the start token is present in the model output, remove it
+        # if it is present.
+        start_token_index = model_output.find(self.think_start_token)
+        if start_token_index != -1:
+            model_output = model_output[start_token_index +
+                                        len(self.think_start_token):]
 
         # DeepSeek R1 doesn't generate <think> now.
         # Thus we assume the reasoning content is always at the start.
         # Ref https://huggingface.co/deepseek-ai/DeepSeek-R1/commit/8a58a132790c9935686eb97f042afa8013451c9f
         if self.think_end_token not in model_output:
             return model_output, None
         else:
-            # Add a start token if it's missing to keep compatibility.
-            if self.think_start_token not in model_output:
-                model_output = f"{self.think_start_token}{model_output}"
-            # Use a regex to find the reasoning content
-            reasoning_content = self.reasoning_regex.findall(model_output)[0]
-
-            end_index = len(
-                f"{self.think_start_token}{reasoning_content}{self.think_end_token}"
-            )
-            final_output = model_output[end_index:]
-
-            if len(final_output) == 0:
+            # Find the end token index in the model output.
+            end_token_index = model_output.find(self.think_end_token)
+            # If the end token is not found, return the model output as is.
+            # It should not happen since we already checked for the presence
+            # of the end token.
+            if end_token_index == -1:
+                return model_output, None
+            # Extract the reasoning content before the end token.
+            reasoning_content = model_output[:end_token_index]
+            # Extract the content after the end token.
+            content = model_output[end_token_index +
+                                   len(self.think_end_token):]
+            if len(content) == 0:
                 return reasoning_content, None
path: vllm/entrypoints/openai/reasoning_parsers/deepseek_r1_reasoning_parser.py
user: login: aarnphm
body: I was just thinking can't we just assume for the thinking models to always output reasoning first until we found `</think>`?
created_at: 2025-03-15T17:34:52Z
author_association: COLLABORATOR
diff_hunk: @@ -124,25 +120,42 @@ def extract_reasoning_content_streaming(
     def extract_reasoning_content(
             self, model_output: str, request: ChatCompletionRequest
     ) -> tuple[Optional[str], Optional[str]]:
+        """
+        Extract reasoning content from the model output.
+
+        For text <think>abc</think>xyz:
+        - 'abc' goes to reasoning_content
+        - 'xyz' goes to content
+
+        Returns:
+            tuple[Optional[str], Optional[str]]: reasoning content and content
+        """
+
+        # Check if the start token is present in the model output, remove it
+        # if it is present.
+        start_token_index = model_output.find(self.think_start_token)
+        if start_token_index != -1:
+            model_output = model_output[start_token_index +
+                                        len(self.think_start_token):]
 
         # DeepSeek R1 doesn't generate <think> now.
         # Thus we assume the reasoning content is always at the start.
         # Ref https://huggingface.co/deepseek-ai/DeepSeek-R1/commit/8a58a132790c9935686eb97f042afa8013451c9f
         if self.think_end_token not in model_output:
             return model_output, None
         else:
-            # Add a start token if it's missing to keep compatibility.
-            if self.think_start_token not in model_output:
-                model_output = f"{self.think_start_token}{model_output}"
-            # Use a regex to find the reasoning content
-            reasoning_content = self.reasoning_regex.findall(model_output)[0]
-
-            end_index = len(
-                f"{self.think_start_token}{reasoning_content}{self.think_end_token}"
-            )
-            final_output = model_output[end_index:]
-
-            if len(final_output) == 0:
+            # Find the end token index in the model output.
+            end_token_index = model_output.find(self.think_end_token)
+            # If the end token is not found, return the model output as is.
+            # It should not happen since we already checked for the presence
+            # of the end token.
+            if end_token_index == -1:
+                return model_output, None
+            # Extract the reasoning content before the end token.
+            reasoning_content = model_output[:end_token_index]
+            # Extract the content after the end token.
+            content = model_output[end_token_index +
+                                   len(self.think_end_token):]
+            if len(content) == 0:
                 return reasoning_content, None
path: vllm/entrypoints/openai/reasoning_parsers/deepseek_r1_reasoning_parser.py
user: login: gaocegege
body: @aarnphm  To maintain compatibility with the old chat template, we must locate the `<think>` token and extract the text between it and the corresponding closing token.

For the new chat template, we can assume `the thinking models to always output reasoning first until we found </think>`
created_at: 2025-03-16T02:45:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -72,6 +72,35 @@
     "reasoning_content": None,
     "content": "This is the rest",
 }
+THINK_NO_END = {
+    "output": "<think>This is a reasoning section",
+    "reasoning_content": "This is a reasoning section",
+    "content": None,
+}
+EMPTY = {
+    "output": "",
+    "reasoning_content": "",
+    "content": None,
+}
+EMPTY_STREAMING = {
+    "output": "",
+    "reasoning_content": None,
+    "content": None,
+}
+NEW_LINE = {
+    "output": "\n<think>This is a reasoning section</think>\nThis is the rest",
path: tests/entrypoints/openai/reasoning_parsers/test_deepseekr1_reasoning_parser.py
user: login: gaocegege
body: I think it is why the previous regex based approach fails sometimes. The old chat template may return `\n<think>...`.
created_at: 2025-03-16T03:06:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -124,25 +120,42 @@ def extract_reasoning_content_streaming(
     def extract_reasoning_content(
             self, model_output: str, request: ChatCompletionRequest
     ) -> tuple[Optional[str], Optional[str]]:
+        """
+        Extract reasoning content from the model output.
+
+        For text <think>abc</think>xyz:
+        - 'abc' goes to reasoning_content
+        - 'xyz' goes to content
+
+        Returns:
+            tuple[Optional[str], Optional[str]]: reasoning content and content
+        """
+
+        # Check if the start token is present in the model output, remove it
+        # if it is present.
+        start_token_index = model_output.find(self.think_start_token)
+        if start_token_index != -1:
+            model_output = model_output[start_token_index +
+                                        len(self.think_start_token):]
 
         # DeepSeek R1 doesn't generate <think> now.
         # Thus we assume the reasoning content is always at the start.
         # Ref https://huggingface.co/deepseek-ai/DeepSeek-R1/commit/8a58a132790c9935686eb97f042afa8013451c9f
         if self.think_end_token not in model_output:
             return model_output, None
         else:
-            # Add a start token if it's missing to keep compatibility.
-            if self.think_start_token not in model_output:
-                model_output = f"{self.think_start_token}{model_output}"
-            # Use a regex to find the reasoning content
-            reasoning_content = self.reasoning_regex.findall(model_output)[0]
-
-            end_index = len(
-                f"{self.think_start_token}{reasoning_content}{self.think_end_token}"
-            )
-            final_output = model_output[end_index:]
-
-            if len(final_output) == 0:
+            # Find the end token index in the model output.
+            end_token_index = model_output.find(self.think_end_token)
+            # If the end token is not found, return the model output as is.
+            # It should not happen since we already checked for the presence
+            # of the end token.
+            if end_token_index == -1:
+                return model_output, None
+            # Extract the reasoning content before the end token.
+            reasoning_content = model_output[:end_token_index]
+            # Extract the content after the end token.
+            content = model_output[end_token_index +
+                                   len(self.think_end_token):]
+            if len(content) == 0:
                 return reasoning_content, None
path: vllm/entrypoints/openai/reasoning_parsers/deepseek_r1_reasoning_parser.py
user: login: aarnphm
body: this is for deepseek r1, all chat templates will have `<think>`
created_at: 2025-03-16T03:09:43Z
author_association: COLLABORATOR
diff_hunk: @@ -124,25 +120,42 @@ def extract_reasoning_content_streaming(
     def extract_reasoning_content(
             self, model_output: str, request: ChatCompletionRequest
     ) -> tuple[Optional[str], Optional[str]]:
+        """
+        Extract reasoning content from the model output.
+
+        For text <think>abc</think>xyz:
+        - 'abc' goes to reasoning_content
+        - 'xyz' goes to content
+
+        Returns:
+            tuple[Optional[str], Optional[str]]: reasoning content and content
+        """
+
+        # Check if the start token is present in the model output, remove it
+        # if it is present.
+        start_token_index = model_output.find(self.think_start_token)
+        if start_token_index != -1:
+            model_output = model_output[start_token_index +
+                                        len(self.think_start_token):]
 
         # DeepSeek R1 doesn't generate <think> now.
         # Thus we assume the reasoning content is always at the start.
         # Ref https://huggingface.co/deepseek-ai/DeepSeek-R1/commit/8a58a132790c9935686eb97f042afa8013451c9f
         if self.think_end_token not in model_output:
             return model_output, None
         else:
-            # Add a start token if it's missing to keep compatibility.
-            if self.think_start_token not in model_output:
-                model_output = f"{self.think_start_token}{model_output}"
-            # Use a regex to find the reasoning content
-            reasoning_content = self.reasoning_regex.findall(model_output)[0]
-
-            end_index = len(
-                f"{self.think_start_token}{reasoning_content}{self.think_end_token}"
-            )
-            final_output = model_output[end_index:]
-
-            if len(final_output) == 0:
+            # Find the end token index in the model output.
+            end_token_index = model_output.find(self.think_end_token)
+            # If the end token is not found, return the model output as is.
+            # It should not happen since we already checked for the presence
+            # of the end token.
+            if end_token_index == -1:
+                return model_output, None
+            # Extract the reasoning content before the end token.
+            reasoning_content = model_output[:end_token_index]
+            # Extract the content after the end token.
+            content = model_output[end_token_index +
+                                   len(self.think_end_token):]
+            if len(content) == 0:
                 return reasoning_content, None
path: vllm/entrypoints/openai/reasoning_parsers/deepseek_r1_reasoning_parser.py
user: login: gaocegege
body: The old chat template does not have `<think>`. Thus the model will generate one. New chat template includes it, thus the model will generate reasoning content directly.
created_at: 2025-03-16T07:01:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -124,25 +120,42 @@ def extract_reasoning_content_streaming(
     def extract_reasoning_content(
             self, model_output: str, request: ChatCompletionRequest
     ) -> tuple[Optional[str], Optional[str]]:
+        """
+        Extract reasoning content from the model output.
+
+        For text <think>abc</think>xyz:
+        - 'abc' goes to reasoning_content
+        - 'xyz' goes to content
+
+        Returns:
+            tuple[Optional[str], Optional[str]]: reasoning content and content
+        """
+
+        # Check if the start token is present in the model output, remove it
+        # if it is present.
+        start_token_index = model_output.find(self.think_start_token)
+        if start_token_index != -1:
+            model_output = model_output[start_token_index +
+                                        len(self.think_start_token):]
 
         # DeepSeek R1 doesn't generate <think> now.
         # Thus we assume the reasoning content is always at the start.
         # Ref https://huggingface.co/deepseek-ai/DeepSeek-R1/commit/8a58a132790c9935686eb97f042afa8013451c9f
         if self.think_end_token not in model_output:
             return model_output, None
         else:
-            # Add a start token if it's missing to keep compatibility.
-            if self.think_start_token not in model_output:
-                model_output = f"{self.think_start_token}{model_output}"
-            # Use a regex to find the reasoning content
-            reasoning_content = self.reasoning_regex.findall(model_output)[0]
-
-            end_index = len(
-                f"{self.think_start_token}{reasoning_content}{self.think_end_token}"
-            )
-            final_output = model_output[end_index:]
-
-            if len(final_output) == 0:
+            # Find the end token index in the model output.
+            end_token_index = model_output.find(self.think_end_token)
+            # If the end token is not found, return the model output as is.
+            # It should not happen since we already checked for the presence
+            # of the end token.
+            if end_token_index == -1:
+                return model_output, None
+            # Extract the reasoning content before the end token.
+            reasoning_content = model_output[:end_token_index]
+            # Extract the content after the end token.
+            content = model_output[end_token_index +
+                                   len(self.think_end_token):]
+            if len(content) == 0:
                 return reasoning_content, None
path: vllm/entrypoints/openai/reasoning_parsers/deepseek_r1_reasoning_parser.py
user: login: gaocegege
body: Ref https://github.com/vllm-project/vllm/pull/13025
created_at: 2025-03-16T07:03:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -124,25 +120,42 @@ def extract_reasoning_content_streaming(
     def extract_reasoning_content(
             self, model_output: str, request: ChatCompletionRequest
     ) -> tuple[Optional[str], Optional[str]]:
+        """
+        Extract reasoning content from the model output.
+
+        For text <think>abc</think>xyz:
+        - 'abc' goes to reasoning_content
+        - 'xyz' goes to content
+
+        Returns:
+            tuple[Optional[str], Optional[str]]: reasoning content and content
+        """
+
+        # Check if the start token is present in the model output, remove it
+        # if it is present.
+        start_token_index = model_output.find(self.think_start_token)
+        if start_token_index != -1:
+            model_output = model_output[start_token_index +
+                                        len(self.think_start_token):]
 
         # DeepSeek R1 doesn't generate <think> now.
         # Thus we assume the reasoning content is always at the start.
         # Ref https://huggingface.co/deepseek-ai/DeepSeek-R1/commit/8a58a132790c9935686eb97f042afa8013451c9f
         if self.think_end_token not in model_output:
             return model_output, None
         else:
-            # Add a start token if it's missing to keep compatibility.
-            if self.think_start_token not in model_output:
-                model_output = f"{self.think_start_token}{model_output}"
-            # Use a regex to find the reasoning content
-            reasoning_content = self.reasoning_regex.findall(model_output)[0]
-
-            end_index = len(
-                f"{self.think_start_token}{reasoning_content}{self.think_end_token}"
-            )
-            final_output = model_output[end_index:]
-
-            if len(final_output) == 0:
+            # Find the end token index in the model output.
+            end_token_index = model_output.find(self.think_end_token)
+            # If the end token is not found, return the model output as is.
+            # It should not happen since we already checked for the presence
+            # of the end token.
+            if end_token_index == -1:
+                return model_output, None
+            # Extract the reasoning content before the end token.
+            reasoning_content = model_output[:end_token_index]
+            # Extract the content after the end token.
+            content = model_output[end_token_index +
+                                   len(self.think_end_token):]
+            if len(content) == 0:
                 return reasoning_content, None
path: vllm/entrypoints/openai/reasoning_parsers/deepseek_r1_reasoning_parser.py
user: login: aarnphm
body: We will always fetch the results from main from `snapshot_download`, so im certain that even if you are using r1 models, it should fetch the latest changes for the tokenizer.
created_at: 2025-03-16T11:00:02Z
author_association: COLLABORATOR
diff_hunk: @@ -124,25 +120,42 @@ def extract_reasoning_content_streaming(
     def extract_reasoning_content(
             self, model_output: str, request: ChatCompletionRequest
     ) -> tuple[Optional[str], Optional[str]]:
+        """
+        Extract reasoning content from the model output.
+
+        For text <think>abc</think>xyz:
+        - 'abc' goes to reasoning_content
+        - 'xyz' goes to content
+
+        Returns:
+            tuple[Optional[str], Optional[str]]: reasoning content and content
+        """
+
+        # Check if the start token is present in the model output, remove it
+        # if it is present.
+        start_token_index = model_output.find(self.think_start_token)
+        if start_token_index != -1:
+            model_output = model_output[start_token_index +
+                                        len(self.think_start_token):]
 
         # DeepSeek R1 doesn't generate <think> now.
         # Thus we assume the reasoning content is always at the start.
         # Ref https://huggingface.co/deepseek-ai/DeepSeek-R1/commit/8a58a132790c9935686eb97f042afa8013451c9f
         if self.think_end_token not in model_output:
             return model_output, None
         else:
-            # Add a start token if it's missing to keep compatibility.
-            if self.think_start_token not in model_output:
-                model_output = f"{self.think_start_token}{model_output}"
-            # Use a regex to find the reasoning content
-            reasoning_content = self.reasoning_regex.findall(model_output)[0]
-
-            end_index = len(
-                f"{self.think_start_token}{reasoning_content}{self.think_end_token}"
-            )
-            final_output = model_output[end_index:]
-
-            if len(final_output) == 0:
+            # Find the end token index in the model output.
+            end_token_index = model_output.find(self.think_end_token)
+            # If the end token is not found, return the model output as is.
+            # It should not happen since we already checked for the presence
+            # of the end token.
+            if end_token_index == -1:
+                return model_output, None
+            # Extract the reasoning content before the end token.
+            reasoning_content = model_output[:end_token_index]
+            # Extract the content after the end token.
+            content = model_output[end_token_index +
+                                   len(self.think_end_token):]
+            if len(content) == 0:
                 return reasoning_content, None
path: vllm/entrypoints/openai/reasoning_parsers/deepseek_r1_reasoning_parser.py
user: login: gaocegege
body: I think so at that time, but some users https://github.com/vllm-project/vllm/pull/13025#discussion_r1950187100 used `TRANSFORMERS_OFFLINE=1`, or just use local model from s3 with the old template. It would be ideal to support both approaches, as doing so would incur minimal additional effort.

created_at: 2025-03-16T11:36:56Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-03-14T14:18:12Z
body: 👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run `fastcheck` CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your `fastcheck` build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping `simon-mo` or `khluu` to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add `ready` label to the PR or enable auto-merge.

🚀
author_association: NONE
user: login: gaocegege
created_at: 2025-03-14T14:19:19Z
body: /assign @aarnphm 
author_association: CONTRIBUTOR
user: login: aarnphm
created_at: 2025-03-15T01:32:33Z
body: Is there a specific bug thst u run into?

This seems like a refactor to me
author_association: COLLABORATOR
user: login: aarnphm
created_at: 2025-03-15T03:28:23Z
body: oh i see. will take a look
author_association: COLLABORATOR
user: login: gaocegege
created_at: 2025-03-16T02:47:34Z
body: @cjackal Thanks for the clean implementation! I addressed your comments. PTAL
author_association: CONTRIBUTOR
user: login: gaocegege
created_at: 2025-03-28T08:18:57Z
body: https://github.com/vllm-project/vllm/pull/14428 has already been merged, which blocks this PR. Since the last review, the only change made was a rebase, with no additional modifications.

@aarnphm @DarkLight1337 Could you please help merge it?
author_association: CONTRIBUTOR