Repository: ggml-org/llama.cpp
Pull request: chat : fix multiple tool_calls on hermes-2-pro

*Make sure to read the [contributing guidelines](https://github.com/ggml-org/llama.cpp/blob/master/CONTRIBUTING.md) before submitting a PR*

Tested with [SmolLM3-3B-GGUF](https://huggingface.co/ggml-org/SmolLM3-3B-GGUF).

<details>
<summary>Outputs</summary>

```

system
## Metadata

Knowledge Cutoff Date: June 2025
Today Date: 30 July 2025
Reasoning Mode: /think

## Custom Instructions

You are a helpful AI assistant named SmolLM, trained by Hugging Face. Your role as an assistant involves thoroughly exploring questions through a systematic thinking process before providing the final precise and accurate solutions. This requires engaging in a comprehensive cycle of analysis, summarizing, exploration, reassessment, reflection, backtracking, and iteration to develop well-considered thinking process. Please structure your response into two main sections: Thought and Solution using the specified format: <think> Thought section </think> Solution section. In the Thought section, detail your reasoning process in steps. Each step should include detailed considerations such as analysing questions, summarizing relevant findings, brainstorming new ideas, verifying the accuracy of the current steps, refining any errors, and revisiting previous steps. In the Solution section, based on various attempts, explorations, and reflections from the Thought section, systematically present the final solution that you deem correct. The Solution section should be logical, accurate, and concise and detail necessary steps needed to reach the conclusion.


user
You are a helpful AI assistant with access to tools. You can call tools to help answer user questions.

You can call any of the following tools to satisfy the user's requests: [
  {
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get current weather information for a location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "The city and country, e.g. \"San Francisco, CA\""
          }
        },
        "required": [
          "location"
        ],
        "additionalProperties": false
      }
    }
  },
  {
    "type": "function",
    "function": {
      "name": "calculate",
      "description": "Perform mathematical calculations",
      "parameters": {
        "type": "object",
        "properties": {
          "expression": {
            "type": "string",
            "description": "Mathematical expression to evaluate, e.g. \"2 + 2 * 3\""
          }
        },
        "required": [
          "expression"
        ],
        "additionalProperties": false
      }
    }
  },
  {
    "type": "function",
    "function": {
      "name": "get_time",
      "description": "Get current time in a specific timezone",
      "parameters": {
        "type": "object",
        "properties": {
          "timezone": {
            "type": "string",
            "description": "Timezone identifier, e.g. \"America/New_York\""
          }
        },
        "required": [
          "timezone"
        ],
        "additionalProperties": false
      }
    }
  }
]

Example tool call syntax:

{
  "tool_calls": [
    {
      "name": "tool_name",
      "arguments": {
        "arg1": "some_value"
      },
      "id": "call_1___"
    }
  ]
}


get weather and time


assistant
{
  "tool_calls": [
    {
      "name": "get_weather",
      "arguments": {
        "location": "New York, NY"
      },
      "id": "call_1___"
    },
    {
      "name": "get_time",
      "arguments": {
        "timezone": "America/New_York"
      },
      "id": "call_2___"
    }
  ],
  "content": "<think>\nOkay, the user wants to get the weather and the current time in a specific timezone. Let me break this down.\n\nFirst, I need to call the get_weather tool with the location provided. The location should be something like \"New York, NY\" or \"London, UK\" depending on the timezone they want. Wait, the user hasn't specified the location yet. Hmm, maybe they expect me to ask for the location, but the example shows a specific call. Wait, the user's instruction says \"get weather and time\", but the example tool calls are for specific names. Maybe they want me to make the calls based on some inputs. Wait, the user hasn't provided the specific location or timezone. Oh, maybe they want me to generate the tool calls for a generic case, but that's not possible without the actual data. Wait, maybe the user is testing if I can structure the tool calls correctly. Let me check the example again.\n\nThe example shows a structure with tool_calls, each having name and arguments. So the user probably expects me to create two tool calls: one for the weather and one for the time. But to do that, I need the location and timezone. Since the user hasn't provided those, perhaps I need to make assumptions. Wait, maybe the user is expecting me to prompt for that information. But the problem statement says \"you can call tools to help answer user questions\" and the example shows tool calls. So maybe the user is giving a scenario where they want me to perform these two tasks. Wait, the user's input is just \"get weather and time\". So perhaps I need to make the tool calls with placeholders. But without specific location and timezone, the tool calls can't be fully defined. Hmm, this is confusing. Maybe the user expects me to structure the tool calls with placeholders, like using variables. But the problem is that the actual content of the tool calls would require specific values. Since the user hasn't provided them, perhaps I should note that in the tool calls. Alternatively, maybe the user is expecting me to make the calls with example locations. For example, if the user wants the weather in New York and the time in New York as well. But that's a guess. Alternatively, maybe the user is expecting me to ask for the location and timezone, but the problem statement says to call the tools to answer the question. Wait, the user's question is \"get weather and time\", so maybe they want me to perform both actions. Let me proceed by assuming that the user wants the weather in a specific location and the time in a specific timezone. Since the user hasn't provided those, perhaps I need to create the tool calls with placeholders. For example, if the location is \"New York, NY\" and the timezone is \"America/New_York\", then the tool calls would be:\n\n{\n  \"tool_calls\": [,]\n}\n\nBut since the user hasn't specified, this is a placeholder. However, the user might have intended to provide the location and timezone. Since the problem statement doesn't include those, perhaps the user is expecting me to structure the calls with the necessary parameters. Alternatively, maybe the user is just testing the format. In that case, I can proceed with the example as above. I'll make the tool calls for the weather and time, using hypothetical locations and timezones. But since the user hasn't provided specific information, this is the best I can do.\n</think>\n\n```json\n{\n  \"tool_calls\": [,]\n}\n"
}


user
{
  "tool_response": {
    "content": "{\"location\":\"New York, NY\",\"temperature\":10,\"condition\":\"Sunny\",\"humidity\":31}",
    "tool_call_id": "call_1___"
  }
}


user
{
  "tool_response": {
    "content": "{\"timezone\":\"America/New_York\",\"time\":\"Wednesday, July 30, 2025 at 04:59:29\"}",
    "tool_call_id": "call_2___"
  }
}


assistant
...
```
</details

Changed files:
filename: common/chat.cpp
patch: @@ -1646,7 +1646,7 @@ static void common_chat_parse_hermes_2_pro(common_chat_msg_parser & builder) {
         "|<function name=\"([^\"]+)\">"  // match 5 (function name again)
     );
 
-    if (auto res = builder.try_find_regex(open_regex)) {
+    while (auto res = builder.try_find_regex(open_regex)) {
         const auto & block_start = res->groups[1];
         std::string block_end = block_start.empty() ? "" : "```";
 
@@ -1668,7 +1668,6 @@ static void common_chat_parse_hermes_2_pro(common_chat_msg_parser & builder) {
                     builder.consume_literal(block_end);
                     builder.consume_spaces();
                 }
-                builder.add_content(builder.consume_rest());
             } else {
                 throw common_chat_msg_partial_exception("failed to parse tool call");
             }
@@ -1693,11 +1692,10 @@ static void common_chat_parse_hermes_2_pro(common_chat_msg_parser & builder) {
                     builder.consume_spaces();
                 }
             }
-            builder.add_content(builder.consume_rest());
         }
-    } else {
-        builder.add_content(builder.consume_rest());
     }
+
+    builder.add_content(builder.consume_rest());
 }
 
 static common_chat_params common_chat_params_init_without_tools(const common_chat_template & tmpl, const struct templates_params & inputs) {

filename: tests/test-chat.cpp
patch: @@ -953,6 +953,33 @@ static void test_template_output_parsers() {
                 /* is_partial= */ false,
                 {COMMON_CHAT_FORMAT_HERMES_2_PRO}));
 
+        // Test multiple tool calls
+        common_chat_msg message_assist_multiple_calls;
+        message_assist_multiple_calls.role = "assistant";
+        message_assist_multiple_calls.content = "";
+        message_assist_multiple_calls.tool_calls.push_back({"special_function", "{\"arg1\": 1}", ""});
+        message_assist_multiple_calls.tool_calls.push_back({"python", "{\"code\":\"print('hello')\"}", ""});
+
+        assert_msg_equals(
+            message_assist_multiple_calls,
+            common_chat_parse(
+                "<tool_call>\n"
+                "{\"name\": \"special_function\", \"arguments\": {\"arg1\": 1}}\n"
+                "</tool_call>\n"
+                "<tool_call>\n"
+                "{\"name\": \"python\", \"arguments\": {\"code\":\"print('hello')\"}}\n"
+                "</tool_call>",
+                /* is_partial= */ false,
+                {COMMON_CHAT_FORMAT_HERMES_2_PRO}));
+
+        assert_msg_equals(
+            message_assist_multiple_calls,
+            common_chat_parse(
+                "<function=special_function>{\"arg1\": 1}</function>\n"
+                "<function=python>{\"code\":\"print('hello')\"}</function>",
+                /* is_partial= */ false,
+                {COMMON_CHAT_FORMAT_HERMES_2_PRO}));
+
         assert_msg_equals(
             simple_assist_msg(
                 "This is not a tool call:",
@@ -1039,6 +1066,22 @@ static void test_template_output_parsers() {
                       "<tool_call>\n"
                       "{\"name\": \"special_function\", \"arguments\": {\"arg1\": 1}}\n"
                       "</tool_call>");
+
+        // Test multiple tool calls with template
+        common_chat_msg message_assist_multiple_calls_template;
+        message_assist_multiple_calls_template.role = "assistant";
+        message_assist_multiple_calls_template.content = "";
+        message_assist_multiple_calls_template.tool_calls.push_back({"special_function", "{\"arg1\": 1}", ""});
+        message_assist_multiple_calls_template.tool_calls.push_back({"python", "{\"code\":\"print('test')\"}", ""});
+
+        test_templates(tmpls.get(), end_tokens, message_assist_multiple_calls_template, tools,
+                      "<tool_call>\n"
+                      "{\"name\": \"special_function\", \"arguments\": {\"arg1\": 1}}\n"
+                      "</tool_call>\n"
+                      "<tool_call>\n"
+                      "{\"name\": \"python\", \"arguments\": {\"code\":\"print('test')\"}}\n"
+                      "</tool_call>");
+
         test_templates(tmpls.get(), end_tokens, message_assist_call_python_lines, tools,
                       "<tool_call>\n"
                       "{\"name\": \"python\", \"arguments\": {\"code\":\"# This is a program:\\nprint('hey')\"}}\n"