Repository: deepset-ai/haystack
Pull request #9303: fix: make `HuggingFaceAPIChatGenerator` convert Tool Call `arguments` from string
Author: login: YassinNouh21
Labels: name: type:documentation
description: Improvements on the docs
name: topic:tests
Created: 2025-04-24T11:54:49Z | Merged: 2025-04-28T13:36:19Z

### Related Issues

- fixes #9298

### Proposed Changes:

Fixed import sorting in `hugging_face_api.py` to comply with Ruff linting rules. The imports are now properly organized in the standard Python order:
1. Standard library imports
2. Third-party imports
3. Local imports

### How did you test it?

- Verified that all pre-commit hooks pass successfully, including:
  - Ruff linting checks
  - Python AST checks
  - Code formatting
  - Other standard checks
- The code functionality remains unchanged as this was purely a style fix

### Notes for the reviewer

The changes only affect import statement ordering in `hugging_face_api.py`. No functional changes were made to the code.

### Checklist

- [x] I have read the [contributors guidelines](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md) and the [code of conduct](https://github.com/deepset-ai/haystack/blob/main/code_of_conduct.txt)
- [x] I have updated the related issue with new insights and changes
- [ ] I added unit tests and updated the docstrings _(N/A - style fix only)_
- [x] I've used one of the [conventional commit types](https://www.conventionalcommits.org/en/v1.0.0/) for my PR title: `fix:`, `feat:`, `build:`, `chore:`, `ci:`, `docs:`, `style:`, `refactor:`, `perf:`, `test:` and added `!` in case the PR includes breaking changes.
- [x] I documented my code
- [x] I ran [pre-commit hooks](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md#installation) and fixed any issue

Linked issue #9298: `HuggingFaceAPIChatGenerator` incorrectly assumes that `arguments` in a prepared Tool Call is always a dict
Investigating on [`test_live_run_with_tools`](https://github.com/deepset-ai/haystack/blob/df662daaef79993d261a66d5ebb5fadcb9c9b531/test/components/generators/chat/test_hugging_face_api.py#L645), I found out that now `arguments` in a Tool Call prepared via the Hugging Face APIs can be a string instead of a dict.

While this is inconsistent with their [docs](https://huggingface.co/docs/huggingface_hub/v0.30.2/en/package_reference/inference_client#huggingface_hub.InferenceClient.chat_completion.example-7), my impression is that in future they will enforce this behavior ([code in main](https://github.com/huggingface/huggingface_hub/blob/a8e321429b5ba08deb87c696c7660a18a33e2916/src/huggingface_hub/inference/_generated/types/chat_completion.py#L191)).

For this reason, I propose handling the conversion to dict on our side if `arguments` is a string.
Linked issue comments: user: login: YassinNouh21
created_at: 2025-04-24T11:24:23Z
body: Hi @anakin87, I looked into it.  implemented a fix—just wanted to confirm if this approach aligns with what you had in mind:
	•	In both _run_non_streaming and _run_non_streaming_async methods of HuggingFaceAPIChatGenerator, I added a check for hfapi_tc.function.arguments.
	•	If it’s a string, I try to parse it as JSON:
	•	If parsing succeeds, it’s treated as a proper dict.
	•	If parsing fails, I wrap the raw string in a dict under the key "raw_arguments" to maintain structure.
	•	I also updated the ToolCall class comment to reflect this potential inconsistency with Hugging Face’s docs.
	•	Added tests for both valid and invalid JSON cases, covering sync and async paths.
	•	Logged this change in the release notes as well.

Let me know if this handling logic makes sense or if you think it should be done differently!

author_association: CONTRIBUTOR
user: login: anakin87
created_at: 2025-04-24T11:31:49Z
body: Hey, thanks!

In general, I more or less agree with you.

If parsing fails: I would simply show a warning similar to what we do for OpenAI:
https://github.com/deepset-ai/haystack/blob/c3bce46663db606d40066335941aa445137e85ef/haystack/components/generators/chat/openai.py#L546-L548

Feel free to open a PR and tag me.

author_association: MEMBER

Diff:
diff --git a/haystack/components/generators/chat/hugging_face_api.py b/haystack/components/generators/chat/hugging_face_api.py
index d52efd0f13..5c2947787f 100644
--- a/haystack/components/generators/chat/hugging_face_api.py
+++ b/haystack/components/generators/chat/hugging_face_api.py
@@ -2,10 +2,11 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
+import json
 from datetime import datetime
 from typing import Any, AsyncIterable, Dict, Iterable, List, Optional, Union
 
-from haystack import component, default_from_dict, default_to_dict
+from haystack import component, default_from_dict, default_to_dict, logging
 from haystack.dataclasses import ChatMessage, StreamingChunk, ToolCall, select_streaming_callback
 from haystack.dataclasses.streaming_chunk import StreamingCallbackT
 from haystack.lazy_imports import LazyImport
@@ -20,17 +21,65 @@
 from haystack.utils.hf import HFGenerationAPIType, HFModelType, check_valid_model, convert_message_to_hf_format
 from haystack.utils.url_validation import is_valid_http_url
 
+logger = logging.getLogger(__name__)
+
 with LazyImport(message="Run 'pip install \"huggingface_hub[inference]>=0.27.0\"'") as huggingface_hub_import:
     from huggingface_hub import (
         AsyncInferenceClient,
         ChatCompletionInputFunctionDefinition,
         ChatCompletionInputTool,
         ChatCompletionOutput,
+        ChatCompletionOutputToolCall,
         ChatCompletionStreamOutput,
         InferenceClient,
     )
 
 
+def _convert_hfapi_tool_calls(hfapi_tool_calls: Optional[List["ChatCompletionOutputToolCall"]]) -> List[ToolCall]:
+    """
+    Convert HuggingFace API tool calls to a list of Haystack ToolCall.
+
+    :param hfapi_tool_calls: The HuggingFace API tool calls to convert.
+    :returns: A list of ToolCall objects.
+
+    """
+    if not hfapi_tool_calls:
+        return []
+
+    tool_calls = []
+
+    for hfapi_tc in hfapi_tool_calls:
+        hf_arguments = hfapi_tc.function.arguments
+
+        arguments = None
+        if isinstance(hf_arguments, dict):
+            arguments = hf_arguments
+        elif isinstance(hf_arguments, str):
+            try:
+                arguments = json.loads(hf_arguments)
+            except json.JSONDecodeError:
+                logger.warning(
+                    "HuggingFace API returned a malformed JSON string for tool call arguments. This tool call "
+                    "will be skipped. Tool call ID: {_id}, Tool name: {_name}, Arguments: {_arguments}",
+                    _id=hfapi_tc.id,
+                    _name=hfapi_tc.function.name,
+                    _arguments=hf_arguments,
+                )
+        else:
+            logger.warning(
+                "HuggingFace API returned tool call arguments of type {_type}. Valid types are dict and str. This tool "
+                "call will be skipped. Tool call ID: {_id}, Tool name: {_name}, Arguments: {_arguments}",
+                _id=hfapi_tc.id,
+                _name=hfapi_tc.function.name,
+                _arguments=hf_arguments,
+            )
+
+        if arguments:
+            tool_calls.append(ToolCall(tool_name=hfapi_tc.function.name, arguments=arguments, id=hfapi_tc.id))
+
+    return tool_calls
+
+
 @component
 class HuggingFaceAPIChatGenerator:
     """
@@ -403,14 +452,8 @@ def _run_non_streaming(
         choice = api_chat_output.choices[0]
 
         text = choice.message.content
-        tool_calls = []
 
-        if hfapi_tool_calls := choice.message.tool_calls:
-            for hfapi_tc in hfapi_tool_calls:
-                tool_call = ToolCall(
-                    tool_name=hfapi_tc.function.name, arguments=hfapi_tc.function.arguments, id=hfapi_tc.id
-                )
-                tool_calls.append(tool_call)
+        tool_calls = _convert_hfapi_tool_calls(choice.message.tool_calls)
 
         meta: Dict[str, Any] = {
             "model": self._client.model,
@@ -486,14 +529,8 @@ async def _run_non_streaming_async(
         choice = api_chat_output.choices[0]
 
         text = choice.message.content
-        tool_calls = []
 
-        if hfapi_tool_calls := choice.message.tool_calls:
-            for hfapi_tc in hfapi_tool_calls:
-                tool_call = ToolCall(
-                    tool_name=hfapi_tc.function.name, arguments=hfapi_tc.function.arguments, id=hfapi_tc.id
-                )
-                tool_calls.append(tool_call)
+        tool_calls = _convert_hfapi_tool_calls(choice.message.tool_calls)
 
         meta: Dict[str, Any] = {
             "model": self._async_client.model,
diff --git a/releasenotes/notes/hfapichatgenerator-toolcall-str-args-b857e65073ba9f2b.yaml b/releasenotes/notes/hfapichatgenerator-toolcall-str-args-b857e65073ba9f2b.yaml
new file mode 100644
index 0000000000..825b8916f6
--- /dev/null
+++ b/releasenotes/notes/hfapichatgenerator-toolcall-str-args-b857e65073ba9f2b.yaml
@@ -0,0 +1,6 @@
+---
+fixes:
+  - |
+    The `HuggingFaceAPIChatGenerator` now checks the type of the `arguments` variable in the tool calls returned by the
+    Hugging Face API. If `arguments` is a JSON string, it is parsed into a dictionary.
+    Previously, the `arguments` type was not checked, which sometimes led to failures later in the tool workflow.
diff --git a/test/components/generators/chat/test_hugging_face_api.py b/test/components/generators/chat/test_hugging_face_api.py
index b9f21f8f3d..0bf01a7751 100644
--- a/test/components/generators/chat/test_hugging_face_api.py
+++ b/test/components/generators/chat/test_hugging_face_api.py
@@ -23,7 +23,7 @@
 )
 from huggingface_hub.utils import RepositoryNotFoundError
 
-from haystack.components.generators.chat.hugging_face_api import HuggingFaceAPIChatGenerator
+from haystack.components.generators.chat.hugging_face_api import HuggingFaceAPIChatGenerator, _convert_hfapi_tool_calls
 from haystack.tools import Tool
 from haystack.dataclasses import ChatMessage, ToolCall
 from haystack.tools.toolset import Toolset
@@ -573,6 +573,73 @@ def test_run_with_tools(self, mock_check_valid_model, tools):
             "usage": {"completion_tokens": 30, "prompt_tokens": 426},
         }
 
+    def test_convert_hfapi_tool_calls_empty(self):
+        hfapi_tool_calls = None
+        tool_calls = _convert_hfapi_tool_calls(hfapi_tool_calls)
+        assert len(tool_calls) == 0
+
+        hfapi_tool_calls = []
+        tool_calls = _convert_hfapi_tool_calls(hfapi_tool_calls)
+        assert len(tool_calls) == 0
+
+    def test_convert_hfapi_tool_calls_dict_arguments(self):
+        hfapi_tool_calls = [
+            ChatCompletionOutputToolCall(
+                function=ChatCompletionOutputFunctionDefinition(
+                    arguments={"city": "Paris"}, name="weather", description=None
+                ),
+                id="0",
+                type="function",
+            )
+        ]
+        tool_calls = _convert_hfapi_tool_calls(hfapi_tool_calls)
+        assert len(tool_calls) == 1
+        assert tool_calls[0].tool_name == "weather"
+        assert tool_calls[0].arguments == {"city": "Paris"}
+        assert tool_calls[0].id == "0"
+
+    def test_convert_hfapi_tool_calls_str_arguments(self):
+        hfapi_tool_calls = [
+            ChatCompletionOutputToolCall(
+                function=ChatCompletionOutputFunctionDefinition(
+                    arguments='{"city": "Paris"}', name="weather", description=None
+                ),
+                id="0",
+                type="function",
+            )
+        ]
+        tool_calls = _convert_hfapi_tool_calls(hfapi_tool_calls)
+        assert len(tool_calls) == 1
+        assert tool_calls[0].tool_name == "weather"
+        assert tool_calls[0].arguments == {"city": "Paris"}
+        assert tool_calls[0].id == "0"
+
+    def test_convert_hfapi_tool_calls_invalid_str_arguments(self):
+        hfapi_tool_calls = [
+            ChatCompletionOutputToolCall(
+                function=ChatCompletionOutputFunctionDefinition(
+                    arguments="not a valid JSON string", name="weather", description=None
+                ),
+                id="0",
+                type="function",
+            )
+        ]
+        tool_calls = _convert_hfapi_tool_calls(hfapi_tool_calls)
+        assert len(tool_calls) == 0
+
+    def test_convert_hfapi_tool_calls_invalid_type_arguments(self):
+        hfapi_tool_calls = [
+            ChatCompletionOutputToolCall(
+                function=ChatCompletionOutputFunctionDefinition(
+                    arguments=["this", "is", "a", "list"], name="weather", description=None
+                ),
+                id="0",
+                type="function",
+            )
+        ]
+        tool_calls = _convert_hfapi_tool_calls(hfapi_tool_calls)
+        assert len(tool_calls) == 0
+
     @pytest.mark.integration
     @pytest.mark.slow
     @pytest.mark.skipif(
@@ -642,9 +709,6 @@ def test_live_run_serverless_streaming(self):
         not os.environ.get("HF_API_TOKEN", None),
         reason="Export an env var called HF_API_TOKEN containing the Hugging Face token to run this test.",
     )
-    @pytest.mark.xfail(
-        reason="The Hugging Face API can be unstable and this test may fail intermittently", strict=False
-    )
     def test_live_run_with_tools(self, tools):
         """
         We test the round trip: generate tool call, pass tool message, generate response.


Reviews:
user: login: anakin87
body: Thanks again.

I found some minor opportunities for improvement.

Let's also add a release note, as explained [here](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md#release-notes).
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-24T15:15:11Z
user: login: anakin87
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-24T15:48:58Z
user: login: anakin87
body: Before merging this PR, we still need to 
- import the object used in type hint
- add a unit test for the new method 
- add a release note

It would be great if you can do that.

I won't manage to get another look before Monday.
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-04-24T17:31:10Z
user: login: anakin87
body: @YassinNouh21 I took care of what was missing and pushed some refinements.

Thanks again.
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-28T13:35:58Z

Review comments:
diff_hunk: @@ -53,6 +53,10 @@ class ToolCall:
     :param id: The ID of the Tool call.
     :param tool_name: The name of the Tool to call.
     :param arguments: The arguments to call the Tool with.
+                     In most cases, this should be a dictionary. However, some LLM APIs might return
+                     a JSON string instead. In those cases, the consumer of this class needs to handle
+                     the conversion appropriately (e.g., in HuggingFaceAPIChatGenerator we convert
+                     string arguments to dict before creating a ToolCall).
path: haystack/dataclasses/chat_message.py
user: login: anakin87
body: Haystack should take care of creating a `ToolCall` object with `arguments` being a dictionary, so I would remove this part from the docstring
created_at: 2025-04-24T15:05:53Z
author_association: MEMBER
diff_hunk: @@ -407,9 +410,21 @@ def _run_non_streaming(
 
         if hfapi_tool_calls := choice.message.tool_calls:
             for hfapi_tc in hfapi_tool_calls:
-                tool_call = ToolCall(
-                    tool_name=hfapi_tc.function.name, arguments=hfapi_tc.function.arguments, id=hfapi_tc.id
-                )
+                arguments = hfapi_tc.function.arguments
+                if isinstance(arguments, str):
+                    try:
+                        arguments = json.loads(arguments)
+                    except json.JSONDecodeError:
+                        logger.warning(
+                            "HuggingFace API returned a malformed JSON string for tool call arguments. "
+                            "The raw string will be wrapped in a 'raw_arguments' field. "
path: haystack/components/generators/chat/hugging_face_api.py
user: login: anakin87
body: please use the same syntax as [OpenAI](https://github.com/deepset-ai/haystack/blob/f97472329f95dba6df24640902b452e5e28b1488/haystack/components/generators/chat/openai.py#L497)

In case the string is not parsable, let's discard the Tool Call
created_at: 2025-04-24T15:10:26Z
author_association: MEMBER
diff_hunk: @@ -407,9 +410,21 @@ def _run_non_streaming(
 
         if hfapi_tool_calls := choice.message.tool_calls:
             for hfapi_tc in hfapi_tool_calls:
-                tool_call = ToolCall(
-                    tool_name=hfapi_tc.function.name, arguments=hfapi_tc.function.arguments, id=hfapi_tc.id
-                )
+                arguments = hfapi_tc.function.arguments
path: haystack/components/generators/chat/hugging_face_api.py
user: login: anakin87
body: I would extract this logic (which is used in two different places) into a small static method.
Something like
```python
@staticmethod
def _convert_hfapi_tool_call(hfapi_tool_call) -> Optional[ToolCall]:
  ...
```

Once you do so, you can easily add a unit test to test it.
created_at: 2025-04-24T15:14:08Z
author_association: MEMBER
diff_hunk: @@ -384,6 +387,34 @@ def _run_streaming(
 
         return {"replies": [message]}
 
+    @staticmethod
+    def _convert_hfapi_tool_call(hfapi_tc) -> Optional[ToolCall]:
+        """
+        Convert a HuggingFace API tool call to a Haystack ToolCall.
+
+        If the arguments cannot be parsed as JSON, returns None.
path: haystack/components/generators/chat/hugging_face_api.py
user: login: anakin87
body: ```suggestion
        If `arguments` is a string and cannot be parsed as JSON, returns None.
```
created_at: 2025-04-24T15:45:58Z
author_association: MEMBER
diff_hunk: @@ -384,6 +387,34 @@ def _run_streaming(
 
         return {"replies": [message]}
 
+    @staticmethod
+    def _convert_hfapi_tool_call(hfapi_tc) -> Optional[ToolCall]:
path: haystack/components/generators/chat/hugging_face_api.py
user: login: anakin87
body: ```suggestion
    def _convert_hfapi_tool_call(hfapi_tc: "ChatCompletionOutputToolCall") -> Optional[ToolCall]:
```

let's add the type hint and also import `ChatCompletionOutputToolCall` in the `LazyImport` block
created_at: 2025-04-24T15:48:21Z
author_association: MEMBER

PR comments:
user: login: YassinNouh21
created_at: 2025-04-24T11:57:03Z
body: @anakin87 
should i add more tests for it ? 
author_association: CONTRIBUTOR
user: login: CLAassistant
created_at: 2025-04-24T12:15:10Z
body: [![CLA assistant check](https://cla-assistant.io/pull/badge/signed)](https://cla-assistant.io/deepset-ai/haystack?pullRequest=9303) <br/>All committers have signed the CLA.
author_association: NONE
user: login: anakin87
created_at: 2025-04-24T12:17:56Z
body: Thanks for the PR!

Could you please sign the CLA?

I'll review it later.
author_association: MEMBER
user: login: YassinNouh21
created_at: 2025-04-24T12:22:06Z
body: > Thanks for the PR!
> 
> Could you please sign the CLA?
> 
> I'll review it later.

Done
author_association: CONTRIBUTOR
user: login: YassinNouh21
created_at: 2025-04-24T15:07:24Z
body: @anakin87 i fixed the unit test 
author_association: CONTRIBUTOR
user: login: coveralls
created_at: 2025-04-24T15:59:54Z
body: ## Pull Request Test Coverage Report for [Build 14708751227](https://coveralls.io/builds/73413105)


### Details

* **0** of **0**   changed or added relevant lines in **0** files are covered.
* **7** unchanged lines in **1** file lost coverage.
* Overall coverage increased (+**0.01%**) to **90.5%**

---


|  Files with Coverage Reduction | New Missed Lines | % |
| :-----|--------------|--: |
| [components/generators/chat/hugging_face_api.py](https://coveralls.io/builds/73413105/source?filename=components%2Fgenerators%2Fchat%2Fhugging_face_api.py#L219) | 7 | 95.58% |
<!-- | **Total:** | **7** |  | -->

|  Totals | [![Coverage Status](https://coveralls.io/builds/73413105/badge)](https://coveralls.io/builds/73413105) |
| :-- | --: |
| Change from base [Build 14680167461](https://coveralls.io/builds/73396614): |  0.01% |
| Covered Lines: | 10879 |
| Relevant Lines: | 12021 |

---
##### 💛  - [Coveralls](https://coveralls.io)

author_association: COLLABORATOR
user: login: YassinNouh21
created_at: 2025-04-24T16:47:36Z
body: @anakin87 can u take a look 
author_association: CONTRIBUTOR