Repository: run-llama/llama_index
Pull request: fix: :bug: [#17555] fix openai.BadRequestError: Invalid value for 'content': expected a string, got null.
# Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #17555 

## Type of Change

Please delete options that are not relevant.

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

Your pull-request will likely not be merged unless it is covered by some form of impactful unit testing.

- [ ] I added new / edit existing unit tests to cover this change
- [x] I believe this change is already covered by existing unit tests

## Suggested Checklist:

- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] I have added Google Colab support for the newly added notebooks.
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I ran `make format; make lint` to appease the lint gods


Problem statement:
[Bug]: openai.BadRequestError: Error code: 400. Invalid value for 'content': expected a string, got null
### Bug Description

OpenAI Didnt Accept ChatMessage Content with Null value, only Accept String.

This issue happened specially when Agent Tools return None or empy string, such as RetrieverTool with Empty Result

### Version

0.12.9 to 0.12.11 (llama-index-llms-openai 0.3.10 to 0.3.13)

### Steps to Reproduce

Create Agent With Retriever Tool
Add Postprocessors to filter out nodes (I Use LLM Rerank)
Adjust filter so that you get 0 Nodes in the end. 



### Relevant Logs/Tracbacks

```shell
Traceback (most recent call last):
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py", line 367, in async_wrapper
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\llama_index\core\chat_engine\types.py", line 234, in awrite_response_to_history       
    async for chat in self.achat_stream:
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\llama_index\core\llms\callbacks.py", line 88, in wrapped_gen
    async for x in f_return_val:
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\llama_index\llms\openai\base.py", line 724, in gen
    async for response in await aclient.chat.completions.create(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\openai\resources\chat\completions.py", line 1720, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\openai\_base_client.py", line 1849, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\openai\_base_client.py", line 1543, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\openai\_base_client.py", line 1644, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "Invalid value for 'content': expected a string, got null.", 'type': 'invalid_request_error', 'param': 'messages.[3].content', 'code': None}}
INFO:     127.0.0.1:51858 - "POST /query HTTP/1.1" 200 OK
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 409, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\fastapi\applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\starlette\applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\starlette\middleware\errors.py", line 187, in __call__
    raise exc
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\starlette\middleware\errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\starlette\middleware\base.py", line 185, in __call__
    with collapse_excgroups():
         ^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\starlette\_utils.py", line 82, in collapse_excgroups
    raise exc
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\starlette\middleware\base.py", line 188, in __call__
    await response(scope, wrapped_receive, send)
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\starlette\middleware\base.py", line 222, in __call__
    async for chunk in self.body_iterator:
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\starlette\middleware\base.py", line 179, in body_stream
    raise app_exc
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\starlette\middleware\base.py", line 149, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\starlette\middleware\base.py", line 185, in __call__
    with collapse_excgroups():
         ^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\starlette\_utils.py", line 82, in collapse_excgroups
    raise exc
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\starlette\responses.py", line 255, in wrap
    await func()
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\starlette\responses.py", line 244, in stream_response
    async for chunk in self.body_iterator:
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\app\index_server.py", line 295, in gen
    async for token in my_query.async_response_gen():
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\llama_index\core\chat_engine\types.py", line 310, in async_response_gen
    raise self.exception
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\llama_index\core\instrumentation\dispatcher.py", line 367, in async_wrapper
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\llama_index\core\chat_engine\types.py", line 234, in awrite_response_to_history       
    async for chat in self.achat_stream:
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\llama_index\core\llms\callbacks.py", line 88, in wrapped_gen
    async for x in f_return_val:
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\llama_index\llms\openai\base.py", line 724, in gen
    async for response in await aclient.chat.completions.create(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\openai\resources\chat\completions.py", line 1720, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\openai\_base_client.py", line 1849, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\openai\_base_client.py", line 1543, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\habi6\Desktop\PROJECT\aidoc-llm-service\.venv\Lib\site-packages\openai\_base_client.py", line 1644, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "Invalid value for 'content': expected a string, got null.", 'type': 'invalid_request_error', 'param': 'messages.[3].content', 'code': None}}
```

Hints from issue discussion:
<!-- Answer -->
To resolve the `openai.BadRequestError` with error code 400 due to an invalid value for 'content' when using the `RetrieverTool` with an empty result in version 0.12.11 of `llama-index-llms-openai`, ensure that the `content` field is not set to an empty string. You can handle this by setting `content_txt` to `None` if it is an empty string:

```python
content_txt = None if content_txt == "" else content_txt
```

This line ensures that if the `content_txt` is empty, it is set to `None` instead of an empty string, which prevents the error when sending the message to OpenAI <sup>[[1]](https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/llms/llama-index-llms-openai/llama_index/llms/openai/utils.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*

---
<sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/97cc08a7-0bdd-4083-be88-d11a8ab0c303?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/97cc08a7-0bdd-4083-be88-d11a8ab0c303?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/97cc08a7-0bdd-4083-be88-d11a8ab0c303?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/97cc08a7-0bdd-4083-be88-d11a8ab0c303?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/97cc08a7-0bdd-4083-be88-d11a8ab0c303?feedback_type=hallucination) | [Other](https://app.dosu.dev/response-feedback/97cc08a7-0bdd-4083-be88-d11a8ab0c303?feedback_type=other)</sup>

no, openai cannot recieve None (null) value, their api only accept for string, empty string `""` is allowed
i already fix this with my PR, please check my PR: https://github.com/run-llama/llama_index/pull/17556

Commit message: Merge branch 'main' into fix/openai.BadRequestError--Error-code--400-Invalid-value-for-'content'--expected-a-string,-got-null-

Patch to review:
diff --git a/llama-index-integrations/llms/llama-index-llms-openai/llama_index/llms/openai/utils.py b/llama-index-integrations/llms/llama-index-llms-openai/llama_index/llms/openai/utils.py
index 6e48083d24..4f1c2d582e 100644
--- a/llama-index-integrations/llms/llama-index-llms-openai/llama_index/llms/openai/utils.py
+++ b/llama-index-integrations/llms/llama-index-llms-openai/llama_index/llms/openai/utils.py
@@ -2,7 +2,13 @@
 import os
 from typing import Any, Callable, Dict, List, Optional, Sequence, Tuple, Type, Union
 
+import openai
 from deprecated import deprecated
+from openai.types.chat import ChatCompletionMessageParam, ChatCompletionMessageToolCall
+from openai.types.chat.chat_completion_chunk import ChoiceDeltaToolCall
+from openai.types.chat.chat_completion_message import ChatCompletionMessage
+from openai.types.chat.chat_completion_token_logprob import ChatCompletionTokenLogprob
+from openai.types.completion_choice import Logprobs
 from tenacity import (
     before_sleep_log,
     retry,
@@ -14,20 +20,9 @@
 )
 from tenacity.stop import stop_base
 
-import openai
 from llama_index.core.base.llms.generic_utils import get_from_param_or_env
-from llama_index.core.base.llms.types import (
-    ChatMessage,
-    ImageBlock,
-    LogProb,
-    TextBlock,
-)
+from llama_index.core.base.llms.types import ChatMessage, ImageBlock, LogProb, TextBlock
 from llama_index.core.bridge.pydantic import BaseModel
-from openai.types.chat import ChatCompletionMessageParam, ChatCompletionMessageToolCall
-from openai.types.chat.chat_completion_chunk import ChoiceDeltaToolCall
-from openai.types.chat.chat_completion_message import ChatCompletionMessage
-from openai.types.chat.chat_completion_token_logprob import ChatCompletionTokenLogprob
-from openai.types.completion_choice import Logprobs
 
 DEFAULT_OPENAI_API_TYPE = "open_ai"
 DEFAULT_OPENAI_API_BASE = "https://api.openai.com/v1"
@@ -292,20 +287,18 @@ def to_openai_message_dict(
             msg = f"Unsupported content block type: {type(block).__name__}"
             raise ValueError(msg)
 
-    # NOTE: Sending a blank string to openai will cause an error.
-    # This will commonly happen with tool calls.
-    content_txt = None if content_txt == "" else content_txt
-
     # NOTE: Despite what the openai docs say, if the role is ASSISTANT, SYSTEM
     # or TOOL, 'content' cannot be a list and must be string instead.
     # Furthermore, if all blocks are text blocks, we can use the content_txt
     # as the content. This will avoid breaking openai-like APIs.
     message_dict = {
         "role": message.role.value,
-        "content": content_txt
-        if message.role.value in ("assistant", "tool", "system")
-        or all(isinstance(block, TextBlock) for block in message.blocks)
-        else content,
+        "content": (
+            content_txt
+            if message.role.value in ("assistant", "tool", "system")
+            or all(isinstance(block, TextBlock) for block in message.blocks)
+            else content
+        ),
     }
 
     # TODO: O1 models do not support system prompts
diff --git a/llama-index-integrations/llms/llama-index-llms-openai/tests/test_openai_utils.py b/llama-index-integrations/llms/llama-index-llms-openai/tests/test_openai_utils.py
index 9f7ec48c5e..b14e223b1e 100644
--- a/llama-index-integrations/llms/llama-index-llms-openai/tests/test_openai_utils.py
+++ b/llama-index-integrations/llms/llama-index-llms-openai/tests/test_openai_utils.py
@@ -59,7 +59,7 @@ def chat_messages_with_function_calling() -> List[ChatMessage]:
 
 
 @pytest.fixture()
-def openi_message_dicts_with_function_calling() -> List[ChatCompletionMessageParam]:
+def openai_message_dicts_with_function_calling() -> List[ChatCompletionMessageParam]:
     return [
         {
             "role": "user",
@@ -67,7 +67,7 @@ def openi_message_dicts_with_function_calling() -> List[ChatCompletionMessagePar
         },
         {
             "role": "assistant",
-            "content": None,
+            "content": "",
             "function_call": {
                 "name": "get_current_weather",
                 "arguments": '{ "location": "Boston, MA"}',
@@ -158,19 +158,19 @@ def test_to_openai_message_dicts_basic_string() -> None:
 
 def test_to_openai_message_dicts_function_calling(
     chat_messages_with_function_calling: List[ChatMessage],
-    openi_message_dicts_with_function_calling: List[ChatCompletionMessageParam],
+    openai_message_dicts_with_function_calling: List[ChatCompletionMessageParam],
 ) -> None:
     message_dicts = to_openai_message_dicts(
         chat_messages_with_function_calling,
     )
-    assert message_dicts == openi_message_dicts_with_function_calling
+    assert message_dicts == openai_message_dicts_with_function_calling
 
 
 def test_from_openai_message_dicts_function_calling(
-    openi_message_dicts_with_function_calling: List[ChatCompletionMessageParam],
+    openai_message_dicts_with_function_calling: List[ChatCompletionMessageParam],
     chat_messages_with_function_calling: List[ChatMessage],
 ) -> None:
-    chat_messages = from_openai_message_dicts(openi_message_dicts_with_function_calling)  # type: ignore
+    chat_messages = from_openai_message_dicts(openai_message_dicts_with_function_calling)  # type: ignore
 
     # assert attributes match
     for chat_message, chat_message_with_function_calling in zip(


Code review comments:
diff_hunk: @@ -292,20 +287,18 @@ def to_openai_message_dict(
             msg = f"Unsupported content block type: {type(block).__name__}"
             raise ValueError(msg)
 
-    # NOTE: Sending a blank string to openai will cause an error.
-    # This will commonly happen with tool calls.
path: llama-index-integrations/llms/llama-index-llms-openai/llama_index/llms/openai/utils.py
text: @user1:
@author I'm pretty sure this introduces another bug.

OpenAI DOES accept None, for example, when making a tool call

```
{"role": "assistant", "content": None, tool_calls: ...}
{"role": "tool", "content": "tool output", "tool_call_id": ...}
```
![image](https://github.com/user-attachments/assets/5b9d1e31-1a29-400f-8890-b0faee2447fc)

I'm pretty sure this change introduces another bug. By sending an empty string, it will cause another error.

I think the issue here is that tool outputs cannot be None (which is what is happening), and this fix is overcorrecting 

![image](https://github.com/user-attachments/assets/37a0ffb7-d2b4-4003-98c0-45e2bb465b3a)

The correct fix should check the roles I think

@author:
i think we cannot make sure the tool output is not None, except if it's a custom tool (which in my case it isn't). the product that i develop requires high data accuracy and precission, that's why we only includes the nodes that relevant, we filter out with LLM Rerank and if the output is None (0 Nodes Passed the filtration) then we must let the Agent to retrieve again using the RetrieverTool.

in my testing, setting it to empty string does fix this kind of issue and doesn't introduce new bugs.

@author:
How about this, if we follow the OpenAI API Reference, we can see that its only allowed to send null if its an Assistant Message, i have changed the code to set it to None only if the MessageRole = Assistant, because for other than that (ex: Tool Message), its always required