Repository: run-llama/llama_index
Pull request: fix react system header formatting in workflow agent
Fixes https://github.com/run-llama/llama_index/issues/19156

We need to set the context properly and respect any existing context that the user may have set on the formatter

Problem statement:
[Bug]: ReactAgent workflow crashes with context in formatter
### Bug Description

Hi,

I noticed that the `llama_index.core.agent.workflow.ReActAgent` workflow crashes when setting a `ReActChatFormatter` formatter with  a context.

Using `ReActChatFormatter.from_defaults()` works fine but using `ReActChatFormatter.from_defaults(context="This is some context")` produces a `KeyError`.

### Version

0.12.43

### Steps to Reproduce

```python
import asyncio

from llama_index.core.agent import ReActChatFormatter
from llama_index.core.agent.workflow import ReActAgent
from llama_index.core.base.llms.types import ChatMessage
from llama_index.llms.openai import OpenAI


async def main() -> None:
    llm = OpenAI(model="gpt-4o")

    formatter = ReActChatFormatter.from_defaults(context="This is some context")

    agent = ReActAgent(
        tools=[], llm=llm, formatter=formatter
    )

    usr_msg = ChatMessage(role="user", content="hello")

    await agent.run(user_msg=usr_msg)


if __name__ == "__main__":
    asyncio.run(main())

```

### Relevant Logs/Tracbacks

```shell
Traceback (most recent call last):
  File "/home/root/test/.venv/lib/python3.12/site-packages/workflows/context/context.py", line 628, in _step_worker
    new_ev = await instrumented_step(**kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/root/test/.venv/lib/python3.12/site-packages/llama_index_instrumentation/dispatcher.py", line 365, in async_wrapper
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/root/test/.venv/lib/python3.12/site-packages/llama_index/core/agent/workflow/base_agent.py", line 346, in run_agent_step
    agent_output = await self.take_step(
                   ^^^^^^^^^^^^^^^^^^^^^
  File "/home/root/test/.venv/lib/python3.12/site-packages/llama_index/core/agent/workflow/react_agent.py", line 86, in take_step
    input_chat = react_chat_formatter.format(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/root/test/.venv/lib/python3.12/site-packages/llama_index/core/agent/react/formatter.py", line 81, in format
    fmt_sys_header = self.system_header.format(**format_args)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'context'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/root/.local/share/uv/python/cpython-3.12.10-linux-x86_64-gnu/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/home/root/test/.venv/lib/python3.12/site-packages/llama_index_instrumentation/dispatcher.py", line 290, in handle_future_result
    raise exception
  File "/home/root/test/.venv/lib/python3.12/site-packages/workflows/workflow.py", line 416, in _run_workflow
    raise exception_raised
  File "/home/root/test/.venv/lib/python3.12/site-packages/workflows/context/context.py", line 637, in _step_worker
    raise WorkflowRuntimeError(
workflows.errors.WorkflowRuntimeError: Error in step 'run_agent_step': 'context'
Traceback (most recent call last):
  File "/home/root/test/.venv/lib/python3.12/site-packages/workflows/context/context.py", line 628, in _step_worker
    new_ev = await instrumented_step(**kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/root/test/.venv/lib/python3.12/site-packages/llama_index_instrumentation/dispatcher.py", line 365, in async_wrapper
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/root/test/.venv/lib/python3.12/site-packages/llama_index/core/agent/workflow/base_agent.py", line 346, in run_agent_step
    agent_output = await self.take_step(
                   ^^^^^^^^^^^^^^^^^^^^^
  File "/home/root/test/.venv/lib/python3.12/site-packages/llama_index/core/agent/workflow/react_agent.py", line 86, in take_step
    input_chat = react_chat_formatter.format(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/root/test/.venv/lib/python3.12/site-packages/llama_index/core/agent/react/formatter.py", line 81, in format
    fmt_sys_header = self.system_header.format(**format_args)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'context'
```

Hints from issue discussion:
<!-- Answer -->
This KeyError happens because when you create a ReActChatFormatter with a context, it sets up a system_header template that expects a {context} variable. If, at any point, the context attribute is missing or empty when format() is called, the formatter tries to fill in the template without the required context key, which triggers the KeyError. This is a mismatch between the template and the arguments provided to it during formatting. You can see this logic in the formatter's implementation: if self.context is set, it adds "context" to format_args; otherwise, it doesn't, but the template may still expect it, leading to the error if context is missing or empty at runtime [formatter.py source](https://github.com/run-llama/llama_index/blob/0b77334fe2df678dd409762f8a62a511ccfc1b20/llama-index-core/llama_index/core/agent/react/formatter.py).

To work around this, make sure that the context attribute is always set (not None or empty) whenever you use a formatter that expects a context in its system_header template. If you need to dynamically update the context, ensure that the new value is not empty before calling format(). Alternatively, you could modify the formatter to use a safer formatting approach or always include a default context value when the template expects it.

There are no existing issues or recent changes in the repo that address this specific bug. If you want a more robust fix, consider updating the formatter logic to always provide a context key (even if it's an empty string) when the template expects it, or use a template that doesn't require context when none is provided.


<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*

---
<sup>How did I do? [Good](https://app.dosu.dev/response-feedback/5746c88b-c3e3-413c-8bb7-734f324a5b57?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/5746c88b-c3e3-413c-8bb7-734f324a5b57?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/5746c88b-c3e3-413c-8bb7-734f324a5b57?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/5746c88b-c3e3-413c-8bb7-734f324a5b57?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/5746c88b-c3e3-413c-8bb7-734f324a5b57?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/5746c88b-c3e3-413c-8bb7-734f324a5b57?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/5746c88b-c3e3-413c-8bb7-734f324a5b57?feedback_type=other)</sup>&nbsp;&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/run-llama/llama_index/issues/19156)
@dosu the formatter and its context remain unchanged after being set in my steps to reproduce code.
<!-- Answer -->
The KeyError happens because, even though you set the context when creating the formatter, the ReActAgent workflow overwrites the formatter's context with the system prompt from the LLM input during execution. If the system prompt is missing or empty at that point, the context becomes empty, but the formatter's template still expects a {context} key—causing the crash. This is a side effect of how the agent workflow manages context, not a bug in your usage or in the formatter itself. You can see this logic in the agent's take_step method, where it sets the formatter's context to the system prompt before formatting the input chat messages ([source](https://github.com/run-llama/llama_index/blob/0b77334fe2df678dd409762f8a62a511ccfc1b20/llama-index-core/llama_index/core/agent/workflow/react_agent.py)).

A workaround is to ensure the system prompt is always set when running the agent, or to modify the agent/formatter logic to provide a default context if none is present. This architectural quirk is not currently addressed by any recent changes or issues in the repo.


<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*

---
<sup>How did I do? [Good](https://app.dosu.dev/response-feedback/4c963789-8e6b-42ab-b408-ec140b0a63f7?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/4c963789-8e6b-42ab-b408-ec140b0a63f7?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/4c963789-8e6b-42ab-b408-ec140b0a63f7?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/4c963789-8e6b-42ab-b408-ec140b0a63f7?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/4c963789-8e6b-42ab-b408-ec140b0a63f7?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/4c963789-8e6b-42ab-b408-ec140b0a63f7?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/4c963789-8e6b-42ab-b408-ec140b0a63f7?feedback_type=other)</sup>&nbsp;&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/run-llama/llama_index/issues/19156)
Alright. I could make it work by doing the following:

```python
agent = ReActAgent(
    tools=tool_list, llm=llm, system_prompt="this is some context",
    formatter=ReActChatFormatter.from_defaults(system_header=CONTEXT_REACT_CHAT_SYSTEM_HEADER)
)
```

It seems to be necessary to set the formatter with `CONTEXT_REACT_CHAT_SYSTEM_HEADER`. It would be nice to create a system that choses the right system header according to whether `system_prompt` is set or not. Or at least write this somewhere in the doc.

Thanks @dosu
@Florian-BACHO this is actually a bug. But also, I really recommend just setting the `system_prompt` in the agent (which ends up getting injected as context in the react header)

If you use both, it starts to get a little confusing 😅 But I can make a PR to fix
@logan-markewich setting the system prompt is not enough to inject it. It seems that we have to explicitly set `formatter=ReActChatFormatter.from_defaults(system_header=CONTEXT_REACT_CHAT_SYSTEM_HEADER)` too (but without context). The default react header does not include the context

Commit message: fix react system header formatting in workflow agent

Patch to review:
diff --git a/llama-index-core/llama_index/core/agent/workflow/react_agent.py b/llama-index-core/llama_index/core/agent/workflow/react_agent.py
index 7e2b98c5cb..09d03b802c 100644
--- a/llama-index-core/llama_index/core/agent/workflow/react_agent.py
+++ b/llama-index-core/llama_index/core/agent/workflow/react_agent.py
@@ -77,7 +77,12 @@ async def take_step(
 
         output_parser = self.output_parser
         react_chat_formatter = self.formatter
-        react_chat_formatter.context = system_prompt
+
+        existing_context = react_chat_formatter.context
+        if existing_context and system_prompt:
+            react_chat_formatter.context = system_prompt + "\n\n" + existing_context
+        elif system_prompt:
+            react_chat_formatter.context = system_prompt
 
         # Format initial chat input
         current_reasoning: list[BaseReasoningStep] = await ctx.get(


Code review comments:
diff_hunk: @@ -77,7 +77,12 @@ async def take_step(
 
         output_parser = self.output_parser
         react_chat_formatter = self.formatter
-        react_chat_formatter.context = system_prompt
+
+        existing_context = react_chat_formatter.context
+        if existing_context and system_prompt:
+            react_chat_formatter.context = system_prompt + "\n\n" + existing_context
path: llama-index-core/llama_index/core/agent/workflow/react_agent.py
text: @user1:
Prependding the formatter context is not a good idea as it will accumulate the system prompt.
For example:
```python
import asyncio

from llama_index.core.agent import ReActChatFormatter
from llama_index.core.agent.workflow import ReActAgent
from llama_index.core.base.llms.types import ChatMessage
from llama_index.llms.openai import OpenAI


async def main() -> None:
    llm = OpenAI(model="gpt-4o")

    formatter = ReActChatFormatter.from_defaults()

    agent = ReActAgent(
        tools=[], llm=llm, formatter=formatter, system_prompt="This is some context",
    )

    usr_msg = ChatMessage(role="user", content="hello")

    for _ in range(3):
        await agent.run(user_msg=usr_msg)

    print(agent.formatter.context)


if __name__ == "__main__":
    asyncio.run(main())
```

Produces:

```
This is some context

This is some context

This is some context
```

@author:
ah shoot good catch. This probably belong in a field validator instead so that it only runs once

@author:
@user2-BACHO pushed a change, it will only do this once on startup