Repository: deepset-ai/haystack

Issue:
Allow Chat Generators to connect to Answer Builder
**Is your feature request related to a problem? Please describe.**
There is no current way for a `ChatGenerator` to directly connect to `AnswerBuilder()` despite similar functionality to a regular Generator.

**Describe the solution you'd like**
We can extract the text from a `ChatMessage` and use it as a `str` as expected in the code for AnswerBuilder.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
ChatGenerators that rely on non-open ai models require a specific chat format to be applied to properly differentiate between system/user/assistant messages. This is done manually (and prone to errors) in regular generators. This is an example I pulled from a Haystack notebook:

```
prompt_template = """


user



Using the information contained in the context, give a comprehensive answer to the question.
If the answer cannot be deduced from the context, do not give an answer.

Context:
  {% for doc in documents %}
  {{ doc.content }} URL:{{ doc.meta['url'] }}
  {% endfor %};
  Question: {{query}}



assistant



"""
prompt_builder = PromptBuilder(template=prompt_template)
```

This can be accomplished by using a `ChatGenerator` as follows:

```
system_message = ChatMessage.from_system(
    """
    Read the context provided and answer the question if possible.If you can not form an answer from the context, reply with "Nah".
    Context:
    {% for doc in documents %}
    {{ doc.content }}
    {% endfor %};
    """
)
user_message = ChatMessage.from_user("query: {{query}}")
assistent_message = ChatMessage.from_assistant("Answer: ")
```

We can then add the output to an `AnswerBuilder` to complete a pipeline ensuring that the proper template is applied.


Discussion:
would also like to add that i have implemented this locally for my hyrbrid pipeline that utilizes LlamaCppChatGenerator and the new ChatPromptBuilder. Everything seems to work. It's just lacking the .meta support, but that can be added at a different time if needed.
@lbux What approach did you use? Did you create a custom component?
@anakin87 There is no need to write an additional component. A `ChatMessage` already contains the necessary information and would only require 3-4 new lines in `AnswerBuilder`. My quick and dirty implementation was allowing for `ChatMessages` or `str` (the default) as input. If a `str` is passed in, we keep the same logic that exists in `AnswerBuilder`. If a `ChatMessage` is passed in, we extract the message string and reuse the existing `str` logic. 
@lbux this is reasonable and useful.

Would you like to open a PR with your changes to the `AnswerBuilder`?

Resolving patch:
diff --git a/haystack/components/builders/answer_builder.py b/haystack/components/builders/answer_builder.py
--- a/haystack/components/builders/answer_builder.py
+++ b/haystack/components/builders/answer_builder.py
@@ -3,9 +3,10 @@
 # SPDX-License-Identifier: Apache-2.0
 
 import re
-from typing import Any, Dict, List, Optional
+from typing import Any, Dict, List, Optional, Union
 
 from haystack import Document, GeneratedAnswer, component, logging
+from haystack.dataclasses.chat_message import ChatMessage
 
 logger = logging.getLogger(__name__)
 
@@ -56,7 +57,7 @@ def __init__(self, pattern: Optional[str] = None, reference_pattern: Optional[st
     def run(
         self,
         query: str,
-        replies: List[str],
+        replies: Union[List[str], List[ChatMessage]],
         meta: Optional[List[Dict[str, Any]]] = None,
         documents: Optional[List[Document]] = None,
         pattern: Optional[str] = None,
@@ -68,7 +69,7 @@ def run(
         :param query:
             The query used in the prompts for the Generator.
         :param replies:
-            The output of the Generator.
+            The output of the Generator. Can be a list of strings or a list of ChatMessage objects.
         :param meta:
             The metadata returned by the Generator. If not specified, the generated answer will contain no metadata.
         :param documents:
@@ -103,14 +104,15 @@ def run(
 
         pattern = pattern or self.pattern
         reference_pattern = reference_pattern or self.reference_pattern
-
         all_answers = []
         for reply, metadata in zip(replies, meta):
+            # Extract content from ChatMessage objects if reply is a ChatMessages, else use the string as is
+            extracted_reply: str = reply.content if isinstance(reply, ChatMessage) else reply  # type: ignore
+            extracted_metadata = reply.meta if isinstance(reply, ChatMessage) else metadata
             referenced_docs = []
             if documents:
-                reference_idxs = []
                 if reference_pattern:
-                    reference_idxs = AnswerBuilder._extract_reference_idxs(reply, reference_pattern)
+                    reference_idxs = AnswerBuilder._extract_reference_idxs(extracted_reply, reference_pattern)
                 else:
                     reference_idxs = [doc_idx for doc_idx, _ in enumerate(documents)]
 
@@ -122,8 +124,10 @@ def run(
                             "Document index '{index}' referenced in Generator output is out of range. ", index=idx + 1
                         )
 
-            answer_string = AnswerBuilder._extract_answer_string(reply, pattern)
-            answer = GeneratedAnswer(data=answer_string, query=query, documents=referenced_docs, meta=metadata)
+            answer_string = AnswerBuilder._extract_answer_string(extracted_reply, pattern)
+            answer = GeneratedAnswer(
+                data=answer_string, query=query, documents=referenced_docs, meta=extracted_metadata
+            )
             all_answers.append(answer)
 
         return {"answers": all_answers}


Test patch:
diff --git a/test/components/builders/test_answer_builder.py b/test/components/builders/test_answer_builder.py
--- a/test/components/builders/test_answer_builder.py
+++ b/test/components/builders/test_answer_builder.py
@@ -5,8 +5,9 @@
 
 import pytest
 
-from haystack import GeneratedAnswer, Document
+from haystack import Document, GeneratedAnswer
 from haystack.components.builders.answer_builder import AnswerBuilder
+from haystack.dataclasses.chat_message import ChatMessage, ChatRole
 
 
 class TestAnswerBuilder:
@@ -150,3 +151,124 @@ def test_run_with_reference_pattern_set_at_runtime(self):
         assert len(answers[0].documents) == 2
         assert answers[0].documents[0].content == "test doc 2"
         assert answers[0].documents[1].content == "test doc 3"
+
+    def test_run_with_chat_message_replies_without_pattern(self):
+        component = AnswerBuilder()
+        replies = [
+            ChatMessage(
+                content="Answer: AnswerString",
+                role=ChatRole.ASSISTANT,
+                name=None,
+                meta={
+                    "model": "gpt-3.5-turbo-0613",
+                    "index": 0,
+                    "finish_reason": "stop",
+                    "usage": {"prompt_tokens": 32, "completion_tokens": 153, "total_tokens": 185},
+                },
+            )
+        ]
+        output = component.run(query="test query", replies=replies, meta=[{}])
+        answers = output["answers"]
+        assert len(answers) == 1
+        assert answers[0].data == "Answer: AnswerString"
+        assert answers[0].meta == {
+            "model": "gpt-3.5-turbo-0613",
+            "index": 0,
+            "finish_reason": "stop",
+            "usage": {"prompt_tokens": 32, "completion_tokens": 153, "total_tokens": 185},
+        }
+        assert answers[0].query == "test query"
+        assert answers[0].documents == []
+        assert isinstance(answers[0], GeneratedAnswer)
+
+    def test_run_with_chat_message_replies_with_pattern(self):
+        component = AnswerBuilder(pattern=r"Answer: (.*)")
+        replies = [
+            ChatMessage(
+                content="Answer: AnswerString",
+                role=ChatRole.ASSISTANT,
+                name=None,
+                meta={
+                    "model": "gpt-3.5-turbo-0613",
+                    "index": 0,
+                    "finish_reason": "stop",
+                    "usage": {"prompt_tokens": 32, "completion_tokens": 153, "total_tokens": 185},
+                },
+            )
+        ]
+        output = component.run(query="test query", replies=replies, meta=[{}])
+        answers = output["answers"]
+        assert len(answers) == 1
+        assert answers[0].data == "AnswerString"
+        assert answers[0].meta == {
+            "model": "gpt-3.5-turbo-0613",
+            "index": 0,
+            "finish_reason": "stop",
+            "usage": {"prompt_tokens": 32, "completion_tokens": 153, "total_tokens": 185},
+        }
+        assert answers[0].query == "test query"
+        assert answers[0].documents == []
+        assert isinstance(answers[0], GeneratedAnswer)
+
+    def test_run_with_chat_message_replies_with_documents(self):
+        component = AnswerBuilder(reference_pattern="\\[(\\d+)\\]")
+        replies = [
+            ChatMessage(
+                content="Answer: AnswerString[2]",
+                role=ChatRole.ASSISTANT,
+                name=None,
+                meta={
+                    "model": "gpt-3.5-turbo-0613",
+                    "index": 0,
+                    "finish_reason": "stop",
+                    "usage": {"prompt_tokens": 32, "completion_tokens": 153, "total_tokens": 185},
+                },
+            )
+        ]
+        output = component.run(
+            query="test query",
+            replies=replies,
+            meta=[{}],
+            documents=[Document(content="test doc 1"), Document(content="test doc 2")],
+        )
+        answers = output["answers"]
+        assert len(answers) == 1
+        assert answers[0].data == "Answer: AnswerString[2]"
+        assert answers[0].meta == {
+            "model": "gpt-3.5-turbo-0613",
+            "index": 0,
+            "finish_reason": "stop",
+            "usage": {"prompt_tokens": 32, "completion_tokens": 153, "total_tokens": 185},
+        }
+        assert answers[0].query == "test query"
+        assert len(answers[0].documents) == 1
+        assert answers[0].documents[0].content == "test doc 2"
+
+    def test_run_with_chat_message_replies_with_pattern_set_at_runtime(self):
+        component = AnswerBuilder(pattern="unused pattern")
+        replies = [
+            ChatMessage(
+                content="Answer: AnswerString",
+                role=ChatRole.ASSISTANT,
+                name=None,
+                meta={
+                    "model": "gpt-3.5-turbo-0613",
+                    "index": 0,
+                    "finish_reason": "stop",
+                    "usage": {"prompt_tokens": 32, "completion_tokens": 153, "total_tokens": 185},
+                },
+            )
+        ]
+        output = component.run(query="test query", replies=replies, meta=[{}], pattern=r"Answer: (.*)")
+        answers = output["answers"]
+        assert len(answers) == 1
+        assert answers[0].data == "AnswerString"
+        assert answers[0].meta == {
+            "model": "gpt-3.5-turbo-0613",
+            "index": 0,
+            "finish_reason": "stop",
+            "usage": {"prompt_tokens": 32, "completion_tokens": 153, "total_tokens": 185},
+        }
+        assert answers[0].query == "test query"
+        assert answers[0].documents == []
+        assert isinstance(answers[0], GeneratedAnswer)