Repository: deepset-ai/haystack

Issue:
The new `PromptNode` cannot work with a `Question-Answering` Pipeline with `Retrievers`
**Describe the bug**
The new `PromptNode` cannot work with a `Question-Answering` Pipeline with `Retrievers`, because the `Retriever` expects the `query` parameter as a string, while the `PromptNode` assumes that all supplied parameters are list and NOT strings. Strings will be silently handled as a list by the `PromptNode` and so split into characters by [this line](https://github.com/deepset-ai/haystack/blob/main/haystack/nodes/prompt/prompt_node.py#L658).

Currently the `PromptNode` will split that string as if it would be a list, so it will only use the first character of the `query` string to be substituted into the constructed prompt.

Mock-example:
```
prompt_node = PromptNode(prompt_model, default_prompt_template="question-answering")

pipeline = Pipeline()
pipeline.add_node(component=bm25_retriever, name="bm25_retriever", inputs=["Query"])
pipeline.add_node(component=prompt_node, name="prompt_node", inputs=["bm25_retriever"])
output = pipeline.run(query="What are subscription warrants?")  # <= if `query` is a list, then that won't work with the retriever,
                                                              # while if it is a string, then that won't with with the PromptNode
```

For the above example the `PromptNode` will construct the prompt such as:
`Given the context please answer the question. Context: ....; Question: W; Answer:'

Observe the `W` in the place of the question inside the prompt. This happens because the `query="What are subscription warrants?"` will be split as if it would be a list, so its first character will be inserted into the prompt instead of the whole question.

**Error message**
No error message, but the prompt sent to the model is wrong - which is why it is a silent error.
You can only observe that something is wrong by inspecting the returned answers and that they have no relationship to the original query.

**Expected behavior**
When a string is supplied as a prompt value to the `PromptNode`, then that should not be split into characters, the PromptNode should be prepared to accept non-lists and turn them into lists before processing them as lists.

**Additional context**
This has been discussed with @vblagoje at https://discord.com/channels/993534733298450452/993539071815200889/1060723643841249320 and agreed that I will submit a PR for it.

**To Reproduce**
See the above code example.

**FAQ Check**
- [x] Have you had a look at [our new FAQ page](https://docs.haystack.deepset.ai/docs/faq)?

**System:**
 - OS: Any
 - GPU/CPU: Any
 - Haystack version (commit or version number): 1.12.2
 - DocumentStore: Any
 - Reader: PromptNode
 - Retriever: Any

The new `PromptNode` cannot work with a `Question-Answering` Pipeline with `Retrievers`
**Describe the bug**
The new `PromptNode` cannot work with a `Question-Answering` Pipeline with `Retrievers`, because the `Retriever` expects the `query` parameter as a string, while the `PromptNode` assumes that all supplied parameters are list and NOT strings. Strings will be silently handled as a list by the `PromptNode` and so split into characters by [this line](https://github.com/deepset-ai/haystack/blob/main/haystack/nodes/prompt/prompt_node.py#L658).

Currently the `PromptNode` will split that string as if it would be a list, so it will only use the first character of the `query` string to be substituted into the constructed prompt.

Mock-example:
```
prompt_node = PromptNode(prompt_model, default_prompt_template="question-answering")

pipeline = Pipeline()
pipeline.add_node(component=bm25_retriever, name="bm25_retriever", inputs=["Query"])
pipeline.add_node(component=prompt_node, name="prompt_node", inputs=["bm25_retriever"])
output = pipeline.run(query="What are subscription warrants?")  # <= if `query` is a list, then that won't work with the retriever,
                                                              # while if it is a string, then that won't with with the PromptNode
```

For the above example the `PromptNode` will construct the prompt such as:
`Given the context please answer the question. Context: ....; Question: W; Answer:'

Observe the `W` in the place of the question inside the prompt. This happens because the `query="What are subscription warrants?"` will be split as if it would be a list, so its first character will be inserted into the prompt instead of the whole question.

**Error message**
No error message, but the prompt sent to the model is wrong - which is why it is a silent error.
You can only observe that something is wrong by inspecting the returned answers and that they have no relationship to the original query.

**Expected behavior**
When a string is supplied as a prompt value to the `PromptNode`, then that should not be split into characters, the PromptNode should be prepared to accept non-lists and turn them into lists before processing them as lists.

**Additional context**
This has been discussed with @vblagoje at https://discord.com/channels/993534733298450452/993539071815200889/1060723643841249320 and agreed that I will submit a PR for it.

**To Reproduce**
See the above code example.

**FAQ Check**
- [x] Have you had a look at [our new FAQ page](https://docs.haystack.deepset.ai/docs/faq)?

**System:**
 - OS: Any
 - GPU/CPU: Any
 - Haystack version (commit or version number): 1.12.2
 - DocumentStore: Any
 - Reader: PromptNode
 - Retriever: Any



Discussion:

Resolving patch:
diff --git a/haystack/nodes/prompt/prompt_node.py b/haystack/nodes/prompt/prompt_node.py
--- a/haystack/nodes/prompt/prompt_node.py
+++ b/haystack/nodes/prompt/prompt_node.py
@@ -645,6 +645,8 @@ def prompt(self, prompt_template: Optional[Union[str, PromptTemplate]], *args, *
             template = Template(prompt_prepared["prompt_template"])
             prompt_context_copy = prompt_prepared.copy()
             prompt_context_copy.pop("prompt_template")
+            # the prompt context values should all be lists, as they will be split as one
+            prompt_context_copy = {k: v if isinstance(v, list) else [v] for k, v in prompt_context_copy.items()}
             for prompt_context_values in zip(*prompt_context_copy.values()):
                 template_input = {key: prompt_context_values[idx] for idx, key in enumerate(prompt_context_copy.keys())}
                 template_prepared: str = template.substitute(template_input)


Test patch:
diff --git a/test/nodes/test_prompt_node.py b/test/nodes/test_prompt_node.py
--- a/test/nodes/test_prompt_node.py
+++ b/test/nodes/test_prompt_node.py
@@ -268,6 +268,35 @@ def test_complex_pipeline(prompt_model):
     assert "berlin" in result["results"][0].casefold()
 
 
+@pytest.mark.parametrize("prompt_model", ["hf", "openai"], indirect=True)
+def test_complex_pipeline_with_qa(prompt_model):
+    """Test the PromptNode where the `query` is a string instead of a list what the PromptNode would expects,
+    because in a question-answering pipeline the retrievers need `query` as a string, so the PromptNode
+    need to be able to handle the `query` being a string instead of a list."""
+    if prompt_model.api_key is not None and not is_openai_api_key_set(prompt_model.api_key):
+        pytest.skip("No API key found for OpenAI, skipping test")
+
+    prompt_template = PromptTemplate(
+        name="question-answering-new",
+        prompt_text="Given the context please answer the question. Context: $documents; Question: $query; Answer:",
+        prompt_params=["documents", "query"],
+    )
+    node = PromptNode(prompt_model, default_prompt_template=prompt_template)
+
+    pipe = Pipeline()
+    pipe.add_node(component=node, name="prompt_node", inputs=["Query"])
+    result = pipe.run(
+        query="Who lives in Berlin?",  # this being a string instead of a list what is being tested
+        documents=[
+            Document("My name is Carla and I live in Berlin"),
+            Document("My name is Christelle and I live in Paris"),
+        ],
+    )
+
+    assert len(result["results"]) == 1
+    assert "carla" in result["results"][0].casefold()
+
+
 def test_complex_pipeline_with_shared_model():
     model = PromptModel()
     node = PromptNode(