Repository: deepset-ai/haystack

Issue:
Add a `verbose` option to PromptNode to let users understand the prompts being used
**Is your feature request related to a problem? Please describe.**
As a developer working with LLMs via the `PromptNode`, I want to have more transparency over the exact prompt being sent to the LLM. This helps my work in several scenarios:
- helps me to debug my pipeline in case the generated predictions are bad (e.g. because wrong variables got ingested or prompt is too long)
- helps me to learn more about prompts and what's behind existing prompt templates
- helps me to understand the flow of more complex pipelines where multiple prompts are generated at different steps

**Describe the solution you'd like**
An optional parameter that I can set to make the logs of the node more verbose.
Could be a log level, could be a `debug` param like we have in [Pipelines.run()](https://docs.haystack.deepset.ai/reference/pipelines-api#pipelinerun) or a `verbose` param.
Ideally, it should be configurable at init time and for each run of the node. 

Add a `verbose` option to PromptNode to let users understand the prompts being used
**Is your feature request related to a problem? Please describe.**
As a developer working with LLMs via the `PromptNode`, I want to have more transparency over the exact prompt being sent to the LLM. This helps my work in several scenarios:
- helps me to debug my pipeline in case the generated predictions are bad (e.g. because wrong variables got ingested or prompt is too long)
- helps me to learn more about prompts and what's behind existing prompt templates
- helps me to understand the flow of more complex pipelines where multiple prompts are generated at different steps

**Describe the solution you'd like**
An optional parameter that I can set to make the logs of the node more verbose.
Could be a log level, could be a `debug` param like we have in [Pipelines.run()](https://docs.haystack.deepset.ai/reference/pipelines-api#pipelinerun) or a `verbose` param.
Ideally, it should be configurable at init time and for each run of the node. 



Discussion:
I agree.
I was working on [this PR](https://github.com/deepset-ai/haystack/pull/3820) related to a bug to how the prompt is constructed in `PromptNode` and I had to use a local version of haystack to hack a print statement into it, so I can see what is the actual prompt sent to the model.
It would be nice to have an option to get back the prompt as part of the results object to help the troubleshooting.
@tholor @zoltan-fedor I opened a PR more as a proposal and consultation before going further. We will likely have a need from either dC or the community to act upon events in PromptNode. Therefore, I made this suggestion, and it is just a suggestion. LMK
I agree.
I was working on [this PR](https://github.com/deepset-ai/haystack/pull/3820) related to a bug to how the prompt is constructed in `PromptNode` and I had to use a local version of haystack to hack a print statement into it, so I can see what is the actual prompt sent to the model.
It would be nice to have an option to get back the prompt as part of the results object to help the troubleshooting.
@tholor @zoltan-fedor I opened a PR more as a proposal and consultation before going further. We will likely have a need from either dC or the community to act upon events in PromptNode. Therefore, I made this suggestion, and it is just a suggestion. LMK

Resolving patch:
diff --git a/haystack/nodes/prompt/prompt_node.py b/haystack/nodes/prompt/prompt_node.py
--- a/haystack/nodes/prompt/prompt_node.py
+++ b/haystack/nodes/prompt/prompt_node.py
@@ -765,6 +765,8 @@ def prompt(self, prompt_template: Optional[Union[str, PromptTemplate]], *args, *
         :return: A list of strings as model responses.
         """
         results = []
+        # we pop the prompt_collector kwarg to avoid passing it to the model
+        prompt_collector: List[str] = kwargs.pop("prompt_collector", [])
         if isinstance(prompt_template, str) and not self.is_supported_template(prompt_template):
             raise ValueError(
                 f"{prompt_template} not supported, please select one of: {self.get_prompt_template_names()} "
@@ -786,12 +788,16 @@ def prompt(self, prompt_template: Optional[Union[str, PromptTemplate]], *args, *
             for prompt in template_to_fill.fill(*args, **kwargs):
                 kwargs_copy = copy.copy(kwargs)
                 # and pass the prepared prompt and kwargs copy to the model
+                prompt_collector.append(prompt)
+                logger.debug("Prompt being sent to LLM with prompt %s and kwargs %s", prompt, kwargs_copy)
                 output = self.prompt_model.invoke(prompt, **kwargs_copy)
                 results.extend(output)
         else:
             # straightforward prompt, no templates used
             for prompt in list(args):
                 kwargs_copy = copy.copy(kwargs)
+                prompt_collector.append(prompt)
+                logger.debug("Prompt being sent to LLM with prompt %s and kwargs %s ", prompt, kwargs_copy)
                 output = self.prompt_model.invoke(prompt, **kwargs_copy)
                 results.extend(output)
         return results
@@ -905,10 +911,15 @@ def run(
         # to pass results from a pipeline node to any other downstream pipeline node.
         invocation_context = invocation_context or {}
 
+        # prompt_collector is an empty list, it's passed to the PromptNode that will fill it with the rendered prompts,
+        # so that they can be returned by `run()` as part of the pipeline's debug output.
+        prompt_collector: List[str] = []
+
         results = self(
             query=query,
             labels=labels,
             documents=[doc.content for doc in documents if isinstance(doc.content, str)] if documents else [],
+            prompt_collector=prompt_collector,
             **invocation_context,
         )
 
@@ -919,6 +930,7 @@ def run(
 
         final_result["results"] = results
         final_result["invocation_context"] = invocation_context
+        final_result["_debug"] = {"prompts_used": prompt_collector}
         return final_result, "output_1"
 
     def run_batch(


Test patch:
diff --git a/test/nodes/test_prompt_node.py b/test/nodes/test_prompt_node.py
--- a/test/nodes/test_prompt_node.py
+++ b/test/nodes/test_prompt_node.py
@@ -340,11 +340,19 @@ def test_complex_pipeline_with_qa(prompt_model):
             Document("My name is Carla and I live in Berlin"),
             Document("My name is Christelle and I live in Paris"),
         ],
+        debug=True,  # so we can verify that the constructed prompt is returned in debug
     )
 
     assert len(result["results"]) == 1
     assert "carla" in result["results"][0].casefold()
 
+    # also verify that the PromptNode has included its constructed prompt LLM model input in the returned debug
+    assert (
+        result["_debug"]["prompt_node"]["runtime"]["prompts_used"][0]
+        == "Given the context please answer the question. Context: My name is Carla and I live in Berlin; "
+        "Question: Who lives in Berlin?; Answer:"
+    )
+
 
 def test_complex_pipeline_with_shared_model():
     model = PromptModel()