Repository: deepset-ai/haystack

Issue:
[2.0.0b8] Generator is being called twice instead of once after loading the pipeline yaml file
**Describe the bug**
Now there is a bug after loading the pipeline yaml file.

**Expected behavior**
Generator is being called twice instead of once.

**Additional context**
If you remove pipeline serialization and loading part, the behavior is normal. Thus I think the problem is due to the pipeline serialization issue.(either saving or loading part)

The example code below is referenced from [the official documentation](https://docs.haystack.deepset.ai/v2.0/docs/openaigenerator#in-a-pipeline)

**To Reproduce**

```python
import backoff
import openai

from typing import Any, Dict, List, Optional

from dotenv import load_dotenv
from haystack import component, Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.builders.prompt_builder import PromptBuilder
from haystack.components.generators import OpenAIGenerator
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.utils.auth import Secret


load_dotenv(override=True)


@component
class CustomOpenAIGenerator(OpenAIGenerator):
    @component.output_types(replies=List[str], meta=List[Dict[str, Any]])
    @backoff.on_exception(backoff.expo, openai.RateLimitError, max_time=60)
    def run(
        self,
        prompt: str,
        generation_kwargs: Optional[Dict[str, Any]] = None
    ):
        print('test')
        return super(CustomOpenAIGenerator, self).run(
            prompt=prompt,
            generation_kwargs=generation_kwargs
        )

docstore = InMemoryDocumentStore()
docstore.write_documents([Document(content="Rome is the capital of Italy"), Document(content="Paris is the capital of France")])

query = "What is the capital of France?"

template = """
Given the following information, answer the question.

Context: 
{% for document in documents %}
    {{ document.content }}
{% endfor %}

{{ embedding|length }}

Question: {{ query }}?
"""
pipe = Pipeline()

pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component("llm", CustomOpenAIGenerator(api_key=Secret.from_env_var('OPENAI_API_KEY')))
pipe.connect("retriever", "prompt_builder.documents")
pipe.connect("prompt_builder", "llm")

with open('test.yaml', "w") as file:
    pipe.dump(file)

with open('test.yaml', "r") as file:
    pipe = Pipeline.load(file)

res=pipe.run({
    "prompt_builder": {
        "query": query
    },
    "retriever": {
        "query": query
    }
})

print(res)
```

**FAQ Check**
- [x] Have you had a look at [our new FAQ page](https://docs.haystack.deepset.ai/docs/faq)?

**System:**
 - OS: MacOS
 - GPU/CPU: CPU
 - Haystack version (commit or version number): 2.0.0b8
 - Generator OpenAIGenerator


Discussion:
Some initial thoughts and findings on this issue.

This issue is also happening with the base `OpenAIGenerator`, to make this visible, add the following lines to the `run()` method, i.e.:

```python
print('Running OpenAIGenerator()...')
print('Prompt:', prompt)
```

and then run the code

```python
from haystack import Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.builders.prompt_builder import PromptBuilder
from haystack.components.generators import OpenAIGenerator
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack import Document
from haystack.utils import Secret

docstore = InMemoryDocumentStore()
docstore.write_documents(
    [Document(content="Rome is the capital of Italy"),
     Document(content="Paris is the capital of France")]
)

query = "What is the capital of France?"

template = """
Given the following information, answer the question.

Context: 
{% for document in documents %}
    {{ document.content }}
{% endfor %}

Question: {{ query }}?
"""
pipe = Pipeline()

pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component("llm", OpenAIGenerator(api_key=Secret.from_env_var('OPENAI_API_KEY')))
pipe.connect("retriever", "prompt_builder.documents")
pipe.connect("prompt_builder", "llm")

print("Running pipeline...")
res = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})
print(res)

with open('test.yaml', "w") as file:
    pipe.dump(file)

with open('test.yaml', "r") as file:
    pipe_from_file = Pipeline.load(file)

print("Running pipeline from file...")
res2 = pipe_from_file.run({"retriever": {"query": query}, "prompt_builder": {"query": query}})
print(res2)
```

This should output the following:

```bash
Running pipeline...
Running OpenAIGenerator()...
Prompt: 
Given the following information, answer the question.

Context: 

    Paris is the capital of France

    Rome is the capital of Italy


Question: What is the capital of France??
Ranking by BM25...: 100%|██████████| 2/2 [00:00<00:00, 27413.75 docs/s]
{'llm': {'replies': ['The capital of France is Paris.'], 'meta': [{'model': 'gpt-3.5-turbo-0125', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 7, 'prompt_tokens': 45, 'total_tokens': 52}}]}}
```

Running the pipeline loaded from file

```bash
Running pipeline from file...
Running OpenAIGenerator()...
Prompt: 
Given the following information, answer the question.

Context: 


Question: What is the capital of France??
Ranking by BM25...: 0 docs [00:00, ? docs/s]
Running OpenAIGenerator()...
Prompt: 
Given the following information, answer the question.

Context: 


Question: What is the capital of France??
{'llm': {'replies': ['The capital of France is Paris.'], 'meta': [{'model': 'gpt-3.5-turbo-0125', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 7, 'prompt_tokens': 29, 'total_tokens': 36}}]}}
```


Loading the saved pipeline and running it results in the `OpenAIGenerator` being run twice, always with an empty context but still with a valid answer. We need to:
- change the prompt to account for an empty context, otherwise  `Paris` is given as an answer because the OpenAI LLM knows it and not because it's read from the retrieved documents
 
```python
template = """
Given the following information, answer the question.
If Context: is empty, the question is asked without context, and return "No Context given"

Context: 
{% for document in documents %}
    {{ document.content }}
{% endfor %}

Question: {{ query }}?
"""
```

- add documents to the DocumentStore from the saved pipeline
```python
with open('test.yaml', "r") as file:
    pipe_from_file = Pipeline.load(file)
docs = [Document(content="Rome is the capital of Italy"), Document(content="Paris is the capital of France")]
pipe_from_file.get_component("retriever").document_store.write_documents(docs)
```

Running the code with the new prompt/template you should now see:

```bash
{'llm': {'replies': ['No Context given'], 'meta': [{'model': 'gpt-3.5-turbo-0125', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 3, 'prompt_tokens': 49, 'total_tokens': 52}}]}}
```
The code above still  doesn't fix the issue since the pipeline loaded from the disk runs twice:

- first, with an empty context prompt
- second with a context prompt filled with docs content

What I noticed is that pipeline created in the code has the following components:

```
🚅 Components
  - retriever: InMemoryBM25Retriever
  - prompt_builder: PromptBuilder
  - llm: OpenAIGenerator
🛤️ Connections
  - retriever.documents -> prompt_builder.documents (List[Document])
  - prompt_builder.prompt -> llm.prompt (str)
```

and the pipeline loaded from disk has the following:

```
🚅 Components
  - llm: OpenAIGenerator
  - prompt_builder: PromptBuilder
  - retriever: InMemoryBM25Retriever
🛤️ Connections
  - prompt_builder.prompt -> llm.prompt (str)
  - retriever.documents -> prompt_builder.documents (List[Document])
  ```
  
 This order in the components makes a difference, and it seems it shouldn't. Looking at `yaml` file created by the `dump()`:
 
 ```yaml
 components:
  llm:
    init_parameters:
      api_base_url: null
      api_key:
        env_vars:
        - OPENAI_API_KEY
        strict: true
        type: env_var
      generation_kwargs: {}
      model: gpt-3.5-turbo
      streaming_callback: null
      system_prompt: null
    type: haystack.components.generators.openai.OpenAIGenerator
  prompt_builder:
    init_parameters:
      template: "\nGiven the following information, answer the question.\n\nContext:\
        \ \n{% for document in documents %}\n    {{ document.content }}\n{% endfor\
        \ %}\n\nQuestion: {{ query }}?\n"
    type: haystack.components.builders.prompt_builder.PromptBuilder
  retriever:
    init_parameters:
      document_store:
        init_parameters:
          bm25_algorithm: BM25L
          bm25_parameters: {}
          bm25_tokenization_regex: (?u)\b\w\w+\b
          embedding_similarity_function: dot_product
        type: haystack.document_stores.in_memory.document_store.InMemoryDocumentStore
      filters: null
      scale_score: false
      top_k: 10
    type: haystack.components.retrievers.in_memory.bm25_retriever.InMemoryBM25Retriever
connections:
- receiver: prompt_builder.documents
  sender: retriever.documents
- receiver: llm.prompt
  sender: prompt_builder.prompt
max_loops_allowed: 100
metadata: {}
```

If we directly edit this file and change the order to be:

```yaml
components:
  prompt_builder:
    init_parameters:
      template: "\nGiven the following information, answer the question.\n\nContext:\
        \ \n{% for document in documents %}\n    {{ document.content }}\n{% endfor\
        \ %}\n\nQuestion: {{ query }}?\n"
    type: haystack.components.builders.prompt_builder.PromptBuilder
  retriever:
    init_parameters:
      document_store:
        init_parameters:
          bm25_algorithm: BM25L
          bm25_parameters: { }
          bm25_tokenization_regex: (?u)\b\w\w+\b
          embedding_similarity_function: dot_product
        type: haystack.document_stores.in_memory.document_store.InMemoryDocumentStore
      filters: null
      scale_score: false
      top_k: 10
    type: haystack.components.retrievers.in_memory.bm25_retriever.InMemoryBM25Retriever
  llm:
    init_parameters:
      api_base_url: null
      api_key:
        env_vars:
        - OPENAI_API_KEY
        strict: true
        type: env_var
      generation_kwargs: {}
      model: gpt-3.5-turbo
      streaming_callback: null
      system_prompt: null
    type: haystack.components.generators.openai.OpenAIGenerator
connections:
- receiver: prompt_builder.documents
  sender: retriever.documents
- receiver: llm.prompt
  sender: prompt_builder.prompt
max_loops_allowed: 100
metadata: {}
```


The pipeline will run the `OpenAIGenerator` only once

So, the issue is related to the default values and the inputs of the `Retriever`.

This can be solved by explicitly naming all the input values of the retriever, i.e.:

```python
pipe_from_file.run(
    {"prompt_builder": {"query": query}, 
      "retriever": {
          'filters': None, 
          'query': 'What is the capital of France?', 
          'scale_score': None, 
          'top_k': None}
      }
  )
```

instead of just:

```python
pipe_from_file.run(
    {"prompt_builder": {"query": query}, 
      "retriever": {'query': 'What is the capital of France?'}
      }
  )
```


This is related to the `run()` method in the `Pipeline()`. The retriever defaults are added only when it’s removed from `waiting_for_input`. If we explicitly pass everything in the `run()` instead the `Retriever` goes straight in `to_run` and runs before the `prompt_builder`.

One solution would be to check for components with default values and explicitly filling them at the beginning of the run() method.

FYI: @silvanocerza @masci 


Resolving patch:
diff --git a/haystack/core/pipeline/pipeline.py b/haystack/core/pipeline/pipeline.py
--- a/haystack/core/pipeline/pipeline.py
+++ b/haystack/core/pipeline/pipeline.py
@@ -948,6 +948,26 @@ def run(self, word: str):
                             if not there_are_only_lazy_variadics:
                                 continue
 
+                        # Components that have defaults for all their inputs must be treated the same identical way as we treat
+                        # lazy variadic components. If there are only components with defaults we can run them.
+                        # If we don't do this the order of execution of the Pipeline's Components will be affected cause we
+                        # enqueue the Components in `to_run` at the start using the order they are added in the Pipeline.
+                        # If a Component A with defaults is added before a Component B that has no defaults, but in the Pipeline
+                        # logic A must be executed after B it could run instead before if we don't do this check.
+                        has_only_defaults = all(
+                            not socket.is_mandatory for socket in comp.__haystack_input__._sockets_dict.values()  # type: ignore
+                        )
+                        if has_only_defaults:
+                            there_are_only_components_with_defaults = True
+                            for other_name, other_comp in waiting_for_input:
+                                if name == other_name:
+                                    continue
+                                there_are_only_components_with_defaults &= all(
+                                    not s.is_mandatory for s in other_comp.__haystack_input__._sockets_dict.values()  # type: ignore
+                                )
+                            if not there_are_only_components_with_defaults:
+                                continue
+
                         # Find the first component that has all the inputs it needs to run
                         has_enough_inputs = True
                         for input_socket in comp.__haystack_input__._sockets_dict.values():  # type: ignore


Test patch:
diff --git a/test/core/pipeline/test_pipeline.py b/test/core/pipeline/test_pipeline.py
--- a/test/core/pipeline/test_pipeline.py
+++ b/test/core/pipeline/test_pipeline.py
@@ -756,3 +756,54 @@ def run(self, word: str, intermediate: Optional[str] = None):
     pipeline.add_component("hello", hello)
     pipeline.connect("hello.intermediate", "hello.intermediate")
     assert [("hello", hello)] == list(pipeline.walk())
+
+
+def test_correct_execution_order_of_components_with_only_defaults(spying_tracer):
+    """
+    We enqueue the Components in internal `to_run` data structure at the start of `Pipeline.run()` using the order
+    they are added in the Pipeline with `Pipeline.add_component()`.
+    If a Component A with defaults is added before a Component B that has no defaults, but in the Pipeline
+    logic A must be executed after B it could run instead before.
+
+    This test verifies that the order of execution is correct.
+    """
+    docs = [Document(content="Rome is the capital of Italy"), Document(content="Paris is the capital of France")]
+    doc_store = InMemoryDocumentStore()
+    doc_store.write_documents(docs)
+    template = (
+        "Given the following information, answer the question.\n"
+        "Context:\n"
+        "{% for document in documents %}"
+        "    {{ document.content }}\n"
+        "{% endfor %}"
+        "Question: {{ query }}"
+    )
+
+    pipe = Pipeline()
+
+    # The order of this addition is important for the test
+    # Do not edit them.
+    pipe.add_component("prompt_builder", PromptBuilder(template=template))
+    pipe.add_component("retriever", InMemoryBM25Retriever(document_store=doc_store))
+    pipe.connect("retriever", "prompt_builder.documents")
+
+    query = "What is the capital of France?"
+    res = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})
+
+    assert len(spying_tracer.spans) == 3
+    assert spying_tracer.spans[0].operation_name == "haystack.pipeline.run"
+    assert spying_tracer.spans[1].operation_name == "haystack.component.run"
+    assert spying_tracer.spans[1].tags["haystack.component.name"] == "retriever"
+    assert spying_tracer.spans[2].operation_name == "haystack.component.run"
+    assert spying_tracer.spans[2].tags["haystack.component.name"] == "prompt_builder"
+
+    print(res["prompt_builder"]["prompt"])
+    assert res == {
+        "prompt_builder": {
+            "prompt": "Given the following information, answer the question.\n"
+            "Context:\n"
+            "    Paris is the capital of France\n"
+            "    Rome is the capital of Italy\n"
+            "Question: What is the capital of France?"
+        }
+    }