Repository: deepset-ai/haystack

Issue:
Rest API doesn't connect to the created document store (?)
The title for this issue may be wrong, the issue might not be that at all..

Related to #4364 and again flagged by @rolandtannous and @recrudesce

With the same pipeline that is used in the issue #4364 another problem that was noticed:
When we initially start the pipeline the first thing we do is use the `file-upload` endpoint to write files to the document store. This works fine and we can see the embeddings being created and written in the logs. 

If we stop the rest API and re-start it with the same document store, the expected behaviour is that we simply connect to and use the embeddings and documents that were written to that document store before. However, this is not the behaviour that we observe and here are the logs that we receive:

(Below I will paste the YAML again for convenience) 

```
Calculating embeddings: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:28<00:00, 28.76s/it]
03/09/2023 11:05:56 AM Index is empty. First add some documents to search them.
03/09/2023 11:05:56 AM No documents with embeddings. Run the document store's update_embeddings() method.
[2023-03-09 11:05:56 +0100] [50290] [ERROR] Exception in ASGI application
Traceback (most recent call last):
  File "/Users/tuanacelik/Documents/git/haystack/haystack/pipelines/base.py", line 555, in run
    node_output, stream_id = self._run_node(node_id, node_input)
  File "/Users/tuanacelik/Documents/git/haystack/haystack/pipelines/base.py", line 469, in _run_node
    return self.graph.nodes[node_id]["component"]._dispatch_run(**node_input)
  File "/Users/tuanacelik/Documents/git/haystack/haystack/nodes/base.py", line 204, in _dispatch_run
    return self._dispatch_run_general(self.run, **kwargs)
  File "/Users/tuanacelik/Documents/git/haystack/haystack/nodes/base.py", line 248, in _dispatch_run_general
    output, stream = run_method(**run_inputs, **run_params)
  File "/Users/tuanacelik/Documents/git/haystack/haystack/nodes/prompt/prompt_node.py", line 663, in run
    results = self(prompt_collector=prompt_collector, **invocation_context)
  File "/Users/tuanacelik/Documents/git/haystack/haystack/nodes/prompt/prompt_node.py", line 475, in __call__
    return self.prompt(self.default_prompt_template, *args, **kwargs)
  File "/Users/tuanacelik/Documents/git/haystack/haystack/nodes/prompt/prompt_node.py", line 509, in prompt
    for prompt in template_to_fill.fill(*args, **kwargs):
  File "/Users/tuanacelik/Documents/git/haystack/haystack/nodes/prompt/prompt_node.py", line 150, in fill
    template_dict = self.prepare(*args, **kwargs)
  File "/Users/tuanacelik/Documents/git/haystack/haystack/nodes/prompt/prompt_node.py", line 129, in prepare
    raise ValueError(f"Expected prompt parameters {self.prompt_params} but got {list(available_params)}.")
ValueError: Expected prompt parameters ['documents', 'query'] but got ['top_k', 'query', 'stop_words'].
```

Example YAML:
```
version: ignore

components:    # define all the building-blocks for Pipeline
  - name: DocumentStore
    type: ElasticsearchDocumentStore
    params:
      host: localhost
      embedding_dim: 1536
      recreate_index: True
  - name: Retriever
    type: EmbeddingRetriever
    params:
      api_key: API-KEY
      document_store: DocumentStore
      embedding_model: text-embedding-ada-002
      top_k: 6
  - name: lfqa_template
    params:
      name: lfqa
      prompt_text: "Given the context please generate a comprehensive, summarized answer. Context: $documents; Question: $query; Answer:"
    type: PromptTemplate
  - name: Prompt
    type: PromptNode
    params:
      api_key: API-KEY
      model_name_or_path: text-davinci-003
      default_prompt_template: lfqa_template
      output_variable: answers
  - name: FileTypeClassifier
    type: FileTypeClassifier
  - name: TextFileConverter
    type: TextConverter
  - name: Preprocessor
    type: PreProcessor
    params:
      split_by: word
      split_length: 180
      split_overlap: 40
      split_respect_sentence_boundary: False

pipelines:
  - name: query    # a sample extractive-qa Pipeline
    nodes:
      - name: Retriever
        inputs: [Query]
      - name: Prompt
        inputs: [Retriever]
  - name: indexing
    nodes:
      - name: FileTypeClassifier
        inputs: [File]
      - name: TextFileConverter
        inputs: [FileTypeClassifier.output_1]
      - name: Preprocessor
        inputs: [TextFileConverter]
      - name: Retriever
        inputs: [Preprocessor]
      - name: DocumentStore
        inputs: [Retriever]
```



Discussion:
Hello 
maybe caused by  **recreate_index: True** 

For my i tried to create a Faissdocumentstore in a seperate Index_pipline with a specifique path, then running the query pipeline with the faiss_path.Faiss and faiss_path.json. And it works fine even with restrating the api 

@TuanaCelik I can confirm @Suhaib441 is right. If we remove **recreate_index: True**, I could re-retrieve the documents that were written previously and also perform QA with ElasticSearch Doc store.

I simplified the yaml significantly but I am taking things step by step to make sure I find the root cause correctly.

Here is the YAML. My next step is to Replace PromptNode with Reader.

```
version: ignore

components:    # define all the building-blocks for Pipeline
  - name: DocumentStore
    type: ElasticsearchDocumentStore
    params:
      embedding_dim: 1536
  - name: Retriever
    type: EmbeddingRetriever
    params:
      api_key: API_KEY
      document_store: DocumentStore
      embedding_model: text-embedding-ada-002
      top_k: 2
  - name: Reader
    params:
      # set model_name_or_path to reflect local model_path or HF model_name
      model_name_or_path: deepset/minilm-uncased-squad2
    type: FARMReader
  - name: TextFileConverter
    type: TextConverter

pipelines:
  - name: query    # a sample extractive-qa Pipeline
    nodes:
      - name: Retriever
        inputs: [Query]
      - name: Reader
        inputs: [Retriever]
  - name: indexing
    nodes:
      - name: TextFileConverter
        inputs: [File]
      - name: Retriever
        inputs: [TextFileConverter]
      - name: DocumentStore
        inputs: [Retriever]
```


Resolving patch:
diff --git a/haystack/nodes/prompt/prompt_template.py b/haystack/nodes/prompt/prompt_template.py
--- a/haystack/nodes/prompt/prompt_template.py
+++ b/haystack/nodes/prompt/prompt_template.py
@@ -268,9 +268,14 @@ def prepare(self, *args, **kwargs) -> Dict[str, Any]:
                 if param in kwargs:
                     params_dict[param] = kwargs[param]
 
-        if set(params_dict.keys()) != set(self.prompt_params):
-            available_params = set(list(params_dict.keys()) + list(set(kwargs.keys())))
-            raise ValueError(f"Expected prompt parameters {self.prompt_params} but got {list(available_params)}.")
+        if not set(self.prompt_params).issubset(params_dict.keys()):
+            available_params = {*params_dict.keys(), *kwargs.keys()}
+            provided = set(self.prompt_params).intersection(available_params)
+            message = f"only {list(provided)}" if provided else "none of these parameters"
+            raise ValueError(
+                f"Expected prompt parameters {self.prompt_params} to be provided but got "
+                f"{message}. Make sure to provide all template parameters."
+            )
 
         template_dict = {"_at_least_one_prompt": True}
         for id, call in self._prompt_params_functions.items():


Test patch:
diff --git a/test/prompt/test_prompt_template.py b/test/prompt/test_prompt_template.py
--- a/test/prompt/test_prompt_template.py
+++ b/test/prompt/test_prompt_template.py
@@ -35,6 +35,27 @@ def test_prompt_templates():
     assert p.prompt_text == "Here is some fake template with variable {baz}"
 
 
+@pytest.mark.unit
+def test_missing_prompt_template_params():
+    template = PromptTemplate("missing_params", "Here is some fake template with variable {foo} and {bar}")
+
+    # both params provided - ok
+    template.prepare(foo="foo", bar="bar")
+
+    # missing one param
+    with pytest.raises(ValueError, match=r".*parameters \['bar', 'foo'\] to be provided but got only \['foo'\].*"):
+        template.prepare(foo="foo")
+
+    # missing both params
+    with pytest.raises(
+        ValueError, match=r".*parameters \['bar', 'foo'\] to be provided but got none of these parameters.*"
+    ):
+        template.prepare(lets="go")
+
+    # more than both params provided - also ok
+    template.prepare(foo="foo", bar="bar", lets="go")
+
+
 @pytest.mark.unit
 def test_prompt_template_repr():
     p = PromptTemplate("t", "Here is variable {baz}")