Repository: deepset-ai/haystack
Pull request: Add export of Pipeline YAML config
## Imlementation
This PR adds an `export_to_yaml()` method for the Pipeline class to create a YAML configuration for a loaded `Pipeline` instance. 

Under-the-hood, the `BaseComponent` saves the init parameters used to create a Component. These parameters are put together in `export_to_yaml() `to create the complete Pipeline config.

### Limitations
With this implementation, the init parameters are captured. Hence, any updates to a Component after it has been initialized are not reflected in the YAML.

## Alternate approach
An alternative approach to generating the YAML by taking a snapshot of instance attributes of Components when calling `export_to_yaml()` was also considered. With this approach, when export is called, we iterate over all the instance attributes of Components & filters the ones that are parameters in the corresponding __init__() of the Component. However, it might not work in case when init parameters like `embedding_model: str` gets loaded as `self.embedding_model: Inferencer`. Additionally, some init parameters like `verify_certs` in ElasticsearchDocumentStore are never saved as instance attributes. 

-----
Resolves #741. 

Problem statement:
Saving Pipeline to YAML
**Question**
Hello everyone , I am looking for a way to save entire pipeline (retriever, reader) so that i can use that later . Like saving deep learning models into h5 is there anyway we can do this to the pipeline?
**Additional context**
Add any other context or screenshots about the question (optional).

Hints from issue discussion:
Hi @SasikiranJ, thank you for raising the issue. It's indeed a feature that would be helpful in many cases and we have it in the backlog with #599. I'll update here when it is implemented.

Commit message: Fix export for case when a parameter can be a dict

Patch to review:
diff --git a/haystack/document_store/elasticsearch.py b/haystack/document_store/elasticsearch.py
index 9ec567e200..29c8ce50bd 100644
--- a/haystack/document_store/elasticsearch.py
+++ b/haystack/document_store/elasticsearch.py
@@ -94,6 +94,16 @@ def __init__(
         :param return_embedding: To return document embedding
 
         """
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            host=host, port=port, username=username, password=password, api_key_id=api_key_id, api_key=api_key,
+            aws4auth=aws4auth, index=index, label_index=label_index, search_fields=search_fields, text_field=text_field,
+            name_field=name_field, embedding_field=embedding_field, embedding_dim=embedding_dim,
+            custom_mapping=custom_mapping, excluded_meta_data=excluded_meta_data, analyzer=analyzer, scheme=scheme,
+            ca_certs=ca_certs, verify_certs=verify_certs, create_index=create_index,
+            update_existing_documents=update_existing_documents, refresh_type=refresh_type, similarity=similarity,
+            timeout=timeout, return_embedding=return_embedding,
+        )
 
         self.client = self._init_elastic_client(host=host, port=port, username=username, password=password,
                                            api_key=api_key, api_key_id=api_key_id, aws4auth=aws4auth, scheme=scheme,
diff --git a/haystack/document_store/faiss.py b/haystack/document_store/faiss.py
index 26e2ef5fe1..50ddacf1a4 100644
--- a/haystack/document_store/faiss.py
+++ b/haystack/document_store/faiss.py
@@ -77,6 +77,15 @@ def __init__(
         :param progress_bar: Whether to show a tqdm progress bar or not.
                              Can be helpful to disable in production deployments to keep the logs clean.
         """
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            sql_url=sql_url, vector_dim=vector_dim, faiss_index_factory_str=faiss_index_factory_str,
+            faiss_index=faiss_index, return_embedding=return_embedding,
+            update_existing_documents=update_existing_documents, index=index, similarity=similarity,
+            embedding_field=embedding_field, progress_bar=progress_bar
+        )
+
         self.vector_dim = vector_dim
         self.faiss_index_factory_str = faiss_index_factory_str
         self.faiss_indexes: Dict[str, faiss.swigfaiss.Index] = {}
diff --git a/haystack/document_store/memory.py b/haystack/document_store/memory.py
index 3a6c417d2e..8c9651df69 100644
--- a/haystack/document_store/memory.py
+++ b/haystack/document_store/memory.py
@@ -44,6 +44,13 @@ def __init__(
         :param progress_bar: Whether to show a tqdm progress bar or not.
                              Can be helpful to disable in production deployments to keep the logs clean.
         """
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            index=index, label_index=label_index, embedding_field=embedding_field, embedding_dim=embedding_dim,
+            return_embedding=return_embedding, similarity=similarity, progress_bar=progress_bar,
+        )
+
         self.indexes: Dict[str, Dict] = defaultdict(dict)
         self.index: str = index
         self.label_index: str = label_index
diff --git a/haystack/document_store/milvus.py b/haystack/document_store/milvus.py
index 549f73693e..eb7195c141 100644
--- a/haystack/document_store/milvus.py
+++ b/haystack/document_store/milvus.py
@@ -94,6 +94,15 @@ def __init__(
         :param progress_bar: Whether to show a tqdm progress bar or not.
                              Can be helpful to disable in production deployments to keep the logs clean.
         """
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            sql_url=sql_url, milvus_url=milvus_url, connection_pool=connection_pool, index=index, vector_dim=vector_dim,
+            index_file_size=index_file_size, similarity=similarity, index_type=index_type, index_param=index_param,
+            search_param=search_param, update_existing_documents=update_existing_documents,
+            return_embedding=return_embedding, embedding_field=embedding_field, progress_bar=progress_bar,
+        )
+
         self.milvus_server = Milvus(uri=milvus_url, pool=connection_pool)
         self.vector_dim = vector_dim
         self.index_file_size = index_file_size
diff --git a/haystack/document_store/sql.py b/haystack/document_store/sql.py
index ab2fabd91e..eed667b928 100644
--- a/haystack/document_store/sql.py
+++ b/haystack/document_store/sql.py
@@ -88,6 +88,12 @@ def __init__(
                                           added already exists. Using this parameter could cause performance degradation
                                           for document insertion.
         """
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            url=url, index=index, label_index=label_index, update_existing_documents=update_existing_documents
+        )
+
         engine = create_engine(url)
         ORMBase.metadata.create_all(engine)
         Session = sessionmaker(bind=engine)
diff --git a/haystack/file_converter/base.py b/haystack/file_converter/base.py
index 09c524f0f9..9ae7e77948 100644
--- a/haystack/file_converter/base.py
+++ b/haystack/file_converter/base.py
@@ -27,6 +27,10 @@ def __init__(self, remove_numeric_tables: bool = False, valid_languages: Optiona
                                 not one of the valid languages, then it might likely be encoding error resulting
                                 in garbled text.
         """
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(remove_numeric_tables=remove_numeric_tables, valid_languages=valid_languages)
+
         self.remove_numeric_tables = remove_numeric_tables
         self.valid_languages = valid_languages
 
diff --git a/haystack/file_converter/pdf.py b/haystack/file_converter/pdf.py
index 03c9bd3fe5..5a644aebfd 100644
--- a/haystack/file_converter/pdf.py
+++ b/haystack/file_converter/pdf.py
@@ -22,6 +22,10 @@ def __init__(self, remove_numeric_tables: bool = False, valid_languages: Optiona
                                 not one of the valid languages, then it might likely be encoding error resulting
                                 in garbled text.
         """
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(remove_numeric_tables=remove_numeric_tables, valid_languages=valid_languages)
+
         verify_installation = subprocess.run(["pdftotext -v"], shell=True)
         if verify_installation.returncode == 127:
             raise Exception(
diff --git a/haystack/file_converter/tika.py b/haystack/file_converter/tika.py
index 8c1f27ae4c..1003b3f1bc 100644
--- a/haystack/file_converter/tika.py
+++ b/haystack/file_converter/tika.py
@@ -58,6 +58,12 @@ def __init__(
                                 not one of the valid languages, then it might likely be encoding error resulting
                                 in garbled text.
         """
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            tika_url=tika_url, remove_numeric_tables=remove_numeric_tables, valid_languages=valid_languages
+        )
+
         ping = requests.get(tika_url)
         if ping.status_code != 200:
             raise Exception(f"Apache Tika server is not reachable at the URL '{tika_url}'. To run it locally"
diff --git a/haystack/file_converter/txt.py b/haystack/file_converter/txt.py
index fa1160fee2..2e08c99150 100644
--- a/haystack/file_converter/txt.py
+++ b/haystack/file_converter/txt.py
@@ -8,22 +8,6 @@
 
 
 class TextConverter(BaseConverter):
-    def __init__(self, remove_numeric_tables: bool = False, valid_languages: Optional[List[str]] = None):
-        """
-        :param remove_numeric_tables: This option uses heuristics to remove numeric rows from the tables.
-                                      The tabular structures in documents might be noise for the reader model if it
-                                      does not have table parsing capability for finding answers. However, tables
-                                      may also have long strings that could possible candidate for searching answers.
-                                      The rows containing strings are thus retained in this option.
-        :param valid_languages: validate languages from a list of languages specified in the ISO 639-1
-                                (https://en.wikipedia.org/wiki/ISO_639-1) format.
-                                This option can be used to add test for encoding errors. If the extracted text is
-                                not one of the valid languages, then it might likely be encoding error resulting
-                                in garbled text.
-        """
-
-        super().__init__(remove_numeric_tables=remove_numeric_tables, valid_languages=valid_languages)
-
     def convert(
         self,
         file_path: Path,
diff --git a/haystack/generator/transformers.py b/haystack/generator/transformers.py
index d32dd7becf..428953b997 100644
--- a/haystack/generator/transformers.py
+++ b/haystack/generator/transformers.py
@@ -94,6 +94,13 @@ def __init__(
         :param use_gpu: Whether to use GPU (if available)
         """
 
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            model_name_or_path=model_name_or_path, model_version=model_version, retriever=retriever,
+            generator_type=generator_type, top_k=top_k, max_length=max_length, min_length=min_length,
+            num_beams=num_beams, embed_title=embed_title, prefix=prefix, use_gpu=use_gpu,
+        )
+
         self.model_name_or_path = model_name_or_path
         self.max_length = max_length
         self.min_length = min_length
diff --git a/haystack/graph_retriever/text_to_sparql.py b/haystack/graph_retriever/text_to_sparql.py
index 5e9c168331..2af82a847c 100644
--- a/haystack/graph_retriever/text_to_sparql.py
+++ b/haystack/graph_retriever/text_to_sparql.py
@@ -22,7 +22,10 @@ def __init__(self, knowledge_graph, model_name_or_path, top_k: int = 1):
         :param model_name_or_path: Name of or path to a pre-trained BartForConditionalGeneration model.
         :param top_k: How many SPARQL queries to generate per text query.
         """
-        
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(knowledge_graph=knowledge_graph, model_name_or_path=model_name_or_path, top_k=top_k)
+
         self.knowledge_graph = knowledge_graph
         # TODO We should extend this to any seq2seq models and use the AutoModel class
         self.model = BartForConditionalGeneration.from_pretrained(model_name_or_path, force_bos_token_to_be_generated=True)
diff --git a/haystack/knowledge_graph/graphdb.py b/haystack/knowledge_graph/graphdb.py
index 42303628bd..e3ffa50594 100644
--- a/haystack/knowledge_graph/graphdb.py
+++ b/haystack/knowledge_graph/graphdb.py
@@ -33,7 +33,12 @@ def __init__(
         :param prefixes: definitions of namespaces with a new line after each namespace, e.g., PREFIX hp: <https://deepset.ai/harry_potter/>
         
         """
-        
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            host=host, port=port, username=username, password=password, index=index, prefixes=prefixes
+        )
+
         self.url = f"http://{host}:{port}"
         self.index = index
         self.username = username
diff --git a/haystack/pipeline.py b/haystack/pipeline.py
index c1560395b9..e793d378d1 100644
--- a/haystack/pipeline.py
+++ b/haystack/pipeline.py
@@ -180,7 +180,7 @@ def load_from_yaml(cls, path: Path, pipeline_name: Optional[str] = None, overwri
         Here's a sample configuration:
 
             ```yaml
-            |   version: '0.7'
+            |   version: '0.8'
             |
             |    components:    # define all the building-blocks for Pipeline
             |    - name: MyReader       # custom-name for the component; helpful for visualization & debugging
@@ -291,6 +291,40 @@ def _overwrite_with_env_variables(cls, definition: dict):
                 param_name = key.replace(env_prefix, "").lower()
                 definition["params"][param_name] = value
 
+    def export_yaml_config(self, path: Path):
+        """
+        Save a YAML configuration for the Pipeline.
+
+        :param path: path of the output YAML file.
+        """
+        nodes = self.graph.nodes
+
+        pipeline_name = self.pipeline_type.lower()
+        pipeline_type = self.pipeline_type
+        pipelines: dict = {pipeline_name: {"name": pipeline_name, "type": pipeline_type, "nodes": []}}
+
+        components = {}
+        for node in nodes:
+            if node == self.root_node_id:
+                continue
+            component_instance = self.graph.nodes.get(node)["component"]
+            component_type = component_instance.pipeline_config["type"]
+            component_params = component_instance.pipeline_config["params"]
+            components[node] = {"name": node, "type": component_type, "params": {}}
+            for key, value in component_params.items():
+                if isinstance(value, dict) and "type" in value.keys():  # the parameter is a Component
+                    components[node]["params"][key] = value["type"]
+                    components[value["type"]] = {"name": value["type"], "type": value["type"], "params": value["params"]}
+                else:
+                    components[node]["params"][key] = value
+
+            pipelines[pipeline_name]["nodes"].append({"name": node, "inputs": list(self.graph.predecessors(node))})
+
+        config = {"components": list(components.values()), "pipelines": list(pipelines.values()), "version": "0.8"}
+
+        with open(path, 'w') as outfile:
+            yaml.dump(config, outfile, default_flow_style=False)
+
 
 class BaseStandardPipeline(ABC):
     pipeline: Pipeline
diff --git a/haystack/preprocessor/preprocessor.py b/haystack/preprocessor/preprocessor.py
index 7252f96408..bbaf62c152 100644
--- a/haystack/preprocessor/preprocessor.py
+++ b/haystack/preprocessor/preprocessor.py
@@ -44,6 +44,14 @@ def __init__(
                                                 to True, the individual split will always have complete sentences &
                                                 the number of words will be <= split_length.
         """
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            clean_whitespace=clean_whitespace, clean_header_footer=clean_header_footer,
+            clean_empty_lines=clean_empty_lines, split_by=split_by, split_length=split_length,
+            split_overlap=split_overlap, split_respect_sentence_boundary=split_respect_sentence_boundary,
+        )
+
         try:
             nltk.data.find('tokenizers/punkt')
         except LookupError:
diff --git a/haystack/reader/farm.py b/haystack/reader/farm.py
index ecb4040c15..b5b21f53c0 100644
--- a/haystack/reader/farm.py
+++ b/haystack/reader/farm.py
@@ -93,6 +93,14 @@ def __init__(
                              Can be helpful to disable in production deployments to keep the logs clean.
         """
 
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            model_name_or_path=model_name_or_path, model_version=model_version, context_window_size=context_window_size,
+            batch_size=batch_size, use_gpu=use_gpu, no_ans_boost=no_ans_boost, return_no_answer=return_no_answer,
+            top_k=top_k, top_k_per_candidate=top_k_per_candidate, top_k_per_sample=top_k_per_sample,
+            num_processes=num_processes, max_seq_len=max_seq_len, doc_stride=doc_stride, progress_bar=progress_bar,
+        )
+
         self.return_no_answers = return_no_answer
         self.top_k = top_k
         self.top_k_per_candidate = top_k_per_candidate
diff --git a/haystack/reader/transformers.py b/haystack/reader/transformers.py
index c194aec6c5..55bc675e0b 100644
--- a/haystack/reader/transformers.py
+++ b/haystack/reader/transformers.py
@@ -57,6 +57,14 @@ def __init__(
         :param doc_stride: length of striding window for splitting long texts (used if len(text) > max_seq_len)
 
         """
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            model_name_or_path=model_name_or_path, model_version=model_version, tokenizer=tokenizer,
+            context_window_size=context_window_size, use_gpu=use_gpu, top_k=top_k, doc_stride=doc_stride,
+            top_k_per_candidate=top_k_per_candidate, return_no_answers=return_no_answers, max_seq_len=max_seq_len,
+        )
+
         self.model = pipeline('question-answering', model=model_name_or_path, tokenizer=tokenizer, device=use_gpu, revision=model_version)
         self.context_window_size = context_window_size
         self.top_k = top_k
diff --git a/haystack/retriever/dense.py b/haystack/retriever/dense.py
index cef64f9d77..4855f3d7da 100644
--- a/haystack/retriever/dense.py
+++ b/haystack/retriever/dense.py
@@ -98,6 +98,16 @@ def __init__(self,
                              Can be helpful to disable in production deployments to keep the logs clean.
         """
 
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            document_store=document_store, query_embedding_model=query_embedding_model,
+            passage_embedding_model=passage_embedding_model, single_model_path=single_model_path,
+            model_version=model_version, max_seq_len_query=max_seq_len_query, max_seq_len_passage=max_seq_len_passage,
+            top_k=top_k, use_gpu=use_gpu, batch_size=batch_size, embed_title=embed_title,
+            use_fast_tokenizers=use_fast_tokenizers, infer_tokenizer_classes=infer_tokenizer_classes,
+            similarity_function=similarity_function, progress_bar=progress_bar,
+        )
+
         self.document_store = document_store
         self.batch_size = batch_size
         self.progress_bar = progress_bar
@@ -461,6 +471,14 @@ def __init__(
                                      Default: -1 (very last layer).
         :param top_k: How many documents to return per query.
         """
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            document_store=document_store, embedding_model=embedding_model, model_version=model_version,
+            use_gpu=use_gpu, model_format=model_format, pooling_strategy=pooling_strategy,
+            emb_extraction_layer=emb_extraction_layer, top_k=top_k,
+        )
+
         self.document_store = document_store
         self.model_format = model_format
         self.pooling_strategy = pooling_strategy
diff --git a/haystack/retriever/sparse.py b/haystack/retriever/sparse.py
index 5b30461b49..38b15415de 100644
--- a/haystack/retriever/sparse.py
+++ b/haystack/retriever/sparse.py
@@ -52,6 +52,10 @@ def __init__(self, document_store: ElasticsearchDocumentStore, top_k: int = 10,
                             ```
         :param top_k: How many documents to return per query.
         """
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(document_store=document_store, top_k=top_k, custom_query=custom_query)
+
         self.document_store: ElasticsearchDocumentStore = document_store
         self.top_k = top_k
         self.custom_query = custom_query
@@ -118,6 +122,10 @@ def __init__(self, document_store: BaseDocumentStore, top_k: int = 10):
         :param document_store: an instance of a DocumentStore to retrieve documents from.
         :param top_k: How many documents to return per query.
         """
+
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(document_store=document_store, top_k=top_k)
+
         self.vectorizer = TfidfVectorizer(
             lowercase=True,
             stop_words=None,
diff --git a/haystack/schema.py b/haystack/schema.py
index 6d2dcc0762..f184194aa1 100644
--- a/haystack/schema.py
+++ b/haystack/schema.py
@@ -1,3 +1,4 @@
+import inspect
 from typing import Any, Optional, Dict, List
 from uuid import uuid4
 import numpy as np
@@ -227,6 +228,7 @@ class BaseComponent:
 
     outgoing_edges: int
     subclasses: dict = {}
+    pipeline_config: dict = {}
 
     def __init_subclass__(cls, **kwargs):
         """ This automatically keeps track of all available subclasses.
@@ -258,4 +260,21 @@ def run(self, *args: Any, **kwargs: Any):
         :param kwargs:
         :return:
         """
-        pass
\ No newline at end of file
+        pass
+
+    def set_pipeline_config(self, **kwargs):
+        """
+        Save the init parameters of a component that later can be used with exporting
+        YAML configuration of a Pipeline.
+
+        :param kwargs: all parameters passed to the __init__() of the Component.
+        """
+        if not self.pipeline_config:
+            self.pipeline_config = {"params": {}, "type": type(self).__name__}
+            parameters = inspect.signature(type(self)).parameters
+            for k, v in kwargs.items():
+                if parameters[k].default != v:
+                    if isinstance(v, BaseComponent):
+                        self.pipeline_config["params"][k] = v.pipeline_config
+                    elif v is not None:
+                        self.pipeline_config["params"][k] = v
diff --git a/haystack/summarizer/transformers.py b/haystack/summarizer/transformers.py
index 671a76685d..4cd0c3f131 100644
--- a/haystack/summarizer/transformers.py
+++ b/haystack/summarizer/transformers.py
@@ -81,6 +81,14 @@ def __init__(
                                         Important: The summary will depend on the order of the supplied documents!
         """
 
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            model_name_or_path=model_name_or_path, model_version=model_version, tokenizer=tokenizer,
+            max_length=max_length, min_length=min_length, use_gpu=use_gpu,
+            clean_up_tokenization_spaces=clean_up_tokenization_spaces,
+            separator_for_single_summary=separator_for_single_summary, generate_single_summary=generate_single_summary,
+        )
+
         # TODO AutoModelForSeq2SeqLM is only necessary with transformers==4.1.1, with newer versions use the pipeline directly
         if tokenizer is None:
             tokenizer = model_name_or_path
diff --git a/haystack/translator/base.py b/haystack/translator/base.py
index d6c75efceb..f284e7605b 100644
--- a/haystack/translator/base.py
+++ b/haystack/translator/base.py
@@ -1,10 +1,10 @@
-from abc import ABC, abstractmethod
+from abc import abstractmethod
 from typing import Any, Dict, List, Mapping, Optional, Union
 
-from haystack import Document
+from haystack import Document, BaseComponent
 
 
-class BaseTranslator(ABC):
+class BaseTranslator(BaseComponent):
     """
     Abstract class for a Translator component that translates either a query or a doc from language A to language B.
     """
@@ -24,7 +24,7 @@ def translate(
         """
         pass
 
-    def run(
+    def run(  # type: ignore
         self,
         query: Optional[str] = None,
         documents: Optional[Union[List[Document], List[str], List[Dict[str, Any]]]] = None,
diff --git a/haystack/translator/transformers.py b/haystack/translator/transformers.py
index 1ba88f16b4..f0ff24a72a 100644
--- a/haystack/translator/transformers.py
+++ b/haystack/translator/transformers.py
@@ -56,6 +56,12 @@ def __init__(
         :param clean_up_tokenization_spaces: Whether or not to clean up the tokenization spaces. (default True)
         """
 
+        # save init parameters to enable export of component config as YAML
+        self.set_pipeline_config(
+            model_name_or_path=model_name_or_path, tokenizer_name=tokenizer_name, max_seq_len=max_seq_len,
+            clean_up_tokenization_spaces=clean_up_tokenization_spaces,
+        )
+
         self.max_seq_len = max_seq_len
         self.clean_up_tokenization_spaces = clean_up_tokenization_spaces
         tokenizer_name = tokenizer_name or model_name_or_path
diff --git a/test/test_pipeline.py b/test/test_pipeline.py
index 8a8d44634b..4e763ac6de 100644
--- a/test/test_pipeline.py
+++ b/test/test_pipeline.py
@@ -10,10 +10,9 @@
 
 
 @pytest.mark.parametrize("document_store_with_docs", ["elasticsearch"], indirect=True)
-def test_load_yaml(document_store_with_docs):
+def test_load_and_save_yaml(document_store_with_docs, tmp_path):
     # test correct load of indexing pipeline from yaml
-    pipeline = Pipeline.load_from_yaml(Path("samples/pipeline/test_pipeline.yaml"),
-                                       pipeline_name="indexing_pipeline")
+    pipeline = Pipeline.load_from_yaml(Path("samples/pipeline/test_pipeline.yaml"), pipeline_name="indexing_pipeline")
     pipeline.run(file_path=Path("samples/pdf/sample_pdf_1.pdf"), top_k_retriever=10, top_k_reader=3)
 
     # test correct load of query pipeline from yaml
@@ -26,6 +25,40 @@ def test_load_yaml(document_store_with_docs):
     with pytest.raises(Exception):
         Pipeline.load_from_yaml(path=Path("samples/pipeline/test_pipeline.yaml"), pipeline_name="invalid")
 
+    # test config export
+    pipeline.export_yaml_config(tmp_path/"test.yaml")
+    with open(tmp_path/"test.yaml", "r", encoding='utf-8') as stream:
+        saved_yaml = stream.read()
+    expected_yaml = '''
+        components:
+        - name: ESRetriever
+          params:
+            document_store: ElasticsearchDocumentStore
+          type: ElasticsearchRetriever
+        - name: ElasticsearchDocumentStore
+          params:
+            index: haystack_test_document
+            label_index: haystack_test_label
+          type: ElasticsearchDocumentStore
+        - name: Reader
+          params:
+            model_name_or_path: deepset/roberta-base-squad2
+            no_ans_boost: -10
+          type: FARMReader
+        pipelines:
+        - name: query
+          nodes:
+          - inputs:
+            - Query
+            name: ESRetriever
+          - inputs:
+            - ESRetriever
+            name: Reader
+          type: Query
+        version: '0.8'
+    '''
+    assert saved_yaml.replace(" ", "").replace("\n", "") == expected_yaml.replace(" ", "").replace("\n", "")
+
 
 @pytest.mark.slow
 @pytest.mark.elasticsearch


Code review comments:
diff_hunk: @@ -258,4 +260,21 @@ def run(self, *args: Any, **kwargs: Any):
         :param kwargs:
         :return:
         """
-        pass
\ No newline at end of file
+        pass
+
+    def set_pipeline_config(self, **kwargs):
path: haystack/schema.py
text: @user1:
I'd suggest to rename this to something independent of pipeline because it could also be used independently for storing a "component representation" or logging params.

Maybe something like `set_config()` or `register_params()`?

diff_hunk: @@ -291,6 +291,40 @@ def _overwrite_with_env_variables(cls, definition: dict):
                 param_name = key.replace(env_prefix, "").lower()
                 definition["params"][param_name] = value
 
+    def export_yaml_config(self, path: Path):
+        """
+        Save a YAML configuration for the Pipeline.
path: haystack/pipeline.py
text: @user1:
"Save the configuration of the current pipeline to a YAML file so that it can later be loaded again via `load_from_yaml()` "

diff_hunk: @@ -291,6 +291,40 @@ def _overwrite_with_env_variables(cls, definition: dict):
                 param_name = key.replace(env_prefix, "").lower()
                 definition["params"][param_name] = value
 
+    def export_yaml_config(self, path: Path):
path: haystack/pipeline.py
text: @user1:
Let's rename the method to `save_to_yaml` to have consistent naming with the equivalent `load_from_yaml()`.

diff_hunk: @@ -258,4 +260,21 @@ def run(self, *args: Any, **kwargs: Any):
         :param kwargs:
         :return:
         """
-        pass
\ No newline at end of file
+        pass
+
+    def set_pipeline_config(self, **kwargs):
+        """
+        Save the init parameters of a component that later can be used with exporting
+        YAML configuration of a Pipeline.
+
+        :param kwargs: all parameters passed to the __init__() of the Component.
+        """
+        if not self.pipeline_config:
+            self.pipeline_config = {"params": {}, "type": type(self).__name__}
+            parameters = inspect.signature(type(self)).parameters
+            for k, v in kwargs.items():
+                if parameters[k].default != v:
path: haystack/schema.py
text: @user1:
Please add a comment that this ensures to only store non-default values. 

I am also not sure if there won't be cases where we need even those defaults. It's not the case for the YAML export, but I can imagine cases where we just want to access all current params (e.g. experiment tracking) and don't want to manually determine the defaults from `inspect.signature()`. Would it be an option to store them all here and add a `get_pipeline_config(return_defaults=True)` that does the optional filtering of defaults?

@author:
> Would it be an option to store them all here and add a get_pipeline_config(return_defaults=True) that does the optional filtering of defaults?

Good idea! I added `return_defaults` flag in `save_to_yaml()`.