Repository: deepset-ai/haystack
Pull request: Update Documentation
Let's update our documentation so that the website better reflects the latest developments in Haystack.

The main changes are:
- Describing Haystack as Search instead of QA
- Replacing Finder with Pipelines
- Summarizer
- Translator
- Crawler
- Evaluation Nodes
- Pipeline Yaml config example
- Adding import statements to code examples

Fixes #793

Problem statement:
Haystack Documentation Backlog
Let's update the usage and API documentation for the following new components:

Summarizer
Translator
Squad to DPR script (Tutorial 9)
Knowledge Graph
Confidence
Evaluation

Website API and Usage for KG
Eval Nodes
Pipeline Config
Confidence Scores
Web Crawler

Add Import statements
Create snippets page

Hints from issue discussion:
I love the usage example snippets and would love to see more there. Its extremely helpful to code things fast.

Could we also add import statements to those usage examples?
Adding imports is a good idea for sure. I think I like having code snippets inside the Usage pages, but I can also imagine it would be useful to have all of them on one page so you can quickly get to them.

Commit message: Update tutorial link

Patch to review:
diff --git a/docs/_src/api/api/crawler.md b/docs/_src/api/api/crawler.md
new file mode 100644
index 0000000000..2f852cdde7
--- /dev/null
+++ b/docs/_src/api/api/crawler.md
@@ -0,0 +1,95 @@
+<a name="crawler"></a>
+# Module crawler
+
+<a name="crawler.Crawler"></a>
+## Crawler Objects
+
+```python
+class Crawler(BaseComponent)
+```
+
+Crawl texts from a website so that we can use them later in Haystack as a corpus for search / question answering etc.
+
+**Example:**
+```python
+|    from haystack.connector import Crawler
+|
+|    crawler = Crawler()
+|    # crawl Haystack docs, i.e. all pages that include haystack.deepset.ai/docs/
+|    docs = crawler.crawl(urls=["https://haystack.deepset.ai/docs/latest/get_startedmd"],
+|                         output_dir="crawled_files",
+|                         filter_urls= ["haystack\.deepset\.ai\/docs\/"])
+```
+
+<a name="crawler.Crawler.__init__"></a>
+#### \_\_init\_\_
+
+```python
+ | __init__(output_dir: str, urls: Optional[List[str]] = None, crawler_depth: int = 1, filter_urls: Optional[List] = None, overwrite_existing_files=True)
+```
+
+Init object with basic params for crawling (can be overwritten later).
+
+**Arguments**:
+
+- `output_dir`: Path for the directory to store files
+- `urls`: List of http(s) address(es) (can also be supplied later when calling crawl())
+- `crawler_depth`: How many sublinks to follow from the initial list of URLs. Current options:
+                      0: Only initial list of urls
+                      1: Follow links found on the initial URLs (but no further)
+- `filter_urls`: Optional list of regular expressions that the crawled URLs must comply with.
+                   All URLs not matching at least one of the regular expressions will be dropped.
+- `overwrite_existing_files`: Whether to overwrite existing files in output_dir with new content
+
+<a name="crawler.Crawler.crawl"></a>
+#### crawl
+
+```python
+ | crawl(output_dir: Union[str, Path, None] = None, urls: Optional[List[str]] = None, crawler_depth: Optional[int] = None, filter_urls: Optional[List] = None, overwrite_existing_files: Optional[bool] = None) -> List[Path]
+```
+
+Craw URL(s), extract the text from the HTML, create a Haystack Document object out of it and save it (one JSON
+file per URL, including text and basic meta data).
+You can optionally specify via `filter_urls` to only crawl URLs that match a certain pattern.
+All parameters are optional here and only meant to overwrite instance attributes at runtime.
+If no parameters are provided to this method, the instance attributes that were passed during __init__ will be used.
+
+**Arguments**:
+
+- `output_dir`: Path for the directory to store files
+- `urls`: List of http addresses or single http address
+- `crawler_depth`: How many sublinks to follow from the initial list of URLs. Current options:
+                      0: Only initial list of urls
+                      1: Follow links found on the initial URLs (but no further)
+- `filter_urls`: Optional list of regular expressions that the crawled URLs must comply with.
+                   All URLs not matching at least one of the regular expressions will be dropped.
+- `overwrite_existing_files`: Whether to overwrite existing files in output_dir with new content
+
+**Returns**:
+
+List of paths where the crawled webpages got stored
+
+<a name="crawler.Crawler.run"></a>
+#### run
+
+```python
+ | run(output_dir: Union[str, Path, None] = None, urls: Optional[List[str]] = None, crawler_depth: Optional[int] = None, filter_urls: Optional[List] = None, overwrite_existing_files: Optional[bool] = None, **kwargs) -> Tuple[Dict, str]
+```
+
+Method to be executed when the Crawler is used as a Node within a Haystack pipeline.
+
+**Arguments**:
+
+- `output_dir`: Path for the directory to store files
+- `urls`: List of http addresses or single http address
+- `crawler_depth`: How many sublinks to follow from the initial list of URLs. Current options:
+                      0: Only initial list of urls
+                      1: Follow links found on the initial URLs (but no further)
+- `filter_urls`: Optional list of regular expressions that the crawled URLs must comply with.
+                   All URLs not matching at least one of the regular expressions will be dropped.
+- `overwrite_existing_files`: Whether to overwrite existing files in output_dir with new content
+
+**Returns**:
+
+Tuple({"paths": List of filepaths, ...}, Name of output edge)
+
diff --git a/docs/_src/api/api/document_store.md b/docs/_src/api/api/document_store.md
index 6c9d1d7571..8b6a128323 100644
--- a/docs/_src/api/api/document_store.md
+++ b/docs/_src/api/api/document_store.md
@@ -657,7 +657,7 @@ Fetch documents by specifying a list of text vector id strings
 
 - `vector_ids`: List of vector_id strings.
 - `index`: Name of the index to get the documents from. If None, the
-              DocumentStore's default index (self.index) will be used.
+DocumentStore's default index (self.index) will be used.
 - `batch_size`: When working with large number of documents, batching can help reduce memory footprint.
 
 <a name="sql.SQLDocumentStore.get_all_documents_generator"></a>
@@ -829,7 +829,6 @@ the vector embeddings are indexed in a FAISS Index.
                                 - "HNSW": Graph-based heuristic. If not further specified,
                                           we use the following config:
                                           HNSW64, efConstruction=80 and efSearch=20
-
                                 - "IVFx,Flat": Inverted Index. Replace x with the number of centroids aka nlist.
                                                   Rule of thumb: nlist = 10 * sqrt (num_docs) is a good starting point.
                                 For more details see:
diff --git a/docs/_src/api/api/evaluation.md b/docs/_src/api/api/evaluation.md
new file mode 100644
index 0000000000..2ce2dfbc25
--- /dev/null
+++ b/docs/_src/api/api/evaluation.md
@@ -0,0 +1,92 @@
+<a name="eval"></a>
+# Module eval
+
+<a name="eval.EvalRetriever"></a>
+## EvalRetriever Objects
+
+```python
+class EvalRetriever()
+```
+
+This is a pipeline node that should be placed after a Retriever in order to assess its performance. Performance
+metrics are stored in this class and updated as each sample passes through it. To view the results of the evaluation,
+call EvalRetriever.print()
+
+<a name="eval.EvalRetriever.__init__"></a>
+#### \_\_init\_\_
+
+```python
+ | __init__(debug=False, open_domain=True)
+```
+
+**Arguments**:
+
+- `open_domain`: When True, a document is considered correctly retrieved so long as the answer string can be found within it.
+                    When False, correct retrieval is evaluated based on document_id.
+:type open_domain: bool
+- `debug`: When True, a record of each sample and its evaluation will be stored in EvalRetriever.log
+:type debug: bool
+
+<a name="eval.EvalRetriever.run"></a>
+#### run
+
+```python
+ | run(documents, labels: dict, **kwargs)
+```
+
+Run this node on one sample and its labels
+
+<a name="eval.EvalRetriever.print"></a>
+#### print
+
+```python
+ | print()
+```
+
+Print the evaluation results
+
+<a name="eval.EvalReader"></a>
+## EvalReader Objects
+
+```python
+class EvalReader()
+```
+
+This is a pipeline node that should be placed after a Reader in order to assess the performance of the Reader
+individually or to assess the extractive QA performance of the whole pipeline. Performance metrics are stored in
+this class and updated as each sample passes through it. To view the results of the evaluation, call EvalReader.print()
+
+<a name="eval.EvalReader.__init__"></a>
+#### \_\_init\_\_
+
+```python
+ | __init__(skip_incorrect_retrieval=True, open_domain=True, debug=False)
+```
+
+**Arguments**:
+
+- `skip_incorrect_retrieval`: When set to True, this eval will ignore the cases where the retriever returned no correct documents
+:type skip_incorrect_retrieval: bool
+- `open_domain`: When True, extracted answers are evaluated purely on string similarity rather than the position of the extracted answer
+:type open_domain: bool
+- `debug`: When True, a record of each sample and its evaluation will be stored in EvalReader.log
+:type debug: bool
+
+<a name="eval.EvalReader.run"></a>
+#### run
+
+```python
+ | run(labels, answers, **kwargs)
+```
+
+Run this node on one sample and its labels
+
+<a name="eval.EvalReader.print"></a>
+#### print
+
+```python
+ | print(mode)
+```
+
+Print the evaluation results
+
diff --git a/docs/_src/api/api/generate_docstrings.sh b/docs/_src/api/api/generate_docstrings.sh
index 66f3e6c3db..1b9837d201 100755
--- a/docs/_src/api/api/generate_docstrings.sh
+++ b/docs/_src/api/api/generate_docstrings.sh
@@ -5,7 +5,14 @@
 pydoc-markdown pydoc-markdown-document-store.yml
 pydoc-markdown pydoc-markdown-file-converters.yml
 pydoc-markdown pydoc-markdown-preprocessor.yml
+pydoc-markdown pydoc-markdown-crawler.yml
 pydoc-markdown pydoc-markdown-reader.yml
 pydoc-markdown pydoc-markdown-generator.yml
 pydoc-markdown pydoc-markdown-retriever.yml
-pydoc-markdown pydoc-markdown-pipelines.yml
\ No newline at end of file
+pydoc-markdown pydoc-markdown-summarizer.yml
+pydoc-markdown pydoc-markdown-translator.yml
+pydoc-markdown pydoc-markdown-pipelines.yml
+pydoc-markdown pydoc-markdown-knowledge-graph.yml
+pydoc-markdown pydoc-markdown-graph-retriever.yml
+pydoc-markdown pydoc-markdown-evaluation.yml
+
diff --git a/docs/_src/api/api/graph_retriever.md b/docs/_src/api/api/graph_retriever.md
new file mode 100644
index 0000000000..ff68c8cf33
--- /dev/null
+++ b/docs/_src/api/api/graph_retriever.md
@@ -0,0 +1,25 @@
+<a name="base"></a>
+# Module base
+
+<a name="text_to_sparql"></a>
+# Module text\_to\_sparql
+
+<a name="text_to_sparql.Text2SparqlRetriever"></a>
+## Text2SparqlRetriever Objects
+
+```python
+class Text2SparqlRetriever(BaseGraphRetriever)
+```
+
+Graph retriever that uses a pre-trained Bart model to translate natural language questions given in text form to queries in SPARQL format.
+The generated SPARQL query is executed on a knowledge graph.
+
+<a name="text_to_sparql.Text2SparqlRetriever.format_result"></a>
+#### format\_result
+
+```python
+ | format_result(result)
+```
+
+Generate formatted dictionary output with text answer and additional info
+
diff --git a/docs/_src/api/api/knowledge_graph.md b/docs/_src/api/api/knowledge_graph.md
new file mode 100644
index 0000000000..467ae4f360
--- /dev/null
+++ b/docs/_src/api/api/knowledge_graph.md
@@ -0,0 +1,15 @@
+<a name="base"></a>
+# Module base
+
+<a name="graphdb"></a>
+# Module graphdb
+
+<a name="graphdb.GraphDBKnowledgeGraph"></a>
+## GraphDBKnowledgeGraph Objects
+
+```python
+class GraphDBKnowledgeGraph(BaseKnowledgeGraph)
+```
+
+Knowledge graph store that runs on a GraphDB instance
+
diff --git a/docs/_src/api/api/pydoc-markdown-crawler.yml b/docs/_src/api/api/pydoc-markdown-crawler.yml
new file mode 100644
index 0000000000..f78330c42a
--- /dev/null
+++ b/docs/_src/api/api/pydoc-markdown-crawler.yml
@@ -0,0 +1,18 @@
+loaders:
+  - type: python
+    search_path: [../../../../haystack/connector]
+    modules: ['crawler']
+    ignore_when_discovered: ['__init__']
+processor:
+  - type: filter
+    expression: not name.startswith('_') and default()
+  - documented_only: true
+  - do_not_filter_modules: false
+  - skip_empty_modules: true
+renderer:
+  type: markdown
+  descriptive_class_title: true
+  descriptive_module_title: true
+  add_method_class_prefix: false
+  add_member_class_prefix: false
+  filename: crawler.md
diff --git a/docs/_src/api/api/pydoc-markdown-evaluation.yml b/docs/_src/api/api/pydoc-markdown-evaluation.yml
new file mode 100644
index 0000000000..11b8696059
--- /dev/null
+++ b/docs/_src/api/api/pydoc-markdown-evaluation.yml
@@ -0,0 +1,18 @@
+loaders:
+  - type: python
+    search_path: [../../../../haystack]
+    modules: ['eval']
+    ignore_when_discovered: ['__init__']
+processor:
+  - type: filter
+    expression: not name.startswith('_') and default()
+  - documented_only: true
+  - do_not_filter_modules: false
+  - skip_empty_modules: true
+renderer:
+  type: markdown
+  descriptive_class_title: true
+  descriptive_module_title: true
+  add_method_class_prefix: false
+  add_member_class_prefix: false
+  filename: evaluation.md
diff --git a/docs/_src/api/api/pydoc-markdown-graph-retriever.yml b/docs/_src/api/api/pydoc-markdown-graph-retriever.yml
new file mode 100644
index 0000000000..e074f2f8c7
--- /dev/null
+++ b/docs/_src/api/api/pydoc-markdown-graph-retriever.yml
@@ -0,0 +1,18 @@
+loaders:
+  - type: python
+    search_path: [../../../../haystack/graph_retriever]
+    modules: ['base', 'text_to_sparql']
+    ignore_when_discovered: ['__init__']
+processor:
+  - type: filter
+    expression: not name.startswith('_') and default()
+  - documented_only: true
+  - do_not_filter_modules: false
+  - skip_empty_modules: true
+renderer:
+  type: markdown
+  descriptive_class_title: true
+  descriptive_module_title: true
+  add_method_class_prefix: false
+  add_member_class_prefix: false
+  filename: graph_retriever.md
diff --git a/docs/_src/api/api/pydoc-markdown-knowledge-graph.yml b/docs/_src/api/api/pydoc-markdown-knowledge-graph.yml
new file mode 100644
index 0000000000..fc73424e91
--- /dev/null
+++ b/docs/_src/api/api/pydoc-markdown-knowledge-graph.yml
@@ -0,0 +1,18 @@
+loaders:
+  - type: python
+    search_path: [../../../../haystack/knowledge_graph]
+    modules: ['base', 'graphdb']
+    ignore_when_discovered: ['__init__']
+processor:
+  - type: filter
+    expression: not name.startswith('_') and default()
+  - documented_only: true
+  - do_not_filter_modules: false
+  - skip_empty_modules: true
+renderer:
+  type: markdown
+  descriptive_class_title: true
+  descriptive_module_title: true
+  add_method_class_prefix: false
+  add_member_class_prefix: false
+  filename: knowledge_graph.md
diff --git a/docs/_src/api/api/pydoc-markdown-summarizer.yml b/docs/_src/api/api/pydoc-markdown-summarizer.yml
new file mode 100644
index 0000000000..0a6ef3d768
--- /dev/null
+++ b/docs/_src/api/api/pydoc-markdown-summarizer.yml
@@ -0,0 +1,18 @@
+loaders:
+  - type: python
+    search_path: [../../../../haystack/summarizer]
+    modules: ['base', 'transformers']
+    ignore_when_discovered: ['__init__']
+processor:
+  - type: filter
+    expression: not name.startswith('_') and default()
+  - documented_only: true
+  - do_not_filter_modules: false
+  - skip_empty_modules: true
+renderer:
+  type: markdown
+  descriptive_class_title: true
+  descriptive_module_title: true
+  add_method_class_prefix: false
+  add_member_class_prefix: false
+  filename: summarizer.md
diff --git a/docs/_src/api/api/pydoc-markdown-translator.yml b/docs/_src/api/api/pydoc-markdown-translator.yml
new file mode 100644
index 0000000000..1e25cb8419
--- /dev/null
+++ b/docs/_src/api/api/pydoc-markdown-translator.yml
@@ -0,0 +1,18 @@
+loaders:
+  - type: python
+    search_path: [../../../../haystack/translator]
+    modules: ['base', 'transformers']
+    ignore_when_discovered: ['__init__']
+processor:
+  - type: filter
+    expression: not name.startswith('_') and default()
+  - documented_only: true
+  - do_not_filter_modules: false
+  - skip_empty_modules: true
+renderer:
+  type: markdown
+  descriptive_class_title: true
+  descriptive_module_title: true
+  add_method_class_prefix: false
+  add_member_class_prefix: false
+  filename: translator.md
diff --git a/docs/_src/api/api/summarizer.md b/docs/_src/api/api/summarizer.md
new file mode 100644
index 0000000000..5eca7d653a
--- /dev/null
+++ b/docs/_src/api/api/summarizer.md
@@ -0,0 +1,131 @@
+<a name="base"></a>
+# Module base
+
+<a name="base.BaseSummarizer"></a>
+## BaseSummarizer Objects
+
+```python
+class BaseSummarizer(BaseComponent)
+```
+
+Abstract class for Summarizer
+
+<a name="base.BaseSummarizer.predict"></a>
+#### predict
+
+```python
+ | @abstractmethod
+ | predict(documents: List[Document], generate_single_summary: Optional[bool] = None) -> List[Document]
+```
+
+Abstract method for creating a summary.
+
+**Arguments**:
+
+- `documents`: Related documents (e.g. coming from a retriever) that the answer shall be conditioned on.
+- `generate_single_summary`: Whether to generate a single summary for all documents or one summary per document.
+                                If set to "True", all docs will be joined to a single string that will then
+                                be summarized.
+                                Important: The summary will depend on the order of the supplied documents!
+
+**Returns**:
+
+List of Documents, where Document.text contains the summarization and Document.meta["context"]
+         the original, not summarized text
+
+<a name="transformers"></a>
+# Module transformers
+
+<a name="transformers.TransformersSummarizer"></a>
+## TransformersSummarizer Objects
+
+```python
+class TransformersSummarizer(BaseSummarizer)
+```
+
+Transformer based model to summarize the documents using the HuggingFace's transformers framework
+
+You can use any model that has been fine-tuned on a summarization task. For example:
+'`bart-large-cnn`', '`t5-small`', '`t5-base`', '`t5-large`', '`t5-3b`', '`t5-11b`'.
+See the up-to-date list of available models on
+`huggingface.co/models <https://huggingface.co/models?filter=summarization>`__
+
+**Example**
+
+```python
+|     docs = [Document(text="PG&E stated it scheduled the blackouts in response to forecasts for high winds amid dry conditions.
+|            The aim is to reduce the risk of wildfires. Nearly 800 thousand customers were scheduled to be affected by
+|            the shutoffs which were expected to last through at least midday tomorrow.")]
+|
+|     # Summarize
+|     summary = summarizer.predict(
+|        documents=docs,
+|        generate_single_summary=True
+|     )
+|
+|     # Show results (List of Documents, containing summary and original text)
+|     print(summary)
+|
+|    [
+|      {
+|        "text": "California's largest electricity provider has turned off power to hundreds of thousands of customers.",
+|        ...
+|        "meta": {
+|          "context": "PGE stated it scheduled the blackouts in response to forecasts for high winds amid dry conditions. ....
+|              },
+|        ...
+|      },
+```
+
+<a name="transformers.TransformersSummarizer.__init__"></a>
+#### \_\_init\_\_
+
+```python
+ | __init__(model_name_or_path: str = "google/pegasus-xsum", model_version: Optional[str] = None, tokenizer: Optional[str] = None, max_length: int = 200, min_length: int = 5, use_gpu: int = 0, clean_up_tokenization_spaces: bool = True, separator_for_single_summary: str = " ", generate_single_summary: bool = False)
+```
+
+Load a Summarization model from Transformers.
+See the up-to-date list of available models on
+`huggingface.co/models <https://huggingface.co/models?filter=summarization>`__
+
+**Arguments**:
+
+- `model_name_or_path`: Directory of a saved model or the name of a public model e.g.
+                           'facebook/rag-token-nq', 'facebook/rag-sequence-nq'.
+                           See https://huggingface.co/models?filter=summarization for full list of available models.
+- `model_version`: The version of model to use from the HuggingFace model hub. Can be tag name, branch name, or commit hash.
+- `tokenizer`: Name of the tokenizer (usually the same as model)
+- `max_length`: Maximum length of summarized text
+- `min_length`: Minimum length of summarized text
+- `use_gpu`: If < 0, then use cpu. If >= 0, this is the ordinal of the gpu to use
+- `clean_up_tokenization_spaces`: Whether or not to clean up the potential extra spaces in the text output
+- `separator_for_single_summary`: If `generate_single_summary=True` in `predict()`, we need to join all docs
+                                     into a single text. This separator appears between those subsequent docs.
+- `generate_single_summary`: Whether to generate a single summary for all documents or one summary per document.
+                                If set to "True", all docs will be joined to a single string that will then
+                                be summarized.
+                                Important: The summary will depend on the order of the supplied documents!
+
+<a name="transformers.TransformersSummarizer.predict"></a>
+#### predict
+
+```python
+ | predict(documents: List[Document], generate_single_summary: Optional[bool] = None) -> List[Document]
+```
+
+Produce the summarization from the supplied documents.
+These document can for example be retrieved via the Retriever.
+
+**Arguments**:
+
+- `documents`: Related documents (e.g. coming from a retriever) that the answer shall be conditioned on.
+- `generate_single_summary`: Whether to generate a single summary for all documents or one summary per document.
+                                If set to "True", all docs will be joined to a single string that will then
+                                be summarized.
+                                Important: The summary will depend on the order of the supplied documents!
+
+**Returns**:
+
+List of Documents, where Document.text contains the summarization and Document.meta["context"]
+         the original, not summarized text
+
diff --git a/docs/_src/api/api/translator.md b/docs/_src/api/api/translator.md
new file mode 100644
index 0000000000..1aace859f0
--- /dev/null
+++ b/docs/_src/api/api/translator.md
@@ -0,0 +1,105 @@
+<a name="base"></a>
+# Module base
+
+<a name="base.BaseTranslator"></a>
+## BaseTranslator Objects
+
+```python
+class BaseTranslator(ABC)
+```
+
+Abstract class for a Translator component that translates either a query or a doc from language A to language B.
+
+<a name="base.BaseTranslator.translate"></a>
+#### translate
+
+```python
+ | @abstractmethod
+ | translate(query: Optional[str] = None, documents: Optional[Union[List[Document], List[str], List[Dict[str, Any]]]] = None, dict_key: Optional[str] = None, **kwargs) -> Union[str, List[Document], List[str], List[Dict[str, Any]]]
+```
+
+Translate the passed query or a list of documents from language A to B.
+
+<a name="base.BaseTranslator.run"></a>
+#### run
+
+```python
+ | run(query: Optional[str] = None, documents: Optional[Union[List[Document], List[str], List[Dict[str, Any]]]] = None, answers: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, dict_key: Optional[str] = None, **kwargs)
+```
+
+Method that gets executed when this class is used as a Node in a Haystack Pipeline
+
+<a name="transformers"></a>
+# Module transformers
+
+<a name="transformers.TransformersTranslator"></a>
+## TransformersTranslator Objects
+
+```python
+class TransformersTranslator(BaseTranslator)
+```
+
+Translator component based on Seq2Seq models from Huggingface's transformers library.
+Exemplary use cases:
+- Translate a query from Language A to B (e.g. if you only have good models + documents in language B)
+- Translate a document from Language A to B (e.g. if you want to return results in the native language of the user)
+
+We currently recommend using OPUS models (see __init__() for details)
+
+**Example:**
+
+```python
+|    DOCS = [
+|        Document(text="Heinz von Foerster was an Austrian American scientist combining physics and philosophy,
+|                       and widely attributed as the originator of Second-order cybernetics.")
+|    ]
+|    translator = TransformersTranslator(model_name_or_path="Helsinki-NLP/opus-mt-en-de")
+|    res = translator.translate(documents=DOCS, query=None)
+```
+
+<a name="transformers.TransformersTranslator.__init__"></a>
+#### \_\_init\_\_
+
+```python
+ | __init__(model_name_or_path: str, tokenizer_name: Optional[str] = None, max_seq_len: Optional[int] = None, clean_up_tokenization_spaces: Optional[bool] = True)
+```
+
+Initialize the translator with a model that fits your targeted languages. While we support all seq2seq
+models from Hugging Face's model hub, we recommend using the OPUS models from Helsiniki NLP. They provide plenty
+of different models, usually one model per language pair and translation direction.
+They have a pretty standardized naming that should help you find the right model:
+- "Helsinki-NLP/opus-mt-en-de" => translating from English to German
+- "Helsinki-NLP/opus-mt-de-en" => translating from German to English
+- "Helsinki-NLP/opus-mt-fr-en" => translating from French to English
+- "Helsinki-NLP/opus-mt-hi-en"=> translating from Hindi to English
+...
+
+They also have a few multilingual models that support multiple languages at once.
+
+**Arguments**:
+
+- `model_name_or_path`: Name of the seq2seq model that shall be used for translation.
+                           Can be a remote name from Huggingface's modelhub or a local path.
+- `tokenizer_name`: Optional tokenizer name. If not supplied, `model_name_or_path` will also be used for the
+                       tokenizer.
+- `max_seq_len`: The maximum sentence length the model accepts. (Optional)
+- `clean_up_tokenization_spaces`: Whether or not to clean up the tokenization spaces. (default True)
+
+<a name="transformers.TransformersTranslator.translate"></a>
+#### translate
+
+```python
+ | translate(query: Optional[str] = None, documents: Optional[Union[List[Document], List[str], List[Dict[str, Any]]]] = None, dict_key: Optional[str] = None, **kwargs) -> Union[str, List[Document], List[str], List[Dict[str, Any]]]
+```
+
+Run the actual translation. You can supply a query or a list of documents. Whatever is supplied will be translated.
+
+**Arguments**:
+
+- `query`: The query string to translate
+:type query: str
+- `documents`: The documents to translate
+:type documents: List[Document] or List[Dict]
+- `dict_key`: 
+:type dict_key:
+
diff --git a/docs/_src/tutorials/tutorials/9.md b/docs/_src/tutorials/tutorials/9.md
index efa44a1f6f..52a091378b 100644
--- a/docs/_src/tutorials/tutorials/9.md
+++ b/docs/_src/tutorials/tutorials/9.md
@@ -69,7 +69,8 @@ They were used by the original DPR authors in an experiment to compare it agains
 In the original DPR paper, these are fetched using a retriever to find the most relevant passages to the query.
 Passages which contain the answer text are filtered out.
 
-We are [currently working](https://github.com/deepset-ai/haystack/issues/705) on a script that will convert SQuAD format data into a DPR dataset!
+If you'd like to convert your SQuAD format data into something that can train a DPR model,
+check out the utility script at [`haystack/retriever/squad_to_dpr.py`](https://github.com/deepset-ai/haystack/blob/master/haystack/retriever/squad_to_dpr.py)
 
 ## Using Question Answering Data
 
diff --git a/docs/_src/usage/usage/document_store.md b/docs/_src/usage/usage/document_store.md
index 57c26b48f1..ce132cbb24 100644
--- a/docs/_src/usage/usage/document_store.md
+++ b/docs/_src/usage/usage/document_store.md
@@ -1,11 +1,11 @@
----
+<!---
 title: "Document Store"
 metaTitle: "Document Store"
 metaDescription: ""
 slug: "/docs/documentstore"
 date: "2020-09-03"
 id: "documentstoremd"
----
+--->
 
 
 # DocumentStores
@@ -40,6 +40,8 @@ docker run -d -p 9200:9200 -e "discovery.type=single-node" elasticsearch:7.9.2
 Next you can initialize the Haystack object that will connect to this instance.
 
 ```python
+from haystack.document_store import ElasticsearchDocumentStore
+
 document_store = ElasticsearchDocumentStore()
 ```
 
@@ -60,6 +62,8 @@ Follow the [official documentation](https://www.milvus.io/docs/v1.0.0/milvus_doc
  
 You can initialize the Haystack object that will connect to this instance as follows:
 ```python
+from haystack.document_store import MilvusDocumentStore
+
 document_store = MilvusDocumentStore()
 ```
 
@@ -73,6 +77,8 @@ document_store = MilvusDocumentStore()
 
 The `FAISSDocumentStore` requires no external setup. Start it by simply using this line. 
 ```python
+from haystack.document_store import FAISSDocumentStore
+
 document_store = FAISSDocumentStore(faiss_index_factory_str="Flat")
 ```
 
@@ -86,6 +92,8 @@ document_store = FAISSDocumentStore(faiss_index_factory_str="Flat")
 
 The `InMemoryDocumentStore()` requires no external setup. Start it by simply using this line.
 ```python
+from haystack.document_store import InMemoryDocumentStore
+
 document_store = InMemoryDocumentStore()
 ```
 
@@ -101,6 +109,8 @@ The `SQLDocumentStore` requires SQLite, PostgresQL or MySQL to be installed and
 Note that SQLite already comes packaged with most operating systems. 
 
 ```python
+from haystack.document_store import SQLDocumentStore
+
 document_store = SQLDocumentStore()
 ```
 
@@ -121,6 +131,8 @@ See [Preprocessing](/docs/latest/preprocessingmd) for more information on the cl
 [//]: # (Add link to preprocessing section)
 
 ```python
+from haystack.document_store import ElasticsearchDocumentStore
+
 document_store = ElasticsearchDocumentStore()
 dicts = [
     {
diff --git a/docs/_src/usage/usage/generator.md b/docs/_src/usage/usage/generator.md
index 8763cb69c3..cdf40ade9c 100644
--- a/docs/_src/usage/usage/generator.md
+++ b/docs/_src/usage/usage/generator.md
@@ -9,14 +9,6 @@ id: "generatormd"
 
 # Generator
 
-<div class="recommendation">
-
-**Example**
-
-See [Tutorial 7](/docs/latest/tutorial7md) for a guide on how to build your own generative QA system.
-
-</div>
-
 While extractive QA highlights the span of text that answers a query,
 generative QA can return a novel text answer that it has composed.
 The best current approaches, such as [Retriever-Augmented Generation](https://arxiv.org/abs/2005.11401),
@@ -25,9 +17,15 @@ as well as passages provided to it with a retriever (non-parametric memory).
 With the advent of Transformer based retrieval methods such as [Dense Passage Retrieval](https://arxiv.org/abs/2004.04906),
 retriever and generator can be trained concurrently from the one loss signal.
 
+<div class="recommendation">
+
+See [Tutorial 7](/docs/latest/tutorial7md) for a guide on how to build your own generative QA system.
+
+</div>
+
 Pros
 * More appropriately phrased answers
-* Able to syntehsize information from different texts
+* Able to synthesize information from different texts
 * Can draw on latent knowledge stored in language model
 
 Cons
diff --git a/docs/_src/usage/usage/get_started.md b/docs/_src/usage/usage/get_started.md
index 3400085afa..69d5bfd03a 100644
--- a/docs/_src/usage/usage/get_started.md
+++ b/docs/_src/usage/usage/get_started.md
@@ -48,14 +48,11 @@ By default, this will give you the latest version of the master branch. Use regu
 
 Note: On Windows add the arg `-f https://download.pytorch.org/whl/torch_stable.html` to install PyTorch correctly
 
-<!-- _comment: !! Have a tab for docker!! -->
-<!-- -comment: !! Have a hello world example!! -->
 ## The Building Blocks of Haystack
 
-Here’s a sample of some Haystack code showing the most important components.
-For a working code example, check out our starter tutorial.
+Here’s a sample of some Haystack code showing a question answering system using a retriever and a reader.
+For a working code example, check out our [starter tutorial](/docs/latest/tutorial1md).
 
-<!-- _comment: !!link!! -->
 ```python
 # DocumentStore: holds all your data
 document_store = ElasticsearchDocumentStore()
@@ -71,12 +68,12 @@ retriever = ElasticsearchRetriever(document_store)
 model_name = "deepset/roberta-base-squad2"
 reader = FARMReader(model_name)
 
-# Finder: Combines Reader and Retriever
-finder = Finder(reader, retriever)
+# Pipeline: Combines all the components
+pipe = ExtractiveQAPipeline(reader, retriever)
 
 # Voilà! Ask a question!
 question = "Who is the father of Sansa Stark?"
-prediction = finder.get_answers(question)
+prediction = pipe.run(query=question)
 print_answers(prediction)
 ```
 
@@ -97,18 +94,25 @@ document_store.write_documents(dicts)
 
 When we talk about Documents in Haystack, we are referring specifically to the individual blocks of text that are being held in the DocumentStore.
 You might want to use all the text in one file as a Document, or split it into multiple Documents.
-This splitting can have a big impact on speed and performance.
+This splitting can have a big impact on speed and performance. See 
 
 <div class="recommendation">
 
 **Tip:** If Haystack is running very slowly, you might want to try splitting your text into smaller Documents.
 If you want an improvement to performance, you might want to try concatenating text to make larger Documents.
+See [Optimization](/docs/latest/optimizationmd) for more details.
+
 
 </div>
 
-## Running Queries
+## Running Search Queries
+
+There are many different flavours of search that can be created using Haystack.
+But to give just one example of what can be achieved, let's look more closely at 
+an Open Domain Question Answering (ODQA) Pipeline.
+
 
-**Querying** involves searching for an answer to a given question within the full document store.
+**Querying** in an ODQA system involves searching for an answer to a given question within the full document store.
 This process will:
 
 
@@ -123,15 +127,16 @@ This process will:
 Usually, there are tight time constraints on querying and so it needs to be a lightweight operation.
 When documents are loaded, Haystack will precompute any of the results that might be useful at query time.
 
-In Haystack, querying is performed with a `Finder` object which connects the reader to the retriever.
+In Haystack, querying is performed with a `Pipeline` object which connects the reader to the retriever.
 
 ```python
-# The Finder sticks together reader and retriever in a pipeline to answer our questions.
-finder = Finder(reader, retriever)
+# Pipeline: Combines all the components
+pipe = ExtractiveQAPipeline(reader, retriever)
 
 # Voilà! Ask a question!
 question = "Who is the father of Sansa Stark?"
-prediction = finder.get_answers(question)
+prediction = pipe.run(query=question)
+print_answers(prediction)
 ```
 
 When the query is complete, you can expect to see results that look something like this:
@@ -146,8 +151,16 @@ When the query is complete, you can expect to see results that look something li
 ]
 ```
 
-<div class="recommendation">
+##Custom Search Pipelines
 
-**Tip:** The Finder class is being deprecated and has been replaced by a more powerful [Pipelines class](/docs/latest/pipelinesmd).
+Haystack providers many different building blocks for you to mix and match.
+They include:
+- Readers
+- Retrievers (sparse and dense)
+- DocumentStores
+- Summarizers
+- Generators
+- Translators
 
-</div>
+These can all be combined in the configuration that you want.
+Have a look at our [Pipelines page](/docs/latest/pipelinesmd) to see what's possible!
\ No newline at end of file
diff --git a/docs/_src/usage/usage/intro.md b/docs/_src/usage/usage/intro.md
index d20372c134..71037d23c9 100644
--- a/docs/_src/usage/usage/intro.md
+++ b/docs/_src/usage/usage/intro.md
@@ -9,27 +9,27 @@ id: "intromd"
 
 # What is Haystack?
 
-Haystack is an **open-source framework** for building **end-to-end question answering systems** for large document collections.
-Recent advances in NLP have enabled the application of QA to real world settings
+Haystack is an **open-source framework** for building **search systems** that work intelligently over large document collections.
+Recent advances in NLP have enabled the application of question answering, retrieval and summarization to real world settings
 and Haystack is designed to be the bridge between research and industry.
 
 
 * **Latest NLP models**: Utilize all transformer based models (BERT, RoBERTa, MiniLM, DPR ...) and smoothly switch when new ones get published
 
-* **Flexible databases**: Load data into and query from a range of databases such as Elasticsearch, FAISS, SQL and more
+* **Flexible databases**: Load data into and query from a range of databases such as Elasticsearch, Milvus, FAISS, SQL and more
 
 * **Scalability**: Production-ready deployments that scale to millions of documents
 
-* **End-to-End**: All tooling you need to implement, evaluate, improve and run a QA system
+* **End-to-End**: All tooling you need to implement, evaluate, improve and run a search system
 
 * **Domain adaptation**: Fine-tune models to your own domain & improve them continuously via user feedback
 
-<!-- _comment: !!Image!! -->
-## Technology
-
-Haystack is powered by a Retriever-Reader pipeline in order to optimise for both speed and accuracy.
-
+## Retriever-Reader
 
+The most common system built with Haystack is the Retriever-Reader Pipeline which is designed to optimize for both
+speed and performance on the task of Open Domain Question Answering.
+In practice, this is a great back bone for creating a search system that can handle detailed full sentence queries.
+ 
 
 ![image](./../../img/retriever_reader.png)
 
diff --git a/docs/_src/usage/usage/languages.md b/docs/_src/usage/usage/languages.md
index 7c5be7e03b..86336d990d 100644
--- a/docs/_src/usage/usage/languages.md
+++ b/docs/_src/usage/usage/languages.md
@@ -28,6 +28,8 @@ See [this page](https://www.elastic.co/guide/en/elasticsearch/reference/current/
 for the full list of language specific analyzers.
 
 ```python
+from haystack.document_store import ElasticsearchDocumentStore
+
 document_store = ElasticsearchDocumentStore(analyzer="thai")
 ```
 
@@ -37,9 +39,12 @@ The default model that is loaded in the DensePassageRetriever is for English.
 We are currently working on training a German DensePassageRetriever model and know other teams who work on further languages.
 If you have a language model and a question answering dataset in your own language, you can also train a DPR model using Haystack!
 Below is a simplified example.
-See the [API reference](/docs/latest/apiretrievermd#train) for `DensePassageRetriever.train()` for more details.
+See [our tutorial](/docs/latest/tutorial9md) and also the [API reference](/docs/latest/apiretrievermd#train) for `DensePassageRetriever.train()` for more details.
 
 ```python
+from haystack.retriever import DensePassageRetriever
+
+dense_passage_retriever = DensePassageRetriever(document_store)
 dense_passage_retriever.train(self,
                               data_dir: str,
                               train_filename: str,
@@ -72,6 +77,8 @@ there are a couple QA models that are directly usable in Haystack.
 <div class="tabcontentinner">
 
 ```python
+from haystack.reader import FARMReader
+
 reader = FARMReader("illuin/camembert-base-fquad")
 ```
 
@@ -84,6 +91,8 @@ reader = FARMReader("illuin/camembert-base-fquad")
 <div class="tabcontentinner">
 
 ```python
+from haystack.reader import FARMReader
+
 reader = FARMReader("mrm8488/bert-italian-finedtuned-squadv1-it-alfa")
 ```
 
@@ -96,6 +105,8 @@ reader = FARMReader("mrm8488/bert-italian-finedtuned-squadv1-it-alfa")
 <div class="tabcontentinner">
 
 ```python
+from haystack.reader import FARMReader
+
 reader = FARMReader("deepset/xlm-roberta-large-squad2")
 ```
 
@@ -120,6 +131,8 @@ reader = FARMReader("deepset/xlm-roberta-large-squad2")
 <div class="tabcontentinner">
 
 ```python
+from haystack.reader import TransformersReader
+
 reader = TransformersReader("illuin/camembert-base-fquad")
 ```
 
@@ -132,6 +145,8 @@ reader = TransformersReader("illuin/camembert-base-fquad")
 <div class="tabcontentinner">
 
 ```python
+from haystack.reader import TransformersReader
+
 reader = TransformersReader("mrm8488/bert-italian-finedtuned-squadv1-it-alfa")
 ```
 
@@ -144,6 +159,8 @@ reader = TransformersReader("mrm8488/bert-italian-finedtuned-squadv1-it-alfa")
 <div class="tabcontentinner">
 
 ```python
+from haystack.reader import TransformersReader
+
 reader = TransformersReader("deepset/xlm-roberta-large-squad2")
 ```
 
diff --git a/docs/_src/usage/usage/optimization.md b/docs/_src/usage/usage/optimization.md
index b2f24b5c67..f129cfa74e 100644
--- a/docs/_src/usage/usage/optimization.md
+++ b/docs/_src/usage/usage/optimization.md
@@ -49,16 +49,9 @@ However, passing more documents to the `Reader` will create a larger workload fo
 
 These parameters can easily be tweaked as follows if using a `Finder`:
 ``` python
-answers = finder.get_answers(retriever_top_k=10,
-                             reader_top_k=5)
+answers = pipeline.run(query="What did Einstein work on?", top_k_retriever=10, top_k_reader=5)
 ```
 or like this if directly calling the `Retriever`:
 ``` python
 retrieved_docs = retriever.retrieve(top_k=10)
 ```
-
-<div class="recommendation">
-
-**Tip:** The Finder class is being deprecated and has been replaced by a more powerful [Pipelines class](/docs/latest/pipelinesmd).
-
-</div>
diff --git a/docs/_src/usage/usage/pipelines.md b/docs/_src/usage/usage/pipelines.md
index 3483481d07..75a902743a 100644
--- a/docs/_src/usage/usage/pipelines.md
+++ b/docs/_src/usage/usage/pipelines.md
@@ -9,25 +9,19 @@ id: "pipelinesmd"
 
 # Pipelines
 
-
-<div class="recommendation">
-
-The new Pipelines class was added in Haystack 0.6.0 to give a more flexible way of defining your processing steps. 
-It replaces the Finder class which will be deprecated in the next version.
-
-</div>
-
-
-### Flexible Pipelines powered by DAGs
-In order to build modern search pipelines, you need two things: powerful building blocks and a flexible way to stick them together.
-The `Pipeline` class is exactly build for this purpose and enables many search scenarios beyond QA. The core idea: you can build a Directed Acyclic Graph (DAG) where each node is one "building block" (Reader, Retriever, Generator ...). Here's a simple example for a "standard" Open-Domain QA Pipeline: 
+### Flexibility powered by DAGs
+In order to build modern search pipelines, you need two things: powerful building blocks and an easy way to stick them together.
+The `Pipeline` class is exactly built for this purpose and enables many search scenarios beyond QA. 
+The core idea is that you can build a Directed Acyclic Graph (DAG) where each node is one building block (Reader, Retriever, Generator ...). 
+Here's a simple example for a standard Open-Domain QA Pipeline: 
 
 ```python
+from haystack import Pipeline
+
 p = Pipeline()
 p.add_node(component=retriever, name="ESRetriever1", inputs=["Query"])
 p.add_node(component=reader, name="QAReader", inputs=["ESRetriever1"])
 res = p.run(query="What did Einstein work on?", top_k_retriever=1)
-
 ```
 
 You can **draw the DAG** to better inspect what you are building:
@@ -36,9 +30,52 @@ p.draw(path="custom_pipe.png")
 ```
 ![image](https://user-images.githubusercontent.com/1563902/102451716-54813700-4039-11eb-881e-f3c01b47ca15.png)
 
+### YAML File Definitions
+
+For your convenience, there is also the option of defining and loading pipelines in YAML files.
+For example, you can define an save a simple Retriever Reader pipeline by saving the following to a file:
+
+```yaml
+version: '0.7'
+
+components:    # define all the building-blocks for Pipeline
+- name: MyReader       # custom-name for the component; helpful for visualization & debugging
+  type: FARMReader    # Haystack Class name for the component
+  params:
+    no_ans_boost: -10
+    model_name_or_path: deepset/roberta-base-squad2
+- name: MyESRetriever
+  type: ElasticsearchRetriever
+  params:
+    document_store: MyDocumentStore    # params can reference other components defined in the YAML
+    custom_query: null
+- name: MyDocumentStore
+  type: ElasticsearchDocumentStore
+  params:
+    index: haystack_test
+
+pipelines:    # multiple Pipelines can be defined using the components from above
+- name: my_query_pipeline    # a simple extractive-qa Pipeline
+  nodes:
+  - name: MyESRetriever
+    inputs: [Query]
+  - name: MyReader
+    inputs: [MyESRetriever]
+```
+
+To load, simply call:
+
+```python
+pipeline.load_from_yaml(Path("sample.yaml"))
+```
+
+For another example YAML config, check out [this file](https://github.com/deepset-ai/haystack/blob/master/rest_api/pipelines.yaml).
+
 ### Multiple retrievers
 You can now also use multiple Retrievers and join their results: 
 ```python
+from haystack import Pipeline
+
 p = Pipeline()
 p.add_node(component=es_retriever, name="ESRetriever", inputs=["Query"])
 p.add_node(component=dpr_retriever, name="DPRRetriever", inputs=["Query"])
@@ -80,6 +117,11 @@ Or you can add decision nodes where only one "branch" is executed afterwards. Th
     res = p.run(query="What did Einstein work on?", top_k_retriever=1)
 ```
 
+### Evaluation nodes
+
+There are nodes in Haystack that are used to evaluate the performance of readers, retrievers and combine systems.
+To get hands on with this kind of node, have a look at the [evaluation tutorial](/docs/latest/tutorial5md).
+
 ### Default Pipelines (replacing the "Finder")
 Last but not least, we added some "Default Pipelines" that allow you to run standard patterns with very few lines of code.
 This is replacing the `Finder` class which is now deprecated.
@@ -106,5 +148,5 @@ res = doc_pipe.run(query="How can I change my address?", top_k_retriever=3)
 ```    
 See also the [Pipelines API documentation](/docs/latest/apipipelinesmd) for more details. 
 
-We plan many more features around the new pipelines incl. parallelized execution, distributed execution, definition via YAML files, dry runs - so stay tuned ...  
+We plan many more features around the new pipelines incl. parallelized execution, distributed execution, dry runs - so stay tuned ...  
 
diff --git a/docs/_src/usage/usage/preprocessing.md b/docs/_src/usage/usage/preprocessing.md
index a4c7b3ed57..c84c478241 100644
--- a/docs/_src/usage/usage/preprocessing.md
+++ b/docs/_src/usage/usage/preprocessing.md
@@ -15,6 +15,12 @@ Haystack includes a suite of tools to:
 * normalize white space
 * split text into smaller pieces to optimize retrieval
 
+<div class="recommendation">
+
+Check out our [preprocessing tutorial](/docs/latest/tutorial8md) if you'd like to start working with code examples already!
+
+</div>
+
 These data preprocessing steps can have a big impact on the systems performance
 and effective handling of data is key to getting the most out of Haystack.
 
@@ -45,6 +51,8 @@ Please refer to [the API docs](/docs/latest/file_convertersmd) to see which conv
 <div class="tabcontent">
 
 ```python
+from haystack.file_converter import PDFToTextConverter
+
 converter = PDFToTextConverter(remove_numeric_tables=True, valid_languages=["de","en"])
 doc = converter.convert(file_path=file, meta=None)
 ```
@@ -58,6 +66,8 @@ doc = converter.convert(file_path=file, meta=None)
 <div class="tabcontent">
 
 ```python
+from haystack.file_converter import DocxToTextConverter
+
 converter = DocxToTextConverter(remove_numeric_tables=True, valid_languages=["de","en"])
 doc = converter.convert(file_path=file, meta=None)
 ```
@@ -75,6 +85,8 @@ Haystack also has a `convert_files_to_dicts()` utility function that will conver
 all txt or pdf files in a given folder into this dictionary format.
 
 ```python
+from haystack.preprocessor.utils import convert_files_to_dicts
+
 docs = convert_files_to_dicts(dir_path=doc_dir)
 ```
 
@@ -83,6 +95,20 @@ docs = convert_files_to_dicts(dir_path=doc_dir)
 
 </div>
 
+## Web Crawler
+
+In Haystack, you will find a web crawler that will help you scrape text from websites and save it to file. 
+See the API documentation for more details.
+
+```python
+from haystack.connector import Crawler
+
+crawler = Crawler()
+docs = crawler.crawl(urls=["https://haystack.deepset.ai/docs/latest/get_startedmd"],
+                     output_dir="crawled_files",
+                     filter_urls= ["haystack\.deepset\.ai\/docs\/"])
+```
+
 ## PreProcessor
 
 While each of the above conversion methods produce documents that are already in the format expected by the Document Store,
@@ -93,6 +119,8 @@ performs various cleaning steps and splits them into multiple smaller documents.
 For suggestions on how best to split your documents, see [Optimization](/docs/latest/optimizationmd)
 
 ```python
+from haystack.preprocessor import PreProcessor
+
 doc = converter.convert(file_path=file, meta=None)
 processor = PreProcessor(
     clean_empty_lines=True,
diff --git a/docs/_src/usage/usage/reader.md b/docs/_src/usage/usage/reader.md
index 8b956aaf55..f764805ffb 100644
--- a/docs/_src/usage/usage/reader.md
+++ b/docs/_src/usage/usage/reader.md
@@ -33,9 +33,10 @@ Haystack’s Readers are:
 <div class="tabcontent">
 
 ```python
+from haystack.reader import FARMReader
+
 model = "deepset/roberta-base-squad2"
 reader = FARMReader(model, use_gpu=True)
-finder = Finder(reader, retriever)
 ```
 
 </div> 
@@ -47,9 +48,10 @@ finder = Finder(reader, retriever)
 <div class="tabcontent">
 
 ```python
+from haystack.reader import TransformersReader
+
 model = "deepset/roberta-base-squad2"
 reader = TransformersReader(model, use_gpu=1)
-finder = Finder(reader, retriever)
 ```
 
 </div> 
@@ -57,12 +59,6 @@ finder = Finder(reader, retriever)
 
 </div>
 
-<div class="recommendation">
-
-**Tip:** The Finder class is being deprecated and has been replaced by a more powerful [Pipelines class](/docs/latest/pipelinesmd).
-
-</div>
-
 While these models can work on CPU, it is recommended that they are run using GPUs to keep query times low.
 
 ## Choosing the Right Model
@@ -91,6 +87,8 @@ To get you started, we have a few recommendations for you to try out.
 **An optimised variant of BERT and a great starting point.**
 
 ```python
+from haystack.reader import FARMReader
+
 reader = FARMReader("deepset/roberta-base-squad2")
 ```
 
@@ -109,6 +107,8 @@ reader = FARMReader("deepset/roberta-base-squad2")
 **A cleverly distilled model that sacrifices a little accuracy for speed.**
 
 ```python
+from haystack.reader import FARMReader
+
 reader = FARMReader("deepset/minilm-uncased-squad2")
 ```
 
@@ -127,6 +127,8 @@ reader = FARMReader("deepset/minilm-uncased-squad2")
 **Large, powerful, SotA model.**
 
 ```python
+from haystack.reader import FARMReader
+
 reader = FARMReader("ahotrod/albert_xxlargev1_squad2_512")
 ```
 
@@ -157,6 +159,8 @@ reader = FARMReader("ahotrod/albert_xxlargev1_squad2_512")
 **An optimised variant of BERT and a great starting point.**
 
 ```python
+from haystack.reader import TransformersReader
+
 reader = TransformersReader("deepset/roberta-base-squad2")
 ```
 
@@ -175,6 +179,8 @@ reader = TransformersReader("deepset/roberta-base-squad2")
 **A cleverly distilled model that sacrifices a little accuracy for speed.**
 
 ```python
+from haystack.reader import TransformersReader
+
 reader = TransformersReader("deepset/minilm-uncased-squad2")
 ```
 
@@ -193,6 +199,8 @@ reader = TransformersReader("deepset/minilm-uncased-squad2")
 **Large, powerful, SotA model.**
 
 ```python
+from haystack.reader import TransformersReader
+
 reader = TransformersReader("ahotrod/albert_xxlargev1_squad2_512")
 ```
 
@@ -232,6 +240,47 @@ you might like to try ALBERT XXL which has set SoTA performance on SQuAD 2.0.
 
 </div>
 
+
+## Confidence Scores
+
+When printing the full results of a Reader, 
+you will see that each prediction is accompanied 
+by a value in the range of 0 to 1 reflecting the model's confidence in that prediction
+
+In the output of `print_answers()`, you will find the model confidence in dictionary key called `probability`.
+
+```python
+from haystack.utils import print_answers
+
+print_answers(prediction, details="all")
+```
+
+```python
+{   
+    'answers': [   
+        {   'answer': 'Eddard',
+            'context': 's Nymeria after a legendary warrior queen. '
+                       'She travels with her father, Eddard, to '
+                       "King's Landing when he is made Hand of the "
+                       'King. Before she leaves,',
+            'probability': 0.9899835586547852,
+            ...
+        },
+    ]
+}
+```
+
+In order to align this probability score with the model's accuracy, finetuning needs to be performed
+on a specific dataset. Have a look at this [FARM tutorial](https://github.com/deepset-ai/FARM/blob/master/examples/question_answering_confidence.py)
+to see how this is done. 
+Note that a finetuned confidence score is specific to the domain that its finetuned on. 
+There is no guarantee that this performance can transfer to a new domain.
+
+Having a confidence score is particularly useful in cases where you need Haystack to work with a certain accuracy threshold.
+Many of our users have built systems where predictions below a certain confidence value are routed
+on to a fallback system.
+
+
 <!-- farm-vs-trans: -->
 ## Deeper Dive: FARM vs Transformers
 
@@ -259,6 +308,8 @@ Haystack also has a close integration with FARM which means that you can further
 See our tutorials for an end-to-end example or below for a shortened example.
 
 ```python
+from haystack.reader import FARMReader
+
 # Initialise Reader
 model = "deepset/roberta-base-squad2"
 reader = FARMReader(model)
@@ -302,6 +353,8 @@ These can be set when the Reader is initialized.
 <div class="tabcontent">
 
 ```python
+from haystack.reader import FARMReader
+
 reader = FARMReader(... max_seq_len=384, doc_stride=128 ...)
 ```
 
@@ -314,7 +367,9 @@ reader = FARMReader(... max_seq_len=384, doc_stride=128 ...)
 <div class="tabcontent">
 
 ```python
-reader = TransformersReader(... max_seq_len=384, doc_stride=128 ...
+from haystack.reader import TransformersReader
+
+reader = TransformersReader(... max_seq_len=384, doc_stride=128 ...)
 ```
 
 </div> 
@@ -324,5 +379,3 @@ reader = TransformersReader(... max_seq_len=384, doc_stride=128 ...
 
 Predictions are made on each individual passage and the process of aggregation picks the best candidates across all passages.
 If you’d like to learn more about what is happening behind the scenes, have a look at [this](https://medium.com/deepset-ai/modern-question-answering-systems-explained-4d0913744097) article.
-
-<!-- _comment: !! Diagram from Blog !! -->
diff --git a/docs/_src/usage/usage/retriever.md b/docs/_src/usage/usage/retriever.md
index d4a52afc7d..94bc1401fe 100644
--- a/docs/_src/usage/usage/retriever.md
+++ b/docs/_src/usage/usage/retriever.md
@@ -9,8 +9,8 @@ id: "retrievermd"
 
 # Retriever
 
-The Retriever is a lightweight filter that can quickly go through the full document store and pass a set of candidate documents to the Reader.
-It is an tool for sifting out the obvious negative cases, saving the Reader from doing more work than it needs to and speeding up the querying process.
+The Retriever is a lightweight filter that can quickly go through the full document store and pass on a set of candidate documents that are relevant to the query.
+When used in combination with a Reader, it is a tool for sifting out the obvious negative cases, saving the Reader from doing more work than it needs to and speeding up the querying process.
 
 <div class="recommendation">
 
@@ -46,7 +46,6 @@ TF-IDF is a commonly used baseline for information retrieval that exploits two k
 
 * documents that have more lexical overlap with the query are more likely to be relevant
 
-
 * words that occur in fewer documents are more significant than words that occur in many documents
 
 Given a query, a tf-idf score is computed for each document as follows:
@@ -68,19 +67,17 @@ In practice, both terms are usually log normalised.
 ### Initialisation
 
 ```python
+from haystack.document_store import InMemoryDocumentStore
+from haystack.retriever.sparse import TfidfRetriever
+from haystack.pipeline import ExtractiveQAPipeline
+
 document_store = InMemoryDocumentStore()
 ...
 retriever = TfidfRetriever(document_store)
 ...
-finder = Finder(reader, retriever)
+p = ExtractiveQAPipeline(reader, retriever)
 ```
 
-<div class="recommendation">
-
-**Tip:** The Finder class is being deprecated and has been replaced by a more powerful [Pipelines class](/docs/latest/pipelinesmd).
-
-</div>
-
 ## BM25 (Recommended)
 
 ### Description
@@ -97,19 +94,17 @@ It improves upon its predecessor in two main aspects:
 ### Initialisation
 
 ```python
+from haystack.document_store import ElasticsearchDocumentStore
+from haystack.retriever import ElasticsearchRetriever
+from haystack.pipeline import ExtractiveQAPipeline
+
 document_store = ElasticsearchDocumentStore()
 ...
 retriever = ElasticsearchRetriever(document_store)
 ...
-finder = Finder(reader, retriever)
+p = ExtractiveQAPipeline(reader, retriever)
 ```
 
-<div class="recommendation">
-
-**Tip:** The Finder class is being deprecated and has been replaced by a more powerful [Pipelines class](/docs/latest/pipelinesmd).
-
-</div>
-
 See [this](https://www.elastic.co/blog/practical-bm25-part-2-the-bm25-algorithm-and-its-variables) blog post for more details about the algorithm.
 
 <!-- _comment: !! Diagram showing TFIDF vs BM25 !! -->
@@ -158,6 +153,10 @@ as is done in the code example below.
 </div>
 
 ```python
+from haystack.document_store import FAISSDocumentStore
+from haystack.retriever import DensePassageRetriever
+from haystack.pipeline import ExtractiveQAPipeline
+
 document_store = FAISSDocumentStore(similarity="dot_product")
 ...
 retriever = DensePassageRetriever(
@@ -166,7 +165,7 @@ retriever = DensePassageRetriever(
     passage_embedding_model="facebook/dpr-ctx_encoder-single-nq-base"
 )
 ...
-finder = Finder(reader, retriever)
+finder = ExtractiveQAPipeline(reader, retriever)
 ```
 
 <div class="recommendation">
@@ -175,14 +174,6 @@ finder = Finder(reader, retriever)
 
 </div>
 
-<div class="recommendation">
-
-**Tip:** The Finder class is being deprecated and has been replaced by a more powerful [Pipelines class](/docs/latest/pipelinesmd).
-
-</div>
-
-
-
 <!-- _comment: !! Training in future? !! -->
 <!-- _comment: !! Talk more about benchmarks, SoTA, results !! -->
 ## Embedding Retrieval
@@ -210,20 +201,18 @@ as is done in the code example below.
 ### Initialisation
 
 ```python
+from haystack.document_store import ElasticsearchDocumentStore
+from haystack.retriever import EmbeddingRetriever
+from haystack.pipeline import ExtractiveQAPipeline
+
 document_store = ElasticsearchDocumentStore(similarity="cosine")
 ...
 retriever = EmbeddingRetriever(document_store=document_store,
                                embedding_model="deepset/sentence_bert")
 ...
-finder = Finder(reader, retriever)
+p = ExtractiveQAPipeline(reader, retriever)
 ```
 
-<div class="recommendation">
-
-**Tip:** The Finder class is being deprecated and has been replaced by a more powerful [Pipelines class](/docs/latest/pipelinesmd).
-
-</div>
-
 ## Deeper Dive: Dense vs Sparse
 
 Broadly speaking, retrieval methods can be split into two categories: **dense** and **sparse**.
diff --git a/docs/_src/usage/usage/summarizer.md b/docs/_src/usage/usage/summarizer.md
new file mode 100644
index 0000000000..93bde9cdaf
--- /dev/null
+++ b/docs/_src/usage/usage/summarizer.md
@@ -0,0 +1,56 @@
+<!---
+title: "Summarizer"
+metaTitle: "Summarizer"
+metaDescription: "Using the Summarizer Node"
+slug: "/docs/summarizer"
+date: "2021-04-16"
+id: "summarizermd"
+--->
+
+# Summarizer
+
+Retrievers are excellent at returning a set of candidate documents,
+but you might not have the time to read through them all.
+Haystack's Summmarizer is here to help you make sense of the documents at a glance.
+
+There is a full integration with Huggingface Transformers and using any of their summarization
+models is as simple as providing the model name.
+See the up-to-date list of available models [here](https://huggingface.co/models?filter=summarization).
+By default, the Google [Pegasus](https://ai.googleblog.com/2020/06/pegasus-state-of-art-model-for.html) model is loaded.
+
+```python
+from haystack.summarizer import TransformersSummarizer
+from haystack.schema import Document
+
+docs = [Document("PG&E stated it scheduled the blackouts in response to forecasts for high winds amid dry conditions.\
+                 The aim is to reduce the risk of wildfires. Nearly 800 thousand customers were scheduled to be affected by\
+                 the shutoffs which were expected to last through at least midday tomorrow.")]
+
+summarizer = TransformersSummarizer(model_name_or_path="google/pegasus-xsum")
+summary = summarizer.predict(documents=docs, generate_single_summary=True)
+```
+
+The contents of summary should contain both the summarization and also the original document text.
+
+```python
+[
+    {
+        "text": "California's largest electricity provider has turned off power to hundreds of thousands of customers.",
+        "meta": {
+            "context": "PGE stated it scheduled the blackouts in response to forecasts for high winds amid dry conditions."
+        },
+        ...
+    }
+]
+```
+
+The summarizer can also functions as a node in a pipeline.
+
+```python
+from haystack.pipeline import Pipeline
+
+p = Pipeline()
+p.add_node(component=retriever, name="ESRetriever1", inputs=["Query"])
+p.add_node(component=summarizer, name="Summarizer", inputs=["ESRetriever1"])
+res = p.run(query="What did Einstein work on?", top_k_retriever=10)
+``` 
diff --git a/docs/_src/usage/usage/terms.md b/docs/_src/usage/usage/terms.md
index d79f677cef..fb77485096 100644
--- a/docs/_src/usage/usage/terms.md
+++ b/docs/_src/usage/usage/terms.md
@@ -24,14 +24,6 @@ Can have a variety of backends such as Elasticsearch, SQL or FAISS.
 **FARM** - An open-source transfer learning [framework](https://github.com/deepset-ai/FARM) by deepset.
 FARM’s question answering models are used in Haystack’s Readers.
 
-**Finder** - The component in Haystack that connects the Retriever to the Reader.
-
-<div class="recommendation">
-
-**Tip:** The Finder class is being deprecated and has been replaced by a more powerful [Pipelines class](/docs/latest/pipelinesmd).
-
-</div>
-
 **Indexing** - To store data in a database in a way that optimises retrieval time.
 The exact steps involved in indexing depend on what kind of retrieval method is chosen.
 
diff --git a/docs/_src/usage/usage/translator.md b/docs/_src/usage/usage/translator.md
index d7b71dadf4..409e30148d 100644
--- a/docs/_src/usage/usage/translator.md
+++ b/docs/_src/usage/usage/translator.md
@@ -24,6 +24,9 @@ You can use a Translator node in your pipeline to
 
 You can use the Translator component directly to translate your query or document(s): 
 ```python
+from haystack.schema import Document
+from haystack.translator import TransformersTranslator
+
 DOCS = [
         Document(
             text="""Heinz von Foerster was an Austrian American scientist 
diff --git a/docs/_src/usage/usage/tutorials.md b/docs/_src/usage/usage/tutorials.md
deleted file mode 100644
index fbff7f68c3..0000000000
--- a/docs/_src/usage/usage/tutorials.md
+++ /dev/null
@@ -1,17 +0,0 @@
-# Tutorials
-
-## 1) Start using Haystack to search through your own documents
-
-> Make this a button?
-
-> A link-  .. ref:: 1) Start using Haystack to search through your own documents
-
-## 2) Make Haystack understand your jargon
-
-## 3) Connect Haystack to your Datastore of choice
-
-## 4) Answer incoming questions using FAQ pages
-
-## 5) Benchmark the different components of Haystack
-
-## 6) SoTA: Powerup Haystack with DPR
diff --git a/haystack/translator/transformers.py b/haystack/translator/transformers.py
index de90a760c4..b068aa764a 100644
--- a/haystack/translator/transformers.py
+++ b/haystack/translator/transformers.py
@@ -73,6 +73,12 @@ def translate(
     ) -> Union[str, List[Document], List[str], List[Dict[str, Any]]]:
         """
         Run the actual translation. You can supply a query or a list of documents. Whatever is supplied will be translated.
+        :param query: The query string to translate
+        :type query: str
+        :param documents: The documents to translate
+        :type documents: List[Document] or List[Dict]
+        :param dict_key:
+        :type dict_key:
         """
         if not query and not documents:
             raise AttributeError("Translator need query or documents to perform translation")
diff --git a/tutorials/Tutorial9_DPR_training.ipynb b/tutorials/Tutorial9_DPR_training.ipynb
index 85bbd0c2fe..54691f6bd3 100644
--- a/tutorials/Tutorial9_DPR_training.ipynb
+++ b/tutorials/Tutorial9_DPR_training.ipynb
@@ -93,7 +93,8 @@
     "In the original DPR paper, these are fetched using a retriever to find the most relevant passages to the query.\n",
     "Passages which contain the answer text are filtered out.\n",
     "\n",
-    "We are [currently working](https://github.com/deepset-ai/haystack/issues/705) on a script that will convert SQuAD format data into a DPR dataset!"
+    "If you'd like to convert your SQuAD format data into something that can train a DPR model,\n",
+    "check out the utility script at [`haystack/retriever/squad_to_dpr.py`](https://github.com/deepset-ai/haystack/blob/master/haystack/retriever/squad_to_dpr.py)"
    ],
    "metadata": {
     "collapsed": false,
@@ -413,6 +414,15 @@
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython2",
    "version": "2.7.6"
+  },
+  "pycharm": {
+   "stem_cell": {
+    "cell_type": "raw",
+    "source": [],
+    "metadata": {
+     "collapsed": false
+    }
+   }
   }
  },
  "nbformat": 4,


Code review comments:
diff_hunk: @@ -0,0 +1,92 @@
+<a name="eval"></a>
+# Module eval
+
+<a name="eval.EvalRetriever"></a>
+## EvalRetriever Objects
+
+```python
+class EvalRetriever()
+```
+
+This is a pipeline node that should be placed after a Retriever in order to assess its performance. Performance
+metrics are stored in this class and updated as each sample passes through it. To view the results of the evaluation,
+call EvalRetriever.print()
+
+<a name="eval.EvalRetriever.__init__"></a>
+#### \_\_init\_\_
+
+```python
+ | __init__(debug=False, open_domain=True)
+```
+
+**Arguments**:
+
+- `open_domain`: When True, a document is considered correctly retrieved so long as the answer string can be found within it.
+                    When False, correct retrieval is evaluated based on document_id.
+:type open_domain: bool
path: docs/_src/api/api/evaluation.md
text: @user1:
This type is not rendered correctly on the website: 
![image](https://user-images.githubusercontent.com/1563902/115515480-76a7f180-a285-11eb-90ee-d9bbfd27816b.png)


@author:
Type annotations now in method signature rather than docstring api

diff_hunk: @@ -73,6 +73,12 @@ def translate(
     ) -> Union[str, List[Document], List[str], List[Dict[str, Any]]]:
         """
         Run the actual translation. You can supply a query or a list of documents. Whatever is supplied will be translated.
+        :param query: The query string to translate
+        :type query: str
path: haystack/translator/transformers.py
text: @user1:
Do we really want to keep the type annotations in the docstring? We have them now already in the method signatures and most other Haystack methods don't have them in the docstring anymore. Are there any advantages of having them for the "API reference" in the docs? 

@author:
Ok let's stick to type annotations in method signatures, not in docstrings

diff_hunk: @@ -0,0 +1,105 @@
+<a name="base"></a>
+# Module base
+
+<a name="base.BaseTranslator"></a>
+## BaseTranslator Objects
+
+```python
+class BaseTranslator(ABC)
+```
+
+Abstract class for a Translator component that translates either a query or a doc from language A to language B.
+
+<a name="base.BaseTranslator.translate"></a>
+#### translate
+
+```python
+ | @abstractmethod
+ | translate(query: Optional[str] = None, documents: Optional[Union[List[Document], List[str], List[Dict[str, Any]]]] = None, dict_key: Optional[str] = None, **kwargs) -> Union[str, List[Document], List[str], List[Dict[str, Any]]]
+```
+
+Translate the passed query or a list of documents from language A to B.
+
+<a name="base.BaseTranslator.run"></a>
+#### run
+
+```python
+ | run(query: Optional[str] = None, documents: Optional[Union[List[Document], List[str], List[Dict[str, Any]]]] = None, answers: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, dict_key: Optional[str] = None, **kwargs)
+```
+
+Method that gets executed when this class is used as a Node in a Haystack Pipeline
+
+<a name="transformers"></a>
+# Module transformers
+
+<a name="transformers.TransformersTranslator"></a>
+## TransformersTranslator Objects
+
+```python
+class TransformersTranslator(BaseTranslator)
+```
+
+Translator component based on Seq2Seq models from Huggingface's transformers library.
+Exemplary use cases:
+- Translate a query from Language A to B (e.g. if you only have good models + documents in language B)
+- Translate a document from Language A to B (e.g. if you want to return results in the native language of the user)
+
+We currently recommend using OPUS models (see __init__() for details)
+
+**Example:**
+
+```python
+|    DOCS = [
+|        Document(text="Heinz von Foerster was an Austrian American scientist combining physics and philosophy,
+|                       and widely attributed as the originator of Second-order cybernetics.")
+|    ]
+|    translator = TransformersTranslator(model_name_or_path="Helsinki-NLP/opus-mt-en-de")
+|    res = translator.translate(documents=DOCS, query=None)
+```
+
+<a name="transformers.TransformersTranslator.__init__"></a>
+#### \_\_init\_\_
+
+```python
+ | __init__(model_name_or_path: str, tokenizer_name: Optional[str] = None, max_seq_len: Optional[int] = None, clean_up_tokenization_spaces: Optional[bool] = True)
+```
+
+Initialize the translator with a model that fits your targeted languages. While we support all seq2seq
+models from Hugging Face's model hub, we recommend using the OPUS models from Helsiniki NLP. They provide plenty
+of different models, usually one model per language pair and translation direction.
+They have a pretty standardized naming that should help you find the right model:
+- "Helsinki-NLP/opus-mt-en-de" => translating from English to German
+- "Helsinki-NLP/opus-mt-de-en" => translating from German to English
+- "Helsinki-NLP/opus-mt-fr-en" => translating from French to English
+- "Helsinki-NLP/opus-mt-hi-en"=> translating from Hindi to English
+...
+
+They also have a few multilingual models that support multiple languages at once.
+
+**Arguments**:
+
+- `model_name_or_path`: Name of the seq2seq model that shall be used for translation.
+                           Can be a remote name from Huggingface's modelhub or a local path.
+- `tokenizer_name`: Optional tokenizer name. If not supplied, `model_name_or_path` will also be used for the
+                       tokenizer.
+- `max_seq_len`: The maximum sentence length the model accepts. (Optional)
+- `clean_up_tokenization_spaces`: Whether or not to clean up the tokenization spaces. (default True)
+
+<a name="transformers.TransformersTranslator.translate"></a>
+#### translate
+
+```python
+ | translate(query: Optional[str] = None, documents: Optional[Union[List[Document], List[str], List[Dict[str, Any]]]] = None, dict_key: Optional[str] = None, **kwargs) -> Union[str, List[Document], List[str], List[Dict[str, Any]]]
+```
+
+Run the actual translation. You can supply a query or a list of documents. Whatever is supplied will be translated.
+
+**Arguments**:
+
+- `query`: The query string to translate
+:type query: str
+- `documents`: The documents to translate
+:type documents: List[Document] or List[Dict]
+- `dict_key`: 
+:type dict_key:
path: docs/_src/api/api/translator.md
text: @user1:
wrongly rendered 
![image](https://user-images.githubusercontent.com/1563902/115516389-59275780-a286-11eb-83e5-05b6b425d7fb.png)


@author:
Sticking to type annotations in signature rather than docstrings

diff_hunk: @@ -36,9 +30,52 @@ p.draw(path="custom_pipe.png")
 ```
 ![image](https://user-images.githubusercontent.com/1563902/102451716-54813700-4039-11eb-881e-f3c01b47ca15.png)
 
+### YAML File Definitions
+
+For your convenience, there is also the option of defining and loading pipelines in YAML files.
+For example, you can define an save a simple Retriever Reader pipeline by saving the following to a file:
path: docs/_src/usage/usage/pipelines.md
text: @user1:
typo "an save"

diff_hunk: @@ -97,18 +94,25 @@ document_store.write_documents(dicts)
 
 When we talk about Documents in Haystack, we are referring specifically to the individual blocks of text that are being held in the DocumentStore.
 You might want to use all the text in one file as a Document, or split it into multiple Documents.
-This splitting can have a big impact on speed and performance.
+This splitting can have a big impact on speed and performance. See 
path: docs/_src/usage/usage/get_started.md
text: @user1:
Typo "See"