Repository: deepset-ai/haystack

Issue:
LFQA: Remove InferenceProcessor dependency from _RetribertEmbeddingEncoder
As part of LFQA, `_RetribertEmbeddingEncoder` uses FARM's `InferenceProcessor`: https://github.com/deepset-ai/haystack/blob/1cd17022af30744e24d3b6af4c0dc346d61813a7/haystack/retriever/dense.py#L699

After migrating FARM to haystack in #1492 and getting rid of haystack's FARM dependency, there is no other use of `InferenceProcessor` in haystack and replacing this single use of it with transformers code instead would allow to remove `InferenceProcessor` and `TextClassificationProcessor` from haystack's code base, which would result in a nice simplification of the code base.

It seems that `_RetribertEmbeddingEncoder` uses `InferenceProcessor` only in `_create_dataloader`. I think, we need a new implementation of that method that makes use of the `embedding_tokenizer` but not the `InferenceProcessor`.

Maybe @vblagoje can provide some help here as he contributed LFQA to haystack? 🙂 


Discussion:
Apologies @julian-risch , atm I can't dedicate any time to this issue. I could in a week from now if that is not too late. Please LMK
> Apologies @julian-risch , atm I can't dedicate any time to this issue. I could in a week from now if that is not too late. Please LMK

Thank you for the quick reply @vblagoje . It's not urgent. Just let us know here in a week if you will have a look at it. 👍 
Julian, to minimize the time cost of the context switch of jumping back into the Haystack codebase, what do you recommend as a replacement for the InferenceProcessor? 
As the InferenceProcessor is only used in `create_dataloader`, I think, using the `embedding_tokenizer` directly is a good way forward.

Resolving patch:
diff --git a/haystack/retriever/dense.py b/haystack/retriever/dense.py
--- a/haystack/retriever/dense.py
+++ b/haystack/retriever/dense.py
@@ -6,6 +6,7 @@
 import numpy as np
 from pathlib import Path
 
+from haystack.modeling.data_handler.dataset import convert_features_to_dataset
 from haystack.modeling.utils import initialize_device_settings
 from tqdm.auto import tqdm
 from transformers import AutoTokenizer, AutoModel
@@ -19,7 +20,7 @@
 from haystack.modeling.model.language_model import LanguageModel
 from haystack.modeling.model.biadaptive_model import BiAdaptiveModel
 from haystack.modeling.model.prediction_head import TextSimilarityHead
-from haystack.modeling.data_handler.processor import TextSimilarityProcessor, InferenceProcessor
+from haystack.modeling.data_handler.processor import TextSimilarityProcessor
 from haystack.modeling.data_handler.data_silo import DataSilo
 from haystack.modeling.data_handler.dataloader import NamedDataLoader
 from haystack.modeling.model.optimization import initialize_optimizer
@@ -693,11 +694,8 @@ def __init__(
         else:
             self.device = torch.device("cpu")
 
-        embedding_tokenizer = AutoTokenizer.from_pretrained(retriever.embedding_model,
-                                                            use_fast_tokenizers=True)
+        self.embedding_tokenizer = AutoTokenizer.from_pretrained(retriever.embedding_model)
         self.embedding_model = AutoModel.from_pretrained(retriever.embedding_model).to(self.device)
-        self.processor = InferenceProcessor(tokenizer=embedding_tokenizer,
-                                            max_seq_len=embedding_tokenizer.max_len_single_sentence)
 
     def embed_queries(self, texts: List[str]) -> List[np.ndarray]:
 
@@ -735,8 +733,29 @@ def embed_passages(self, docs: List[Document]) -> List[np.ndarray]:
 
     def _create_dataloader(self, text_to_encode: List[dict]) -> NamedDataLoader:
 
-        dataset, tensor_names, _ = self.processor.dataset_from_dicts(text_to_encode,
-                                                                     indices=[i for i in range(len(text_to_encode))])
+        dataset, tensor_names = self.dataset_from_dicts(text_to_encode)
         dataloader = NamedDataLoader(dataset=dataset, sampler=SequentialSampler(dataset),
                                      batch_size=32, tensor_names=tensor_names)
         return dataloader
+
+    def dataset_from_dicts(self, dicts: List[dict]):
+        texts = [x["text"] for x in dicts]
+        tokenized_batch = self.embedding_tokenizer(
+            texts,
+            return_token_type_ids=True,
+            return_attention_mask=True,
+            truncation=True,
+            padding=True
+        )
+
+        features_flat = []
+        for input_ids, segment_ids, padding_mask in zip(
+                tokenized_batch["input_ids"], tokenized_batch["token_type_ids"], tokenized_batch["attention_mask"]
+        ):
+            feat_dict = {"input_ids": input_ids,
+                         "padding_mask": padding_mask,
+                         "segment_ids": segment_ids}
+            features_flat.append(feat_dict)
+
+        dataset, tensornames = convert_features_to_dataset(features=features_flat)
+        return dataset, tensornames


Test patch: