Repository: deepset-ai/haystack

Issue:
EmbeddingRetriever.retrieve_batch has a tracker-less batching loop
**Describe the bug**

There is a batching loop inside `EmbeddingRetriever.retrieve_batch` [here](https://github.com/deepset-ai/haystack/blob/378a3fd2e77a1ed66e2c3ab84f6088e6a8713004/haystack/nodes/retriever/dense.py#L1763) that doesn't have a tqdm tracker attached. As such, there isn't an informative progress bar for the whole process.

```python
for batch in self._get_batches(queries=queries, batch_size=batch_size):
    query_embs.extend(self.embed_queries(queries=batch))
```

As such, say with `batch_size=8`, for 24 queries, the output log received is something like:
```
Batches: 100% ██████████ 1/1 [00:00<00:00, 24.23it/s]
Batches: 100% ██████████ 1/1 [00:00<00:00, 25.09it/s]
Batches: 100% ██████████ 1/1 [00:00<00:00, 28.02it/s]
```

**Error message**
Not an error. Just a case of quality-of-life improvement.

**Expected behavior**
A more informative progress tracker. 

So for the `batch_size=8` and 24 queries case, it would have a single progress tracker in terms of the 3 batches instead of 1 tracker for every batch.

```
Batches: 100% ███░░░░░░  1/3 [...]
```

**Additional context**

The relevant call stack is as:

`EmbeddingRetriever.retrieve_batch`:
- loops over batches and calls `EmbeddingRetriever.embed_queries`
    - calls `_SentenceTransformersEmbeddingEncoder.embed_queries`
        - calls `_SentenceTransformersEmbeddingEncoder.embed`
            - calls `SentenceTransformer.encode` which has tqdm inside with batch_size=self.batch_size=retriever.batch_size

Because the batching is also done further down the call stack, **a possibly better alternative** to adding a tqdm tracker on the loop in question could be to remove the loop and just call `embed_queries` directly.

```python
query_embs = self.embed_queries(queries=queries)
```

**To Reproduce**
[Colab notebook](https://colab.research.google.com/drive/1os-VuHTpAoCJ6Bt179kuKdeqT54dya-e?usp=sharing)

Relevant code:
```python
retriever = EmbeddingRetriever(
    batch_size=8,
    document_store=document_store,
    embedding_model="sentence-transformers/multi-qa-mpnet-base-dot-v1",
    model_format="sentence_transformers",
)

document_store.update_embeddings(retriever)

# Produces 3 batches of 8 samples each
# Creates 3 different 1/1 progress trackers instead of a single x/3 one
retriever.retrieve_batch(queries=['sample query?']*24)
```

**FAQ Check**
- [x] Have you had a look at [our new FAQ page](https://docs.haystack.deepset.ai/docs/faq)?

**System:**
 - OS: 
 - GPU/CPU:
 - Haystack version (commit or version number): 1.12.2
 - DocumentStore:
 - Reader:
 - Retriever: EmbeddingRetriever



Discussion:

Resolving patch:
diff --git a/haystack/nodes/retriever/dense.py b/haystack/nodes/retriever/dense.py
--- a/haystack/nodes/retriever/dense.py
+++ b/haystack/nodes/retriever/dense.py
@@ -1075,11 +1075,18 @@ def retrieve_batch(
         if scale_score is None:
             scale_score = self.scale_score
 
-        query_embs: List[np.ndarray] = []
-        for batch in self._get_batches(queries=queries, batch_size=batch_size):
-            query_embs.extend(self.embed_queries(queries=batch))
+        # embed_queries is already batched within by batch_size, so no need to batch the input here
+        query_embs: np.ndarray = self.embed_queries(queries=queries)
+        batched_query_embs: List[np.ndarray] = []
+        for i in range(0, len(query_embs), batch_size):
+            batched_query_embs.extend(query_embs[i : i + batch_size])
         documents = document_store.query_by_embedding_batch(
-            query_embs=query_embs, top_k=top_k, filters=filters, index=index, headers=headers, scale_score=scale_score
+            query_embs=batched_query_embs,
+            top_k=top_k,
+            filters=filters,
+            index=index,
+            headers=headers,
+            scale_score=scale_score,
         )
 
         return documents
@@ -1769,11 +1776,18 @@ def retrieve_batch(
         if scale_score is None:
             scale_score = self.scale_score
 
-        query_embs: List[np.ndarray] = []
-        for batch in self._get_batches(queries=queries, batch_size=batch_size):
-            query_embs.extend(self.embed_queries(queries=batch))
+        # embed_queries is already batched within by batch_size, so no need to batch the input here
+        query_embs: np.ndarray = self.embed_queries(queries=queries)
+        batched_query_embs: List[np.ndarray] = []
+        for i in range(0, len(query_embs), batch_size):
+            batched_query_embs.extend(query_embs[i : i + batch_size])
         documents = document_store.query_by_embedding_batch(
-            query_embs=query_embs, top_k=top_k, filters=filters, index=index, headers=headers, scale_score=scale_score
+            query_embs=batched_query_embs,
+            top_k=top_k,
+            filters=filters,
+            index=index,
+            headers=headers,
+            scale_score=scale_score,
         )
 
         return documents


Test patch:
diff --git a/test/nodes/test_retriever.py b/test/nodes/test_retriever.py
--- a/test/nodes/test_retriever.py
+++ b/test/nodes/test_retriever.py
@@ -172,6 +172,7 @@ def test_retrieval_empty_query(document_store: BaseDocumentStore):
     assert result[0]["documents"][0][0] == mock_document
 
 
+@pytest.mark.parametrize("retriever_with_docs", ["embedding", "dpr", "tfidf"], indirect=True)
 def test_batch_retrieval_single_query(retriever_with_docs, document_store_with_docs):
     if not isinstance(retriever_with_docs, (BM25Retriever, FilterRetriever, TfidfRetriever)):
         document_store_with_docs.update_embeddings(retriever_with_docs)
@@ -189,6 +190,7 @@ def test_batch_retrieval_single_query(retriever_with_docs, document_store_with_d
     assert res[0][0].meta["name"] == "filename1"
 
 
+@pytest.mark.parametrize("retriever_with_docs", ["embedding", "dpr", "tfidf"], indirect=True)
 def test_batch_retrieval_multiple_queries(retriever_with_docs, document_store_with_docs):
     if not isinstance(retriever_with_docs, (BM25Retriever, FilterRetriever, TfidfRetriever)):
         document_store_with_docs.update_embeddings(retriever_with_docs)