Repository: deepset-ai/haystack

Issue:
Check document_store and embedding_model dimensions before calculating embeddings
**Is your feature request related to a problem? Please describe.**
When running document_store.update_embeddings(retriever=embedding_retriever), embeddings will first be calculated then saved to the doc store.

However, if the embedding dimensions differ, you get an error and you have now calculated the embeddings wastefully. E.g.

RuntimeError: Embedding dimensions of the model (1024) don't match the embedding dimensions of the document store (768). Initiate FAISSDocumentStore again with arg embedding_dim=1024.

This is more of a problem when paying (e.g. open_ai).

**Describe the solution you'd like**
A check against both document_store and embedding dimensions before running the embedding calculations.


Discussion:
@AlexGWOmron Thank you for this suggestion. I agree that it makes a lot of sense to check document_store and embedding dimensions before running the embedding calculations. Would you maybe like to contribute this feature and open a PR? We can give early feedback if you make it a draft PR. Guidelines are [here](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md) 🙂 
@julian-risch I would like to work on this issue. What would be the preferred implementation that I should follow?
@awinml That's great to hear! You can find our general contributor guidelines here: https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md
For this particular issue, I'd say in the beginning of the update_embeddings method, their should be a check for whether the embedding dimensions set in the document store and the embedding dimensions of the model used by the retriever for the embeddings are the same. This check could be implemented in a separate function that just get's a document store and a retriever as input maybe? In case we can't infer the embedding dimensions from the model, the alternative would be to calculate the embeddings of a first document and then check whether it's the same as the setting of the document store. That's still better than first calculating the. embeddings for dozens of documents and then noticing that they can't be stored.
If you open a draft PR, we can give early feedback! No need to wait until you have a solution ready. Before the PR can be merged, we will need to have some unit tests but we can guide you there. Looking forward to your PR! 🙂 
@julian-risch Thank you for the detailed explanation. I understand the overall implementation suggestion and will open a draft PR shortly. I'll make sure to follow the contributor guidelines and include unit tests as well.
Hi @julian-risch! I'd like to work on this if it's still open.
Hi @AnushreeBannadabhavi, I am working on other issues at the moment, feel free to take it up.

Resolving patch:
diff --git a/haystack/document_stores/faiss.py b/haystack/document_stores/faiss.py
--- a/haystack/document_stores/faiss.py
+++ b/haystack/document_stores/faiss.py
@@ -324,6 +324,18 @@ def write_documents(
     def _create_document_field_map(self) -> Dict:
         return {self.index: self.embedding_field}
 
+    def _validate_embedding_dimension(self, retriever: DenseRetriever, index: Optional[str] = None):
+        """
+        Verify if the embedding dimension set in the document store and embedding dimension of the retriever are the same.
+        This check is done before calculating embeddings for all documents.
+        :param retriever: Retriever to use to get embeddings for text
+        :param index: Index name for which embeddings are to be updated. If set to None, the default self.index is used.
+        :return: None
+        """
+        first_document = self.get_all_documents(index=index)[0]
+        embeddings = retriever.embed_documents([first_document])
+        self._validate_embeddings_shape(embeddings=embeddings, num_documents=1, embedding_dim=self.embedding_dim)
+
     def update_embeddings(
         self,
         retriever: DenseRetriever,
@@ -373,6 +385,8 @@ def update_embeddings(
             logger.warning("Calling DocumentStore.update_embeddings() on an empty index")
             return
 
+        self._validate_embedding_dimension(retriever, index)
+
         logger.info("Updating embeddings for %s docs...", document_count)
         vector_id = self.faiss_indexes[index].ntotal
 


Test patch:
diff --git a/test/document_stores/test_faiss.py b/test/document_stores/test_faiss.py
--- a/test/document_stores/test_faiss.py
+++ b/test/document_stores/test_faiss.py
@@ -42,6 +42,21 @@ def test_index_mutual_exclusive_args(self, tmp_path):
                 isolation_level="AUTOCOMMIT",
             )
 
+    @pytest.mark.unit
+    def test_validate_embedding_dimension_unequal_embedding_dim(self, ds, documents):
+        retriever = MockDenseRetriever(document_store=ds, embedding_dim=384)
+        ds.write_documents(documents)
+        assert ds.get_document_count() == len(documents)
+        with pytest.raises(RuntimeError):
+            ds._validate_embedding_dimension(retriever)
+
+    @pytest.mark.unit
+    def test_validate_embedding_dimension_equal_embedding_dim(self, ds, documents):
+        retriever = MockDenseRetriever(document_store=ds, embedding_dim=768)
+        ds.write_documents(documents)
+        assert ds.get_document_count() == len(documents)
+        ds._validate_embedding_dimension(retriever)
+
     @pytest.mark.integration
     def test_delete_index(self, ds, documents):
         """Contrary to other Document Stores, FAISSDocumentStore doesn't raise if the index is empty"""