Repository: deepset-ai/haystack
Pull request: Delete documents by ID in all document stores
Related to #1532, #1383,  #880 and #1269

Allows `delete_documents` to accept a list of IDs to delete in all document stores.


Problem statement:
Allow document stores to delete documents by ID
Currently `delete_documents()` accepts filters to narrow down the amount of documents to delete, but does not allow targeting IDs - only `meta` fields.

We should modify `delete_documents` to be able to accept a list (or a range) of IDs to delete, in all document stores.

Hints from issue discussion:
Just a comment: There is this PR #1383 with some code that maybe could be reused.
Related https://github.com/deepset-ai/haystack/pull/880 and #1269

Commit message: Add latest docstring and tutorial changes

Patch to review:
diff --git a/docs/_src/api/api/document_store.md b/docs/_src/api/api/document_store.md
index 68bfe2f8fd..21061c2cc0 100644
--- a/docs/_src/api/api/document_store.md
+++ b/docs/_src/api/api/document_store.md
@@ -456,7 +456,7 @@ None
 #### delete\_documents
 
 ```python
- | delete_documents(index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None)
+ | delete_documents(index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None)
 ```
 
 Delete documents in an index. All documents are deleted if no filters are passed.
@@ -464,8 +464,12 @@ Delete documents in an index. All documents are deleted if no filters are passed
 **Arguments**:
 
 - `index`: Index name to delete the document from.
+- `ids`: Optional list of IDs to narrow down the documents to be deleted.
 - `filters`: Optional filters to narrow down the documents to be deleted.
-    Example filters: {"name": ["some", "more"], "category": ["only_one"]}
+    Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+    If filters are provided along with a list of IDs, this method deletes the
+    intersection of the two query results (documents that match the filters and
+    have their ID in the list).
 
 **Returns**:
 
@@ -737,7 +741,7 @@ None
 #### delete\_documents
 
 ```python
- | delete_documents(index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None)
+ | delete_documents(index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None)
 ```
 
 Delete documents in an index. All documents are deleted if no filters are passed.
@@ -746,8 +750,12 @@ Delete documents in an index. All documents are deleted if no filters are passed
 
 - `index`: Index name to delete the document from. If None, the
               DocumentStore's default index (self.index) will be used.
+- `ids`: Optional list of IDs to narrow down the documents to be deleted.
 - `filters`: Optional filters to narrow down the documents to be deleted.
-                Example filters: {"name": ["some", "more"], "category": ["only_one"]}
+    Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+    If filters are provided along with a list of IDs, this method deletes the
+    intersection of the two query results (documents that match the filters and
+    have their ID in the list).
 
 **Returns**:
 
@@ -954,7 +962,7 @@ None
 #### delete\_documents
 
 ```python
- | delete_documents(index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None)
+ | delete_documents(index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None)
 ```
 
 Delete documents in an index. All documents are deleted if no filters are passed.
@@ -963,8 +971,12 @@ Delete documents in an index. All documents are deleted if no filters are passed
 
 - `index`: Index name to delete the document from. If None, the
               DocumentStore's default index (self.index) will be used.
+- `ids`: Optional list of IDs to narrow down the documents to be deleted.
 - `filters`: Optional filters to narrow down the documents to be deleted.
-                Example filters: {"name": ["some", "more"], "category": ["only_one"]}
+    Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+    If filters are provided along with a list of IDs, this method deletes the
+    intersection of the two query results (documents that match the filters and
+    have their ID in the list).
 
 **Returns**:
 
@@ -1153,7 +1165,7 @@ Delete all documents from the document store.
 #### delete\_documents
 
 ```python
- | delete_documents(index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None)
+ | delete_documents(index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None)
 ```
 
 Delete documents from the document store. All documents are deleted if no filters are passed.
@@ -1162,8 +1174,12 @@ Delete documents from the document store. All documents are deleted if no filter
 
 - `index`: Index name to delete the document from. If None, the
               DocumentStore's default index (self.index) will be used.
+- `ids`: Optional list of IDs to narrow down the documents to be deleted.
 - `filters`: Optional filters to narrow down the documents to be deleted.
-                Example filters: {"name": ["some", "more"], "category": ["only_one"]}
+    Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+    If filters are provided along with a list of IDs, this method deletes the
+    intersection of the two query results (documents that match the filters and
+    have their ID in the list).
 
 **Returns**:
 
@@ -1418,7 +1434,7 @@ None
 #### delete\_documents
 
 ```python
- | delete_documents(index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None)
+ | delete_documents(index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None)
 ```
 
 Delete documents in an index. All documents are deleted if no filters are passed.
@@ -1427,8 +1443,12 @@ Delete documents in an index. All documents are deleted if no filters are passed
 
 - `index`: Index name to delete the document from. If None, the
               DocumentStore's default index (self.index) will be used.
-- `filters`: Optional filters to narrow down the search space.
-                Example: {"name": ["some", "more"], "category": ["only_one"]}
+- `ids`: Optional list of IDs to narrow down the documents to be deleted.
+- `filters`: Optional filters to narrow down the documents to be deleted.
+    Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+    If filters are provided along with a list of IDs, this method deletes the
+    intersection of the two query results (documents that match the filters and
+    have their ID in the list).
 
 **Returns**:
 
@@ -1787,7 +1807,7 @@ None
 #### delete\_documents
 
 ```python
- | delete_documents(index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None)
+ | delete_documents(index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None)
 ```
 
 Delete documents in an index. All documents are deleted if no filters are passed.
@@ -1796,8 +1816,12 @@ Delete documents in an index. All documents are deleted if no filters are passed
 
 - `index`: Index name to delete the document from. If None, the
               DocumentStore's default index (self.index) will be used.
+- `ids`: Optional list of IDs to narrow down the documents to be deleted.
 - `filters`: Optional filters to narrow down the documents to be deleted.
-                Example filters: {"name": ["some", "more"], "category": ["only_one"]}
+    Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+    If filters are provided along with a list of IDs, this method deletes the
+    intersection of the two query results (documents that match the filters and
+    have their ID in the list).
 
 **Returns**:
 
diff --git a/haystack/document_store/base.py b/haystack/document_store/base.py
index bb6133d887..45be0ae31a 100644
--- a/haystack/document_store/base.py
+++ b/haystack/document_store/base.py
@@ -245,7 +245,7 @@ def delete_all_documents(self, index: Optional[str] = None, filters: Optional[Di
         pass
 
     @abstractmethod
-    def delete_documents(self, index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None):
+    def delete_documents(self, index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None):
         pass
 
     def run(self, documents: List[dict], index: Optional[str] = None):  # type: ignore
diff --git a/haystack/document_store/elasticsearch.py b/haystack/document_store/elasticsearch.py
index 03f355d82c..af40587481 100644
--- a/haystack/document_store/elasticsearch.py
+++ b/haystack/document_store/elasticsearch.py
@@ -991,15 +991,19 @@ def delete_all_documents(self, index: Optional[str] = None, filters: Optional[Di
                 For more details, please refer to the issue: https://github.com/deepset-ai/haystack/issues/1045
                 """
         )
-        self.delete_documents(index, filters)
+        self.delete_documents(index, None, filters)
 
-    def delete_documents(self, index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None):
+    def delete_documents(self, index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None):
         """
         Delete documents in an index. All documents are deleted if no filters are passed.
 
         :param index: Index name to delete the document from.
+        :param ids: Optional list of IDs to narrow down the documents to be deleted.
         :param filters: Optional filters to narrow down the documents to be deleted.
-            Example filters: {"name": ["some", "more"], "category": ["only_one"]}
+            Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+            If filters are provided along with a list of IDs, this method deletes the
+            intersection of the two query results (documents that match the filters and
+            have their ID in the list).
         :return: None
         """
         index = index or self.index
@@ -1013,6 +1017,13 @@ def delete_documents(self, index: Optional[str] = None, filters: Optional[Dict[s
                         }
                 )
                 query["query"]["bool"] = {"filter": filter_clause}
+            
+            if ids:
+                query["query"]["bool"]["must"] = {"ids": {"values": ids}}
+
+        elif ids:
+            if ids:
+                query["query"]["ids"] = {"values": ids}
         else:
             query["query"] = {"match_all": {}}
         self.client.delete_by_query(index=index, body=query, ignore=[404])
diff --git a/haystack/document_store/faiss.py b/haystack/document_store/faiss.py
index 9ded4f281c..c498a080ef 100644
--- a/haystack/document_store/faiss.py
+++ b/haystack/document_store/faiss.py
@@ -397,27 +397,34 @@ def delete_all_documents(self, index: Optional[str] = None, filters: Optional[Di
                 For more details, please refer to the issue: https://github.com/deepset-ai/haystack/issues/1045
                 """
         )
-        self.delete_documents(index, filters)
+        self.delete_documents(index, None, filters)
 
-    def delete_documents(self, index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None):
+    def delete_documents(self, index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None):
         """
         Delete documents from the document store. All documents are deleted if no filters are passed.
 
         :param index: Index name to delete the document from. If None, the
                       DocumentStore's default index (self.index) will be used.
-        :param filters: Optional filters to narrow down the documents to be deleted. 
-                        Example filters: {"name": ["some", "more"], "category": ["only_one"]}
+        :param ids: Optional list of IDs to narrow down the documents to be deleted.
+        :param filters: Optional filters to narrow down the documents to be deleted.
+            Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+            If filters are provided along with a list of IDs, this method deletes the
+            intersection of the two query results (documents that match the filters and
+            have their ID in the list).
         :return: None
         """
         index = index or self.index
         if index in self.faiss_indexes.keys():
-            if filters:
+            if not filters and not ids:
+                self.faiss_indexes[index].reset()
+            else:
                 affected_docs = self.get_all_documents(filters=filters)
+                if ids:
+                    affected_docs = [doc for doc in affected_docs if doc.id in ids]
                 doc_ids = [doc.meta.get("vector_id") for doc in affected_docs if doc.meta and doc.meta.get("vector_id") is not None]
                 self.faiss_indexes[index].remove_ids(np.array(doc_ids, dtype="int64"))
-            else:
-                self.faiss_indexes[index].reset()
-        super().delete_documents(index=index, filters=filters)
+
+        super().delete_documents(index=index, ids=ids, filters=filters)
 
     def query_by_embedding(
         self,
diff --git a/haystack/document_store/memory.py b/haystack/document_store/memory.py
index b51ff59e0e..a08466b672 100644
--- a/haystack/document_store/memory.py
+++ b/haystack/document_store/memory.py
@@ -389,21 +389,29 @@ def delete_all_documents(self, index: Optional[str] = None, filters: Optional[Di
                 For more details, please refer to the issue: https://github.com/deepset-ai/haystack/issues/1045
                 """
         )
-        self.delete_documents(index, filters)
+        self.delete_documents(index, None, filters)
 
-    def delete_documents(self, index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None):
+    def delete_documents(self, index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None):
         """
         Delete documents in an index. All documents are deleted if no filters are passed.
 
         :param index: Index name to delete the document from. If None, the
                       DocumentStore's default index (self.index) will be used.
+        :param ids: Optional list of IDs to narrow down the documents to be deleted.
         :param filters: Optional filters to narrow down the documents to be deleted.
-                        Example filters: {"name": ["some", "more"], "category": ["only_one"]}
+            Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+            If filters are provided along with a list of IDs, this method deletes the
+            intersection of the two query results (documents that match the filters and
+            have their ID in the list).
+
         :return: None
         """
         index = index or self.index
-        if not filters:
+        if not filters and not ids:
             self.indexes[index] = {}
-            return            
-        for doc in self.get_all_documents(filters=filters):
+            return
+        docs_to_delete = self.get_all_documents(filters=filters)
+        if ids:
+            docs_to_delete = [doc for doc in docs_to_delete if doc.id in ids]
+        for doc in docs_to_delete:
             del self.indexes[index][doc.id]
diff --git a/haystack/document_store/milvus.py b/haystack/document_store/milvus.py
index f5137051f6..9ff0ce75d6 100644
--- a/haystack/document_store/milvus.py
+++ b/haystack/document_store/milvus.py
@@ -389,16 +389,20 @@ def delete_all_documents(self, index: Optional[str] = None, filters: Optional[Di
                 For more details, please refer to the issue: https://github.com/deepset-ai/haystack/issues/1045
                 """
         )
-        self.delete_documents(index, filters)
+        self.delete_documents(index, None, filters)
 
-    def delete_documents(self, index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None):
+    def delete_documents(self, index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None):
         """
         Delete documents in an index. All documents are deleted if no filters are passed.
 
         :param index: Index name to delete the document from. If None, the
                       DocumentStore's default index (self.index) will be used.
-        :param filters: Optional filters to narrow down the search space.
-                        Example: {"name": ["some", "more"], "category": ["only_one"]}
+        :param ids: Optional list of IDs to narrow down the documents to be deleted.
+        :param filters: Optional filters to narrow down the documents to be deleted.
+            Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+            If filters are provided along with a list of IDs, this method deletes the
+            intersection of the two query results (documents that match the filters and
+            have their ID in the list).
         :return: None
         """
         index = index or self.index
@@ -406,19 +410,21 @@ def delete_documents(self, index: Optional[str] = None, filters: Optional[Dict[s
         if status.code != Status.SUCCESS:
             raise RuntimeError(f'Milvus has collection check failed: {status}')
         if ok:
-            if filters:
-                existing_docs = super().get_all_documents(filters=filters, index=index)
-                self._delete_vector_ids_from_milvus(documents=existing_docs, index=index)
-            else:
+            if not filters and not ids:
                 status = self.milvus_server.drop_collection(collection_name=index)
                 if status.code != Status.SUCCESS:
                     raise RuntimeError(f'Milvus drop collection failed: {status}')
+            else:
+                affected_docs = super().get_all_documents(filters=filters, index=index)
+                if ids:
+                    affected_docs = [doc for doc in affected_docs if doc.id in ids]
+                self._delete_vector_ids_from_milvus(documents=affected_docs, index=index)
 
             self.milvus_server.flush([index])
             self.milvus_server.compact(collection_name=index)
 
         # Delete from SQL at the end to allow the above .get_all_documents() to work properly
-        super().delete_documents(index=index, filters=filters)
+        super().delete_documents(index=index, ids=ids, filters=filters)
 
     def get_all_documents_generator(
         self,
diff --git a/haystack/document_store/sql.py b/haystack/document_store/sql.py
index b5c7a9ef89..4f92789788 100644
--- a/haystack/document_store/sql.py
+++ b/haystack/document_store/sql.py
@@ -514,33 +514,37 @@ def delete_all_documents(self, index: Optional[str] = None, filters: Optional[Di
                 For more details, please refer to the issue: https://github.com/deepset-ai/haystack/issues/1045
                 """
         )
-        self.delete_documents(index, filters)
+        self.delete_documents(index, None, filters)
 
-    def delete_documents(self, index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None):
+    def delete_documents(self, index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None):
         """
         Delete documents in an index. All documents are deleted if no filters are passed.
 
         :param index: Index name to delete the document from. If None, the
                       DocumentStore's default index (self.index) will be used.
+        :param ids: Optional list of IDs to narrow down the documents to be deleted.
         :param filters: Optional filters to narrow down the documents to be deleted.
-                        Example filters: {"name": ["some", "more"], "category": ["only_one"]}
+            Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+            If filters are provided along with a list of IDs, this method deletes the
+            intersection of the two query results (documents that match the filters and
+            have their ID in the list).
         :return: None
         """
         index = index or self.index
-
+        # documents_query = documents_query.join(MetaORM)
+        document_ids_to_delete = self.session.query(DocumentORM.id).filter(DocumentORM.index==index)
         if filters:
-            # documents_query = documents_query.join(MetaORM)
-            document_ids_to_delete = self.session.query(DocumentORM.id).filter_by(index=index)
             for key, values in filters.items():
                 document_ids_to_delete = document_ids_to_delete.filter(
                     MetaDocumentORM.name == key,
-                        MetaDocumentORM.value.in_(values),
+                    MetaDocumentORM.value.in_(values),
                     DocumentORM.id == MetaDocumentORM.document_id
                 )
-            self.session.query(DocumentORM).filter(DocumentORM.id.in_(document_ids_to_delete)).delete(
-                    synchronize_session=False)
-        else:
-            self.session.query(DocumentORM).filter_by(index=index).delete(synchronize_session=False)
+        if ids:
+            document_ids_to_delete = document_ids_to_delete.filter(DocumentORM.id.in_(ids))
+
+        self.session.query(DocumentORM).filter(DocumentORM.id.in_(document_ids_to_delete)).delete(
+                synchronize_session=False)
 
         self.session.commit()
 
diff --git a/haystack/document_store/weaviate.py b/haystack/document_store/weaviate.py
index 07ce846869..92d152da23 100644
--- a/haystack/document_store/weaviate.py
+++ b/haystack/document_store/weaviate.py
@@ -681,26 +681,33 @@ def delete_all_documents(self, index: Optional[str] = None, filters: Optional[Di
                 For more details, please refer to the issue: https://github.com/deepset-ai/haystack/issues/1045
                 """
         )
-        self.delete_documents(index, filters)
+        self.delete_documents(index, None, filters)
 
-    def delete_documents(self, index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None):
+    def delete_documents(self, index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None):
         """
         Delete documents in an index. All documents are deleted if no filters are passed.
 
         :param index: Index name to delete the document from. If None, the
                       DocumentStore's default index (self.index) will be used.
+        :param ids: Optional list of IDs to narrow down the documents to be deleted.
         :param filters: Optional filters to narrow down the documents to be deleted.
-                        Example filters: {"name": ["some", "more"], "category": ["only_one"]}
+            Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+            If filters are provided along with a list of IDs, this method deletes the
+            intersection of the two query results (documents that match the filters and
+            have their ID in the list).
         :return: None
         """
         index = index or self.index
-        if filters:
+        if not filters and not ids:
+            self.weaviate_client.schema.delete_class(index)
+            self._create_schema_and_index_if_not_exist(index)
+        else:
             docs_to_delete = self.get_all_documents(index, filters=filters)
+            if ids:
+                docs_to_delete = [doc for doc in docs_to_delete if doc.id in ids]
             for doc in docs_to_delete:
                 self.weaviate_client.data_object.delete(doc.id)
-        else:
-            self.weaviate_client.schema.delete_class(index)
-            self._create_schema_and_index_if_not_exist(index)
+            
 
 
 
diff --git a/test/test_document_store.py b/test/test_document_store.py
index 650af7390f..e29b20518b 100644
--- a/test/test_document_store.py
+++ b/test/test_document_store.py
@@ -345,6 +345,32 @@ def test_delete_documents(document_store_with_docs):
     assert len(documents) == 0
 
     
+def test_delete_documents_by_id(document_store_with_docs):
+    doc_ids = [doc.id for doc in document_store_with_docs.get_all_documents()]
+    assert len(doc_ids) == 3
+    docs_to_delete = doc_ids[0:2]
+
+    document_store_with_docs.delete_documents(ids=docs_to_delete)
+
+    documents = document_store_with_docs.get_all_documents()
+    assert len(documents) == 1
+    assert documents[0].id == doc_ids[2]
+
+
+def test_delete_documents_by_id_with_filters(document_store_with_docs):
+    docs_to_delete = document_store_with_docs.get_all_documents(filters={"meta_field": ["test1", "test2"]})
+    docs_not_to_delete = document_store_with_docs.get_all_documents(filters={"meta_field": ["test3"]})
+
+    document_store_with_docs.delete_documents(ids=[doc.id for doc in docs_to_delete], filters={"meta_field": ["test1"]})
+
+    all_docs_left = document_store_with_docs.get_all_documents()
+    assert len(all_docs_left) == 2
+    assert all(doc.meta["meta_field"] != "test1" for doc in all_docs_left)
+
+    all_ids_left = [doc.id for doc in all_docs_left]
+    assert all(doc.id in all_ids_left for doc in docs_not_to_delete)
+
+    
 def test_delete_documents_with_filters(document_store_with_docs):
     document_store_with_docs.delete_documents(filters={"meta_field": ["test1", "test2"]})
     documents = document_store_with_docs.get_all_documents()
diff --git a/test/test_faiss_and_milvus.py b/test/test_faiss_and_milvus.py
index daf3ca2834..a8e6b8af8d 100644
--- a/test/test_faiss_and_milvus.py
+++ b/test/test_faiss_and_milvus.py
@@ -210,6 +210,51 @@ def test_delete_docs_with_filters(document_store, retriever):
     assert {doc.meta["name"] for doc in documents} == {"name_5", "name_6"}
 
 
+@pytest.mark.slow
+@pytest.mark.parametrize("retriever", ["dpr"], indirect=True)
+@pytest.mark.parametrize("document_store", ["faiss", "milvus"], indirect=True)
+def test_delete_docs_by_id(document_store, retriever):
+    document_store.write_documents(DOCUMENTS)
+    document_store.update_embeddings(retriever=retriever, batch_size=4)
+    assert document_store.get_embedding_count() == 6
+    doc_ids = [doc.id for doc in document_store.get_all_documents()]
+    ids_to_delete = doc_ids[0:3]
+
+    document_store.delete_documents(ids=ids_to_delete)
+
+    documents = document_store.get_all_documents()
+    assert len(documents) == len(doc_ids) - len(ids_to_delete)
+    assert document_store.get_embedding_count() == len(doc_ids) - len(ids_to_delete)
+
+    remaining_ids = [doc.id for doc in documents]
+    assert all(doc_id not in remaining_ids for doc_id in ids_to_delete)
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize("retriever", ["dpr"], indirect=True)
+@pytest.mark.parametrize("document_store", ["faiss", "milvus"], indirect=True)
+def test_delete_docs_by_id_with_filters(document_store, retriever):
+    document_store.write_documents(DOCUMENTS)
+    document_store.update_embeddings(retriever=retriever, batch_size=4)
+    assert document_store.get_embedding_count() == 6
+
+    ids_to_delete = [doc.id for doc in document_store.get_all_documents(filters={"name": ["name_1", "name_2"]})]
+    ids_not_to_delete = [doc.id for doc in document_store.get_all_documents(filters={"name": ["name_3", "name_4", "name_5", "name_6"]})]
+
+    document_store.delete_documents(ids=ids_to_delete, filters={"name": ["name_1", "name_2", "name_3", "name_4"]})
+
+    documents = document_store.get_all_documents()
+    assert len(documents) == len(DOCUMENTS) - len(ids_to_delete)
+    assert document_store.get_embedding_count() == len(DOCUMENTS) - len(ids_to_delete)
+
+    assert all(doc.meta["name"] != "name_1" for doc in documents)
+    assert all(doc.meta["name"] != "name_2" for doc in documents)
+
+    all_ids_left = [doc.id for doc in documents]
+    assert all(doc_id in all_ids_left for doc_id in ids_not_to_delete)
+
+ 
+
 @pytest.mark.parametrize("retriever", ["embedding"], indirect=True)
 @pytest.mark.parametrize("document_store", ["faiss", "milvus"], indirect=True)
 def test_pipeline(document_store, retriever):
diff --git a/test/test_weaviate.py b/test/test_weaviate.py
index b646d211e8..2f96192263 100644
--- a/test/test_weaviate.py
+++ b/test/test_weaviate.py
@@ -314,7 +314,7 @@ def test_query(document_store_with_docs):
 
 @pytest.mark.weaviate
 @pytest.mark.parametrize("document_store_with_docs", ["weaviate"], indirect=True)
-def test_delete_all_documents(document_store_with_docs):
+def test_delete_documents(document_store_with_docs):
     assert len(document_store_with_docs.get_all_documents()) == 3
 
     document_store_with_docs.delete_documents()
@@ -324,8 +324,36 @@ def test_delete_all_documents(document_store_with_docs):
 @pytest.mark.weaviate
 @pytest.mark.parametrize("document_store_with_docs", ["weaviate"], indirect=True)
 def test_delete_documents_with_filters(document_store_with_docs):
-    document_store_with_docs.delete_all_documents(filters={"metafield": ["test1", "test2"]})
+    assert len(document_store_with_docs.get_all_documents()) == 3
+
+    document_store_with_docs.delete_documents(filters={"metafield": ["test1", "test2"]})
     documents = document_store_with_docs.get_all_documents()
     assert len(documents) == 1
     assert documents[0].meta["metafield"] == "test3"
 
+
+@pytest.mark.weaviate
+@pytest.mark.parametrize("document_store_with_docs", ["weaviate"], indirect=True)
+def test_delete_documents_by_id(document_store_with_docs):
+    assert len(document_store_with_docs.get_all_documents()) == 3
+    ids_to_delete = [doc.id for doc in document_store_with_docs.get_all_documents()[0:2]]
+
+    document_store_with_docs.delete_documents(ids=ids_to_delete)
+    documents = document_store_with_docs.get_all_documents()
+    assert len(documents) == 1
+    assert documents[0].id not in ids_to_delete
+
+
+@pytest.mark.weaviate
+@pytest.mark.parametrize("document_store_with_docs", ["weaviate"], indirect=True)
+def test_delete_documents_by_id_with_filters(document_store_with_docs):
+    docs_to_delete = document_store_with_docs.get_all_documents(filters={"metafield": ["test1", "test2"]})
+    docs_not_to_delete = document_store_with_docs.get_all_documents(filters={"metafield": ["test3"]})
+
+    document_store_with_docs.delete_documents(ids=[doc.id for doc in docs_to_delete], filters={"metafield": ["test1"]})
+
+    all_docs_left = document_store_with_docs.get_all_documents()
+    assert len(all_docs_left) == 2
+    assert all(doc.meta["metafield"] != "test1" for doc in all_docs_left)
+    all_ids_left = [doc.id for doc in all_docs_left]
+    assert all(doc.id in all_ids_left for doc in docs_not_to_delete)


Code review comments:
diff_hunk: @@ -514,33 +514,37 @@ def delete_all_documents(self, index: Optional[str] = None, filters: Optional[Di
                 For more details, please refer to the issue: https://github.com/deepset-ai/haystack/issues/1045
                 """
         )
-        self.delete_documents(index, filters)
+        self.delete_documents(index, None, filters)
 
-    def delete_documents(self, index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None):
+    def delete_documents(self, index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None):
         """
         Delete documents in an index. All documents are deleted if no filters are passed.
 
         :param index: Index name to delete the document from. If None, the
                       DocumentStore's default index (self.index) will be used.
+        :param ids: Optional list of IDs to narrow down the documents to be deleted.
         :param filters: Optional filters to narrow down the documents to be deleted.
-                        Example filters: {"name": ["some", "more"], "category": ["only_one"]}
+            Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+            If filters are provided along with a list of IDs, this method deletes the
+            intersection of the two query results (documents that match the filters and
+            have their ID in the list).
         :return: None
         """
         index = index or self.index
-
+        # documents_query = documents_query.join(MetaORM)
path: haystack/document_store/sql.py
text: @user1:
I think we can get rid of this commented line here

diff_hunk: @@ -514,33 +514,37 @@ def delete_all_documents(self, index: Optional[str] = None, filters: Optional[Di
                 For more details, please refer to the issue: https://github.com/deepset-ai/haystack/issues/1045
                 """
         )
-        self.delete_documents(index, filters)
+        self.delete_documents(index, None, filters)
 
-    def delete_documents(self, index: Optional[str] = None, filters: Optional[Dict[str, List[str]]] = None):
+    def delete_documents(self, index: Optional[str] = None, ids: Optional[List[str]] = None, filters: Optional[Dict[str, List[str]]] = None):
         """
         Delete documents in an index. All documents are deleted if no filters are passed.
 
         :param index: Index name to delete the document from. If None, the
                       DocumentStore's default index (self.index) will be used.
+        :param ids: Optional list of IDs to narrow down the documents to be deleted.
         :param filters: Optional filters to narrow down the documents to be deleted.
-                        Example filters: {"name": ["some", "more"], "category": ["only_one"]}
+            Example filters: {"name": ["some", "more"], "category": ["only_one"]}.
+            If filters are provided along with a list of IDs, this method deletes the
+            intersection of the two query results (documents that match the filters and
+            have their ID in the list).
         :return: None
         """
         index = index or self.index
-
+        # documents_query = documents_query.join(MetaORM)
+        document_ids_to_delete = self.session.query(DocumentORM.id).filter(DocumentORM.index==index)
         if filters:
-            # documents_query = documents_query.join(MetaORM)
-            document_ids_to_delete = self.session.query(DocumentORM.id).filter_by(index=index)
             for key, values in filters.items():
                 document_ids_to_delete = document_ids_to_delete.filter(
                     MetaDocumentORM.name == key,
-                        MetaDocumentORM.value.in_(values),
+                    MetaDocumentORM.value.in_(values),
                     DocumentORM.id == MetaDocumentORM.document_id
                 )
-            self.session.query(DocumentORM).filter(DocumentORM.id.in_(document_ids_to_delete)).delete(
-                    synchronize_session=False)
-        else:
-            self.session.query(DocumentORM).filter_by(index=index).delete(synchronize_session=False)
path: haystack/document_store/sql.py
text: @user1:
It might make sense to keep this query for the case of deleting all documents (filters=none, ids =none) to optimize for speed. Otherwise, it might become slow to query **all** doc ids first, and then delete by the below filter condition where you check if an id is in the list of all ids.

@author:
Right! Should be fixed now