Repository: deepset-ai/haystack

Issue:
Writing preprocessed documents to Weaviate fails 
**Describe the bug**
When writing documents that have been preprocessed to Weaviate it fails.

**Error message**
```
Traceback (most recent call last):
  File "/bug.py", line 39, in <module>
    document_store.write_documents(docs_to_index)
  File "/.env/lib/python3.9/site-packages/haystack/document_stores/weaviate.py", line 687, in write_documents
    self._update_schema(property, property_value, index)
  File "/.env/lib/python3.9/site-packages/haystack/document_stores/weaviate.py", line 518, in _update_schema
    data_type = self._get_weaviate_type_of_value(property_value)
  File "/.env/lib/python3.9/site-packages/haystack/document_stores/weaviate.py", line 538, in _get_weaviate_type_of_value
    value = value[0]

IndexError: list index out of range
```

**Expected behavior**
It should write the documents to the index.

**To Reproduce**
```python
document_store = WeaviateDocumentStore(
    host="http://localhost", port=8080, embedding_dim=384, index="test"
)
documents = [Document(content="some text to index")]

preprocessor = PreProcessor(
    clean_empty_lines=True,
    clean_whitespace=True,
    clean_header_footer=True,
    split_by="word",
    split_length=512,
    split_overlap=32,
    split_respect_sentence_boundary=True,
)

docs_to_index = preprocessor.process(documents)

# this line throws the error:
document_store.write_documents(docs_to_index)

# call the above with documents instead of docs_to_index works fine
```


**System:**
 - OS: Mac Ventura (M1)
 - GPU/CPU: CPU
 - Haystack version (commit or version number): 1.22.1
 - DocumentStore: WeaviateDocumentStore


Discussion:
Hello there, i've got same issue. Have you had a solution?
Sadly not, the 2 objects look identical but would need the Haystack team to debug the issue further.
The issue is easily reproducible and can sometimes make it impossible to write preprocessed docs.

Resolving patch:
diff --git a/haystack/document_stores/weaviate.py b/haystack/document_stores/weaviate.py
--- a/haystack/document_stores/weaviate.py
+++ b/haystack/document_stores/weaviate.py
@@ -684,8 +684,10 @@ def write_documents(
                             property_value = _doc[property]
                             if property in json_fields:
                                 property_value = doc.meta[property]
-                            self._update_schema(property, property_value, index)
-                            current_properties.append(property)
+                            # if the property_value is an empty list, we can't infer the type
+                            if not isinstance(property_value, list) or len(property_value) > 0:
+                                self._update_schema(property, property_value, index)
+                                current_properties.append(property)
                         # update the date fields as there might be new ones
                         date_fields = self._get_date_properties(index)
 


Test patch:
diff --git a/test/document_stores/test_weaviate.py b/test/document_stores/test_weaviate.py
--- a/test/document_stores/test_weaviate.py
+++ b/test/document_stores/test_weaviate.py
@@ -9,6 +9,7 @@
 from haystack.document_stores.weaviate import WeaviateDocumentStore
 from haystack.schema import Document
 from haystack.testing import DocumentStoreBaseTestAbstract
+from haystack.nodes.preprocessor import PreProcessor
 
 embedding_dim = 768
 
@@ -267,6 +268,33 @@ def test_get_embedding_count(self, ds, documents):
         ds.write_documents(documents)
         assert ds.get_embedding_count() == 9
 
+    @pytest.mark.integration
+    def test_write_preprocessed_docs(self, ds, documents):
+        """
+        Test that preprocessed documents can be correctly written to Weaviate
+        even if the meta field `_split_overlap` is an empty list for some documents.
+        """
+        preprocessor = PreProcessor(
+            clean_empty_lines=True,
+            clean_whitespace=True,
+            clean_header_footer=True,
+            split_by="word",
+            split_length=5,
+            split_overlap=2,
+            split_respect_sentence_boundary=False,
+        )
+
+        longer_doc = Document(content="This is a longer document that will be split into multiple parts.")
+        documents.append(longer_doc)
+
+        preprocessed_docs = preprocessor.process(documents)
+
+        ds.write_documents(preprocessed_docs)
+
+        docs_from_weaviate = ds.get_all_documents()
+        for doc in docs_from_weaviate:
+            assert "_split_overlap" in doc.meta
+
     @pytest.mark.unit
     def test__get_auth_secret(self):
         # Test with username and password