Repository: elastic/elasticsearch-dsl-py (language: python)

Issue:
 Inconsistent handling of empty lists in Document partial updates.
Hi! At work we are currently developing an API that relies on Elasticsearch responses. Within our document mapping, we have defined multi-value fields, which we aim to represent as empty arrays (**`[]`**) in the API response when these fields are empty. When initially indexing the document, it successfully store empty lists into ES as **`[]`** by passing these fields with **`skip_empty=False`**.

However, we are encountering an issue with partial updates. After updating a field to become empty, it is stored as **`None`** instead of **`[]`**, even though we are passing the field values as **`[]`**. 

The root cause appears to be within the current implementation of the **`Document.update`** method for partial document updates. This method prepares data for Elasticsearch by invoking the [**`to_dict`**](https://github.com/elastic/elasticsearch-dsl-py/blob/3e880fd4c18b61874884b3e9a4040204f8a9f507/elasticsearch_dsl/_sync/document.py#L296) method, which, by default, ignores empty fields. Consequently, empty lists are treated as **`None`** and updated accordingly.

**Proposed Fix:**
To address this issue, I propose modifying the **`update`** method to accept **`skip_empty=False`** and passing this parameter to the **`to_dict`** method. This adjustment will ensure that empty lists are included in the update payload as empty lists instead of being interpreted as **`None`**.

**Steps to Reproduce:**

1. Save a document with multi-value fields.
2. Perform a partial update using the **`Document.update`** method, passing multi-value fields as empty lists **`[]`**.
3. Verify that the empty lists are updated as **`None`** instead of remaining as empty lists.


I am willing to contribute to fixing this issue if you believe it should be fixed in this way.


Discussion / hints:
I was able to reproduce this easily. I think `update()` should always use `to_dict(skip_empty=False)`, I don't see why you would send an empty value to update but would like it to be skipped. Working on a fix.
You're right; there is no reason to pass `skip_empty=True` on an update. The default option, `False`, is correct.

Thank you!

Fix patch:
diff --git a/elasticsearch_dsl/_async/document.py b/elasticsearch_dsl/_async/document.py
index 23d3c282..89ed06f4 100644
--- a/elasticsearch_dsl/_async/document.py
+++ b/elasticsearch_dsl/_async/document.py
@@ -295,7 +295,7 @@ async def update(
             merge(self, fields)
 
             # prepare data for ES
-            values = self.to_dict()
+            values = self.to_dict(skip_empty=False)
 
             # if fields were given: partial update
             body["doc"] = {k: values.get(k) for k in fields.keys()}
diff --git a/elasticsearch_dsl/_sync/document.py b/elasticsearch_dsl/_sync/document.py
index 7567ff1a..c851c8e8 100644
--- a/elasticsearch_dsl/_sync/document.py
+++ b/elasticsearch_dsl/_sync/document.py
@@ -293,7 +293,7 @@ def update(
             merge(self, fields)
 
             # prepare data for ES
-            values = self.to_dict()
+            values = self.to_dict(skip_empty=False)
 
             # if fields were given: partial update
             body["doc"] = {k: values.get(k) for k in fields.keys()}


Test patch:
diff --git a/tests/test_integration/_async/test_document.py b/tests/test_integration/_async/test_document.py
index e18b9467..10677fc5 100644
--- a/tests/test_integration/_async/test_document.py
+++ b/tests/test_integration/_async/test_document.py
@@ -120,6 +120,13 @@ class Index:
         name = "test-serialization"
 
 
+class Tags(AsyncDocument):
+    tags = Keyword(multi=True)
+
+    class Index:
+        name = "tags"
+
+
 @pytest.mark.asyncio
 async def test_serialization(async_write_client):
     await SerializationDoc.init()
@@ -504,6 +511,19 @@ async def test_save_updates_existing_doc(async_data_client):
     assert new_repo["_seq_no"] == elasticsearch_repo.meta.seq_no
 
 
+@pytest.mark.asyncio
+async def test_update_empty_field(async_client):
+    await Tags._index.delete(ignore_unavailable=True)
+    await Tags.init()
+    d = Tags(id="123", tags=["a", "b"])
+    await d.save(refresh=True)
+    await d.update(tags=[], refresh=True)
+    assert d.tags == []
+
+    r = await Tags.search().execute()
+    assert r.hits[0].tags == []
+
+
 @pytest.mark.asyncio
 async def test_save_automatically_uses_seq_no_and_primary_term(async_data_client):
     elasticsearch_repo = await Repository.get("elasticsearch-dsl-py")
diff --git a/tests/test_integration/_sync/test_document.py b/tests/test_integration/_sync/test_document.py
index 16f39ad9..9718f080 100644
--- a/tests/test_integration/_sync/test_document.py
+++ b/tests/test_integration/_sync/test_document.py
@@ -120,6 +120,13 @@ class Index:
         name = "test-serialization"
 
 
+class Tags(Document):
+    tags = Keyword(multi=True)
+
+    class Index:
+        name = "tags"
+
+
 @pytest.mark.sync
 def test_serialization(write_client):
     SerializationDoc.init()
@@ -498,6 +505,19 @@ def test_save_updates_existing_doc(data_client):
     assert new_repo["_seq_no"] == elasticsearch_repo.meta.seq_no
 
 
+@pytest.mark.sync
+def test_update_empty_field(client):
+    Tags._index.delete(ignore_unavailable=True)
+    Tags.init()
+    d = Tags(id="123", tags=["a", "b"])
+    d.save(refresh=True)
+    d.update(tags=[], refresh=True)
+    assert d.tags == []
+
+    r = Tags.search().execute()
+    assert r.hits[0].tags == []
+
+
 @pytest.mark.sync
 def test_save_automatically_uses_seq_no_and_primary_term(data_client):
     elasticsearch_repo = Repository.get("elasticsearch-dsl-py")