Repository: stac-utils/stac-fastapi-elasticsearch-opensearch (language: python)

Issue:
null fields in item search
**Describe the bug**
When the fields extension is enabled, the search response will include null for every field that is not defined in an item.

**To Reproduce**
Steps to reproduce the behavior:
1. Ingest STAC Item with minimal metadata, eg. https://github.com/radiantearth/stac-spec/blob/master/examples/simple-item.json
2. Send GET or POST `/search` request
3. The response contains null for the following fields of the features:
    - geometry.bbox
    - assets.*.description
    - assets.*.start_datetime
    - assets.*.created
    - ...
    - links[*].title
    - links[*].label


**Expected behavior**
According to the STAC API Fields extension, a field should not be specified in the search response if it does not apply to an Item.
https://github.com/stac-api-extensions/fields/blob/6662435f98b8fff6e5d8ffa0f212e3241a10bc6f/README.md?plain=1#L58-L61

**Additional context**
This bug can be solved in the following ways:
Fix it in stac-fastapi-elasticsearch by adding the `exclude_unset=True` parameter to the `.json()` call. 
https://github.com/stac-utils/stac-fastapi-elasticsearch/blob/5f26c325940761da7ecd86677da7acceea69bb78/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/core.py#L512

Or fix it in stac-fastapi by adding `exclude_unset` to the `PostFieldsExtension:filter_fields` dictionary.
https://github.com/stac-utils/stac-fastapi/blob/09dac221d86fe70035aa6cddbc9a3f0de304aff5/stac_fastapi/extensions/stac_fastapi/extensions/core/fields/request.py#L57-L60


Discussion / hints:
Do you want to make a pr for this here? Theoretically it can be set both places?
It can only be set in one place, as you can only supply one value for a keyword argument.
Judging from a quick GitHub search, it seems like `stac-fastapi-elasticsearch` is the only project using the `filter_fields` property to get the desired output from pydantic, so I'll create a PR here.

Fix patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7f9226fa..d50d6ff9 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 
 ### Fixed
 
+- Exclude unset fields in search response [#166](https://github.com/stac-utils/stac-fastapi-elasticsearch/issues/166)
 
 ## [v1.0.0]
 
diff --git a/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/core.py b/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/core.py
index 65e9a458..76bb04cd 100644
--- a/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/core.py
+++ b/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/core.py
@@ -509,7 +509,9 @@ async def post_search(
             filter_kwargs = search_request.fields.filter_fields
 
             items = [
-                orjson.loads(stac_pydantic.Item(**feat).json(**filter_kwargs))
+                orjson.loads(
+                    stac_pydantic.Item(**feat).json(**filter_kwargs, exclude_unset=True)
+                )
                 for feat in items
             ]
 


Test patch:
diff --git a/stac_fastapi/elasticsearch/tests/api/test_api.py b/stac_fastapi/elasticsearch/tests/api/test_api.py
index 932edc08..c50256da 100644
--- a/stac_fastapi/elasticsearch/tests/api/test_api.py
+++ b/stac_fastapi/elasticsearch/tests/api/test_api.py
@@ -163,6 +163,23 @@ async def test_app_fields_extension_no_properties_post(app_client, ctx, txn_clie
     assert "properties" not in resp_json["features"][0]
 
 
+@pytest.mark.asyncio
+async def test_app_fields_extension_no_null_fields(app_client, ctx, txn_client):
+    resp = await app_client.get("/search", params={"collections": ["test-collection"]})
+    assert resp.status_code == 200
+    resp_json = resp.json()
+    # check if no null fields: https://github.com/stac-utils/stac-fastapi-elasticsearch/issues/166
+    for feature in resp_json["features"]:
+        # assert "bbox" not in feature["geometry"]
+        for link in feature["links"]:
+            assert all(a not in link or link[a] is not None for a in ("title", "asset"))
+        for asset in feature["assets"]:
+            assert all(
+                a not in asset or asset[a] is not None
+                for a in ("start_datetime", "created")
+            )
+
+
 @pytest.mark.asyncio
 async def test_app_fields_extension_return_all_properties(app_client, ctx, txn_client):
     item = ctx.item