Repository: ecmwf/earthkit-data
Pull request #401 (2024-06-06 10:10:23)

Fix reading netcdf generated from FA file
This PR fixes the following NetCDF problems:
- fix issue when fields were assigned to variables without lat-lon/x-y dimensions
- fix getting the correct latitudes-longitudes when the variables "lat" and "lon" are used with the following mapping from x and y:
<img width="559" alt="image" src="https://github.com/ecmwf/earthkit-data/assets/9033020/af0ed0f0-160b-46b9-8fcd-d0651c03f02e">


codecov-commenter: ## [Codecov](https://app.codecov.io/gh/ecmwf/earthkit-data/pull/401?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 89.78%. Comparing base [(`36b2d9c`)](https://app.codecov.io/gh/ecmwf/earthkit-data/commit/36b2d9c6d4e54706077fc825653bd79151c4561a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf) to head [(`f74a79f`)](https://app.codecov.io/gh/ecmwf/earthkit-data/commit/f74a79f53104cd00128d7b01764bb96ba4b8d8fa?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf).


<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff            @@
##           develop     #401   +/-   ##
========================================
  Coverage    89.78%   89.78%           
========================================
  Files          120      120           
  Lines         7653     7653           
  Branches       675      675           
========================================
  Hits          6871     6871           
  Misses         651      651           
  Partials       131      131           
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/ecmwf/earthkit-data/pull/401?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf).


Patch:
diff --git a/src/earthkit/data/readers/netcdf/dataset.py b/src/earthkit/data/readers/netcdf/dataset.py
index feec536..a952456 100644
--- a/src/earthkit/data/readers/netcdf/dataset.py
+++ b/src/earthkit/data/readers/netcdf/dataset.py
@@ -117,9 +117,18 @@ class DataSet:
         keys, coords = self._get_xy_coords(data_array)
 
         points = dict()
-        if "latitude" in self._ds and "longitude" in self._ds:
-            latitude = self._ds["latitude"]
-            longitude = self._ds["longitude"]
+
+        def _get_ll(keys):
+            for key in keys:
+                if key in self._ds:
+                    return self._ds[key]
+
+        lat_keys = ["latitude", "lat"]
+        lon_keys = ["longitude", "lon"]
+        latitude = _get_ll(lat_keys)
+        longitude = _get_ll(lon_keys)
+
+        if latitude is not None and longitude is not None:
             if latitude.dims == coords and longitude.dims == coords:
                 latitude = latitude.data
                 longitude = longitude.data
diff --git a/src/earthkit/data/readers/netcdf/fieldlist.py b/src/earthkit/data/readers/netcdf/fieldlist.py
index 0de8074..a0bb40b 100644
--- a/src/earthkit/data/readers/netcdf/fieldlist.py
+++ b/src/earthkit/data/readers/netcdf/fieldlist.py
@@ -27,9 +27,6 @@ def get_fields_from_ds(
     field_type=None,
     check_only=False,
 ):  # noqa C901
-    # Select only geographical variables
-    has_lat = False
-    has_lon = False
 
     fields = []
 
@@ -47,6 +44,10 @@ def get_fields_from_ds(
         _skip_attr(v, "grid_mapping")
 
     for name in ds.data_vars:
+        # Select only geographical variables
+        has_lat = False
+        has_lon = False
+
         if name in skip:
             continue
 


Test patch:
diff --git a/tests/netcdf/test_netcdf_geography.py b/tests/netcdf/test_netcdf_geography.py
index bae6c9d..cd6dcba 100644
--- a/tests/netcdf/test_netcdf_geography.py
+++ b/tests/netcdf/test_netcdf_geography.py
@@ -239,6 +239,14 @@ def test_netcdf_to_latlon_laea():
             assert np.isclose(v["lat"][x], ref[i]), f"{i=}, {x=}"
 
 
+def test_netcdf_forecast_reference_time():
+    ds = from_source("url", earthkit_remote_test_data_file("test-data", "fa_ta850.nc"))
+
+    assert len(ds) == 37
+    assert ds[0].metadata("valid_datetime") == "2020-01-23T00:00:00"
+    assert ds[5].metadata("valid_datetime") == "2020-01-23T05:00:00"
+
+
 if __name__ == "__main__":
     from earthkit.data.testing import main