Repository: ecmwf/earthkit-data
Pull request #551 (2024-11-28 10:29:44)

Fix failing to_numpy() call on a wrapped field
Fixes https://github.com/ecmwf-lab/ai-models/issues/63

Note: the code in ai-models `src/ai_models/inputs/transform.py` should be removed and the `Field.clone()` method should be used instead.

codecov-commenter: ## [Codecov](https://app.codecov.io/gh/ecmwf/earthkit-data/pull/551?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf) Report
Attention: Patch coverage is `69.56522%` with `14 lines` in your changes missing coverage. Please review.
> Please [upload](https://docs.codecov.com/docs/codecov-uploader) report for BASE (`main@ecd14e0`). [Learn more](https://docs.codecov.io/docs/error-reference?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf#section-missing-base-commit) about missing BASE report.

| [Files with missing lines](https://app.codecov.io/gh/ecmwf/earthkit-data/pull/551?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf) | Patch % | Lines |
|---|---|---|
| [tests/ai-models/test\_ai\_warpped\_field.py](https://app.codecov.io/gh/ecmwf/earthkit-data/pull/551?src=pr&el=tree&filepath=tests%2Fai-models%2Ftest_ai_warpped_field.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf#diff-dGVzdHMvYWktbW9kZWxzL3Rlc3RfYWlfd2FycHBlZF9maWVsZC5weQ==) | 69.56% | [13 Missing and 1 partial :warning: ](https://app.codecov.io/gh/ecmwf/earthkit-data/pull/551?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf) |

<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##             main     #551   +/-   ##
=======================================
  Coverage        ?   90.30%           
=======================================
  Files           ?      149           
  Lines           ?    10884           
  Branches        ?      515           
=======================================
  Hits            ?     9829           
  Misses          ?      890           
  Partials        ?      165           
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/ecmwf/earthkit-data/pull/551?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf).


Patch:
diff --git a/src/earthkit/data/core/fieldlist.py b/src/earthkit/data/core/fieldlist.py
index 612b652..abd3a98 100644
--- a/src/earthkit/data/core/fieldlist.py
+++ b/src/earthkit/data/core/fieldlist.py
@@ -1045,7 +1045,7 @@ class FieldList(Index):
         if n > 0:
             it = iter(self)
             first = next(it)
-            is_property = isinstance(getattr(first.__class__, accessor), property)
+            is_property = not callable(getattr(first, accessor, None))
             vals = _vals(first)
             ns = array_namespace(vals)
             shape = (n, *vals.shape)


Test patch:
diff --git a/tests/ai-models/test_ai_warpped_field.py b/tests/ai-models/test_ai_warpped_field.py
new file mode 100644
index 0000000..3c0a9a5
--- /dev/null
+++ b/tests/ai-models/test_ai_warpped_field.py
@@ -0,0 +1,76 @@
+#!/usr/bin/env python3
+
+# (C) Copyright 2020 ECMWF.
+#
+# This software is licensed under the terms of the Apache Licence Version 2.0
+# which can be obtained at http://www.apache.org/licenses/LICENSE-2.0.
+# In applying this licence, ECMWF does not waive the privileges and immunities
+# granted to it by virtue of its status as an intergovernmental organisation
+# nor does it submit to any jurisdiction.
+#
+
+import logging
+
+import numpy as np
+
+from earthkit.data import SimpleFieldList
+from earthkit.data import from_source
+from earthkit.data.testing import earthkit_examples_file
+
+LOG = logging.getLogger(__name__)
+
+
+class WrappedField:
+    def __init__(self, field):
+        self._field = field
+
+    def __getattr__(self, name):
+        return getattr(self._field, name)
+
+    def __repr__(self) -> str:
+        return repr(self._field)
+
+
+class NewDataField(WrappedField):
+    def __init__(self, field, data):
+        super().__init__(field)
+        self._data = data
+        self.shape = data.shape
+
+    def to_numpy(self, flatten=False, dtype=None, index=None):
+        data = self._data
+        if dtype is not None:
+            data = data.astype(dtype)
+        if flatten:
+            data = data.flatten()
+        if index is not None:
+            data = data[index]
+        return data
+
+
+class NewMetadataField(WrappedField):
+    def __init__(self, field, **kwargs):
+        super().__init__(field)
+        self._metadata = kwargs
+
+    def metadata(self, *args, **kwargs):
+        if len(args) == 1 and args[0] in self._metadata:
+            return self._metadata[args[0]]
+        return self._field.metadata(*args, **kwargs)
+
+
+def test_ai_models_wrapped_field_grib():
+
+    ds = from_source("file", earthkit_examples_file("test.grib"))
+    np_ref = ds.to_numpy()
+    vals_ref = ds.values
+
+    f1 = NewMetadataField(ds[0], shortName="sstk")
+    f2 = NewMetadataField(ds[1], shortName="2d")
+
+    r = SimpleFieldList([f1, f2])
+
+    assert r[0].metadata("shortName") == "sstk"
+    assert r[1].metadata("shortName") == "2d"
+    assert np.allclose(r.to_numpy(), np_ref)
+    assert np.allclose(r.values, vals_ref)