Repository: bjlittle/geovista

Issue:
rasterio 1.4.x regression
## 🐛 Bug Report
<!-- Provide a clear description of the bug/issue that you're experiencing -->

The recent release of [rasterio 1.4.0](https://github.com/rasterio/rasterio/releases/tag/1.4.0) has resulted in a behaviour regression, see https://github.com/rasterio/rasterio/issues/3191.

The following [rasterio 1.4.1](https://github.com/rasterio/rasterio/releases/tag/1.4.1) patch to resolve this causes similar but different issues within `geovista.bridge.Transform.from_tiff`.

All of these issues can be resolved entirely within `geovista` without pinning or skipping `rasterio` versions by explicitly managing the shape of data passed into and out of `rasterio.transform.xy`.


Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/bjlittle/geovista/pull/1132?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Bill+Little) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 91.32%. Comparing base [(`57fc7ed`)](https://app.codecov.io/gh/bjlittle/geovista/commit/57fc7edd7c12d554f35e50afbce40967bc4df913?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Bill+Little) to head [(`22841f1`)](https://app.codecov.io/gh/bjlittle/geovista/commit/22841f102aac258f2feec4bf4c448e307e06db34?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Bill+Little).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           v0.5.x    #1132      +/-   ##
==========================================
+ Coverage   91.30%   91.32%   +0.01%     
==========================================
  Files          58       58              
  Lines        2934     2939       +5     
==========================================
+ Hits         2679     2684       +5     
  Misses        255      255              
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/bjlittle/geovista/pull/1132?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Bill+Little).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Bill+Little).


Patch:
diff --git a/src/geovista/bridge.py b/src/geovista/bridge.py
index eafc184..f95f56b 100644
--- a/src/geovista/bridge.py
+++ b/src/geovista/bridge.py
@@ -867,7 +867,19 @@ class Transform:  # numpydoc ignore=PR01
             cols, rows = np.meshgrid(
                 np.arange(src.width), np.arange(src.height), indexing="xy"
             )
-            xs, ys = rio.transform.xy(src.transform, rows, cols)
+            # rasterio 1.4.0 (regression) expects 1-D arrays, fixed in 1.4.1
+            # see https://github.com/rasterio/rasterio/issues/3191
+            xs, ys = rio.transform.xy(src.transform, rows.flatten(), cols.flatten())
+
+            # ensure we have arrays, rather than a list of arrays
+            xs, ys = np.asanyarray(xs), np.asanyarray(ys)
+
+            # ensure shape is maintained (rasterio 1.4.1 regression)
+            if xs.shape != src.shape:
+                xs = xs.reshape(src.shape)
+
+            if ys.shape != src.shape:
+                ys = ys.reshape(src.shape)
 
             # create the geotiff mesh
             mesh = cls.from_2d(
diff --git a/src/geovista/examples/rectilinear/from_1d__oisst.py b/src/geovista/examples/rectilinear/from_1d__oisst.py
index aebc82d..0695465 100755
--- a/src/geovista/examples/rectilinear/from_1d__oisst.py
+++ b/src/geovista/examples/rectilinear/from_1d__oisst.py
@@ -18,8 +18,8 @@ Creates a mesh from 1-D latitude and longitude rectilinear cell bounds.
 The resulting mesh contains quad cells.
 
 The example uses NOAA/NECI 1/4° Daily Optimum Interpolation Sea Surface Temperature
-(OISST) v2.1 Advanced Very High Resolution Radiometer (AVHRR) gridded data
-(https://doi.org/10.25921/RE9P-PT57). The data targets the mesh faces/cells.
+(OISST) v2.1 Advanced Very High Resolution Radiometer (AVHRR) gridded data.
+The data targets the mesh faces/cells.
 
 Note that, a threshold is also applied to remove land ``NaN`` cells, and a
 NASA Blue Marble base layer is rendered along with Natural Earth coastlines.
diff --git a/src/geovista/examples/rectilinear/from_1d__oisst_eqc.py b/src/geovista/examples/rectilinear/from_1d__oisst_eqc.py
index ff1e64d..8291ed6 100755
--- a/src/geovista/examples/rectilinear/from_1d__oisst_eqc.py
+++ b/src/geovista/examples/rectilinear/from_1d__oisst_eqc.py
@@ -18,8 +18,8 @@ Creates a mesh from 1-D latitude and longitude rectilinear cell bounds.
 The resulting mesh contains quad cells.
 
 The example uses NOAA/NECI 1/4° Daily Optimum Interpolation Sea Surface Temperature
-(OISST) v2.1 Advanced Very High Resolution Radiometer (AVHRR) gridded data
-(https://doi.org/10.25921/RE9P-PT57). The data targets the mesh faces/cells.
+(OISST) v2.1 Advanced Very High Resolution Radiometer (AVHRR) gridded data.
+The data targets the mesh faces/cells.
 
 Note that, a threshold is also applied to remove land ``NaN`` cells, and a
 NASA Blue Marble base layer is rendered along with Natural Earth coastlines.


Test patch:
diff --git a/.github/workflows/ci-tests.yml b/.github/workflows/ci-tests.yml
index 30ca93d..3ed5309 100644
--- a/.github/workflows/ci-tests.yml
+++ b/.github/workflows/ci-tests.yml
@@ -50,7 +50,7 @@ jobs:
         posargs: ["--xvfb-backend xvfb --durations=10"]
         include:
           - version: "py312"
-            coverage: "--cov-report=xml --cov=geovista"
+            coverage: "--cov-report= --cov=geovista"
 
     steps:
     - uses: actions/checkout@v4
@@ -131,13 +131,13 @@ jobs:
     - name: "prepare coverage"
       if: ${{ matrix.coverage }}
       run: |
-        mv .coverage .coverage${{ strategy.job-index }}
+        mv .coverage ci-test-coverage${{ strategy.job-index }}
 
     - if: ${{ matrix.coverage }}
       uses: actions/upload-artifact@v4
       with:
         name: coverage-artifacts-${{ github.job }}-${{ strategy.job-index }}
-        path: ${{ github.workspace }}/.coverage*
+        path: ${{ github.workspace }}/ci-test-coverage*
 
 
   coverage:
@@ -169,7 +169,7 @@ jobs:
 
       - name: "create coverage report"
         run: |
-          coverage combine .coverage*
+          coverage combine ci-test-coverage*
           coverage xml --omit=*/_version.py
 
       - name: "upload coverage report"
diff --git a/tests/bridge/test_from_tiff.py b/tests/bridge/test_from_tiff.py
index d331dd7..ea82d9f 100644
--- a/tests/bridge/test_from_tiff.py
+++ b/tests/bridge/test_from_tiff.py
@@ -44,7 +44,7 @@ def test_rgb_band_fail(mocker):
         _ = Transform.from_tiff(fname, rgb=True)
 
 
-@pytest.mark.parametrize("rgb", [True])
+@pytest.mark.parametrize("rgb", [True, False])
 @pytest.mark.parametrize("band", [1, 2, 3])
 @pytest.mark.parametrize("unit", ["m", None])
 def test_rgb_band(mocker, rgb, band, unit):
@@ -55,12 +55,14 @@ def test_rgb_band(mocker, rgb, band, unit):
     data = mocker.sentinel.data
     mocked_read = mocker.MagicMock(return_value=data)
     transform = mocker.sentinel.transform
-    height, width = 2, 3
+    height, width = pixels_shape = 2, 3
+    n_pixels = height * width
     kwargs = {
         "count": band,
         "crs": crs,
         "height": height,
         "read": mocked_read,
+        "shape": pixels_shape,
         "transform": transform,
         "units": [unit] * band,
         "width": width,
@@ -74,10 +76,13 @@ def test_rgb_band(mocker, rgb, band, unit):
             "numpy.dstack", return_value=mocker.MagicMock(reshape=mocked_reshape)
         )
 
-    cols, rows = mocker.sentinel.cols, mocker.sentinel.rows
+    cols_flatten = mocker.sentinel.cols_flatten
+    rows_flatten = mocker.sentinel.rows_flatten
+    cols = mocker.MagicMock(flatten=mocker.MagicMock(return_value=cols_flatten))
+    rows = mocker.MagicMock(flatten=mocker.MagicMock(return_value=rows_flatten))
     mocked_meshgrid = mocker.patch("numpy.meshgrid", return_value=(cols, rows))
 
-    xs, ys = mocker.sentinel.xs, mocker.sentinel.ys
+    xs, ys = np.arange(n_pixels), np.arange(n_pixels)
     mocked_xy = mocker.patch("rasterio.transform.xy", return_value=(xs, ys))
 
     expected = mocker.sentinel.mesh
@@ -102,19 +107,26 @@ def test_rgb_band(mocker, rgb, band, unit):
     assert len(args) == 2
     np.testing.assert_array_equal(args[0], np.arange(width))
     np.testing.assert_array_equal(args[1], np.arange(height))
+    assert mocked_meshgrid.call_args.kwargs == {"indexing": "xy"}
 
-    mocked_xy.assert_called_once_with(transform, rows, cols)
-
-    kwargs = {"radius": None, "zlevel": None, "zscale": None, "clean": None}
-    mocked_from_2d.assert_called_once_with(
-        xs,
-        ys,
-        data=data,
-        name=name.format(units=str(unit)),
-        crs=crs,
-        rgb=rgb,
-        **kwargs,
-    )
+    mocked_xy.assert_called_once_with(transform, rows_flatten, cols_flatten)
+
+    expected_kwargs = {
+        "data": data,
+        "name": name.format(units=str(unit)),
+        "crs": crs,
+        "rgb": rgb,
+        "radius": None,
+        "zlevel": None,
+        "zscale": None,
+        "clean": None,
+    }
+    mocked_from_2d.assert_called_once()
+    args = mocked_from_2d.call_args.args
+    assert len(args) == 2
+    assert args[0].shape == pixels_shape
+    assert args[1].shape == pixels_shape
+    assert mocked_from_2d.call_args.kwargs == expected_kwargs
 
 
 @pytest.mark.parametrize("sieve", [True])
@@ -122,7 +134,8 @@ def test_rgb_band(mocker, rgb, band, unit):
 @pytest.mark.parametrize("masked", [False, True])
 def test_extract(mocker, masked, rgb, sieve):
     """Test extract behaviour with and without image masking."""
-    height, width = 2, 3
+    pixels_shape = height, width = 2, 3
+    n_pixels = height * width
     band = 3 if rgb else 1
     crs = mocker.sentinel.crs
     dtypes = ["uint8"] * band
@@ -145,16 +158,20 @@ def test_extract(mocker, masked, rgb, sieve):
         "dtypes": dtypes,
         "height": height,
         "read": mocked_read,
+        "shape": pixels_shape,
         "transform": transform,
         "width": width,
     }
     dataset = mocker.MagicMock(**kwargs)
     mocker.patch("rasterio.open").return_value.__enter__.return_value = dataset
 
-    cols, rows = mocker.sentinel.cols, mocker.sentinel.rows
+    cols_flatten = mocker.sentinel.cols_flatten
+    rows_flatten = mocker.sentinel.rows_flatten
+    cols = mocker.MagicMock(flatten=mocker.MagicMock(return_value=cols_flatten))
+    rows = mocker.MagicMock(flatten=mocker.MagicMock(return_value=rows_flatten))
     mocked_meshgrid = mocker.patch("numpy.meshgrid", return_value=(cols, rows))
 
-    xs, ys = mocker.sentinel.xs, mocker.sentinel.ys
+    xs, ys = np.arange(n_pixels), np.arange(n_pixels)
     mocked_xy = mocker.patch("rasterio.transform.xy", return_value=(xs, ys))
 
     expected_mesh = mocker.sentinel.mesh
@@ -186,15 +203,14 @@ def test_extract(mocker, masked, rgb, sieve):
     assert len(args) == 2
     np.testing.assert_array_equal(args[0], np.arange(width))
     np.testing.assert_array_equal(args[1], np.arange(height))
+    assert mocked_meshgrid.call_args.kwargs == {"indexing": "xy"}
 
-    mocked_xy.assert_called_once_with(transform, rows, cols)
+    mocked_xy.assert_called_once_with(transform, rows_flatten, cols_flatten)
 
     if rgb:
         data = np.dstack(data).reshape(-1, band)
         mask = mask[0]
 
-    mocked_from_2d.assert_called_once()
-    assert mocked_from_2d.call_args.args == (xs, ys)
     expected_kwargs = {
         "name": None,
         "crs": crs,
@@ -204,6 +220,11 @@ def test_extract(mocker, masked, rgb, sieve):
         "zscale": None,
         "clean": None,
     }
+    mocked_from_2d.assert_called_once()
+    args = mocked_from_2d.call_args.args
+    assert len(args) == 2
+    assert args[0].shape == pixels_shape
+    assert args[1].shape == pixels_shape
     kwargs = mocked_from_2d.call_args.kwargs
     actual = kwargs.pop("data")
     assert kwargs == expected_kwargs