Repository: SciTools/cartopy

Issue:
BUG: incompatibility with scipy 1.11.0
### Description
Scipy 1.11.0 (released yesterday) appear to break [use cases exercised in yt's CI](https://github.com/yt-project/yt/issues/4540)

More specifically the breaking change seems to be https://github.com/scipy/scipy/pull/18502


#### Code to reproduce

```python
import numpy as np
import cartopy.crs as ccrs
from matplotlib.figure import Figure

fig = Figure()
ax = fig.add_axes((0, 0, 1, 1), projection=ccrs.Mollweide())
ax.imshow([[0, 1], [0, 1]], transform=ccrs.PlateCarree())
```

#### Traceback

```python-traceback
Traceback (most recent call last):
  File "/private/tmp/t.py", line 7, in <module>
    ax.imshow([[0, 1], [0, 1]], transform=ccrs.PlateCarree())
  File "/private/tmp/.venv/lib/python3.11/site-packages/cartopy/mpl/geoaxes.py", line 318, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/.venv/lib/python3.11/site-packages/cartopy/mpl/geoaxes.py", line 1331, in imshow
    img, extent = warp_array(img,
                  ^^^^^^^^^^^^^^^
  File "/private/tmp/.venv/lib/python3.11/site-packages/cartopy/img_transform.py", line 192, in warp_array
    array = regrid(array, source_native_xy[0], source_native_xy[1],
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/.venv/lib/python3.11/site-packages/cartopy/img_transform.py", line 278, in regrid
    _, indices = kdtree.query(target_xyz, k=1)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "_ckdtree.pyx", line 795, in scipy.spatial._ckdtree.cKDTree.query
ValueError: 'x' must be finite, check for nan or inf values
```

<details>
 <summary>Full environment definition</summary>

<!-- fill in the following information as appropriate -->

### Operating system

Discovered in CI on Ubuntu, and reproduced on macOS


### Cartopy version
0.21.1

### pip list

```
Package         Version
--------------- --------
Cartopy         0.21.1
certifi         2023.5.7
contourpy       1.1.0
cycler          0.11.0
fonttools       4.40.0
kiwisolver      1.4.4
matplotlib      3.7.1
numpy           1.25.0
packaging       23.1
Pillow          9.5.0
pip             23.1.2
pyparsing       3.1.0
pyproj          3.6.0
pyshp           2.3.1
python-dateutil 2.8.2
scipy           1.11.0
setuptools      65.5.0
shapely         2.0.1
six             1.16.0
```

</details>



Discussion:
I was able to reproduce this error with the cartopy example for plotting images ([link](https://scitools.org.uk/cartopy/docs/v0.13/matplotlib/advanced_plotting.html#images)) for a number of projection choices: Mollweide, Orthographic, LambertAzimuthalEqualArea, LambertCylindrical all result in the same error as above (simply changing the `ax = plt.axes(projection=ccrs.PlateCarree())` line to use different projections). 
So I think the issue is simply that previously cartopy didn't worry if the proj4 projection returned invalid `target_xyz` values and relied on masking after any kdtree calls. Here's a diff that attempts to fix the error in this PR by filtering the `target_xyz` values before evaluating with the kdtree: 

```
diff --git a/lib/cartopy/img_transform.py b/lib/cartopy/img_transform.py
index 14af226e..a1d8b726 100644
--- a/lib/cartopy/img_transform.py
+++ b/lib/cartopy/img_transform.py
@@ -268,6 +268,7 @@ def regrid(array, source_x_coords, source_y_coords, source_proj, target_proj,
                                               target_x_points.flatten(),
                                               target_y_points.flatten())
 
     if _is_pykdtree:
         kdtree = pykdtree.kdtree.KDTree(xyz)
         # Use sqr_dists=True because we don't care about distances,
@@ -277,7 +278,11 @@ def regrid(array, source_x_coords, source_y_coords, source_proj, target_proj,
         # Versions of scipy >= v0.16 added the balanced_tree argument,
         # which caused the KDTree to hang with this input.
         kdtree = scipy.spatial.cKDTree(xyz, balanced_tree=False)
-        _, indices = kdtree.query(target_xyz, k=1)
+
+        # precompute finite mask to avoid scipy kdtree error
+        target_finite_mask = np.array([np.isfinite(target_xyz[:, i]) for i in range(3)]).all(axis=0)
+        indices = np.zeros(target_xyz.shape[0], dtype=int)
+        _, indices[target_finite_mask] = kdtree.query(target_xyz[target_finite_mask, :], k=1)
     mask = indices >= len(xyz)
     indices[mask] = 0
 ```
In my limited testing that seems to fix the problem, but I'm not familiar enough with the cartopy codebase to feel comfortable submitting this as PR. Still, thought it might be helpful for you all. 
@chrishavlin there is only one way to get comfortable and that is to contribute :) Your PR would be more than welcome!

Seems like a pretty reasonable fix, and may even be appropriate to put above that block and use it regardless of whether we are using scipy or pykdtree. We might need to update the `mask` below too? I think you can probably do an `np.any(..., axis=-1)` or something similar rather than iterating over indices too.
well I was worried about having the bandwidth in the short term to work out the related `mask` refactoring that you mention. I can certainly submit a PR based on the above diff to get it started, I just might be a little slow to respond to reviews and thought this might be a critical enough issue that someone more familiar might want to get a fix in faster :) But I'll go ahead and submit it.

Resolving patch:
diff --git a/lib/cartopy/img_transform.py b/lib/cartopy/img_transform.py
--- a/lib/cartopy/img_transform.py
+++ b/lib/cartopy/img_transform.py
@@ -13,11 +13,11 @@
 
 
 try:
-    import pykdtree.kdtree
+    from pykdtree.kdtree import KDTree as _kdtreeClass
     _is_pykdtree = True
 except ImportError:
     try:
-        import scipy.spatial
+        from scipy.spatial import cKDTree as _kdtreeClass
     except ImportError as e:
         raise ImportError("Using image transforms requires either "
                           "pykdtree or scipy.") from e
@@ -268,17 +268,25 @@ def regrid(array, source_x_coords, source_y_coords, source_proj, target_proj,
                                               target_x_points.flatten(),
                                               target_y_points.flatten())
 
+    # Find mask of valid points before querying kdtree: scipy >= 1.11 errors
+    # when querying nan points, might as well use for pykdtree too.
+    indices = np.zeros(target_xyz.shape[0], dtype=int)
+    finite_xyz = np.all(np.isfinite(target_xyz), axis=-1)
+
     if _is_pykdtree:
-        kdtree = pykdtree.kdtree.KDTree(xyz)
+        kdtree = _kdtreeClass(xyz)
         # Use sqr_dists=True because we don't care about distances,
         # and it saves a sqrt.
-        _, indices = kdtree.query(target_xyz, k=1, sqr_dists=True)
+        _, indices[finite_xyz] = kdtree.query(target_xyz[finite_xyz, :],
+                                              k=1,
+                                              sqr_dists=True)
     else:
         # Versions of scipy >= v0.16 added the balanced_tree argument,
         # which caused the KDTree to hang with this input.
-        kdtree = scipy.spatial.cKDTree(xyz, balanced_tree=False)
-        _, indices = kdtree.query(target_xyz, k=1)
-    mask = indices >= len(xyz)
+        kdtree = _kdtreeClass(xyz, balanced_tree=False)
+        _, indices[finite_xyz] = kdtree.query(target_xyz[finite_xyz, :], k=1)
+
+    mask = ~finite_xyz | (indices >= len(xyz))
     indices[mask] = 0
 
     desired_ny, desired_nx = target_x_points.shape


Test patch:
diff --git a/lib/cartopy/tests/test_img_transform.py b/lib/cartopy/tests/test_img_transform.py
--- a/lib/cartopy/tests/test_img_transform.py
+++ b/lib/cartopy/tests/test_img_transform.py
@@ -7,6 +7,7 @@
 import numpy as np
 from numpy.testing import assert_array_equal
 import pytest
+import scipy.spatial
 
 import cartopy.crs as ccrs
 import cartopy.img_transform as img_trans
@@ -108,3 +109,25 @@ def test_gridding_data_outside_projection():
     assert_array_equal([-180, 180, -90, 90], extent)
     assert_array_equal(expected, image)
     assert_array_equal(expected_mask, image.mask)
+
+
+@pytest.mark.parametrize("target_prj",
+                         (ccrs.Mollweide(), ccrs.Orthographic()))
+@pytest.mark.parametrize("use_scipy", (True, False))
+def test_regridding_with_invalid_extent(target_prj, use_scipy, monkeypatch):
+    # tests that when a valid extent results in invalid points in the
+    # transformed coordinates, the regridding does not error.
+
+    # create 3 data points
+    lats = np.array([65, 10, -45])
+    lons = np.array([-170, 10, 170])
+    data = np.array([1, 2, 3])
+    data_trans = ccrs.Geodetic()
+
+    target_x, target_y, extent = img_trans.mesh_projection(target_prj, 8, 4)
+
+    if use_scipy:
+        monkeypatch.setattr(img_trans, "_is_pykdtree", False)
+        monkeypatch.setattr(img_trans, "_kdtreeClass", scipy.spatial.cKDTree)
+    _ = img_trans.regrid(data, lons, lats, data_trans, target_prj,
+                         target_x, target_y)