Repository: pysal/momepy

Issue:
Handle `All-NaN slice encountered` warning in diversity.Percentiles()
### Describe the problem

A `RuntimeWarning` (`All-NaN slice encountered`) is thrown from within `diversity.Percentiles()` by `numpy.nanpercentile` (see [here](https://github.com/pysal/momepy/blob/b2d88846a2a3c4162682aa7ce0ec0d4040260975/momepy/diversity.py#L913)) when [`a`](https://numpy.org/doc/stable/reference/generated/numpy.nanpercentile.html#numpy-nanpercentile) is all `numpy.nan`s. This scenario should be caught and handle prior to the `numpy.nanpercentile`, like what was done in #389. MWE below.

This warning can also potentially be thrown in [`utils.limit_range`](https://github.com/pysal/momepy/blob/67d9509c5e6a7122998e175dc47606f61f242c97/momepy/utils.py#L475-L486).

### Steps to reproduce

```python
import numpy
percentiles = [25, 50, 75]
value_arrays = [
    numpy.array([[10, numpy.nan, 0]]),                # not all NaNs
    numpy.array([[numpy.nan, numpy.nan, numpy.nan]])  # all NaNs
]
for a in value_arrays:
    a_all_nan = numpy.isnan(a).all()
    if a_all_nan:
        values_list = a
    else:
        values_list = numpy.nanpercentile(a, percentiles, method="midpoint")
    print(values_list)
# [5. 5. 5.]       # return percentiles of `a` object
# [[nan nan nan]]  # return original `a` object
```

### Versions of your packages

```momepy='0.5.3+16.gb30e8cc.dirty'```

### Your operating system

_No response_

### Additional context

_No response_

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/pysal/momepy/pull/395?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pysal) Report
> Merging [#395](https://codecov.io/gh/pysal/momepy/pull/395?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pysal) (13da137) into [main](https://codecov.io/gh/pysal/momepy/commit/afcb85b89195f3ba2c5c869fb2e5f5add357c927?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pysal) (afcb85b) will **not change** coverage.
> The diff coverage is `90.9%`.

[![Impacted file tree graph](https://codecov.io/gh/pysal/momepy/pull/395/graphs/tree.svg?width=650&height=150&src=pr&token=VNn0WR5JWT&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pysal)](https://codecov.io/gh/pysal/momepy/pull/395?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pysal)

```diff
@@          Coverage Diff          @@
##            main    #395   +/-   ##
=====================================
  Coverage   95.9%   95.9%           
=====================================
  Files         13      13           
  Lines       2965    2965           
=====================================
  Hits        2842    2842           
  Misses       123     123           
```

| [Impacted Files](https://codecov.io/gh/pysal/momepy/pull/395?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pysal) | Coverage Δ | |
|---|---|---|
| [momepy/diversity.py](https://codecov.io/gh/pysal/momepy/pull/395/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pysal#diff-bW9tZXB5L2RpdmVyc2l0eS5weQ==) | `97.7% <85.7%> (+<0.1%)` | :arrow_up: |
| [momepy/utils.py](https://codecov.io/gh/pysal/momepy/pull/395/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pysal#diff-bW9tZXB5L3V0aWxzLnB5) | `98.8% <100.0%> (-<0.1%)` | :arrow_down: |

jGaboardi: Here's a timing update for `diversity.Percentiles()`:
* 2.65 s ± 4.95 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) ---- no check for nans
* 3.89 s ± 306 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) ---- check for all nans (wrong shape)
* 3.31 s ± 25.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) ---- nan filler (correct shape)
martinfleis: > Here's a timing update

does this suggest that the best solution would be to keep the implementation we have and just suppress the warning?
jGaboardi: > > Here's a timing update
> 
> does this suggest that the best solution would be to keep the implementation we have and just suppress the warning?

Seems like yes. Let me test the warning timing again to make sure.
jGaboardi: > > > Here's a timing update
> > 
> > 
> > does this suggest that the best solution would be to keep the implementation we have and just suppress the warning?
> 
> Seems like yes. Let me test the warning timing again to make sure.

So the answer is 'yes,' simply catching the `All-NaN slice encountered` warning is the most efficient solution currently (nearly 1 second faster for one column in the sdss2022 notebook).

* 2.51 s ± 14.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Shall I push that up for the code chunk in `Percentiles()` and update `limit_range()` with the actual `nump.isnan()` checker?
martinfleis: Okay. Sounds good. 
jGaboardi: > Okay. Sounds good.

Still feel like I might be missing something. Going try out a couple more things.
martinfleis: I think you haven't pushed. 
jGaboardi: Weird. [CI ran for the push](https://github.com/pysal/momepy/actions/runs/3038949553/jobs/4893299442#step:5:160) and the [files changes are current](https://github.com/pysal/momepy/pull/395/files), but no commit/run report here in the actual PR.

Patch:
diff --git a/ci/envs/38-minimal.yaml b/ci/envs/38-minimal.yaml
index 9476ef8..9f4bde1 100644
--- a/ci/envs/38-minimal.yaml
+++ b/ci/envs/38-minimal.yaml
@@ -3,15 +3,16 @@ channels:
   - conda-forge
 dependencies:
   - python=3.8
+  - codecov
   - geopandas=0.8
+  - inequality
   - libpysal=4.2.2
-  - networkx=2.3
-  - tqdm=4.27.0
-  - pytest
-  - pytest-cov
-  - codecov
   - mapclassify
+  - networkx=2.3
+  - numpy=1.21
   - osmnx
-  - pygeos
-  - inequality
   - packaging
+  - pygeos
+  - pytest
+  - pytest-cov
+  - tqdm=4.27.0
diff --git a/momepy/diversity.py b/momepy/diversity.py
index 451e058..da4539a 100644
--- a/momepy/diversity.py
+++ b/momepy/diversity.py
@@ -5,11 +5,12 @@
 # definitions of diversity characters
 
 import numpy as np
+from numpy.lib import NumpyVersion
 import pandas as pd
 import scipy as sp
 from tqdm.auto import tqdm  # progress bar
+import warnings
 
-from numpy.lib import NumpyVersion
 
 __all__ = [
     "Range",
@@ -899,19 +900,23 @@ class Percentiles:
         elif weighted is None:
             data = data.set_index(unique_id)[values]
 
+            if NumpyVersion(np.__version__) >= "1.22.0":
+                method = dict(method=interpolation)
+            else:
+                method = dict(interpolation=interpolation)
+
             for index in tqdm(data.index, total=data.shape[0], disable=not verbose):
                 if index in spatial_weights.neighbors.keys():
                     neighbours = [index]
                     neighbours += spatial_weights.neighbors[index]
-
                     values_list = data.loc[neighbours]
-                    if NumpyVersion(np.__version__) >= "1.22.0":
-                        method = dict(method=interpolation)
-                    else:
-                        method = dict(interpolation=interpolation)
-                    results_list.append(
-                        np.nanpercentile(values_list, percentiles, **method)
-                    )
+                    with warnings.catch_warnings():
+                        warnings.filterwarnings(
+                            "ignore", message="All-NaN slice encountered"
+                        )
+                        results_list.append(
+                            np.nanpercentile(values_list, percentiles, **method)
+                        )
                 else:
                     results_list.append(np.nan)
 
diff --git a/momepy/utils.py b/momepy/utils.py
index 962572a..5cdc52e 100644
--- a/momepy/utils.py
+++ b/momepy/utils.py
@@ -8,9 +8,9 @@ import geopandas as gpd
 import libpysal
 import networkx as nx
 import numpy as np
+from numpy.lib import NumpyVersion
 from shapely.geometry import Point, LineString
 
-from numpy.lib import NumpyVersion
 
 __all__ = [
     "unique_id",
@@ -470,20 +470,24 @@ def limit_range(vals, rng):
     array
         limited array
     """
+
     vals = np.asarray(vals)
-    if len(vals) > 2:
+    nan_tracker = np.isnan(vals)
+
+    if (len(vals) > 2) and (not nan_tracker.all()):
         if NumpyVersion(np.__version__) >= "1.22.0":
             method = dict(method="nearest")
         else:
             method = dict(interpolation="nearest")
         rng = sorted(rng)
-        if np.isnan(vals).any():
+        if nan_tracker.any():
             lower = np.nanpercentile(vals, rng[0], **method)
             higher = np.nanpercentile(vals, rng[1], **method)
         else:
             lower = np.percentile(vals, rng[0], **method)
             higher = np.percentile(vals, rng[1], **method)
-        return vals[(lower <= vals) & (vals <= higher)]
+        vals = vals[(lower <= vals) & (vals <= higher)]
+
     return vals
 
 


Test patch:
diff --git a/tests/test_utils.py b/tests/test_utils.py
index 88d85b9..4393b22 100644
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -190,3 +190,7 @@ class TestUtils:
         assert list(
             mm.limit_range(np.array([0, 1, 2, 3, 4, np.nan]), rng=(25, 75))
         ) == [1, 2, 3]
+        np.testing.assert_array_equal(
+            mm.limit_range(np.array([np.nan, np.nan, np.nan]), rng=(25, 75)),
+            np.array([np.nan, np.nan, np.nan]),
+        )