Repository: librosa/librosa

Issue:
Regression: bandwidth estimation now requires all vertices to have edges
**Describe the bug**

In 0.9 and earlier, the bandwidth for `affinity` mode in recurrence matrix calculation was inferred as the median distance to the k'th nearest neighbor of each data point:
https://github.com/librosa/librosa/blob/b9830b4a335b1e7e4be66ee81870573bf3d516d0/librosa/segment.py#L524-L526

Because we were using `nanmedian(rec.max(axis=1))`, any empty rows of `rec` would be implicitly ignored in the median calculation.  This, I believe, is a good and correct thing.

After merging #1495 , the behavior has changed:
https://github.com/librosa/librosa/blob/e18d544cbebfb590da34710ba2de86292179ec39/librosa/segment.py#L1357-L1367

we now explicitly construct an array of the k'th neighbor distance for each point, and then derive bandwidths later on.  If a point does not have any neighbors, an exception is raised.  This is correct and necessary if we're doing individual or pairwise bandwidth estimates (eg `mean_k` or `gmean_k`), but I think the requirement is too strict if we're doing global bandwidth estimation.


**To Reproduce**

Behavior is triggered by our segmentation example in the advanced gallery, resulting in the following:

```
Unexpected failing examples:
/home/bmcfee/git/librosa/docs/examples/plot_segmentation.py failed leaving traceback:
Traceback (most recent call last):
  File "/home/bmcfee/git/librosa/docs/examples/plot_segmentation.py", line 77, in <module>
    R = librosa.segment.recurrence_matrix(Csync, width=3, mode='affinity',
  File "/home/bmcfee/git/librosa/librosa/segment.py", line 685, in recurrence_matrix
    aff_bandwidth = __affinity_bandwidth(rec, bandwidth, bandwidth_k)
  File "/home/bmcfee/git/librosa/librosa/segment.py", line 1363, in __affinity_bandwidth
    raise ParameterError("The sample at time point {} has no neighbors".format(i))
librosa.util.exceptions.ParameterError: The sample at time point 0 has no neighbors
```

**Expected behavior**

I think that if we're estimating scalar bandwidths, we can support empty rows in the recurrence matrix.  In this case, we can fill in `nan` if the link set is empty, and then use nan-friendly estimators if we're producing a scalar bandwidth.

Does this sound reasonable?  If so, I'll make a quick PR for it.

Regression: bandwidth estimation now requires all vertices to have edges
**Describe the bug**

In 0.9 and earlier, the bandwidth for `affinity` mode in recurrence matrix calculation was inferred as the median distance to the k'th nearest neighbor of each data point:
https://github.com/librosa/librosa/blob/b9830b4a335b1e7e4be66ee81870573bf3d516d0/librosa/segment.py#L524-L526

Because we were using `nanmedian(rec.max(axis=1))`, any empty rows of `rec` would be implicitly ignored in the median calculation.  This, I believe, is a good and correct thing.

After merging #1495 , the behavior has changed:
https://github.com/librosa/librosa/blob/e18d544cbebfb590da34710ba2de86292179ec39/librosa/segment.py#L1357-L1367

we now explicitly construct an array of the k'th neighbor distance for each point, and then derive bandwidths later on.  If a point does not have any neighbors, an exception is raised.  This is correct and necessary if we're doing individual or pairwise bandwidth estimates (eg `mean_k` or `gmean_k`), but I think the requirement is too strict if we're doing global bandwidth estimation.


**To Reproduce**

Behavior is triggered by our segmentation example in the advanced gallery, resulting in the following:

```
Unexpected failing examples:
/home/bmcfee/git/librosa/docs/examples/plot_segmentation.py failed leaving traceback:
Traceback (most recent call last):
  File "/home/bmcfee/git/librosa/docs/examples/plot_segmentation.py", line 77, in <module>
    R = librosa.segment.recurrence_matrix(Csync, width=3, mode='affinity',
  File "/home/bmcfee/git/librosa/librosa/segment.py", line 685, in recurrence_matrix
    aff_bandwidth = __affinity_bandwidth(rec, bandwidth, bandwidth_k)
  File "/home/bmcfee/git/librosa/librosa/segment.py", line 1363, in __affinity_bandwidth
    raise ParameterError("The sample at time point {} has no neighbors".format(i))
librosa.util.exceptions.ParameterError: The sample at time point 0 has no neighbors
```

**Expected behavior**

I think that if we're estimating scalar bandwidths, we can support empty rows in the recurrence matrix.  In this case, we can fill in `nan` if the link set is empty, and then use nan-friendly estimators if we're producing a scalar bandwidth.

Does this sound reasonable?  If so, I'll make a quick PR for it.



Discussion:

Resolving patch:
diff --git a/librosa/segment.py b/librosa/segment.py
--- a/librosa/segment.py
+++ b/librosa/segment.py
@@ -289,7 +289,7 @@ def cross_similarity(
 
     # using k for bandwidth estimation also and decouple k for full mode
     bandwidth_k = k
-    if full and (mode != 'connectivity'):
+    if full and (mode != "connectivity"):
         k = n
 
     # Build the neighbor search object
@@ -614,7 +614,7 @@ def recurrence_matrix(
 
     # using k for bandwidth estimation also and decouple k for full mode
     bandwidth_k = k
-    if full and (mode != 'connectivity'):
+    if full and (mode != "connectivity"):
         k = t
 
     # Build the neighbor search object
@@ -1315,7 +1315,11 @@ def path_enhance(
     return np.asanyarray(R_smooth)
 
 
-def __affinity_bandwidth(rec: scipy.sparse.csr_matrix, bw_mode: Optional[Union[np.ndarray, _FloatLike_co, str]], k: int) -> Union[float, np.ndarray]:
+def __affinity_bandwidth(
+    rec: scipy.sparse.csr_matrix,
+    bw_mode: Optional[Union[np.ndarray, _FloatLike_co, str]],
+    k: int,
+) -> Union[float, np.ndarray]:
     # rec should be a csr_matrix
 
     # the api allows users to specify a scalar bandwidth directly, besides the string based options.
@@ -1337,14 +1341,23 @@ def __affinity_bandwidth(rec: scipy.sparse.csr_matrix, bw_mode: Optional[Union[n
         scalar_bandwidth = float(bw_mode)
         if scalar_bandwidth <= 0:
             raise ParameterError(
-                "Invalid scalar bandwidth={}. Must be strictly positive.".format(scalar_bandwidth)
+                "Invalid scalar bandwidth={}. Must be strictly positive.".format(
+                    scalar_bandwidth
+                )
             )
         return scalar_bandwidth
 
     if bw_mode is None:
-        bw_mode = 'med_k_scalar'
-
-    if bw_mode not in ['med_k_scalar', 'mean_k', 'gmean_k', 'mean_k_avg', 'gmean_k_avg', 'mean_k_avg_and_pair']:
+        bw_mode = "med_k_scalar"
+
+    if bw_mode not in [
+        "med_k_scalar",
+        "mean_k",
+        "gmean_k",
+        "mean_k_avg",
+        "gmean_k_avg",
+        "mean_k_avg_and_pair",
+    ]:
         raise ParameterError(
             (
                 "Invalid bandwidth='{}'. Must be either a positive scalar or one of "
@@ -1360,47 +1373,59 @@ def __affinity_bandwidth(rec: scipy.sparse.csr_matrix, bw_mode: Optional[Union[n
         links = rec[i].nonzero()[1]
         # catch empty dists lists in knn_dists
         if len(links) == 0:
-            raise ParameterError("The sample at time point {} has no neighbors".format(i))
-
-        # Compute k nearest neighbors' distance and sort ascending
-        knn_dist_row = np.sort(rec[i, links].toarray()[0])[:k]
-        knn_dists.append(knn_dist_row)
+            # Disconnected vertices are only a problem for point-wise bandwidth estimation
+            if bw_mode not in ["med_k_scalar"]:
+                raise ParameterError(f"The sample at time point {i} has no neighbors")
+            else:
+                # If we have no links, then there's no distance
+                # shove a nan in here
+                knn_dists.append(np.array([np.nan]))
+        else:
+            # Compute k nearest neighbors' distance and sort ascending
+            knn_dist_row = np.sort(rec[i, links].toarray()[0])[:k]
+            knn_dists.append(knn_dist_row)
 
     # take the last element of each list for the distance to kth neighbor
     dist_to_k = np.asarray([dists[-1] for dists in knn_dists])
     avg_dist_to_first_ks = np.asarray([np.mean(dists) for dists in knn_dists])
 
-    if bw_mode == 'med_k_scalar':
+    if bw_mode == "med_k_scalar":
+        if not np.any(np.isfinite(dist_to_k)):
+            raise ParameterError("Cannot estimate bandwidth from an empty graph")
         return float(np.nanmedian(dist_to_k))
 
-    if bw_mode in ['mean_k', 'gmean_k']:
+    if bw_mode in ["mean_k", "gmean_k"]:
         # building bandwidth components (sigma) using sparse matrix structures and indices
         sigma_i_data = np.empty_like(rec.data)
         sigma_j_data = np.empty_like(rec.data)
         for row in range(t):
-            sigma_i_data[rec.indptr[row]:rec.indptr[row + 1]] = dist_to_k[row]
-            col_idx = rec.indices[rec.indptr[row]:rec.indptr[row + 1]]
-            sigma_j_data[rec.indptr[row]:rec.indptr[row + 1]] = dist_to_k[col_idx]
+            sigma_i_data[rec.indptr[row] : rec.indptr[row + 1]] = dist_to_k[row]
+            col_idx = rec.indices[rec.indptr[row] : rec.indptr[row + 1]]
+            sigma_j_data[rec.indptr[row] : rec.indptr[row + 1]] = dist_to_k[col_idx]
 
-        if bw_mode == 'mean_k':
+        if bw_mode == "mean_k":
             out = np.array((sigma_i_data + sigma_j_data) / 2)
-        elif bw_mode == 'gmean_k':
+        elif bw_mode == "gmean_k":
             out = np.array((sigma_i_data * sigma_j_data) ** 0.5)
 
-    if bw_mode in ['mean_k_avg', 'gmean_k_avg', 'mean_k_avg_and_pair']:
+    if bw_mode in ["mean_k_avg", "gmean_k_avg", "mean_k_avg_and_pair"]:
         # building bandwidth components (sigma) using sparse matrix structures and indices
         sigma_i_data = np.empty_like(rec.data)
         sigma_j_data = np.empty_like(rec.data)
         for row in range(t):
-            sigma_i_data[rec.indptr[row]:rec.indptr[row + 1]] = avg_dist_to_first_ks[row]
-            col_idx = rec.indices[rec.indptr[row]:rec.indptr[row + 1]]
-            sigma_j_data[rec.indptr[row]:rec.indptr[row + 1]] = avg_dist_to_first_ks[col_idx]
-
-        if bw_mode == 'mean_k_avg':
+            sigma_i_data[rec.indptr[row] : rec.indptr[row + 1]] = avg_dist_to_first_ks[
+                row
+            ]
+            col_idx = rec.indices[rec.indptr[row] : rec.indptr[row + 1]]
+            sigma_j_data[rec.indptr[row] : rec.indptr[row + 1]] = avg_dist_to_first_ks[
+                col_idx
+            ]
+
+        if bw_mode == "mean_k_avg":
             out = np.array((sigma_i_data + sigma_j_data) / 2)
-        elif bw_mode == 'gmean_k_avg':
+        elif bw_mode == "gmean_k_avg":
             out = np.array((sigma_i_data * sigma_j_data) ** 0.5)
-        elif bw_mode == 'mean_k_avg_and_pair':
+        elif bw_mode == "mean_k_avg_and_pair":
             out = np.array((sigma_i_data + sigma_j_data + rec.data) / 3)
-    
-    return out
\ No newline at end of file
+
+    return out


Test patch:
diff --git a/tests/test_segment.py b/tests/test_segment.py
--- a/tests/test_segment.py
+++ b/tests/test_segment.py
@@ -285,6 +285,18 @@ def test_empty_data_recurrence():
     librosa.segment.recurrence_matrix(data, mode="affinity")
 
 
+@pytest.mark.xfail(raises=librosa.ParameterError)
+def test_empty_rows_recurrence():
+    data = np.zeros((10, 10))
+    data[0, 5] = 1
+    librosa.segment.recurrence_matrix(data, mode="affinity", bandwidth="mean_k")
+
+def test_empty_rows_recurrence_okay():
+    data = np.zeros((10, 10))
+    data[0, 5] = 1
+    librosa.segment.recurrence_matrix(data, mode="affinity", bandwidth="med_k_scalar")
+
+
 def test_recurrence_multi():
     srand()
     X = np.random.randn(2, 10, 100)