Repository: librosa/librosa

Issue:
Numpy array truthiness error during effects.split
When loading a file and trying to run librosa.effects.split() on it, I get this error:

```
  File "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/librosa/effects.py", line 574, in split
    if non_silent[0]:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```


**To Reproduce**
```
import librosa
import numpy as np

y, sr = librosa.load("path/to/file.mp3", sr=44100, mono=False)
intervals = librosa.effects.split(y, top_db=22, ref=np.max, frame_length=44100, hop_length=44100)
```

**Expected behavior**
The split effect returning an array of non-silent intervals.

**Software versions***
```
INSTALLED VERSIONS
------------------
python: 3.8.12 (default, Oct 13 2021, 06:42:42) 
[Clang 13.0.0 (clang-1300.0.29.3)]

librosa: 0.9.1

audioread: 2.1.9
numpy: 1.22.4
scipy: 1.8.1
sklearn: 1.1.1
joblib: 1.1.0
decorator: 5.1.1
soundfile: 0.10.3
resampy: 0.2.2
numba: 0.55.2

numpydoc: None
sphinx: None
sphinx_rtd_theme: None
sphinxcontrib.versioning: None
sphinx-gallery: None
pytest: None
pytest-mpl: None
pytest-cov: None
matplotlib: None
presets: None
```

**Additional context**
This is a file I haven't touched in a while, so I apologize if it is something that is covered in a changelog somewhere. However, I was unable find any similar issues.

Numpy array truthiness error during effects.split
When loading a file and trying to run librosa.effects.split() on it, I get this error:

```
  File "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/librosa/effects.py", line 574, in split
    if non_silent[0]:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```


**To Reproduce**
```
import librosa
import numpy as np

y, sr = librosa.load("path/to/file.mp3", sr=44100, mono=False)
intervals = librosa.effects.split(y, top_db=22, ref=np.max, frame_length=44100, hop_length=44100)
```

**Expected behavior**
The split effect returning an array of non-silent intervals.

**Software versions***
```
INSTALLED VERSIONS
------------------
python: 3.8.12 (default, Oct 13 2021, 06:42:42) 
[Clang 13.0.0 (clang-1300.0.29.3)]

librosa: 0.9.1

audioread: 2.1.9
numpy: 1.22.4
scipy: 1.8.1
sklearn: 1.1.1
joblib: 1.1.0
decorator: 5.1.1
soundfile: 0.10.3
resampy: 0.2.2
numba: 0.55.2

numpydoc: None
sphinx: None
sphinx_rtd_theme: None
sphinxcontrib.versioning: None
sphinx-gallery: None
pytest: None
pytest-mpl: None
pytest-cov: None
matplotlib: None
presets: None
```

**Additional context**
This is a file I haven't touched in a while, so I apologize if it is something that is covered in a changelog somewhere. However, I was unable find any similar issues.



Discussion:
Thanks for reporting!  I think I understand what's going on here.  The offending code segments are:
https://github.com/librosa/librosa/blob/a19da5075663321d49afd9beca574e4ee527f5b7/librosa/effects.py#L573-L579

There are two things wrong with this when applied to multi-channel inputs:

1. `non_silent[0]` is an array, not a scalar, so we need to aggregate somehow.  This is suggested by the error message in the initial post; I would expect `any()` to be the correct aggregator here, as non-silence should be disjunctive across channels.
2. The array index should be trailing, not leading.  That is, we should have `non_silent[..., 0]`.  

Of course, this also applies to the trailing frame logic, `non_silent[..., -1]`.

I'm not sure how this one slipped past our CR for 0.9, but I guess it did.

@timlundy  Can you confirm that this error goes away if you load with `mono=True`?


Using `mono=True` does solve the issue. Thanks so much for looking into it!
Great, thanks for confirming.

Looking into more detail here, I don't think the resolution is exactly as I suggested above.

When we generalized trim and split to support multichannel #1369 we made the decision to aggregate silence detection across all channels, disjunctively by default using `aggregate=np.max`, so that `non_silent` should always be a 1d array.  This is precisely what the helper function https://github.com/librosa/librosa/blob/a19da5075663321d49afd9beca574e4ee527f5b7/librosa/effects.py#L403 does.

Testing this out with a minimal reproducible example:
```python
In [1]: import librosa

In [2]: import numpy as np

In [4]: y, sr = librosa.load(librosa.ex('nutcracker', hq=True), mono=False, sr=None)

In [5]: librosa.effects.split(y=y, sr=sr)

```
does trigger the error.  Digging under the hood, we can call the helper function directly:
```python
In [8]: librosa.effects._signal_to_frame_nonsilent(y)
Out[8]: array([[False, False, False, ..., False, False, False]])
```
and note that the output is of size `(1, n_frames)`.  This is a result of the `apply_along_axis` call that we make in the helper:
https://github.com/librosa/librosa/blob/a19da5075663321d49afd9beca574e4ee527f5b7/librosa/effects.py#L445-L447

If we replace that call by slicing in different ways, we get predictable behavior
```python
In [9]: librosa.effects._signal_to_frame_nonsilent(y[:1])  # Still 2d input
Out[9]: array([[False, False, False, ..., False, False, False]])

In [10]: librosa.effects._signal_to_frame_nonsilent(y[0])  # 1d input
Out[10]: array([False, False, False, ..., False, False, False])

In [11]: librosa.effects._signal_to_frame_nonsilent(y[np.newaxis, ...])  # 3d input
Out[11]: array([[[False, False, False, ..., False, False, False]]])
```
We can also try it with a non-trivial expansion tiling `y` in a leading dimension:
```python
In [18]: y_copy = np.tile(y, (4, 1, 1))

In [19]: y_copy.shape
Out[19]: (4, 2, 5286528)

In [20]: librosa.effects._signal_to_frame_nonsilent(y_copy)
Out[20]: array([[[False, False, False, ..., False, False, False]]])
```
and still get a `(1, 1, N)` output.

I think the solution here is simply to do `np.squeeze` in the helper and everything else should work itself out naturally.  We should also put in some unit tests to prevent this from recurring.  Ditto we should cover `trim` as well.
Thanks for reporting!  I think I understand what's going on here.  The offending code segments are:
https://github.com/librosa/librosa/blob/a19da5075663321d49afd9beca574e4ee527f5b7/librosa/effects.py#L573-L579

There are two things wrong with this when applied to multi-channel inputs:

1. `non_silent[0]` is an array, not a scalar, so we need to aggregate somehow.  This is suggested by the error message in the initial post; I would expect `any()` to be the correct aggregator here, as non-silence should be disjunctive across channels.
2. The array index should be trailing, not leading.  That is, we should have `non_silent[..., 0]`.  

Of course, this also applies to the trailing frame logic, `non_silent[..., -1]`.

I'm not sure how this one slipped past our CR for 0.9, but I guess it did.

@timlundy  Can you confirm that this error goes away if you load with `mono=True`?


Using `mono=True` does solve the issue. Thanks so much for looking into it!
Great, thanks for confirming.

Looking into more detail here, I don't think the resolution is exactly as I suggested above.

When we generalized trim and split to support multichannel #1369 we made the decision to aggregate silence detection across all channels, disjunctively by default using `aggregate=np.max`, so that `non_silent` should always be a 1d array.  This is precisely what the helper function https://github.com/librosa/librosa/blob/a19da5075663321d49afd9beca574e4ee527f5b7/librosa/effects.py#L403 does.

Testing this out with a minimal reproducible example:
```python
In [1]: import librosa

In [2]: import numpy as np

In [4]: y, sr = librosa.load(librosa.ex('nutcracker', hq=True), mono=False, sr=None)

In [5]: librosa.effects.split(y=y, sr=sr)

```
does trigger the error.  Digging under the hood, we can call the helper function directly:
```python
In [8]: librosa.effects._signal_to_frame_nonsilent(y)
Out[8]: array([[False, False, False, ..., False, False, False]])
```
and note that the output is of size `(1, n_frames)`.  This is a result of the `apply_along_axis` call that we make in the helper:
https://github.com/librosa/librosa/blob/a19da5075663321d49afd9beca574e4ee527f5b7/librosa/effects.py#L445-L447

If we replace that call by slicing in different ways, we get predictable behavior
```python
In [9]: librosa.effects._signal_to_frame_nonsilent(y[:1])  # Still 2d input
Out[9]: array([[False, False, False, ..., False, False, False]])

In [10]: librosa.effects._signal_to_frame_nonsilent(y[0])  # 1d input
Out[10]: array([False, False, False, ..., False, False, False])

In [11]: librosa.effects._signal_to_frame_nonsilent(y[np.newaxis, ...])  # 3d input
Out[11]: array([[[False, False, False, ..., False, False, False]]])
```
We can also try it with a non-trivial expansion tiling `y` in a leading dimension:
```python
In [18]: y_copy = np.tile(y, (4, 1, 1))

In [19]: y_copy.shape
Out[19]: (4, 2, 5286528)

In [20]: librosa.effects._signal_to_frame_nonsilent(y_copy)
Out[20]: array([[[False, False, False, ..., False, False, False]]])
```
and still get a `(1, 1, N)` output.

I think the solution here is simply to do `np.squeeze` in the helper and everything else should work itself out naturally.  We should also put in some unit tests to prevent this from recurring.  Ditto we should cover `trim` as well.

Resolving patch:
diff --git a/librosa/effects.py b/librosa/effects.py
--- a/librosa/effects.py
+++ b/librosa/effects.py
@@ -445,6 +445,9 @@ def _signal_to_frame_nonsilent(
     # Aggregate everything but the time dimension
     if db.ndim > 1:
         db = np.apply_over_axes(aggregate, db, range(db.ndim - 1))
+        # Squeeze out leading singleton dimensions here
+        # We always want to keep the trailing dimension though
+        db = np.squeeze(db, axis=tuple(range(db.ndim - 1)))
 
     return db > -top_db
 


Test patch:
diff --git a/tests/test_effects.py b/tests/test_effects.py
--- a/tests/test_effects.py
+++ b/tests/test_effects.py
@@ -252,6 +252,19 @@ def test_trim_empty():
     assert idx[1] == 0
 
 
+def test_trim_multi(y_multi):
+    # Test for https://github.com/librosa/librosa/issues/1489
+    y, sr = y_multi
+    librosa.effects.trim(y=y)
+
+
+def test_split_multi(y_multi):
+    # Test for https://github.com/librosa/librosa/issues/1489
+    y, sr = y_multi
+
+    librosa.effects.split(y=y)
+
+
 @pytest.fixture(
     scope="module",
     params=[0, 1, 2, 3],