Repository: statsmodels/statsmodels

Issue:
select_bandwidth raising ValueError in version 0.11.0 for multi-dimensional input
#### Describe the bug

After updating to version 0.11.0, `statsmodels.nonparametric.bandwidths.select_bandwidth()` raises a ValueError due to the addition of PR #6292 when the input array `x` is multi-dimensional. I believe it is due to the following [lines](https://github.com/statsmodels/statsmodels/blob/033d56de5b7036bc763764b2cf5e903091d6916e/statsmodels/nonparametric/bandwidths.py#L171) in `bandwidths.py`.

```python
...
bandwidth = bandwidth_funcs[bw](x, kernel)
if bandwidth == 0:
...
```

because if `x` is multi-dimensional, the output of the bandwidth function is an array. This should be a simple fix but the question is whether PR #6292 wanted to check if *all* or *any* of the values were zero.

#### Code Sample

```python
from statsmodels.nonparametric.bandwidths import select_bandwidth

x = [[1, 2, 3], [4, 5, 6]]
bw = select_bandwidth(x, 'scott', None)
print(bw)
```

this produces the following exception:

```none
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-10-d22be3b83d7d> in <module>
      1 x = [[1, 2, 3], [4, 5, 6]]
      2 
----> 3 bw = select_bandwidth(x, 'scott', None)

~/.virtualenvs/base/lib/python3.7/site-packages/statsmodels/nonparametric/bandwidths.py in select_bandwidth(x, bw, kernel)
    169         raise ValueError("Bandwidth %s not understood" % bw)
    170 #TODO: uncomment checks when we have non-rule of thumb bandwidths for diff. kernels
--> 171 #    if kernel == "gauss":
    172     return bandwidth_funcs[bw](x, kernel)
    173 #    else:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

<details>

**Note**: As you can see, there are many issues on our GitHub tracker, so it is very possible that your issue has been posted before. Please check first before submitting so that we do not have to handle and close duplicates.

**Note**: Please be sure you are using the latest released version of `statsmodels`, or a recent build of `master`. If your problem has been fixed in an unreleased version, you might be able to use `master` until a new release occurs. 

**Note**: If you are using a released version, have you verified that the bug exists in the master branch of this repository? It helps the limited resources if we know problems exist in the current master so that they do not need to check whether the code sample produces a bug in the next release.

</details>


If the issue has not been resolved, please file it in the issue tracker.

#### Expected Output

```none
[1.70851195 1.70851195 1.70851195]
```

#### Output of ``import statsmodels.api as sm; sm.show_versions()``

<details>

[paste the output of ``import statsmodels.api as sm; sm.show_versions()`` here below this line]

INSTALLED VERSIONS
------------------
Python: 3.7.4.final.0
OS: Darwin 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8

statsmodels
===========

Installed: 0.11.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/statsmodels)

Required Dependencies
=====================

cython: 0.29.13 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/Cython)
numpy: 1.17.3 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/numpy)
scipy: 1.3.1 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/scipy)
pandas: 0.25.1 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/pandas)
    dateutil: 2.8.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/dateutil)
patsy: 0.5.1 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/patsy)

Optional Dependencies
=====================

matplotlib: 3.1.3 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/matplotlib)
    backend: module://ipykernel.pylab.backend_inline 
cvxopt: Not installed
joblib: 0.14.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/joblib)

Developer Tools
================

IPython: 7.8.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/IPython)
    jinja2: 2.10.3 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/jinja2)
sphinx: 2.2.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/sphinx)
    pygments: 2.4.2 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/pygments)
pytest: Not installed
virtualenv: Not installed

</details>

select_bandwidth raising ValueError in version 0.11.0 for multi-dimensional input
#### Describe the bug

After updating to version 0.11.0, `statsmodels.nonparametric.bandwidths.select_bandwidth()` raises a ValueError due to the addition of PR #6292 when the input array `x` is multi-dimensional. I believe it is due to the following [lines](https://github.com/statsmodels/statsmodels/blob/033d56de5b7036bc763764b2cf5e903091d6916e/statsmodels/nonparametric/bandwidths.py#L171) in `bandwidths.py`.

```python
...
bandwidth = bandwidth_funcs[bw](x, kernel)
if bandwidth == 0:
...
```

because if `x` is multi-dimensional, the output of the bandwidth function is an array. This should be a simple fix but the question is whether PR #6292 wanted to check if *all* or *any* of the values were zero.

#### Code Sample

```python
from statsmodels.nonparametric.bandwidths import select_bandwidth

x = [[1, 2, 3], [4, 5, 6]]
bw = select_bandwidth(x, 'scott', None)
print(bw)
```

this produces the following exception:

```none
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-10-d22be3b83d7d> in <module>
      1 x = [[1, 2, 3], [4, 5, 6]]
      2 
----> 3 bw = select_bandwidth(x, 'scott', None)

~/.virtualenvs/base/lib/python3.7/site-packages/statsmodels/nonparametric/bandwidths.py in select_bandwidth(x, bw, kernel)
    169         raise ValueError("Bandwidth %s not understood" % bw)
    170 #TODO: uncomment checks when we have non-rule of thumb bandwidths for diff. kernels
--> 171 #    if kernel == "gauss":
    172     return bandwidth_funcs[bw](x, kernel)
    173 #    else:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

<details>

**Note**: As you can see, there are many issues on our GitHub tracker, so it is very possible that your issue has been posted before. Please check first before submitting so that we do not have to handle and close duplicates.

**Note**: Please be sure you are using the latest released version of `statsmodels`, or a recent build of `master`. If your problem has been fixed in an unreleased version, you might be able to use `master` until a new release occurs. 

**Note**: If you are using a released version, have you verified that the bug exists in the master branch of this repository? It helps the limited resources if we know problems exist in the current master so that they do not need to check whether the code sample produces a bug in the next release.

</details>


If the issue has not been resolved, please file it in the issue tracker.

#### Expected Output

```none
[1.70851195 1.70851195 1.70851195]
```

#### Output of ``import statsmodels.api as sm; sm.show_versions()``

<details>

[paste the output of ``import statsmodels.api as sm; sm.show_versions()`` here below this line]

INSTALLED VERSIONS
------------------
Python: 3.7.4.final.0
OS: Darwin 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8

statsmodels
===========

Installed: 0.11.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/statsmodels)

Required Dependencies
=====================

cython: 0.29.13 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/Cython)
numpy: 1.17.3 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/numpy)
scipy: 1.3.1 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/scipy)
pandas: 0.25.1 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/pandas)
    dateutil: 2.8.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/dateutil)
patsy: 0.5.1 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/patsy)

Optional Dependencies
=====================

matplotlib: 3.1.3 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/matplotlib)
    backend: module://ipykernel.pylab.backend_inline 
cvxopt: Not installed
joblib: 0.14.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/joblib)

Developer Tools
================

IPython: 7.8.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/IPython)
    jinja2: 2.10.3 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/jinja2)
sphinx: 2.2.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/sphinx)
    pygments: 2.4.2 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/pygments)
pytest: Not installed
virtualenv: Not installed

</details>

select_bandwidth raising ValueError in version 0.11.0 for multi-dimensional input
#### Describe the bug

After updating to version 0.11.0, `statsmodels.nonparametric.bandwidths.select_bandwidth()` raises a ValueError due to the addition of PR #6292 when the input array `x` is multi-dimensional. I believe it is due to the following [lines](https://github.com/statsmodels/statsmodels/blob/033d56de5b7036bc763764b2cf5e903091d6916e/statsmodels/nonparametric/bandwidths.py#L171) in `bandwidths.py`.

```python
...
bandwidth = bandwidth_funcs[bw](x, kernel)
if bandwidth == 0:
...
```

because if `x` is multi-dimensional, the output of the bandwidth function is an array. This should be a simple fix but the question is whether PR #6292 wanted to check if *all* or *any* of the values were zero.

#### Code Sample

```python
from statsmodels.nonparametric.bandwidths import select_bandwidth

x = [[1, 2, 3], [4, 5, 6]]
bw = select_bandwidth(x, 'scott', None)
print(bw)
```

this produces the following exception:

```none
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-10-d22be3b83d7d> in <module>
      1 x = [[1, 2, 3], [4, 5, 6]]
      2 
----> 3 bw = select_bandwidth(x, 'scott', None)

~/.virtualenvs/base/lib/python3.7/site-packages/statsmodels/nonparametric/bandwidths.py in select_bandwidth(x, bw, kernel)
    169         raise ValueError("Bandwidth %s not understood" % bw)
    170 #TODO: uncomment checks when we have non-rule of thumb bandwidths for diff. kernels
--> 171 #    if kernel == "gauss":
    172     return bandwidth_funcs[bw](x, kernel)
    173 #    else:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

<details>

**Note**: As you can see, there are many issues on our GitHub tracker, so it is very possible that your issue has been posted before. Please check first before submitting so that we do not have to handle and close duplicates.

**Note**: Please be sure you are using the latest released version of `statsmodels`, or a recent build of `master`. If your problem has been fixed in an unreleased version, you might be able to use `master` until a new release occurs. 

**Note**: If you are using a released version, have you verified that the bug exists in the master branch of this repository? It helps the limited resources if we know problems exist in the current master so that they do not need to check whether the code sample produces a bug in the next release.

</details>


If the issue has not been resolved, please file it in the issue tracker.

#### Expected Output

```none
[1.70851195 1.70851195 1.70851195]
```

#### Output of ``import statsmodels.api as sm; sm.show_versions()``

<details>

[paste the output of ``import statsmodels.api as sm; sm.show_versions()`` here below this line]

INSTALLED VERSIONS
------------------
Python: 3.7.4.final.0
OS: Darwin 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8

statsmodels
===========

Installed: 0.11.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/statsmodels)

Required Dependencies
=====================

cython: 0.29.13 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/Cython)
numpy: 1.17.3 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/numpy)
scipy: 1.3.1 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/scipy)
pandas: 0.25.1 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/pandas)
    dateutil: 2.8.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/dateutil)
patsy: 0.5.1 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/patsy)

Optional Dependencies
=====================

matplotlib: 3.1.3 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/matplotlib)
    backend: module://ipykernel.pylab.backend_inline 
cvxopt: Not installed
joblib: 0.14.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/joblib)

Developer Tools
================

IPython: 7.8.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/IPython)
    jinja2: 2.10.3 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/jinja2)
sphinx: 2.2.0 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/sphinx)
    pygments: 2.4.2 (/Users/ajl573/.virtualenvs/base/lib/python3.7/site-packages/pygments)
pytest: Not installed
virtualenv: Not installed

</details>



Discussion:
Clearly a bug.  I wrongly assumed that BW was scalar.
Thanks for the speedy response. It looks like a fairly easy fix at least, I'd imagine that if any of `bandwidth` is zero then that will cause problems in `KDEMultivariate` (which is what I am determining the bandwidth for), similarly to how `bandwidth == 0` cannot work with `KDEUnivariate`.
Just needs and `np.any` wrapped around the `==` test.
I notice that you are looking for contributors, I'd be happy to sort this one as it's straightforward and a chance to learn how to contribute to statsmodels (I have only contributed to smaller projects so far but am keen to get more experience helping open source projects). If that's fine by you, I'll read the guidelines and submit a pull request.
Please do!

On Sat, Feb 15, 2020, 12:34 Alex Lyttle <notifications@github.com> wrote:

> I notice that you are looking for contributors, I'd be happy to sort this
> one as it's straightforward and a chance to learn how to contribute to
> statsmodels (I have only contributed to smaller projects so far but am keen
> to get more experience helping open source projects). If that's fine by
> you, I'll read the guidelines and submit a pull request.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/statsmodels/statsmodels/issues/6511?email_source=notifications&email_token=ABKTSRMZYTK5BKLSRGR7M3LRC7OOZA5CNFSM4KVL2IX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEL3JPJQ#issuecomment-586586022>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABKTSRK6O7DDQEH7EENDNPTRC7OOZANCNFSM4KVL2IXQ>
> .
>

Clearly a bug.  I wrongly assumed that BW was scalar.
Thanks for the speedy response. It looks like a fairly easy fix at least, I'd imagine that if any of `bandwidth` is zero then that will cause problems in `KDEMultivariate` (which is what I am determining the bandwidth for), similarly to how `bandwidth == 0` cannot work with `KDEUnivariate`.
Just needs and `np.any` wrapped around the `==` test.
I notice that you are looking for contributors, I'd be happy to sort this one as it's straightforward and a chance to learn how to contribute to statsmodels (I have only contributed to smaller projects so far but am keen to get more experience helping open source projects). If that's fine by you, I'll read the guidelines and submit a pull request.
Please do!

On Sat, Feb 15, 2020, 12:34 Alex Lyttle <notifications@github.com> wrote:

> I notice that you are looking for contributors, I'd be happy to sort this
> one as it's straightforward and a chance to learn how to contribute to
> statsmodels (I have only contributed to smaller projects so far but am keen
> to get more experience helping open source projects). If that's fine by
> you, I'll read the guidelines and submit a pull request.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/statsmodels/statsmodels/issues/6511?email_source=notifications&email_token=ABKTSRMZYTK5BKLSRGR7M3LRC7OOZA5CNFSM4KVL2IX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEL3JPJQ#issuecomment-586586022>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABKTSRK6O7DDQEH7EENDNPTRC7OOZANCNFSM4KVL2IXQ>
> .
>

Clearly a bug.  I wrongly assumed that BW was scalar.
Thanks for the speedy response. It looks like a fairly easy fix at least, I'd imagine that if any of `bandwidth` is zero then that will cause problems in `KDEMultivariate` (which is what I am determining the bandwidth for), similarly to how `bandwidth == 0` cannot work with `KDEUnivariate`.
Just needs and `np.any` wrapped around the `==` test.
I notice that you are looking for contributors, I'd be happy to sort this one as it's straightforward and a chance to learn how to contribute to statsmodels (I have only contributed to smaller projects so far but am keen to get more experience helping open source projects). If that's fine by you, I'll read the guidelines and submit a pull request.
Please do!

On Sat, Feb 15, 2020, 12:34 Alex Lyttle <notifications@github.com> wrote:

> I notice that you are looking for contributors, I'd be happy to sort this
> one as it's straightforward and a chance to learn how to contribute to
> statsmodels (I have only contributed to smaller projects so far but am keen
> to get more experience helping open source projects). If that's fine by
> you, I'll read the guidelines and submit a pull request.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/statsmodels/statsmodels/issues/6511?email_source=notifications&email_token=ABKTSRMZYTK5BKLSRGR7M3LRC7OOZA5CNFSM4KVL2IX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEL3JPJQ#issuecomment-586586022>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABKTSRK6O7DDQEH7EENDNPTRC7OOZANCNFSM4KVL2IXQ>
> .
>


Resolving patch:
diff --git a/statsmodels/nonparametric/bandwidths.py b/statsmodels/nonparametric/bandwidths.py
--- a/statsmodels/nonparametric/bandwidths.py
+++ b/statsmodels/nonparametric/bandwidths.py
@@ -168,7 +168,7 @@ def select_bandwidth(x, bw, kernel):
     if bw not in bandwidth_funcs:
         raise ValueError("Bandwidth %s not understood" % bw)
     bandwidth = bandwidth_funcs[bw](x, kernel)
-    if bandwidth == 0:
+    if np.any(bandwidth == 0):
         # eventually this can fall back on another selection criterion.
         err = "Selected KDE bandwidth is 0. Cannot estimate density."
         raise RuntimeError(err)


Test patch:
diff --git a/statsmodels/nonparametric/tests/test_bandwidths.py b/statsmodels/nonparametric/tests/test_bandwidths.py
--- a/statsmodels/nonparametric/tests/test_bandwidths.py
+++ b/statsmodels/nonparametric/tests/test_bandwidths.py
@@ -15,6 +15,7 @@
 
 
 from numpy.testing import assert_allclose
+import pytest
 
 # setup test data
 
@@ -70,3 +71,25 @@ class TestTriweight(CheckNormalReferenceConstant):
 
     kern = kernels.Triweight()
     constant = 3.15
+
+
+class BandwidthZero(object):
+
+    def test_bandwidth_zero(self):
+
+        kern = kernels.Gaussian()
+        for bw in ['scott', 'silverman', 'normal_reference']:
+            with pytest.raises(RuntimeError,
+                               match="Selected KDE bandwidth is 0"):
+                select_bandwidth(self.xx, bw, kern)
+
+
+class TestAllBandwidthZero(BandwidthZero):
+
+    xx = np.ones((100, 3))
+
+
+class TestAnyBandwidthZero(BandwidthZero):
+
+    xx = np.random.normal(size=(100, 3))
+    xx[:, 0] = 1.0