Repository: pandas-dev/pandas
Pull request: BUG: Fixing regression in DataFrame.all and DataFrame.any with bool_only=True
* `bool_only` parameter is supported again
* Commit 36ab8c95a41bbadc286a5a520883b149c86931bd created this regression
  due to a bug in all/any (pandas-dev/pandas#23070)
* Reverted the regression and fixed the bug with a condition
* Added tests for `bool_only` parameter

- [x] closes #25101 
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Problem statement:
all/any ignoring `bool_only` parameter
#### Code Sample, a copy-pastable example if possible

```python
# Your code here
import pandas as pd
print(pd.DataFrame({"col1": [1,2,3], "col2": [4,5,6], "col3": [None, None, None]}).all(bool_only=True))
print(pd.DataFrame({"col1": [1,2,3], "col2": [4,5,6], "col3": [None, None, None], "col4":[False, False, True]}).all(bool_only=True))
```
#### Problem description

`DataFrame.all` and `DataFrame.any` are both ignoring `bool_only` parameter. It does not seem to matter if there are booleans in the input or not.

This was a regression in 0.24.0. On 0.23.4 it was working, expected output from code sample below.

#### Expected Output

Expected output is:
```python
Series([], dtype: bool)

col4    False
dtype: bool
```

#### Output of ``pd.show_versions()``

<details>

[paste the output of ``pd.show_versions()`` here below this line]
```
INSTALLED VERSIONS
------------------
commit: None
python: 3.6.6.final.0
python-bits: 64
OS: Darwin
OS-release: 16.7.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.24.0
pytest: 3.8.2
pip: 18.0
setuptools: 40.4.3
Cython: 0.29
numpy: 1.15.0
scipy: 1.2.0
pyarrow: 0.12.0-RAY
xarray: None
IPython: 7.0.1
sphinx: 1.8.1
patsy: None
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: 0.4.0
matplotlib: 3.0.0
openpyxl: 2.5.8
xlrd: 1.1.0
xlwt: None
xlsxwriter: None
lxml.etree: 4.2.5
bs4: None
html5lib: None
sqlalchemy: 1.2.17
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: 0.6.1
pandas_datareader: None
gcsfs: None
```
</details>

Hints from issue discussion:
It appears to be due to this PR: #24434, where the filter for boolean data was removed from the `_reduce` method to fix a bug for `axis=1`. I created #25102 to solve (both) problems.

Commit message: Merge branch 'master' into issues/25101

Patch to review:
diff --git a/doc/source/whatsnew/v0.24.1.rst b/doc/source/whatsnew/v0.24.1.rst
index 2e4d96dcf9f11..a94d0fc0aa55d 100644
--- a/doc/source/whatsnew/v0.24.1.rst
+++ b/doc/source/whatsnew/v0.24.1.rst
@@ -58,6 +58,7 @@ Fixed Regressions
 - Fixed regression in :func:`merge` when merging an empty ``DataFrame`` with multiple timezone-aware columns on one of the timezone-aware columns (:issue:`25014`).
 - Fixed regression in :meth:`Series.rename_axis` and :meth:`DataFrame.rename_axis` where passing ``None`` failed to remove the axis name (:issue:`25034`)
 - Fixed regression in :func:`to_timedelta` with `box=False` incorrectly returning a ``datetime64`` object instead of a ``timedelta64`` object (:issue:`24961`)
+- Fixed regression in :meth:`DataFrame.all` and :meth:`DataFrame.any` where ``bool_only`` as ``True`` was ignored (:issue:`25101`)
 
 .. _whatsnew_0241.bug_fixes:
 
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
index ade05ab27093e..afe09df8b13ab 100644
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -7476,7 +7476,8 @@ def f(x):
                 if filter_type is None or filter_type == 'numeric':
                     data = self._get_numeric_data()
                 elif filter_type == 'bool':
-                    data = self
+                    # GH 25101, # GH 24434
+                    data = self._get_bool_data() if axis == 0 else self
                 else:  # pragma: no cover
                     msg = ("Generating numeric_only data with filter_type {f}"
                            "not supported.".format(f=filter_type))
diff --git a/pandas/tests/frame/test_analytics.py b/pandas/tests/frame/test_analytics.py
index 386e5f57617cf..c0f4fb3b4d36e 100644
--- a/pandas/tests/frame/test_analytics.py
+++ b/pandas/tests/frame/test_analytics.py
@@ -1442,6 +1442,26 @@ def test_any_datetime(self):
         expected = Series([True, True, True, False])
         tm.assert_series_equal(result, expected)
 
+    def test_any_all_bool_only(self):
+
+        # GH 25101
+        df = DataFrame({"col1": [1, 2, 3],
+                        "col2": [4, 5, 6],
+                        "col3": [None, None, None]})
+
+        result = df.all(bool_only=True)
+        expected = Series(dtype=np.bool)
+        tm.assert_equal(result, expected)
+
+        df = DataFrame({"col1": [1, 2, 3],
+                        "col2": [4, 5, 6],
+                        "col3": [None, None, None],
+                        "col4": [False, False, True]})
+
+        result = df.all(bool_only=True)
+        expected = Series({"col4": False})
+        tm.assert_equal(result, expected)
+
     @pytest.mark.parametrize('func, data, expected', [
         (np.any, {}, False),
         (np.all, {}, True),


Code review comments:
diff_hunk: @@ -58,6 +58,7 @@ Fixed Regressions
 - Fixed regression in :func:`merge` when merging an empty ``DataFrame`` with multiple timezone-aware columns on one of the timezone-aware columns (:issue:`25014`).
 - Fixed regression in :meth:`Series.rename_axis` and :meth:`DataFrame.rename_axis` where passing ``None`` failed to remove the axis name (:issue:`25034`)
 - Fixed regression in :func:`to_timedelta` with `box=False` incorrectly returning a ``datetime64`` object instead of a ``timedelta64`` object (:issue:`24961`)
+- Fixed regression in :meth:`DataFrame.all` and :meth:`DataFrame.any` where ``bool_only`` as ``True`` was ignored (:issue:`25101`)
path: doc/source/whatsnew/v0.24.1.rst
text: @user1:
move to 0.24.2

@user1:
say ``bool_only=True``

diff_hunk: @@ -1442,6 +1442,26 @@ def test_any_datetime(self):
         expected = Series([True, True, True, False])
         tm.assert_series_equal(result, expected)
 
+    def test_any_all_bool_only(self):
+
+        # GH 25101
+        df = DataFrame({"col1": [1, 2, 3],
+                        "col2": [4, 5, 6],
+                        "col3": [None, None, None]})
+
+        result = df.all(bool_only=True)
+        expected = Series(dtype=np.bool)
+        tm.assert_equal(result, expected)
path: pandas/tests/frame/test_analytics.py
text: @user1:
use assert_series_equal