Repository: pandas-dev/pandas
Pull request: DOC: Correct/update skipna docstrings for `any` and `all` (#23109)
Also, include examples with NA values and describe treatment of NA with `skipna == False`

- [x] closes #23109
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Problem statement:
BUG: skipna parameter in series.any() returns wrong result

```
#importing pandas module 
import pandas as pd 

#importing numpy module
import numpy as np

data=pd.DataFrame({'A':[1,2,3,4,0,np.nan,3],
                  'B':[3,1,4,5,0,np.nan,5]})

data.any(axis=1,skipna=True)
```

Expected output:
0     True
1     True
2     True
3     True
4    False
5    True
6     True
dtype: bool


Returned output:

0     True
1     True
2     True
3     True
4    False
5    False
6     True
dtype: bool



As written in documentation,  If an entire row/column is NA, the result will be NA
But NA isn't returned in any of the cases (Keeping skipna True or False)

Hints from issue discussion:
> As written in documentation, If an entire row/column is NA, the result will be NA

I think the docs at https://github.com/pandas-dev/pandas/blob/12a0dc49ac63b63bcc5f93bccf71bce51c60bdad/pandas/core/generic.py#L9728-L9730 are incorrect. Skipna should be the same as the operation on the values with NAs removed (is that right @jorisvandenbossche?).
Isn't this a better way? ↓
If skipana is None which is default then it returns NA on whole NaN rows/column
If True/False then return True/False respectively for whole NA 

Because if the docs are incorrect then there is probably no way to return NA for Null values
min_count I thikn

On Fri, Oct 12, 2018 at 11:25 AM Kartikay Bhutani <notifications@github.com>
wrote:

> Isn't this a better way? ↓
> If skipana is None which is default then it returns NA on whole NaN
> rows/column
> If True/False then return True/False respectively for whole NA
>
> Because if the docs are incorrect then there is probably no way to return
> NA for Null values
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/issues/23109#issuecomment-429382736>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIhLJLLq1TczRspP-JDekrFBh8mKiks5ukMJtgaJpZM4XZlWL>
> .
>

My opinion: I think the problem is the documentation; the result is actually correct.  If you ask if any of an empty set of statements is True, the answer is no.  This is consistent with `numpy`:

```
In [1]: import numpy as np

In [2]: np.any([])
Out[2]: False
```
> Skipna should be the same as the operation on the values with NAs removed (is that right @jorisvandenbossche?).

I suppose this as well. `any`/`all` can be seen as reductions like `sum` or `prod`, so we should probably follow their design.

So I think @dsaxton is right that it is only the documentation that is incorrect.
@jorisvandenbossche What would you say is the appropriate fix for this?  If the documentation is a general statement about the `skipna` parameter, maybe it makes sense to just remove the claim that the result will be NA (since it's not true for `any`, but presumably would be true in other contexts)?

Commit message: DOC: Correct/update skipna docstrings for `any` and `all` (#23109)

Also include examples with NA values and clarify treatment of NA with `skipna == False`

Patch to review:
diff --git a/doc/source/whatsnew/v0.24.0.rst b/doc/source/whatsnew/v0.24.0.rst
index 7617ad5b428a2..5317a44e86102 100644
--- a/doc/source/whatsnew/v0.24.0.rst
+++ b/doc/source/whatsnew/v0.24.0.rst
@@ -366,6 +366,7 @@ Other Enhancements
 - :meth:`DataFrame.to_stata` and :class:` pandas.io.stata.StataWriter117` can write mixed sting columns to Stata strl format (:issue:`23633`)
 - :meth:`DataFrame.between_time` and :meth:`DataFrame.at_time` have gained the an ``axis`` parameter (:issue: `8839`)
 - :class:`IntervalIndex` has gained the :attr:`~IntervalIndex.is_overlapping` attribute to indicate if the ``IntervalIndex`` contains any overlapping intervals (:issue:`23309`)
+- :meth:`any` and :meth:`all` aggregation methods have corrected/improved docstrings for the ``skipna`` parameter (:issue:`23019`).
 
 .. _whatsnew_0240.api_breaking:
 
diff --git a/pandas/core/generic.py b/pandas/core/generic.py
index c58c84b422209..4cdbb21776062 100644
--- a/pandas/core/generic.py
+++ b/pandas/core/generic.py
@@ -9862,10 +9862,10 @@ def _add_numeric_operations(cls):
         axis_descr, name, name2 = _doc_parms(cls)
 
         cls.any = _make_logical_function(
-            cls, 'any', name, name2, axis_descr,
+            cls, 'any', name, name2, False, axis_descr,
             _any_desc, nanops.nanany, _any_examples, _any_see_also)
         cls.all = _make_logical_function(
-            cls, 'all', name, name2, axis_descr, _all_doc,
+            cls, 'all', name, name2, True, axis_descr, _all_doc,
             nanops.nanall, _all_examples, _all_see_also)
 
         @Substitution(outname='mad',
@@ -10188,8 +10188,10 @@ def _doc_parms(cls):
     Include only boolean columns. If None, will attempt to use everything,
     then use only boolean data. Not implemented for Series.
 skipna : boolean, default True
-    Exclude NA/null values. If an entire row/column is NA, the result
-    will be NA.
+    Exclude NA/null values. If the entire row/column is NA and skipna is
+    True, then the result will be %(empty_value)s, as for an empty row/column.
+    If skipna is False, then NA are treated as True, because these are not
+    equal to zero.
 level : int or level name, default None
     If the axis is a MultiIndex (hierarchical), count along a
     particular level, collapsing into a %(name1)s.
@@ -10219,6 +10221,10 @@ def _doc_parms(cls):
 True
 >>> pd.Series([True, False]).all()
 False
+>>> pd.Series([]).all()
+True
+>>> pd.Series([np.nan]).all()
+True
 
 DataFrames
 
@@ -10576,6 +10582,13 @@ def _doc_parms(cls):
 
 >>> pd.Series([True, False]).any()
 True
+>>> pd.Series([]).any()
+False
+>>> pd.Series([np.nan]).any()
+False
+>>> pd.Series([np.nan]).any(skipna=False)
+True
+
 
 **DataFrame**
 
@@ -10860,10 +10873,11 @@ def cum_func(self, axis=None, skipna=True, *args, **kwargs):
     return set_function_name(cum_func, name, cls)
 
 
-def _make_logical_function(cls, name, name1, name2, axis_descr, desc, f,
-                           examples, see_also):
+def _make_logical_function(cls, name, name1, name2, empty_value, axis_descr,
+                           desc, f, examples, see_also):
     @Substitution(outname=name, desc=desc, name1=name1, name2=name2,
-                  axis_descr=axis_descr, examples=examples, see_also=see_also)
+                  empty_value=empty_value, axis_descr=axis_descr,
+                  examples=examples, see_also=see_also)
     @Appender(_bool_doc)
     def logical_func(self, axis=0, bool_only=None, skipna=True, level=None,
                      **kwargs):


Code review comments:
diff_hunk: @@ -9862,10 +9862,10 @@ def _add_numeric_operations(cls):
         axis_descr, name, name2 = _doc_parms(cls)
 
         cls.any = _make_logical_function(
-            cls, 'any', name, name2, axis_descr,
+            cls, 'any', name, name2, False, axis_descr,
path: pandas/core/generic.py
text: @user1:
I can't guess what's the `False` without checking the function signature, we should use keyword arguments.

@author:
Would you prefer me to move the argument to the end of the list, or add keywords to all of the arguments?

@user1:
you can add the argument names to all params, or to all the ones after the `True`/`False`, whatever you think it's clearer.