Repository: scipy/scipy

Issue:
rv.expect() == rv.mean() is false for rv.mean() == nan (and inf)
<<Please describe the issue in detail here, and for bug reports fill in the fields below.>>



### Reproducing code example:
```
(scipy-dev) Lucass-MacBook:scipy rlucas$ python -V 
Python 3.6.6 :: Anaconda, Inc.
(scipy-dev) Lucass-MacBook:scipy rlucas$ python 
Python 3.6.6 |Anaconda, Inc.| (default, Jun 28 2018, 11:07:29) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from scipy.stats import cauchy
>>> rv = cauchy(0)
>>> rv.mean() == rv.expect()
False
>>> 
The equality comparison should result in true if the mean() is numpy.inf or numpy.nan
```

### Error message:
```
<<Full error message, if any (starting from line Traceback: ...)>>
```
There isn't really any error message but there should be one. 

The first step is add this as a unit test into `scipy/stats/tests/test_distributions.py` by starting a `class TestCauchy(object):` test and test that the equality above holds. 

Second Step is fix the expect() method. I was thinking a simple fix could be to have the expect() check if mean() is implemented, if it is then call mean(). If not then do what it usually does. I'm not sure if this fully fixes the problem though. 
### Scipy/Numpy/Python version information:
```
<<Output from 'import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)'>>
>>> import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
1.2.0.dev0+b8baffe 1.15.1 sys.version_info(major=3, minor=6, micro=6, releaselevel='final', serial=0)
```




Discussion:
This came up during my implementation of support for yule-simon on a different open issue:

https://github.com/scipy/scipy/issues/9183

There is some discussion of the issue on there too. 
The check I'm suggesting to make in the above comment is a check in the expect methods starting @ LOCs 508, 2339, 3133 of _distn_infrastructure.py file,
i.e. checking if the mean() is nan (or inf) and then returning that rather than doing the numeric integral. 
I'm not sure it is useful to go into the direction of checking existence or finiteness of the integral.

(aside rv.mean() == rv.expect()  will never be True in general because of floating point noise and integral approximation error, and it is not true for nan either)

I don't see a general way to check whether a integral is finite or exists. Also we would have to identify this case for arbitrary user functions.
`mean` is just a special case even though it's the default. The same would apply to other moments, e.g. variance for cauchy would be inf also. I don't think it's an important use case because when want the mean, then we can use the mean or moment methods.
On the other hand, users can specify bounds to get a trimmed mean which will exist in most cases.

IMO, this could lead to a messy and difficult feature creep, with adding more and more special cases.
Instead, maybe we can just make the warning in the docstring more explicit instead of just pointing to integrate quad.


@josef-pkt, that is another option too. There could be valid use cases where the mean in undefined or infinite and you still want a numeric approximation to the expectation. 

Also, exact equality (==) isn't what I would test, I'd likely use numpy.isclose() which returns reasonable output for `numpy.isclose(numpy.nan, numpy.nan)` and `numpy.isclose(numpy.inf, numpy.inf)` and the like...

If changing the docstring(s) to explicitly state that the solution is a numeric approximation.  Then going with changing the docstrings in expect() from: 
( _distn_infrastructure.py file line numbers)
"""Calculate expected value of a function with respect to the
2342         distribution.
2343 
2344         The expected value of a function ``f(x)`` with respect to a
2345         distribution ``dist`` is defined as::
2346 
2347                     ubound
2348             E[x] = Integral(f(x) * dist.pdf(x))
2349                     lbound

to something like: 

 """Calculate a numeric approximation of the expected value of a function with respect to the
2342         distribution.
2343 
2344         The expected value of a function ``f(x)`` with respect to a
2345         distribution ``dist`` is defined as::
2346 
2347                     ubound
2348             E[f(x)] = Integral(f(x) * dist.pdf(x))
2349                     lbound

Note I'm also proposing to change a typo on line 2348 => (E[x] to E[f(x)])
would be the change. 

I'd also missed that the caller can specify their lower and upper bounds to the integral which may differ from the support of the distribution. 
 
Does that sound like a reasonable change that addresses the issue? 
Changing the docstring to explicitly mention that `expect` performs a numerical computation is a good idea.  And the change `E[x]` to `E[f(x)]` should also be made.
typo fix is good.

IMO, numeric approximation sound too strong, i.e. everything we do is a numeric approximation to some extend.
e.g. integrate.quad doesn't phrase it that way.

maybe just add "by numeric integration" to the summary line
(in the discrete case, the expected value will be exact at floating point summation when we completely enumerate all cases.)

my original expect function had an additional sentence in the Notes section:
"This function has not been checked for it's behavior when the integral is not finite. "

I think something along the lines that the `expect` function, and the underlying integrate `quad` function`, cannot verify whether the integral exists or is finite. 
I think pointing this out explicitly is useful given that the non-existence and non-finite cases are quite common in the distributions.

(also: in the unit tests, AFAIR, we explicitly exclude checking that expect is allclose to mean if mean is nan or inf.)

Resolving patch:
diff --git a/scipy/stats/_distn_infrastructure.py b/scipy/stats/_distn_infrastructure.py
--- a/scipy/stats/_distn_infrastructure.py
+++ b/scipy/stats/_distn_infrastructure.py
@@ -2339,14 +2339,22 @@ def integ(x):
     def expect(self, func=None, args=(), loc=0, scale=1, lb=None, ub=None,
                conditional=False, **kwds):
         """Calculate expected value of a function with respect to the
-        distribution.
+        distribution by numerical integration.
 
         The expected value of a function ``f(x)`` with respect to a
         distribution ``dist`` is defined as::
 
-                    ubound
-            E[x] = Integral(f(x) * dist.pdf(x))
-                    lbound
+                    ub
+            E[f(x)] = Integral(f(x) * dist.pdf(x)),
+                    lb
+
+        where ``ub`` and ``lb`` are arguments and ``x`` has the ``dist.pdf(x)`` 
+        distribution. If the bounds ``lb`` and ``ub`` correspond to the 
+        support of the distribution, e.g. ``[-inf, inf]`` in the default 
+        case, then the integral is the unrestricted expectation of ``f(x)``. 
+        Also, the function ``f(x)`` may be defined such that ``f(x)`` is ``0`` 
+        outside a finite interval in which case the expectation is 
+        calculated within the finite range ``[lb, ub]``. 
 
         Parameters
         ----------
@@ -2378,8 +2386,30 @@ def expect(self, func=None, args=(), loc=0, scale=1, lb=None, ub=None,
         Notes
         -----
         The integration behavior of this function is inherited from
-        `integrate.quad`.
+        `scipy.integrate.quad`. Neither this function nor
+        `scipy.integrate.quad` can verify whether the integral exists or is
+        finite. For example ``cauchy(0).mean()`` returns ``np.nan`` and
+        ``cauchy(0).expect()`` returns ``0.0``.
+
+        Examples
+        --------
+
+        To understand the effect of the bounds of integration consider 
+        >>> from scipy.stats import expon
+        >>> expon(1).expect(lambda x: 1, lb=0.0, ub=2.0)
+        0.6321205588285578
+
+        This is close to 
+
+        >>> expon(1).cdf(2.0) - expon(1).cdf(0.0)
+        0.6321205588285577
+
+        If ``conditional=True``
+
+        >>> expon(1).expect(lambda x: 1, lb=0.0, ub=2.0, conditional=True)
+        1.0000000000000002
 
+        The slight deviation from 1 is due to numerical integration.
         """
         lockwds = {'loc': loc,
                    'scale': scale}
@@ -3134,7 +3164,7 @@ def expect(self, func=None, args=(), loc=0, lb=None, ub=None,
                conditional=False, maxcount=1000, tolerance=1e-10, chunksize=32):
         """
         Calculate expected value of a function with respect to the distribution
-        for discrete distribution.
+        for discrete distribution by numerical summation.
 
         Parameters
         ----------


Test patch: