Use np.random's RandomState when seed is None
State: closed

- [x] closes #13143
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

The handle for numpy's current random state is
`np.random.mtrand._rand`.

Compare https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/utils/validation.py#L573


Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
@ariddell pls also add a whatsnew entry. I would do it in API changes section.

@jreback added. Thanks!

@ariddell could you please fix the use of NumPy's private state as noted above? This is something simple that could avoid significant pain down the road.

Fair enough. What should the seed be? Did you mean `2 ** 32` above?

@ariddell Yes, I meant `2 ** 32` :).

Using `np.random.mtrand._rand` is the right approach. It's not going to disappear. This is an authorized use.

Let's just return `np.random` from `_random_state` instead of `np.random.mtrand._rand`, using @rkern's suggestion of duck typing for RandomState objects from the related thread on numpy-discussion: https://mail.scipy.org/pipermail/numpy-discussion/2016-May/075489.html

@shoyer duck typing it is.

you have a failing test

@ariddell can you rebase / update

OK. I think I've got it.

thanks @ariddell 

I'm not getting the behavior I'd expect from this fix in Pandas 0.19.2. Example:

```
>>> import pandas as pd
>>> import numpy as np
>>> np.random.seed(12345678)
>>> df = pd.DataFrame(np.random.randn(10, 5), columns=['a', 'b', 'c', 'd', 'e'])
>>> df.sample(n=2)
          a         b         c         d         e
8  0.365786 -0.855795 -0.511062 -0.116734  1.133541
0  1.281455 -0.423852  0.011236 -0.945194 -0.088124
>>> df.sample(n=2)
          a         b         c         d         e
2  0.341751 -1.158737  0.313814  1.827552 -0.351045
9  1.355003  0.532651  0.250463 -0.281751 -0.342741
>>> df.sample(n=2, random_state=12345678)
          a         b         c         d         e
8  0.365786 -0.855795 -0.511062 -0.116734  1.133541
6 -0.401077 -0.047115 -0.410951 -1.608354  0.290594
>>> df.sample(n=2, random_state=12345678)
          a         b         c         d         e
8  0.365786 -0.855795 -0.511062 -0.116734  1.133541
6 -0.401077 -0.047115 -0.410951 -1.608354  0.290594
>>> pd.__version__
u'0.19.2'
>>> np.__version__
'1.12.0'
```

Any thoughts on why `df.sample(n=2)` doesn't match `df.sample(n=2, random_state=12345678)`?

@pydolan is there a missing `np.random.seed()` call somewhere in your example?
@shoyer -- sorry about that; will update example with my actual code (i.e., with the one missing line)
@pydolan It looks like you are calling `np.random.randn()` after setting the seed before `df.sample()`
If I move the `seed()` call after instantiating the dataframe, I still get inconsistent behavior with calls to `sample()`, except when I provide the `random_state` arg. Example:

```
>>> import pandas as pd
>>> import numpy as np
>>> df = pd.DataFrame(np.random.randn(10, 5), columns=['a', 'b', 'c', 'd', 'e'])
>>> np.random.seed(12345678)
>>> df.sample(n=2)
          a         b         c         d         e
8 -1.250204  0.551508  1.408080  0.397452  0.424326
6 -0.028298  0.203270  0.939094 -1.802227 -0.088679
>>> df.sample(n=2)
          a         b         c         d         e
4  0.895497  0.609853 -1.548664 -1.238415 -1.058904
5  0.196420  0.472877 -0.918205  1.019862 -0.631993
>>> df.sample(n=2, random_state=12345678)
          a         b         c         d         e
8 -1.250204  0.551508  1.408080  0.397452  0.424326
6 -0.028298  0.203270  0.939094 -1.802227 -0.088679
>>> df.sample(n=2, random_state=12345678)
          a         b         c         d         e
8 -1.250204  0.551508  1.408080  0.397452  0.424326
6 -0.028298  0.203270  0.939094 -1.802227 -0.088679
```

Note that this time, the first call to `sample()` uses the seed, but the second call does not use the seed. Is it expected that `seed()` needs to be called before every sample call? I thought it is supposed to be, "set once", and all future randomization-related calls should use (including my original example, where `randn()` is called after `seed()` and before `sample()`).

(Also note that I did verify that calling `seed()` before every call to `sample()` does indeed produce the same sampled rows.)
> Is it expected that seed() needs to be called before every sample call? I thought it is supposed to be, "set once", and all future randomization-related calls should use (including my original example, where randn() is called after seed() and before sample()).

This is working as intended. Try just sampling random numbers from numpy -- it does the exact same thing.
Good point; I completely spaced on the nature of random sampling. Sorry for the waste of time!