Repository: scipy/scipy

Issue:
DOC: helper function to seed examples
Currently we are rewriting `rng = np.random.default_rng()` so that a common seed is used in the background:

https://github.com/scipy/scipy/blob/78ff9f16c4619759fc59f1e1813900738db8d7d2/scipy/_lib/_util.py#L548-L556

From https://github.com/scipy/scipy/pull/14182#issuecomment-855318533:

We should document that somewhere in the development docs, and expose that example seed somewhere so docstring writers have an easy way to write those examples. So I can write

```python
>>> from scipy._lib.docstrings import doc_rng
>>> rng = doc_rng()
>>> # My example follows
```

Which is easy to change to 

```python
>>> rng = np.random.default_rng()
>>> # My example follows
```

when copying into the docstring.

---

Or even simpler if we manage to render something different, we can just do:

```python
>>> rng = np.random.default_rng("any seed")
>>> # My example follows
```

And get back

```python
>>> rng = np.random.default_rng()
>>> # My example follows
```
Simpler.


Discussion:
Actually this is easy to change when the example is being run during the doctests. But I don't know how to change the rendering itself. I don't know what can be wrapped here.
I think I can mock what Sphinx is getting from numpydoc here https://github.com/numpy/numpydoc/blob/fe95e8b9332ed4ebc154db08b00232f5f67f9330/numpydoc/docscrape_sphinx.py#L340
I got this!! Started to play with mocks and others, but in the end it's way simpler than that:

```python
# put this at the end of doc/source/conf.py.
# And I suppose we can just put this in utils (or elsewhere to have a better name)

from numpydoc.docscrape_sphinx import SphinxDocString


def side_effect(func):
    def _wrapped(*args, **kwargs):
        res = func(*args, **kwargs)

        lines = []
        for line in res:
            if "np.random.default_rng" in line:
                prefix = line.split(">>>")[0]
                # just putting some random value to see it change in the doc
                line = f"{prefix}>>> rng = np.random.default_rng(66666666)"
            lines.append(line)

        return lines
    return _wrapped


SphinxDocString._str_examples = side_effect(SphinxDocString._str_examples)
```

If we do this, we can put the seed we want in the source, but the rendered doc will just display the fake thing. So the rendered example is using this. BUT, doctests would use the value you set (we need to remove the current doctest wrapper).

@mdhaber what do you think?? (Just don't mind yet the code, this is quick and there might be faster/better ways to do that in the inner function.)
To make sure I understand: with this, we would write examples like:
```
rng = np.random.default_rng(90783456987345690)
```
and that's how they would be executed in the refguide check, but the rendered documentation would look like:
> ```
> rng = np.random.default_rng()
> ```

Hopefully that's right, but I'm still not sure what the rendered example will use when it _executes_ - "the fake thing", presumably `default_rng()`, or "the value you set", `default_rng(90783456987345690)`?
Yes!

When the example runs to render the doc it would use the rendered value without the seed. Only running the refguide check would use what's in the source.

Since I now have a way to overwrite things, we could have the logic we want.
A lengthier explanation for completeness and if someone else stop by.

The examples are run twice:

1. doc building: this renders the doc and make the HTML. From playing around, the code is just run for plotting purposes. So no plot, no run and in any case we don't validate anything. Build would pass if the code is wrong/malformed and no plot is involved. Also even when plots are involved and the code runs, the output like printing an array is not changed and stays what we put in the source.
2. doctests: this is the refguide check part. Here we run the examples AND check the output.

We don't want seed in the HTML: `rng = np.random.default_rng()`. And we need a seed for doctests. Hence, at leat one overwrite is needed to switch the behaviour.

Right now the switch is on the doctest side. It works, but not convenient for devs as we need to go find the seed value used for doctests.

A better experience could be to do the switch on the rendering side. i.e. put a seed in the example, anything the dev wants, and overwrite this for rendering the doc with `rng = np.random.default_rng()`.

(Since we already have the first way, we would have both ways at the same time. Can be done with little to no adjustments)

Resolving patch:
diff --git a/doc/source/conf.py b/doc/source/conf.py
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -1,31 +1,32 @@
-# -*- coding: utf-8 -*-
-import sys, os, re
-import glob
-from datetime import date
+import math
+import os
+from os.path import relpath, dirname
+import re
+import sys
 import warnings
+from datetime import date
 
-import numpy as np
-
-# Currently required to build scipy.fft docs
-os.environ['_SCIPY_BUILDING_DOC'] = 'True'
-
-# Check Sphinx version
-import sphinx
-if sphinx.__version__ < "2.0":
-    raise RuntimeError("Sphinx 2.0 or newer required")
-
-needs_sphinx = '2.0'
+import matplotlib
+import matplotlib.pyplot as plt
+from numpydoc.docscrape_sphinx import SphinxDocString
+from sphinx.util import inspect
 
+import scipy
+from scipy._lib._util import _rng_html_rewrite
 # Workaround for sphinx-doc/sphinx#6573
 # ua._Function should not be treated as an attribute
-from sphinx.util import inspect
 import scipy._lib.uarray as ua
 from scipy.stats._distn_infrastructure import rv_generic  # noqa: E402
 from scipy.stats._multivariate import multi_rv_generic  # noqa: E402
+
+
 old_isdesc = inspect.isdescriptor
 inspect.isdescriptor = (lambda obj: old_isdesc(obj)
                         and not isinstance(obj, ua._Function))
 
+# Currently required to build scipy.fft docs
+os.environ['_SCIPY_BUILDING_DOC'] = 'True'
+
 # -----------------------------------------------------------------------------
 # General configuration
 # -----------------------------------------------------------------------------
@@ -50,16 +51,10 @@
     'matplotlib.sphinxext.plot_directive',
 ]
 
-# Determine if the matplotlib has a recent enough version of the
-# plot_directive.
-from matplotlib.sphinxext import plot_directive
-if plot_directive.__version__ < 2:
-    raise RuntimeError("You need a recent enough version of matplotlib")
+
 # Do some matplotlib config in case users have a matplotlibrc that will break
 # things
-import matplotlib
 matplotlib.use('agg')
-import matplotlib.pyplot as plt
 plt.ioff()
 
 # Add any paths that contain templates here, relative to this directory.
@@ -77,7 +72,6 @@
 
 # The default replacements for |version| and |release|, also used in various
 # other places throughout the built documents.
-import scipy
 version = re.sub(r'\.dev-.*$', r'.dev', scipy.__version__)
 release = scipy.__version__
 
@@ -86,7 +80,7 @@
     version = os.environ['CIRCLE_BRANCH']
     release = version
 
-print("%s (VERSION %s)" % (project, version))
+print(f"{project} (VERSION {version})")
 
 # There are two options for replacing |today|: either, you set today to some
 # non-false value, then it is used:
@@ -192,7 +186,7 @@
 if 'dev' in version:
     html_theme_options["switcher"]["version_match"] = "dev"
 
-if 'versionwarning' in tags:
+if 'versionwarning' in tags:  # noqa
     # Specific to docs.scipy.org deployment.
     # See https://github.com/scipy/docs.scipy.org/blob/main/_static/versionwarning.js_t
     src = ('var script = document.createElement("script");\n'
@@ -204,7 +198,7 @@
     }
     html_js_files = ['versioncheck.js']
 
-html_title = "%s v%s Manual" % (project, version)
+html_title = f"{project} v{version} Manual"
 html_static_path = ['_static']
 html_last_updated_fmt = '%b %d, %Y'
 
@@ -314,7 +308,6 @@
 plot_html_show_formats = False
 plot_html_show_source_link = False
 
-import math
 phi = (math.sqrt(5) + 1)/2
 
 font_size = 13*72/96.0  # 13 px
@@ -338,11 +331,6 @@
 # -----------------------------------------------------------------------------
 # Source code links
 # -----------------------------------------------------------------------------
-
-import re
-import inspect
-from os.path import relpath, dirname
-
 for name in ['sphinx.ext.linkcode', 'linkcode', 'numpydoc.linkcode']:
     try:
         __import__(name)
@@ -353,6 +341,7 @@
 else:
     print("NOTE: linkcode extension not found -- no links to source generated")
 
+
 def linkcode_resolve(domain, info):
     """
     Determine the URL corresponding to Python object
@@ -408,14 +397,18 @@ def linkcode_resolve(domain, info):
 
     if fn.startswith('scipy/'):
         m = re.match(r'^.*dev0\+([a-f0-9]+)$', scipy.__version__)
+        base_url = "https://github.com/scipy/scipy/blob"
         if m:
-            return "https://github.com/scipy/scipy/blob/%s/%s%s" % (
-                m.group(1), fn, linespec)
+            return f"{base_url}/{m.group(1)}/{fn}{linespec}"
         elif 'dev' in scipy.__version__:
-            return "https://github.com/scipy/scipy/blob/main/%s%s" % (
-                fn, linespec)
+            return f"{base_url}/main/{fn}{linespec}"
         else:
-            return "https://github.com/scipy/scipy/blob/v%s/%s%s" % (
-                scipy.__version__, fn, linespec)
+            return f"{base_url}/v{scipy.__version__}/{fn}{linespec}"
     else:
         return None
+
+
+# Tell overwrite numpydoc's logic to render examples containing rng.
+SphinxDocString._str_examples = _rng_html_rewrite(
+    SphinxDocString._str_examples
+)
diff --git a/scipy/_lib/_util.py b/scipy/_lib/_util.py
--- a/scipy/_lib/_util.py
+++ b/scipy/_lib/_util.py
@@ -1,3 +1,4 @@
+import re
 from contextlib import contextmanager
 import functools
 import operator
@@ -521,6 +522,29 @@ def _fixed_default_rng(seed=1638083107694713882823079058616272161):
         np.random.default_rng = orig_fun
 
 
+def _rng_html_rewrite(func):
+    """Rewrite the HTML rendering of ``np.random.default_rng``.
+
+    This is intended to decorate
+    ``numpydoc.docscrape_sphinx.SphinxDocString._str_examples``.
+
+    Examples are only run by Sphinx when there are plot involved. Even so,
+    it does not change the result values getting printed.
+    """
+    # hexadecimal or number seed, case-insensitive
+    pattern = re.compile(r'np.random.default_rng\((0x[0-9A-F]+|\d+)\)', re.I)
+
+    def _wrapped(*args, **kwargs):
+        res = func(*args, **kwargs)
+        lines = [
+            re.sub(pattern, 'np.random.default_rng()', line)
+            for line in res
+        ]
+        return lines
+
+    return _wrapped
+
+
 def _argmin(a, keepdims=False, axis=None):
     """
     argmin with a `keepdims` parameter.
diff --git a/scipy/stats/_hypotests.py b/scipy/stats/_hypotests.py
--- a/scipy/stats/_hypotests.py
+++ b/scipy/stats/_hypotests.py
@@ -537,17 +537,17 @@ def cramervonmises(rvs, cdf, args=()):
 
     Suppose we wish to test whether data generated by ``scipy.stats.norm.rvs``
     were, in fact, drawn from the standard normal distribution. We choose a
-    significance level of alpha=0.05.
+    significance level of ``alpha=0.05``.
 
     >>> import numpy as np
     >>> from scipy import stats
-    >>> rng = np.random.default_rng()
+    >>> rng = np.random.default_rng(165417232101553420507139617764912913465)
     >>> x = stats.norm.rvs(size=500, random_state=rng)
     >>> res = stats.cramervonmises(x, 'norm')
     >>> res.statistic, res.pvalue
-    (0.49121480855028343, 0.04189256516661377)
+    (0.1072085112565724, 0.5508482238203407)
 
-    The p-value 0.79 exceeds our chosen significance level, so we do not
+    The p-value exceeds our chosen significance level, so we do not
     reject the null hypothesis that the observed sample is drawn from the
     standard normal distribution.
 
@@ -557,7 +557,7 @@ def cramervonmises(rvs, cdf, args=()):
     >>> y = x + 2.1
     >>> res = stats.cramervonmises(y, 'norm', args=(2,))
     >>> res.statistic, res.pvalue
-    (0.07400330012187435, 0.7274595666160468)
+    (0.8364446265294695, 0.00596286797008283)
 
     Here we have used the `args` keyword to specify the mean (``loc``)
     of the normal distribution to test the data against. This is equivalent
@@ -567,11 +567,11 @@ def cramervonmises(rvs, cdf, args=()):
     >>> frozen_dist = stats.norm(loc=2)
     >>> res = stats.cramervonmises(y, frozen_dist.cdf)
     >>> res.statistic, res.pvalue
-    (0.07400330012187435, 0.7274595666160468)
+    (0.8364446265294695, 0.00596286797008283)
 
     In either case, we would reject the null hypothesis that the observed
     sample is drawn from a normal distribution with a mean of 2 (and default
-    variance of 1) because the p-value 0.04 is less than our chosen
+    variance of 1) because the p-value is less than our chosen
     significance level.
 
     """


Test patch:
diff --git a/scipy/_lib/tests/test__util.py b/scipy/_lib/tests/test__util.py
--- a/scipy/_lib/tests/test__util.py
+++ b/scipy/_lib/tests/test__util.py
@@ -14,7 +14,7 @@
 from scipy._lib._util import (_aligned_zeros, check_random_state, MapWrapper,
                               getfullargspec_no_self, FullArgSpec,
                               rng_integers, _validate_int, _rename_parameter,
-                              _contains_nan)
+                              _contains_nan, _rng_html_rewrite)
 
 
 def test__aligned_zeros():
@@ -378,3 +378,24 @@ def test_contains_nan_2d(self):
 
         data4 = np.array([["1", 2], [3, np.nan]], dtype='object')
         assert _contains_nan(data4)[0]
+
+
+def test__rng_html_rewrite():
+    def mock_str():
+        lines = [
+            'np.random.default_rng(8989843)',
+            'np.random.default_rng(seed)',
+            'np.random.default_rng(0x9a71b21474694f919882289dc1559ca)',
+            ' bob ',
+        ]
+        return lines
+
+    res = _rng_html_rewrite(mock_str)()
+    ref = [
+        'np.random.default_rng()',
+        'np.random.default_rng(seed)',
+        'np.random.default_rng()',
+        ' bob ',
+    ]
+
+    assert res == ref