Repository: joblib/joblib

Issue:
DOC: `joblib.parallel_backend` no longer in intersphinx
Sometime in the last day we started getting:
```
/home/larsoner/python/mne-python/mne/viz/_3d.py:docstring of mne.viz._3d.plot_evoked_field:16: WARNING: py:func reference target not found: joblib:joblib.parallel_backend
```
And indeed it appears to be missing
```
$ python -m sphinx.ext.intersphinx https://joblib.readthedocs.io/en/latest/objects.inv | grep parallel_backend
	joblib.register_parallel_backend         parallel.html#joblib.register_parallel_backend
```
And the old link is broken/does not have an anchor:

- https://joblib.readthedocs.io/en/latest/parallel.html#joblib.parallel_backend

Looking into the changelog/history to find the culprit now...

Discussion:
codecov[bot]: ## [Codecov](https://codecov.io/gh/joblib/joblib/pull/1437?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) Report
Patch coverage has no change and project coverage change: **`-0.88`** :warning:
> Comparison is base [(`ebacd8c`)](https://codecov.io/gh/joblib/joblib/commit/ebacd8cf0bcc52f1884098df8fbc3a93df91e30e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) 94.75% compared to head [(`7204da9`)](https://codecov.io/gh/joblib/joblib/pull/1437?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) 93.88%.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #1437      +/-   ##
==========================================
- Coverage   94.75%   93.88%   -0.88%     
==========================================
  Files          44       44              
  Lines        7306     7306              
==========================================
- Hits         6923     6859      -64     
- Misses        383      447      +64     
```


| [Impacted Files](https://codecov.io/gh/joblib/joblib/pull/1437?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) | Coverage Δ | |
|---|---|---|
| [joblib/compressor.py](https://codecov.io/gh/joblib/joblib/pull/1437?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL2NvbXByZXNzb3IucHk=) | `89.14% <ø> (ø)` | |
| [joblib/hashing.py](https://codecov.io/gh/joblib/joblib/pull/1437?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL2hhc2hpbmcucHk=) | `89.47% <ø> (-1.76%)` | :arrow_down: |
| [joblib/numpy\_pickle.py](https://codecov.io/gh/joblib/joblib/pull/1437?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL251bXB5X3BpY2tsZS5weQ==) | `99.14% <ø> (ø)` | |
| [joblib/numpy\_pickle\_compat.py](https://codecov.io/gh/joblib/joblib/pull/1437?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL251bXB5X3BpY2tsZV9jb21wYXQucHk=) | `91.08% <ø> (ø)` | |
| [joblib/parallel.py](https://codecov.io/gh/joblib/joblib/pull/1437?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3BhcmFsbGVsLnB5) | `95.12% <ø> (-2.12%)` | :arrow_down: |

... and [11 files with indirect coverage changes](https://codecov.io/gh/joblib/joblib/pull/1437/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib)


</details>

[:umbrella: View full report in Codecov by Sentry](https://codecov.io/gh/joblib/joblib/pull/1437?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).

larsoner: Yes I would change the RTD build command. I can look into that or feel free to push a commit

This fixes the original problem for me:
```
$ python -m sphinx.ext.intersphinx _build/html/objects.inv | grep "joblib\.parallel_backend"
	joblib.parallel_backend                  parallel.html#joblib.parallel_backend
```
larsoner: ... actually IIRC the RTD stuff lives on the RTD site itself (?) so someone else might need to change it. I did at least make it that locally now it builds with `-nWT --keep-going` by default

Patch:
diff --git a/.readthedocs.yaml b/.readthedocs.yaml
new file mode 100644
index 0000000..c451ca2
--- /dev/null
+++ b/.readthedocs.yaml
@@ -0,0 +1,13 @@
+version: 2
+
+python:
+  # make sure joblib is installed in the virtualenv (it is imported in
+  # conf.py)
+  install:
+    - requirements: .readthedocs-requirements.txt
+    - method: pip
+      path: .
+
+sphinx:
+  fail_on_warning: true
+
diff --git a/doc/Makefile b/doc/Makefile
index bae1bcd..547f3f2 100644
--- a/doc/Makefile
+++ b/doc/Makefile
@@ -1,5 +1,5 @@
 # You can set these variables from the command lines
-SPHINXOPTS   ?= -j $(shell nproc)
+SPHINXOPTS   ?= -j $(shell nproc) -nWT --keep-going
 SPHINXBUILD  ?= sphinx-build
 
 .PHONY: all html clean
@@ -8,8 +8,13 @@ all: html
 
 # generate html documentation with warning as errors
 html:
-	$(SPHINXBUILD) -W -b html . ./_build/html/
+	$(SPHINXBUILD) $(SPHINXOPTS) -b html . ./_build/html/
 
 # remove generated sphinx gallery examples and sphinx documentation
 clean:
 	rm -rf auto_examples && rm -rf generated && rm -rf _build
+
+view:
+	@python -c "import webbrowser; webbrowser.open_new_tab('file://$(PWD)/_build/html/index.html')"
+
+show: view
diff --git a/doc/_templates/class.rst b/doc/_templates/class.rst
index 93b430e..ec25aad 100644
--- a/doc/_templates/class.rst
+++ b/doc/_templates/class.rst
@@ -1,25 +1,11 @@
-:mod:`{{module}}`.{{objname}}
-{{ underline }}==============
+{{ fullname | escape | underline }}
 
 .. currentmodule:: {{ module }}
 
 .. autoclass:: {{ objname }}
+   :members:
 
-   {% block methods %}
-   .. automethod:: __init__
+.. _sphx_glr_backreferences_{{ fullname }}:
 
-   {% if methods %}
-   .. rubric:: Methods
-
-   .. autosummary::
-   {% for item in methods %}
-      ~{{ name }}.{{ item }}
-   {%- endfor %}
-   {% endif %}
-   {% endblock %}
-
-.. include:: {{module}}.{{objname}}.examples
-
-.. raw:: html
-
-    <div class="clearer"></div>
+.. minigallery:: {{ fullname }}
+    :add-heading:
diff --git a/doc/memory.rst b/doc/memory.rst
index 901ccd0..7b6b176 100644
--- a/doc/memory.rst
+++ b/doc/memory.rst
@@ -15,20 +15,9 @@ On demand recomputing: the `Memory` class
 Use case
 --------
 
-The `Memory` class defines a context for lazy evaluation of function, by
-putting the results in a store, by default using a disk, and not re-running
-the function twice for the same arguments.
-
-..
- Commented out in favor of briefness
-
-    You can use it as a context, with its `eval` method:
-
-    .. automethod:: Memory.eval
-
-    or decorate functions with the `cache` method:
-
-    .. automethod:: Memory.cache
+The :class:`~joblib.Memory` class defines a context for lazy evaluation of
+function, by putting the results in a store, by default using a disk, and not
+re-running the function twice for the same arguments.
 
 It works by explicitly saving the output to a file and it is designed to
 work with non-hashable and potentially large input and output data types
@@ -102,7 +91,7 @@ An example
 
   Define two functions: the first with a number as an argument,
   outputting an array, used by the second one. Both functions are decorated
-  with `Memory.cache`::
+  with :meth:`Memory.cache <joblib.Memory.cache>`::
 
     >>> import numpy as np
 
@@ -413,16 +402,19 @@ change, for instance a debug flag. `Memory` provides the `ignore` list::
 
 .. _memory_reference:
 
-Reference documentation of the `Memory` class
----------------------------------------------
+Reference documentation of the :class:`~joblib.Memory` class
+------------------------------------------------------------
 
-.. autoclass:: Memory
-    :members: __init__, cache, eval, clear
+.. autoclass:: joblib.Memory
+    :members: __init__, cache, eval, clear, reduce_size, format
+    :no-inherited-members:
+    :noindex:
 
 Useful methods of decorated functions
 -------------------------------------
 
-Functions decorated by :meth:`Memory.cache` are :class:`MemorizedFunc`
+Functions decorated by :meth:`Memory.cache <joblib.Memory.cache>` are
+:class:`MemorizedFunc`
 objects that, in addition of behaving like normal functions, expose
 methods useful for cache exploration and management. For example, you can
 use :meth:`func.check_call_in_cache <MemorizedFunc.check_call_in_cache>` to
diff --git a/doc/parallel.rst b/doc/parallel.rst
index b4c5071..b63e0ae 100644
--- a/doc/parallel.rst
+++ b/doc/parallel.rst
@@ -397,10 +397,22 @@ does not exist (but multiprocessing has more overhead).
 ==================================
 
 .. autoclass:: joblib.Parallel
-    :noindex:
+   :members: dispatch_next, dispatch_one_batch, format, print_progress
+   :no-inherited-members:
+   :noindex:
 
 .. autofunction:: joblib.delayed
 
 .. autofunction:: joblib.register_parallel_backend
 
 .. autofunction:: joblib.parallel_backend
+
+.. autofunction:: joblib.wrap_non_picklable_objects
+
+.. autofunction:: joblib.parallel.register_parallel_backend
+
+.. autoclass:: joblib.parallel.parallel_config
+
+.. autoclass:: joblib.parallel.ParallelBackendBase
+
+.. autoclass:: joblib.parallel.AutoBatchingMixin
diff --git a/examples/parallel_generator.py b/examples/parallel_generator.py
index 0e4dbc3..43b84ea 100644
--- a/examples/parallel_generator.py
+++ b/examples/parallel_generator.py
@@ -81,7 +81,7 @@ def return_big_object(i):
 
 ##############################################################################
 # We create a reduce step. The input will be a generator on big objects
-# generated in parallel by several instances of :func:`return_big_object`.
+# generated in parallel by several instances of ``return_big_object``.
 
 def accumulator_sum(generator):
     result = 0
diff --git a/examples/serialization_and_wrappers.py b/examples/serialization_and_wrappers.py
index d03f012..d463242 100644
--- a/examples/serialization_and_wrappers.py
+++ b/examples/serialization_and_wrappers.py
@@ -141,7 +141,7 @@ print("With pickle from stdlib and wrapper: {:.3f}s"
 
 ###############################################################################
 # The same wrapper can also be used for non-picklable classes. Note that the
-# side effects of :func:`wrap_non_picklable_objects` on objects can break magic
+# side effects of ``wrap_non_picklable_objects`` on objects can break magic
 # methods such as ``__add__`` and can mess up the ``isinstance`` and
 # ``issubclass`` functions. Some improvements will be considered if use-cases
 # are reported.
diff --git a/joblib/_cloudpickle_wrapper.py b/joblib/_cloudpickle_wrapper.py
index 3dbe3ae..daf899d 100644
--- a/joblib/_cloudpickle_wrapper.py
+++ b/joblib/_cloudpickle_wrapper.py
@@ -7,11 +7,13 @@ multiprocessing is not available.
 from ._multiprocessing_helpers import mp
 
 
-def my_wrap_non_picklable_objects(obj, keep_wrapper=True):
+def _my_wrap_non_picklable_objects(obj, keep_wrapper=True):
     return obj
 
 
-if mp is None:
-    wrap_non_picklable_objects = my_wrap_non_picklable_objects
+if mp is not None:
+    from .externals.loky import wrap_non_picklable_objects
 else:
-    from .externals.loky import wrap_non_picklable_objects # noqa
+    wrap_non_picklable_objects = _my_wrap_non_picklable_objects
+
+__all__ = ["wrap_non_picklable_objects"]
diff --git a/joblib/compressor.py b/joblib/compressor.py
index 8361d37..0d9e261 100644
--- a/joblib/compressor.py
+++ b/joblib/compressor.py
@@ -47,7 +47,7 @@ def register_compressor(compressor_name, compressor,
     """Register a new compressor.
 
     Parameters
-    -----------
+    ----------
     compressor_name: str.
         The name of the compressor.
     compressor: CompressorWrapper
diff --git a/joblib/hashing.py b/joblib/hashing.py
index b983e84..65b0ce8 100644
--- a/joblib/hashing.py
+++ b/joblib/hashing.py
@@ -245,9 +245,8 @@ def hash(obj, hash_name='md5', coerce_mmap=False):
     """ Quick calculation of a hash to identify uniquely Python objects
         containing numpy arrays.
 
-
         Parameters
-        -----------
+        ----------
         hash_name: 'md5' or 'sha1'
             Hashing algorithm used. sha1 is supposedly safer, but md5 is
             faster.
diff --git a/joblib/numpy_pickle.py b/joblib/numpy_pickle.py
index 371ee96..3c6974a 100644
--- a/joblib/numpy_pickle.py
+++ b/joblib/numpy_pickle.py
@@ -427,7 +427,7 @@ def dump(value, filename, compress=0, protocol=None, cache_size=None):
     Read more in the :ref:`User Guide <persistence>`.
 
     Parameters
-    -----------
+    ----------
     value: any Python object
         The object to store to disk.
     filename: str, pathlib.Path, or file object.
@@ -611,7 +611,7 @@ def load(filename, mmap_mode=None):
     to load files from untrusted sources.
 
     Parameters
-    -----------
+    ----------
     filename: str, pathlib.Path, or file object.
         The file object or path of the file from which to load the object
     mmap_mode: {None, 'r+', 'r', 'w+', 'c'}, optional
diff --git a/joblib/numpy_pickle_compat.py b/joblib/numpy_pickle_compat.py
index 7bf979e..3261284 100644
--- a/joblib/numpy_pickle_compat.py
+++ b/joblib/numpy_pickle_compat.py
@@ -203,7 +203,7 @@ def load_compatibility(filename):
     (<= 0.9.3).
 
     Parameters
-    -----------
+    ----------
     filename: string
         The name of the file from which to load the object
 
diff --git a/joblib/parallel.py b/joblib/parallel.py
index a4700a7..cf528dc 100644
--- a/joblib/parallel.py
+++ b/joblib/parallel.py
@@ -210,10 +210,10 @@ def _get_active_backend(
 
 
 class parallel_config:
-    """Change the default backend or configuration used by :class:`~Parallel`
+    """Set the default backend or configuration for :class:`~joblib.Parallel`.
 
     This is an alternative to directly passing keyword arguments to the
-    :class:`~Parallel` class constructor. It is particularly useful when
+    :class:`~joblib.Parallel` class constructor. It is particularly useful when
     calling into library code that uses joblib internally but does not expose
     the various parallel configuration arguments in its own API.
 
@@ -237,7 +237,7 @@ class parallel_config:
         multiprocessing and loky may not be available, in which case joblib
         defaults to threading.
 
-        In addition, if the `dask` and `distributed` Python packages are
+        In addition, if the ``dask`` and ``distributed`` Python packages are
         installed, it is possible to use the 'dask' backend for better
         scheduling of nested parallel calls without over-subscription and
         potentially distribute parallel calls over a networked cluster of
@@ -251,14 +251,14 @@ class parallel_config:
 
     n_jobs : int, default=None
         The maximum number of concurrently running jobs, such as the number
-        of Python worker processes when `backend="loky"` or the size of the
-        thread-pool when `backend="threading"`.
+        of Python worker processes when ``backend="loky"`` or the size of the
+        thread-pool when ``backend="threading"``.
         If -1 all CPUs are used. If 1 is given, no parallel computing code
-        is used at all, which is useful for debugging. For `n_jobs` below -1,
-        (n_cpus + 1 + n_jobs) are used. Thus for `n_jobs=-2`, all
+        is used at all, which is useful for debugging. For ``n_jobs`` below -1,
+        (n_cpus + 1 + n_jobs) are used. Thus for ``n_jobs=-2``, all
         CPUs but one are used.
-        `None` is a marker for 'unset' that will be interpreted as `n_jobs=1`
-        in most backends.
+        ``None`` is a marker for 'unset' that will be interpreted as
+        ``n_jobs=1`` in most backends.
 
     verbose : int, default=0
         The verbosity level: if non zero, progress messages are
@@ -271,14 +271,14 @@ class parallel_config:
         for sharing memory with worker processes. If None, this will try in
         order:
 
-        - a folder pointed by the `JOBLIB_TEMP_FOLDER` environment
+        - a folder pointed by the ``JOBLIB_TEMP_FOLDER`` environment
           variable,
-        - `/dev/shm` if the folder exists and is writable: this is a
+        - ``/dev/shm`` if the folder exists and is writable: this is a
           RAM disk filesystem available by default on modern Linux
           distributions,
         - the default system temporary folder that can be
-          overridden with `TMP`, `TMPDIR` or `TEMP` environment
-          variables, typically `/tmp` under Unix operating systems.
+          overridden with ``TMP``, ``TMPDIR`` or ``TEMP`` environment
+          variables, typically ``/tmp`` under Unix operating systems.
 
     max_nbytes int, str, or None, optional, default='1M'
         Threshold on the size of arrays passed to the workers that
@@ -305,7 +305,7 @@ class parallel_config:
     inner_max_num_threads : int, default=None
         If not None, overwrites the limit set on the number of threads
         usable in some third-party library threadpools like OpenBLAS,
-        MKL or OpenMP. This is only used with the `loky` backend.
+        MKL or OpenMP. This is only used with the ``loky`` backend.
 
     backend_params : dict
         Additional parameters to pass to the backend constructor when
@@ -320,6 +320,8 @@ class parallel_config:
     overwritten with the ``inner_max_num_threads`` argument which will be used
     to set this limit in the child processes.
 
+    .. versionadded:: 1.3
+
     Examples
     --------
     >>> from operator import neg
@@ -328,7 +330,7 @@ class parallel_config:
     ...
     [-1, -2, -3, -4, -5]
 
-    To use the 'ray' joblib backend add the following lines::
+    To use the 'ray' joblib backend add the following lines:
 
     >>> from ray.util.joblib import register_ray  # doctest: +SKIP
     >>> register_ray()  # doctest: +SKIP
@@ -336,7 +338,6 @@ class parallel_config:
     ...     print(Parallel()(delayed(neg)(i + 1) for i in range(5)))
     [-1, -2, -3, -4, -5]
 
-    .. versionadded:: 1.3
     """
     def __init__(
         self,
@@ -441,8 +442,9 @@ class parallel_config:
 class parallel_backend(parallel_config):
     """Change the default backend used by Parallel inside a with block.
 
-    It is advised to use the `parallel_config` context manager instead,
-    which allows more fine-grained control over the backend configuration.
+    It is advised to use the :class:`~joblib.parallel.parallel_config` context
+    manager instead, which allows more fine-grained control over the backend
+    configuration.
 
     If ``backend`` is a string it must match a previously registered
     implementation using the :func:`~register_parallel_backend` function.
@@ -510,12 +512,12 @@ class parallel_backend(parallel_config):
     overwritten with the ``inner_max_num_threads`` argument which will be used
     to set this limit in the child processes.
 
-    See Also
-    --------
-    parallel_config : context manager to change the backend configuration
-
     .. versionadded:: 0.10
 
+    See Also
+    --------
+    joblib.parallel.parallel_config : context manager to change the backend
+                                      configuration
     """
     def __init__(self, backend, n_jobs=-1, inner_max_num_threads=None,
                  **backend_params):
@@ -880,7 +882,6 @@ def register_parallel_backend(name, factory, make_default=False):
     version of joblib.
 
     .. versionadded:: 0.10
-
     """
     BACKENDS[name] = factory
     if make_default:
@@ -908,7 +909,6 @@ def effective_n_jobs(n_jobs=-1):
     version of joblib.
 
     .. versionadded:: 0.10
-
     """
     if n_jobs == 1:
         return 1
@@ -926,7 +926,7 @@ class Parallel(Logger):
         Read more in the :ref:`User Guide <parallel>`.
 
         Parameters
-        -----------
+        ----------
         n_jobs: int, default: None
             The maximum number of concurrently running jobs, such as the number
             of Python worker processes when backend="multiprocessing"
diff --git a/readthedocs.yml b/readthedocs.yml
deleted file mode 100644
index b69d3ab..0000000
--- a/readthedocs.yml
+++ /dev/null
@@ -1,6 +0,0 @@
-python:
-  # make sure joblib is installed in the virtualenv (it is imported in
-  # conf.py)
-  pip_install: true
-
-requirements_file: .readthedocs-requirements.txt


Test patch:
diff --git a/joblib/test/test_cloudpickle_wrapper.py b/joblib/test/test_cloudpickle_wrapper.py
index 733f51c..57bf92f 100644
--- a/joblib/test/test_cloudpickle_wrapper.py
+++ b/joblib/test/test_cloudpickle_wrapper.py
@@ -4,7 +4,7 @@ properly the loky implementation.
 """
 
 from .._cloudpickle_wrapper import wrap_non_picklable_objects
-from .._cloudpickle_wrapper import my_wrap_non_picklable_objects
+from .._cloudpickle_wrapper import _my_wrap_non_picklable_objects
 
 
 def a_function(x):
@@ -23,5 +23,5 @@ def test_wrap_non_picklable_objects():
     # upstream one
     for obj in (a_function, AClass()):
         wrapped_obj = wrap_non_picklable_objects(obj)
-        my_wrapped_obj = my_wrap_non_picklable_objects(obj)
+        my_wrapped_obj = _my_wrap_non_picklable_objects(obj)
         assert wrapped_obj(1) == my_wrapped_obj(1)