Repository: joblib/joblib
Pull request #1289 (2022-04-27 12:33:40)

FIX memory for function that are call before and after clearance

codecov[bot]: # [Codecov](https://codecov.io/gh/joblib/joblib/pull/1289?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) Report
> Merging [#1289](https://codecov.io/gh/joblib/joblib/pull/1289?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) (c4a128d) into [master](https://codecov.io/gh/joblib/joblib/commit/4dafaff788a3b5402acfed091558b4c511982959?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) (4dafaff) will **decrease** coverage by `23.19%`.
> The diff coverage is `100.00%`.

```diff
@@             Coverage Diff             @@
##           master    #1289       +/-   ##
===========================================
- Coverage   93.80%   70.60%   -23.20%     
===========================================
  Files          50       50               
  Lines        7268     7084      -184     
===========================================
- Hits         6818     5002     -1816     
- Misses        450     2082     +1632     
```


| [Impacted Files](https://codecov.io/gh/joblib/joblib/pull/1289?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) | Coverage Δ | |
|---|---|---|
| [joblib/memory.py](https://codecov.io/gh/joblib/joblib/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL21lbW9yeS5weQ==) | `94.42% <100.00%> (-0.81%)` | :arrow_down: |
| [joblib/test/test\_memory.py](https://codecov.io/gh/joblib/joblib/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Rlc3QvdGVzdF9tZW1vcnkucHk=) | `95.80% <100.00%> (-2.78%)` | :arrow_down: |
| [joblib/test/test\_dask.py](https://codecov.io/gh/joblib/joblib/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Rlc3QvdGVzdF9kYXNrLnB5) | `3.59% <0.00%> (-87.54%)` | :arrow_down: |
| [joblib/executor.py](https://codecov.io/gh/joblib/joblib/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL2V4ZWN1dG9yLnB5) | `30.95% <0.00%> (-69.05%)` | :arrow_down: |
| [joblib/\_dask.py](https://codecov.io/gh/joblib/joblib/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19kYXNrLnB5) | `23.76% <0.00%> (-68.85%)` | :arrow_down: |
| [joblib/test/test\_memmapping.py](https://codecov.io/gh/joblib/joblib/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Rlc3QvdGVzdF9tZW1tYXBwaW5nLnB5) | `34.12% <0.00%> (-65.13%)` | :arrow_down: |
| [joblib/\_multiprocessing\_helpers.py](https://codecov.io/gh/joblib/joblib/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19tdWx0aXByb2Nlc3NpbmdfaGVscGVycy5weQ==) | `26.92% <0.00%> (-61.54%)` | :arrow_down: |
| [joblib/pool.py](https://codecov.io/gh/joblib/joblib/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Bvb2wucHk=) | `27.04% <0.00%> (-60.76%)` | :arrow_down: |
| [joblib/\_memmapping\_reducer.py](https://codecov.io/gh/joblib/joblib/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19tZW1tYXBwaW5nX3JlZHVjZXIucHk=) | `38.34% <0.00%> (-57.54%)` | :arrow_down: |
| [joblib/test/test\_module.py](https://codecov.io/gh/joblib/joblib/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Rlc3QvdGVzdF9tb2R1bGUucHk=) | `52.94% <0.00%> (-47.06%)` | :arrow_down: |
| ... and [28 more](https://codecov.io/gh/joblib/joblib/pull/1289/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/joblib/joblib/pull/1289?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/joblib/joblib/pull/1289?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib). Last update [4dafaff...c4a128d](https://codecov.io/gh/joblib/joblib/pull/1289?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).


Patch:
diff --git a/azure-pipelines.yml b/azure-pipelines.yml
index 90e59f6..a3c1e40 100644
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -3,6 +3,13 @@
 # Add steps that analyze code, save the dist with the build record, publish to a PyPI-compatible index, and more:
 # https://docs.microsoft.com/azure/devops/pipelines/languages/python
 
+schedules:
+- cron: "0 9 * * *"
+  displayName: Daily build
+  branches:
+    include:
+    - master
+
 trigger:
 - master
 
diff --git a/doc/parallel.rst b/doc/parallel.rst
index 8514e2d..c4d3dd3 100644
--- a/doc/parallel.rst
+++ b/doc/parallel.rst
@@ -69,7 +69,7 @@ In prior versions, the same effect could be achieved by hardcoding a
 specific backend implementation such as ``backend="threading"`` in the
 call to :class:`joblib.Parallel` but this is now considered a bad pattern
 (when done in a library) as it does not make it possible to override that
-choice with the ``parallel_backend`` context manager.
+choice with the :func:`~joblib.parallel_backend` context manager.
 
 
 .. topic:: The loky backend may not always be available
@@ -199,7 +199,7 @@ libraries:
 
 Since joblib 0.14, it is also possible to programmatically override the default
 number of threads using the ``inner_max_num_threads`` argument of the
-``parallel_backend`` function as follows:
+:func:`~joblib.parallel_backend` function as follows:
 
 .. code-block:: python
 
diff --git a/joblib/memory.py b/joblib/memory.py
index b761345..fdc58fd 100644
--- a/joblib/memory.py
+++ b/joblib/memory.py
@@ -987,6 +987,12 @@ class Memory(Logger):
         if self.store_backend is not None:
             self.store_backend.clear()
 
+            # As the cache in completely clear, make sure the _FUNCTION_HASHES
+            # cache is also reset. Else, for a function that is present in this
+            # table, results cached after this clear will be have cache miss
+            # as the function code is not re-written.
+            _FUNCTION_HASHES.clear()
+
     def reduce_size(self):
         """Remove cache elements to make cache size fit in ``bytes_limit``."""
         if self.bytes_limit is not None and self.store_backend is not None:
diff --git a/joblib/parallel.py b/joblib/parallel.py
index 9eb6308..f9c8454 100644
--- a/joblib/parallel.py
+++ b/joblib/parallel.py
@@ -135,7 +135,7 @@ class parallel_backend(object):
     """Change the default backend used by Parallel inside a with block.
 
     If ``backend`` is a string it must match a previously registered
-    implementation using the ``register_parallel_backend`` function.
+    implementation using the :func:`~register_parallel_backend` function.
 
     By default the following backends are available:
 
@@ -172,9 +172,9 @@ class parallel_backend(object):
     caller passes an explicit value for the ``n_jobs`` parameter.
 
     This is an alternative to passing a ``backend='backend_name'`` argument to
-    the ``Parallel`` class constructor. It is particularly useful when calling
-    into library code that uses joblib internally but does not expose the
-    backend argument in its own API.
+    the :class:`~Parallel` class constructor. It is particularly useful when
+    calling into library code that uses joblib internally but does not expose
+    the backend argument in its own API.
 
     >>> from operator import neg
     >>> with parallel_backend('threading'):
@@ -389,8 +389,8 @@ def register_parallel_backend(name, factory, make_default=False):
     """Register a new Parallel backend factory.
 
     The new backend can then be selected by passing its name as the backend
-    argument to the Parallel class. Moreover, the default backend can be
-    overwritten globally by setting make_default=True.
+    argument to the :class:`~Parallel` class. Moreover, the default backend can
+    be overwritten globally by setting make_default=True.
 
     The factory can be any callable that takes no argument and return an
     instance of ``ParallelBackendBase``.
@@ -453,8 +453,8 @@ class Parallel(Logger):
             CPUs but one are used.
             None is a marker for 'unset' that will be interpreted as n_jobs=1
             (sequential execution) unless the call is performed under a
-            parallel_backend context manager that sets another value for
-            n_jobs.
+            :func:`~parallel_backend` context manager that sets another value
+            for n_jobs.
         backend: str, ParallelBackendBase instance or None, default: 'loky'
             Specify the parallelization backend implementation.
             Supported backends are:
@@ -474,18 +474,18 @@ class Parallel(Logger):
               in a "with nogil" block or an expensive call to a library such
               as NumPy).
             - finally, you can register backends by calling
-              register_parallel_backend. This will allow you to implement
-              a backend of your liking.
+              :func:`~register_parallel_backend`. This will allow you to
+              implement a backend of your liking.
 
             It is not recommended to hard-code the backend name in a call to
-            Parallel in a library. Instead it is recommended to set soft hints
-            (prefer) or hard constraints (require) so as to make it possible
-            for library users to change the backend from the outside using the
-            parallel_backend context manager.
+            :class:`~Parallel` in a library. Instead it is recommended to set
+            soft hints (prefer) or hard constraints (require) so as to make it
+            possible for library users to change the backend from the outside
+            using the :func:`~parallel_backend` context manager.
         prefer: str in {'processes', 'threads'} or None, default: None
             Soft hint to choose the default backend if no specific backend
-            was selected with the parallel_backend context manager. The
-            default process-based backend is 'loky' and the default
+            was selected with the :func:`~parallel_backend` context manager.
+            The default process-based backend is 'loky' and the default
             thread-based backend is 'threading'. Ignored if the ``backend``
             parameter is specified.
         require: 'sharedmem' or None, default None
@@ -541,7 +541,7 @@ class Parallel(Logger):
             Use None to disable memmapping of large arrays.
             Only active when backend="loky" or "multiprocessing".
         mmap_mode: {None, 'r+', 'r', 'w+', 'c'}, default: 'r'
-            Memmapping mode for numpy arrays passed to workers. None will 
+            Memmapping mode for numpy arrays passed to workers. None will
             disable memmapping, other modes defined in the numpy.memmap doc:
             https://numpy.org/doc/stable/reference/generated/numpy.memmap.html
             Also, see 'max_nbytes' parameter documentation for more details.


Test patch:
diff --git a/joblib/test/test_memory.py b/joblib/test/test_memory.py
index 687f036..aaa7d16 100644
--- a/joblib/test/test_memory.py
+++ b/joblib/test/test_memory.py
@@ -978,11 +978,14 @@ def test_memory_reduce_size(tmpdir):
 
 
 def test_memory_clear(tmpdir):
-    memory, _, _ = _setup_toy_cache(tmpdir)
+    memory, _, g = _setup_toy_cache(tmpdir)
     memory.clear()
 
     assert os.listdir(memory.store_backend.location) == []
 
+    # Check that the cache for functions hash is also reset.
+    assert not g._check_previous_func_code(stacklevel=4)
+
 
 def fast_func_with_complex_output():
     complex_obj = ['a' * 1000] * 1000