Repository: joblib/joblib
Pull request #1374 (2022-12-27 14:36:15)

FIX Avoid collisions when caching nested functions
the current implementation do not support nested function
 if two nested function in different function have same name
 such as this
```python
def func1():
    def nested():
        ...
def func2():
    def nested():
        ...
```
then the get_func_name will return same module and name pair

codecov[bot]: # [Codecov](https://codecov.io/gh/joblib/joblib/pull/1374?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) Report
Base: **94.01**% // Head: **70.79**% // Decreases project coverage by **`-23.21%`** :warning:
> Coverage data is based on head [(`59cf40a`)](https://codecov.io/gh/joblib/joblib/pull/1374?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) compared to base [(`6836640`)](https://codecov.io/gh/joblib/joblib/commit/6836640abba55611d6e57f20338ea54b3e27f296?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).
> Patch coverage: 100.00% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##           master    #1374       +/-   ##
===========================================
- Coverage   94.01%   70.79%   -23.22%     
===========================================
  Files          52       52               
  Lines        7300     7115      -185     
===========================================
- Hits         6863     5037     -1826     
- Misses        437     2078     +1641     
```


| [Impacted Files](https://codecov.io/gh/joblib/joblib/pull/1374?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) | Coverage Δ | |
|---|---|---|
| [joblib/func\_inspect.py](https://codecov.io/gh/joblib/joblib/pull/1374/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL2Z1bmNfaW5zcGVjdC5weQ==) | `87.02% <100.00%> (-5.33%)` | :arrow_down: |
| [joblib/test/test\_dask.py](https://codecov.io/gh/joblib/joblib/pull/1374/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Rlc3QvdGVzdF9kYXNrLnB5) | `3.92% <0.00%> (-87.27%)` | :arrow_down: |
| [joblib/executor.py](https://codecov.io/gh/joblib/joblib/pull/1374/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL2V4ZWN1dG9yLnB5) | `30.95% <0.00%> (-69.05%)` | :arrow_down: |
| [joblib/\_dask.py](https://codecov.io/gh/joblib/joblib/pull/1374/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19kYXNrLnB5) | `24.01% <0.00%> (-68.67%)` | :arrow_down: |
| [joblib/test/testutils.py](https://codecov.io/gh/joblib/joblib/pull/1374/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Rlc3QvdGVzdHV0aWxzLnB5) | `33.33% <0.00%> (-66.67%)` | :arrow_down: |
| [joblib/test/test\_memmapping.py](https://codecov.io/gh/joblib/joblib/pull/1374/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Rlc3QvdGVzdF9tZW1tYXBwaW5nLnB5) | `35.30% <0.00%> (-63.95%)` | :arrow_down: |
| [joblib/\_multiprocessing\_helpers.py](https://codecov.io/gh/joblib/joblib/pull/1374/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19tdWx0aXByb2Nlc3NpbmdfaGVscGVycy5weQ==) | `26.92% <0.00%> (-61.54%)` | :arrow_down: |
| [joblib/pool.py](https://codecov.io/gh/joblib/joblib/pull/1374/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Bvb2wucHk=) | `27.04% <0.00%> (-60.76%)` | :arrow_down: |
| [joblib/\_memmapping\_reducer.py](https://codecov.io/gh/joblib/joblib/pull/1374/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19tZW1tYXBwaW5nX3JlZHVjZXIucHk=) | `38.72% <0.00%> (-57.16%)` | :arrow_down: |
| [joblib/\_parallel\_backends.py](https://codecov.io/gh/joblib/joblib/pull/1374/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19wYXJhbGxlbF9iYWNrZW5kcy5weQ==) | `46.84% <0.00%> (-47.65%)` | :arrow_down: |
| ... and [28 more](https://codecov.io/gh/joblib/joblib/pull/1374/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/joblib/joblib/pull/1374?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).


Patch:
diff --git a/CHANGES.rst b/CHANGES.rst
index 6f2dd8a..3f1bf8e 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -27,6 +27,11 @@ In development
   errors, a new warning has been introduced: `joblib.memory.CacheWarning`.
   https://github.com/joblib/joblib/pull/1359
 
+- Avoid (module, name) collisions when caching nested functions. This fix
+  changes the module name of nested functions, invalidating caches from
+  previous versions of Joblib.
+  https://github.com/joblib/joblib/pull/1374
+
 Release 1.2.0
 -------------
 
diff --git a/joblib/func_inspect.py b/joblib/func_inspect.py
index d334a2b..58d9751 100644
--- a/joblib/func_inspect.py
+++ b/joblib/func_inspect.py
@@ -166,6 +166,10 @@ def get_func_name(func, resolv_alias=True, win_characters=True):
         if hasattr(func, 'func_globals') and name in func.func_globals:
             if not func.func_globals[name] is func:
                 name = '%s-alias' % name
+    if hasattr(func, '__qualname__') and func.__qualname__ != name:
+        # Extend the module name in case of nested functions to avoid
+        # (module, name) collisions
+        module.extend(func.__qualname__.split(".")[:-1])
     if inspect.ismethod(func):
         # We need to add the name of the class
         if hasattr(func, 'im_class'):


Test patch:
diff --git a/joblib/test/test_func_inspect.py b/joblib/test/test_func_inspect.py
index b3b7b7b..a2a010e 100644
--- a/joblib/test/test_func_inspect.py
+++ b/joblib/test/test_func_inspect.py
@@ -146,6 +146,26 @@ def test_func_name_on_inner_func(cached_func):
     assert get_func_name(cached_func)[1] == 'cached_func_inner'
 
 
+def test_func_name_collision_on_inner_func():
+    # Check that two functions defining and caching an inner function
+    # with the same do not cause (module, name) collision
+    def f():
+        def inner_func():
+            return  # pragma: no cover
+        return get_func_name(inner_func)
+
+    def g():
+        def inner_func():
+            return  # pragma: no cover
+        return get_func_name(inner_func)
+
+    module, name = f()
+    other_module, other_name = g()
+
+    assert name == other_name
+    assert module != other_module
+
+
 def test_func_inspect_errors():
     # Check that func_inspect is robust and will work on weird objects
     assert get_func_name('a'.lower)[-1] == 'lower'