Repository: joblib/joblib
Pull request #1272 (2022-02-23 09:31:14)

Bump up dependency versions on the CI config
Test against more recent versions of Python and numpy on all platforms.

codecov[bot]: # [Codecov](https://codecov.io/gh/joblib/joblib/pull/1272?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) Report
> Merging [#1272](https://codecov.io/gh/joblib/joblib/pull/1272?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) (0cf6684) into [master](https://codecov.io/gh/joblib/joblib/commit/4c1643a124b988da6c097750678608ff94f26f24?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) (4c1643a) will **decrease** coverage by `17.89%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/joblib/joblib/pull/1272/graphs/tree.svg?width=650&height=150&src=pr&token=gA6LF5DGTW&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib)](https://codecov.io/gh/joblib/joblib/pull/1272?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib)

```diff
@@             Coverage Diff             @@
##           master    #1272       +/-   ##
===========================================
- Coverage   88.28%   70.38%   -17.90%     
===========================================
  Files          50       50               
  Lines        7118     6936      -182     
===========================================
- Hits         6284     4882     -1402     
- Misses        834     2054     +1220     
```


| [Impacted Files](https://codecov.io/gh/joblib/joblib/pull/1272?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) | Coverage Δ | |
|---|---|---|
| [joblib/executor.py](https://codecov.io/gh/joblib/joblib/pull/1272/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL2V4ZWN1dG9yLnB5) | `30.95% <0.00%> (-69.05%)` | :arrow_down: |
| [joblib/test/test\_memmapping.py](https://codecov.io/gh/joblib/joblib/pull/1272/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Rlc3QvdGVzdF9tZW1tYXBwaW5nLnB5) | `34.12% <0.00%> (-65.13%)` | :arrow_down: |
| [joblib/\_multiprocessing\_helpers.py](https://codecov.io/gh/joblib/joblib/pull/1272/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19tdWx0aXByb2Nlc3NpbmdfaGVscGVycy5weQ==) | `26.92% <0.00%> (-61.54%)` | :arrow_down: |
| [joblib/pool.py](https://codecov.io/gh/joblib/joblib/pull/1272/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Bvb2wucHk=) | `27.04% <0.00%> (-60.76%)` | :arrow_down: |
| [joblib/\_memmapping\_reducer.py](https://codecov.io/gh/joblib/joblib/pull/1272/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19tZW1tYXBwaW5nX3JlZHVjZXIucHk=) | `38.34% <0.00%> (-58.29%)` | :arrow_down: |
| [joblib/test/test\_module.py](https://codecov.io/gh/joblib/joblib/pull/1272/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Rlc3QvdGVzdF9tb2R1bGUucHk=) | `52.94% <0.00%> (-47.06%)` | :arrow_down: |
| [joblib/backports.py](https://codecov.io/gh/joblib/joblib/pull/1272/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL2JhY2twb3J0cy5weQ==) | `39.47% <0.00%> (-44.74%)` | :arrow_down: |
| [joblib/\_parallel\_backends.py](https://codecov.io/gh/joblib/joblib/pull/1272/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL19wYXJhbGxlbF9iYWNrZW5kcy5weQ==) | `46.84% <0.00%> (-44.71%)` | :arrow_down: |
| [joblib/test/test\_parallel.py](https://codecov.io/gh/joblib/joblib/pull/1272/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Rlc3QvdGVzdF9wYXJhbGxlbC5weQ==) | `51.94% <0.00%> (-42.28%)` | :arrow_down: |
| [joblib/test/test\_store\_backends.py](https://codecov.io/gh/joblib/joblib/pull/1272/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib#diff-am9ibGliL3Rlc3QvdGVzdF9zdG9yZV9iYWNrZW5kcy5weQ==) | `50.00% <0.00%> (-41.43%)` | :arrow_down: |
| ... and [24 more](https://codecov.io/gh/joblib/joblib/pull/1272/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/joblib/joblib/pull/1272?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/joblib/joblib/pull/1272?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib). Last update [4c1643a...0cf6684](https://codecov.io/gh/joblib/joblib/pull/1272?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=joblib).

ogrisel: The flaky tests seem unrelated. Let's merge.

Patch:
diff --git a/azure-pipelines.yml b/azure-pipelines.yml
index b15e394..90e59f6 100644
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -43,20 +43,20 @@ jobs:
         PYTHON_VERSION: "3.9"
         # SKIP_TESTS: "true"
         SKLEARN_TESTS: "true"
-      linux_py38_distributed:
+      linux_py310_distributed:
         # To be updated regularly to use the most recent versions of the
         # dependencies.
         imageName: 'ubuntu-latest'
-        PYTHON_VERSION: "3.8"
-        EXTRA_CONDA_PACKAGES: "numpy=1.18 distributed=2.17"
+        PYTHON_VERSION: "3.10"
+        EXTRA_CONDA_PACKAGES: "numpy=1.22 distributed=2022.2.0"
       linux_py37_distributed:
         imageName: 'ubuntu-latest'
         PYTHON_VERSION: "3.7"
         EXTRA_CONDA_PACKAGES: "numpy=1.15 distributed=2.13"
-      linux_py37_cython:
+      linux_py310_cython:
         imageName: 'ubuntu-latest'
-        PYTHON_VERSION: "3.7"
-        EXTRA_CONDA_PACKAGES: "numpy=1.15"
+        PYTHON_VERSION: "3.10"
+        EXTRA_CONDA_PACKAGES: "numpy=1.22"
         CYTHON: "true"
       linux_py37_no_multiprocessing_no_lzma:
         imageName: 'ubuntu-latest'
@@ -68,15 +68,15 @@ jobs:
         imageName: 'ubuntu-latest'
         PYTHON_VERSION: "3.7"
 
-      windows_py38:
+      windows_py310:
         imageName: "windows-latest"
-        PYTHON_VERSION: "3.8"
-        EXTRA_CONDA_PACKAGES: "numpy=1.18"
+        PYTHON_VERSION: "3.10"
+        EXTRA_CONDA_PACKAGES: "numpy=1.22"
 
-      macos_py38:
+      macos_py310:
         imageName: "macos-latest"
-        PYTHON_VERSION: "3.8"
-        EXTRA_CONDA_PACKAGES: "numpy=1.18"
+        PYTHON_VERSION: "3.10"
+        EXTRA_CONDA_PACKAGES: "numpy=1.22"
       macos_py37_no_numpy:
         imageName: "macos-latest"
         PYTHON_VERSION: "3.7"
diff --git a/continuous_integration/install.sh b/continuous_integration/install.sh
index c942d89..13cb8df 100755
--- a/continuous_integration/install.sh
+++ b/continuous_integration/install.sh
@@ -15,7 +15,7 @@ create_new_conda_env() {
     # TODO: unpin pytest once it no longer causes test errors because of
     # PytestRemovedIn8Warning warnings
     TO_INSTALL="python=$PYTHON_VERSION pip pytest<7.0 $EXTRA_CONDA_PACKAGES"
-    conda create -n testenv --yes $TO_INSTALL
+    conda create -n testenv --yes -c conda-forge $TO_INSTALL
     source activate testenv
 }
 
diff --git a/joblib/backports.py b/joblib/backports.py
index cb2f723..3a14f10 100644
--- a/joblib/backports.py
+++ b/joblib/backports.py
@@ -2,13 +2,112 @@
 Backports of fixes for joblib dependencies
 """
 import os
+import re
 import time
 
-from distutils.version import LooseVersion
 from os.path import basename
 from multiprocessing import util
 
 
+class Version:
+    """Backport from deprecated distutils
+
+    We maintain this backport to avoid introducing a new dependency on
+    `packaging`.
+
+    We might rexplore this choice in the future if all major Python projects
+    introduce a dependency on packaging anyway.
+    """
+
+    def __init__(self, vstring=None):
+        if vstring:
+            self.parse(vstring)
+
+    def __repr__(self):
+        return "%s ('%s')" % (self.__class__.__name__, str(self))
+
+    def __eq__(self, other):
+        c = self._cmp(other)
+        if c is NotImplemented:
+            return c
+        return c == 0
+
+    def __lt__(self, other):
+        c = self._cmp(other)
+        if c is NotImplemented:
+            return c
+        return c < 0
+
+    def __le__(self, other):
+        c = self._cmp(other)
+        if c is NotImplemented:
+            return c
+        return c <= 0
+
+    def __gt__(self, other):
+        c = self._cmp(other)
+        if c is NotImplemented:
+            return c
+        return c > 0
+
+    def __ge__(self, other):
+        c = self._cmp(other)
+        if c is NotImplemented:
+            return c
+        return c >= 0
+
+
+class LooseVersion(Version):
+    """Backport from deprecated distutils
+
+    We maintain this backport to avoid introducing a new dependency on
+    `packaging`.
+
+    We might rexplore this choice in the future if all major Python projects
+    introduce a dependency on packaging anyway.
+    """
+
+    component_re = re.compile(r'(\d+ | [a-z]+ | \.)', re.VERBOSE)
+
+    def __init__(self, vstring=None):
+        if vstring:
+            self.parse(vstring)
+
+    def parse(self, vstring):
+        # I've given up on thinking I can reconstruct the version string
+        # from the parsed tuple -- so I just store the string here for
+        # use by __str__
+        self.vstring = vstring
+        components = [x for x in self.component_re.split(vstring)
+                      if x and x != '.']
+        for i, obj in enumerate(components):
+            try:
+                components[i] = int(obj)
+            except ValueError:
+                pass
+
+        self.version = components
+
+    def __str__(self):
+        return self.vstring
+
+    def __repr__(self):
+        return "LooseVersion ('%s')" % str(self)
+
+    def _cmp(self, other):
+        if isinstance(other, str):
+            other = LooseVersion(other)
+        elif not isinstance(other, LooseVersion):
+            return NotImplemented
+
+        if self.version == other.version:
+            return 0
+        if self.version < other.version:
+            return -1
+        if self.version > other.version:
+            return 1
+
+
 try:
     import numpy as np
 
diff --git a/joblib/compressor.py b/joblib/compressor.py
index 840fabe..8361d37 100644
--- a/joblib/compressor.py
+++ b/joblib/compressor.py
@@ -2,7 +2,7 @@
 
 import io
 import zlib
-from distutils.version import LooseVersion
+from joblib.backports import LooseVersion
 
 try:
     from threading import RLock


Test patch:
diff --git a/conftest.py b/conftest.py
index e246e95..875e9b9 100644
--- a/conftest.py
+++ b/conftest.py
@@ -1,10 +1,10 @@
-from distutils.version import LooseVersion
 
 import pytest
 from _pytest.doctest import DoctestItem
 
 import logging
 from joblib.parallel import mp
+from joblib.backports import LooseVersion
 try:
     import lz4
 except ImportError:
diff --git a/joblib/test/test_numpy_pickle.py b/joblib/test/test_numpy_pickle.py
index 2135e8f..8019ca5 100644
--- a/joblib/test/test_numpy_pickle.py
+++ b/joblib/test/test_numpy_pickle.py
@@ -282,11 +282,13 @@ def test_compress_mmap_mode_warning(tmpdir):
     numpy_pickle.dump(a, this_filename, compress=1)
     with warns(UserWarning) as warninfo:
         numpy_pickle.load(this_filename, mmap_mode='r+')
+    warninfo = [w.message for w in warninfo]
     assert len(warninfo) == 1
-    assert (str(warninfo[0].message) ==
-            'mmap_mode "%(mmap_mode)s" is not compatible with compressed '
-            'file %(filename)s. "%(mmap_mode)s" flag will be ignored.' %
-            {'filename': this_filename, 'mmap_mode': 'r+'})
+    assert (
+        str(warninfo[0]) ==
+        'mmap_mode "r+" is not compatible with compressed '
+        f'file {this_filename}. "r+" flag will be ignored.'
+    )
 
 
 @with_numpy