Repository: mne-tools/mne-python

Issue:
BUG: _compare_version backward incompat
cc @hoechenberger:
```
>>> import mne
>>> mne.fixes._compare_version('1.0.dev0', '>=', '1.0')
False
>>> from distutils.version import LooseVersion
>>> LooseVersion('1.0.dev0') >= LooseVersion('1.0')
<stdin>:1: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
True
```
This is a problem because we usually want the second result :(

The `git grep check_version | wc -l`  count is 67 and `git grep _compare_version | wc -l` is 26, I'd rather not have to change all of these to allow for `.dev0` extensions. Plus other submodules like `mne-nirs` that already use this (where I saw the problem actually!) will not work properly...

I'm tempted to change this to cut off any `.dev` from the end to make life easy, but then we depart from standard. But it seems a bit like the lesser of two evils here.


Discussion:
I believe there were some relatively recent changes by @cbrnr if I'm not mistaken (cannot check right now), could you have a look at the history of that function and see if this problem maybe only started showing up after these changes?
This is a bit unfortunate and maybe even a bug? They are explicitly recommending to switch to `packaging.version.parse`, but it doesn't behave identically. I've open an [issue](https://github.com/pypa/packaging/issues/514).
I have to say that IMHO the new behavior is more correct than the old one though. 1.0 **is** newer than 1.0dev0. So the version comparison should be `>=1.0dev0`, no? I think this is how conda behaves too if I'm not mistaken. 
Technically yes, `dev0` is a *pre*release, so it is less than the final release. However, speaking of [semantic versioning](https://semver.org/), I think we're not using those prerelease tags correctly, because they should be separated by a hyphen (e.g. `1.0.0-dev0`).
As @hoechenberger suspected this is expected behavior, and `distutils.version` was doing its own thing. I'd change the comparisons in our code. Alternatively, we could vendor the old versioning code, but I'd rather not do that.
Let's change our code to make it more consistent with my thinking 😅🥴😁
I'd rather add a loose=True that prunes the dev. Most of the time if a person is using 1.0.dev0, they probably have what's needed and if they don't they know how to update. Plus it's backward compatible this way. And it's much easier than changing 100+ places we use (and think about) these comparisons.
So this would basically affect equality comparisons, right? Using `loose=True` would mean that `1.0.dev0 >= 1.0`, `1.0.dev0 <= 1.0`, and `1.0.dev0 == 1.0` are all `True`. Does this make sense? I don't really understand why we rely on such strange comparisons.
I can document this inaccuracy with `==` and `loose=True`. In practice we (almost?) never use `==`. To me practicality -- and backward compat for other libs and even users that make use of `check_version` (and have only recently silently been getting the wrong result with `parse`, at least according to how `check_version` has behaved for the last 9 years) -- beats purity here by a long shot.
Even if documented just stripping the `dev0` suffix doesn't make sense – this doesn't restore the old behavior, which is:

```python
>>>  LooseVersion("1.0.dev0") >= LooseVersion("1.0")
True
>>>  LooseVersion("1.0.dev0") <= LooseVersion("1.0")
False
>>>  LooseVersion("1.0.dev0") == LooseVersion("1.0")
False
```

So the difference is that a `dev0` version was *newer* previously, but is now considered *older*.
The most important thing is that it restores the behavior of `check_version`. This uses `_compare_version` under the hood with a `<` operator to say if something is not okay:

https://github.com/mne-tools/mne-python/blob/6608b720b214529732542cb2aac11b401770eddf/mne/utils/check.py#L100

So for example:
```
>>> LooseVersion('1.0.dev0') < LooseVersion('1.0')
False
>>> mne.fixes._compare_version('1.0.dev0', '<', '1.0')
True
```
So I think it should work. So maybe we should add `loose=True` to `check_version`, do the stripping there, and not change `_compare_version` (and just fix the way fewer of those we use, where needed).
> So maybe we should add loose=True to check_version, do the stripping there, and not change _compare_version (and just fix the way fewer of those we use, where needed).

This sounds good!

Resolving patch:
diff --git a/mne/utils/check.py b/mne/utils/check.py
--- a/mne/utils/check.py
+++ b/mne/utils/check.py
@@ -4,13 +4,14 @@
 #
 # License: BSD-3-Clause
 
-import importlib
 from builtins import input  # no-op here but facilitates testing
 from difflib import get_close_matches
+from importlib import import_module
 import operator
 import os
 import os.path as op
 from pathlib import Path
+import re
 import sys
 import warnings
 import numbers
@@ -64,7 +65,8 @@ def check_fname(fname, filetype, endings, endings_err=()):
              % (fname, filetype, print_endings))
 
 
-def check_version(library, min_version='0.0', *, return_version=False):
+def check_version(library, min_version='0.0', *, strip=True,
+                  return_version=False):
     r"""Check minimum library version required.
 
     Parameters
@@ -75,6 +77,15 @@ def check_version(library, min_version='0.0', *, return_version=False):
         The minimum version string. Anything that matches
         ``'(\d+ | [a-z]+ | \.)'``. Can also be empty to skip version
         check (just check for library presence).
+    strip : bool
+        If True (default), then PEP440 development markers like ``.devN``
+        will be stripped from the version. This makes it so that
+        ``check_version('mne', '1.1')`` will be ``True`` even when on version
+        ``'1.1.dev0'`` (prerelease/dev version). This option is provided for
+        backward compatibility with the behavior of ``LooseVersion``, and
+        diverges from how modern parsing in ``packaging.version.parse`` works.
+
+        .. versionadded:: 1.0
     return_version : bool
         If True (default False), also return the version (can be None if the
         library is missing).
@@ -91,18 +102,42 @@ def check_version(library, min_version='0.0', *, return_version=False):
     ok = True
     version = None
     try:
-        library = __import__(library)
+        library = import_module(library)
     except ImportError:
         ok = False
     else:
-        if min_version and min_version != '0.0':
+        check_version = min_version and min_version != '0.0'
+        get_version = check_version or return_version
+        if get_version:
             version = library.__version__
+            if strip:
+                version = _strip_dev(version)
+        if check_version:
             if _compare_version(version, '<', min_version):
                 ok = False
     out = (ok, version) if return_version else ok
     return out
 
 
+def _strip_dev(version):
+    # First capturing group () is what we want to keep, at the beginning:
+    #
+    # - at least one numeral, then
+    # - repeats of {dot, at least one numeral}
+    #
+    # The rest (consume to the end of the string) is the stuff we want to cut
+    # off:
+    #
+    # - A period (maybe), then
+    # - "dev", "rc", or "+", then
+    # - numerals, periods, dashes, and "a" through "g" (hex chars)
+    #
+    # Thanks https://www.regextester.com !
+    exp = r'^([0-9]+(?:\.[0-9]+)*)\.?(?:dev|rc|\+)[0-9+a-g\.\-]+$'
+    match = re.match(exp, version)
+    return match.groups()[0] if match is not None else version
+
+
 def _require_version(lib, what, version='0.0'):
     """Require library for a purpose."""
     ok, got = check_version(lib, version, return_version=True)
@@ -297,7 +332,7 @@ def _soft_import(name, purpose, strict=True):
         Whether to raise an error if module import fails.
     """
     try:
-        mod = importlib.import_module(name)
+        mod = import_module(name)
         return mod
     except (ImportError, ModuleNotFoundError):
         if strict:


Test patch:
diff --git a/mne/utils/tests/test_check.py b/mne/utils/tests/test_check.py
--- a/mne/utils/tests/test_check.py
+++ b/mne/utils/tests/test_check.py
@@ -16,7 +16,7 @@
 from mne.datasets import testing
 from mne.io.pick import pick_channels_cov, _picks_to_idx
 from mne.utils import (check_random_state, _check_fname, check_fname, _suggest,
-                       _check_subject, _check_info_inv, _check_option,
+                       _check_subject, _check_info_inv, _check_option, Bunch,
                        check_version, _path_like, _validate_type, _on_missing,
                        requires_nibabel, _safe_input, _check_ch_locs)
 
@@ -248,3 +248,55 @@ def test_check_ch_locs():
     # tests for other (and "all") channels should still pass
     assert _check_ch_locs(info=info)
     assert _check_ch_locs(info=info, ch_type='mag')
+
+
+# Check a bunch of version schemes as of 2022/03/01
+# We don't have to get this 100% generalized, but it would be nice if all
+# of these worked.
+@pytest.mark.parametrize('version, want, have_unstripped', [
+    # test some dev cases
+    ('1.23.0.dev0+782.g1168868df6', '1.23', False),  # NumPy
+    ('1.9.0.dev0+1485.b06254e', '1.9', False),  # SciPy
+    ('3.6.0.dev1651+g30d6161406', '3.6', False),  # matplotlib
+    ('1.1.dev0', '1.1', False),  # sklearn
+    ('0.56.0dev0+39.gef1ba4c10', '0.56', False),  # numba
+    ('9.1.0.rc1', '9.1', False),  # VTK
+    ('0.3dev0', '0.3', False),  # mne-connectivity
+    ('0.2.2.dev0', '0.2.2', False),  # mne-qt-browser
+    ('3.2.2+150.g1e93bd5d', '3.2.2', True),  # nibabel
+    # test some stable cases
+    ('1.2.3', '1.2.3', True),
+    ('1.2', '1.2', True),
+    ('1', '1', True),
+])
+def test_strip_dev(version, want, have_unstripped, monkeypatch):
+    """Test that stripping dev works."""
+    monkeypatch.setattr(
+        mne.utils.check, 'import_module',
+        lambda x: Bunch(__version__=version))
+    got_have_unstripped, same_version = check_version(
+        version, want, strip=False, return_version=True)
+    assert same_version == version
+    assert got_have_unstripped is have_unstripped
+    have, simpler_version = check_version(
+        'foo', want, return_version=True)  # strip=True is the default
+    assert have, (simpler_version, version)
+
+    def looks_stable(version):
+        try:
+            [int(x) for x in version.split('.')]
+        except ValueError:
+            return False
+        else:
+            return True
+
+    if looks_stable(version):
+        assert 'dev' not in version
+        assert 'rc' not in version
+        assert simpler_version == version
+    else:
+        assert simpler_version != version
+    assert 'dev' not in simpler_version
+    assert 'rc' not in simpler_version
+    assert not simpler_version.endswith('.')
+    assert looks_stable(simpler_version)