Repository: DataBiosphere/toil

Issue:
htcondor extra not installable with Toil > 5.5
This came up here: https://github.com/ComparativeGenomicsToolkit/cactus/issues/903

example
```
pip install -U toil[htcondor]
toil --version
cwltool is not installed.
5.5.0
```
trying to force a more recent version
```
pip install toil[htcondor]==5.8.0
Collecting toil[htcondor]==5.8.0
  Using cached toil-5.8.0-py3-none-any.whl (679 kB)
Requirement already satisfied: python-dateutil in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.8.0) (2.8.2)
Requirement already satisfied: dill<0.4,>=0.3.2 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.8.0) (0.3.6)
Requirement already satisfied: requests<3,>=2 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.8.0) (2.28.2)
Requirement already satisfied: enlighten<2,>=1.5.2 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.8.0) (1.11.1)
Collecting PyPubSub<5,>=4.0.3
  Using cached Pypubsub-4.0.3-py3-none-any.whl (61 kB)
Collecting py-tes<1,>=0.4.2
  Using cached py_tes-0.4.2-py3-none-any.whl
Requirement already satisfied: psutil<6,>=3.0.1 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.8.0) (5.9.4)
Requirement already satisfied: docker<6,>=3.7.2 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.8.0) (4.3.1)
Requirement already satisfied: pytz>=2012 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.8.0) (2022.7.1)
Collecting typing-extensions
  Using cached typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Requirement already satisfied: addict<2.5,>=2.2.1 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.8.0) (2.2.1)
ERROR: Ignored the following versions that require a different python version: 3.4.2 Requires-Python ==2.7.*
ERROR: Could not find a version that satisfies the requirement htcondor<9,>=8.6.0; sys_platform != "darwin" and extra == "htcondor" (from toil[htcondor]) (from versions: 9.4.0, 9.5.0, 9.7.0, 9.8.0, 9.8.1, 9.9.0, 9.10.0.post1, 9.11.0, 9.11.1, 9.12.0, 10.0.0, 10.0.1, 10.0.1.post1, 10.1.0, 10.1.1, 10.2.0, 10.2.0.post1)
ERROR: No matching distribution found for htcondor<9,>=8.6.0; sys_platform != "darwin" and extra == "htcondor"
```

```
pip install toil[htcondor]==5.6.0
Collecting toil[htcondor]==5.6.0
  Using cached toil-5.6.0-py3-none-any.whl (592 kB)
Collecting py-tes<1,>=0.4.2
  Using cached py_tes-0.4.2-py3-none-any.whl
Requirement already satisfied: addict<2.5,>=2.2.1 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.6.0) (2.2.1)
Requirement already satisfied: python-dateutil in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.6.0) (2.8.2)
Requirement already satisfied: requests<3,>=2 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.6.0) (2.28.2)
Requirement already satisfied: psutil<6,>=3.0.1 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.6.0) (5.9.4)
Requirement already satisfied: dill<0.4,>=0.3.2 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.6.0) (0.3.6)
Requirement already satisfied: enlighten<2,>=1.5.2 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.6.0) (1.11.1)
Requirement already satisfied: docker<6,>=3.7.2 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.6.0) (4.3.1)
Requirement already satisfied: pytz>=2012 in ./test-venv/lib/python3.10/site-packages (from toil[htcondor]==5.6.0) (2022.7.1)
ERROR: Could not find a version that satisfies the requirement htcondor<9,>=8.6.0; sys_platform != "darwin" and extra == "htcondor" (from toil[htcondor]) (from versions: 9.4.0, 9.5.0, 9.7.0, 9.8.0, 9.8.1, 9.9.0, 9.10.0.post1, 9.11.0, 9.11.1, 9.12.0, 10.0.0, 10.0.1, 10.0.1.post1, 10.1.0, 10.1.1, 10.2.0, 10.2.0.post1)
ERROR: No matching distribution found for htcondor<9,>=8.6.0; sys_platform != "darwin" and extra == "htcondor"
```

┆Issue is synchronized with this [Jira Story](https://ucsc-cgl.atlassian.net/browse/TOIL-1261)
┆Issue Number: TOIL-1261



Discussion:
I think the problem is [we don't want htcondor 9 or later](https://github.com/DataBiosphere/toil/blob/1e94d76640be0ee58c5c746eb03fe98863fdc869/requirements-htcondor.txt#L1), while I think `htcondor==10.2.0.post1` should work fine (although obviously we can't actually test it without a live cluster).

Resolving patch:
diff --git a/src/toil/cwl/__init__.py b/src/toil/cwl/__init__.py
--- a/src/toil/cwl/__init__.py
+++ b/src/toil/cwl/__init__.py
@@ -15,6 +15,14 @@
 from functools import lru_cache
 
 from pkg_resources import DistributionNotFound, get_distribution
+try:
+    # Setuptools 66+ will raise this if any package on the system has a version that isn't PEP440.
+    # See https://github.com/pypa/setuptools/issues/3772
+    from setuptools.extern.packaging.version import InvalidVersion # type: ignore
+except ImportError:
+    # It's not clear that this exception is really part fo the public API, so fake it.
+    class InvalidVersion(Exception): # type: ignore
+        pass
 
 from toil.version import cwltool_version
 
@@ -38,6 +46,9 @@ def check_cwltool_version() -> None:
             )
     except DistributionNotFound:
         logger.debug("cwltool is not installed.")
+    except InvalidVersion as e:
+        logger.warning(f"Could not determine the installed version of cwltool because a package "
+                       f"with an unacceptable version is installed: {e}")
 
 
 check_cwltool_version()


Test patch: