Repository: pmeier/light-the-torch (language: python)

Issue:
The found PyTorch version is too old
Hi, @pmeier. Thanks for this package, it looks very useful. I've always wanted something like this.

I am on Linux, NVIDIA driver 430.50. According to [Table 1 here](https://docs.nvidia.com/deploy/cuda-compatibility/index.html), the latest CUDA version I can install is 10.1. When I ran `ltt install torch`, I expected it to run

```
pip install torch==1.8.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html
```

i.e., the latest PyTorch version that supports 10.1 (as far as I know), but it does 

```
Collecting torch==1.4.0+cu100
  Downloading https://download.pytorch.org/whl/cu100/torch-1.4.0%2Bcu100-cp36-cp36m-linux_x86_64.whl (723.9 MB)
     |██▏                             | 50.2 MB 43.4 MB/s eta 0:00:16^C
```

which I killed. It looks like instead of installing 1.8.0 with CUDA 10.1, it installs 1.4.0 with CUDA 10.0. Is this expected?


Discussion / hints:
Hey @fepegar 

> It looks like instead of installing 1.8.0 with CUDA 10.1, it installs 1.4.0 with CUDA 10.0. Is this expected?

Partially yes. It has detected that you are using CUDA 10.0 and `torch==1.4.0` is the latest version built for that. So this behavior is correct. 

Now we need to know *why* it detected CUDA 10.0, but you think you have CUDA 10.1. Internally `light-the-torch` uses `$nvcc --version` as source for the information. Could you run that manually and post the output here?


Thanks for your reply. I'm not in front of my computer at the moment, but I do have 10.0 installed, you're right. [The internet says](https://stackoverflow.com/q/53422407/3956024) that the version shown by `nvidia-smi` is the latest that the installed drivers (`nvidia-smi --query-gpu=driver_version --format=csv`, second line) support.

It might be better to retrieve the version shown by `nvidia-smi` instead of the installed one shown by `nvcc`. What do you think? I'm happy to contribute, if necessary.
> the version shown by nvidia-smi is the latest that the installed drivers support.

[And that's why I can install `torch==1.8.0+cu101`.]
However, [it looks like](https://stackoverflow.com/a/55717476/3956024) the highest supported version is shown is printed with drivers >= 410.72. Therefore it might be safer to retrieve the driver version and calculate the supported CUDA manually using the lookup table I shared above (https://github.com/pmeier/light-the-torch/issues/30#issue-927470215).
Well, it won't help you much if your driver supports up to 10.1, but you only have 10.0 installed, or am I wrong here? For example in my current setup this looks like this

```shell
$nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Mon_Oct_12_20:09:46_PDT_2020
Cuda compilation tools, release 11.1, V11.1.105
Build cuda_11.1.TC455_06.29190527_0
$nvidia-smi
Wed Jun 23 09:48:45 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.31       Driver Version: 465.31       CUDA Version: 11.3     |
|-------------------------------+----------------------+----------------------+
```

So although my driver supports CUDA 11.3, I only have 11.1 installed. IMO in turn that means I will not be able to run a PyTorch compiled for CUDA 11.3.


I don't know much about CUDA, but I'm pretty sure a PyTorch compiled for CUDA 11.3 (if there is one) would indeed work for you. I think the CUDA toolkit installed with PyTorch is minimal and independent of whatever you have installed and is shown by NVCC. As I said, that works for me: I have 10.0 installed, but PyTorch compiled for 10.1 works fine.
Let me check that with the PyTorch folks and get back to you.
It appears you are right. Thanks a lot for the heads-up! Here is my plan fixing this:

1. Expand 
    https://github.com/pmeier/light-the-torch/blob/49264ab486f16f556987deea2c9688876169025e/light_the_torch/compatibility.py#L6
    to handle version comparison besides equality check. Maybe it is easier to simply rely on [`packaging`](https://pypi.org/project/packaging/) or the vendored version that [ships with `pip`](https://github.com/pypa/pip/tree/main/src/pip/_vendor/packaging).
2. Hardcode the table you mentioned above.
3. Adapt 
    https://github.com/pmeier/light-the-torch/blob/49264ab486f16f556987deea2c9688876169025e/light_the_torch/computation_backend.py#L81
    to use `nvidia-smi --query-gpu=driver_version --format=csv` together with mapping from 2. to determine the computation backend.

One problem that I see is that currently the implementation relies on the fact that there is only a single CUDA version. So this would also needs to be fixed to work with multiple versions.

> I'm happy to contribute, if necessary.

Happy to take you up on the offer. I will probably not be able to get onto this for a couple of days.
No worries! I'd probably feel more comfortable with points 2. and 3. I won't be able to help until next week, in the context of the NA-MIC Project Week: https://github.com/NA-MIC/ProjectWeek/tree/master/PW35_2021_Virtual/Projects/PyTorchIntegration#light-the-torch

> One problem that I see is that currently the implementation relies on the fact that there is only a single CUDA version. So this would also needs to be fixed to work with multiple versions.

I'm not sure what you mean by "there is only a single CUDA version". Couldn't we just ignore the installed CUDA versions and choose the installer based on the drivers?
>  in the context of the NA-MIC Project Week

That being said, I might not be able to use `ltt` within the Slicer Python console...

```python-traceback
>>> import light_the_torch as ltt
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/fernando/opt/Slicer/Nightly/lib/Python/lib/python3.6/site-packages/light_the_torch/__init__.py", line 3, in <module>
    from ._pip import *
  File "/home/fernando/opt/Slicer/Nightly/lib/Python/lib/python3.6/site-packages/light_the_torch/_pip/__init__.py", line 1, in <module>
    from .extract import *
  File "/home/fernando/opt/Slicer/Nightly/lib/Python/lib/python3.6/site-packages/light_the_torch/_pip/extract.py", line 5, in <module>
    from pip._internal.req.req_install import InstallRequirement
  File "/home/fernando/opt/Slicer/Nightly/lib/Python/lib/python3.6/site-packages/pip/_internal/req/__init__.py", line 10, in <module>
    from .req_install import InstallRequirement
  File "/home/fernando/opt/Slicer/Nightly/lib/Python/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 35, in <module>
    from pip._internal.utils.direct_url_helpers import direct_url_from_link
  File "/home/fernando/opt/Slicer/Nightly/lib/Python/lib/python3.6/site-packages/pip/_internal/utils/direct_url_helpers.py", line 12, in <module>
    from pip._internal.vcs import vcs
  File "/home/fernando/opt/Slicer/Nightly/lib/Python/lib/python3.6/site-packages/pip/_internal/vcs/__init__.py", line 9, in <module>
    import pip._internal.vcs.subversion  # noqa: F401
  File "/home/fernando/opt/Slicer/Nightly/lib/Python/lib/python3.6/site-packages/pip/_internal/vcs/subversion.py", line 336, in <module>
    vcs.register(Subversion)
  File "/home/fernando/opt/Slicer/Nightly/lib/Python/lib/python3.6/site-packages/pip/_internal/vcs/versioncontrol.py", line 317, in register
    self._registry[cls.name] = cls()
  File "/home/fernando/opt/Slicer/Nightly/lib/Python/lib/python3.6/site-packages/pip/_internal/vcs/subversion.py", line 191, in __init__
    use_interactive = is_console_interactive()
  File "/home/fernando/opt/Slicer/Nightly/lib/Python/lib/python3.6/site-packages/pip/_internal/utils/misc.py", line 906, in is_console_interactive
    return sys.stdin is not None and sys.stdin.isatty()
AttributeError: 'PythonQtStdInRedirect' object has no attribute 'isatty'
```

So the problem is in `from pip._internal.req.req_install import InstallRequirement`. But this is probably not your problem, so I will discuss it with the Slicer devs if necessary.
> I'm not sure what you mean by "there is only a single CUDA version". Couldn't we just ignore the installed CUDA versions and choose the installer based on the drivers?

The current implementation is built on the assumption that there is only one computation backend. With the addition we are about to make this is no longer true, since a specific driver version supports multiple CUDA versions. Currently, the implementation restricts the field of possible installation candidates to one exact CUDA version. So this needs to be relaxed.

> So the problem is in `from pip._internal.req.req_install import InstallRequirement`. But this is probably not your problem, so I will discuss it with the Slicer devs if necessary.

Yeah, sorry, I can't offer support there. I'll fix that this works with `pip` but to get this working within other projects is out of scope (at least for now).

> The current implementation is built on the assumption that there is only one computation backend. With the addition we are about to make this is no longer true, since a specific driver version supports multiple CUDA versions. Currently, the implementation restricts the field of possible installation candidates to one exact CUDA version. So this needs to be relaxed.

Ok, I think I understand. Thanks!

> Yeah, sorry, I can't offer support there. I'll fix that this works with pip but to get this working within other projects is out of scope (at least for now).

Sure! I just thought I'd let you know in case it was helpful.

Fix patch:
diff --git a/light_the_torch/computation_backend.py b/light_the_torch/computation_backend.py
index 636c1cf..38e916c 100644
--- a/light_the_torch/computation_backend.py
+++ b/light_the_torch/computation_backend.py
@@ -1,7 +1,8 @@
+import os
 import re
 import subprocess
 from abc import ABC, abstractmethod
-from typing import Any
+from typing import Any, List, Optional
 
 __all__ = [
     "ComputationBackend",
@@ -75,24 +76,64 @@ def local_specifier(self) -> str:
         return f"cu{self.major}{self.minor}"
 
 
-NVCC_RELEASE_PATTERN = re.compile(r"release (?P<major>\d+)[.](?P<minor>\d+)")
-
-
-def detect_computation_backend() -> ComputationBackend:
-    fallback = CPUBackend()
+def detect_nvidia_driver() -> Optional[str]:
+    driver: Optional[str]
     try:
-        output = (
-            subprocess.check_output(
-                "nvcc --version", shell=True, stderr=subprocess.DEVNULL,
-            )
-            .decode("utf-8")
-            .strip()
+        output = subprocess.check_output(
+            "nvidia-smi --query-gpu=driver_version --format=csv",
+            shell=True,
+            stderr=subprocess.DEVNULL,
         )
-        match = NVCC_RELEASE_PATTERN.findall(output)
-        if not match:
-            return fallback
+        driver = output.decode("utf-8").splitlines()[-1]
+        pattern = re.compile(r"(\d+\.\d+)")  # match at least major and minor
+        if not pattern.match(driver):
+            driver = None
+    except subprocess.CalledProcessError:
+        driver = None
+    return driver
+
+
+def get_supported_cuda_version() -> Optional[str]:
+    def split(version_string: str) -> List[int]:
+        return [int(n) for n in version_string.split(".")]
+
+    nvidia_driver = detect_nvidia_driver()
+    if nvidia_driver is None:
+        return None
+
+    nvidia_driver = split(nvidia_driver)
+    cuda_version = None
+    if os.name == "nt":  # windows
+        # Table 3 from https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html
+        if nvidia_driver >= split("456.38"):
+            cuda_version = "11.1"
+        elif nvidia_driver >= split("451.22"):
+            cuda_version = "11.0"
+        elif nvidia_driver >= split("441.22"):
+            cuda_version = "10.2"
+        elif nvidia_driver >= split("418.96"):
+            cuda_version = "10.1"
+        elif nvidia_driver >= split("398.26"):
+            cuda_version = "9.2"
+    else:  # linux
+        # Table 1 from https://docs.nvidia.com/deploy/cuda-compatibility/index.html
+        if nvidia_driver >= split("450.80.02"):
+            cuda_version = "11.1"
+        elif nvidia_driver >= split("450.36.06"):
+            cuda_version = "11.0"
+        elif nvidia_driver >= split("440.33"):
+            cuda_version = "10.2"
+        elif nvidia_driver >= split("418.39"):
+            cuda_version = "10.1"
+        elif nvidia_driver >= split("396.26"):
+            cuda_version = "9.2"
+    return cuda_version
 
-        major, minor = match[0]
+
+def detect_computation_backend() -> ComputationBackend:
+    cuda_version = get_supported_cuda_version()
+    if cuda_version is None:
+        return CPUBackend()
+    else:
+        major, minor = cuda_version.split(".")
         return CUDABackend(int(major), int(minor))
-    except subprocess.CalledProcessError:
-        return fallback


Test patch:
diff --git a/tests/unit/test_computation_backend.py b/tests/unit/test_computation_backend.py
index 2f8e0ef..e70c100 100644
--- a/tests/unit/test_computation_backend.py
+++ b/tests/unit/test_computation_backend.py
@@ -107,17 +107,18 @@ def test_detect_computation_backend_unknown_release(mocker):
 
 
 def test_detect_computation_backend_cuda(mocker):
-    major = 42
-    minor = 21
+    major = 460
+    minor = 20
+    patch = 30
     mocker.patch(
         "light_the_torch.computation_backend.subprocess.check_output",
-        return_value=f"foo\nbar, release {major}.{minor}, baz".encode("utf-8"),
+        return_value=f"foo\n{major}.{minor}.{patch}".encode("utf-8"),
     )
 
     backend = cb.detect_computation_backend()
     assert isinstance(backend, cb.CUDABackend)
-    assert backend.major == major
-    assert backend.minor == minor
+    assert backend.major == 11
+    assert backend.minor == 1
 
 
 @skip_if_cuda_unavailable