Repository: cupy/cupy

Issue:
Replace `CUDA_VERSION` with `cudaRuntimeGetVersion()` for `if`/`IF` conditionals
This is needed to support CUDA Python as `CUDA_VERSION` in CUDA Python does not mean the current CUDA version.


Discussion:
`CUDA_VERSION` in Cython files?
```
$ find . -name '*.pyx' -or -name '*.px[id]' | xargs grep CUDA_VERSION
./cupy/cuda/memory.pyx:    if CUDA_VERSION < 11020:
./cupy/fft/_callback.pyx:                        '-E', 'CUDA_VERSION=' + _build_ver,
./cupy_backends/cuda/api/_driver_extern.pxi:    int CUDA_VERSION
./cupy_backends/cuda/api/_runtime_typedef.pxi:    IF CUDA_VERSION > 0:
./cupy_backends/cuda/api/_runtime_typedef.pxi:    IF CUDA_VERSION >= 11000:
./cupy_backends/cuda/api/_runtime_typedef.pxi:    ELIF CUDA_VERSION >= 10000:
./cupy_backends/cuda/api/_runtime_typedef.pxi:    ELIF CUDA_VERSION == 9020:
./cupy_backends/cuda/api/runtime.pyx:    IF CUDA_VERSION > 0 or HIP_VERSION > 0:
./cupy_backends/cuda/api/runtime.pyx:    IF CUDA_VERSION >= 9020:
./cupy_backends/cuda/api/runtime.pyx:    IF CUDA_VERSION >= 10000:
./cupy_backends/cuda/api/runtime.pyx:    IF CUDA_VERSION >= 11000:
./cupy_backends/cuda/api/runtime.pyx:    if CUDA_VERSION < 11020:
./cupy_backends/cuda/api/runtime.pyx:    if CUDA_VERSION < 11020:
./cupy_backends/cuda/api/runtime.pyx:    IF CUDA_VERSION > 0:
./cupy_backends/cuda/api/runtime.pyx:    if CUDA_VERSION < 11020:
./cupy_backends/cuda/api/runtime.pyx:    if CUDA_VERSION < 11020:
./cupy_backends/cuda/api/runtime.pyx:    if CUDA_VERSION < 11020:
./cupy_backends/cuda/api/runtime.pyx:    if CUDA_VERSION < 11020:
./cupy_backends/cuda/api/runtime.pyx:    if CUDA_VERSION < 11020:
./cupy_backends/cuda/api/runtime.pyx:    if CUDA_VERSION < 11020:
./cupy_backends/cuda/api/runtime.pyx:IF CUDA_VERSION >= 10000:
./cupy_backends/cuda/api/runtime.pyx:    if CUDA_VERSION < 10000:
./cupy_backends/cuda/api/driver.pyx:    # Note: CUDA_VERSION is defined either in CUDA Python or _driver_extern.pxi
./cupy_backends/cuda/api/driver.pyx:    if CUDA_VERSION > 0:
./cupy_backends/cuda/api/driver.pyx:        return CUDA_VERSION
./cupy_backends/cuda/api/driver.pyx:    if CUDA_VERSION < 9000:
./cupy_backends/cuda/libs/nvrtc.pyx:    if CUDA_VERSION < 11020 or runtime._is_hip_environment:
./cupy_backends/cuda/libs/nvrtc.pyx:    if CUDA_VERSION < 11010 or runtime._is_hip_environment:
```
Yes, but `_runtime_typedef.pxi` can be kept as-is as `IF CUDA_VERSION` condition cannot be removed and it is not used by CUDA Python.
Seems some codes depend on Cython's dead code elimination:

https://github.com/cupy/cupy/blob/f81c85f1239d1b3da3fb3d7be3dd701b988526da/cupy_backends/cuda/api/driver.pyx#L278-L286

https://github.com/cupy/cupy/blob/f81c85f1239d1b3da3fb3d7be3dd701b988526da/cupy_backends/cuda/api/runtime.pyx#L771-L777

As only a few modules, `driver`, `runtme` and `nvrtc`, import CUDA Python, giving an alias to `CUDA_VERSION` in a C header and importing it may be a handy way.
You can remove these conditionals as we no longer support CUDA 9.x 😄 

Besides that, in general, build-time conditionals (`IF`) must be replaced with `if`, as the CuPy built with CUDA Python will not depend on a specific CUDA version.

> As only a few modules, driver, runtme and nvrtc, import CUDA Python, giving an alias to CUDA_VERSION in a C header and importing it may be a handy way.

We are going to use CUDA Python in other modules e.g. cublas once CUDA Python support is available
`get_build_version` wants `CUDA_VERSION` at the build time. How about this?

https://github.com/cupy/cupy/blob/f81c85f1239d1b3da3fb3d7be3dd701b988526da/cupy_backends/cuda/api/driver.pyx#L78-L79


I think it's ok to leave it as-is. `show_config()` uses it like this:
https://github.com/cupy/cupy/blob/f81c85f1239d1b3da3fb3d7be3dd701b988526da/cupyx/_runtime.py#L152-L159

Other use of `get_build_version()` may need to be rewritten.
Okay, we still have the build-time CUDA version so we need to see and bake `CUDA_VERSION` in `cuda.h` there instad of CUDA Python's somehow, right? Also, I'll check where `get_build_version()` is used.
Yes. Currently only `{driver,runtime,nvrtc}.pyx` uses CUDA Python, whereas other modules still directly link to CUDA Toolkit libraries. So we still need to burn-in build-time `CUDA_VERSION` from the `cuda.h` (not CUDA Python). However CUDA Python also exports `CUDA_VERSION`, so we have to do this bunr-in somewhere else than `driver.pyx`.
My idea is to do the burn-in in every module using `driver.get_build_version()`. For example, burn-in `CUDA_VERSION` in `cupy_backends.cuda.libs.cusolver`, then use it in 
https://github.com/cupy/cupy/blob/f81c85f1239d1b3da3fb3d7be3dd701b988526da/cupy/cusolver.pyx#L109
Let me clarify that `driver.get_build_version()` returns CUDA Python's `CUDA_VERSION` if it is left as it-is (when CUDA Python is enabled). So I think we need to change `driver.get_build_version()` somehow to see `cuda.h`'s `CUDA_VERSION`.

Also, I agree that making each module have the burn-in `CUDA_VERSION` is reasonable, though I'm not sure `driver.get_build_version()` is expanded to `CUDA_VERSION` at build-time and it may be left as a runtime call. How about directly declaring extern `CUDA_VERSION` from `cupy_backend.h` to make it burned-in at each module?
Ah, let me make myself understand the case further...
> Let me clarify that `driver.get_build_version()` returns CUDA Python's `CUDA_VERSION` if it is left as it-is (when CUDA Python is enabled).

Yes.

> So I think we need to change `driver.get_build_version()` somehow to see `cuda.h`'s `CUDA_VERSION`.

I think it's better to expose `CUDA_VERSION` as `cupy_backends.cuda.libs.cusolver._get_cuda_build_version()` etc.
Phew, now I got your point about how `driver.get_build_version` should work. I misunderstood that.
In an offline talk, we found that Cython looks up and finds `cuda.h`'s `CUDA_VERSION` first even when `cuda.ccuda.*` is cimported.
Let's prefix all `compile_time_env`s with `CUPY_` to avoid confusion and conflict like this 😅 
https://github.com/cupy/cupy/blob/master/cupy_setup_build.py#L947-L957

Resolving patch:
diff --git a/cupy/cuda/compiler.py b/cupy/cuda/compiler.py
--- a/cupy/cuda/compiler.py
+++ b/cupy/cuda/compiler.py
@@ -160,8 +160,12 @@ def _get_arch_for_options_for_nvrtc(arch=None):
     # generate cubin (SASS) instead of PTX. See #5097 for details.
     if arch is None:
         arch = _get_arch()
+    if driver._is_cuda_python():
+        version = runtime.runtimeGetVersion()
+    else:
+        version = _cuda_hip_version
     if (
-        not _use_ptx and _cuda_hip_version >= 11010
+        not _use_ptx and version >= 11010
         and arch < _get_max_compute_capability()
     ):
         return f'-arch=sm_{arch}', 'cubin'
diff --git a/cupyx/scipy/sparse/csr.py b/cupyx/scipy/sparse/csr.py
--- a/cupyx/scipy/sparse/csr.py
+++ b/cupyx/scipy/sparse/csr.py
@@ -9,7 +9,6 @@
 except ImportError:
     _scipy_available = False
 
-from cupy_backends.cuda.api import driver
 import cupy
 from cupy._core import _accelerator
 from cupy.cuda import cub
@@ -186,7 +185,7 @@ def __mul__(self, other):
                                > self.indptr.size * self.indptr.dtype.itemsize)
                 # CUB spmv is buggy since CUDA 11.0, see
                 # https://github.com/cupy/cupy/issues/3822#issuecomment-782607637
-                is_cub_safe &= (driver.get_build_version() < 11000)
+                is_cub_safe &= (cub._get_cuda_build_version() < 11000)
                 for accelerator in _accelerator.get_routine_accelerators():
                     if (accelerator == _accelerator.ACCELERATOR_CUB
                             and not runtime.is_hip


Test patch:
diff --git a/tests/cupy_tests/core_tests/test_raw.py b/tests/cupy_tests/core_tests/test_raw.py
--- a/tests/cupy_tests/core_tests/test_raw.py
+++ b/tests/cupy_tests/core_tests/test_raw.py
@@ -16,7 +16,6 @@
 from cupy._core import _accelerator
 from cupy.cuda import compiler
 from cupy.cuda import memory
-from cupy_backends.cuda.api import driver
 
 
 _test_source1 = r'''
@@ -636,10 +635,13 @@ def test_dynamical_parallelism_compile_failure(self):
         x = cupy.zeros((N,), dtype=cupy.float32)
         use_ptx = os.environ.get(
             'CUPY_COMPILE_WITH_PTX', False)
-        if (self.backend == 'nvrtc' and (
-                use_ptx or (
-                not cupy.cuda.runtime.is_hip
-                and driver.get_build_version() < 11010))):
+        if self.backend == 'nvrtc' and (
+                use_ptx or
+                (cupy.cuda.driver._is_cuda_python()
+                 and cupy.cuda.runtime.runtimeGetVersion() < 11010) or
+                (not cupy.cuda.driver._is_cuda_python()
+                 and not cupy.cuda.runtime.is_hip
+                 and cupy.cuda.driver.get_build_version() < 11010)):
             # raised when calling ls.complete()
             error = cupy.cuda.driver.CUDADriverError
         else:  # nvcc, hipcc, hiprtc
diff --git a/tests/cupy_tests/cuda_tests/test_memory.py b/tests/cupy_tests/cuda_tests/test_memory.py
--- a/tests/cupy_tests/cuda_tests/test_memory.py
+++ b/tests/cupy_tests/cuda_tests/test_memory.py
@@ -52,9 +52,14 @@ def check(self, device_id):
                 raise unittest.SkipTest('HIP does not support managed memory')
             if self.allocator is memory.malloc_async:
                 raise unittest.SkipTest('HIP does not support async mempool')
-        elif cupy.cuda.driver.get_build_version() < 11020:
-            raise unittest.SkipTest('malloc_async is supported since '
-                                    'CUDA 11.2')
+        else:
+            if cupy.cuda.driver._is_cuda_python():
+                version = cupy.cuda.runtime.runtimeGetVersion()
+            else:
+                version = cupy.cuda.driver.get_build_version()
+            if version < 11020:
+                raise unittest.SkipTest('malloc_async is supported since '
+                                        'CUDA 11.2')
 
         size = 24
         shape = (2, 3)
@@ -741,7 +746,11 @@ def setUp(self):
         if self.mempool == 'MemoryAsyncPool':
             if cupy.cuda.runtime.is_hip:
                 pytest.skip('HIP does not support async allocator')
-            if cupy.cuda.driver.get_build_version() < 11020:
+            if cupy.cuda.driver._is_cuda_python():
+                version = cupy.cuda.runtime.runtimeGetVersion()
+            else:
+                version = cupy.cuda.driver.get_build_version()
+            if version < 11020:
                 pytest.skip('malloc_async is supported since CUDA 11.2')
             if cupy.cuda.runtime.driverGetVersion() < 11030:
                 pytest.skip('pool statistics is supported with driver 11.3+')
@@ -977,7 +986,11 @@ def test(self):
 @testing.gpu
 @pytest.mark.skipif(cupy.cuda.runtime.is_hip,
                     reason='HIP does not support async allocator')
-@pytest.mark.skipif(cupy.cuda.driver.get_build_version() < 11020,
+@pytest.mark.skipif(cupy.cuda.driver._is_cuda_python()
+                    and cupy.cuda.runtime.runtimeGetVersion() < 11020,
+                    reason='malloc_async is supported since CUDA 11.2')
+@pytest.mark.skipif(not cupy.cuda.driver._is_cuda_python()
+                    and cupy.cuda.driver.get_build_version() < 11020,
                     reason='malloc_async is supported since CUDA 11.2')
 class TestMallocAsync(unittest.TestCase):
 
@@ -1043,7 +1056,11 @@ def test_stream5(self):
 @testing.gpu
 @pytest.mark.skipif(cupy.cuda.runtime.is_hip,
                     reason='HIP does not support async allocator')
-@pytest.mark.skipif(cupy.cuda.driver.get_build_version() < 11020,
+@pytest.mark.skipif(cupy.cuda.driver._is_cuda_python()
+                    and cupy.cuda.runtime.runtimeGetVersion() < 11020,
+                    reason='malloc_async is supported since CUDA 11.2')
+@pytest.mark.skipif(not cupy.cuda.driver._is_cuda_python()
+                    and cupy.cuda.driver.get_build_version() < 11020,
                     reason='malloc_async is supported since CUDA 11.2')
 class TestMemoryAsyncPool(unittest.TestCase):
 
diff --git a/tests/cupy_tests/cuda_tests/test_stream.py b/tests/cupy_tests/cuda_tests/test_stream.py
--- a/tests/cupy_tests/cuda_tests/test_stream.py
+++ b/tests/cupy_tests/cuda_tests/test_stream.py
@@ -115,8 +115,6 @@ def _callback(s, _, t):
 
     @unittest.skipIf(cuda.runtime.is_hip,
                      'HIP does not support launch_host_func')
-    @unittest.skipIf(cuda.driver.get_build_version() < 10000,
-                     'Only CUDA 10.0+ supports this')
     def test_launch_host_func(self):
         N = 100
         cupy_arrays = [testing.shaped_random((2, 3)) for _ in range(N)]
@@ -268,8 +266,6 @@ def test_get_and_add_callback(self):
 
     @unittest.skipIf(cuda.runtime.is_hip,
                      'HIP does not support launch_host_func')
-    @unittest.skipIf(cuda.driver.get_build_version() < 10000,
-                     'Only CUDA 10.0+ supports this')
     def test_launch_host_func(self):
         N = 100
         cupy_arrays = [testing.shaped_random((2, 3)) for _ in range(N)]
diff --git a/tests/cupyx_tests/scipy_tests/sparse_tests/test_csr.py b/tests/cupyx_tests/scipy_tests/sparse_tests/test_csr.py
--- a/tests/cupyx_tests/scipy_tests/sparse_tests/test_csr.py
+++ b/tests/cupyx_tests/scipy_tests/sparse_tests/test_csr.py
@@ -1760,7 +1760,7 @@ def test_getitem_slice_stop_too_large(self, xp, sp):
 @testing.with_requires('scipy')
 @testing.gpu
 @pytest.mark.skipif(
-    driver.get_build_version() >= 11000,
+    cupy.cuda.cub._get_cuda_build_version() >= 11000,
     reason='CUDA built-in CUB SpMV is buggy, see cupy/cupy#3822')
 @pytest.mark.skipif(runtime.is_hip, reason='hipCUB does not provide spmv')
 @pytest.mark.skipif(