Repository: cupy/cupy
Pull request: Eliminate random test ids
`pytest -n 2 tests` looks OK with pytest-xdist 2.3.0

Let's close #5623, while the parameters are still randomly generated in the test collection.

Depends #5599 because
- the new option `_ids=False` is implemented only for pytest-backend `cupy.testing.parameterize`; and
- the test collection should be fast to benefit from pytest-xdist.

Problem statement:
Avoid using random in test parameterization value
Currently, the following tests are using random values in test parameterization. I'd like to remove use of random values so that we can use pytest-xdist to run tests in parallel.

```
tests/cupy_tests/core_tests/test_ndarray_adv_indexing.py
tests/cupy_tests/core_tests/test_ndarray_scatter.py
```

When using pytest-xdist, each test process collects tests. The list of collected tests must match between all processes, but if random values are used, pytest-xdist fails with `Different tests were collected between ...` error.

Hints from issue discussion:

Commit message: Fix unpredictive test ids

Patch to review:
diff --git a/.pfnci/config.pbtxt b/.pfnci/config.pbtxt
index 703c81da6dd..5289f7093a6 100644
--- a/.pfnci/config.pbtxt
+++ b/.pfnci/config.pbtxt
@@ -43,7 +43,7 @@ configs {
   value {
     requirement {
       cpu: 8
-      memory: 36
+      memory: 50
       disk: 10
       gpu: 2
     }
@@ -61,7 +61,7 @@ configs {
   value {
     requirement {
       cpu: 8
-      memory: 36
+      memory: 50
       disk: 10
       gpu: 2
     }
diff --git a/.pfnci/linux/tests/_build_and_test.sh b/.pfnci/linux/tests/_build_and_test.sh
index 1703878e30b..047b53897b2 100644
--- a/.pfnci/linux/tests/_build_and_test.sh
+++ b/.pfnci/linux/tests/_build_and_test.sh
@@ -9,13 +9,14 @@ pytest_opts=(
     --timeout 300
     --maxfail 500
     --showlocals
+    --numprocesses 2
 )
 
 if [[ "${MARKER}" != "" ]]; then
     pytest_opts+=(-m "${MARKER}")
 fi
 
-python3 -m pip install --user pytest-timeout
+python3 -m pip install --user pytest-timeout pytest-xdist
 python3 -m pip install --user -v ".[all,test]"
 
 pushd tests
diff --git a/cupy/testing/_parameterized.py b/cupy/testing/_parameterized.py
index ec79b215dec..203f78c31ae 100644
--- a/cupy/testing/_parameterized.py
+++ b/cupy/testing/_parameterized.py
@@ -1,6 +1,5 @@
 import itertools
 import types
-import typing as tp  # NOQA
 import unittest
 
 from cupy.testing import _bundle
@@ -74,7 +73,7 @@ def new_v(self, *args, **kwargs):
     return (cls_name, mb, lambda method: method)
 
 
-def parameterize(*params):
+def parameterize(*params, **kwargs):
     """Generates test classes with given sets of additional attributes
 
     >>> @parameterize({"a": 1}, {"b": 2, "c": 3})
@@ -91,10 +90,11 @@ def parameterize(*params):
     """
     def f(cls):
         if issubclass(cls, unittest.TestCase):
+            assert not kwargs
             deco = _bundle.make_decorator(
                 lambda base: _parameterize_test_case_generator(base, params))
         else:
-            deco = _pytest_impl.parameterize(*params)
+            deco = _pytest_impl.parameterize(*params, **kwargs)
         return deco(cls)
     return f
 
diff --git a/cupy/testing/_pytest_impl.py b/cupy/testing/_pytest_impl.py
index 2f188acb0f2..1ef802aa660 100644
--- a/cupy/testing/_pytest_impl.py
+++ b/cupy/testing/_pytest_impl.py
@@ -38,9 +38,14 @@ def _cupy_testing_parameterize(self, _cupy_testing_param):
             self.__dict__.update(_cupy_testing_param)
 
 
-def parameterize(*params):
+def parameterize(*params, _ids=True):
     check_available()
-    param_name = cupy.testing._parameterized._make_class_name
+    if _ids:
+        param_name = cupy.testing._parameterized._make_class_name
+    else:
+        def param_name(_, i, param):
+            return str(i)
+
     # TODO(kataoka): Give better names (`id`).
     # For now, use legacy `_make_class_name` just for consistency. Here,
     # a generated name is `TestFoo::test_bar[_param_0_{...}]`, whereas
diff --git a/tests/conftest.py b/tests/conftest.py
index ec0e07fb216..521da96647c 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -1,7 +1,10 @@
+import collections
 import os
 import subprocess
 import sys
 
+import pytest
+
 
 # Enable `testdir` fixture to test `cupy.testing`.
 # `pytest_plugins` cannot be locally configured. See also
@@ -27,3 +30,37 @@ def pytest_configure(config):
     if _is_in_ci() and _is_pip_installed():
         print("***** Installed packages *****", flush=True)
         subprocess.check_call([sys.executable, '-m', 'pip', 'freeze', '--all'])
+    if config.pluginmanager.hasplugin("xdist"):
+        config.pluginmanager.register(DeferPlugin())
+
+
+# https://docs.pytest.org/en/latest/how-to/writing_hook_functions.html#optionally-using-hooks-from-3rd-party-plugins
+class DeferPlugin:
+    """Simple plugin to defer pytest-xdist hook functions."""
+
+    # Edit the environment variable `CUDA_VISIBLE_DEVICES` for each session.
+    # Cannot use `pytest_configure_node` nor `pytest_testnodeready` hook,
+    # because they are called in the `master` node (process).
+    # See also https://github.com/pytest-dev/pytest-xdist/issues/179.
+    @pytest.fixture(autouse=True, scope='session')
+    def _rotate_cuda_visible_devices(self, worker_id):
+        n_gpu = os.environ.get('CUPY_TEST_GPU_LIMIT')
+        if n_gpu is None:
+            return
+        n_gpu = int(n_gpu)
+
+        assert worker_id.startswith('gw')
+        w = int(worker_id[2:])
+
+        devices = os.environ.get('CUDA_VISIBLE_DEVICES')
+        if devices is None:
+            devices = [str(k) for k in range(n_gpu)]
+        else:
+            devices = devices.split(',')[:n_gpu]
+        devices = collections.deque(devices)
+        devices.rotate(w)
+        devices = ','.join(devices)
+        os.environ['CUDA_VISIBLE_DEVICES'] = devices
+        # With PyTest's default, the print will be shown as
+        # "--- Captured stdout setup ---" on failure.
+        print(f'CUDA_VISIBLE_DEVICES={devices}')
diff --git a/tests/cupy_tests/core_tests/test_ndarray_adv_indexing.py b/tests/cupy_tests/core_tests/test_ndarray_adv_indexing.py
index 733f26f6826..5f9bce8350f 100644
--- a/tests/cupy_tests/core_tests/test_ndarray_adv_indexing.py
+++ b/tests/cupy_tests/core_tests/test_ndarray_adv_indexing.py
@@ -110,6 +110,7 @@ def test_adv_getitem(self, xp, dtype):
     {'shape': (0, 3), 'indexes': (slice(None), [1])},
     # TODO(niboshi): pass the following commented out tests
     # {'shape': (0,), 'indexes': (False, True, True)},
+    _ids=False,  # Do not generate ids from randomly generated params
 )
 @testing.gpu
 class TestArrayAdvancedIndexingGetitemParametrized:
@@ -324,6 +325,7 @@ def test_invalid_adv_getitem(self):
      'indexes': (slice(None), numpy.random.choice([False, True], (3, 1)))},
     {'shape': (2, 3, 4),
      'indexes': numpy.random.choice([False, True], (1, 3))},
+    _ids=False,  # Do not generate ids from randomly generated params
 )
 @testing.gpu
 class TestArrayInvalidIndexAdvGetitem2:
@@ -462,6 +464,7 @@ def test_invalid_adv_getitem(self):
     {'shape': (0,), 'indexes': (), 'value': 1},
     # TODO(niboshi): pass the following commented out tests
     # {'shape': (0,), 'indexes': (False, True, True), 'value': 1},
+    _ids=False,  # Do not generate ids from randomly generated params
 )
 @testing.gpu
 class TestArrayAdvancedIndexingSetitemScalarValue:
@@ -579,6 +582,7 @@ def test_adv_setitem(self):
     # list indexes
     {'shape': (2, 3, 4), 'indexes': [1],
      'value': numpy.arange(3 * 4).reshape(3, 4)},
+    _ids=False,  # Do not generate ids from randomly generated params
 )
 @testing.gpu
 class TestArrayAdvancedIndexingVectorValue:
diff --git a/tests/cupy_tests/core_tests/test_ndarray_scatter.py b/tests/cupy_tests/core_tests/test_ndarray_scatter.py
index e09be7d4a73..2150a3c87a5 100644
--- a/tests/cupy_tests/core_tests/test_ndarray_scatter.py
+++ b/tests/cupy_tests/core_tests/test_ndarray_scatter.py
@@ -107,6 +107,7 @@
     {'shape': (2, 3, 4), 'slices': ([1, 1], 1), 'value': 1},
     {'shape': (2, 3, 4), 'slices': ([1], slice(1, 2)), 'value': 1},
     {'shape': (2, 3, 4), 'slices': ([[1]], slice(1, 2)), 'value': 1},
+    _ids=False,  # Do not generate ids from randomly generated params
 )
 @testing.gpu
 class TestScatterParametrized:
diff --git a/tests/cupy_tests/linalg_tests/test_einsum.py b/tests/cupy_tests/linalg_tests/test_einsum.py
index 35ac8b6325a..d60770dd942 100644
--- a/tests/cupy_tests/linalg_tests/test_einsum.py
+++ b/tests/cupy_tests/linalg_tests/test_einsum.py
@@ -7,6 +7,9 @@
 from cupy import testing
 
 
+rng = numpy.random.default_rng(seed=0)
+
+
 def _dec_shape(shape, dec):
     # Test smaller shape
     return tuple(1 if s == 1 else max(0, s - dec) for s in shape)
@@ -15,12 +18,12 @@ def _dec_shape(shape, dec):
 def _rand1_shape(shape, prob):
     # Test broadcast
     # If diagonals are "broadcasted" we can simply:
-    # return tuple(1 if numpy.random.rand() < prob else s for s in shape)
+    # return tuple(1 if rng.uniform() < prob else s for s in shape)
     table = {}
     new_shape = []
     for s in shape:
         if s not in table:
-            table[s] = 1 if numpy.random.rand() < prob else s
+            table[s] = 1 if rng.uniform() < prob else s
         new_shape.append(table[s])
     return tuple(new_shape)
 
@@ -46,7 +49,7 @@ def augment_einsum_testcases(*params):
                     if k.startswith('shape_'):
                         new_shape = _dec_shape(param[k], dec)
                         if drop:
-                            prob = numpy.random.rand()
+                            prob = rng.uniform()
                             new_shape = _rand1_shape(new_shape, prob)
                         param_new[k] = new_shape
                 param_new['_raw_params'] = {
diff --git a/tests/cupy_tests/logic_tests/test_type_test.py b/tests/cupy_tests/logic_tests/test_type_test.py
index e0c08f25605..95bf3595697 100644
--- a/tests/cupy_tests/logic_tests/test_type_test.py
+++ b/tests/cupy_tests/logic_tests/test_type_test.py
@@ -21,7 +21,7 @@ def test_argspec(self):
             numpy.int32(1), numpy.array([1, 2], numpy.int32),
             numpy.complex128(1), numpy.complex128(1j),
             numpy.complex128(1 + 1j),
-            None, object(), 'abc', '', int, numpy.int32]}))
+            None, 'abc', '', int, numpy.int32]}))
 class TestIsScalarValues(testing.NumpyAliasValuesTestBase):
 
     func = 'isscalar'
@@ -30,6 +30,15 @@ def setUp(self):
         self.args = (self.value,)
 
 
+class TestIsScalarValues2(testing.NumpyAliasValuesTestBase):
+
+    func = 'isscalar'
+
+    def setUp(self):
+        value = object()
+        self.args = (value,)
+
+
 @testing.parameterize(
     *testing.product({
         'value': [
diff --git a/tests/cupy_tests/sorting_tests/test_search.py b/tests/cupy_tests/sorting_tests/test_search.py
index 004e471d251..9b48cbd792a 100644
--- a/tests/cupy_tests/sorting_tests/test_search.py
+++ b/tests/cupy_tests/sorting_tests/test_search.py
@@ -1,5 +1,3 @@
-import unittest
-
 import numpy
 import pytest
 
@@ -10,7 +8,7 @@
 
 
 @testing.gpu
-class TestSearch(unittest.TestCase):
+class TestSearch:
 
     @testing.for_all_dtypes(no_complex=True)
     @testing.numpy_cupy_allclose()
@@ -176,25 +174,26 @@ def _skip_cuda90(dtype):
     'backend': ('device', 'block'),
 }))
 @testing.gpu
-@unittest.skipUnless(cupy.cuda.cub.available, 'The CUB routine is not enabled')
-class TestCubReduction(unittest.TestCase):
+@pytest.mark.skipif(
+    not cupy.cuda.cub.available, reason='The CUB routine is not enabled')
+class TestCubReduction:
 
+    @pytest.fixture(autouse=True)
     def setUp(self):
         self.order, self.axis = self.order_and_axis
-        self.old_routine_accelerators = _acc.get_routine_accelerators()
-        self.old_reduction_accelerators = _acc.get_reduction_accelerators()
+        old_routine_accelerators = _acc.get_routine_accelerators()
+        old_reduction_accelerators = _acc.get_reduction_accelerators()
         if self.backend == 'device':
             if self.axis is not None:
-                raise unittest.SkipTest('does not support')
+                pytest.skip('does not support')
             _acc.set_routine_accelerators(['cub'])
             _acc.set_reduction_accelerators([])
         elif self.backend == 'block':
             _acc.set_routine_accelerators([])
             _acc.set_reduction_accelerators(['cub'])
-
-    def tearDown(self):
-        _acc.set_routine_accelerators(self.old_routine_accelerators)
-        _acc.set_reduction_accelerators(self.old_reduction_accelerators)
+        yield
+        _acc.set_routine_accelerators(old_routine_accelerators)
+        _acc.set_reduction_accelerators(old_reduction_accelerators)
 
     @testing.for_dtypes('bhilBHILefdFD')
     @testing.numpy_cupy_allclose(rtol=1E-5, contiguous_check=False)
@@ -273,7 +272,7 @@ def test_cub_argmax(self, xp, dtype):
     'is_module': [True, False],
     'shape': [(3, 4), ()],
 }))
-class TestArgMinMaxDtype(unittest.TestCase):
+class TestArgMinMaxDtype:
 
     @testing.for_dtypes(
         dtypes=[numpy.int8, numpy.int16, numpy.int32, numpy.int64],
@@ -298,7 +297,7 @@ def test_argminmax_dtype(self, in_dtype, result_dtype):
     {'cond_shape': (3, 4),    'x_shape': (2, 3, 4), 'y_shape': (4,)},
 )
 @testing.gpu
-class TestWhereTwoArrays(unittest.TestCase):
+class TestWhereTwoArrays:
 
     @testing.for_all_dtypes_combination(
         names=['cond_type', 'x_type', 'y_type'])
@@ -320,7 +319,7 @@ def test_where_two_arrays(self, xp, cond_type, x_type, y_type):
     {'cond_shape': (3, 4)},
 )
 @testing.gpu
-class TestWhereCond(unittest.TestCase):
+class TestWhereCond:
 
     @testing.for_all_dtypes()
     @testing.numpy_cupy_array_equal()
@@ -331,7 +330,7 @@ def test_where_cond(self, xp, dtype):
 
 
 @testing.gpu
-class TestWhereError(unittest.TestCase):
+class TestWhereError:
 
     def test_one_argument(self):
         for xp in (numpy, cupy):
@@ -347,9 +346,10 @@ def test_one_argument(self):
     {'array': numpy.empty((0,))},
     {'array': numpy.empty((0, 2))},
     {'array': numpy.empty((0, 2, 0))},
+    _ids=False,  # Do not generate ids from randomly generated params
 )
 @testing.gpu
-class TestNonzero(unittest.TestCase):
+class TestNonzero:
 
     @testing.for_all_dtypes()
     @testing.numpy_cupy_array_equal()
@@ -364,7 +364,7 @@ def test_nonzero(self, xp, dtype):
 )
 @testing.gpu
 @testing.with_requires('numpy>=1.17.0')
-class TestNonzeroZeroDimension(unittest.TestCase):
+class TestNonzeroZeroDimension:
 
     @testing.for_all_dtypes()
     @testing.numpy_cupy_array_equal()
@@ -382,9 +382,10 @@ def test_nonzero(self, xp, dtype):
     {'array': numpy.empty((0,))},
     {'array': numpy.empty((0, 2))},
     {'array': numpy.empty((0, 2, 0))},
+    _ids=False,  # Do not generate ids from randomly generated params
 )
 @testing.gpu
-class TestFlatNonzero(unittest.TestCase):
+class TestFlatNonzero:
 
     @testing.for_all_dtypes()
     @testing.numpy_cupy_array_equal()
@@ -399,9 +400,10 @@ def test_flatnonzero(self, xp, dtype):
     {'array': numpy.empty((0,))},
     {'array': numpy.empty((0, 2))},
     {'array': numpy.empty((0, 2, 0))},
+    _ids=False,  # Do not generate ids from randomly generated params
 )
 @testing.gpu
-class TestArgwhere(unittest.TestCase):
+class TestArgwhere:
 
     @testing.for_all_dtypes()
     @testing.numpy_cupy_array_equal()
@@ -416,7 +418,7 @@ def test_argwhere(self, xp, dtype):
 )
 @testing.gpu
 @testing.with_requires('numpy>=1.18')
-class TestArgwhereZeroDimension(unittest.TestCase):
+class TestArgwhereZeroDimension:
 
     @testing.for_all_dtypes()
     @testing.numpy_cupy_array_equal()
@@ -426,7 +428,7 @@ def test_argwhere(self, xp, dtype):
 
 
 @testing.gpu
-class TestNanArgMin(unittest.TestCase):
+class TestNanArgMin:
 
     @testing.for_all_dtypes(no_complex=True)
     @testing.numpy_cupy_allclose()
@@ -518,7 +520,7 @@ def test_nanargmin_zero_size_axis1(self, xp, dtype):
 
 
 @testing.gpu
-class TestNanArgMax(unittest.TestCase):
+class TestNanArgMax:
 
     @testing.for_all_dtypes(no_complex=True)
     @testing.numpy_cupy_allclose()
@@ -627,7 +629,7 @@ def test_nanargmax_zero_size_axis1(self, xp, dtype):
         'side': ['left', 'right'],
         'shape': [(), (10,), (6, 3, 3)]})
 )
-class TestSearchSorted(unittest.TestCase):
+class TestSearchSorted:
 
     @testing.for_all_dtypes(no_bool=True)
     @testing.numpy_cupy_array_equal()
@@ -642,7 +644,7 @@ def test_searchsorted(self, xp, dtype):
 @testing.parameterize(
     {'side': 'left'},
     {'side': 'right'})
-class TestSearchSortedNanInf(unittest.TestCase):
+class TestSearchSortedNanInf:
 
     @testing.numpy_cupy_array_equal()
     def test_searchsorted_nanbins(self, xp):
@@ -702,7 +704,7 @@ def test_searchsorted_minf(self, xp):
 
 
 @testing.gpu
-class TestSearchSortedInvalid(unittest.TestCase):
+class TestSearchSortedInvalid:
 
     # Cant test unordered bins due to numpy undefined
     # behavior for searchsorted
@@ -716,7 +718,7 @@ def test_searchsorted_ndbins(self):
 
 
 @testing.gpu
-class TestSearchSortedWithSorter(unittest.TestCase):
+class TestSearchSortedWithSorter:
 
     @testing.numpy_cupy_array_equal()
     def test_sorter(self, xp):
diff --git a/tests/cupy_tests/testing_tests/test_parameterized.py b/tests/cupy_tests/testing_tests/test_parameterized.py
index 8907e8b0364..226489a4698 100644
--- a/tests/cupy_tests/testing_tests/test_parameterized.py
+++ b/tests/cupy_tests/testing_tests/test_parameterized.py
@@ -45,6 +45,9 @@ def f(x):
 
 class C(object):
 
+    def __repr__(self):
+        return '<C object>'
+
     def __call__(self, x):
         return x
 
diff --git a/tests/cupyx_tests/scipy_tests/ndimage_tests/test_filters.py b/tests/cupyx_tests/scipy_tests/ndimage_tests/test_filters.py
index 00b1691846c..76a31f1d1eb 100644
--- a/tests/cupyx_tests/scipy_tests/ndimage_tests/test_filters.py
+++ b/tests/cupyx_tests/scipy_tests/ndimage_tests/test_filters.py
@@ -480,7 +480,7 @@ def shift_pyfunc(src, dst):
     testing.product_dict(
         testing.product({
             'filter': ['generic_filter1d'],
-            'func_or_kernel': [(shift_raw, shift_pyfunc)],
+            'func_or_kernel': [('shift_raw', 'shift_pyfunc')],
             'axis': [0, 1, -1],
         }),
 
@@ -507,11 +507,20 @@ def shift_pyfunc(src, dst):
 @testing.gpu
 @testing.with_requires('scipy')
 class TestGeneric1DFilter(FilterTestCaseBase):
+    _func_or_kernels = {
+        'shift_raw': shift_raw,
+        'shift_pyfunc': shift_pyfunc,
+    }
+
+    def get_func_or_kernel(self, xp):
+        return self._func_or_kernels[
+            self.func_or_kernel[1 if xp == numpy else 0]]
+
     @testing.numpy_cupy_allclose(atol=1e-5, rtol=1e-5, scipy_name='scp')
     def test_filter(self, xp, scp):
         # Need to deal with the different versions of the functions given to
         # numpy vs cupy
-        self.function = self.func_or_kernel[int(xp == numpy)]
+        self.function = self.get_func_or_kernel(xp)
         return self._filter(xp, scp)
 
 


Code review comments:
diff_hunk: @@ -27,3 +30,37 @@ def pytest_configure(config):
     if _is_in_ci() and _is_pip_installed():
         print("***** Installed packages *****", flush=True)
         subprocess.check_call([sys.executable, '-m', 'pip', 'freeze', '--all'])
+    if config.pluginmanager.hasplugin("xdist"):
+        config.pluginmanager.register(DeferPlugin())
+
+
+# https://docs.pytest.org/en/latest/how-to/writing_hook_functions.html#optionally-using-hooks-from-3rd-party-plugins
+class DeferPlugin:
+    """Simple plugin to defer pytest-xdist hook functions."""
+
+    # Edit the environment variable `CUDA_VISIBLE_DEVICES` for each session.
+    # Cannot use `pytest_configure_node` nor `pytest_testnodeready` hook,
+    # because they are called in the `master` node (process).
+    # See also https://github.com/pytest-dev/pytest-xdist/issues/179.
+    @pytest.fixture(autouse=True, scope='session')
+    def _rotate_cuda_visible_devices(self, worker_id):
+        n_gpu = os.environ.get('CUPY_TEST_GPU_LIMIT')
+        if n_gpu is None:
+            return
path: tests/conftest.py
text: @user1:
```suggestion
            print('Tip: when using pytest-xdist, you can automatically rotate'
                  ' CUDA_VISIBLE_DEVICES for each test worker by setting'
                  ' CUPY_TEST_GPU_LIMIT environment variable.')
            return
```

diff_hunk: @@ -27,3 +30,37 @@ def pytest_configure(config):
     if _is_in_ci() and _is_pip_installed():
         print("***** Installed packages *****", flush=True)
         subprocess.check_call([sys.executable, '-m', 'pip', 'freeze', '--all'])
+    if config.pluginmanager.hasplugin("xdist"):
+        config.pluginmanager.register(DeferPlugin())
+
+
+# https://docs.pytest.org/en/latest/how-to/writing_hook_functions.html#optionally-using-hooks-from-3rd-party-plugins
+class DeferPlugin:
+    """Simple plugin to defer pytest-xdist hook functions."""
+
+    # Edit the environment variable `CUDA_VISIBLE_DEVICES` for each session.
+    # Cannot use `pytest_configure_node` nor `pytest_testnodeready` hook,
+    # because they are called in the `master` node (process).
+    # See also https://github.com/pytest-dev/pytest-xdist/issues/179.
+    @pytest.fixture(autouse=True, scope='session')
+    def _rotate_cuda_visible_devices(self, worker_id):
+        n_gpu = os.environ.get('CUPY_TEST_GPU_LIMIT')
path: tests/conftest.py
text: @user1:
Can we ensure that CUDA is not loaded (so that we can overwrite `CUDA_VISIBLE_DEVICES`)? Maybe `assert 'cupy' not in sys.modules` can be used.

@author:
I tried
```python
        assert not {
            'cupy', 'cupy_backends', 'cupyx',
        }.intersection(sys.modules)
```
in this function, but it failed.  However, `CUDA_VISIBLE_DEVICES` looks working to me because `cupy.cuda.Device(0).attributes["PciBusId"]` returned different values.

I didn't come up with an idea for testing this in a portable manner.

@user1:
I see, thanks for testing!