Repository: cupy/cupy

Issue:
Installation with pip results in broken installation
### Description

This is reopening #7253 since it still persists with CUDA 12.1 and cupy-cuda12x on Python 3.11 in a fresh environment. CC @kmaehashi 

Importing `cupy` fails with an `ImportError`:

```
In [1]: import cupy
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
File ~\miniconda3\envs\spaconv311\Lib\site-packages\cupy\__init__.py:17
     16 try:
---> 17     from cupy import _core  # NOQA
     18 except ImportError as exc:

File ~\miniconda3\envs\spaconv311\Lib\site-packages\cupy\_core\__init__.py:3
      1 # mypy: ignore-errors
----> 3 from cupy._core import core  # NOQA
      4 from cupy._core import fusion  # NOQA

File cupy\_core\core.pyx:1, in init cupy._core.core()

File ~\miniconda3\envs\spaconv311\Lib\site-packages\cupy\cuda\__init__.py:8
      7 from cupy._environment import get_hipcc_path  # NOQA
----> 8 from cupy.cuda import compiler  # NOQA
      9 from cupy.cuda import device  # NOQA

File ~\miniconda3\envs\spaconv311\Lib\site-packages\cupy\cuda\compiler.py:13
     11 import warnings
---> 13 from cupy.cuda import device
     14 from cupy.cuda import function

File cupy\cuda\device.pyx:1, in init cupy.cuda.device()

ImportError: DLL load failed while importing runtime: The specified module could not be found.

The above exception was the direct cause of the following exception:

ImportError                               Traceback (most recent call last)
Cell In[1], line 1
----> 1 import cupy

File ~\miniconda3\envs\spaconv311\Lib\site-packages\cupy\__init__.py:19
     17     from cupy import _core  # NOQA
     18 except ImportError as exc:
---> 19     raise ImportError(f'''
     20 ================================================================
     21 {_environment._diagnose_import_error()}
     22
     23 Original error:
     24   {type(exc).__name__}: {exc}
     25 ================================================================
     26 ''') from exc
     29 from cupy import cuda  # NOQA
     30 # Do not make `cupy.cupyx` available because it is confusing.

ImportError:
================================================================
Failed to import CuPy.

If you installed CuPy via wheels (cupy-cudaXXX or cupy-rocm-X-X), make sure that the package matches with the version of CUDA or ROCm installed.

On Linux, you may need to set LD_LIBRARY_PATH environment variable depending on how you installed CUDA/ROCm.
On Windows, try setting CUDA_PATH environment variable.

Check the Installation Guide for details:
  https://docs.cupy.dev/en/latest/install.html

CUDA Path: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1
DLL dependencies:
  KERNEL32.dll -> C:\WINDOWS\System32\KERNEL32.DLL
  MSVCP140.dll -> C:\Users\weber\miniconda3\envs\spaconv311\MSVCP140.dll
  VCRUNTIME140.dll -> C:\Users\weber\miniconda3\envs\spaconv311\VCRUNTIME140.dll
  api-ms-win-crt-convert-l1-1-0.dll -> C:\WINDOWS\System32\ucrtbase.dll
  api-ms-win-crt-environment-l1-1-0.dll -> C:\WINDOWS\System32\ucrtbase.dll
  api-ms-win-crt-filesystem-l1-1-0.dll -> C:\WINDOWS\System32\ucrtbase.dll
  api-ms-win-crt-heap-l1-1-0.dll -> C:\WINDOWS\System32\ucrtbase.dll
  api-ms-win-crt-runtime-l1-1-0.dll -> C:\WINDOWS\System32\ucrtbase.dll
  api-ms-win-crt-stdio-l1-1-0.dll -> C:\WINDOWS\System32\ucrtbase.dll
  cuTENSOR.dll -> not found
  cublas64_12.dll -> not found
  cudart64_12.dll -> not found
  cudnn64_8.dll -> not found
  cufft64_11.dll -> not found
  curand64_10.dll -> not found
  cusolver64_11.dll -> not found
  cusparse64_12.dll -> not found
  nvcuda.dll -> not found
  nvrtc64_120_0.dll -> not found
  python311.dll -> C:\Users\weber\miniconda3\envs\spaconv311\python311.dll

Original error:
  ImportError: DLL load failed while importing runtime: The specified module could not be found.
================================================================
```

The environment variable is set correctly: `CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1`

### To Reproduce

```py
import cupy
```


### Installation

Wheel (`pip install cupy-***`)

### Environment

```
(spaconv311) C:\Users\weber>python -c "import cupy; cupy.show_config()"
Traceback (most recent call last):
  File "C:\Users\weber\miniconda3\envs\spaconv311\Lib\site-packages\cupy\__init__.py", line 17, in <module>
    from cupy import _core  # NOQA
    ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\weber\miniconda3\envs\spaconv311\Lib\site-packages\cupy\_core\__init__.py", line 3, in <module>
    from cupy._core import core  # NOQA
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "cupy\_core\core.pyx", line 1, in init cupy._core.core
  File "C:\Users\weber\miniconda3\envs\spaconv311\Lib\site-packages\cupy\cuda\__init__.py", line 8, in <module>
    from cupy.cuda import compiler  # NOQA
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\weber\miniconda3\envs\spaconv311\Lib\site-packages\cupy\cuda\compiler.py", line 13, in <module>
    from cupy.cuda import device
  File "cupy\cuda\device.pyx", line 1, in init cupy.cuda.device
ImportError: DLL load failed while importing runtime: The specified module could not be found.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\weber\miniconda3\envs\spaconv311\Lib\site-packages\cupy\__init__.py", line 19, in <module>
    raise ImportError(f'''
ImportError:
================================================================
Failed to import CuPy.

If you installed CuPy via wheels (cupy-cudaXXX or cupy-rocm-X-X), make sure that the package matches with the version of CUDA or ROCm installed.

On Linux, you may need to set LD_LIBRARY_PATH environment variable depending on how you installed CUDA/ROCm.
On Windows, try setting CUDA_PATH environment variable.

Check the Installation Guide for details:
  https://docs.cupy.dev/en/latest/install.html

CUDA Path: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1
DLL dependencies:
  KERNEL32.dll -> C:\WINDOWS\System32\KERNEL32.DLL
  MSVCP140.dll -> C:\Users\weber\miniconda3\envs\spaconv311\MSVCP140.dll
  VCRUNTIME140.dll -> C:\Users\weber\miniconda3\envs\spaconv311\VCRUNTIME140.dll
  api-ms-win-crt-convert-l1-1-0.dll -> C:\WINDOWS\System32\ucrtbase.dll
  api-ms-win-crt-environment-l1-1-0.dll -> C:\WINDOWS\System32\ucrtbase.dll
  api-ms-win-crt-filesystem-l1-1-0.dll -> C:\WINDOWS\System32\ucrtbase.dll
  api-ms-win-crt-heap-l1-1-0.dll -> C:\WINDOWS\System32\ucrtbase.dll
  api-ms-win-crt-runtime-l1-1-0.dll -> C:\WINDOWS\System32\ucrtbase.dll
  api-ms-win-crt-stdio-l1-1-0.dll -> C:\WINDOWS\System32\ucrtbase.dll
  cuTENSOR.dll -> not found
  cublas64_12.dll -> not found
  cudart64_12.dll -> not found
  cudnn64_8.dll -> not found
  cufft64_11.dll -> not found
  curand64_10.dll -> not found
  cusolver64_11.dll -> not found
  cusparse64_12.dll -> not found
  nvcuda.dll -> not found
  nvrtc64_120_0.dll -> not found
  python311.dll -> C:\Users\weber\miniconda3\envs\spaconv311\python311.dll

Original error:
  ImportError: DLL load failed while importing runtime: The specified module could not be found.
================================================================
```

### Additional Information

_No response_


Discussion:
After getting the error on `import cupy`, could you run the following and share the output?

```py
import sys
print(sys.modules['cupy._environment']._get_preload_logs())
```
This is on a diferent computer with Win 10 that has the same issue. Here's the output:

```
>>> import sys
>>> print(sys.modules['cupy._environment']._get_preload_logs())
CUDA_PATH: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2
Wheel shared libraries: C:\Users\dwebe\.conda\envs\cupytest\Lib\site-packages\cupy\.data\lib
Adding DLL search path: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2
Adding DLL search path: C:\Users\dwebe\.conda\envs\cupytest\Lib\site-packages\cupy\.data\lib
Preloading triggered for library: cutensor
CuPy wheel package built for CUDA 12.x
CuPy CUDA library directory: C:\Users\dwebe/.cupy/cuda_lib
Looking for cutensor version 1.6.2 (cutensor.dll)
Rejected candidate (not found): C:\Users\dwebe/.cupy/cuda_lib\12.x\cutensor\1.6.2\lib\cutensor.dll
Rejected candidate (not found): C:\Users\dwebe/.cupy/cuda_lib\12.x\cutensor\1.6.2\lib64\cutensor.dll
Rejected candidate (not found): C:\Users\dwebe/.cupy/cuda_lib\12.x\cutensor\1.6.2\bin\cutensor.dll
File cutensor.dll could not be found
Trying to load cutensor.dll from default search path
Library cutensor could not be preloaded: Could not find module 'cutensor.dll' (or one of its dependencies). Try using the full path with constructor syntax.
Preloading triggered for library: nccl
Preload nccl not configured in wheel
```
> After getting the error on `import cupy`, could you run the following and share the output?
> 
> ```python
> import sys
> print(sys.modules['cupy._environment']._get_preload_logs())
> ```
@kmaehashi 
Hi, I got almost the same issue on linux: importing `nccl` got the error `AttributeError: module 'cupy.cuda.nccl' has no attribute 'get_version'`.
## Environment
```
OS: Linux aarch64
cuda version: 12.0
python version: 3.9
cupy version: cupy-cuda12x
```
## To Reproduce
```python
from cupy.cuda import nccl
nccl_version = str(nccl.get_version())
```

## Check logs
By running the scripts
```python
import sys
print(sys.modules['cupy._environment']._get_preload_logs())
```
I got outputs
```
Preloading triggered for library: cutensor
Preload cutensor not configured in wheel
Preloading triggered for library: nccl
Preload nccl not configured in wheel
```
Thanks for sharing the details, @uellue!

```
CUDA_PATH: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2
...
Adding DLL search path: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2
```

@leofang It seems we need some fix on here.
https://github.com/cupy/cupy/blob/b6e2664c835664c531354c5776a160cd3e493d16/cupy/_environment.py#L202-L211

When wheels are installed in conda env without `cudatoolkit`, CuPy fails to load CUDA Toolkit because DLL search path becomes `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2` instead of `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2\bin` (which contains DLLs).
@AlbertZhangHIT This is a different issue. Currently, CuPy wheels for aarch64 are shipped without NCCL/cuDNN/cuTENSOR support. Please file an issue along with your usecase if you need NCCL support in Arm. You can use `cupy.cuda.nccl.available` flag (which is guaranteed to be always available) to test if NCCL is available in the current environment.
> It seems we need some fix on here.
> 
> When wheels are installed in conda env without `cudatoolkit`, CuPy fails to load CUDA Toolkit because DLL search path becomes `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2` instead of `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2\bin` (which contains DLLs).

@kmaehashi Yes indeed. I think we've been overlooking the use case that a wheel is installed to a conda env (which is also what I do for my non-CuPy dev environment). This is just one of the corners that exposed the issue. I have been thinking about a way to identify how a package (self) is installed. This can be fixed by querying the installation source. Alternatively, here we can check multiple possible paths. This would not need any self-identification mechanism. WDYT?

I think existing `_wheel.json` can be used to identify how the package is installed.

https://github.com/cupy/cupy/blob/ecc51eb9d5444fb5815207f85d53860d4e37dfab/cupy/_environment.py#L287

Right, nice, I forget!

Resolving patch:
diff --git a/cupy/_environment.py b/cupy/_environment.py
--- a/cupy/_environment.py
+++ b/cupy/_environment.py
@@ -200,8 +200,10 @@ def _setup_win32_dll_directory():
     # Setup DLL directory to load CUDA Toolkit libs and shared libraries
     # added during the build process.
     if sys.platform.startswith('win32'):
-        is_conda = ((os.environ.get('CONDA_PREFIX') is not None)
-                    or (os.environ.get('CONDA_BUILD_STATE') is not None))
+        # see _can_attempt_preload()
+        config = get_preload_config()
+        is_conda = (config is not None and (config['packaging'] == 'conda'))
+
         # Path to the CUDA Toolkit binaries
         cuda_path = get_cuda_path()
         if cuda_path is not None:


Test patch: