Could NOT find CUDA (missing: CUDA_CUDART_LIBRARY) (found version "12.5")

Geremia: <p>With CMake version 3.30.2 and CUDA 12.5.1, I get the following error when trying to build PyTorch 2.4.0:</p>
<pre><code class="lang-auto">-- Could NOT find CUDA (missing: CUDA_CUDART_LIBRARY) (found version "12.5")
CMake Warning at cmake/public/cuda.cmake:31 (message):
  Caffe2: CUDA cannot be found.  Depending on whether you are building Caffe2
  or a Caffe2 dependent library, the next warning / error will give you more
  info.
Call Stack (most recent call first):
  cmake/Dependencies.cmake:43 (include)
  CMakeLists.txt:853 (include)


CMake Warning at cmake/Dependencies.cmake:74 (message):
  Not compiling with CUDA.  Suppress this warning with -DUSE_CUDA=OFF.
</code></pre>
<p>Perhaps this is a CMake issue? I didn’t have the problem with CMake 3.29.4, and I’ve since upgraded to 3.30.2.</p>

ptrblck: <aside class="quote no-group" data-username="Geremia" data-post="1" data-topic="207960">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/geremia/48/70165_2.png" class="avatar"> Geremia:</div>
<blockquote>
<p>Perhaps this is a CMake issue? I didn’t have the problem with CMake 3.29.4, and I’ve since upgraded to 3.30.2.</p>
</blockquote>
</aside>
<p>Might be as <code>cmake==3.30.2</code> was just released a week ago. Am I understanding your claim correctly that <code>cmake==3.29.4</code> works for you?</p>

Geremia: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="207960">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Am I understanding your claim correctly that <code>cmake==3.29.4</code> works for you?</p>
</blockquote>
</aside>
<p>I downgraded CMake to 3.29.4, but I’m still getting the same issue:<br>
<code>disabling CUDA because USE_CUDA is set false</code><br>
Yet I have <code>USE_CUDA</code> explicitly set: <code>export USE_CUDA=1</code> and <code>-DUSE_CUDA=ON</code>.</p>

Geremia: <p>It does say: <code>Found CUDA with FP16 support, compiling with torch.cuda.HalfTensor</code><br>
But then it says <code>disabling CUDA because NOT USE_CUDA is set</code>.</p>

ptrblck: <p>I haven’t seen this issue myself and can find a few references pointing to a wrong <code>docker run</code> command not allowing GPU access. I don’t know if this is related to your issue or not.<br>
You could check if you are able to source build any other CUDA application (e.g. the CUDA samples) first.</p>

Geremia: <aside class="quote no-group" data-username="ptrblck" data-post="5" data-topic="207960">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>You could check if you are able to source build any other CUDA application (e.g. the CUDA samples) first.</p>
</blockquote>
</aside>
<p>Yes. I can CUDA example (which doesn’t use CMake). It seems to be a CMake issue.</p>

janeyx99: <p>Hi, I mentioned this on the issue and this is a noob guess, but could try recompiling after cleaning the build? e.g., add the line right <code>python3 setup.py clean</code> before <code>python3 setup.py install</code>?</p>

Geremia: <aside class="quote no-group" data-username="janeyx99" data-post="7" data-topic="207960">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/janeyx99/48/45020_2.png" class="avatar"> janeyx99:</div>
<blockquote>
<p>add the line right <code>python3 setup.py clean</code> before <code>python3 setup.py install</code></p>
</blockquote>
</aside>
<p>Where exactly? This is how <a href="https://slackbuilds.org/slackbuilds/15.0/libraries/pytorch/pytorch.SlackBuild" rel="noopener nofollow ugc">the SlackBuild script</a> does it:</p>
<pre data-code-wrap="bash"><code class="lang-bash">cd build
  unshare -n cmake \
    -G Ninja \
    -DCMAKE_C_FLAGS:STRING="$SLKCFLAGS" \
    -DCMAKE_CXX_FLAGS:STRING="$SLKCFLAGS" \
    -DCMAKE_CXX_STANDARD=17 \
    -DCMAKE_INSTALL_PREFIX="/usr" \
    -DLIBSHM_INSTALL_LIB_SUBDIR="lib$LIBDIRSUFFIX" \
    -DTORCH_INSTALL_LIB_DIR="lib$LIBDIRSUFFIX" \
    -DPYTHON_EXECUTABLE=$(which python3) \
    -DBUILD_CUSTOM_PROTOBUF=OFF \
    -DBUILD_TEST=OFF \
    -DUSE_FFMPEG=ON \
    -DUSE_GOLD_LINKER=ON \
    -DUSE_OPENCL=ON \
    -DUSE_OPENCV=ON \
    -DUSE_VULKAN=ON \
    -DCMAKE_BUILD_TYPE=Release ..
  "${NINJA:=ninja}"
  DESTDIR=tmpxxx $NINJA install/strip

  mkdir -p $PKG/usr/{share,lib$LIBDIRSUFFIX}
  mv tmpxxx/usr/bin $PKG/usr
  mv tmpxxx/usr/include $PKG/usr
  mv tmpxxx/usr/share/cmake $PKG/usr/share
  mv tmpxxx/usr/lib$LIBDIRSUFFIX/*.so $PKG/usr/lib$LIBDIRSUFFIX
cd ..
python3 setup.py install --root=$PKG
</code></pre>
<p>Adding <code>python3 setup.py clean</code> right before the last line above didn’t change anything. I still get:</p>
<p><code>Found CUDA with FP16 support, compiling with torch.cuda.HalfTensor</code><br>
and<br>
<code>disabling CUDA because NOT USE_CUDA is set</code></p>

geduo15: <p>Hi, did you solve the issue? I also met the same problem.</p>

Geremia: <aside class="quote no-group" data-username="geduo15" data-post="9" data-topic="207960" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/geduo15/48/72163_2.png" class="avatar"> geduo15:</div>
<blockquote>
<p>Hi, did you solve the issue?</p>
</blockquote>
</aside>
<p>Not yet.</p>
<p>I should also mention that I fixed the</p>
<pre><code class="lang-auto">Building wheel torch-2.4.0a0+gitUnknown
-- Building version 2.4.0a0+gitUnknown
Could not find any of CMakeLists.txt, Makefile, setup.py, LICENSE, LICENSE.md, LICENSE.txt in /tmp/SBo/pytorch-v2.4.0/third_party/QNNPACK
Did you run 'git submodule update --init --recursive'?
</code></pre>
<p>by running <code>sed -i '/"QNNPACK"/d' setup.py</code>. I’m wondering if that triggered the CUDA issue I’m having?</p>

Geremia: <aside class="quote no-group" data-username="Geremia" data-post="8" data-topic="207960">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/geremia/48/70165_2.png" class="avatar"> Geremia:</div>
<blockquote>
<p><code>Found CUDA with FP16 support, compiling with torch.cuda.HalfTensor</code><br>
and<br>
<code>disabling CUDA because NOT USE_CUDA is set</code></p>
</blockquote>
</aside>
<p>I’m still getting this issue when building Pytorch 2.5.1.</p>

Geremia: <p>I updated to CUDA toolkit 12.6.2 and NVIDIA 560.35.03, and now it works.</p>

geduo15: <p>Thanks a lot. I’ll check it.</p>

drewjin: <p>Chat!</p>
<p>Although not the same project, I met the similar issue in <code>vllm</code> which looks like:</p>
<pre data-code-wrap="bash"><code class="lang-bash">running egg_info
writing vllm.egg-info/PKG-INFO
writing dependency_links to vllm.egg-info/dependency_links.txt
writing entry points to vllm.egg-info/entry_points.txt
writing requirements to vllm.egg-info/requires.txt
writing top-level names to vllm.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'vllm.egg-info/SOURCES.txt'
running build_ext
-- Build type: RelWithDebInfo
-- Target device: cuda
-- Found python matching: /data1/jyj/micromamba/envs/vllm/bin/python.
-- Could NOT find CUDA (missing: CUDA_INCLUDE_DIRS) (found version "12.4")
CMake Warning at /data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:31 (message):
  PyTorch: CUDA cannot be found.  Depending on whether you are building
  PyTorch or a PyTorch dependent library, the next warning / error will give
  you more info.
Call Stack (most recent call first):
  /data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
  /data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  CMakeLists.txt:85 (find_package)


CMake Error at /data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:90 (message):
  Your installed Caffe2 version uses CUDA but I cannot find the CUDA
  libraries.  Please set the proper CUDA prefixes and / or install CUDA.
Call Stack (most recent call first):
  /data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  CMakeLists.txt:85 (find_package)


-- Configuring incomplete, errors occurred!
Traceback (most recent call last):
  File "/data1/jyj/DengSeek/others/vllm/setup.py", line 676, in &lt;module&gt;
    setup(
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/__init__.py", line 117, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 186, in setup
    return run_commands(dist)
           ^^^^^^^^^^^^^^^^^^
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
    dist.run_commands()
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 983, in run_commands
    self.run_command(cmd)
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/dist.py", line 999, in run_command
    super().run_command(command)
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
    cmd_obj.run()
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/command/develop.py", line 35, in run
    self.install_for_development()
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/command/develop.py", line 112, in install_for_development
    self.run_command('build_ext')
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 339, in run_command
    self.distribution.run_command(command)
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/dist.py", line 999, in run_command
    super().run_command(command)
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
    cmd_obj.run()
  File "/data1/jyj/DengSeek/others/vllm/setup.py", line 267, in run
    super().run()
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 99, in run
    _build_ext.run(self)
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 365, in run
    self.build_extensions()
  File "/data1/jyj/DengSeek/others/vllm/setup.py", line 226, in build_extensions
    self.configure(ext)
  File "/data1/jyj/DengSeek/others/vllm/setup.py", line 204, in configure
    subprocess.check_call(
  File "/data1/jyj/micromamba/envs/vllm/lib/python3.12/subprocess.py", line 415, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/data1/jyj/DengSeek/others/vllm', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DVLLM_TARGET_DEVICE=cuda', '-DVLLM_PYTHON_EXECUTABLE=/data1/jyj/micromamba/envs/vllm/bin/python', '-DVLLM_PYTHON_PATH=/data1/jyj/DengSeek/others/vllm:/data1/jyj/micromamba/envs/vllm/lib/python312.zip:/data1/jyj/micromamba/envs/vllm/lib/python3.12:/data1/jyj/micromamba/envs/vllm/lib/python3.12/lib-dynload:/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages:/data1/jyj/micromamba/envs/vllm/lib/python3.12/site-packages/setuptools/_vendor', '-DFETCHCONTENT_BASE_DIR=/data1/jyj/DengSeek/others/vllm/.deps', '-DNVCC_THREADS=1']' returned non-zero exit status 1.
</code></pre>
<p>I have no sudo right in my cluster, thus, I downloaded my CUDA in the micromamba venv.</p>
<p>It turns out to be ok to find <code>nvcc</code>, however, CUDA can’t be linked.</p>
<p>For instance, you can’t just set <code>your_mamba_venv/</code> or <code>your_conda_venv/</code> as the <code>CUDA_HOME</code>, because in this folder, the <code>lib/</code> and <code>include/</code> does not have and cuda -related files in it, only <code>bin</code> has <code>nvcc</code>, which makes it weird, but it is true.</p>
<p>This is because it has been actually downloaded in <code>/your_env/targets/x86_64-linux</code>, and the <code>bin</code> inside this folder also contains <code>nvcc</code>, <code>include</code> and <code>lib</code> in this folder truly make it to be <code>CUDA_HOME</code>.</p>
<p>In the venv, however, you can’t also compile, you have to change the CMakeLists.txt:</p>
<pre data-code-wrap="CMakeLists"><code class="lang-CMakeLists">set(CUDA_TOOLKIT_ROOT_DIR "your_venv/targets/x86_64-linux")
set(CUDA_INCLUDE_DIRS "${CUDA_TOOLKIT_ROOT_DIR}/include")
</code></pre>

opposj: <p>When compiling v2.8.0 or others? with a conda-fetched CUDA, simply inserting the <code>set</code> command causes cmake cache to be missing in the post-installation phase. The <code>CACHE</code> should be added, for example:</p>
<pre><code class="lang-auto">sed -i '1i set(CUDA_INCLUDE_DIRS "${CUDA_TOOLKIT_ROOT_DIR}/include" CACHE STRING "The include directory of CUDA.")' CMakeLists.txt
sed -i '1i set(CUDA_TOOLKIT_ROOT_DIR "your_venv/targets/x86_64-linux" CACHE STRING "The toolkit root directory of CUDA.")' CMakeLists.txt
</code></pre>