USE_CUDNN always set to 0

username: Diazepam
cooked: <p>Dear Developers:</p>
<p>I’m new to libtorch and I met a problem in setting the cudnn. I’m not sure what I did is wrong… <img src="https://discuss.pytorch.org/images/emoji/apple/face_with_thermometer.png?v=12" title=":face_with_thermometer:" class="emoji" alt=":face_with_thermometer:" loading="lazy" width="20" height="20"></p>
<p>Environment:</p>
<ol>
<li>
<p>NVCC (/usr/local/cuda-12.1/bin/nvcc)<br>
Cuda compilation tools, release 12.1, V12.1.66<br>
Build cuda_12.1.r12.1/compiler.32415258_0</p>
</li>
<li>
<p>CUDNN (/usr/include/cudnn.h)<br>
CUDNN_MAJOR 9</p>
</li>
<li>
<p>cmake<br>
cmake version 3.28.1 (CMake; JetBrains IDE bundle; build 16)</p>
</li>
<li>
<p>CLion<br>
2024.1.1</p>
</li>
<li>
<p>Ubuntu<br>
22.04</p>
</li>
</ol>
<p>I did these things below:</p>
<ol>
<li>
<p>Download <a href="https://download.pytorch.org/libtorch/cu121/libtorch-cxx11-abi-shared-with-deps-2.3.0%2Bcu121.zip" rel="noopener nofollow ugc">libtorch</a>, and unzip.</p>
</li>
<li>
<p>Write some files:</p>
</li>
</ol>
<ul>
<li>CMakeLists.txt</li>
</ul>
<pre><code class="lang-auto">cmake_minimum_required(VERSION 3.18)
project(Test)

set(CMAKE_CUDA_ARCHITECTURES "native")
set(PYTHON_EXECUTABLE "/usr/bin/python3")

find_package(Torch REQUIRED)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")

add_executable(Test main.cpp)
target_link_libraries(Test "${TORCH_LIBRARIES}")
set_property(TARGET Test PROPERTY CXX_STANDARD 17)

# The following code block is suggested to be used on Windows.
# According to https://github.com/pytorch/pytorch/issues/25457,
# the DLLs need to be copied to avoid memory errors.
if (MSVC)
    file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
    add_custom_command(TARGET Test
            POST_BUILD
            COMMAND ${CMAKE_COMMAND} -E copy_if_different
            ${TORCH_DLLS}
            $&lt;TARGET_FILE_DIR:example-app&gt;)
endif (MSVC)
</code></pre>
<ul>
<li>main.cpp</li>
</ul>
<pre><code class="lang-auto">#include &lt;torch/torch.h&gt;
#include &lt;iostream&gt;

int main() {
    torch::Tensor tensor = torch::rand({2, 3});
    std::cout &lt;&lt; tensor &lt;&lt; std::endl;
    std::cout &lt;&lt; torch::cuda::is_available();
}

</code></pre>
<ol start="3">
<li>Set environment variables in CLion</li>
</ol>
<pre><code class="lang-auto">CUDACXX=/usr/local/cuda-12.1/bin/nvcc;USE_CUDNN=1;CAFFE2_USE_CUDNN=1
</code></pre>
<ol start="4">
<li>Set CMake Options</li>
</ol>
<pre><code class="lang-auto">-DCMAKE_PREFIX_PATH=/path/to/libtorch
</code></pre>
<p>Results from CLion of cmake:</p>
<pre><code class="lang-auto">/snap/clion/275/bin/cmake/linux/x64/bin/cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=/path/to/libtorch -S /home/i/Documents/Test -B /home/i/Documents/Test/cmake-build-debug
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found CUDA: /usr/local/cuda (found version "12.1") 
-- The CUDA compiler identification is NVIDIA 12.1.66
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda-12.1/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Found CUDAToolkit: /usr/local/cuda-12.1/include (found version "12.1.66") 
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Caffe2: CUDA detected: 12.1
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 12.1
-- /usr/local/cuda/lib64/libnvrtc.so shorthash is d540eb83
-- USE_CUDNN is set to 0. Compiling without cuDNN support
-- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support
-- Autodetected CUDA architecture(s):  7.5
-- Added CUDA NVCC flags for: -gencode;arch=compute_75,code=sm_75
-- Found Torch: /path/to/libtorch/lib/libtorch.so  
-- Configuring done (8.3s)
-- Generating done (0.0s)
-- Build files have been written to: /home/i/Documents/Test/cmake-build-debug
</code></pre>
<p>The <code>USE_CUDNN</code> still 0 even though I set USE_CUDNN=1;CAFFE2_USE_CUDNN=1. <img src="https://discuss.pytorch.org/images/emoji/apple/exploding_head.png?v=12" title=":exploding_head:" class="emoji" alt=":exploding_head:" loading="lazy" width="20" height="20"></p>
<p>I have read <code>https://discuss.pytorch.org/t/use-cudnn-set-to-zero-after-upgrade-to-libtorch-2-0/175413/4</code> and <code>https://github.com/pytorch/pytorch/blob/main/cmake/public/cuda.cmake#L268</code> . I also tried <a href="https://download.pytorch.org/libtorch/cu121/libtorch-cxx11-abi-shared-with-deps-2.3.0%2Bcu121.zip" rel="noopener nofollow ugc">libtorch 2.2.0</a>. I tried to set environment variables in terminal. But it still not work.</p>
<p>Thank you for your patience to read my topic, and I would so appreciate if you could help me to solve this problem. Thanks! <img src="https://discuss.pytorch.org/images/emoji/apple/smiling_face_with_three_hearts.png?v=12" title=":smiling_face_with_three_hearts:" class="emoji" alt=":smiling_face_with_three_hearts:" loading="lazy" width="20" height="20"></p>

username: ptrblck
cooked: <p>Could you check where your cuDNN libraries are located on your system?</p>

username: Diazepam
cooked: <p>Yes. cuDNN 9 is located in /usr/include/cudnn.h</p>

username: ptrblck
cooked: <p>That’s the header file not the library. Look for the <code>.so</code> files.</p>

username: Diazepam
cooked: <p>Thanks for your patience!<br>
I found <code>libcudnn.so</code> in <code>/usr/lib/x86_64-linux-gnu/libcudnn.so</code>.<br>
By the way, I install cudnn by deb package.</p>

username: Diazepam
cooked: <p>By adding “message(STATUS “CAFFE2_USE_CUDNN is: ${CAFFE2_USE_CUDNN}”)” in <code>libtorch/share/cmake/Caffe2/public/cuda.cmake</code>, I found CAFFE2_USE_CUDNN is None, even though I set CAFFE2_USE_CUDNN=1 in environment variables.</p>
<p>Finally, I add <code>-DCAFFE2_USE_CUDNN=True</code> in cmake option and make everything works fine.</p>

username: CityHunter71
cooked: <p>This post was also very useful to compile TensorRT-LLM correctly, in fact even after installing all the necessary packages it continued to say:<br>
– USE_CUDNN is set to 0. Compiling without cuDNN support<br>
setting in cmake -DCAFFE2_USE_CUDNN=ON</p>
<p>finally it works:</p>
<p>– Found CUDNN: /usr/lib/x86_64-linux-gnu/libcudnn.so</p>
<p>Thanks a lot for the precious support!!</p>

username: Diazepam
cooked: <p>Wow, thanks for your reply. I’m so happy to see this post helps others. <img src="https://discuss.pytorch.org/images/emoji/apple/smiling_face_with_three_hearts.png?v=14" title=":smiling_face_with_three_hearts:" class="emoji" alt=":smiling_face_with_three_hearts:" loading="lazy" width="20" height="20"></p>