Cuda not found in CMake when upgrading to libtorch1.11

Xavier31: <p>I have several libtorch versions installed (1.3, 1.5, 1.10) on Ubuntu 20.04 with which I have no problem building and running.<br>
However when I tried to upgrade to libtorch, cmake threw the following errror when configuring:</p>
<pre><code class="lang-auto">Found CUDA: /usr/local/cuda (found version "11.5") 
The CUDA compiler identification is unknown
CMake Error at /snap/cmake/1088/share/cmake-3.23/Modules/CMakeDetermineCUDACompiler.cmake:633 (message):
  Failed to detect a default CUDA architecture.  

  Compiler output:

Call Stack (most recent call first):
  /home/fbx/libs/libtorch1.11/share/cmake/Caffe2/public/cuda.cmake:41 (enable_language)
  /home/fbx/libs/libtorch1.11/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
  /home/fbx/libs/libtorch1.11/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  CMakeLists.txt:35 (find_package)
</code></pre>
<p>Configuring incomplete, errors occurred!<br>
I installed the latest version of cmake (3.23.2) bu it did not fix the issue.<br>
Any idea why this got broken in 1.11 ?<br>
Thanks</p>

ptrblck: <p>In case you were using different <code>cmake</code> versions, make sure to <code>clean</code> the build as the build files might be dirty and interact in strange ways.<br>
If this also doesn’t help, check the cmake error log and see if e.g. an incompatible C++ compiler was found for the currently used CUDA version.</p>

Xavier31: <p>I usually remove the build directory and clean the CMake cache, yes.</p>
<p>for a different problem, I had to revert to CMake 3.16 and got a different error</p>
<pre><code class="lang-auto">Found CUDA: /usr/local/cuda (found version "11.5")
The CUDA compiler identification is NVIDIA 11.5.50
Check for working CUDA compiler: /usr/local/cuda/bin/nvcc
Check for working CUDA compiler: /usr/local/cuda/bin/nvcc -- works
Detecting CUDA compiler ABI info
Detecting CUDA compiler ABI info - done
Caffe2: CUDA detected: 11.5
Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
Caffe2: CUDA toolkit directory: /usr/local/cuda
Caffe2: Header version is: 11.5
Found CUDNN: /usr/lib/x86_64-linux-gnu/libcudnn.so
Found cuDNN: v8.4.0 (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so)
/usr/local/cuda/lib64/libnvrtc.so shorthash is 6dcccaf3
CMake Error in cpp_export/build/CMakeFiles/CMakeTmp/CMakeLists.txt:
Target "cmTC_b3f90" requires the language dialect "CUDA17" (with compiler
extensions), but CMake does not know the compile flags to use to enable it.
</code></pre>
<p>the cmake error log shows an error in pthread, which is weird (?)</p>
<pre><code class="lang-auto">Performing C SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output:
Change Dir: cpp_export/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/ninja cmTC_54de7 &amp;&amp; [1/2] Building C object CMakeFiles/cmTC_54de7.dir/src.c.o
[2/2] Linking C executable cmTC_54de7
FAILED: cmTC_54de7 
: &amp;&amp; /usr/bin/cc -DCMAKE_HAVE_LIBC_PTHREAD   CMakeFiles/cmTC_54de7.dir/src.c.o  -o cmTC_54de7   &amp;&amp; :
/usr/bin/ld : CMakeFiles/cmTC_54de7.dir/src.c.o : dans la fonction « main » :
src.c:(.text+0x46) : référence indéfinie vers « pthread_create »
/usr/bin/ld : src.c:(.text+0x52) : référence indéfinie vers « pthread_detach »
/usr/bin/ld : src.c:(.text+0x63) : référence indéfinie vers « pthread_join »
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
</code></pre>

dfalbel: <aside class="quote no-group" data-username="Xavier31" data-post="1" data-topic="153972">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/xavier31/48/11725_2.png" class="avatar"> Xavier31:</div>
<blockquote>
<p><code>Failed to detect a default CUDA architecture. </code></p>
</blockquote>
</aside>
<p>I had to use something like this at some point:</p>
<pre><code class="lang-auto">set_property(TARGET yourtarget PROPERTY CUDA_STANDARD 14)
</code></pre>
<p>as it seems that by default CMake will use CUDA_STANDARD = C++standard but that might not be supported depending on your CUDA Toolkit version.</p>

Xavier31: <p>I tried</p>
<pre><code class="lang-auto">set_property(TARGET ${PROJ_NAME} PROPERTY CUDA_STANDARD 14)
</code></pre>
<p>with different values for c++standard (14,17,20)<br>
but did not work</p>
<pre><code class="lang-auto">CMake Error at /usr/share/cmake-3.16/Modules/CMakeDetermineCUDACompiler.cmake:25 (message):
  Could not find compiler set in environment variable CUDACXX:
  CMAKE_CUDA_COMPILER-NOTFOUND.
CMake Error: CMAKE_CUDA_COMPILER not set, after EnableLanguage
</code></pre>

dfalbel: <p>I think the old CMake scripts from LibTorch were smarter at finding <code>nvcc</code>. You can set <code>CUDACXX</code> env var to the nvcc location to fix, or make sure <code>nvcc</code> is in the PATH.</p>
<p>FWIW I think this is mostly related to this PR:</p>
<aside class="onebox githubpullrequest" data-onebox-src="https://github.com/pytorch/pytorch/pull/62445">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/pull/62445" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Pull Request">
    <svg width="60" height="60" class="github-icon" viewbox="0 0 12 16" aria-hidden="true"><path d="M11 11.28V5c-.03-.78-.34-1.47-.94-2.06C9.46 2.35 8.78 2.03 8 2H7V0L4 3l3 3V4h1c.27.02.48.11.69.31.21.2.3.42.31.69v6.28A1.993 1.993 0 0 0 10 15a1.993 1.993 0 0 0 1-3.72zm-1 2.92c-.66 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2zM4 3c0-1.11-.89-2-2-2a1.993 1.993 0 0 0-1 3.72v6.56A1.993 1.993 0 0 0 2 15a1.993 1.993 0 0 0 1-3.72V4.72c.59-.34 1-.98 1-1.72zm-.8 10c0 .66-.55 1.2-1.2 1.2-.65 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2zM2 4.2C1.34 4.2.8 3.65.8 3c0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/pull/62445" target="_blank" rel="noopener nofollow ugc">Update CMake and use native CUDA language support</a>
    </h4>

    <div class="branches">
      <code>pytorch:gh/peterbell10/108/base</code> ← <code>pytorch:gh/peterbell10/108/head</code>
    </div>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2021-07-29" data-time="23:55:43" data-timezone="UTC">11:55PM - 29 Jul 21 UTC</span>
      </div>

      <div class="user">
        <a href="https://github.com/peterbell10" target="_blank" rel="noopener nofollow ugc">
          <img alt="peterbell10" src="https://avatars.githubusercontent.com/u/13238737?v=4" class="onebox-avatar-inline" width="20" height="20">
          peterbell10
        </a>
      </div>

      <div class="lines" title="63 commits changed 23 files with 264 additions and 161 deletions">
        <a href="https://github.com/pytorch/pytorch/pull/62445/files" target="_blank" rel="noopener nofollow ugc">
          <span class="added">+264</span>
          <span class="removed">-161</span>
        </a>
      </div>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">Stack from [ghstack](https://github.com/ezyang/ghstack):
* #62550
* __-&gt;__ #6244<span class="show-more-container"><a href="https://github.com/pytorch/pytorch/pull/62445" target="_blank" rel="noopener nofollow ugc" class="show-more">…</a></span><span class="excerpt hidden">5

PyTorch currently uses the old style of compiling CUDA in CMake which is just a
bunch of scripts in `FindCUDA.cmake`. Newer versions support CUDA natively as
a language just like C++ or C.

Differential Revision: [](https://our.internmc.facebook.com/intern/diff/)

cc @malfet @seemethere

Differential Revision: [D31503350](https://our.internmc.facebook.com/intern/diff/D31503350)</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


Xavier31: <pre><code class="lang-auto">set(CMAKE_CUDA_COMPILER /usr/local/cuda-11.5/bin/nvcc)
</code></pre>
<p>did the trick. Thanks !</p>