Failed to find nvToolsExt

maxim1: <p>Hello,</p>
<p>libtorch 2.0.1 with CUDA 11.8 and cudNN 8.5 works perfectly fine for me in Visual Studio Community 2022.</p>
<p>However, i now wanted to upgrade to libtorch Preview (nightly) with CUDA 12.1 and cudNN 8.9, but now i get an error message when trying to build with cmake:</p>
<p>[CMake] …<br>
[CMake] – Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/include (found version “12.1.105”)<br>
[CMake] CMake Error at C:/…/libtorch/share/cmake/Caffe2/public/cuda.cmake:74 (message):<br>
[CMake] Failed to find nvToolsExt.</p>
<p>I saw in NVTX github that the most recent NVTX version (nvtx3) is using a header only file instead of a library.<br>
Could that be the reason for the error i get?<br>
Is this a known bug or am i doing something wrong?</p>
<p>Best Regards,<br>
Max</p>

LogicPy: <aside class="quote no-group" data-username="maxim1" data-post="1" data-topic="179635">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/85e7bf/48.png" class="avatar"> maxim1:</div>
<blockquote>
<p>libtorch 2.0.1 with CUDA 11.8 and cudNN 8.5 works perfectly fine for me in Visual Studio Community 2022.</p>
<p>However, i now wanted to upgrade to libtorch Preview (nightly) with CUDA 12.1 and cudNN 8.9, but now i get an error message when trying to build with cmake:</p>
<p>[CMake] …<br>
[CMake] – Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/include (found version “12.1.105”)<br>
[CMake] CMake Error at C:/…/libtorch/share/cmake/Caffe2/public/cuda.cmake:74 (message):<br>
[CMake] Failed to find nvToolsExt.</p>
<p>I saw in NVTX github that the most recent NVTX version (nvtx3) is using a header only file instead of a library.<br>
Could that be the reason for the error i get?<br>
Is this a known bug or am i doing something wrong?</p>
<p>Best Regards,<br>
Max</p>
</blockquote>
</aside>
<p>Hello Max,</p>
<p>The error message you are encountering indicates that CMake is unable to find the <code>nvToolsExt</code> library, which is typically provided by the NVIDIA CUDA Toolkit. This library is used for GPU profiling and debugging.</p>
<p>The issue could be related to the version mismatch between your CUDA installation and the libtorch Preview version you are trying to use. It’s possible that the libtorch Preview version you have is expecting a different version of <code>nvToolsExt</code> than the one provided by CUDA 12.1.</p>
<p>To address this issue, you can try the following steps:</p>
<ol>
<li>
<p>Confirm that the <code>nvToolsExt</code> library is indeed present in your CUDA 12.1 installation. Check the <code>C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/lib64</code> directory for the library file (<code>nvToolsExt64_1.lib</code> on Windows).</p>
</li>
<li>
<p>Ensure that the CUDA 12.1 installation directory is correctly set in your system’s environment variables (<code>PATH</code> variable) or in the CMake configuration. This ensures that CMake can locate the necessary CUDA files.</p>
</li>
<li>
<p>If the library is present and the environment variables are correctly set, you can try manually adding the path to the <code>nvToolsExt</code> library in your CMakeLists.txt file. Add the following line before the <code>find_package(Torch REQUIRED)</code> line:</p>
<pre data-code-wrap="cmake"><code class="lang-cmake">link_directories("C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/lib64")
</code></pre>
<p>Replace the path with the correct location of the <code>lib64</code> directory on your system.</p>
</li>
<li>
<p>If the above steps do not resolve the issue, it’s possible that there is an incompatibility between the libtorch Preview version and CUDA 12.1. In that case, you might need to wait for a newer libtorch Preview version that is compatible with CUDA 12.1 or consider downgrading your CUDA installation to a version that is compatible with the libtorch Preview version you want to use.</p>
</li>
</ol>
<p>I hope this helps! Let me know if you have any further questions.</p>

sander.vanl: <p>Thank you for the extensive answer. I am running into this problem too, but I cannot find a <code>nvToolsExt</code> lib anywhere under <code>C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1</code>. I ran the NVIDIA CUDA installer as specified <a href="https://developer.nvidia.com/cuda-downloads" rel="noopener nofollow ugc">on the download page</a> and installed all possible packages.</p>
<p>How can I best install the <code>nvToolsExt</code> lib? It does not seem to come with the default installer</p>

Te93: <p>I had a similar problem and what I did was install NVTX with CUDA 11.8 installer. With CUDA 12 installed, run CUDA 11.8 installer and select <code>custom installation</code>. Then select <code>Nsight NVTX</code> alone and leave everything else unchecked.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/5/e534b177747e78d4aa786936d04876f9c621b33d.png" alt="image" data-base62-sha1="wHEeB5AiPlUUW4NSmX94HOJ78oR" width="594" height="444"></p>

BasisPoints: <p><a class="mention" href="/u/te93">@Te93</a> This worked for me!  Now have PyTorch working on CUDA 12.2</p>
<p>Thanks <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

rsegecin: <p>I’ve just sign into this forum to say thank you. I was about to create an issue on pytorch’s github saying that link libraries paths to cuda didn’t match the installation of Nvidia’s new 12.3 toolkit, but your direction to install NVTX from an older version of the toolkit saved the day. I think Nvidia should include NVTX as well on newer versions.</p>
<p>Thank you.</p>

Katrin_Schmid: <p>Odd, the headers are still in 12.1<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/7/9/79a745a3b4f23088d88f1523cfe12e426a24ac5c.png" alt="image" data-base62-sha1="hmc9szrQ56v2H6aLgO51eYauMTq" width="648" height="297"></p>
<p>For anyone else wondering its “C:\Program Files\NVIDIA Corporation\NvToolsExt\lib\x64\nvToolsExt64_1.lib”<br>
No idea if that is recent</p>

QFuser: <p>Its not like this since the last time I looked at this since cuda 12 NVIDA dropped support of NVTX in favor of NVTX3 which can be found under …/CUDA/%version%/NvtoolsExt which isn’t NVTX but NVTX3. Pytorch tried to update/upgrade to NVTX3 from what I see in the github logs but it failed to go through all test and so they reverted the changes and nobody fixed it. What you have are maybe remnant of an old or other installation.</p>
<p>A bit a rant Pytroch is at the moment far from stable only the bugs which are with the big frameworks and libraries are getting fixed and there is so much stuff to get fixed it starts to be a drag and there is more coming python will come with real mulithreading a lot of stuff will break because Pytroch isn’t build around it and has many static functions e.g like seed() . It maybe take a bit longer but MS want to drop the console for powershell = old help scripts won’t work anymore also in not so far future the next VS version is coming.</p>

AidanShipperley: <h2><a name="update-1" class="anchor" href="#update-1"></a>UPDATE:</h2>
<p>For anybody still encountering this issue, <strong>there is still no official solution</strong> quite yet as of 6/5/2024 except to just use CUDA 11.8 on Windows, or to install CUDA 11.8 (making sure that you specifically check the box to install <code>Nsight NVTX</code>) and copy the <code>nvToolsExt64_1.lib</code> from 11.8 to 21.x’s <code>\lib\x64</code> directory.</p>
<p>There is already a <a href="https://github.com/pytorch/pytorch/issues/116242" rel="noopener nofollow ugc">GitHub issue</a> for this, and the PyTorch developers already have a <a href="https://github.com/pytorch/pytorch/pull/116926" rel="noopener nofollow ugc">pull request addressing this bug</a> and <a href="https://github.com/pytorch/pytorch/pull/109843" rel="noopener nofollow ugc">another pull request specifically addressing the NVTX changes</a>. Currently they are <a href="https://github.com/pytorch/pytorch/pull/109843#issuecomment-2143970242" rel="noopener nofollow ugc">working to merge their fix for this</a> (looks like they are just reviewing changes before merging) which will stop Cmake from attempting to look for the library in CUDA 12.x+.</p>
<p>The reason this is happening is because Nvidia stopped including NVTX as a compiled library in CUDA 12.x+ with NVTX version 3, which is great if you’re a developer as you only have to include it’s header files now. <a href="https://nvidia.github.io/NVTX/#how-do-i-use-nvtx-in-my-code" rel="noopener nofollow ugc">Nvidia explains this here</a>. Unfortunately, the LibTorch developers did not have time to account for this for cmake, which is still looking for the compiled library. That’s why you have to steal the <code>.lib</code> file from 11.8 in order to build on 12.x+.</p>
<p>I’ve been stuck on this for a few weeks, so hopefully this helps somebody out.</p>

Fitanium: <p>Maybe you need to modify some cmake files shipped with libtorch, please read this post <a href="https://zhuanlan.zhihu.com/p/695586484" rel="noopener nofollow ugc">修复Windows下LibTorch+CUDA12.4执行CMake报错：Failed to find nvToolsExt</a> as reference. (Maybe you need to use a translator if you are not familar with Chinese)</p>

rbaleksandar: <aside class="onebox githubpullrequest" data-onebox-src="https://github.com/pytorch/pytorch/pull/116926">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/pull/116926" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">



    <div class="github-icon-container" title="Pull Request">
      <svg width="60" height="60" class="github-icon" viewBox="0 0 12 16" aria-hidden="true"><path fill-rule="evenodd" d="M11 11.28V5c-.03-.78-.34-1.47-.94-2.06C9.46 2.35 8.78 2.03 8 2H7V0L4 3l3 3V4h1c.27.02.48.11.69.31.21.2.3.42.31.69v6.28A1.993 1.993 0 0 0 10 15a1.993 1.993 0 0 0 1-3.72zm-1 2.92c-.66 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2zM4 3c0-1.11-.89-2-2-2a1.993 1.993 0 0 0-1 3.72v6.56A1.993 1.993 0 0 0 2 15a1.993 1.993 0 0 0 1-3.72V4.72c.59-.34 1-.98 1-1.72zm-.8 10c0 .66-.55 1.2-1.2 1.2-.65 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2zM2 4.2C1.34 4.2.8 3.65.8 3c0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2z"></path></svg>
    </div>

  <div class="github-info-container">



      <h4>
        <a href="https://github.com/pytorch/pytorch/pull/116926" target="_blank" rel="noopener nofollow ugc">Fix MSVC build with CUDA enabled</a>
      </h4>

    <div class="branches">
      <code>pytorch:main</code> ← <code>pbelcak:nvtx-3-fix</code>
    </div>

      <div class="github-info">
        <div class="date">
          opened <span class="discourse-local-date" data-format="ll" data-date="2024-01-06" data-time="17:25:23" data-timezone="UTC">05:25PM - 06 Jan 24 UTC</span>
        </div>

        <div class="user">
          <a href="https://github.com/pbelcak" target="_blank" rel="noopener nofollow ugc">
            <img alt="pbelcak" src="https://avatars.githubusercontent.com/u/107618858?v=4" class="onebox-avatar-inline" width="20" height="20">
            pbelcak
          </a>
        </div>

        <div class="lines" title="1 commits changed 4 files with 24 additions and 11 deletions">
          <a href="https://github.com/pytorch/pytorch/pull/116926/files" target="_blank" rel="noopener nofollow ugc">
            <span class="added">+24</span>
            <span class="removed">-11</span>
          </a>
        </div>
      </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">Fixes #115922 and #101135 



cc @ptrblck @msaroufim @jgong5 @mingfeima @Xiaob<span class="show-more-container"><a href="https://github.com/pytorch/pytorch/pull/116926" target="_blank" rel="noopener nofollow ugc" class="show-more">…</a></span><span class="excerpt hidden">ingSuper @sanchitintel @ashokei @jingxu10</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>The issue is ongoing. Long story short: Nvidia did a number and changed the here mentioned library from binary (a <code>lib</code> file) to a header-only. <code>libtorch</code> still hasn’t caught up with this change leading to perpetual complains of the few who dare use <code>libtorch</code> (I myself am trying ONNX right now to get away from <code>libtorch</code>) about this issue.</p>
<p>The solution given by <a class="mention" href="/u/aidanshipperley">@AidanShipperley</a> is no longer working since the binaries are no longer shipped. I checked with 11.8 and 12.2. The header can be found under <code>&lt;CUDA_PATH&gt;\include\nvtx3</code>.</p>
<p>However, I still have an older <code>libtorch</code> <strong>2.1.0</strong> where the DLL is actually shipped and can be found (once you extract the <code>libtorch</code> ZIP archive) under <code>lib</code> subdirectory.</p>

phetdam: <p>Sadly still an issue with PyTorch 2.5.0 with CUDA 12.2 (12.4 max) on Windows 11.</p>
<p>I tried fiddling with <code>torch/share/cmake/Caffe2/public/cuda.cmake</code> since I am using Visual Studio 2022 with CMake 3.26; nvToolsExt has been <a href="https://cmake.org/cmake/help/latest/module/FindCUDAToolkit.html#nvtoolsext" rel="noopener nofollow ugc">deprecated since 3.25</a>. I did notice that the <a href="https://cmake.org/cmake/help/latest/command/find_path.html" rel="noopener nofollow ugc"><code>find_path</code></a> call specifies <code>NO_DEFAULT_PATH</code> to suppress looking through system paths. However, the path <code>"${PROJECT_SOURCE_DIR}/third_party/NVTX/c/include"</code> isn’t one I can find in the PyTorch distribution I installed via pip. In fact, <code>PROJECT_SOURCE_DIR</code> doesn’t resolve to any directory inside the <code>site-packages/torch</code> but rather still refers to <em>my</em> project’s top-level directory. That’s a bit strange but I guess the Caffe2 find module doesn’t call <code>project()</code> directly.</p>
<p>Edit: I had a few questions I wanted to ask but commit <a href="https://github.com/pytorch/pytorch/commit/3cfd2444955a81b5cd6655c085b7eba0574db8da" rel="noopener nofollow ugc"><code>3cfd244</code></a> gives me some answers. I.e.:</p>
<ol>
<li>Is there any reason <code>NO_DEFAULT_PATH</code> is being used?
<blockquote>
<p>This supports building LibTorch from source using the CMake build config to use the vendored <code>third_party/NVTX</code> submodule tracking the <a href="https://github.com/NVIDIA/NVTX/" rel="noopener nofollow ugc">NVTX</a> repo.</p>
</blockquote>
</li>
<li>Can we relax this requirement on Windows?
<blockquote>
<p>Probably yes in general but only with an explicit CMake build flag option; this option would also be <code>OFF</code> to preserve existing source build behavior. <code>3cfd244</code> adds a <code>USE_SYSTEM_NVTX</code> CMake option to the LibTorch CMake config that could ostensibly be used in <code>cuda.cmake</code> if the user specified <code>-DUSE_SYSTEM_NVTX=ON</code> in their project.</p>
</blockquote>
</li>
<li>And does <code>/path/to/third_party/NVTX/c/include</code> exist?
<blockquote>
<p>Yes, in the PyTorch source repo. This path should be hardcoded in <code>cuda.cmake</code> in the first place but unfortunately it’s there in the 2.5.0 release.</p>
</blockquote>
</li>
</ol>
<p>I suppose building LibTorch from source may be friendlier for those using CUDA 12+ but this is of course not yet in an official PyTorch pip release.</p>

phetdam: <p>Good news is that I <em>did</em> find a user workaround that does <em>not</em> require tweaking any of the CMake modules that come with a PyTorch distribution. Assuming that <code>PYTORCH_VERSION</code> and <code>PYTORCH_CUDA_VERSION</code> are available, with these two CMake variables defined as values from running the following Python script and reading the output:</p>
<pre data-code-wrap="python"><code class="lang-python">from torch import version


if __name__ == "__main__":
    print(version.__version__.split("+")[0])  # for PYTORCH_VERSION
    print(version.cuda)                       # for PYTORCH_CUDA_VERSION
</code></pre>
<p>Right before your <code>find_package(Torch CONFIG)</code> call you can do the following:</p>
<pre><code class="lang-auto"># assumes find_package(CUDAToolkit) was already done
if(
    PYTORCH_VERSION VERSION_GREATER_EQUAL 2.5.0 AND
    PYTORCH_CUDA_VERSION VERSION_GREATER_EQUAL 12
)
    message(STATUS "PyTorch NVTX headers workaround: Yes")
    # only do this if nvToolsExt is not defined and CUDA::nvtx3 exists
    if(NOT TARGET CUDA::nvToolsExt AND TARGET CUDA::nvtx3)
        add_library(CUDA::nvToolsExt INTERFACE IMPORTED)
        # ensure that PyTorch is told to use NVTX3 headers
        target_compile_definitions(
            CUDA::nvToolsExt INTERFACE
            TORCH_CUDA_USE_NVTX3
        )
        target_link_libraries(CUDA::nvToolsExt INTERFACE CUDA::nvtx3)
    endif()
else()
    message(STATUS "PyTorch NVTX headers workaround: No")
endif()
# find Torch C++ as usual
find_package(Torch CONFIG)
</code></pre>
<p>This has pacified CMake nicely for me so hopefully it will be of help to others too.</p>

Cynician: <p>You are such a genuis,now my torch with cuda version of 12.6 can run successfully!</p>

key_H: <p>you are genuis !!! you save my day!!!</p>

realiti: <p>Thank you, this worked for me!</p>