How to install specific version of torch 2.0.0

david.waterworth: <p>In the past in order to ensure I get the specific version of torch and cude I need (rather than the latest release at the time of install), I’ve used the following:</p>
<pre><code class="lang-auto">--find-links https://download.pytorch.org/whl/torch_stable.html
torch==1.13.1+cu117
</code></pre>
<p>If I upgrade this for 2.0.0, i.e.</p>
<pre><code class="lang-auto">--find-links https://download.pytorch.org/whl/torch_stable.html
torch==2.0.0+cu117
</code></pre>
<p>It works, but I noticed if follow the current installation instructions.</p>
<pre><code class="lang-auto">pip install torch
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/c/9c206163d2ab951f1e6b67fdfc015048e1191ea2.png" data-download-href="https://discuss.pytorch.org/uploads/default/9c206163d2ab951f1e6b67fdfc015048e1191ea2" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/c/9c206163d2ab951f1e6b67fdfc015048e1191ea2_2_690x267.png" alt="image" data-base62-sha1="mh9RkntOzODK4jFVhnMidnJ1KQG" width="690" height="267" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/c/9c206163d2ab951f1e6b67fdfc015048e1191ea2_2_690x267.png, https://discuss.pytorch.org/uploads/default/original/3X/9/c/9c206163d2ab951f1e6b67fdfc015048e1191ea2.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/9/c/9c206163d2ab951f1e6b67fdfc015048e1191ea2.png 2x" data-dominant-color="F5D9D4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">824×320 22.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>It’s not installing the same packages, specifically</p>
<pre><code class="lang-auto">pip install torch==2.0.0+cu117 -f https://download.pytorch.org/whl/torch_stable.html
</code></pre>
<p>Just installs <code>https://download.pytorch.org/whl/cu117/torch-2.0.0 cu117-cp310-cp310-linux_x86_64.whl</code> (and <code>triton==2.0.0</code>)</p>
<pre><code class="lang-auto">pip install torch==2.0.0
</code></pre>
<p>On the other hand installs <code>torch-2.0.0-cp310-cp310-manylinux1_x86_64.whl</code> and a bunch of extra packages, specifically</p>
<p><code>nvidia-nvtx-cu11</code><br>
<code>nvidia-nccl-cu11</code><br>
<code>nvidia-cusparse-cu11</code><br>
<code>nvidia-curand-cu11</code><br>
<code>nvidia-cufft-cu11</code><br>
<code>nvidia-cuda-runtime-cu11</code><br>
<code>nvidia-cuda-nvrtc-cu11</code><br>
<code>nvidia-cuda-cupti-cu11</code><br>
<code>nvidia-cublas-cu11</code></p>
<p>Does this mean I no longer have to install cuda using <code>apt install cuda-11-7</code>? Do these nvidia packages take care of that?</p>

ptrblck: <aside class="quote no-group" data-username="david.waterworth" data-post="1" data-topic="177812">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/e68b1a/48.png" class="avatar"> david.waterworth:</div>
<blockquote>
<p>Does this mean I no longer have to install cuda using <code>apt install cuda-11-7</code>?</p>
</blockquote>
</aside>
<p>You never needed to install a local CUDA toolkit since the PyTorch binaries were shipping with their own CUDA runtime and other CUDA libs already.<br>
The locally installed CUDA toolkit will be used if you build PyTorch from source or a custom CUDA extension.</p>
<p>The two workflows you’ve posted differ in their dependency resolution:</p>
<ul>
<li><code>pip install torch</code> uses the “default” pip wheel hosted on PyPI and will download the CUDA dependencies also from PyPI (seen as e.g. <code>nvidia-cublas-cu11</code>).</li>
<li><code>pip install torch==2.0.0+cu117 -f https://download.pytorch.org/whl/torch_stable.html</code> installs the “large” binary from the custom PyTorch download mirror (not from PyPI) and ships with all dependencies inside the wheel (which is also why their size is large).</li>
</ul>

david.waterworth: <p>Interesting - thanks.</p>
<p>When you say never needed to, I wasn’t aware of that. But I’m fairly sure that even when you install the “large” binary, it uses the system cuda if it finds it? The reason I say this is by installing the C++ cuda runtime and adding to my <code>.bashrc</code></p>
<pre><code class="lang-auto">export LIBTORCH=/usr/local/libtorch
export LD_LIBRARY_PATH=${LIBTORCH}/lib::$LD_LIBRARY_PATH
</code></pre>
<p>Allows me to run <code>tch-rs</code> (rust C++ bindings to libtorch) but doing so broke a python virtualenv containing <code>torch==1.13.1+cu117</code> (since it seemed to be loading torch 2.0 libs from libtorch) and removing the exports fixed it again. Same thing happens if I use <code>update alternatives</code> to switch between system CUDA versions.</p>
<p>Poking around a venv I created, I see the nvidia libs are installed in</p>
<pre><code class="lang-auto">.venv/lib64/python3.10/site-packages/nvidia/*/lib/*
</code></pre>
<p>But I’m not sure they’re in my LD_LIBRARY_PATH (unless activating a venv modifies it) but I am sure system CUDA is.</p>
<p>Is there a “best practice” to ensure that the correct <code>.so</code>’s are loaded? I typically have a venv per project, at the moment I’ve managed to keep all projects using the same version of cuda (1.11.7) but it would be nice to isolate each one.</p>

ptrblck: <p>I’m not familiar with <code>tch-rs</code> and don’t know what kind of dependencies are used. However, to build a local <code>libtorch</code> application your local CUDA and C++ toolchains would be used, so it could be related.<br>
The wheels <em>should</em> force the usage of the (legacy) <code>RATH</code> to point to the wheel dependencies, but in the past system libs <em>could</em> be preloaded since the <code>RUNPATH</code> was specified.</p>
<aside class="quote no-group" data-username="david.waterworth" data-post="3" data-topic="177812">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/e68b1a/48.png" class="avatar"> david.waterworth:</div>
<blockquote>
<p>Is there a “best practice” to ensure that the correct <code>.so</code>’s are loaded?</p>
</blockquote>
</aside>
<p>As a user you shouldn’t care about your system-wise installed libs as the binaries should use their own dependencies, unless you want to build PyTorch or any extension from source.</p>

ptrblck: <p>Please let me know if you see any issues where the CUDA libs packaged in the binaries (coming from PyPI or directly shipped inside the wheel etc.) are not working as expected and you see some mismatches.<br>
Also, let me know if my assumption about <code>tch-rs</code> is correct and it indeed needs to build <code>libtorch</code> applications in which case your locally installed CUDA toolkit should be used.</p>

david.waterworth: <p>It seems to be working for <code>torch 2.0.0</code>. I’m unfamiliar with RPATH, but I removed the <code>cuda</code> exports from my <code>.bashrc</code>, and <code>torch</code> still worked fine. I might dig into it a bit further to see exactly what libraries are being loaded.</p>
<p>When I get time, I’ll try torch 1.13.1 and see if that works.</p>
<pre><code class="lang-auto"># for tch-rs
#export LIBTORCH=/usr/local/libtorch
#export LD_LIBRARY_PATH=${LIBTORCH}/lib::$LD_LIBRARY_PATH
# no longer required for torch
#export CUDA_HOME=/usr/local/cuda
#export PATH=${CUDA_HOME}/bin:${PATH}
#export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH
# Protects agains accidental package install in system python
export PIP_REQUIRE_VIRTUALENV=true
</code></pre>
<p>Yes you’re correct about <code>tch-rs</code> - it requires <code>libtorch</code>.</p>