When I install torch==2.6.0 with whl/cu126, none of cuda dependencies get installed. I have cu126 in the environment already. Is this expected?

username: kofuji
cooked: <p><strong>Background</strong></p>
<p>I’m trying to build pytorch docker image here <a href="https://github.com/microsoft/dstoolkit-devcontainers/blob/main/src/sample_pytorch_gpu_project/.devcontainer/Dockerfile" rel="noopener nofollow ugc">dstoolkit-devcontainers/src/sample_pytorch_gpu_project/.devcontainer/Dockerfile at main · microsoft/dstoolkit-devcontainers</a>. I was creating a pytorch container with nvidia/cuda as base and then installing pytorch on top of that. I was hoping that pytorch uses already installed cuda dependencies in that way. Recently I learned that this way, pytorch actually doesn’t use available cuda dependencies in the base container but it just installs them as part of <code>pip install torch==2.6.0</code>. So I’ve been investigating how to make pytorch not install cuda dependencies but use already installed ones</p>
<p><strong>Question</strong></p>
<p>As part of this exploration, I realized that <code>pip install torch==2.6.0</code> actually installs CUDA 12.4 version that mismatches with the base container I use <code>nvidia/cuda:12.6.3-cudnn-runtime-ubuntu24.04</code>.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/e/0ed4e9db0ae58fbdcaac67a5d5a4183acf24f67f.png" data-download-href="https://discuss.pytorch.org/uploads/default/0ed4e9db0ae58fbdcaac67a5d5a4183acf24f67f" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/e/0ed4e9db0ae58fbdcaac67a5d5a4183acf24f67f_2_690x280.png" alt="image" data-base62-sha1="27cQG48Kc1zkX4qMcRyFeeTe6FV" width="690" height="280" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/e/0ed4e9db0ae58fbdcaac67a5d5a4183acf24f67f_2_690x280.png, https://discuss.pytorch.org/uploads/default/optimized/3X/0/e/0ed4e9db0ae58fbdcaac67a5d5a4183acf24f67f_2_1035x420.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/0/e/0ed4e9db0ae58fbdcaac67a5d5a4183acf24f67f_2_1380x560.png 2x" data-dominant-color="F2DFDC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1680×684 52.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>When I switched to <code>--index-url https://download.pytorch.org/whl/cu126</code>, it stopped installing cuda dependencies and now torch tries to refer to locally preinstalled cuda dependencies so I achieved the goal I mentioned in Background. But I only get this behavior when I choose cu126. When I use cu118 or default (cuda 12.4), it still installs cuda dependencies. Is this inconsistent behavior expected? Does <code>pip install torch</code> process check local cuda version and switch whether to install dependency or not? If this is expected and this is how to use locally installed cuda dependencies, I’m happy with it but wanted to make sure that this is not bug.</p>
<p><strong>Details</strong><br>
Actually I’m currently testing uv as replacement of pip</p>
<p>So this is what I configure in pyproject.toml for pytorch 2.6.0 with cu126</p>
<pre><code class="lang-auto">[[tool.uv.index]]
name = "pytorch-cu126"
url = "https://download.pytorch.org/whl/cu126"
explicit = true

[tool.uv.sources]
torch = [
    { index = "pytorch-cu126" },
]
torchvision = [
    { index = "pytorch-cu126" },
]
</code></pre>
<p>When I swtich this to cu118 or get rid of these (default, cuda 12.4), I get the follwing with <code>uv lock</code> so I see cuda dependencies only come either with cu118 or cu124 and not with cu126</p>
<p>cu126 to cu124</p>
<pre><code class="lang-auto">Resolved 126 packages in 44.21s
Add nvidia-cublas-cu12 v12.4.5.8
Add nvidia-cuda-cupti-cu12 v12.4.127
Add nvidia-cuda-nvrtc-cu12 v12.4.127
Add nvidia-cuda-runtime-cu12 v12.4.127
Add nvidia-cudnn-cu12 v9.1.0.70
Add nvidia-cufft-cu12 v11.2.1.3
Add nvidia-curand-cu12 v10.3.5.147
Add nvidia-cusolver-cu12 v11.6.1.9
Add nvidia-cusparse-cu12 v12.3.1.170
Add nvidia-cusparselt-cu12 v0.6.2
Add nvidia-nccl-cu12 v2.21.5
Add nvidia-nvjitlink-cu12 v12.4.127
Add nvidia-nvtx-cu12 v12.4.127
Update torch v2.6.0+cu126 -&gt; v2.6.0
Update torchvision v0.21.0, v0.21.0+cu126 -&gt; v0.21.0
Add triton v3.2.0
</code></pre>
<p>cu126 to cu118</p>
<pre><code class="lang-auto">Resolved 124 packages in 5.03s
Add nvidia-cublas-cu11 v11.11.3.6
Add nvidia-cuda-cupti-cu11 v11.8.87
Add nvidia-cuda-nvrtc-cu11 v11.8.89
Add nvidia-cuda-runtime-cu11 v11.8.89
Add nvidia-cudnn-cu11 v9.1.0.70
Add nvidia-cufft-cu11 v10.9.0.58
Add nvidia-curand-cu11 v10.3.0.86
Add nvidia-cusolver-cu11 v11.4.1.48
Add nvidia-cusparse-cu11 v11.7.5.86
Add nvidia-nccl-cu11 v2.21.5
Add nvidia-nvtx-cu11 v11.8.86
Update torch v2.6.0+cu126 -&gt; v2.6.0+cu118
Update torchvision v0.21.0, v0.21.0+cu126 -&gt; v0.21.0+cu118
Add triton v3.2.0
</code></pre>

username: ptrblck
cooked: <aside class="quote no-group" data-username="kofuji" data-post="1" data-topic="218014">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/91b2a8/48.png" class="avatar"> kofuji:</div>
<blockquote>
<p>When I switched to <code>--index-url https://download.pytorch.org/whl/cu126</code>, it stopped installing cuda dependencies and now torch tries to refer to locally preinstalled cuda dependencies so I achieved the goal I mentioned in Background.</p>
</blockquote>
</aside>
<p>That’s not the case as the PyTorch binaries build with CUDA 12.6 are still shipping with CUDA runtime dependencies and the install log will also show it in the same way as for CUDA 12.4.</p>
<p>If you want to use your locally installed CUDA toolkit you can build PyTorch from source.</p>

username: twoudsma
cooked: <p>I have the same issue. When going from torch==2.6.0+cu124 to torch==2.6.0+cu126, the CUDA packages are removed.</p>
<pre><code class="lang-auto">Uninstalled 16 packages in 133ms
Installed 2 packages in 125ms
 - nvidia-cublas-cu12==12.4.5.8
 - nvidia-cuda-cupti-cu12==12.4.127
 - nvidia-cuda-nvrtc-cu12==12.4.127
 - nvidia-cuda-runtime-cu12==12.4.127
 - nvidia-cudnn-cu12==9.1.0.70
 - nvidia-cufft-cu12==11.2.1.3
 - nvidia-curand-cu12==10.3.5.147
 - nvidia-cusolver-cu12==11.6.1.9
 - nvidia-cusparse-cu12==12.3.1.170
 - nvidia-cusparselt-cu12==0.6.2
 - nvidia-nccl-cu12==2.21.5
 - nvidia-nvjitlink-cu12==12.4.127
 - nvidia-nvtx-cu12==12.4.127
 - torch==2.6.0+cu124
 + torch==2.6.0+cu126
 - torchvision==0.21.0+cu124
 + torchvision==0.21.0+cu126
 - triton==3.2.0
</code></pre>

username: ptrblck
cooked: <p>This is expected since the PyTorch binaries ship with their own CUDA runtime dependencies as already explained.<br>
If you install PyTorch built with CUDA 12.6, the previous CUDA libs should be uninstalled and replaced with the ones from 12.6.</p>
<p>Running <code>pip install torch --index-url https://download.pytorch.org/whl/cu126</code> shows:</p>
<pre data-code-wrap="python"><code class="lang-python">Collecting nvidia-cuda-nvrtc-cu12==12.6.77 (from torch)
  Downloading https://download.pytorch.org/whl/cu126/nvidia_cuda_nvrtc_cu12-12.6.77-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cuda-runtime-cu12==12.6.77 (from torch)
  Downloading https://download.pytorch.org/whl/cu126/nvidia_cuda_runtime_cu12-12.6.77-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cuda-cupti-cu12==12.6.80 (from torch)
  Downloading https://download.pytorch.org/whl/cu126/nvidia_cuda_cupti_cu12-12.6.80-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cudnn-cu12==9.5.1.17 (from torch)
  Downloading https://download.pytorch.org/whl/cu126/nvidia_cudnn_cu12-9.5.1.17-py3-none-manylinux_2_28_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cublas-cu12==12.6.4.1 (from torch)
  Downloading https://download.pytorch.org/whl/cu126/nvidia_cublas_cu12-12.6.4.1-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cufft-cu12==11.3.0.4 (from torch)
  Downloading https://download.pytorch.org/whl/cu126/nvidia_cufft_cu12-11.3.0.4-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-curand-cu12==10.3.7.77 (from torch)
  Downloading https://download.pytorch.org/whl/cu126/nvidia_curand_cu12-10.3.7.77-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cusolver-cu12==11.7.1.2 (from torch)
  Downloading https://download.pytorch.org/whl/cu126/nvidia_cusolver_cu12-11.7.1.2-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cusparse-cu12==12.5.4.2 (from torch)
  Downloading https://download.pytorch.org/whl/cu126/nvidia_cusparse_cu12-12.5.4.2-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cusparselt-cu12==0.6.3 (from torch)
  Downloading https://download.pytorch.org/whl/cu126/nvidia_cusparselt_cu12-0.6.3-py3-none-manylinux2014_x86_64.whl.metadata (6.8 kB)
Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in /usr/local/lib/python3.12/dist-packages (from torch) (2.21.5)
Collecting nvidia-nvtx-cu12==12.6.77 (from torch)
  Downloading https://download.pytorch.org/whl/cu126/nvidia_nvtx_cu12-12.6.77-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-nvjitlink-cu12==12.6.85 (from torch)
  Downloading https://download.pytorch.org/whl/cu126/nvidia_nvjitlink_cu12-12.6.85-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.5 kB)
Requirement already satisfied: triton==3.2.0 in /usr/local/lib/python3.12/dist-packages (from torch) (3.2.0)
</code></pre>
<p>which is correct since the command explicitly specifies to install PyTorch binaries with CUDA 12.6 runtime dependencies.</p>

username: twoudsma
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="218014">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>pip install torch --index-url <a href="https://download.pytorch.org/whl/cu126" rel="noopener nofollow ugc">https://download.pytorch.org/whl/cu126</a></p>
</blockquote>
</aside>
<p>This indeed works, but when using uv as OP also did it does not install the CUDA dependencies.</p>

username: ptrblck
cooked: <p>I’m not familiar with <code>uv</code>, sorry.<br>
CC <a class="mention" href="/u/malfet">@malfet</a> in case you’ve seen some unexpected behavior.</p>