Any change of using CUDA 12.2?

hgfernan: <p>1st of all, congratulations for your amazing work.</p>
<p>I would like to use CUDA support, but as I can see in <a href="https://pytorch.org/get-started/locally/" rel="noopener nofollow ugc">Start Locally | PyTorch</a>, the current PyTorch package demands CUDA 11.7 or 11.8</p>
<p>However, NVIDIA is distributing the binaries of version 12.2</p>
<p>Is the upgrade to CUDA 12.2 in the near term plans of PyTorch development team ?</p>
<p>Best regards,<br>
Hilton</p>

ptrblck: <p>The nightly binaries ship with 11.8 and 12.1 already which would be the target for the next PyTorch 2.1.0 release. We will start bringing up 12.2 soon, but it would miss the 2.1.0 release. In the meantime you can build from source.</p>

hgfernan: <p>Thanks a lot, kind sir !</p>
<p>I will take a look at building PyTorch from sources. PyTorch is certainly a huge software in any sense of the word, but NVIDIA seems more complex to be built, as it has so many interactions with hardware.</p>

bhack: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Are we going to have some issue with nightly with Torch compile for the Triton cuda version <a href="https://github.com/openai/triton/issues/1955" class="inline-onebox" rel="noopener nofollow ugc">After build from source, Error occured "device kernel image is invalid" · Issue #1955 · openai/triton · GitHub</a>?</p>

ptrblck: <p>OpenAI/Triton should not depend on any PyTorch CUDA libs, so I would not expect to see issues. However, I’m not familiar enough with Triton issues and their roadmap.</p>

bhack: <p>It is more about using nightly pytorch where the CUDA version of trition it seems harcorded:</p><aside class="onebox githubblob" data-onebox-src="https://github.com/openai/triton/blob/9e3e10c5edb4a062cf547ae73e6ebfb19aad7bdf/python/setup.py#L128-L129">
  <header class="source">

      <a href="https://github.com/openai/triton/blob/9e3e10c5edb4a062cf547ae73e6ebfb19aad7bdf/python/setup.py#L128-L129" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/openai/triton/blob/9e3e10c5edb4a062cf547ae73e6ebfb19aad7bdf/python/setup.py#L128-L129" target="_blank" rel="noopener nofollow ugc">openai/triton/blob/9e3e10c5edb4a062cf547ae73e6ebfb19aad7bdf/python/setup.py#L128-L129</a></h4>



    <pre class="onebox"><code class="lang-py">
      <ol class="start lines" start="128" style="counter-reset: li-counter 127 ;">
          <li>version = "12.2.91"</li>
          <li>url = f"https://conda.anaconda.org/nvidia/label/cuda-12.2.0/linux-64/cuda-nvcc-{version}-0.tar.bz2"</li>
      </ol>
    </code></pre>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>I am on the same commit as our CI:</p><aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/pytorch/blob/main/.ci/docker/ci_commit_pins/triton.txt">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/blob/main/.ci/docker/ci_commit_pins/triton.txt" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/main/.ci/docker/ci_commit_pins/triton.txt" target="_blank" rel="noopener nofollow ugc">pytorch/pytorch/blob/main/.ci/docker/ci_commit_pins/triton.txt</a></h4>


      <pre><code class="lang-txt">9e3e10c5edb4a062cf547ae73e6ebfb19aad7bdf
</code></pre>




  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>How we are going to still release working (<code>torch.compile</code>) cuda 11.8 nightly wheels?</p>

ptrblck: <p>It’s still unclear what kind of issues you are seeing. Triton should use its own <code>ptxas</code>, shouldn’t it?<br>
Which dependencies would be conflicting and what could fail? Again, I’m not the code owner of Triton, but expect to see no Triton-&gt;PyTorch dependencies for CUDA.</p>

bhack: <p>What I meant is that the official conda pytorch nightly CUDA 11.8 is going to trigger Triton <code>device kernel image is invalid</code></p>

stebix: <p>Yup, seeing this problem as well. I installed pytorch-nightly in a fresh conda environment, but any attempt at compilation fails due to</p>
<pre><code class="lang-auto">[2023-07-29 04:04:26,800] torch._dynamo.convert_frame: [WARNING]   File "/home/user/anaconda3/envs/torchnightly/lib/python3.11/site-packages/triton/compiler/compiler.py", line 589, in _init_handles
[2023-07-29 04:04:26,800] torch._dynamo.convert_frame: [WARNING]     mod, func, n_regs, n_spills = fn_load_binary(self.metadata["name"], self.asm[bin_path], self.shared, device)
[2023-07-29 04:04:26,800] torch._dynamo.convert_frame: [WARNING]                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2023-07-29 04:04:26,800] torch._dynamo.convert_frame: [WARNING] torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
[2023-07-29 04:04:26,800] torch._dynamo.convert_frame: [WARNING] RuntimeError: Triton Error [CUDA]: device kernel image is invalid
</code></pre>

maxpain: <p>Any updates on this?</p>

bhack: <p>There is a ticket <a href="https://github.com/pytorch/pytorch/issues/106144" class="inline-onebox" rel="noopener nofollow ugc">Pytorch nighlty and openAI/triton cuda · Issue #106144 · pytorch/pytorch · GitHub</a></p>