Any wheel for CUDA 12 coming?

username: meme
cooked: <p>I’ve been struggling with this for ages. NVIDIA only supports CUDA 12 for Fedora 36 and beyond.  Fedora 35 is deprecated. CUDA 12 has been out since September. PyTorch supposedly has supported CUDA 12 since late December, but it’s mid-March and there’s still no wheel for it - half a year after it came out.</p>
<p><a href="https://download.pytorch.org/whl/torch_stable.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://download.pytorch.org/whl/torch_stable.html</a></p>
<p>I’m struggling to compile it (gcc 12.2.1), but that’s just been a fail thusfar.  Appeared to be failing due to a ton of warnings and PyTorch’s use of -Werror (I’m a big fan of the use of Werror in development… but in production?).  I guessed that my gcc is too new and so downgraded it, but then I got even more errors.  Now I’m trying again by trying to suppress Werror and seeing how that goes…</p>
<p>Any chance we’ll get a wheel at any point? Thanks!  <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>(ED: Nope, compile failed… Fbgemm is still using Werror for some reason and complaining about possible uninitialized variables… sigh…)</p>
<p>(ED2: Well, I managed to shoehorn it into compiling by literally replacing my systemwide gcc/g++/cpp/c++ binaries with wrapper scripts that strip out all -Werror arguments.  But now when I actually include torch/python.h, type_caster is broken:</p>
<p>/usr/local/lib64/python3.10/site-packages/torch/include/pybind11/detail/…/cast.h:42:120: error: expected template-name before ‘&lt;’ token<br>
42 |     return caster.operator typename make_caster::template cast_op_type();<br>
|                                                                                                                        ^<br>
/usr/local/lib64/python3.10/site-packages/torch/include/pybind11/detail/…/cast.h:42:120: error: expected identifier before ‘&lt;’ token</p>
<p>… I give up  <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=12" title=":frowning:" class="emoji" alt=":frowning:" loading="lazy" width="20" height="20">  )</p>

username: ptrblck
cooked: <p>We are still in the process of discussing and merging the last few needed changes into the code base before starting with the CUDA 12.x bringup for the binaries.<br>
If you need to use this CUDA version, you would indeed need to build PyTorch from source or just use the latest NGC containers.<br>
As a workaround for now you could use the current binaries (stable or nightly) with CUDA 11.7 or 11.8 which already support all released GPU architectures.<br>
Also note that your locally installed CUDA toolkit won’t be used when executing the binaries as they ship with their own dependencies unless you build a custom CUDA extension.</p>

username: meme
cooked: <p>Okay, thanks for the update (I know that maintaining packages is a rather thankless job!)</p>

username: AidedSine035533
cooked: <p>My gamer box has CUDA 12.0 drivers. I installed CUDA 11.8 and it seemed to work, but should it have?</p>

username: ptrblck
cooked: <p>Yes, since the PyTorch binaries ship with their own CUDA dependencies and work with a properly installed driver. Your locally installed CUDA toolkit will be used if you build PyTorch from source or a custom CUDA extension.<br>
You can also install the nightlies with 12.1 now.</p>

username: Joshua9
cooked: <p>Are there nightly docker images? I found <a href="https://github.com/orgs/pytorch/packages/container/package/pytorch-nightly" class="inline-onebox" rel="noopener nofollow ugc">Package pytorch-nightly · GitHub</a> but there is no information about the used version of CUDA in those.</p>

username: ptrblck
cooked: <p>I’m not familiar with the nightly docker build process, but you might want to pull the container and check the used CUDA version via <code>print(torch.version.cuda)</code>.</p>

username: Joshua9
cooked: <p>Thanks, I will do that.</p>
<p>I am asking because he last pytorch docker 2.0.1-cuda11.7-cudnn8-runtime does not seem to support A100 and H100 GPUs (arch ‘sm_90’).</p>

username: ptrblck
cooked: <p>Yes, CUDA 11.7 does not support sm_90, but sm_80, and you would need CUDA &gt;= 11.8.</p>

username: cbcase
cooked: <p>Bumping this thread – is there a plan for a CUDA 12-compatible stable release imminently, or will it be just nightlies for the foreseeable future?</p>

username: ptrblck
cooked: <p>The next stable <code>2.1.0</code> release will use 11.8 and 12.1.</p>

username: cbcase
cooked: <p>Thanks, Piotr! Is there a planned release date for 2.1.0, or is that still TBD?</p>
<p>(Context is I need to build an H100-compatible image soon, and I’m figuring out whether to go to 11.8 now or if I can wait a few weeks and skip to 12.x – while sticking with wheel installation.)</p>

username: ptrblck
cooked: <p><code>2.1.0</code> is supposed to be released ~Oct. 2023, but note that the nightly binaries already support CUDA 12.1 and ship with it, so you could try these out in case nightly works for you.</p>

username: cbcase
cooked: <p><img src="https://discuss.pytorch.org/images/emoji/apple/+1.png?v=12" title=":+1:" class="emoji" alt=":+1:" loading="lazy" width="20" height="20">, thanks as always</p>