Cuda version conundrum

username: Ed_Moman
cooked: <p>Hello,</p>
<p>Transformers relies on Pytorch, Tensorflow or Flax. I typically use the first.</p>
<p>In any case, the latest versions of Pytorch and Tensorflow are, at the time of this writing, compatible with Cuda 11.8.</p>
<p>Lucky me, for Cuda 11.8 is supposed to be the first version to support the RTX 4090 cards.</p>
<p>Well, not fully, apparently:</p>
<pre><code class="lang-auto">MapSMtoCores for SM 8.9 is undefined.  Default to use 128 Cores/SM
MapSMtoCores for SM 8.9 is undefined.  Default to use 128 Cores/SM
MapSMtoArchName for SM 8.9 is undefined.  Default to use Hopper
GPU Device 0: "Hopper" with compute capability 8.9
</code></pre>
<p>I believe the 4090 to be an Ada Lovelace, not a Hopper.</p>
<p>Will the fact that the card is not correctly identified by Cuda have any effect in resource utilisation and/or performance?</p>
<p>Is there anything we could do about that?</p>
<p>Does anyone know if Torch works with a more recent Cuda? Or can the MapSMtoCores and MapSMtoArchName variables be somehow hard-coded? Or is this completely irrelevant?</p>
<p>Best,</p>
<p>Ed</p>

username: Ed_Moman
cooked: <p>For Pytorch, the nightly version is compatible with Cuda 12.1, which fully supports the card and simplifies things considerably.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Ed_Moman" data-post="1" data-topic="185714">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ed_moman/48/62894_2.png" class="avatar"> Ed_Moman:</div>
<blockquote>
<p>Lucky me, for Cuda 11.8 is supposed to be the first version to support the RTX 4090 cards.</p>
</blockquote>
</aside>
<p>That’s not true, since <code>sm_89</code> is binary compatible to <code>sm_86</code> and <code>sm_80</code> and will thus work with any PyTorch binary using CUDA &gt;= 11.1.</p>
<p>I don’t know where the error message is raised from, but your 4090 is also not a Hopper GPU.</p>
<aside class="quote no-group" data-username="Ed_Moman" data-post="1" data-topic="185714">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ed_moman/48/62894_2.png" class="avatar"> Ed_Moman:</div>
<blockquote>
<p>Will the fact that the card is not correctly identified by Cuda have any effect in resource utilisation and/or performance?</p>
</blockquote>
</aside>
<p>Again, I doubt it’s not detected by CUDA, since the error message is also pointing to the wrong architecture name, so I guess it’s a 3rd party library,<br>
Run a pure PyTorch workload and check that the device is properly recognized and working. A simple smoke test would be <code>python -c "import torch; print(torch.cuda.get_device_properties(0); print(torch.randn(1).cuda())"</code>.</p>
<aside class="quote no-group" data-username="Ed_Moman" data-post="1" data-topic="185714">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ed_moman/48/62894_2.png" class="avatar"> Ed_Moman:</div>
<blockquote>
<p>Does anyone know if Torch works with a more recent Cuda?</p>
</blockquote>
</aside>
<p>PyTorch supports all recent CUDA versions and the nightly binaries ship with 11.8 and 12.1.<br>
Both releases support GPU architectures up to sm_90 (Hopper).</p>

username: Ed_Moman
cooked: <p>Thanks.</p>
<p>I have installed Cuda 12.1 and Pytorch nightly. Everything is fine now.</p>
<p>The messages came from the Cuda Samples (11.8). With Cuda Samples 12.1 the message is gone and the card is correctly recognised as Ada.</p>
<p>The messages are now:</p>
<pre><code class="lang-auto">GPU Device 0: "Ada" with compute capability 8.9
GPU Device 1: "Ada" with compute capability 8.9
</code></pre>