Rtx 3070/3080 support

username: yoad.tewel
cooked: <p>Hello!<br>
So I’ve got a machine with ubuntu 20.04 and rtx 3070. Is it possible to run pytorch at this time with support to the new GPUs?<br>
From my understanding for the rtx 3070 I need cudnn 8.0.5 and cuda 11.1, is there a way to get pytorch to work this these versions?</p>
<ol>
<li>What are my current options to install pytorch? (for example, should I install cuda 11.1, cudnn 8.0.5 and build from source, or can I install binaries)</li>
<li>For the currently supported options, are there speed problems? And if so, how much of a speed decrease should I expect in comparison to optimized setup?</li>
</ol>
<p>Thanks!</p>

username: albanD
cooked: <p>Hi,</p>
<ol>
<li>Current binaries for cuda 11.0 will work with these cards</li>
<li>There are perf issues with these because the current cuda libraries are not properly optimized for them. We will release 1.7.1 soon to update to cudnn 8.0.5 to fix some of these. But it won’t fix everything I’m afraid and we’ll have to wait for newer versions of the binaries.</li>
</ol>
<p>You can compile from source with 11.1 and cudnn 8.0.5 but I am not sure if it will fix all of the perf regressions. I don’t remember if there is another version of cudnn coming with more fixes soon or not? cc <a class="mention" href="/u/ptrblck">@ptrblck</a></p>

username: yoad.tewel
cooked: <p><a class="mention" href="/u/alband">@albanD</a>  Thanks for answering!<br>
So just make sure:</p>
<ol>
<li>If I will use the current binaries (1.7.0), I should install cuda 11.0, and which cudnn version?</li>
<li>Anothe option is to install cuda 11.1, cudnn 8.0.5, and then build compile pytorch from source.</li>
<li>Last option is to wait for 1.7.1, and then install it with cuda 11.1 and cudnn 8.0.5</li>
</ol>
<p>Is this right?</p>

username: albanD
cooked: <ol>
<li>Both cuda and cudnn are shipped with the binary. No need for you to worry about that. But the current binaries ship with cudnn 8.0.4 (the only thing that was available when they were built).</li>
<li>Yes that is the only option to get these two versions</li>
<li>The 1.7.1 binaries will ship with cuda 11.0 (EDIT: see <a class="mention" href="/u/ptrblck">@ptrblck</a> comment below, this might change)  because conda still does not support 11.1 <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:">  But it will have cudnn 8.0.5 which should fix most perf regressions!</li>
</ol>

username: ptrblck
cooked: <p>Minor corrections:</p>
<ul>
<li>1.7.0 ships with CUDA11.0 + cudnn8.0.3. Unfortunately, we didn’t trigger a rebuild of the binaries after bumping cudnn to 8.0.4. <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:">
</li>
<li>We are working on workarounds for the pruning issues with 11.1 and think we could ship 1.7.1 with CUDA11.1 + cudnn8.0.5</li>
</ul>
<p>cudnn 8.0.5 ships with updated heuristics for 3090. In 8.1.x the full sm86 heuristics should be added.</p>

username: yoad.tewel
cooked: <p>Hey, so now that the 1.7.1 version is out, will it ship with cuda 11.1? because currently the stable version I saw is 1.7.1 with cudnn 8.0.5 and cuda 11.0</p>

username: ptrblck
cooked: <p>No, the build team focused on the Python3.9 support and didn’t have enough resources to target CUDA11.1 as well for this release.</p>

username: yoad.tewel
cooked: <p>Ok thanks,<br>
and do you know if there are supposed to be any performence difference between cuda 11.0 and 11.1 (both using cudnn 8.0.5), with rtx 3070?<br>
And also, should there be any difference between performence of pytorch nightly and pytorch 1.7.1 right now reagarding rtx 3070?</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="yoad.tewel" data-post="8" data-topic="104895">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yoad.tewel/48/30861_2.png" class="avatar"> yoad.tewel:</div>
<blockquote>
<p>any performence difference between cuda 11.0 and 11.1 (both using cudnn 8.0.5), with rtx 3070?</p>
</blockquote>
</aside>
<p>Not as of now, since most (if not all) native PyTorch kernels are memory bandwidth bound and wouldn’t benefit from an increase in compute performance, while future cudnn and cublas versions should ship with sm86-specific instructions to increase the performance on the 30xx series.</p>
<aside class="quote no-group" data-username="yoad.tewel" data-post="8" data-topic="104895">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yoad.tewel/48/30861_2.png" class="avatar"> yoad.tewel:</div>
<blockquote>
<p>difference between performence of pytorch nightly and pytorch 1.7.1 right now reagarding rtx 3070?</p>
</blockquote>
</aside>
<p>The nightly release gets the latest updates, e.g. such as the usage of <a href="https://github.com/pytorch/pytorch/pull/48675"><code>fastAtomicAdd</code> for trilinear upsampling</a>, and could thus improve the performance.<br>
However, while I don’t have a bad experience with the stability of the nightly binaries, you might want/need to stick to the stable release.</p>

username: Felix_Lessange
cooked: <p>Hi. I tested 3d convolution models with 3070, nvidia’s official containers, pytorch 1.7.1, 1.8 nightly release, cudnn 8.0.4 and 8.0.5 and all combinations show the same speed.</p>
<p>Also I’m expecting similar to better performance than 2080 Ti in FP16 inference  (don’t know if I’m right though !), and I’m instead observing about 75% the perf right now.</p>
<p>I knnow this isn’t a benchmark post, but just to say : this soft will only <strong>technically</strong> support rtx 3000 in the “short” term.</p>

username: gelli_tarun
cooked: <p>Hii please help me with this is 3070 ti compatible with PyTorch??and is 3070ti better than 4070 for deep learning??</p>

username: ptrblck
cooked: <p>Yes, all Ampere GPUs, including the 3070 (TI), are compatible and will work.</p>

username: gelli_tarun
cooked: <p>In laptops?? Like built in gpus</p>

username: ptrblck
cooked: <p>Yes, as all NVIDIA GPUs support CUDA.</p>

username: gelli_tarun
cooked: <p>Hi sir,can you please help me that is tensorflow supported in 12.3 cuda toolkit or can i install lesser if i have 3070ti please</p>

username: ptrblck
cooked: <p>I don’t know which CUDA version Tensorflow supports.</p>