Why does the performance of pytorch generated by compilation degrade. （cudnn）

cxhmyself: <p>Recently, I compiled pytorch according to the tips above GitHub, but I found that the pytorch library generated by compilation will slow down when running convolution operation. Through NSYS, I found that the API duration on the CPU side has become longer. What is the reason and how to solve it.</p>

ptrblck: <p>Could you explain which calls are seeing a slowdown on the CPU and how this would be related to cuDNN, please?</p>

cxhmyself: <p>This is my code.</p>
<pre><code class="lang-auto">input_size = [128, 128, 112, 112]
conv_param = [128, 256, [3,3], 1, 1, 1, 1, False]
input = torch.randn([128, input_size[1], input_size[2], input_size[3]], device = 'cuda:0')
cnt = 100
net = torch.nn.Conv2d(*conv_param).to('cuda:0')
output = net(input)
start  = time.time()
torch.cuda.synchronize()
while cnt &gt; 0:
    output = net(input)
    cnt -= 1
torch.cuda.synchronize()
end  = time.time()
print('Duration is ', end - start)
</code></pre>
<p>Compile pytorch and run the above code. The result is 7.132707357406616s<br>
PIP installs pytorch and runs the above code. The result is 4.728105068206787s</p>
<p>The operation of NSYS (another program, but the problems described are consistent) is shown in the following figure. Figure 1 shows the results generated by compilation, and Figure 2 shows the results generated by direct installation. We can see that the CPU duration of generatewinogradtileskernel in Figure 1 is 5.863 milliseconds. In Figure 2, the CPU duration of generatewinogradtileskernel is 178.632 microseconds.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/f/eff60344a71b9395d7b6076742b9f360c3d8e0d1.png" data-download-href="https://discuss.pytorch.org/uploads/default/eff60344a71b9395d7b6076742b9f360c3d8e0d1" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/f/eff60344a71b9395d7b6076742b9f360c3d8e0d1_2_690x439.png" alt="image" data-base62-sha1="yeNc6Xlt6aCNjFxt9UbG3A7Tyyl" width="690" height="439" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/f/eff60344a71b9395d7b6076742b9f360c3d8e0d1_2_690x439.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/f/eff60344a71b9395d7b6076742b9f360c3d8e0d1_2_1035x658.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/e/f/eff60344a71b9395d7b6076742b9f360c3d8e0d1.png 2x" data-dominant-color="D8E9D9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1205×768 33.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Which CUDA and cuDNN version are you using in the source build vs. the binary (i.e. which PyTorch version and CUDA runtime did you select for the binaries)?<br>
I would assume that the change in all 3 libs (torch, CUDA, cuDNN) would create different runtimes, but it’s hard to tell where the difference is coming from when all these libs were moved.</p>

cxhmyself: <p>Source build：</p>
<pre><code class="lang-auto">cuda: 11.2
cudnn: 8201
torch: 1.13.0a0+git7360b53
</code></pre>
<p>binary:</p>
<pre><code class="lang-auto">cuda: 10.2
cudnn: 7605
torch: 1.10.0+cu102
</code></pre>

ptrblck: <p>You could try the latest releases for the source build (CUDA 11.7.0 and cuDNN 8.4) and/or the CUDA11.6 binary which comes with cuDNN 8.3.2 and see how it compares to these old releases.</p>

cxhmyself: <p>Thank you for your patient reply. I’ll try what you said.</p>