The CUDA kernel running on python and C++ is different

111480: <p>GPU profiling has shown that the kinds of CUDA kernels that run internally are different between python and C++.</p>
<p>Python</p>
<blockquote>
<p>models.densenet201(pretrained=True).cuda().eval()</p>
</blockquote>
<blockquote>
<p>torch.jit.load(“densenet201_model.pt”).cuda().eval()</p>
</blockquote>
<p>C++</p>
<blockquote>
<p>torch::jit::load(“densenet201_model.pt”);</p>
</blockquote>
<p>The above three results and the layers running are the same, but the types and numbers of CUDA kernels running internally are all different.</p>
<p>There were more kernel types running on python.</p>
<p>What’s the reason for this result?</p>

ptrblck: <p>Do you know which kernels were called additionally in Python and were you using the same setup? E.g. in case you were using cudnn’s benchmark mode, different kernels will be profiled for each new input shape, which would also show up in the profile.</p>

111480: <p>I did not explicitly use the following code.</p>
<blockquote>
<p>torch.backends.cudnn.enabled == True</p>
</blockquote>
<p>Is using cudnn default?</p>
<p>The list of kernels each uses is as follows.</p>
<p>Python</p>
<blockquote>
<p>models.densenet201(pretrained=True).cuda().eval()</p>
</blockquote>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/3/1332ceab66af687296ef2cddd272cb8aaa780a91.png" data-download-href="https://discuss.pytorch.org/uploads/default/1332ceab66af687296ef2cddd272cb8aaa780a91" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/3/1332ceab66af687296ef2cddd272cb8aaa780a91.png" alt="image" data-base62-sha1="2JPVpRudze0NRoe0HbJKciHXgMp" width="517" height="341" data-dominant-color="EBEBEC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1256×830 46.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<blockquote>
<p>torch.jit.load(“densenet201_model.pt”).cuda().eval()</p>
</blockquote>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/3/c36f2fcdf6bc9fb706237ed22b5ef1a0b05094f5.png" data-download-href="https://discuss.pytorch.org/uploads/default/c36f2fcdf6bc9fb706237ed22b5ef1a0b05094f5" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/c/3/c36f2fcdf6bc9fb706237ed22b5ef1a0b05094f5.png" alt="image" data-base62-sha1="rSTk2sPhf69xXOQdocd3lEPwIGF" width="517" height="344" data-dominant-color="ECECEC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1252×834 44.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>C++</p>
<blockquote>
<p>torch::jit::load(“densenet201_model.pt”);</p>
</blockquote>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/8/68bbc02d9de092d92d22f963983f4f6664447f15.png" data-download-href="https://discuss.pytorch.org/uploads/default/68bbc02d9de092d92d22f963983f4f6664447f15" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/8/68bbc02d9de092d92d22f963983f4f6664447f15.png" alt="image" data-base62-sha1="eWvUtDZnFhfHna3AarmZbvm2Asd" width="517" height="341" data-dominant-color="F4F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1254×828 27.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Why are all the results different even though it’s the same DenseNet201?</p>