Calculation time of 'torch.nn.functional.conv2d '

is_Juncheng: <p>Why is it so time-consuming！！！<br>
I need to use custom parameters in the network. So I used this function in forward propagation, but practice shows that its calculation is very time-consuming! Before the calculation, I have placed the input and weight on the GPU, but the time is as high as 2S. I don’t know why. Can someone help me?<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/d/4/d42ac002f0cf4ae56595305cf7eefe5ff7a8dd27.png" alt="image" data-base62-sha1="ugUOxWZ85Q5iCAGUDN60cBpu0xF" width="462" height="175"></p>

is_Juncheng: <p>this code is integrated in yolov5, the img shape is [16,640,640,3], the x_ shape is [16,1,640,640], the weight shape is [1,1,25,25].<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/c/4cf1b7b761d72b7c068950dff64143a461e4ba1b.png" data-download-href="https://discuss.pytorch.org/uploads/default/4cf1b7b761d72b7c068950dff64143a461e4ba1b" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/c/4cf1b7b761d72b7c068950dff64143a461e4ba1b_2_690x82.png" alt="image" data-base62-sha1="aYGafQGjW4izHwTOs2Jwuiee4T9" width="690" height="82" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/c/4cf1b7b761d72b7c068950dff64143a461e4ba1b_2_690x82.png, https://discuss.pytorch.org/uploads/default/optimized/3X/4/c/4cf1b7b761d72b7c068950dff64143a461e4ba1b_2_1035x123.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/c/4cf1b7b761d72b7c068950dff64143a461e4ba1b_2_1380x164.png 2x" data-dominant-color="DCDFE8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1448×173 28.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
As can be seen from the figure, the time is up to 2 ~ 3 seconds.</p>

eqy: <p>Depending on the hardware (GPU) used this isn’t entirely unexpected. Keep in mind that scaling is basically quadratic with respect to the spatial dimension, with <code>(640**2)/(224**2)</code> being about 8x.</p>
<p>However, indexing the channel dimensions of the image seems highly unusual (what is the purpose of this?) and would also slow things down due to the multiple kernel invocations.</p>
<p>What is the GPU in this setup and are there any other optimizations being applied (e.g., fp16)?</p>

is_Juncheng: <p>Thank you for your reply. I did an independent test later: in the same device, when I randomly initialize the input and weight of the same dimension for operation, the time is greatly reduced, and I don’t know what the problem is. Is it because of my integration in yolov5?<br>
This is an attempt of depth-wise convolution.<br>
GPU settings are the same as yolov5.</p>

eqy: <blockquote>
<p>GPU settings are the same as yolov5.<br>
I assume that means you are using V100?</p>
</blockquote>
<p>How is <code>img</code> initialized in this setup? It would be very helpful if you could post a runnable code snippet that reproduces the problem (e.g., make a dummy conv2d layer that has the same shapes) for debugging.</p>