Group convolution takes much longer than normal convolution

yjy765: <p>Hi,</p>
<p>I have an issue with time to be taken when I use group convolution</p>
<p>I compared time with normal convolution which have same number of parameters and it takes much shorter time</p>
<p>Test code is as follows :</p>
<pre><code class="lang-auto">% dataloader is given. dataloader provides two batch data of (32,12,50,50) shape
loader = enumerate(dataloader)
net1 = torch.nn.Conv2d(12, 300, kernel_size=5, stride=1, padding=(2,2), bias=False) % normal convolution
net2 = torch.nn.Conv2d(12, 12*300, groups=12, kernel_size=5, stride=1, padding=(2,2), bias=False) % group convolution
net1.cuda()
net2.cuda()
% compare normal conv and group conv
for _ in range(3):
  _, (a,b) = loader.__next__()
  a = a.cuda()
  torch.cuda.synchronize()
  start = time.time()
  o = net1(a) % line**
  torch.cuda.synchronize()
  end = time.time()
</code></pre>
<p>I recorded a time to take for convolution operation and repeat again replacing line** to “o = net2(a)”<br>
Following is when I used normal convolution</p>
<blockquote>
<p>iter 0 : 0.12s<br>
iter 1 : 0.00536s<br>
iter 2 : 0.00494s</p>
</blockquote>
<p>And then I replaced net1-&gt;net2 i.e., normal conv to group conv</p>
<blockquote>
<p>iter 0 : 0.2497s<br>
iter 1 : 0.2162s<br>
iter 2 : 0.20505s</p>
</blockquote>
<p>I guess that net1 and net2 have the same number of weight parameters and they give exactly the same result when reshape and addition over input channel are performed to group convolution result. However time cost is so different. Especially, time for normal convolution dramatically reduces after one iteration while not for group convolution</p>
<p>Can I run group convolution as fast as normal convolution??</p>
<p>p.s.<br>
I need group convolution to implement following custom layer<br>
X1…Xn (Xi is i-th channel) → Y1 … Ym (Yj is j-th channel)<br>
where Yj = \Sigma_i (Xi * Kij) @ (Xi * L)<br>
Kij is kernel, * is convolution, @ is element-wise multiplication and L is another fixed kernel.<br>
To implement this I used group convolution for two convolution and then elementwise-multipy and add over one channel. But it is too slow.</p>

ipostr08: <p>Yes, it’s very slow <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_frowning_face.png?v=12" title=":slightly_frowning_face:" class="emoji" alt=":slightly_frowning_face:" loading="lazy" width="20" height="20">. This has been mentioned many times in github issues but there haven’t been any fixes and I don’t know if anybody is working on this. E.g.<br>
<a href="https://github.com/pytorch/pytorch/issues/73764" class="inline-onebox" rel="noopener nofollow ugc">Group convolution slower than manually running separate convolutions in CUDA streams · Issue #73764 · pytorch/pytorch · GitHub</a>,<br>
<a href="https://github.com/pytorch/pytorch/issues/18631" class="inline-onebox" rel="noopener nofollow ugc">FP32 depthwise convolution is slow in GPU · Issue #18631 · pytorch/pytorch · GitHub</a>,<br>
<a href="https://github.com/pytorch/pytorch/issues/70954" class="inline-onebox" rel="noopener nofollow ugc">Training grouped Conv2D is slow · Issue #70954 · pytorch/pytorch · GitHub</a></p>