How to sum every k channels for a CNN feature map

username: kevin-ssy
cooked: <p>Hey guys, I want to sum every k consecutive channels of a variable together. Assume that the input before summing is with the shape like NxCxWxH, and the output after the summation should be Nx(C/k)xWxH. I implemented the method using the code below. the function sum_up is called in the network’s forward function, but it seems to be extremely inefficient. Is there any better way to implement this in pytorch?</p>
<pre><code class="lang-auto">def sum_up(input, bottle_neck=2):
      output= []
      for ch in xrange(0, input.data.shape[1], bottle_neck):
           output.append(torch.sum(input[:, ch:ch+bottle_neck, :, :], 1))
      return torch.stack(output).permute(1, 0, 2, 3).cuda()
</code></pre>

username: tom
cooked: <p>You can split the channel dimension with a view/reshape and sum</p>
<pre><code class="lang-auto">n,c,h,w = input.size()
res = input.reshape(n, c//k, k, h, w).sum(2)
</code></pre>
<p>Best regards</p>
<p>Thomas</p>

username: kevin-ssy
cooked: <p>Thanks! This works for me!</p>

username: SimonW
cooked: <p>Btw, if you are trying to do group sum, it will be available in the next release soon.</p>

username: akhilpan
cooked: <p>Could you also tell some efficient way of adding channels in an interleaved fashion? For example if there are 6 channels [0-5], I wish to add 0 th and 3rd, 1st and 4th, 2nd and 5th?</p>

username: kelam_goutam
cooked: <p>I have not tried it, but I have a vague idea.</p>
<pre><code class="lang-auto">for i in range(No_of_channels):
    new_channel = old_feature_map[i]+old_feature_map[i+1]
   # now concatenate the new channels one after another to get a new feature map.


</code></pre>

username: akhilpan
cooked: <p>I have implemented this but it seems to be inefficient  as we are using for loop. I am actually looking for some efficient way to do that. Anyways thanks for your answer.</p>

username: justusschock
cooked: <p>This should work with advanced indexing like this (haven’t tested it yet):</p>
<pre><code class="lang-auto">new_tensor = tensor[:, ::2] + tensor[:, 1::2]
</code></pre>
<p>Edit: sorry, you would of course have to specify the middle channel like this:</p>
<pre><code class="lang-auto">middle_channel = tensor.size(1)//2
new_tensor = tensor[:, :middle_channel] + tensor[:, middle_channel:] 
</code></pre>
<p>Sorry for the first part, I misunderstood your goal.</p>

username: akhilpan
cooked: <p>Thanks <a class="mention" href="/u/justusschock">@justusschock</a> for the answer.</p>

username: justusschock
cooked: <p>Did you try it? Does it work?</p>

username: akhilpan
cooked: <p>Not yet. Will try it out tomorrow.</p>

username: tom
cooked: <p>Well, you can extend the trick above, if you have <code>c</code> channels, c divisible by k*l:</p>
<pre><code class="lang-python">n,c,h,w = input.size()
res = input.reshape(n, c//k//l, k, l, l, h, w).sum(2).view(n, c//k, h, w)
</code></pre>
<p>This will sum k channels from an interleaving grid of spacing l.</p>
<p>Best regards</p>
<p>Thomas</p>

username: akhilpan
cooked: <p>Thanks for the answer <a class="mention" href="/u/tom">@tom</a></p>

username: AtenaNguyen
cooked: <p>Did group sum release yet? Thanks</p>