Convolution operator with groups for taking each convolved result

username: Ja-Keoung_Koo
cooked: <p>I recently saw that there is an option of <em>groups</em> for Conv2d. But I don’t understand how to use it.</p>
<p>In my case, I want to take each convolution output.</p>
<p>In traditional convolution networks, if we assume that input channel (R,G,B) is 3 and output channel (J1, J2) is 2,</p>
<pre><code class="lang-auto">J1 = F1 * R + F2 * G + F3 * B
J1 = F4 * R + F5 * G + F6 * B
</code></pre>
<p>where F1,F2,… denotes filters (say 3x3 filters) and * denotes convolution operator.</p>
<p>In my case, I want to manipulate the each convolved result. So I want to take the followings, not output images:</p>
<pre><code class="lang-auto">F1 * R, F2 * G, F3 * B,
F4 * R, F5 * G, F6 * B
</code></pre>
<p>Is this possible by using <code>group</code> option? Thanks.</p>

username: apaszke
cooked: <p>Yes, <code>nn.Conv2d(3, 6, groups=6)</code> will do the operation you want. Each output channel will be a result of a convolution over a single input channel.</p>

username: Ja-Keoung_Koo
cooked: <p>Thanks you for kind reply!</p>
<p>Minor question: Is this right that <code>nn.Conv2d(3, 6, groups=3)</code>? This is because I saw the following in <code>ConvNd</code> module,</p>
<pre><code>       self.weight = Parameter(torch.Tensor(
            out_channels, in_channels // groups, *kernel_size))</code></pre>

username: apaszke
cooked: <p>Yes, you’re right. It should be <code>groups=3</code>.</p>

username: siki
cooked: <p>Thanks to this Q&amp;A, it helps me to understand groups operation.</p>

username: juliohm
cooked: <p><a class="mention" href="/u/apaszke">@apaszke</a>, thank you for confirming the feature. As a more general question, when is it interesting to convolve channels separately like this as opposed to the default <code>groups=1</code>?</p>

username: hinken
cooked: <p>In a backprop context, you’d use groupwise convolution to either make the model splittable into more than one GPU (since the groups won’t have to “cross” data), or to force the training to split channels into different groupings of features (you can google on this, there are some nice explanations available). In a more general context, you could find any number of applications, lets say for simplicity that you just want to blur each R, G and B channel separately with a blur kernel and want RGB output (and you definitely don’t want to blur R with G etc.).</p>

username: Yohan_Sumathipala
cooked: <p>Quick follow-up: What is the order of the channels in the output image?</p>
<p>Assuming input is RGB image and output is 6 channels (as in OP) is the order:</p>
<p>Guess 1:<br>
J1 = F1 * R<br>
J2 = F2 * G<br>
J3 = F3 * B<br>
J4 = F4 * R<br>
J5 = F5 * G<br>
J6 = F6 * B</p>
<p>Or<br>
Guess 2:<br>
J1 = F1 * R<br>
J2 = F2 * R<br>
J3 = F3 * G<br>
J4 = F4 * G<br>
J5 = F5 * B<br>
J6 = F6 * B</p>

username: ptrblck
cooked: <p>Let’s create a small example and have a look at the order:</p>
<pre><code class="lang-auto"># Set R=100, G=200, B=300
x = torch.FloatTensor([100, 200, 300]).view(1, -1, 1, 1)
x = Variable(x)

conv = nn.Conv2d(in_channels=3,
                 out_channels=6,
                 kernel_size=1,
                 stride=1,
                 padding=0,
                 groups=3,
                 bias=False)

# Set Conv weight to [0, 1, 2, 3, 4 ,5]
conv.weight.data = torch.arange(6).view(-1, 1, 1, 1)
output = conv(x)
print(output)
&gt; Variable containing:
(0 ,0 ,.,.) = 
     0

(0 ,1 ,.,.) = 
   100

(0 ,2 ,.,.) = 
   400

(0 ,3 ,.,.) = 
   600

(0 ,4 ,.,.) = 
  1200

(0 ,5 ,.,.) = 
  1500
[torch.FloatTensor of size (1,6,1,1)]
</code></pre>
<p>The only way to get this result is the second guess.</p>
<p>J1 = F1 * R = 0 * 100 = 0<br>
J2 = F2 * R = 1 = 100 = 100<br>
J3 = F3 * G = 2 * 200 = 400<br>
J4 = F4 * G = 3 * 200 = 600<br>
J5 = F5 * B = 4 * 300 = 1200<br>
J6 = F6 * B = 5 * 300 = 1500</p>

username: Yohan_Sumathipala
cooked: <p>Thanks so much, this is super helpful!</p>

username: ixez
cooked: <p>I try the following codes:</p>
<pre><code class="lang-auto">import torch
from torch import nn

# Set (RGB)*2 = (0, 100, 200), (300, 400, 500)
x = (torch.arange(6, dtype=torch.double)*100).view(1, -1, 1, 1)

conv = nn.Conv2d(in_channels=6,
                 out_channels=2,
                 kernel_size=1,
                 stride=1,
                 padding=0,
                 groups=2,
                 bias=False)

# Set Conv weight to 1
conv.weight.data = torch.ones(6, dtype=torch.double).view(2, 3, 1, 1)

output = conv(x)

print(output)
</code></pre>
<p>output = (500, 5000). The result seems to be your Guess 2, RR GG BB.</p>