Sharing weights between Conv3d groups

jfolz: <p>Hey guys,</p>
<p>The documentation for the <code>Conv3d</code> module states that inputs and output can be grouped together, each group with its own set of weights:</p>
<blockquote>
<p><strong>groups</strong> - controls the connections between inputs and outputs. in_channels and out_channels must both be divisible by groups.<br>
At groups=1, all inputs are convolved to all outputs.<br>
At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, <strong>and producing half the output channels</strong>, and both subsequently concatenated. At groups=<code>in_channels</code>, <strong>each input channel is convolved with its own set of filters</strong> (of size out_channels // in_channels).</p>
</blockquote>
<p>I did a small experiment to confirm this works as I expect it to (and btw, awesome framework where you can just whip something like this together in no time at all <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20">):</p>
<pre><code>import torch
from torch.autograd import Variable
from torch.nn import Conv3d
v = Variable(torch.randn((1, 20, 10, 10)))
v[:, 10:] = 0
c = Conv3d(20, 40, (3, 3), groups=2, bias=False)
print(c.weight.size())
r = c(v)
print(r[:, :20].sum())
print(r[:, 20:].sum())
</code></pre>
<p>Result:</p>
<pre><code>(40L, 10L, 3L, 3L)
Variable containing:
 14.2016
[torch.FloatTensor of size 1]

Variable containing:
 0
[torch.FloatTensor of size 1]
</code></pre>
<p>So the input is indeed treated as two separate groups of inputs, each corresponding to a proportionate number of outputs, and (judging from the number of parameters) each group has its own set of weights.</p>
<p>This is, however, not really what I want to do here. I would like all groups to share the same set of weights (i.e., a 20x10x3x3 tensor in the example).</p>
<p>I first tried concatenating several output of a smaller <code>Conv2d</code> layer. It works as expected, but it’s considerably slower (because operations have to be done sequentially?). 10% on my potato-testing-card and 40+% with multiple fast GPUs.<br>
I then tried several combinations of 3d kernels with some <code>unsqueeze</code> and <code>view</code> operations for good measure, but they were just as slow and didn’t produce the result I wanted.</p>
<p>Does anyone have an idea on how to share weights between groups?<br>
For now I’m fine with separate weights (it works quite well), but I would really like to test a theory.<br>
However, blocking resources for several days with an architecture I know is inefficient is not something I’m comfortable with <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

Luis_Salgueiro: <p>Hello guys,</p>
<p>I’m newbie in pytorch and also interested in the shared weights topic. If we use groups==in_channels, we know that each input channel would have its own set of filters, but different, so I did a test to clarify this concept.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/c/6c7c6df7508514e1bc91d2d66281155d359fafed.png" data-download-href="https://discuss.pytorch.org/uploads/default/6c7c6df7508514e1bc91d2d66281155d359fafed" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/c/6c7c6df7508514e1bc91d2d66281155d359fafed.png" alt="image" data-base62-sha1="ftI9FhsycaHvTbBg9l1Ktw8VrDn" width="666" height="499" data-dominant-color="1A1B1C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1055×792 43.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>It would be nice to have this feature of shared_by_group or something in future release.</p>
<p>How did you solved <a class="mention" href="/u/jfolz">@jfolz</a>?</p>

Ja-Keoung_Koo: <p>I’m also interested in this… Is there any news?</p>