Is it a silent error to use layers added to self in a nn.sequential also assigned to self?

nicholdav: <p>Hi,</p>
<p>Sorry if this has answered before, it’s a bit hard to search for.</p>
<p>If I write a class where I instantiate <code>torch.nn</code> modules inside <code>__init__</code> and make them attributes of <code>self</code>, then use those same layers in a <code>Sequential</code> model, is that a silent error?</p>
<p>Like this, I mean:</p>
<pre><code class="lang-auto">class MyModule(nn.Module):
    def __init__(self):
        self.conv1 = nn.Conv2d(3, 64)
        self.conv2 = nn.Conv2d(64, 64)

        self.net = nn.Sequential(self.conv1, self.conv2)

    def forward(self, x):
        return self.net(x)
</code></pre>
<p>Would parameters get updated twice somehow, or something like that?</p>
<p>I realized I had done this without thinking about it when <code>torchinfo</code> showed me double the number of parameters for a layer – it counted once for the <code>self.conv</code> attribute and again for the parameters from the same module instance inside the <code>Sequential</code>.</p>
<p>Guessing the answer is “no” since this could result in some unpleasant surprises but just want to be sure.</p>
<p>Thanks!</p>
<p>edit: this looks close to what I’m asking</p><aside class="quote quote-modified" data-post="1" data-topic="98689">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/9e8a1a/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/avoid-parameter-sharing-when-calling-modules-multiple-times/98689">Avoid parameter sharing when calling modules multiple times!</a> 
  </div>
  <blockquote>
    Hi everyone, 
I am new at Pytorch, I am trying to generate a classifier net that contains a mix of plain and residual modules. However, I got an error of: 

ValueError: TracedModules don’t support parameter sharing between modules 

when trying to visualize it by xtensorboard, can you please help to understand where are the shared parameters, and how to avoid that! 
my net: 
import torch
import torch.nn as nn
import torch.nn.functional as F

# DCR block
class DCRBlock(nn.Module):

    def __init…
  </blockquote>
</aside>
<p>
Not sure if that post is saying that what I’ve done above is a problem though.</p>

ptrblck: <p>It shouldn’t be a problem, since the <code>nn.Sequential</code> container will share the same parameters with the original layers.<br>
However, TorchScript (and maybe other backends) might not accept models with shared parameters and could raise an error as described in the linked post.<br>
The <code>torchinfo</code> output seems to be misleading, since you are not increasing the number of parameters:</p>
<pre><code class="lang-python">=================================================================
Layer (type:depth-idx)                   Param #
=================================================================
MyModule                                 --
├─Conv2d: 1-1                            1,792
├─Conv2d: 1-2                            36,928
├─Sequential: 1-3                        38,720
│    └─Conv2d: 2-1                       (recursive)
│    └─Conv2d: 2-2                       (recursive)
=================================================================
Total params: 77,440
Trainable params: 77,440
Non-trainable params: 0
=================================================================
</code></pre>
<p>and you can also verify it e.g. my manipulating a parameter inplace:</p>
<pre><code class="lang-python">class MyModule(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 64, 3)
        self.conv2 = nn.Conv2d(64, 64, 3)

        self.net = nn.Sequential(self.conv1, self.conv2)

    def forward(self, x):
        return self.net(x)
    
model = MyModule()
with torch.no_grad():
    model.conv1.weight[0, 0].fill_(1.)

model.conv1.weight[0, 0]
# tensor([[1., 1., 1.],
#         [1., 1., 1.],
#         [1., 1., 1.]], grad_fn=&lt;SelectBackward0&gt;)
model.net[0].weight[0, 0]
# tensor([[1., 1., 1.],
#         [1., 1., 1.],
#         [1., 1., 1.]], grad_fn=&lt;SelectBackward0&gt;)
</code></pre>

nicholdav: <p>Got it, good to know this could be an issue for TorchScript down the road. I rewrote to avoid that.</p>
<p>Agreed the <code>torchinfo.summary</code> output is a bit misleading, although I don’t envy them the job of trying to parse eager graphs.</p>
<p>Thank you <a class="mention" href="/u/ptrblck">@ptrblck</a>, you’re the undisputed forum king! Much appreciated</p>