Number of parameters when using a shared module?

Victor_Butoi: <p>Hello there! I’m experimenting with a network where I reuse several layers that are initialized in the init portion of my network, in my forward pass, so that I can tie the weights together. However, when I use a “parameter counter” like torchsummary, it shows me that I have an identical number of parameters to a net with a unique module for each layer.</p>
<p>My question is, is reusing a layer module enough to tie the weights? This to me sounds like a stupid question because how would backprop have additional parameters if an additional module doesn’t exist, but I was thinking maybe reusing the layer creates some kind of shadow layer that caches the parameters for each individual layer.</p>

ptrblck: <aside class="quote no-group" data-username="Victor_Butoi" data-post="1" data-topic="82887">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/victor_butoi/48/13438_2.png" class="avatar"> Victor_Butoi:</div>
<blockquote>
<p>My question is, is reusing a layer module enough to tie the weights? This to me sounds like a stupid question because how would backprop have additional parameters if an additional module doesn’t exist, but I was thinking maybe reusing the layer creates some kind of shadow layer that caches the parameters for each individual layer.</p>
</blockquote>
</aside>
<p>Your approach should be correct and I get the same output for this dummy model:</p>
<pre data-code-wrap="python"><code class="lang-python">class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc1 = nn.Linear(1, 1, bias=False)
        
    def forward(self, x):
        for _ in range(10):
            x = self.fc1(x)
        return x

model = MyModel()
x = torch.randn(1, 1)

summary(model, [(1, 1)])
&gt; ----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Linear-1                 [-1, 1, 1]               1
            Linear-2                 [-1, 1, 1]               1
            Linear-3                 [-1, 1, 1]               1
            Linear-4                 [-1, 1, 1]               1
            Linear-5                 [-1, 1, 1]               1
            Linear-6                 [-1, 1, 1]               1
            Linear-7                 [-1, 1, 1]               1
            Linear-8                 [-1, 1, 1]               1
            Linear-9                 [-1, 1, 1]               1
           Linear-10                 [-1, 1, 1]               1
================================================================
Total params: 10
Trainable params: 10
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.00
Params size (MB): 0.00
Estimated Total Size (MB): 0.00
</code></pre>
<p>so I assume <code>torchsummary</code> might not have implemented this featrure.</p>

wlp: <p>Have you solved it? I meet the same problem with you. If there are other parameter counter can calculate rightly?</p>