Missing weights in model.parameters

username: CedricLy
cooked: <p>Happy New Year community,</p>
<p>I stumbled across a strange phenomena.<br>
I define 3 Sequentials in a model and when I go for model.parameters, I expect 6 parameters.<br>
One weight and one bias parameter for each sequential.</p>
<p>Instead I get only the first sequentials.<br>
Isn’t this problematic? The optimizer will later get all the model.parameters, but if 2 Sequentials are missing, they will not get trained as a consequence right?</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

class model(nn.Module):
    
    def __init__(self):
        super(model,self).__init__()
        sequence_generated_var = [nn.Linear(5, 1)]
        self.gen_energy = nn.Sequential(*sequence_generated_var)
        self.gen_theta = nn.Sequential(*sequence_generated_var)
        self.gen_phi = nn.Sequential(*sequence_generated_var)
    def forward(self,x):
        return
   
print([x[0] for x in m.named_parameters()])#outputs  ['gen_energy.0.weight', 'gen_energy.0.bias']
</code></pre>
<p>I guess there is a good explanation for the case and I hope someone can enlighten me.</p>
<p>Thanks in advance.</p>

username: JuanFMontesinos
cooked: <p>hmmm Can you try to instantiate the linear model once per definition?<br>
I wonder if it can detect they all are pointing to the same object</p>

username: CedricLy
cooked: <aside class="quote no-group" data-username="JuanFMontesinos" data-post="2" data-topic="108011">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juanfmontesinos/48/76115_2.png" class="avatar"> JuanFMontesinos:</div>
<blockquote>
<p>antiate the linear model once per defi</p>
</blockquote>
</aside>
<p>Well i get a response if I go model.gen_theta or model.gen_energy.</p>
<p>They even have different weights.</p>

username: JuanFMontesinos
cooked: <p>Hmm<br>
The modules should be registered at<br>
<code>model._modules</code><br>
So you can check the keys at<br>
<code>model._modules.keys()</code><br>
Can you post a reproducible script?<br>
I think you may be facing a silly error.</p>

username: CedricLy
cooked: <pre><code class="lang-auto">import torch
import torch.nn as nn

class model(nn.Module):
    
    def __init__(self):
        super(model,self).__init__()
        sequence_generated_var = [nn.Linear(5, 1)]
        self.gen_energy = nn.Sequential(*sequence_generated_var)
        self.gen_theta = nn.Sequential(*sequence_generated_var)
        self.gen_phi = nn.Sequential(*sequence_generated_var)
    def forward(self,x):
        return
  
m = model()
print([x[0] for x in m.named_parameters()])#outputs  ['gen_energy.0.weight', 'gen_energy.0.bias']
m._modules.keys() # outputs  odict_keys(['gen_energy', 'gen_theta', 'gen_phi'])
</code></pre>
<p>This is my minimal example.</p>

username: JuanFMontesinos
cooked: <p>Hmm I don’t know why but for some reason instantiating from the same module causes this error. If you create one instance per nn.Sequential it does work.</p>
<p>Internally the id of</p>
<pre><code class="lang-auto">id(self._modules['gen_energy']._modules['0'].weight)
</code></pre>
<p>is the same than</p>
<pre><code class="lang-auto">id(self._modules['gen_theta']._modules['0'].weight)
</code></pre>
<p>Soo it’s somehow recognizing that when generating the params.</p>
<p>If you want to do that you need to deepcopy the nn.Linear</p>

username: CedricLy
cooked: <aside class="quote no-group" data-username="JuanFMontesinos" data-post="6" data-topic="108011">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juanfmontesinos/48/76115_2.png" class="avatar"> JuanFMontesinos:</div>
<blockquote>
<p>o that you need to deepcopy the nn.Linea</p>
</blockquote>
</aside>
<p>Ok I suspected something like that.<br>
Is this intentional? I can’t think of a use case. Maybe something with recurrent layers?</p>

username: JuanFMontesinos
cooked: <p>Well, I don’t know at which point of the code these happens but makes sense since gradients are tracked at a tensor level. Sooo when you call the iterable the parameters are returned only once (you don’t want several backprop updates).</p>
<p>If you look at pieces of code which uses an already-instantiated layer to create several ones deepcopy is always used.</p>