AttributeError: 'Sequential' object has no attribute 'weight'

username: ammalikwaterloo
cooked: <p>Hello,</p>
<p>I am using a model from</p>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/vision/blob/master/torchvision/models/vgg.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/vision/blob/master/torchvision/models/vgg.py" target="_blank" rel="nofollow noopener">pytorch/vision/blob/master/torchvision/models/vgg.py</a></h4>
<pre><code class="lang-py">import torch
import torch.nn as nn
from .utils import load_state_dict_from_url


__all__ = [
    'VGG', 'vgg11', 'vgg11_bn', 'vgg13', 'vgg13_bn', 'vgg16', 'vgg16_bn',
    'vgg19_bn', 'vgg19',
]


model_urls = {
    'vgg11': 'https://download.pytorch.org/models/vgg11-bbd30ac9.pth',
    'vgg13': 'https://download.pytorch.org/models/vgg13-c768596a.pth',
    'vgg16': 'https://download.pytorch.org/models/vgg16-397923af.pth',
    'vgg19': 'https://download.pytorch.org/models/vgg19-dcbb9e9d.pth',
    'vgg11_bn': 'https://download.pytorch.org/models/vgg11_bn-6002323d.pth',
    'vgg13_bn': 'https://download.pytorch.org/models/vgg13_bn-abd245e5.pth',
    'vgg16_bn': 'https://download.pytorch.org/models/vgg16_bn-6c64b313.pth',
    'vgg19_bn': 'https://download.pytorch.org/models/vgg19_bn-c79401a0.pth',
</code></pre>

  This file has been truncated. <a href="https://github.com/pytorch/vision/blob/master/torchvision/models/vgg.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>However when I use<br>
print(“weight[0,0] b4 average: %f” % model.classifier.weight[0,0].item())</p>
<p>I got the error.<br>
Thanks,</p>

username: ptrblck
cooked: <p><code>model.classifier</code> is defines as <code>nn.Sequential</code>, so you would have to index the module inside it, e.g.</p>
<pre><code class="lang-python">model.classifier[0].weight
</code></pre>

username: TO_Kocsis
cooked: <p>Using:</p>
<blockquote>
<p><span class="hashtag-raw">#load</span><br>
pretrained_dict  = torch.load(‘H:/workspace/pretrain/epoch_500_pretrain_resnet50.pth’)<br>
model_dict = model.state_dict()<br>
# 1. filter out unnecessary keys<br>
pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}<br>
# 2. overwrite entries in the existing state dict<br>
model_dict.update(pretrained_dict)<br>
# 3. load the new state dict<br>
model.load_state_dict(model_dict)<br>
# copy braches 0 → branches 1/2<br>
model.branches[1].weight.copy_(model.branches[0].weight)</p>
</blockquote>
<p>I get an error: AttributeError: ‘Sequential’ object has no attribute ‘weight’</p>
<p>Im not sure whats wrong?<br>
Here is an image of the model structure that might reveal the issue?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/6/164e395f6e2e3c415bbb6ceaf93acac744688e04.png" data-download-href="https://discuss.pytorch.org/uploads/default/164e395f6e2e3c415bbb6ceaf93acac744688e04" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/6/164e395f6e2e3c415bbb6ceaf93acac744688e04.png" alt="image" data-base62-sha1="3bk67xYnistUGkyfGWVfdQ8gwdu" width="427" height="500" data-dominant-color="48494C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">499×584 24 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Eta_C
cooked: <p>It seems that this module is <strong>a Sequential of Sequentials</strong>, try</p>
<pre><code class="lang-python">model.branches[0][1].weight
</code></pre>

username: TO_Kocsis
cooked: <blockquote>
<p>model.branches[0][1].weight.copy_(model.branches[0][0].weight)</p>
</blockquote>
<p>AttributeError: ‘AdaptiveAvgPool2d’ object has no attribute ‘weight’</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/b/8b5e59edbca5b82bceacda1e484299b0f679aef6.png" data-download-href="https://discuss.pytorch.org/uploads/default/8b5e59edbca5b82bceacda1e484299b0f679aef6" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/8/b/8b5e59edbca5b82bceacda1e484299b0f679aef6.png" alt="image" data-base62-sha1="jSUvVZlMsqQq3vCrwZqPehvr4km" width="379" height="500" data-dominant-color="47484B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">578×761 47.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>If I understand correctly, this adresses as follows:<br>
model → _modules:branches → _modules: 0 → _modules:1 = AAP2d</p>
<p>I tried to track down where the weights are in debugging:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/5/65da576aa7e05471cb8ff141a1ef75258cb2b227.png" data-download-href="https://discuss.pytorch.org/uploads/default/65da576aa7e05471cb8ff141a1ef75258cb2b227" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/5/65da576aa7e05471cb8ff141a1ef75258cb2b227_2_445x499.png" alt="image" data-base62-sha1="ex21aSGhSUgaJR3j6uJ1rNMIBMP" width="445" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/5/65da576aa7e05471cb8ff141a1ef75258cb2b227_2_445x499.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/5/65da576aa7e05471cb8ff141a1ef75258cb2b227_2_667x748.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/5/65da576aa7e05471cb8ff141a1ef75258cb2b227_2_890x998.png 2x" data-dominant-color="434547"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1043×1171 94.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>But I still cant find how to address it correctly, when to use the number indexing, or the . indexing?<br>
I tried for example:</p>
<blockquote>
<p>model.branches[1][0].conv1.weight   .copy_(  model.branches[0][0].conv1.weight  )</p>
</blockquote>
<p>AttributeError: ‘Sequential’ object has no attribute ‘conv1’</p>

username: Eta_C
cooked: <aside class="quote no-group" data-username="TO_Kocsis" data-post="5" data-topic="53855">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/58f4c7/48.png" class="avatar"> TO_Kocsis:</div>
<blockquote>
<p>But I still cant find how to address it correctly, when to use the number indexing, or the . indexing?<br>
I tried for example:</p>
</blockquote>
</aside>
<p>You don’t understand the key points.</p>
<ul>
<li>Use number indexing for <code>Sequential</code> and <code>ModuleList</code></li>
<li>Use key for <code>ModuleDict</code></li>
<li>Use .module_name otherwise</li>
</ul>
<p>For example, if I have a network like</p>
<pre data-code-wrap="python"><code class="lang-python">import torch.nn as nn

class SubNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = nn.Conv2d(2, 2, 2)
        self.relu = nn.ReLU(inplace=True)
    def forward(self, x):
        return x

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.module = nn.Sequential(
            nn.Sequential(
                nn.Conv2d(1, 1, 1),
                SubNet(),
            ),
            nn.Conv2d(3, 3, 3)
        )
        self.conv = nn.Conv2d(4, 4, 4)
    def forward(self, x):
        return x

net = Net()
print(net)
</code></pre>
<p>The output is</p>
<pre><code class="lang-auto">Net(
  (module): Sequential(
    (0): Sequential(
      (0): Conv2d(1, 1, kernel_size=(1, 1), stride=(1, 1))
      (1): SubNet(
        (conv): Conv2d(2, 2, kernel_size=(2, 2), stride=(1, 1))
        (relu): ReLU(inplace=True)
      )
    )
    (1): Conv2d(3, 3, kernel_size=(3, 3), stride=(1, 1))
  )
  (conv): Conv2d(4, 4, kernel_size=(4, 4), stride=(1, 1))
)
</code></pre>
<p>Then we have<br>
<code>net.module[0][0].weight</code> → <code>Conv2d(1, 1, 1)</code>'s weight<br>
<code>net.module[0][1].conv.weight</code> → <code>Conv2d(2, 2, 2)</code>'s weight<br>
<code>net.module[1].weight</code> → <code>Conv2d(3, 3, 3)</code>'s weight<br>
<code>net.conv.weight</code> → <code>Conv2d(4, 4, 4)</code>'s weight</p>

username: TO_Kocsis
cooked: <p>Thanks you example was very informative.<br>
I had problems understanding and seeing clearly from pycharm debug, but printing the net makes it much more clear.</p>

username: Kamil_Adamczewski
cooked: <p>What would we do without you.</p>

username: Fathima
cooked: <p>I am trying to initialise the weight but I get this error and I followed the thread but I don’t understand how I could fix in my code. I have 2 con layers and 2 fc layers</p>
<p>this is my architecture</p>
<pre><code class="lang-auto">#original code 
class ConvNet(nn.Module):
    def __init__(self):
        super(ConvNet, self).__init__()
        
        #refer the article ... 
        self.conv1 = nn.Sequential(
            nn.Conv1d(1, 32, kernel_size=2, stride=1),
            nn.Dropout(0.5),
            nn.ReLU())
        
        self.conv2 = nn.Sequential(
            nn.Conv1d(32,32, kernel_size=2, stride=1),
            nn.Dropout(0.5),
            nn.ReLU(),
            nn.MaxPool1d(2,stride=3))
       
        self.conv3 = nn.Sequential(
            nn.Conv1d(32, 32, kernel_size=2, stride=1),
            nn.Dropout(0.5),
            nn.ReLU())
        
        
        #fully connected layers
        self.fc1 = nn.Linear(32*47,32)
        #self.fc2 = nn.Linear(32,1)
        self.fc2 = nn.Linear(32,1)
        #self.activation = nn.Softmax()
        #self.activation = nn.Sigmoid()
        
         # Initialization
        nn.init.zeros_(self.conv1.weight)
        nn.init.zeros_(self.conv2.bias)
    
    
    def forward(self, x):
        # input x : 
        #expected conv1d input = minibatch_size * num_channel * width
        batch_size=x.size(0)
        y = self.conv1(x.view(batch_size,1,-1))
        y = self.conv2(y)
        y = self.conv3(y)
        
        #print(y.size())
        batch_size= y.size(0)
        y = y.flatten(start_dim=1)
        #print(y.size())
        y = self.fc1(y.view(y.size(0), -1))
        #y = self.fc1(y.view(batch_size,1,-1))
        y = self.fc2(y.view(batch_size,1,-1))

        return y  
</code></pre>
<pre><code class="lang-auto">
</code></pre>
<pre><code class="lang-auto">AttributeError                            Traceback (most recent call last)
&lt;ipython-input-66-063f0bb327dd&gt; in &lt;module&gt;
      1 #check the output size
----&gt; 2 Model = ConvNet()
      3 X = (torch.randn(32, 1, 145))
      4 output=Model(X)
      5 #print(output.size())

&lt;ipython-input-65-154d3f618194&gt; in __init__(self)
     30 
     31          # Initialization
---&gt; 32         nn.init.zeros_(self.conv1.weight)
     33         nn.init.zeros_(self.conv2.bias)
     34 

~\anaconda3\lib\site-packages\torch\nn\modules\module.py in __getattr__(self, name)
   1183             if name in modules:
   1184                 return modules[name]
-&gt; 1185         raise AttributeError("'{}' object has no attribute '{}'".format(
   1186             type(self).__name__, name))
   1187 

AttributeError: 'Sequential' object has no attribute 'weight'```
</code></pre>

username: ptrblck
cooked: <p>This line of code:</p>
<pre><code class="lang-python">nn.init.zeros_(self.conv1.weight)
</code></pre>
<p>won’t work, since <code>self.conv1</code> is an <code>nn.Sequential</code> block and thus does not have any parameters.<br>
You would need to index the actual conv layer via <code>self.conv1[0]</code> and it should work:</p>
<pre><code class="lang-python">nn.init.zeros_(self.conv1[0].weight)
</code></pre>