TypeError: ‘Conv2d’ object is not iterable

Niki: <p>I want to use Xavier initialization scheme, for initializing weights, for Resnet in <a href="http://www.pabloruizruiz10.com/resources/CNNs/ResNet-PyTorch.html" rel="nofollow noopener">http://www.pabloruizruiz10.com/resources/CNNs/ResNet-PyTorch.html </a></p>
<p>So I added this in the network:</p>
<pre><code class="lang-auto">def xavier_init(ms):
       for m in ms :
            if isinstance(m, nn.Linear) or isinstance(m, nn.Conv2d):
                nn.init.xavier_uniform(m.weight,gain=nn.init.calculate_gain('relu'))
                m.bias.data.zero_()
</code></pre>
<p>and then calling this in the main code:</p>
<pre><code class="lang-auto">net.weight_init()
</code></pre>
<p>It is giving this error:</p>
<pre><code class="lang-auto">TypeError: ‘Conv2d’ object is not iterable
</code></pre>
<p>how can I solve this error?</p>

Upgrade_Yourself: <p>Hi I don’t know if this’ll help but I found that in order to do xavier_uniform one should do:</p>
<blockquote>
<p>linear1=torch.nn.Linear(N_FEATURES, hiddenLayerSize, bias=True)<br>
torch.nn.init.xavier_uniform(linear1.weight)</p>
</blockquote>
<p>instead of:</p>
<blockquote>
<p>nn.init.xavier_uniform(m.weight,gain=nn.init.calculate_gain(‘relu’))</p>
</blockquote>
<p>It’s discussed here: <a href="https://discuss.pytorch.org/t/how-to-initialize-the-conv-layers-with-xavier-weights-initialization/8419" class="inline-onebox">How to initialize the conv layers with xavier weights initialization?</a></p>

ptrblck: <p>I’m not sure, how you are calling <code>xavier_init</code>, but if <code>ms</code> is a module, the for loop will throw this error.<br>
Your current code (<code>net.weight_init()</code>) is not passing any arguments, and <code>xavier_init</code> is not defined as a class function.</p>

Niki: <p>Thank you for your reply.<br>
I’m calling <code>xavier_init</code> as follows:</p>
<pre><code class="lang-auto">def xavier_init(ms):
    for m in ms :
        if isinstance(m, nn.Linear) or isinstance(m, nn.Conv2d):
            nn.init.xavier_uniform(m.weight,gain=nn.init.calculate_gain('relu'))
            m.bias.data.zero_()


class ResNet(nn.Module):
    def __init__(self, block, num_blocks, num_classes=10):
        super(ResNet, self).__init__()
        self.in_planes = 64
        
        
        self.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1, bias=False)
        self.bn1 = nn.BatchNorm2d(64)
        self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1)
        self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2)
        self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2)
        self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2)
        self.linear = nn.Linear(512*block.expansion, num_classes)

      
    
    
    def _make_layer(self, block, planes, num_blocks, stride):
        strides = [stride] + [1]*(num_blocks-1)
        layers = []
        for stride in strides:
            layers.append(block(self.in_planes, planes, stride))
            self.in_planes = planes * block.expansion
        return nn.Sequential(*layers)


    def forward(self, x, lin=0, lout=5):
        out = x
        if lin &lt; 1 and lout &gt; -1:
            out = self.conv1(out)
            out = self.bn1(out)
            out = F.relu(out)
        if lin &lt; 2 and lout &gt; 0:
            out = self.layer1(out)
        if lin &lt; 3 and lout &gt; 1:
            out = self.layer2(out)
        if lin &lt; 4 and lout &gt; 2:
            out = self.layer3(out)
        if lin &lt; 5 and lout &gt; 3:
            out = self.layer4(out)
        if lout &gt; 4:
            out = F.avg_pool2d(out, 4)
            out = out.view(out.size(0), -1)
            out = self.linear(out)  
        return out   
    
   
    def weight_init(self):
        for m in self._modules:
            xavier_init(self._modules[m])
</code></pre>
<p>and in the main code, I’m calling resnet and weight_init as follows:</p>
<pre><code class="lang-auto">net = models.__dict__[args.model](num_classes)
net.weight_init() 
</code></pre>
<p>What should I add as an argument for <code>(net.weight_init())</code>?</p>

ptrblck: <p>I would recommend to call <code>model.apply</code> with your weight init method and remove the loop inside <code>xavier_init</code>:</p>
<pre><code class="lang-python">def xavier_init(ms):
    if isinstance(m, nn.Linear) or isinstance(m, nn.Conv2d):
        nn.init.xavier_uniform(m.weight,gain=nn.init.calculate_gain('relu'))
        m.bias.data.zero_()

model.apply(xavier_init)
</code></pre>
<p>Also, your instantiation of <code>net</code> looks as if you are importing the model directly from <code>torchvision.models</code> instead of your custom <code>ResNet</code> definition.</p>

Niki: <aside class="quote no-group" data-username="ptrblck" data-post="5" data-topic="61958">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>model.apply(xavier_init)</p>
</blockquote>
</aside>
<p>Actually, I am using <code>argparse.</code> for importing my custom <code>Resnet</code>. If I understood, <code>model.apply</code> or (I think in my case <code>net.apply</code> should be in the main code where I have <code>net = models.__dict__[args.model](num_classes)</code>, while <code>xavier_init</code> function has to be in the model.py (where the model is, I mean in a separate .py file). My question is, I have to put the model in the main .py file for using <code>model.apply</code> or there is a way that I keep separate .py files for main and model and still using <code>model.apply</code>?</p>

ptrblck: <p>You could call <code>self.apply</code> from some internal method (e.g. <code>__init__</code>) calling <code>self.xavier_init</code>.</p>

Niki: <p>Thank you for your reply.</p>
<p>I used <code>model.apply</code>, it is giving this error:</p>
<pre><code class="lang-auto">File "...", line 308, in &lt;module&gt;
    net.apply(xavier_init)      
  File "...", line 293, in apply
    module.apply(fn)
  File "...", line 293, in apply
    module.apply(fn)
  File "...", line 294, in apply
    fn(self)
  File "...", line 191, in xavier_init
    m.bias.data.zero_()
AttributeError: 'NoneType' object has no attribute 'data'
</code></pre>

ptrblck: <p>Probably some layers do not have a <code>bias</code> parameters, so you should add a condition to the weight init function.</p>

Niki: <p>It is solved. Thank you for your help, <a class="mention" href="/u/ptrblck">@ptrblck</a>.</p>

leilei: <p><a class="mention" href="/u/niki">@Niki</a>  Could you please show how you solved the problem? thanks</p>