Fine tuning resnet: 'DataParallel' object has no attribute 'fc'

username: yang_yang1
cooked: <p>When I tried to fine tuning my resnet module, and run the following code:</p>
<p>ignored_params = list(map(id, model.fc.parameters()))<br>
base_params = filter(lambda p: id§ not in ignored_params,<br>
model.parameters())</p>
<p>optimizer = optim.Adam([<br>
{‘params’: base_params},<br>
{‘params’: model.fc.parameters(), ‘lr’: opt.lr}<br>
], lr=opt.lr*0.1, momentum=0.9)</p>
<p>it raise a error:<br>
AttributeError                            Traceback (most recent call last)<br>
 in ()<br>
----&gt; 1 ignored_params = list(map(id, model.module.fc.parameters()))<br>
2 base_params = filter(lambda p: id§ not in ignored_params,<br>
3                      model.parameters())<br>
4<br>
5 optimizer = optim.Adam([</p>
<p>/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in <strong>getattr</strong>(self, name)<br>
364                 return modules[name]<br>
365         raise AttributeError("’{}’ object has no attribute ‘{}’".format(<br>
–&gt; 366             type(self).<strong>name</strong>, name))<br>
367<br>
368     def <strong>setattr</strong>(self, name, value):</p>
<p>AttributeError: ‘DataParallel’ object has no attribute ‘fc’</p>
<p>How to fix it? thanks.<br>
and here is my model:</p>
<p>DataParallel(<br>
(module): DataParallel(<br>
(module): DataParallel(<br>
(module): DataParallel(<br>
(module): Net(<br>
(conv1): Conv2d (3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)<br>
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(maxpool): MaxPool2d(kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), dilation=(1, 1))<br>
(layer1): Sequential(<br>
(0): BasicBlock(<br>
(conv1): Conv2d (64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
(1): BasicBlock(<br>
(conv1): Conv2d (64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
(2): BasicBlock(<br>
(conv1): Conv2d (64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
)<br>
(layer2): Sequential(<br>
(0): BasicBlock(<br>
(conv1): Conv2d (64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True)<br>
(downsample): Sequential(<br>
(0): Conv2d (64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)<br>
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
)<br>
(1): BasicBlock(<br>
(conv1): Conv2d (128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
(2): BasicBlock(<br>
(conv1): Conv2d (128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
(3): BasicBlock(<br>
(conv1): Conv2d (128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
)<br>
(layer3): Sequential(<br>
(0): BasicBlock(<br>
(conv1): Conv2d (128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)<br>
(downsample): Sequential(<br>
(0): Conv2d (128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)<br>
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
)<br>
(1): BasicBlock(<br>
(conv1): Conv2d (256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
(2): BasicBlock(<br>
(conv1): Conv2d (256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
(3): BasicBlock(<br>
(conv1): Conv2d (256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
(4): BasicBlock(<br>
(conv1): Conv2d (256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
(5): BasicBlock(<br>
(conv1): Conv2d (256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
)<br>
(layer4): Sequential(<br>
(0): BasicBlock(<br>
(conv1): Conv2d (256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True)<br>
(downsample): Sequential(<br>
(0): Conv2d (256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)<br>
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
)<br>
(1): BasicBlock(<br>
(conv1): Conv2d (512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
(2): BasicBlock(<br>
(conv1): Conv2d (512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True)<br>
(relu): ReLU(inplace)<br>
(conv2): Conv2d (512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True)<br>
)<br>
)<br>
(avgpool): AdaptiveAvgPool2d(output_size=1)<br>
(fc): Linear(in_features=512, out_features=10)<br>
)<br>
)<br>
)<br>
)<br>
)</p>

username: ptrblck
cooked: <p>When using <code>DataParallel</code> your <code>nn.Module</code> will be in <code>.module</code>:</p>
<pre><code class="lang-auto">model = Net()
model.fc

model_parallel = nn.DataParallel(model)
model_parallel.module.fc
</code></pre>

username: yang_yang1
cooked: <p>Thanks for your reply, but after initialized with model = Net () , how to load my pretrained weights? my pretrained model is saved by torch.save()</p>

username: zakaria_laskar
cooked: <p>model.load_state_dict(torch.load(’/path/to/model.pth.tar’))</p>

username: Ivamcoder
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>Assume  that I have another attribute from a second model which is called in attribute of first model,<br>
How should I do that,<br>
example</p>
<pre><code class="lang-auto">model_parallel.module.inference(inputs_var)
</code></pre>
<p>and in inference attribute I have</p>
<pre><code class="lang-auto">    def inference(self, img):
        feat = self.feature_extractor(img)
        x = self.second_net.inference(feat)
        return x
</code></pre>
<p>I get the error that <code>second net</code> does not have <code>inference</code> attribute<br>
What should I do ?</p>

username: ptrblck
cooked: <p>Could you post the complete model definition so that we can have a look?</p>