Pre-trained model and finetuning

username: mab73
cooked: <p>I am using an off-shelf action recognition system(TSN) that I pretrained on 195 classes. Now I want to finetune it on another dataset that has 25 classes. When I load my pretrained model I do NOT get an error of a mismatch between classes even though I have not changed the last layer. Did the system adopt to the new last layer byitself or I am doing something wrong because I am getting very low accuracy?</p>

username: ptrblck
cooked: <p>Most likely your model didn’t adapt itself to the new number of classes. Since the new dataset contains less classes than the one the model was pre-trained on, you won’t get an out of index error.<br>
However, you are “wasting” model capacity, since the majority of the output neurons aren’t used (neurons corresponding to class26 to class195).<br>
I would recommend to change the last layer corresponding to the new number of classes as I would guess you’ll see a performance boost.</p>

username: mab73
cooked: <p>Thanks your very much for your fast reply.Do you think the accuracy will increase?</p>

username: ptrblck
cooked: <p>That would be my guess, but I can’t promise anything. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=6" title=":wink:" class="emoji" alt=":wink:"></p>

username: mab73
cooked: <p>So I added this line  model.fc = torch.nn.Linear(195,25) to change the last layer from 195 to 25, but I am getting this error<br>
raise KeyError(‘missing keys in state_dict: “{}”’.format(missing))<br>
KeyError: ‘missing keys in state_dict: “set([‘fc.weight’, ‘fc.bias’])”’</p>

username: ptrblck
cooked: <p>It seems you are using a pre-trained model. If that’s the case, you should load the <code>state_dict</code> with the old architecture and change the last layer afterwards. This will make sure that all parameters will be found.</p>

username: mab73
cooked: <p>Thank you very much for answering my questions. Now that I froze all the layers except the last fully connected layer and after changing my optimizer from</p>
<blockquote>
<pre><code>for group in policies:
    print(('group: {} has {} params, lr_mult: {}, decay_mult: {}'.format(
        group['name'], len(group['params']), group['lr_mult'], group['decay_mult'])))

optimizer = torch.optim.SGD(policies,
                           args.lr,
                           momentum=args.momentum,
                           weight_decay=args.weight_decay)
</code></pre>
</blockquote>
<p>to</p>
<blockquote>
<p>optimizer = torch.optim.SGD(filter(lambda p: p.requires_grad, model.parameters()), args.lr )</p>
</blockquote>
<p>I am getting this error</p>
<blockquote>
<p>File “main.py”, line 307, in adjust_learning_rate<br>
param_group[‘lr’] = lr * param_group[‘lr_mult’]<br>
KeyError: ‘lr_mult’</p>
</blockquote>
<p>Any idea how I can fix it, thank you again.</p>

username: ptrblck
cooked: <p>Could you explain a bit, how you’ve created <code>policies</code>?<br>
Is it a custom <code>dict</code>?</p>

username: mab73
cooked: <p>I did not create it. I am using an off shelf action recognition system TSN pytorch. They defined it as</p>
<blockquote>
<pre><code>policies = model.get_optim_policies()
</code></pre>
</blockquote>

username: ptrblck
cooked: <p>I’m not familiar with TSN, but apparently <code>policies</code> does not contain the key <code>'lr_mult'</code>.<br>
Could you check the repo and see if your usage is correct?<br>
If you can’t figure out the problem, I think the best approach would be to create an issue in the TSN repo.</p>

username: mab73
cooked: <p>before doing my edits, the code worked fine. But when I added my own optimizer it gave me this error.</p>

username: ptrblck
cooked: <p>I guess you are using <a href="https://github.com/yjxiong/tsn-pytorch" rel="nofollow noopener">this repo</a>.<br>
In your code you are filtering out all parameters which do not require gradients, thus you are probably breaking the intended usage of the code.<br>
Have a look at <a href="https://github.com/yjxiong/tsn-pytorch/blob/6b51fc28e87d61070b26d7454e5b169042fc851f/models.py#L177" rel="nofollow noopener">these lines of code</a>.<br>
I think if you create your own <code>policies</code> dict using this code for your filtered parameters, it should work again.</p>

username: Yang_Zheng
cooked: <p>Have you finished the problem? i would be appreciated it if you could share with me.</p>