Load weights from trained models for intialization

Rosa: <p>Hi,<br>
I used a resnet50 model(pretrained=True) for training.  I saved the best model in training function like:</p>
<pre><code class="lang-auto">model.load_state_dict(best_model_wts)
return model
</code></pre>
<p>then i called my training function:</p>
<pre><code class="lang-auto">trained_model = training_func(.....)
torch.save(trained_model, 'trained.pth')
</code></pre>
<p>Now I want to train again using the weights of my trained model.  So what I did is:</p>
<pre><code class="lang-auto">pretrained_weights = torch.load('trained.pth'')
model = resnet50(pretrained=False)
model.load_state_dict(pretrained_weights)
</code></pre>
<p>But it throws error:</p>
<pre><code class="lang-auto">model.load_state_dict(pretrained_weights)
  File "/home/user/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 751, in load_state_dict
    state_dict = state_dict.copy()
  File "/home/user/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 539, in __getattr__
    type(self).__name__, name))
AttributeError: 'ResNet' object has no attribute 'copy'
</code></pre>
<p>What I am doing wrong, can you please tell me ? I want to use my trained model’s weights to initialize my new training, same model as before resnet50.</p>

ilyes: <p>This should work !</p>
<pre><code class="lang-auto">trained_model = training_func(.....)
torch.save(trained_model.state_dict(), 'trained.pth')
</code></pre>

benoriol: <p>There are two ways of saving and loading models in Pytorch. You can either save/load the whole python class, architecture, weights or only the weights.</p>
<p>It is explained <a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html" rel="nofollow noopener">here</a></p>
<p>In your case, you can load it using.</p>
<p><code>model = torch.load('trained.pth')</code></p>

autocyz: <p>when training:</p>
<pre><code class="lang-auto">trained_model = training_func(.....)
torch.save(trained_model.state_dict(), 'trained.pth')
</code></pre>
<p>then:</p>
<pre><code class="lang-auto">pretrained_weights = torch.load('trained.pth'')
model = resnet50(pretrained=False)
model.load_state_dict(pretrained_weights)
</code></pre>
<p>you save your model state_dict with model structure by using <code>torch.save(trained_model, 'trained.pth')</code> , but you just load state_dict  by <code>model.load_state_dict(pretrained_weights)</code></p>

Rosa: <p>Thanks, but , if you look , what I am returning is</p>
<pre><code class="lang-auto">trained_model.state_dict()
</code></pre>
<p>So , no difference between your suggestion and my implementation , is it ?</p>
<pre><code class="lang-auto">torch.save(trained_model, 'trained.pth')
</code></pre>

ilyes: <p>if this is your return, then you are returning the whole model</p>
<pre><code class="lang-auto">model.load_state_dict(best_model_wts)
return model
</code></pre>

Rosa: <p>I see, that does make sense!! Thanks.</p>

Rosa: <p>Just for others,<br>
Depending on the saving options, my code also works. Just in below way:</p>
<pre><code class="lang-auto">pretrained_model = torch.load('trainedModel.pth')
model = resnet50(pretrained=False)
model.load_state_dict(pretrained_model.state_dict(),strict=False)
</code></pre>

ilyes: <p>Yes it works this way !  but with two drawbacks:</p>
<ul>
<li>now you are saving large files ( memory issue)</li>
<li>now you are saving as a pickle object which doesn’t save the model’s classes, but saves paths to them. As mentioned in the documentation, this may cause a problem during load time.</li>
</ul>