How to train model from remaining interrupted epoch?

mathematics: <p>I had save a model upto 7 epochs,</p>
<details>
<summary>
Summary</summary>
<p>Epoch: 1 	Training Loss: 4.736156 	Validation Loss: 5.014328<br>
Validation has decreased<br>
Saving Model…<br>
Epoch: 2 	Training Loss: 4.607684 	Validation Loss: 5.020730<br>
Epoch: 3 	Training Loss: 4.722754 	Validation Loss: 4.918604<br>
Validation has decreased<br>
Saving Model…<br>
Epoch: 4 	Training Loss: 4.724532 	Validation Loss: 4.631703<br>
Validation has decreased<br>
Saving Model…<br>
Epoch: 5 	Training Loss: 4.621485 	Validation Loss: 4.857414<br>
Epoch: 6 	Training Loss: 4.601058 	Validation Loss: 5.013983<br>
Epoch: 7 	Training Loss: 4.561738 	Validation Loss: 4.354510<br>
Validation has decreased<br>
Saving Model…<br>
Epoch: 8 	Training Loss: 4.606921 	Validation Loss: 4.746311<br>
Epoch: 9 	Training Loss: 4.655972 	Validation Loss: 4.657127<br>
Epoch: 10 	Training Loss: 4.661118 	Validation Loss: 4.698534<br>
Epoch: 11 	Training Loss: 4.605792 	Validation Loss: 4.700062<br>
Epoch: 12 	Training Loss: 4.711023 	Validation Loss: 4.680906<br>
Epoch: 13 	Training Loss: 4.442619 	Validation Loss: 4.678782<br>
Epoch: 14 	Training Loss: 4.598712 	Validation Loss: 4.609448<br>
Epoch: 15 	Training Loss: 4.525731 	Validation Loss: 4.983483<br>
Epoch: 16 	Training Loss: 4.530140 	Validation Loss: 4.464461<br>
Epoch: 17 	Training Loss: 4.501666 	Validation Loss: 4.358377<br>
Epoch: 18 	Training Loss: 4.614322 	Validation Loss: 4.664511<br>
Epoch: 19 	Training Loss: 4.537676 	Validation Loss: 4.367801<br>
Epoch: 21 	Training Loss: 4.442828 	Validation Loss: 4.889397<br>
​</p>
</details>
<p>Unfortunately internet gone.<br>
Now i want to again train interrupted training , As  i had saved model using</p>
<pre><code class="lang-auto"> state = {'epoch': epoch + 1, 'state_dict': model.state_dict(), 
'optimizer': optimizer.state_dict(), 'loss': loss, }
torch.save(state, 'saved.pt')
</code></pre>
<p>How do i train from that <em>saved.pt</em> again after 7 epoch?<br>
Thank you for answer</p>

spacemeerkat: <p>Maybe my answer here is too basic but this is how I generally do this:</p>
<p>Load the model:<br>
<code>from model_location import model</code></p>
<p>Instantiate the model:<br>
<code>m = model()</code></p>
<p>Load the state dictionary required:<br>
<code>m.load_state_dict(torch.load(state_dictionary_saved_at_epoch_7.pt'))</code></p>
<p>(as you have saved it as <code>saved.pt</code> you might have to preload this dictionary and then replace <code>state_dictionary_saved_at_epoch_7.pt'</code> with whatever is in <code>'state_dict': model.state_dict()</code> at epoch 7. (perhaps this is the subtelty of your question that most of this answer will be not addressing properly?); you will also have to push the optimizer state dictionary and loss value too but I don’t know how to do that.)</p>
<p>Set the training state of the model <code>m</code> to true (perhaps this is done by default):<br>
<code>m.train(True)</code></p>
<p>…and then continue training using the model now called <code>m</code> in this example.</p>
<p>Sorry if that’s too basic an answer!</p>

mathematics: <p>Thank you for you explanation <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> . but im not getting over How to replace<br>
<em>‘state_dictionary_saved_at_epoch_7.pt’</em> from <em>‘saved.pt’</em>  and all below, also code example.</p>

boto: <p>Hi!</p>
<p>The answer by <a class="mention" href="/u/spacemeerkat">@spacemeerkat</a> seems to be the correct way to go (see <a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html" rel="nofollow noopener">PyTorch: Saving and Loading Models</a>)</p>
<pre><code class="lang-auto">from your_model import model  # Replace your_model with the name of your model class
m = model()  # Create a new model
state = torch.load(saved.pt)  # Load the whole dictionary, as that's what you have saved
m.load_state_dict(state[state_dict]) 
# There is no need for m.train() if you want to continue with training
</code></pre>
<p>P.S.<br>
You are always saving the state in the same file. This leads to only the newest model state to be saved.</p>

mathematics: <p>Does it runs previously retrained weights or starts from new.</p>

spacemeerkat: <p>Yes, loading a saved state dictionary loads pretrained weights so it won’t be starting from new.</p>
<p>Just note that with this method, be careful not to change the model architecture and then try to load the state dictionary to a new model as obviously they will no longer share the same shapes…something that has stung me in the past.</p>

mathematics: <p>Thank you ,  for helping. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>