Loss increases after loading model from checkpoint

username: KUSHAL_JAIN
cooked: <p>I am experiencing this weird model behavior.  I am training a video captioning model. The training loss for epochs decreases for each epoch during the first 8 epochs. When I load the model again from the last checkpoint to train again for 4 epochs, the loss is greater than the last recorded loss and does not drop in subsequent epochs.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/9/49bebd58c6d85041c1b326421cc34aa054a9d1d7.png" data-download-href="https://discuss.pytorch.org/uploads/default/49bebd58c6d85041c1b326421cc34aa054a9d1d7" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/9/49bebd58c6d85041c1b326421cc34aa054a9d1d7_2_567x500.png" alt="image" data-base62-sha1="awnvOsKLYPJGD8E7OUHtXiPspJZ" width="567" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/9/49bebd58c6d85041c1b326421cc34aa054a9d1d7_2_567x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/4/9/49bebd58c6d85041c1b326421cc34aa054a9d1d7_2_850x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/4/9/49bebd58c6d85041c1b326421cc34aa054a9d1d7.png 2x" data-dominant-color="F3F3F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1014×893 66.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I load my model in this way<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/8/28ef0a337cb36a4f3935becb81265cb06afc1979.png" data-download-href="https://discuss.pytorch.org/uploads/default/28ef0a337cb36a4f3935becb81265cb06afc1979" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/8/28ef0a337cb36a4f3935becb81265cb06afc1979_2_690x349.png" alt="image" data-base62-sha1="5Q7ePZbB55U17nmjUMacDyIW4kx" width="690" height="349" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/8/28ef0a337cb36a4f3935becb81265cb06afc1979_2_690x349.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/8/28ef0a337cb36a4f3935becb81265cb06afc1979_2_1035x523.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/8/28ef0a337cb36a4f3935becb81265cb06afc1979_2_1380x698.png 2x" data-dominant-color="EEEEED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1459×738 69.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
The loss on retraining<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/0/907c03baebb7eb9e343f2f202778d640cd1fd0c2.png" data-download-href="https://discuss.pytorch.org/uploads/default/907c03baebb7eb9e343f2f202778d640cd1fd0c2" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/0/907c03baebb7eb9e343f2f202778d640cd1fd0c2_2_690x361.png" alt="image" data-base62-sha1="kCasfR0mOd7ikDrrFP0iAfCsNKW" width="690" height="361" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/0/907c03baebb7eb9e343f2f202778d640cd1fd0c2_2_690x361.png, https://discuss.pytorch.org/uploads/default/optimized/3X/9/0/907c03baebb7eb9e343f2f202778d640cd1fd0c2_2_1035x541.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/9/0/907c03baebb7eb9e343f2f202778d640cd1fd0c2.png 2x" data-dominant-color="F4F3F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1093×573 40.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Is this common? What are the reasons for it and how can I improve/avoid this?<br>
Thank you.</p>