Overfitting or underfitting

ays: <p>Hi,<br>
the concept of overfitting and undercutting is still quite confusing to me. Is this plot of training/validation loss below overfitting or underfitting?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/7/c7e8ef04f584a278f6b27f98703d827c155d1459.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/c7e8ef04f584a278f6b27f98703d827c155d1459" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c7e8ef04f584a278f6b27f98703d827c155d1459_2_249x499.jpeg" alt="image" data-base62-sha1="swu4Daadl9mcB6xYKWUc8FeEoTD" width="249" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c7e8ef04f584a278f6b27f98703d827c155d1459_2_249x499.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c7e8ef04f584a278f6b27f98703d827c155d1459_2_373x748.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c7e8ef04f584a278f6b27f98703d827c155d1459_2_498x998.jpeg 2x" data-dominant-color="FBFAF9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">620×1240 88.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>I would claim it’s neither as the validation loss is still “close” to the training loss, doesn’t change a lot and is thus also quite noisy. Plot both curves into the same plot and the gap should be small, if I interpret the values correctly.</p>

ays: <p>Thanks for your response. This is both plots together. So can this be considered a “reasonable” loss and means the model is actually learning?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/8/4880df4ee87cae55e9e959175af0cbf8af2b20e1.png" data-download-href="https://discuss.pytorch.org/uploads/default/4880df4ee87cae55e9e959175af0cbf8af2b20e1" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/8/4880df4ee87cae55e9e959175af0cbf8af2b20e1_2_621x500.png" alt="image" data-base62-sha1="alou6fwbkz6Wp4xnodOYD5tY4Eh" width="621" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/8/4880df4ee87cae55e9e959175af0cbf8af2b20e1_2_621x500.png, https://discuss.pytorch.org/uploads/default/original/3X/4/8/4880df4ee87cae55e9e959175af0cbf8af2b20e1.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/4/8/4880df4ee87cae55e9e959175af0cbf8af2b20e1.png 2x" data-dominant-color="FBFAF9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">766×616 48.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>The model reduced the loss initially at least and might not be stuck on a plateau. You might want to continue with the training to see if the loss would still decrease or not.</p>

Mah_Neh: <p>There is also learning schedulers that may be useful later on</p>
<p><a href="https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.ReduceLROnPlateau.html#torch.optim.lr_scheduler.ReduceLROnPlateau" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.ReduceLROnPlateau.html#torch.optim.lr_scheduler.ReduceLROnPlateau</a></p>

ays: <p>Im using pytorch lightning EarlyStopping with a patience of 8, so I guess the training stopped when the validation loss wasn’t improving?</p>

ays: <p>Is this similar to using <a href="https://pytorch-lightning.readthedocs.io/en/1.0.8/api/pytorch_lightning.callbacks.early_stopping.html#pytorch_lightning.callbacks.early_stopping.EarlyStopping" rel="noopener nofollow ugc">Early stopping</a> with pytorch lightning?</p>

Mah_Neh: <p>I try to keep deps to the minimum and haven’t used lightining, but I think it is not. Early Stopping does not change the learning rate, but stops the training process when the validation loss is stagnant.</p>