Yolov5 with Pytorch Overfitting Problem

Yasar_Niyazoglu: <p>Hello there. I trained in yolov5 with my own data set and my map values ​​are like this. As a result of the model I made with 30 epochs, it did a normal quality job when testing with video, but when I tested the same video with the model formed as a result of 300 epochs, it did a worse job than 30 epochs. how can i solve this.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/3/933d36274432d44498c2e1a596be85ee37b7817e.png" data-download-href="https://discuss.pytorch.org/uploads/default/933d36274432d44498c2e1a596be85ee37b7817e" title="Ekran Alıntısı2"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/3/933d36274432d44498c2e1a596be85ee37b7817e_2_690x314.png" alt="Ekran Alıntısı2" data-base62-sha1="l0xkF5cOOINwafnLPhxEzUDoi3c" width="690" height="314" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/3/933d36274432d44498c2e1a596be85ee37b7817e_2_690x314.png, https://discuss.pytorch.org/uploads/default/optimized/3X/9/3/933d36274432d44498c2e1a596be85ee37b7817e_2_1035x471.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/9/3/933d36274432d44498c2e1a596be85ee37b7817e_2_1380x628.png 2x" data-dominant-color="FBFBFC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Ekran Alıntısı2</span><span class="informations">1805×823 97.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/f/9fe9c5c971afcee69a0f462041c796d9e88466d7.png" data-download-href="https://discuss.pytorch.org/uploads/default/9fe9c5c971afcee69a0f462041c796d9e88466d7" title="Ekran Alıntısı1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/f/9fe9c5c971afcee69a0f462041c796d9e88466d7_2_690x311.png" alt="Ekran Alıntısı1" data-base62-sha1="mOELWyvRTq1nwsvcJKZ4552S4bd" width="690" height="311" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/f/9fe9c5c971afcee69a0f462041c796d9e88466d7_2_690x311.png, https://discuss.pytorch.org/uploads/default/optimized/3X/9/f/9fe9c5c971afcee69a0f462041c796d9e88466d7_2_1035x466.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/9/f/9fe9c5c971afcee69a0f462041c796d9e88466d7_2_1380x622.png 2x" data-dominant-color="FAFBFB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Ekran Alıntısı1</span><span class="informations">1785×806 75.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/5/a56dd35d018bdedf65067b80524ea478e5cc46ef.png" data-download-href="https://discuss.pytorch.org/uploads/default/a56dd35d018bdedf65067b80524ea478e5cc46ef" title="Ekran Alıntısı"><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/5/a56dd35d018bdedf65067b80524ea478e5cc46ef.png" alt="Ekran Alıntısı" data-base62-sha1="nBs58VCQ1tEsh2NObOuFXgn7PQj" width="690" height="347" data-dominant-color="F5F5F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Ekran Alıntısı</span><span class="informations">1024×515 25.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

SarthakJain: <p>Hi,</p>
<p>As you have correctly identified in the title, the model suffers from overfitting.<br>
Especially for object detection these are some specific ways you can solve them:</p>
<p><strong>Use Data Augmentation using the Albumentations Library</strong><br>
Data Augmentations is a way to prevent your model from learning on noise by simply altering the data to express more variety. This can introduce helpful aspects of your model like translational or rotational invariance. You can apply these augmentations to the frames of your video and a great thing is that these albumentations work to transform both boxes and images so you don’t have to worry about that.<br>
I suggest you check out this github and doc to get started:</p>
<ul>
<li><a href="https://albumentations.ai/docs/" class="inline-onebox" rel="noopener nofollow ugc">Albumentations Documentation</a></li>
<li><a href="https://github.com/albumentations-team/albumentations" class="inline-onebox" rel="noopener nofollow ugc">GitHub - albumentations-team/albumentations: Fast image augmentation library and an easy-to-use wrapper around other libraries. Documentation: https://albumentations.ai/docs/ Paper about the library: https://www.mdpi.com/2078-2489/11/2/125</a></li>
</ul>
<p>Additionally you can try to obtain more data by looking for other well known dataset like openImages, Imagenet,  COCO etc. and altering your dataclass to work on images that share the same classes you are already testing for.</p>
<p>Another thing to try is to increase the weight decay in your model. Note: You can also find more geeneral advice on overfitting on the internet but these are some specific ways you can tackle the issue you have right now.</p>
<p>Sarthak</p>

Yasar_Niyazoglu: <p>Thank you for your interest.<br>
I will try to increase data. You said weight reduction in the model, what exactly is this? you mean my L2 norm</p>

SarthakJain: <p>It depends on your optimizer.<br>
Use this documentation to find out more on whether it L1, L2 or some smart combination.<br>
<a href="https://pytorch.org/docs/stable/optim.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/optim.html</a></p>
<p>Check this image as an example.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/7/874467dbc728f6572172b3fcff2d12445a3007b5.png" data-download-href="https://discuss.pytorch.org/uploads/default/874467dbc728f6572172b3fcff2d12445a3007b5" title="Screen Shot 2021-07-03 at 7.41.12 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/7/874467dbc728f6572172b3fcff2d12445a3007b5_2_690x341.png" alt="Screen Shot 2021-07-03 at 7.41.12 PM" data-base62-sha1="jiD13vnRND6ECtPFProIC9BKW6p" width="690" height="341" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/7/874467dbc728f6572172b3fcff2d12445a3007b5_2_690x341.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/7/874467dbc728f6572172b3fcff2d12445a3007b5_2_1035x511.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/7/874467dbc728f6572172b3fcff2d12445a3007b5_2_1380x682.png 2x" data-dominant-color="F3F1F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2021-07-03 at 7.41.12 PM</span><span class="informations">2188×1082 274 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Also, in addtion to more data, I really suggest you check out the albumentations library as that can add a lot to your model.</p>
<p>Sarthak Jain</p>

Yasar_Niyazoglu: <p>thank you my friend. i will do what you say</p>