Lane / line detection loss and training

vermont_waters: <p>I am working on a lane line detection tasks. I use a strategy similar to the ones used in many papers on <a href="https://paperswithcode.com/task/lane-detection" class="inline-onebox" rel="noopener nofollow ugc">Lane Detection | Papers With Code</a>.</p>
<p>Similar to object detection all lanes are also given by anchors. Lanes are assumed to be more or less vertical lines, starting from the bottom. In contrast to standard object detection tasks the lanes are NOT parameteized by 4 box coordinates and a class score but by the following outputs:</p>
<ul>
<li>class score, CrossEntropyLoss</li>
<li>lane end in percentage of the vertical space of the image, SmoothL1Loss</li>
<li>horizontal offsets from anchor coordinates at equally spaced positions along the vertical  axis of the image (e.g. 40 offsets along an image height of 360 pixels, normalized to lie betwenn 0-1 for each element, SmoothL1Loss per element.</li>
</ul>
<p>Now, I also want to predict the starting point of each anchor, for lanes which are not starting from the image border. For this I added another output:</p>
<ul>
<li>line start index in percentage of the vertical space of the image, SmoothL1Loss</li>
</ul>
<p>The F1 score for the loss without the lane start is given in dark blue, with the lane start loss in light blue:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/6/2689c5b201763a71d909d8345d982b0f48d42a5e.png" data-download-href="https://discuss.pytorch.org/uploads/default/2689c5b201763a71d909d8345d982b0f48d42a5e" title="F1-score"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/6/2689c5b201763a71d909d8345d982b0f48d42a5e_2_690x353.png" alt="F1-score" data-base62-sha1="5uVjOn5Ft8XeX5bEDukRf4vb8J0" width="690" height="353" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/6/2689c5b201763a71d909d8345d982b0f48d42a5e_2_690x353.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/6/2689c5b201763a71d909d8345d982b0f48d42a5e_2_1035x529.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/6/2689c5b201763a71d909d8345d982b0f48d42a5e_2_1380x706.png 2x" data-dominant-color="FBFCFD"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">F1-score</span><span class="informations">1871×959 76.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I cant get my head around the root cause for the drastically higher variance in the F1-score. What strategy could I use to mitigate this? Is there anything in pytorch that could help here. Maybe regularization?</p>
<p>Here is the total loss of all terms added and weighted:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/7/b768fb406d036bfd86b2a4913467183688910d6e.png" data-download-href="https://discuss.pytorch.org/uploads/default/b768fb406d036bfd86b2a4913467183688910d6e" title="Loss comparison"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/7/b768fb406d036bfd86b2a4913467183688910d6e_2_690x315.png" alt="Loss comparison" data-base62-sha1="qawiwE42PQzvGNFKFf1lzRiX0Hs" width="690" height="315" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/7/b768fb406d036bfd86b2a4913467183688910d6e_2_690x315.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/7/b768fb406d036bfd86b2a4913467183688910d6e_2_1035x472.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/7/b768fb406d036bfd86b2a4913467183688910d6e_2_1380x630.png 2x" data-dominant-color="D9E9F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Loss comparison</span><span class="informations">2126×973 349 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>In the loss I see that in the new configurations there are more frequent but smaller jumps in the overall loss value. I assume that these jumps are due to some badly labeled data in interaction with augmentation. Does someone has some more insights/intuition for this?</p>