Loss value 0 indication

username: lycaenidae
cooked: <p>Hi,</p>
<p>When I get 0 loss, is it always the indication of overfitting? Because I am getting accuracy around %50 on even train set, although my loss is 0 after a while.</p>
<p>What could be the problem in this case? Could you please help me if you have any idea?  Thanks a lot<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/8/781b455a47b0ca89227c09f2eb4a907b8960d4bc.png" data-download-href="https://discuss.pytorch.org/uploads/default/781b455a47b0ca89227c09f2eb4a907b8960d4bc" title="Screenshot 2020-05-15 at 16.18.20"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/8/781b455a47b0ca89227c09f2eb4a907b8960d4bc_2_690x472.png" alt="Screenshot 2020-05-15 at 16.18.20" data-base62-sha1="h8vITsRL17SHl6NilVkIalgdW9K" width="690" height="472" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/8/781b455a47b0ca89227c09f2eb4a907b8960d4bc_2_690x472.png, https://discuss.pytorch.org/uploads/default/original/3X/7/8/781b455a47b0ca89227c09f2eb4a907b8960d4bc.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/8/781b455a47b0ca89227c09f2eb4a907b8960d4bc.png 2x" data-dominant-color="F9F5F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2020-05-15 at 16.18.20</span><span class="informations">750×514 28.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: KFrank
cooked: <p>Hi Lyca!</p>
<aside class="quote no-group" data-username="lycaenidae" data-post="1" data-topic="81377" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/5f9b8f/48.png" class="avatar"> lycaenidae:</div>
<blockquote>
<p>I am getting accuracy around %50 on even train set, although my loss is 0 after a while.</p>
</blockquote>
</aside>
<p>You don’t provide any detail on what you are doing, so it’s hard to<br>
be certain, but you very likely have a bug somewhere.</p>
<p>Common loss functions only go to zero when the predictions are<br>
“completely right.”  Such predictions should – on your training<br>
set – produce perfect accuracy.  So something isn’t lining up here.</p>
<p>Best.</p>
<p>K. Frank</p>

username: lycaenidae
cooked: <p>Thanks a lot <a class="mention" href="/u/kfrank">@KFrank</a>, I would expect the same thing on training set therefore I asked. This loss function corresponds to standart triplet loss which corresponds to minimising energy between positive and anchor and maximising between anchor and negative.</p>
<p>I am evaluating the accuracy on the same set that I used for training. Is there any chance that the problem is related to batching somehow?</p>
<p>Best</p>

username: KFrank
cooked: <p>Hi Lyca!</p>
<aside class="quote no-group" data-username="lycaenidae" data-post="3" data-topic="81377" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/5f9b8f/48.png" class="avatar"> lycaenidae:</div>
<blockquote>
<p>This loss function corresponds to standart triplet loss</p>
</blockquote>
</aside>
<p>Let me take back what I said.  Triplet loss is not a loss function for<br>
which zero loss means “completely right.”  (In fact, it’s not even<br>
clear to me what “completely right” would mean in practice in a<br>
case where one would use triplet loss.)</p>
<p>Best.</p>
<p>K. Frank</p>

username: lycaenidae
cooked: <aside class="quote no-group" data-username="KFrank" data-post="4" data-topic="81377">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>Triplet loss is not a loss function for<br>
which zero loss means “completely right.”</p>
</blockquote>
</aside>
<p>What is the reason for that? I wonder if we get 0 loss function, I would expect to overfit to training data.  Is this statement wrong?</p>