ChauffeurNet Loss functions

username: Isaac_Kargar
cooked: <p>Hi,<br>
I’m trying to understand loss functions used in the ChauffeurNet paper from Waymo.</p>
<aside class="onebox pdf" data-onebox-src="https://www.roboticsproceedings.org/rss15/p31.pdf">
  <header class="source">

      <a href="https://www.roboticsproceedings.org/rss15/p31.pdf" target="_blank" rel="noopener nofollow ugc">roboticsproceedings.org</a>
  </header>

  <article class="onebox-body">
    <a href="https://www.roboticsproceedings.org/rss15/p31.pdf" target="_blank" rel="noopener nofollow ugc"><span class="pdf-onebox-logo"></span></a>

<h3><a href="https://www.roboticsproceedings.org/rss15/p31.pdf" target="_blank" rel="noopener nofollow ugc">p31.pdf</a></h3>

  <p class="filesize">1616.99 KB</p>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>They want to predict a mask and a point for the future position of the car. The loss function is as follows:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/7/67690eb261661ff114c6fa37f88ac8ba042a7003.png" data-download-href="https://discuss.pytorch.org/uploads/default/67690eb261661ff114c6fa37f88ac8ba042a7003" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/67690eb261661ff114c6fa37f88ac8ba042a7003_2_529x500.png" alt="image" data-base62-sha1="eKOgmuZkP6U2zBO7eoNujzo2WdB" width="529" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/67690eb261661ff114c6fa37f88ac8ba042a7003_2_529x500.png, https://discuss.pytorch.org/uploads/default/original/3X/6/7/67690eb261661ff114c6fa37f88ac8ba042a7003.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/7/67690eb261661ff114c6fa37f88ac8ba042a7003.png 2x" data-dominant-color="E8E8E8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">626×591 99.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
The masks are as follows:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/6/669ebb4790a578877fb725dc1f48adac880d2cc6.png" data-download-href="https://discuss.pytorch.org/uploads/default/669ebb4790a578877fb725dc1f48adac880d2cc6" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/6/669ebb4790a578877fb725dc1f48adac880d2cc6_2_601x500.png" alt="image" data-base62-sha1="eDOMx1SRRmQWwJPe86HHGvTaelU" width="601" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/6/669ebb4790a578877fb725dc1f48adac880d2cc6_2_601x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/6/669ebb4790a578877fb725dc1f48adac880d2cc6_2_901x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/6/669ebb4790a578877fb725dc1f48adac880d2cc6.png 2x" data-dominant-color="EBECEC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1032×858 54.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I want to implement these loss functions in pytorch. For the Bk prediction, I used a BCELoss and sigmoid function on the output of the network. I don’t understand two sigmas in the eq(4) for the L_B loss. The BLELoss has sigma in itself. I think it is a multi-label binary classification. value of each pixel would be between 0 and 1 in output of the network. After reading pytorch forum I realized that I can use BCEWithLogits loss and assign pos_weight, because as you can see in the image, the classes are imbalanced.<br>
And for the L_P loss I used a spatial_softamx to convert the output of the network to probabilities. I think the sum of the value of all pixels in output should be 1. right? and I think it is a one-label binary classification.<br>
Now I don’t understand which loss function I have to use in pytorch.<br>
What is the difference between BCELoss for one-label vs multi-label binary classification?<br>
And if I use BCEWithLogits, how should I use network in test time? I use a separate sigmoid but it seems that the result is not good.</p>
<p>Thanks</p>

username: Isaac_Kargar
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Do you have any idea about this problem?</p>