Mask Loss in Mask R-CNN returns negative values

sigma_x: <p>Apparently Mask head uses BCEWithLogits, so it combines sigmoid+CE, to avoid competition between masks. I get a negative value. Is this a bug, or someting’s wrong with my encoding? I encoded each target mask as a uint8 array, where all pixels are 0 except the object’s mask, which are set to the masks’s class (i.e it matches the target class).</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/4/34087a133d5881c455dc0fad81f562f68b827ecb.png" data-download-href="https://discuss.pytorch.org/uploads/default/34087a133d5881c455dc0fad81f562f68b827ecb" title="loss_mask"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/4/34087a133d5881c455dc0fad81f562f68b827ecb_2_690x25.png" alt="loss_mask" data-base62-sha1="7qiZnJ8MdMSJw54zg835ApkpqHx" width="690" height="25" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/4/34087a133d5881c455dc0fad81f562f68b827ecb_2_690x25.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/4/34087a133d5881c455dc0fad81f562f68b827ecb_2_1035x37.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/3/4/34087a133d5881c455dc0fad81f562f68b827ecb_2_1380x50.png 2x" data-dominant-color="3E1C33"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">loss_mask</span><span class="informations">1717×64 19.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Could you post a link to the repository you are using?<br>
Also, could you post the inputs and targets for a negative loss value?<br>
Negative values for <code>nn.BCE(WithLogits)Loss</code> can be created if your target is out of bounds.</p>

sigma_x: <p>I’m using this one:<br>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicon.ico" class="site-icon" width="16" height="16">
      <a href="https://github.com/pytorch/vision/tree/master/torchvision/models/detection" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars2.githubusercontent.com/u/21003710?s=400&amp;v=4" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://github.com/pytorch/vision/tree/master/torchvision/models/detection" target="_blank" rel="nofollow noopener">pytorch/vision</a></h3>

<p>Datasets, Transforms and Models specific to Computer Vision - pytorch/vision</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

ptrblck: <p>Thanks for the link. Have you checked the target values?</p>

sigma_x: <p>How do I correctly label the mask? If I have K objects across N classes in one image, the mask label array must be HxWxK, but in each map, do the mask values have to be =1 or =Class_number?</p>

ptrblck: <p>If you are using <code>nn.BCE(WithLogits)Loss</code>, the target should have the same shape as your output activation, where the channel index corresponds to the class index.<br>
A one indicates the presence of the class at this spatial position, while a zero does not.</p>

sigma_x: <p>This is from the mask_rcnn.py:</p>
<pre><code class="lang-auto">    During training, the model expects both the input tensors, as well as a targets dictionary,
    containing:
        - boxes (Tensor[N, 4]): the ground-truth boxes in [x0, y0, x1, y1] format, with values
          between 0 and H and 0 and W
        - labels (Tensor[N]): the class label for each ground-truth box
        - masks (Tensor[N, H, W]): the segmentation binary masks for each instance
</code></pre>
<p>here N is a ground-truth box, not class index.</p>
<p>So if I have (for example) 3 objects of class 1 and 2 objects of class2 and 1 object of class 3, image size 224x224, the mask array will be size (6x224x224), each map will be 0 for background and 1 for object. During training BCE is applied pixelwise, and the correct class is taken from the <code>labels</code> vector. Since for each object its class/label, bbox and mask are appended to a list, the order stays the same. so the correct class is taken from the <code>labels</code> vector.</p>