Unable to get Masks with Mask R-CNN Model during inference

agn3ya: <p>Hi Everyone,<br>
I am working on a project involving Mask R-CNN for video frame analysis. The model is trained on a custom dataset and annotation from CVAT. It works fine for detecting bounding boxes, but I face issues with visualising masks. I am not sure whether the model is generating masks correctly.</p>
<p><strong>Relevant Code:</strong><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/1/71e39c2617b6b0ac1548a4d2d6c6f5154fc179e2.png" data-download-href="https://discuss.pytorch.org/uploads/default/71e39c2617b6b0ac1548a4d2d6c6f5154fc179e2" title="Screenshot from 2024-05-28 12-28-16"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/1/71e39c2617b6b0ac1548a4d2d6c6f5154fc179e2_2_690x253.png" alt="Screenshot from 2024-05-28 12-28-16" data-base62-sha1="gfvBAoFUjDkflh6BfzaVEdlmsca" width="690" height="253" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/1/71e39c2617b6b0ac1548a4d2d6c6f5154fc179e2_2_690x253.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/1/71e39c2617b6b0ac1548a4d2d6c6f5154fc179e2_2_1035x379.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/1/71e39c2617b6b0ac1548a4d2d6c6f5154fc179e2.png 2x" data-dominant-color="95988D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2024-05-28 12-28-16</span><span class="informations">1104×406 77.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<pre><code class="lang-auto">def postprocess_detection(frames, outputs, threshold=0.5):
    frames = frames.cpu().numpy().transpose(1, 2, 0)

    boxes = outputs['boxes'].cpu().detach().numpy()
    labels = outputs['labels'].cpu().detach().numpy()
    scores = outputs['scores'].cpu().detach().numpy()
    masks = outputs['masks'].cpu().detach().numpy()

    indices = scores &gt;= threshold
    boxes = boxes[indices]
    labels = labels[indices]
    scores = scores[indices]
    masks = masks[indices]

    mask_canvas = np.zeros_like(frames, dtype=np.uint8)

    for i, mask in enumerate(masks):
        mask = mask[0, :, :]
        
        mask = preprocessing.normalize(mask)
        
        # Apply threshold to binarize mask
        mask = (mask &gt; 0.5).astype(np.uint8)
        
        color = [random.randint(0, 255) for _ in range(3)]
        for c in range(3):
            mask_canvas[:, :, c] = np.where(mask == 1, color[c], mask_canvas[:, :, c])
    result_image = cv2.addWeighted(frames, 1, mask_canvas.astype(np.float32), 0.5, 0)

    return result_image, mask_canvas, boxes, labels, scores
</code></pre>
<p><strong>Issue:</strong></p>
<ul>
<li>Bounding Boxes are drawn correctly</li>
<li>Masks and not visible and I am not sure whether they are being generated correctly</li>
</ul>
<p><strong>Additional Details:</strong></p>
<ul>
<li>The masks are thresholded and resized before blending.</li>
<li>The output shape and type of the image array are checked and seem to be correct.</li>
<li>The device used for inference is CUDA.</li>
</ul>