How to get mask scores for image segmentation in Detectron2?

dangmanhtruong1995: <p>Hi, I’m trying to use Detectron2 to extract masks for image segmentation using Mask-RCNN. I used the command:</p>
<blockquote>
<p><code>outputs = predictor(im) where predictor is a DefaultPredictor</code></p>
</blockquote>
<p>However, the output has a field called pred_masks which returns only True or False values, while I want it to return a value from 0 to 1 in each pixel (from what I understand while reading the Mask-RCNN paper, it is supposed to return from 0 to 1). Please help me, thank you very much.</p>

ptrblck: <p><code>outputs</code> should be a <code>dict</code> containing <code>panoptic_seg</code>, <code>sem_seg</code> or <code>instances</code> as seen <a href="https://github.com/facebookresearch/detectron2/blob/0b500b4d7ed11e9728867298f5f168fb34908e2c/demo/predictor.py#L48-L66">here</a>. Could you post a minimal code snippet to reproduce the issue?</p>

dangmanhtruong1995: <p>The thing is that <code>outputs</code> is from a DefaultPredictor which provides only <code>pred_masks</code> (binary mask) which is a field in <code>instances</code>. But I want a score associated with each pixel.</p>
<p>Here is the example:</p>
<blockquote>
<p>for img_name in img_list:<br>
im = cv2.imread(os.path.join(TEST_DIR, img_name))<br>
# if img_name == ‘438.jpg’:<br>
#     pdb.set_trace()<br>
# pdb.set_trace()<br>
outputs = predictor(im)<br>
mask = outputs[‘instances’].pred_masks.to(‘cpu’).numpy()<br>
mask = mask.astype(np.uint8)<br>
pdb.set_trace()<br>
mask[mask &gt; 0] = 255<br>
(N, H, W) = mask.shape<br>
if N == 1:<br>
mask = mask.squeeze()<br>
else:<br>
# Many objects<br>
mask1 = np.zeros((H, W), dtype=np.uint8)<br>
for i in range(N):<br>
mask1[mask[i, :] &gt; 0] = 255<br>
mask = mask1<br>
cv2.imwrite(os.path.join(OUTPUT_DIR, img_name), mask)</p>
<pre><code>num += 1
</code></pre>
</blockquote>

ptrblck: <p>Your code snippet is unfortunately not executable. Could you use random inputs and define all classes, so that we could run the code and debug it, please?</p>

dangmanhtruong1995: <aside class="quote no-group" data-username="dangmanhtruong1995" data-post="3" data-topic="84209">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dangmanhtruong1995/48/59870_2.png" class="avatar"> dangmanhtruong1995:</div>
<blockquote>
<p>outputs[‘instances’].pred_masks.to(‘cpu’).numpy()</p>
</blockquote>
</aside>
<p>Here is the code (I used from <a href="https://colab.research.google.com/drive/16jcaJoc6bCFAQ96jDe2HwtXj7BMD_-m5#scrollTo=dq9GY37ml1kr" class="inline-onebox" rel="noopener nofollow ugc">Google Colab</a>):</p>
<blockquote>
<p>import detectron2<br>
from detectron2.utils.logger import setup_logger<br>
setup_logger()</p>
<p>import numpy as np<br>
import cv2<br>
import random<br>
import pdb<br>
import matplotlib as mpl<br>
mpl.use(‘Agg’)<br>
import matplotlib.pyplot as plt</p>
<p>from detectron2 import model_zoo<br>
from detectron2.engine import DefaultPredictor<br>
from detectron2.config import get_cfg<br>
from detectron2.utils.visualizer import Visualizer<br>
from detectron2.data import MetadataCatalog</p>
<p>cfg = get_cfg()<br>
cfg.merge_from_file(model_zoo.get_config_file(“COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml”))<br>
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5  # set threshold for this model<br>
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(“COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml”)<br>
predictor = DefaultPredictor(cfg)<br>
im = cv2.imread(“./input.jpg”)<br>
outputs = predictor(im)<br>
mask = outputs[‘instances’].pred_masks.to(‘cpu’).numpy()<br>
pdb.set_trace()<br>
v = Visualizer(im[:, :, ::-1], MetadataCatalog.get(cfg.DATASETS.TRAIN[0]), scale=1.2)<br>
v = v.draw_instance_predictions(outputs[“instances”].to(“cpu”))<br>
cv2.imwrite(‘./out.png’, v.get_image()[:, :, ::-1])</p>
</blockquote>
<p>The image can be downloaded with:</p>
<p><code>wget http://images.cocodataset.org/val2017/000000439715.jpg -O input.jpg</code></p>
<p>The problem is that the <code>mask</code> variable only returns either 0 or 1, but I want the score for each pixel from 0 to 1 (because the Mask-RCNN paper returns so). Thank you very much.</p>

dangmanhtruong1995: <p>So after a while, I was able to figure out a makeshift solution (by looking at a number of places:  , github .com / facebookresearch /detectron2/blob/master/detectron2/modeling/postprocessing.py, <a href="https://github.com/facebookresearch/detectron2/blob/98739e17ca01a46fe1da2db9ec09ed03564088bd/detectron2/layers/mask_ops.py#L67" class="inline-onebox" rel="noopener nofollow ugc">detectron2/detectron2/layers/mask_ops.py at 98739e17ca01a46fe1da2db9ec09ed03564088bd · facebookresearch/detectron2 · GitHub</a>). I copied the code from those links and modified only the part where the thresholding is involved (which is the last part). Here is the full code:</p>
<blockquote>
<p>import detectron2<br>
from detectron2.utils.logger import setup_logger<br>
setup_logger()</p>
<h1>import some common libraries</h1>
<p>import numpy as np<br>
import cv2<br>
import random<br>
import pdb<br>
import matplotlib as mpl<br>
mpl.use(‘Agg’)<br>
import matplotlib.pyplot as plt<br>
import torch</p>
<p>from detectron2 import model_zoo<br>
from detectron2.engine import DefaultPredictor<br>
from detectron2.config import get_cfg<br>
from detectron2.utils.visualizer import Visualizer<br>
from detectron2.data import MetadataCatalog</p>
<p>from detectron2.checkpoint import DetectionCheckpointer<br>
from detectron2.modeling import build_model<br>
import detectron2.data.transforms as T<br>
from detectron2.structures import ImageList<br>
from detectron2.modeling.backbone.build import build_backbone<br>
from torch.nn import functional as F<br>
from detectron2.structures import Instances<br>
from detectron2.utils.memory import retry_if_cuda_oom<br>
from PIL import Image</p>
<p>BYTES_PER_FLOAT = 4</p>
<h1>TODO: This memory limit may be too much or too little. It would be better to</h1>
<h1>determine it based on available resources.</h1>
<p>GPU_MEM_LIMIT = 1024 ** 3  # 1 GB memory limit</p>
<p>def preprocess_image(batched_inputs, cfg):<br>
“”"<br>
Normalize, pad and batch the input images.<br>
“”"<br>
pixel_mean = torch.Tensor(cfg.MODEL.PIXEL_MEAN).view(-1, 1, 1)<br>
pixel_std = torch.Tensor(cfg.MODEL.PIXEL_STD).view(-1, 1, 1)<br>
backbone = build_backbone(cfg)</p>
<pre><code>images = [x["image"].to(pixel_mean.device) for x in batched_inputs]
images = [(x - pixel_mean) / pixel_std for x in images]
images = ImageList.from_tensors(images, backbone.size_divisibility)
return images
</code></pre>
<p>def custom_postprocess(instances, batched_inputs, image_sizes):<br>
“”"<br>
Rescale the output instances to the target size.<br>
“”"<br>
# note: private function; subject to changes<br>
processed_results = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
for results_per_image, input_per_image, image_size in zip(<br>
instances, batched_inputs, image_sizes<br>
):<br>
height = input_per_image.get(“height”, image_size[0])<br>
width = input_per_image.get(“width”, image_size[1])<br>
r = detector_postprocess(results_per_image, height, width)<br>
processed_results.append({“instances”: r})<br>
return processed_results</p>
<p>def detector_postprocess(results, output_height, output_width, mask_threshold=0.5):<br>
“”"<br>
Resize the output instances.<br>
The input images are often resized when entering an object detector.<br>
As a result, we often need the outputs of the detector in a different<br>
resolution from its inputs.<br>
This function will resize the raw outputs of an R-CNN detector<br>
to produce outputs according to the desired output resolution.<br>
Args:<br>
results (Instances): the raw outputs from the detector.<br>
<code>results.image_size</code> contains the input image resolution the detector sees.<br>
This object might be modified in-place.<br>
output_height, output_width: the desired output resolution.<br>
Returns:<br>
Instances: the resized output from the model, based on the output resolution<br>
“”"<br>
scale_x, scale_y = (output_width / results.image_size[1], output_height / results.image_size[0])<br>
results = Instances((output_height, output_width), **results.get_fields())</p>
<pre><code>if results.has("pred_boxes"):
    output_boxes = results.pred_boxes
elif results.has("proposal_boxes"):
    output_boxes = results.proposal_boxes

output_boxes.scale(scale_x, scale_y)
output_boxes.clip(results.image_size)

results = results[output_boxes.nonempty()]

if results.has("pred_masks"):
    results.pred_masks = retry_if_cuda_oom(paste_masks_in_image)(
        results.pred_masks[:, 0, :, :],  # N, 1, M, M
        results.pred_boxes,
        results.image_size,
        threshold=mask_threshold,
    )

if results.has("pred_keypoints"):
    results.pred_keypoints[:, :, 0] *= scale_x
    results.pred_keypoints[:, :, 1] *= scale_y

return results
</code></pre>
<p>def custom_do_paste_mask(masks, boxes, img_h, img_w, skip_empty=True):<br>
“”"<br>
Args:<br>
masks: N, 1, H, W<br>
boxes: N, 4<br>
img_h, img_w (int):<br>
skip_empty (bool): only paste masks within the region that<br>
tightly bound all boxes, and returns the results this region only.<br>
An important optimization for CPU.<br>
Returns:<br>
if skip_empty == False, a mask of shape (N, img_h, img_w)<br>
if skip_empty == True, a mask of shape (N, h’, w’), and the slice<br>
object for the corresponding region.<br>
“”"<br>
# On GPU, paste all masks together (up to chunk size)<br>
# by using the entire image to sample the masks<br>
# Compared to pasting them one by one,<br>
# this has more operations but is faster on COCO-scale dataset.<br>
device = masks.device<br>
if skip_empty:<br>
x0_int, y0_int = torch.clamp(boxes.min(dim=0).values.floor()[:2] - 1, min=0).to(<br>
dtype=torch.int32<br>
)<br>
x1_int = torch.clamp(boxes[:, 2].max().ceil() + 1, max=img_w).to(dtype=torch.int32)<br>
y1_int = torch.clamp(boxes[:, 3].max().ceil() + 1, max=img_h).to(dtype=torch.int32)<br>
else:<br>
x0_int, y0_int = 0, 0<br>
x1_int, y1_int = img_w, img_h<br>
x0, y0, x1, y1 = torch.split(boxes, 1, dim=1)  # each is Nx1</p>
<pre><code>N = masks.shape[0]

img_y = torch.arange(y0_int, y1_int, device=device, dtype=torch.float32) + 0.5
img_x = torch.arange(x0_int, x1_int, device=device, dtype=torch.float32) + 0.5
img_y = (img_y - y0) / (y1 - y0) * 2 - 1
img_x = (img_x - x0) / (x1 - x0) * 2 - 1
# img_x, img_y have shapes (N, w), (N, h)

gx = img_x[:, None, :].expand(N, img_y.size(1), img_x.size(1))
gy = img_y[:, :, None].expand(N, img_y.size(1), img_x.size(1))
grid = torch.stack([gx, gy], dim=3)

img_masks = F.grid_sample(masks.to(dtype=torch.float32), grid, align_corners=False)

if skip_empty:
    return img_masks[:, 0], (slice(y0_int, y1_int), slice(x0_int, x1_int))
else:
    return img_masks[:, 0], ()
</code></pre>
<p>def paste_masks_in_image(masks, boxes, image_shape, threshold=0.5):<br>
“”"<br>
Paste a set of masks that are of a fixed resolution (e.g., 28 x 28) into an image.<br>
The location, height, and width for pasting each mask is determined by their<br>
corresponding bounding boxes in boxes.<br>
Note:<br>
This is a complicated but more accurate implementation. In actual deployment, it is<br>
often enough to use a faster but less accurate implementation.<br>
See :func:<code>paste_mask_in_image_old</code> in this file for an alternative implementation.<br>
Args:<br>
masks (tensor): Tensor of shape (Bimg, Hmask, Wmask), where Bimg is the number of<br>
detected object instances in the image and Hmask, Wmask are the mask width and mask<br>
height of the predicted mask (e.g., Hmask = Wmask = 28). Values are in [0, 1].<br>
boxes (Boxes or Tensor): A Boxes of length Bimg or Tensor of shape (Bimg, 4).<br>
boxes[i] and masks[i] correspond to the same object instance.<br>
image_shape (tuple): height, width<br>
threshold (float): A threshold in [0, 1] for converting the (soft) masks to<br>
binary masks.<br>
Returns:<br>
img_masks (Tensor): A tensor of shape (Bimg, Himage, Wimage), where Bimg is the<br>
number of detected object instances and Himage, Wimage are the image width<br>
and height. img_masks[i] is a binary mask for object instance i.<br>
“”"</p>
<pre><code>assert masks.shape[-1] == masks.shape[-2], "Only square mask predictions are supported"
N = len(masks)
if N == 0:
    return masks.new_empty((0,) + image_shape, dtype=torch.uint8)
if not isinstance(boxes, torch.Tensor):
    boxes = boxes.tensor
device = boxes.device
assert len(boxes) == N, boxes.shape

img_h, img_w = image_shape

# The actual implementation split the input into chunks,
# and paste them chunk by chunk.
if device.type == "cpu":
    # CPU is most efficient when they are pasted one by one with skip_empty=True
    # so that it performs minimal number of operations.
    num_chunks = N
else:
    # GPU benefits from parallelism for larger chunks, but may have memory issue
    # int(img_h) because shape may be tensors in tracing
    num_chunks = int(np.ceil(N * int(img_h) * int(img_w) * BYTES_PER_FLOAT / GPU_MEM_LIMIT))
    assert (
        num_chunks &lt;= N
    ), "Default GPU_MEM_LIMIT in mask_ops.py is too small; try increasing it"
chunks = torch.chunk(torch.arange(N, device=device), num_chunks)

# img_masks = torch.zeros(
#     N, img_h, img_w, device=device, dtype=torch.bool if threshold &gt;= 0 else torch.uint8
# )
img_masks = torch.zeros(
      N, img_h, img_w, device=device, dtype=torch.uint8
)
for inds in chunks:
    masks_chunk, spatial_inds = custom_do_paste_mask(
        masks[inds, None, :, :], boxes[inds], img_h, img_w, skip_empty=device.type == "cpu"
    )

    # if threshold &gt;= 0:
    #     masks_chunk = (masks_chunk &gt;= threshold).to(dtype=torch.bool)
    # else:
    # for visualization and debugging
    masks_chunk = (masks_chunk * 255).to(dtype=torch.uint8)

    img_masks[(inds,) + spatial_inds] = masks_chunk
return img_masks
</code></pre>
<p>cfg = get_cfg()</p>
<h1>add project-specific config (e.g., TensorMask) here if you’re not running a model in detectron2’s core library</h1>
<p>cfg.merge_from_file(model_zoo.get_config_file(“COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml”))<br>
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5  # set threshold for this model</p>
<h1>Find a model from detectron2’s model zoo. You can use the <a href="https://dl.fbaipublicfiles" rel="noopener nofollow ugc">https://dl.fbaipublicfiles</a>… url as well</h1>
<p>cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(“COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml”)</p>
<h1>predictor = DefaultPredictor(cfg)</h1>
<p>model = build_model(cfg)  # returns a torch.nn.Module<br>
DetectionCheckpointer(model).load(cfg.MODEL.WEIGHTS)<br>
model.eval()</p>
<p>im = cv2.imread(“./input.jpg”)<br>
predictor = DefaultPredictor(cfg)<br>
outputs = predictor(im)<br>
mask = outputs[‘instances’].pred_masks.to(‘cpu’).numpy()<br>
v = Visualizer(im[:, :, ::-1], MetadataCatalog.get(cfg.DATASETS.TRAIN[0]), scale=1.2)<br>
v = v.draw_instance_predictions(outputs[“instances”].to(“cpu”))<br>
cv2.imwrite(‘./out.png’, v.get_image()[:, :, ::-1])<br>
plt.close()</p>
<p>original_image = im<br>
original_image = original_image[:, :, ::-1]<br>
transform_gen = T.ResizeShortestEdge(<br>
[cfg.INPUT.MIN_SIZE_TEST, cfg.INPUT.MIN_SIZE_TEST], cfg.INPUT.MAX_SIZE_TEST<br>
)<br>
height, width = original_image.shape[:2]<br>
image = transform_gen.get_transform(original_image).apply_image(original_image)<br>
image = torch.as_tensor(image.astype(“float32”).transpose(2, 0, 1))<br>
inputs = {“image”: image, “height”: height, “width”: width}<br>
outputs = model.inference([inputs], do_postprocess=False)<br>
inputs_1 = preprocess_image([inputs], cfg)<br>
image_sizes = inputs_1.image_sizes<br>
processed_outputs = custom_postprocess(outputs, [inputs], image_sizes)</p>
<h1>mask = outputs[‘instances’].pred_masks.to(‘cpu’).numpy()</h1>
<h1>v = Visualizer(im[:, :, ::-1], MetadataCatalog.get(cfg.DATASETS.TRAIN[0]), scale=1.2)</h1>
<h1>v = v.draw_instance_predictions(outputs[“instances”].to(“cpu”))</h1>
<h1>cv2.imwrite(‘./out.png’, v.get_image()[:, :, ::-1])</h1>
<p>mask = processed_outputs[0][‘instances’].pred_masks.to(‘cpu’).numpy()<br>
v = Visualizer(im[:, :, ::-1], MetadataCatalog.get(cfg.DATASETS.TRAIN[0]), scale=1.2)<br>
processed_outputs[0][‘instances’].pred_boxes.tensor= processed_outputs[0][‘instances’].pred_boxes.tensor.detach()<br>
processed_outputs[0][‘instances’].scores = processed_outputs[0][‘instances’].scores.detach()<br>
pdb.set_trace()<br>
v = v.draw_instance_predictions(processed_outputs[0][“instances”].to(“cpu”))<br>
cv2.imwrite(‘./out2.png’, v.get_image()[:, :, ::-1])</p>
</blockquote>
<p>So once we get <code>processed_outputs</code>, the mask can be fetched by:</p>
<blockquote>
<p>mask = processed_outputs[0]‘instances’].pred_masks.to(‘cpu’).numpy()</p>
</blockquote>
<p>The masks will be integers from 0 to 255 which is good enough. Here are the image from soft (non-binary) masks <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/e/9e11142f1dadddacfd37ced2971b53a5d6db796d.png" data-download-href="https://discuss.pytorch.org/uploads/default/9e11142f1dadddacfd37ced2971b53a5d6db796d" title="out2"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/e/9e11142f1dadddacfd37ced2971b53a5d6db796d_2_666x500.png" alt="out2" data-base62-sha1="myk1XpFgFwBDN2FGocoCjP3sAG1" width="666" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/e/9e11142f1dadddacfd37ced2971b53a5d6db796d_2_666x500.png, https://discuss.pytorch.org/uploads/default/original/3X/9/e/9e11142f1dadddacfd37ced2971b53a5d6db796d.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/9/e/9e11142f1dadddacfd37ced2971b53a5d6db796d.png 2x" data-dominant-color="5E6651"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">out2</span><span class="informations">768×576 812 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

hamzagorgulu: <p>I encountered the same problem and tried to resolve it with <a class="mention" href="/u/dangmanhtruong1995">@dangmanhtruong1995</a> s way. But I still get the binary result. I change the threshold and check it at the end of the function of paste_masks_in_image(), it seems to be non binary, however, the predictor still gives a non binary output. Since the detectron architecture is too complex, I am lost and open to suggestions.</p>