Extracting Bounding Box Coordinates from mask

Sabbir_Ahmed: <p>I have a model that predicts a binary mask. I want to extract the bounding box coordinates without calling cpu or numpy. Is there any efficient way to do this in pytorch??</p>

spanev: <p>Hi <a class="mention" href="/u/sabbir_ahmed">@Sabbir_Ahmed</a>,</p>
<p>What format do you want the coordinates to be extracted in?</p>
<p>You can access the raw data pointer with</p>
<pre><code class="lang-auto">t.data_ptr()
</code></pre>
<p>then you can cast/interpret it as a ctype with <code>ctypes</code>:</p>
<pre><code class="lang-auto">import ctypes
c_type_pointer = ctypes.c_void_p(t.data_ptr())
# Then you can process or copy from c_type_pointer
</code></pre>

Sabbir_Ahmed: <pre><code class="lang-auto">#I have a predicted mask
pred_mask = model(input)

# I just want an efficient function that maps the mask to coordinates of bounding boxes without calling cpu 
coordinates = function(pred_mask)
</code></pre>
<p>And i am sorry, but i don’t have a strong understanding of raw data pointer.<br>
Could u please suggest a code snippet of the function in pytorch <a class="mention" href="/u/spanev">@spanev</a>.</p>

Qing_En: <p>You can reference the code below</p>
<pre><code class="lang-auto">def extract_bboxes(mask):

    """Compute bounding boxes from masks.

    mask: [height, width, num_instances]. Mask pixels are either 1 or 0.

 

    Returns: bbox array [num_instances, (y1, x1, y2, x2)].

    """

    boxes = np.zeros([mask.shape[-1], 4], dtype=np.int32)

    for i in range(mask.shape[-1]):

        m = mask[:, :, i]

        # Bounding box.

        horizontal_indicies = np.where(np.any(m, axis=0))[0]

        print("np.any(m, axis=0)",np.any(m, axis=0))

        print("p.where(np.any(m, axis=0))",np.where(np.any(m, axis=0)))

        vertical_indicies = np.where(np.any(m, axis=1))[0]

        if horizontal_indicies.shape[0]:

            x1, x2 = horizontal_indicies[[0, -1]]

            y1, y2 = vertical_indicies[[0, -1]]

            # x2 and y2 should not be part of the box. Increment by 1.

            x2 += 1

            y2 += 1

        else:

            # No mask for this instance. Might happen due to

            # resizing or cropping. Set bbox to zeros

            x1, x2, y1, y2 = 0, 0, 0, 0

        boxes[i] = np.array([y1, x1, y2, x2])

    return boxes.astype(np.int32)
</code></pre>
<p>Hope it will help you~ <a class="mention" href="/u/sabbir_ahmed">@Sabbir_Ahmed</a></p>

Jaideep_Valani: <aside class="quote no-group" data-username="Qing_En" data-post="4" data-topic="61179">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/qing_en/48/3701_2.png" class="avatar"> Qing_En:</div>
<blockquote>
<p><code>np.any(m, axis=0)</code></p>
</blockquote>
</aside>
<p><span class="mention">@QingEn</span><br>
Thanks if i want to also capture along with bounding box some area around it … what should i do.<br>
intent is to capture bounding box image portion and vicinity around it</p>

Jean_Da_Rolt: <p>Using skimage, np and pandas (pandas is not necessary:</p>
<pre><code class="lang-auto">from typing import Dict, List, Optional

import numpy as np
import pandas as pd
from skimage.measure import label, regionprops
import torch as th

def simple_boxing(
    classmasks: th.Tensor,
    file_ids: Optional[List[int]] = None,
    channel2class: Dict[int, int] = None,
) -&gt; pd.DataFrame:
    """
    Convert a tensor of shape (B, C, H, W) to a pandas DataFrame with bounding boxes.

    Args:
        classmasks: Tensor with shape (batch_size, channels, height, width)
    """
    assert classmasks.dtype == th.bool, "classmasks data type must be boolean."
    assert (
        len(classmasks.shape) == 4
    ), f"classmasks with shape {classmasks.shape} should have 4 dimensions"

    # to reduce timing, skip all empty channels
    has_detections = (
        classmasks.view(classmasks.shape[0:2] + (-1,))
        .any(dim=2)
        .nonzero()
        .cpu()
        .numpy()
    )

    boxes_list = []
    classmasks = classmasks.cpu().numpy()

    for img_idx, channel_idx in has_detections:
        labels = label(classmasks[img_idx, channel_idx], background=0, connectivity=2)
        props = regionprops(labels)
        file_id = file_ids[img_idx] if file_ids is not None else img_idx
        boxes_list.extend([(file_id, *x.bbox, channel_idx) for x in props])

    boxes = pd.DataFrame(boxes_list, columns=COLUMN_NAMES_SEGMENTATION)

    if channel2class is not None:
        boxes["class_id"] = boxes["channel_id"].map(channel2class)

    return boxes
</code></pre>
<p>Just in case someone needs it.</p>

Peter_O_Connor: <p>The above answers address how to do it when you just want one box per mask.  If you stumbled upon this question looking for how to predict multiple boxes per mask (e.g. when regions are disconnected) see:</p>
<aside class="onebox stackexchange" data-onebox-src="https://stackoverflow.com/questions/58419893/generating-bounding-boxes-from-heatmap-data">
  <header class="source">

      <a href="https://stackoverflow.com/questions/58419893/generating-bounding-boxes-from-heatmap-data" target="_blank" rel="noopener nofollow ugc">stackoverflow.com</a>
  </header>

  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/11699125/brokenfulcrum" target="_blank" rel="noopener nofollow ugc">
    <img alt="brokenfulcrum" src="https://www.gravatar.com/avatar/8e56ef6070f444db4ae38a95423ca71e?s=256&amp;d=identicon&amp;r=PG&amp;f=1" class="thumbnail onebox-avatar" width="256" height="256">
  </a>

<h4>
  <a href="https://stackoverflow.com/questions/58419893/generating-bounding-boxes-from-heatmap-data" target="_blank" rel="noopener nofollow ugc">Generating bounding boxes from heatmap data</a>
</h4>

<div class="tags">
  <strong>python, numpy, opencv, object-detection</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/11699125/brokenfulcrum" target="_blank" rel="noopener nofollow ugc">
    brokenfulcrum
  </a>
  on <a href="https://stackoverflow.com/questions/58419893/generating-bounding-boxes-from-heatmap-data" target="_blank" rel="noopener nofollow ugc">06:48PM - 16 Oct 19 UTC</a>
</div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<aside class="onebox stackexchange" data-onebox-src="https://stackoverflow.com/questions/73282135/computing-bounding-boxes-from-a-mask-image-tensorflow-or-other">
  <header class="source">

      <a href="https://stackoverflow.com/questions/73282135/computing-bounding-boxes-from-a-mask-image-tensorflow-or-other" target="_blank" rel="noopener nofollow ugc">stackoverflow.com</a>
  </header>

  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/851699/peter" target="_blank" rel="noopener nofollow ugc">
    <img alt="Peter" src="https://i.stack.imgur.com/jDiPy.png?s=256&amp;g=1" class="thumbnail onebox-avatar" width="256" height="256">
  </a>

<h4>
  <a href="https://stackoverflow.com/questions/73282135/computing-bounding-boxes-from-a-mask-image-tensorflow-or-other" target="_blank" rel="noopener nofollow ugc">Computing Bounding Boxes from a Mask-Image (Tensorflow or other)</a>
</h4>

<div class="tags">
  <strong>tensorflow, computer-vision, image-segmentation, tensorflow-lite</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/851699/peter" target="_blank" rel="noopener nofollow ugc">
    Peter
  </a>
  on <a href="https://stackoverflow.com/questions/73282135/computing-bounding-boxes-from-a-mask-image-tensorflow-or-other" target="_blank" rel="noopener nofollow ugc">05:50PM - 08 Aug 22 UTC</a>
</div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


Jinx_L: <p>Suppose mask_np is the numpy array from a binary mask, then the following  codes will help you obtain the bounding box coordinates:</p>
<pre><code class="lang-python"># the fuction
def bounding_box(img):
    rows = np.any(img, axis=1)
    cols = np.any(img, axis=0)
    rmin, rmax = np.where(rows)[0][[0, -1]]
    cmin, cmax = np.where(cols)[0][[0, -1]]
    return rmin, rmax, cmin, cmax # y1, y2, x1, x2 

# process the mask array with the above function
bounding_box(img=mask_up)

</code></pre>