How to handle overlapping segmentation mask in v2 transformations

root_zero: <p>Segmentation masks are provided as polygons from which i am generating Binary masks for each class as different channel.<br>
For eg. I have 13 classes so my mask contain 13 channels with each channel as a class containing Binary mask.<br>
Now I want to pass this through image transformations. I tried using skimage transformation wrote a custom one, but due to scaling down it is changing binary mask from 0-1 to &lt;1 in floats.<br>
I tried to use v2.transformations but wrapping 13 channel array/tensor with Mask Datapoint is making 12GB RAM out of memory.<br>
I don’t want to store masks in RGB format because some of the masks are overlapping and it is causing color change which will in turn lead to incorrect label.<br>
Any insights how I can approach this problem?<br>
Thanks</p>
<p>Skimage based resizing for image and mask which is leading to change in Binary mask values due to interpolation.</p>
<pre data-code-wrap="python"><code class="lang-python">class Rescale(object):
    """Rescale the image in a sample to a given size.

    Args:
        output_size (tuple or int): Desired output size. If tuple, output is
            matched to output_size. If int, smaller of image edges is matched
            to output_size keeping aspect ratio the same.
    """

    def __init__(self, output_size):
        assert isinstance(output_size, (int, tuple))
        self.output_size = output_size

    def __call__(self, sample):
        image, landmarks = sample["img"],sample["mask"]

        h, w = image.shape[:2]
        if isinstance(self.output_size, int):
            if h &gt; w:
                new_h, new_w = self.output_size * h / w, self.output_size
            else:
                new_h, new_w = self.output_size, self.output_size * w / h
        else:
            new_h, new_w = self.output_size

        new_h, new_w = int(new_h), int(new_w)

        img = transform.resize(image, (new_h, new_w),interpolation=Image.NEAREST)

        landmarks = transform.resize(landmarks,(new_h,new_w),interpolation=Image.NEAREST)

        return {'img': img, 'mask': landmarks}
</code></pre>
<p>This is code in my dataset class where I am wrapping mask in Mask Datapoint.</p>
<pre data-code-wrap="python"><code class="lang-python">        mask = create_channel_masks((1024,1024,13),self.annotation_json[self.files[index]]["polygons"],self.annotation_json[self.files[index]]["syms"])
        # sample = {"img":img,"mask":mask}
        img, mask = self.transform(img,tv_tensors.Mask(mask,dtype=torch.uint8))
</code></pre>
<p>This is where I am generating multichannel masks</p>
<pre data-code-wrap="python"><code class="lang-python">def create_channel_masks(image_shape, polygons, syms,disease_color_map):
    
    mask = np.full(image_shape,0)

    for idx, (polygon, sym) in enumerate(zip(polygons, syms)):
      ch = disease_color_map[sym][0] - 1
      mask[:,:,ch] = mask[:,:,ch] | get_mask(polygon)

    return mask



def get_mask(points):
    mask = np.full((1024,1024),False)
    if len(points) ==0:
        return mask
    img = Image.new('L', (1024,1024), 0)
    poly = [(x,y) for x,y in points]
    ImageDraw.Draw(img).polygon(poly, outline=1, fill=1)
    curr_mask = np.array(img)

    return curr_mask
</code></pre>
<p>Image containing overlapping colored masks which I want to avoid so I created Multichannel Binary Masks(You can see white color is overlapped)<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/9/996090f8d55ddc6af231e8a5222b63fff129a969.png" data-download-href="https://discuss.pytorch.org/uploads/default/996090f8d55ddc6af231e8a5222b63fff129a969" title="36206"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/9/996090f8d55ddc6af231e8a5222b63fff129a969.png" alt="36206" data-base62-sha1="lSPWBNbnBYQHgJDhmEqMdHDRwlH" width="500" height="500" data-dominant-color="140809"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">36206</span><span class="informations">1024×1024 3.93 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>