Mask_b_box errors

username: Sushmanth_reddy_Mere
cooked: <p>I have been extracting the bounding box from masks to give and train segment anything model, I have a 16 uint datatype image,masks are also 16 uint image.</p>
<pre><code class="lang-auto">class segmentationdataset(Dataset):
    def __init__(self, csv, augmentation =None, transform_image = None, transform_label = None,transform_bbox= None):
        self.df=pd.read_csv(csv)
        self.ids = self.df["file_ids"]
        self.transform_image = transform_image
        self.transform_label = transform_label
        self.transform_bbox = transform_bbox
        self.augmentation = augmentation
        
    def __getitem__(self,idx):
        image=np.array(Image.open("/kaggle/input/nucleus-data-c-elegans/nucleus_data/features/F"+self.ids[idx]))
        mask=np.array(Image.open("/kaggle/input/nucleus-data-c-elegans/nucleus_data/segmentation_maps/L"+self.ids[idx]))
        b_box=np.array(Image.open("/kaggle/input/nucleus-data-c-elegans/nucleus_data/segmentation_maps/L"+self.ids[idx]))
        if self.augmentation is not None:
            augmented=self.augmentation(image=image,mask=mask,b_box=b_box)
            image = augmented["image"]
            mask = augmented["mask"]
            b_box = augmented["b_box"]
            
        image = self.transform_image(image)
        mask = self.transform_label(mask)
        b_box = self.transform_bbox(b_box)
        b_box = np.array(b_box, dtype=np.uint8)  # Convert PIL Image to NumPy array
        b_box = torch.as_tensor(b_box)
        obj_ids = torch.unique(b_box)
        obj_ids = obj_ids[1:]
        b_boxes = b_box == obj_ids[:, None, None]
        boxes = masks_to_boxes(b_boxes)
        return image.float(), mask.float(), boxes.float()
    
    def __len__(self):
        return len(self.ids)
              

</code></pre>
<p>this is dataset class</p>
<p>here are data augmentation which , I am doing</p>
<pre><code class="lang-auto">full_dataset = segmentationdataset(csv = "file_ids.csv",
                                    augmentation =  Compose([
                                                            #GridDistortion(p=0.5),
                                                            Transpose(p=0.5),
                                                            VerticalFlip(p=0.5),
                                                            HorizontalFlip(p=0.5),
                                                            RandomRotate90(p=0.5),
                                                            ShiftScaleRotate(p=0.1),
                                                            OpticalDistortion(distort_limit=0.3, shift_limit=0.3, p=1),
                                                            ]),

                                    transform_image = transforms.Compose([ 
                                                                    
                                                                    transforms.ToPILImage(),
                                                                    ToTensor(),
                                                                    transforms.RandomApply([AddGaussianNoise( mean = 0.5,std= 0.05)], p=0.5)
                                                                ]),                                  
                                    transform_label = transforms.Compose([ 
                                                                                                      
                                                                    transforms.ToPILImage(),
                                                                    ToTensor(),

                                                                ]),
                                   transform_bbox = transforms.Compose([ 
                                                                                                      
                                                                    transforms.ToPILImage(),
                                                                    ToTensor()

                                                                ]))
</code></pre>
<pre><code class="lang-auto">train_batch_size = 32
train_size = int(0.8* len(full_dataset))  ## 80/20 split
test_size = len(full_dataset) - train_size
train_dataset, test_dataset = random_split(full_dataset, [train_size, test_size])

train_loader = DataLoader(dataset=train_dataset, batch_size=16, shuffle=True)
val_loader = DataLoader(dataset=test_dataset, batch_size= 4, shuffle = True)

print(len(train_loader), "batches ")
print(len(val_loader), " batches ")

for image, mask, bboxes in train_loader:
    # img_embed: (B, 256, 64, 64), gt2D: (B, 1, 256, 256), bboxes: (B, 4)
    print(f"{image.shape=}, {mask.shape=}, {bboxes.shape=}")
    break
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/1/41c05fd1c84d66170a008a836d1fd6bc0eaf9cec.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/41c05fd1c84d66170a008a836d1fd6bc0eaf9cec" title="Screenshot 2023-06-14 at 12.54.14 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/41c05fd1c84d66170a008a836d1fd6bc0eaf9cec_2_690x347.jpeg" alt="Screenshot 2023-06-14 at 12.54.14 PM" data-base62-sha1="9nFc7U7upqieVKDprC0rdE7F08k" width="690" height="347" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/41c05fd1c84d66170a008a836d1fd6bc0eaf9cec_2_690x347.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/41c05fd1c84d66170a008a836d1fd6bc0eaf9cec_2_1035x520.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/41c05fd1c84d66170a008a836d1fd6bc0eaf9cec_2_1380x694.jpeg 2x" data-dominant-color="F4D8D5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2023-06-14 at 12.54.14 PM</span><span class="informations">1920×968 155 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/7/57d2ff27402e2ee6ecb86bf06780802e8e92bce5.png" data-download-href="https://discuss.pytorch.org/uploads/default/57d2ff27402e2ee6ecb86bf06780802e8e92bce5" title="Screenshot 2023-06-14 at 12.54.26 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/7/57d2ff27402e2ee6ecb86bf06780802e8e92bce5_2_690x296.png" alt="Screenshot 2023-06-14 at 12.54.26 PM" data-base62-sha1="cwVB5HGKq8OmC0rZFcTq6Hc3IyN" width="690" height="296" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/7/57d2ff27402e2ee6ecb86bf06780802e8e92bce5_2_690x296.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/7/57d2ff27402e2ee6ecb86bf06780802e8e92bce5_2_1035x444.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/5/7/57d2ff27402e2ee6ecb86bf06780802e8e92bce5_2_1380x592.png 2x" data-dominant-color="EBD4CE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2023-06-14 at 12.54.26 PM</span><span class="informations">2130×916 214 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>When I tried to extract the shapes of the image, mask, and bounding_box ,I encountered this error. What does the error mean??</p>
<p>This is what my mask looks like!<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/7/a/7a48ae4df56ca280799ec28d2d6b20c0dc936b77.png" alt="Screenshot 2023-06-06 at 10.14.11 AM" data-base62-sha1="hrLY6Arkcxh03hkd0D65XhdaH4P" width="306" height="218"></p>
<p>thanks in advance</p>

username: ptrblck
cooked: <p>Based on the error message it seems the bounding boxes are empty for some samples (indicated by the shape <code>[0, 4]</code>) while other samples seem to contain a lot of bounding boxes (indicated by the shape <code>[114, 4]</code>). Your current data loading pipeline then tries to stack these tensors and fails.<br>
The <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html">object detection tutorial</a> might be helpful as it shows how the dataset is implemented and how a <code>dict</code> containing all needed data is returned.</p>