TypeError: can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, complex64, complex128, int64, int32, int16, int8, uint8, and bool

username: k3vinli
cooked: <p>Hi, I am trying to create a segmentation model with a pretrained FCN ResNet50 model but I am getting an error from the segmentation reference code. I’m not sure if I am creating the targets correctly in my __getitem__ function in my dataloader class and was wondering if anyone could help.</p>
<p>I was trying to follow this tutorial <a href="https://github.com/voxel51/fiftyone-examples/blob/master/examples/pytorch_detection_training.ipynb" rel="noopener nofollow ugc">PyTorch object detection model training</a> from fiftyone. I know the tutorial is for detection but I thought I could adapt it to a segmentation model. I’m also using the semantic segmentation reference code in the main branch from the vision repo <a href="https://github.com/pytorch/vision/tree/main/references/segmentation" rel="noopener nofollow ugc">here</a>. The error is happening in the transforms.py file from the reference in the PILToTensor class.</p>
<pre><code class="lang-auto">     78 def __call__(self, image, target):
     79     image = F.pil_to_tensor(image)
---&gt; 80     target = torch.as_tensor(np.array(target), dtype=torch.int64)
     81     return image, target
</code></pre>
<p>Here is my __getitem__ code for my custom Dataloader</p>
<pre><code class="lang-auto">    def __getitem__(self, idx):
        img_path = self.img_paths[idx]
        sample = self.samples[img_path]
        metadata = sample.metadata
        frame_size = (metadata['width'], metadata['height'])
        print("frame_size", frame_size)
        img = Image.open(img_path).convert("RGB")

        boxes = []
        labels = []
        area = []
        iscrowd = []
        masks = []
        detections = sample[self.gt_field].detections
        for det in detections:
            category_id = self.labels_map_rev[det.label]
            coco_obj = fouc.COCOObject.from_label(
                det, metadata, category_id=category_id,
            )
            x, y, w, h = coco_obj.bbox
            boxes.append([x, y, x + w, y + h])
            labels.append(coco_obj.category_id)
            area.append(coco_obj.area)
            iscrowd.append(coco_obj.iscrowd)
            masks.append(det.to_segmentation(frame_size=frame_size).mask)
            # masks.append(det.mask)

        target = {}
        target["boxes"] = torch.as_tensor(boxes, dtype=torch.float32)
        target["labels"] = torch.as_tensor(labels, dtype=torch.int64)
        target["image_id"] = torch.as_tensor([idx])
        target["area"] = torch.as_tensor(area, dtype=torch.float32)
        target["iscrowd"] = torch.as_tensor(iscrowd, dtype=torch.int64)

        masks=torch.as_tensor(masks, dtype=torch.float)
        print("mask shape", masks.shape)
        target["masks"] = masks

        if self.transforms is not None:
            img, target = self.transforms(img, target)

        return img, target 
</code></pre>
<p>I tried debugging it and this is the structure of the target dict.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/8/6/86619c5a06b51564fbdabe1bdca5a924799be7fe.png" alt="image" data-base62-sha1="jaN6T5n3g5wH8Kd8JMAcc9jfxIW" width="485" height="335"></p>
<p>I saw in another post that everything has to be the same length to be turned into a tensor but I am not sure how to do that without modifying the target dict.</p>