Why are duplicate samples showing up in batches - custom PyTorch dataset for detection

toomanyeggs: <p>I’m fairly new to PyTorch, so maybe the answer will be obvious to someone else. I’ve got a custom dataset for object detection that returns batches of images and their objects (dictionaries containing boxes, labels, image_id, area, iscrowd). However, when I instantiate and load the custom dataset (see code below), duplicate samples (with the exact same transformations applied) show up in a batch. I’ve included a figure at the bottom showing the duplicate samples in batch.</p>
<p>Can anyone tell me what I’m doing wrong?</p>
<br>
<p><strong>Custom transforms applied in custom dataset’s <code>__getitem__():</code></strong></p>
<pre><code class="lang-auto">def transforms(image, boxes, labels, split):
    """
    Apply the transformations above.

    :param image: image, an ndarray 
    :param boxes: boxes in boundary coordinates, a tensor of dimensions (n_objects, 4)
    :param labels: labels of targets, a tensor of dimensions (n_objects)
    :param split: one of 'TRAIN' or 'TEST', since different sets of transformations are applied
    :return: transformed image, transformed boxes, transformed labels
    """
    
    np.random.seed(666)

    if split == 'TRAIN':
        new_image, new_boxes = Rescale(image, boxes, 
                                      output_size = (400))

        new_image, new_boxes, new_labels = RandomCrop(new_image, new_boxes, labels, output_size = (300))

        new_image, new_boxes = RandomHorizontalFlip(new_image, new_boxes)

    else:
        new_image, new_boxes = Rescale(image, boxes,
                                        output_size = (300))

    new_image = image_to_tensor(new_image)

    new_image = FT.normalize(new_image, mean, std)

    return new_image, new_boxes, new_labels
</code></pre>
<br>
<p><strong>Custom Dataset :</strong></p>
<pre><code class="lang-auto">class Dataset(Dataset):
    """Dataset"""

    def __init__(self, csv_file, root_dir, split):
        """
        Arguments:
            csv_file (string): Path to the CSV file with annotations. (image name, xmin, ymin, xmax, ymax, label)
            root_dir (string): Directory containing all images with annotations.
            split (string): Train or test, determines which transforms are applied during loading. 
        """
        self.df = pd.read_csv(csv_file)
        self.root_dir = root_dir

        self.split = split.upper()

        assert self.split in {'TRAIN', 'TEST'}


    def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()

        image_path = os.path.join(self.root_dir, self.df.iloc[idx, 0])
        image = io.imread(image_path)

        boxes = self.df[self.df['img_name'] == self.df.iloc[idx, 0]][['xmin', 'ymin', 'xmax', 'ymax']].values # why are xmin/xmax switched during loading?
        labels = self.df[self.df['img_name'] == self.df.iloc[idx, 0]]['label'].values

        labels = torch.LongTensor(labels) # (n_objects)

        image, boxes, labels = transforms(image, boxes, labels, split = self.split)

        boxes = torch.FloatTensor(boxes).reshape(-1, 4) # (n_objects, [xmin, ymin, xmax, ymax])

        target = {'boxes': boxes, 'labels': labels, 'image_id': self.df.iloc[idx, 0], 
                   'area': (boxes[:, 3] - boxes[:, 1]) * (boxes[:, 2] - boxes[:, 0]), 
                   'iscrowd': torch.zeros((len(labels,)), dtype=torch.int64)}

        return image, target

    def __len__(self):
        return len(self.df['img_name'].unique())


    def collate_fn(self, batch):
        """
        Since each image may have a different number of objects, we need a collate function (to be passed to the DataLoader).

        This describes how to combine these tensors of different sizes. Use lists.

        :param batch: an iterable of N sets from __getitem__()
        :return: a tensor of images, dictionary containing lists of varying-size tensors of polygons and targets
        """

        # collate function to combine images with varying numbers of objects into batches
        images = list()
        boxes = list()
        labels = list()
        image_ids = list()
        areas = list()
        iscrowds = list()

        for b in batch:
            images.append(b[0])
            boxes.append(b[1]['boxes'])
            labels.append(b[1]['labels'])
            image_ids.append(b[1]['image_id'])
            areas.append(b[1]['area'])
            iscrowds.append(b[1]['iscrowd'])

        images = torch.stack(images, dim=0)

        targets = {'boxes': boxes, 'labels': labels, 'image_ids': image_ids, 'areas': areas, 'iscrowds': iscrowds}

        return images, targets # tensor (N, 3, 300, 300), dictionary containing 2 lists of N tensors each
</code></pre>
<br>
<p><strong>Training dataset:</strong></p>
<pre><code class="lang-auto">train_dataset = Dataset(csv_file = '/train/label.csv',
                                root_dir = '/train/images/', 
                                split = 'TRAIN')
</code></pre>
<br>
<p><strong>DataLoader using custom collate function</strong></p>
<pre><code class="lang-auto">train_loader = torch.utils.data.DataLoader(train_dataset, batch_size = 16, shuffle = True, 
                                             collate_fn = train_dataset.collate_fn, num_workers = 0)
</code></pre>
<br>
<p><strong>Sample batch images and targets from data loader:</strong></p>
<pre><code class="lang-auto">images, targets = next(iter(train_loader))
</code></pre>
<br>
<p><strong>Plotting images, targets from sample batch:</strong></p>
<pre><code class="lang-auto">plt.figure(figsize = (16, 64))
for i in range(batch_size):
    ax = plt.subplot(16, 4, 1 + i)
    plot_detections(images[i], targets['boxes'][i], targets['labels'][i], ax = ax)
    plt.axis('off')
    plt.title(f"Sample {i + 1}")

plt.show()
</code></pre>
<br>
<p><strong>Output:</strong></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/e/1e7fc75bd203f74f73e426ede104ac327d93e112.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/1e7fc75bd203f74f73e426ede104ac327d93e112" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/e/1e7fc75bd203f74f73e426ede104ac327d93e112_2_507x500.jpeg" alt="image" data-base62-sha1="4lO5voiQlyHFtAmlMiZ3R3kvLDY" width="507" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/e/1e7fc75bd203f74f73e426ede104ac327d93e112_2_507x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/1/e/1e7fc75bd203f74f73e426ede104ac327d93e112_2_760x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/e/1e7fc75bd203f74f73e426ede104ac327d93e112_2_1014x1000.jpeg 2x" data-dominant-color="898274"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1278×1259 269 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Random <code>torchvision.transforms</code> will use the pseudo-random number generator to sample the random parameters of each transform. Repeating a transformation thus depends on the setup (i.e. the possible random values) and is often quite unlikely. I see that you are already seeding the <code>numpy</code> code in the <code>transforms</code> call so could you check if you are seeing also PyTorch somewhere?</p>

toomanyeggs: <p>I am not seeding PyTorch in the application. I didn’t realize random torchvision.transforms use a pseudo-random number generator to sample the random parameters of each transform. If I follow you correctly, what you’re saying is that applying <code>torchvision.transforms.functional.normalize</code> to the images at the end of the transforms function is overwriting the random seed set by <code>numpy</code> at the beginning of transforms with another random state?</p>
<p>Would you recommend I set PyTorch and <code>numpy</code> seeds equal, or only seed PyTorch?</p>

ptrblck: <p>No, <code>normalize</code> won’t overwrite any seeds and you would need to re-seed the PRNG manually.<br>
You should not set any seeds repeatedly if you want to get random outcomes. If you get stuck, post a minimal and executable code snippet reproducing the issue, please.</p>

toomanyeggs: <p>Got it figured out. Thanks for the help!</p>