Cropping batches at the same position

mhusseinsh: <p>As far as I understand, that <code>RandomCropping</code> is performing random crops of given size for each batch, is there a way, that I can strict the same location, a crop is taken from ?</p>

InnovArul: <p>we have <code>torchvision.transforms.CenterCrop</code>. Does that help you?</p>

mhusseinsh: <p>it always give a crop around the center, right ?, so if all images/tensors, are of same size, the crop place will be the same</p>

mhusseinsh: <p><a class="mention" href="/u/innovarul">@InnovArul</a>  I will try to say clarify exactly what I want</p>
<p>It is ok to have <code>RandomCrop</code> in my case, but what I want that the random position changes every 2nd batch</p>
<p>so for batch 1, the crop is taken from position <code>(x,y)</code>, and from batch 2, the same position <code>(x,y)</code>, but batch 3 and 4, will be from a different random position, and so on</p>
<p>I know this is something really special in my case, but is there any way to have this ?</p>

ptrblck: <p>You could use a counter to chose to resample the random crop parameters or reuse them.<br>
Here is a small (untested) example:</p>
<pre><code class="lang-python">class MyDataset(Dataset):
    def __init__(self, image_paths):
        self.image_paths = image_paths
        self.crop_indices = []

    def transform(self, image, resample):
        # Resize
        resize = transforms.Resize(size=(520, 520))
        image = resize(image)

        # Random crop
        if resample:
            self.crop_indices = transforms.RandomCrop.get_params(
                image, output_size=(512, 512))
        i, j, h, w = self.crop_indices
        image = TF.crop(image, i, j, h, w)

        # Random horizontal flipping
        if random.random() &gt; 0.5:
            image = TF.hflip(image)

        # Random vertical flipping
        if random.random() &gt; 0.5:
            image = TF.vflip(image)

        # Transform to tensor
        image = TF.to_tensor(image)
        return image

    def __getitem__(self, index):
        image = Image.open(self.image_paths[index])
        resample = index % 2 == 0
        x = self.transform(image, resample)
        return x 

    def __len__(self):
        return len(self.image_paths)
</code></pre>
<p>Let me know, if that works for you.</p>

mhusseinsh: <p>Hello <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
Sorry for my late reply</p>
<p>Can <code>RandomCrop</code> works on tensors not images ?<br>
I will try to explain why</p>
<p>I have like this</p>
<pre><code class="lang-auto">def get_data_loader_folder(input_folder, batch_size, train, new_size=None,
                           height=256, width=256, num_workers=4, crop=True):
    transform_list = [transforms.ToTensor(),
                      transforms.Normalize((0.5, 0.5, 0.5),
                                           (0.5, 0.5, 0.5))]
    transform_list = [transforms.RandomCrop((height, width))] + transform_list if crop else transform_list
    transform_list = [transforms.Resize((256, 256))] + transform_list if new_size is not None else transform_list
    transform_list = [transforms.RandomHorizontalFlip()] + transform_list if train else transform_list
    transform = transforms.Compose(transform_list)
    dataset = ImageFolder(input_folder, transform=transform)
    loader = DataLoader(dataset=dataset, batch_size=batch_size, shuffle=train, drop_last=True, num_workers=num_workers)
    return loader

class ImageFolder(data.Dataset):

    def __init__(self, root, transform=None, return_paths=False,
                 loader=default_loader):
        #imgs = sorted(make_dataset(root))
        # shuffle implicit pairs
        if "test" in root: 
            imgs = sorted(make_dataset(root))
        else:
            imgs = shuffle_pairs(sorted(make_dataset(root)))
        if len(imgs) == 0:
            raise(RuntimeError("Found 0 images in: " + root + "\n"
                               "Supported image extensions are: " +
                               ",".join(IMG_EXTENSIONS)))

        self.root = root
        self.imgs = imgs
        self.transform = transform
        self.return_paths = return_paths
        self.loader = loader

    def __getitem__(self, index):
        path = self.imgs[index]
        img = self.loader(path)
        if self.transform is not None:
            img = self.transform(img)
        if self.return_paths:
            return img, path
        else:
            return img

    def __len__(self):
        return len(self.imgs)
</code></pre>
<p>So when I call <code>get_data_loader_folder(</code>), I set the <code>crop=False</code>, because I don’t want <code>RandomCrop</code> in all batches, but as I told you before, that I want to crop at the same location for every 2 sequential batches</p>
<p>in the <code>class ImageFolder()</code><br>
here is what happens, it applies all the transformation composed based on the transform_list, then it returns a tensor with the transformations applied … back again to the RandomCrop</p>
<p>Is there a way to apply it here ?</p>
<pre><code class="lang-auto">    def __getitem__(self, index):
        path = self.imgs[index]
        img = self.loader(path)
        if self.transform is not None:
            img = self.transform(img)
            ##RANDOM CROP##
        if self.return_paths:
            return img, path
</code></pre>
<p>By taking into the consideration that the returned <code>img</code> from <code>self.transform</code> is a tensor now … and if not ? do you have any tips or hints how to achieve what I want in this case ?</p>
<p>Thanks a lot … your help is totally appreciated</p>

mhusseinsh: <p>hey <a class="mention" href="/u/ptrblck">@ptrblck</a>, sorry to disturb<br>
any help here ?</p>

ptrblck: <p>Hi Mostafa,<br>
sorry for the late reply. I’ve seen the thread but have completely forgotten to answer. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=6" title=":wink:" class="emoji" alt=":wink:"></p>
<p><code>RandomCrop</code> works on images and your current code should also work as you put all image transformations before <code>ToTensor</code>. So if no image transformations are given, your data will just be transformed to a tensor and normalized. Otherwise it will be cropped randomly etc. before the <code>ToTensor</code> transform.<br>
Your code makes sense and I would like to stick to it.<br>
Therefore we would need to create our own <code>RandomCrop</code> class.<br>
We can derive from <code>transforms.RandomCrop</code> and just add the “counter” from my previous example so that the crop indices will be resampled in every second iteration:</p>
<pre><code class="lang-python">class MyRandomCrop(transforms.RandomCrop):
    def __init__(self, size, padding=0, pad_if_needed=False):
        super(MyRandomCrop, self).__init__(size, padding, pad_if_needed)
        self.counter = 0
        self.crop_indices = []
        
    def __call__(self, img):
        if self.padding is not None:
            img = F.pad(img, self.padding, self.fill, self.padding_mode)

        # pad the width if needed
        if self.pad_if_needed and img.size[0] &lt; self.size[1]:
            img = F.pad(img, (self.size[1] - img.size[0], 0), self.fill, self.padding_mode)
        # pad the height if needed
        if self.pad_if_needed and img.size[1] &lt; self.size[0]:
            img = F.pad(img, (0, self.size[0] - img.size[1]), self.fill, self.padding_mode)
        
        resample = self.counter % 2 == 0
        self.counter += 1
        if resample:
            self.crop_indices = self.get_params(img, self.size)
        i, j, h, w = self.crop_indices

        return F.crop(img, i, j, h, w)


transform = transforms.Compose([
    MyRandomCrop((10, 10)),
    transforms.ToTensor()
])
        
class MyDataset(Dataset):
    def __init__(self, image_paths, transform):
        self.image_paths = image_paths
        self.transform = transform
    
    def __getitem__(self, index):
        image = Image.open(self.image_paths[index])
        x = self.transform(image)
        return x
    
    def __len__(self):
        return len(self.image_paths)
</code></pre>
<p>Depending on the <code>torchvision</code> version you are using you might need to adapt the arguments to <code>RandomCrop</code>.<br>
Let me know, if that works for you!</p>

mhusseinsh: <p>no problem <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
so you say that I create this class on my own, then here I do like this ?</p>
<pre><code class="lang-auto">def get_data_loader_folder(input_folder, batch_size, train, new_size=None,
                           height=256, width=256, num_workers=4, crop=True):
    transform_list = [transforms.ToTensor(),
                      transforms.Normalize((0.5, 0.5, 0.5),
                                           (0.5, 0.5, 0.5))]
    transform_list = [MyRandomCrop((height, width))] + transform_list if crop else transform_list
    transform_list = [transforms.Resize((256, 256))] + transform_list if new_size is not None else transform_list
    transform_list = [transforms.RandomHorizontalFlip()] + transform_list if train else transform_list
    transform = transforms.Compose(transform_list)
    dataset = ImageFolder(input_folder, transform=transform)
    loader = DataLoader(dataset=dataset, batch_size=batch_size, shuffle=train, drop_last=True, num_workers=num_workers)
    return loader
</code></pre>
<p>correct me if I am wrong</p>

ptrblck: <p>Exactly! Let me know if you get any errors.</p>

mhusseinsh: <p>but in the arguments, do I need to path width and height as I am doing ?</p>

ptrblck: <p>Yes you are using it right.<br>
I used up the current master branch of <code>torchvision</code> and tried to post code for <code>0.2.1</code> forgetting about some fixes.<br>
Here is the fixed code:</p>
<pre><code class="lang-python">import torchvision.transforms.functional as TF

class MyRandomCrop(transforms.RandomCrop):
    def __init__(self, size, padding=0, pad_if_needed=False):
        super(MyRandomCrop, self).__init__(size, padding, pad_if_needed)
        self.counter = 0
        self.crop_indices = []
        
    def __call__(self, img):
        if self.padding &gt; 0:
            img = TF.pad(img, self.padding)

        # pad the width if needed
        if self.pad_if_needed and img.size[0] &lt; self.size[1]:
            img = TF.pad(img, (int((1 + self.size[1] - img.size[0]) / 2), 0))
        # pad the height if needed
        if self.pad_if_needed and img.size[1] &lt; self.size[0]:
            img = TF.pad(img, (0, int((1 + self.size[0] - img.size[1]) / 2)))
        
        resample = self.counter % 2 == 0
        self.counter += 1
        if resample:
            self.crop_indices = self.get_params(img, self.size)
        i, j, h, w = self.crop_indices
        print('Using {} {} {} {}'.format(i, j, h, w))

        return TF.crop(img, i, j, h, w)
</code></pre>
<p>Let me know, if that works now!</p>

mhusseinsh: <p><a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>I tried it, and it is always giving the same <code>(i,j)</code></p>
<pre><code class="lang-auto">Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
Using 0 0 256 256
</code></pre>

mhusseinsh: <p>This is how I have done it</p>
<pre><code class="lang-auto">def get_data_loader_folder(input_folder, batch_size, train, new_size=None,
                           height=256, width=256, num_workers=4, crop=True):
    transform_list = [transforms.ToTensor(),
                      transforms.Normalize((0.5, 0.5, 0.5),
                                           (0.5, 0.5, 0.5))]
    #transform_list = [transforms.RandomCrop((height, width))] + transform_list if crop else transform_list
    transform_list = [MyRandomCrop((height, width))] + transform_list
    transform_list = [transforms.Resize((256, 256))] + transform_list if new_size is not None else transform_list
    transform_list = [transforms.RandomHorizontalFlip()] + transform_list if train else transform_list
    transform = transforms.Compose(transform_list)
    dataset = ImageFolder(input_folder, transform=transform)
    loader = DataLoader(dataset=dataset, batch_size=batch_size, shuffle=train, drop_last=True, num_workers=num_workers)
    return loader

class ImageFolder(data.Dataset):

    def __init__(self, root, transform=None, return_paths=False,
                 loader=default_loader):
        #imgs = sorted(make_dataset(root))
        # shuffle implicit pairs
        if "test" in root: 
            imgs = sorted(make_dataset(root))
        else:
            imgs = shuffle_pairs(sorted(make_dataset(root)))
        if len(imgs) == 0:
            raise(RuntimeError("Found 0 images in: " + root + "\n"
                               "Supported image extensions are: " +
                               ",".join(IMG_EXTENSIONS)))

        self.root = root
        self.imgs = imgs
        self.transform = transform
        self.return_paths = return_paths
        self.loader = loader

    def __getitem__(self, index):
        path = self.imgs[index]
        img = self.loader(path)
        if self.transform is not None:
            img.save('x_a1.png')
            img = self.transform(img)
            vutils.save_image(img, 'x_a2.png', nrow=1)
            #exit()
        if self.return_paths:
            return img, path
        else:
            return img

    def __len__(self):
        return len(self.imgs)

# RandomCrop modified
class MyRandomCrop(transforms.RandomCrop):
    def __init__(self, size, padding=0, pad_if_needed=False):
        super(MyRandomCrop, self).__init__(size, padding, pad_if_needed)
        self.counter = 0
        self.crop_indices = []
        
    def __call__(self, img):
        if self.padding &gt; 0:
            img = TF.pad(img, self.padding)

        # pad the width if needed
        if self.pad_if_needed and img.size[0] &lt; self.size[1]:
            img = TF.pad(img, (int((1 + self.size[1] - img.size[0]) / 2), 0))
        # pad the height if needed
        if self.pad_if_needed and img.size[1] &lt; self.size[0]:
            img = TF.pad(img, (0, int((1 + self.size[0] - img.size[1]) / 2)))
        
        resample = self.counter % 2 == 0
        self.counter += 1
        if resample:
            self.crop_indices = self.get_params(img, self.size)
        i, j, h, w = self.crop_indices
        print('Using {} {} {} {}'.format(i, j, h, w))

        return TF.crop(img, i, j, h, w)
</code></pre>

ptrblck: <p>What is your image size? The random crop size should be smaller than the image size.<br>
Also the <code>Resize</code> transformation won’t do anything, if your crops are already in the same size.</p>

mhusseinsh: <p>I am loading 2 datasets,<br>
one has size <code>(400, 300)</code>, and the other has size <code>(800,600)</code></p>
<p>concerning the resizing, yes for sure, it won’t do anything, i know this, it can be ignored</p>

ptrblck: <p>That’s strange. I’m testing it with images of size <code>[3, 690, 334]</code> and it seems to work:</p>
<pre><code class="lang-auto">Using 34 337 256 256
Using 34 337 256 256
Using 72 102 256 256
Using 72 102 256 256
Using 57 382 256 256
Using 57 382 256 256
</code></pre>

mhusseinsh: <p>it is weird, I just copied what you wrote without editing anything</p>

ptrblck: <p>Could you run this code and see it the last line returns different values?</p>
<pre><code class="lang-python">img = transforms.ToPILImage()(torch.randn(3, 600, 600))
crop = MyRandomCrop((256, 256))
crop.get_params(img, (256, 256))
</code></pre>
<p>If so, could you try this afterwards:</p>
<pre><code class="lang-auto">cropped = crop(img)
</code></pre>

mhusseinsh: <p>I did that</p>
<pre><code class="lang-auto"> for i in range(10):
        img = transforms.ToPILImage()(torch.randn(3, 600, 600))
        crop = MyRandomCrop((256, 256))
        print(crop.get_params(img, (256, 256)))
</code></pre>
<p>and I got</p>
<pre><code class="lang-auto">(228, 344, 256, 256)
(75, 317, 256, 256)
(190, 65, 256, 256)
(152, 243, 256, 256)
(74, 312, 256, 256)
(246, 237, 256, 256)
(137, 43, 256, 256)
(29, 186, 256, 256)
(320, 141, 256, 256)
(334, 326, 256, 256)
</code></pre>