Cropping images into 128*128 images

chiheb48: <p>Hello,<br>
I’m working with Challenge Learned Image Compression dataset, it has different size images, I am using the code below to crop the images, one i use dataloader and want to start to train this error occurs:<br>
RuntimeError: each element in list of batch should be of equal size.</p>
<p>But I made sure that every block is 128<em>128</em>3.</p>
<p>import os<br>
import numpy as np<br>
import torch<br>
from PIL import Image<br>
from torch.utils.data import Dataset</p>
<p>class CustomDataset(Dataset):<br>
def <strong>init</strong>(self, root_dir, transform =None):<br>
self.root_dir = root_dir<br>
self.image_filenames = os.listdir(root_dir)<br>
self.transform = transform</p>
<pre><code>def __len__(self):
    return len(self.image_filenames)


def __getitem__(self, idx):
    image_path = os.path.join(self.root_dir, self.image_filenames[idx])
    image = Image.open(image_path).convert('RGB')
    image = np.array(image)
    
    h,w = image.shape[0], image.shape[1]

    mod_h = h%128
    mod_width = w%128

    pad1 = 128-mod_h
    pad2 = 128-mod_width

    quotient_h = (h+pad1)//128
    quotient_w = (w+pad2)//128

    pad = ((pad1, 0), (pad2, 0), (0, 0))
    # img = np.pad(img, pad, 'constant', constant_values=0) / 255
    image = np.pad(image, pad, mode="edge") 
    # Crop the image into 128x128 blocks
    height, width, _ = image.shape
    cropped_images = []
    for i in range(0, height, 128):
        for j in range(0, width, 128):
            cropped_image = image[i:i+128, j:j+128, :]
            if cropped_image.shape == (128, 128, 3):
                cropped_images.append(cropped_image)

    # Convert the cropped images to PyTorch tensors
    cropped_images = [torch.from_numpy(cropped_image.transpose((2, 0, 1))).float() / 255.0 for cropped_image in cropped_images]
   
    if self.transform:
        cropped_images = [self.transform(cropped_image) for cropped_image in cropped_images]
    return cropped_images
</code></pre>
<p>import torch.optim as optim<br>
from torch.utils.data import DataLoader<br>
from torchvision import transforms<br>
from torchvision.transforms import ToTensor</p>
<h1>
<a name="load-the-dataset-1" class="anchor" href="#load-the-dataset-1"></a>Load the dataset</h1>
<p>dataset = CustomDataset(‘C:/Users/mhund/Downloads/AE/data_1/train’,transform=transforms.Compose([<br>
transforms.ToPILImage(),<br>
transforms.ToTensor()<br>
]))</p>
<p>dataloader = DataLoader(dataset, batch_size=32, shuffle=True)</p>
<h1>
<a name="define-the-autoencoder-model-and-optimizer-2" class="anchor" href="#define-the-autoencoder-model-and-optimizer-2"></a>Define the autoencoder model and optimizer</h1>
<p>autoencoder = Autoencoder().cuda()<br>
optimizer = optim.Adam(autoencoder.parameters(), lr=1e-4)</p>
<h1>
<a name="train-the-autoencoder-3" class="anchor" href="#train-the-autoencoder-3"></a>Train the autoencoder</h1>
<p>for epoch in range(0,100):<br>
for i,batch  in enumerate(dataloader): # here it occurs<br>
print(i)</p>
<p>Any suggestions? Thank you</p>

eqy: <p>I think the issue is that you are effectively returning a “batch” of images (e.g., each crop can be considered a separate image) in your <code>__getitem__</code> function. When the dataloader is is assembling a batch from your dataset, it will call <code>__getitem__</code> 32 times (your batch size) so the batch it will collate will really look something like</p>
<pre><code class="lang-auto">[[image0crop1, image0crop2, ..., image0cropn], [image1crop1, image1crop2, ..., image1cropm], ..., [imagekcrop1, imagekcrop2, ..., imagekcropl]]
</code></pre>
<p>where n, m, l, can all be different.<br>
. If different images in your dataset are different sizes, then each could potentially have different numbers of crops, leading to the error. You could work around this by resizing each image in the dataset to a fixed resolution before cropping, or rethink how you want to batch your crops.</p>

chiheb48: <p>Thank you, I took your advice, and it solved every problem I had.</p>