Batch shape coming out to be wrong when iterating through dataloader

username: sparshgarg23
cooked: <p>While iterating through the dataloader,I noticed that for all samples,the shape of the image and y was coming out to be [16,256<em>256] and [16,3] but at the end the output of the batch shape came out to be [12,256</em>256] and [12,3].<br>
Why was the batch size set to 12 and not 16.<br>
Code is listed below</p>
<pre><code class="lang-auto">   dataset=NumbersDataset(paths,batch_size=16,shuffle=True)
    print(len(dataset)) #449900
    dataset_loader=DataLoader(dataset,batch_size=16,shuffle=True,num_workers=4)
    for image,coord in dataset_loader:
        print("Batch of images has shape:",image.shape)
        print("Batch of coordinates has shape:",coord.shape)

Batch of images has shape: torch.Size([16, 65536])
Batch of coordinates has shape: torch.Size([16, 3])
Batch of images has shape: torch.Size([16, 65536])
Batch of coordinates has shape: torch.Size([16, 3])
Batch of images has shape: torch.Size([12, 65536])
Batch of coordinates has shape: torch.Size([12, 3])
</code></pre>
<p>My custom dataset is as follows</p>
<pre><code class="lang-auto">class NumbersDataset(Dataset):
    def __init__(self,paths,batch_size=16,shuffle=True):
        self.img_dir='blender_files/Model_0/images'
        self.coord_files=paths
        self.batch_size=batch_size
        self.shuffle=shuffle
        self.transforms=None
        self.image_shape=(256,256)
        self.data=[]
        img_list=glob.glob("blender_files/Model_0/images/*")
        for img_path in img_list:
            for file in self.coord_files:
                self.data.append([img_path,file])

        
        
        
        

    def __len__(self):
        self.num_batches=int(np.floor(len(self.coord_files)/self.batch_size))
        return len(self.data)

    def __getitem__(self, idx):
        
        if self.transforms is not None:
            self.image=self.transform(self.image)
         
        img_path,coord_path=self.data[idx]
        image=cv2.imread(img_path,0)
        image=cv2.resize(image,self.image_shape)
        coord=np.load(coord_path)
        return image.ravel(),coord
        #for i in range(len(self.coord_files)):
            #X_coord=np.load(coord_files[i])
            #return self.image.ravel(),X_coord
</code></pre>
<p>The number of files in paths is 89980 and there are 4 images</p>

username: ptrblck
cooked: <p>The last batch is smaller, since you have <code>5623</code> full batches (<code>5623 * 16 = 89968</code>) and <code>12</code> samples at the end (<code>89980 % 16 = 12</code>). To drop the last batch you could use <code>drop_last=True</code> in the <code>DataLoader</code>.</p>

username: sparshgarg23
cooked: <p>that makes sense,thanks</p>

username: sparshgarg23
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>  I have another question while training the autoencoder on my dataset,at the end of the training ,the output came out to be in the following format</p>
<pre><code class="lang-auto">[ 70.6182,  70.8503,  70.6278,  ..., 243.5364, 330.0468,   5.6832]
</code></pre>
<p>But at evaluation stage,when I run the autoencoder on a single image and a random vector of 3x1,the autoencoder generates the following ouptut</p>
<pre><code class="lang-auto">tensor([[3.2402e+16, 3.4111e+16, 3.2839e+16,  ..., 4.4640e+16, 4.4089e+16,
         7.7656e+15]], device='cuda:0', dtype=torch.float64)
</code></pre>
<p>As you can see the values are not close to the ones obtained during training,any ideas why this is happening.</p>

username: ptrblck
cooked: <p>If calling <code>model.eval()</code> yields worse results on the same input data (or similar data), it could mean that e.g. the running stats of the batchnorm layers are off and you could change the momentum or check if you are dealing with data samples from different distributions.</p>

username: sparshgarg23
cooked: <p>hmm,i checked the data distribution ,its coming from the same set,plus during validation stage,as for momentum i will try to see.</p>