Batch size match

nfreitas: <p>I’m trying to create a CNN to do image recognition, but my training loop is failing in the loss check with the error “Expected input batch_size (1) to match target batch_size (50)”. I see many examples in the internt like mine and a look for solution here and in other pages but could’t figure out how to solve it.<br>
Mainly I got that the output of the model has batch size 1 while the target has batch size 50 as I defined in the dataloader. That’s the code:</p>
<pre><code class="lang-auto">train_dl = torch.utils.data.DataLoader(dataset = train_ds,
                                       batch_size = 50,
                                      shuffle = True) 


test_dl = torch.utils.data.DataLoader(dataset = test_ds,
                                       batch_size = 50,
                                      shuffle = False)

class ConvNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(50, 5, 3, bias=False)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(5, 1, 3, bias=False)
        self.fc1 = nn.Linear(25, 10)
        
    def forward(self, x):
        #N 50 28 28
        x = F.relu(self.conv1(x)) # N 5 26 26
        x = self.pool(x)          # N 5 13 13
        x = F.relu(self.conv2(x)) # N 1 11 11
        x = self.pool(x)          # N 1 5 5
        x = torch.flatten(x, 1)   # N 25
        x = self.fc1(x)           # N 10
        
        return x

CNN = ConvNet()

criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(CNN.parameters(), lr=0.001)

n_total_steps = len(train_dl)

for epoch in range(100):
    
    running_loss = 0.0
    
    for i, (images,labels) in enumerate(train_dl):
        images = images.float()
        labels = labels

        outputs = CNN(images)
        loss = criterion(outputs, labels)

        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        running_loss += loss.item()

    print(f'[{epoch + 1}] loss: {running_loss / n_total_steps:.3f}')

print('Finished Training')
</code></pre>
<p>Thanks a lot in advance!!</p>

ptrblck: <p>Your code works fine using:</p>
<pre><code class="lang-python">train_ds = TensorDataset(torch.randn(101, 50, 28, 28), torch.randint(0, 10, (101,)))

train_dl = torch.utils.data.DataLoader(dataset = train_ds,
                                       batch_size = 50,
                                      shuffle = True) 
</code></pre>
<p>which creates a last batch with a single sample.</p>

nfreitas: <p>thanks a lot! So the issue is on my Dataset!<br>
I’m using a dataset in csv format and converting it to a tensor, so I suppose the issue is on my Dataset Class.</p>
<p>That’s what I’m doing:</p>
<pre><code class="lang-auto">data = pd.read_csv('train.csv')

X1 = data.drop(['label'], axis=1)
y = data[['label']]

X1_train, X1_test, y1_train, y1_test = train_test_split(X1, y, test_size=0.25, stratify=y, random_state=108)

y1_train.reset_index(inplace=True)
y1_train = y1_train['label']

class CustDF(torch.utils.data.Dataset):
    def __init__(self, features, target):
        self.features = features
        self.target = target
    
    def __len__(self):
        return self.features.shape[0]
    
    def __getitem__(self, idx):
        dp = self.features.iloc[idx].values
        targ = self.target[idx]
        dp = torch.from_numpy(dp)
        dp = dp.view(28,28)
        targ = torch.tensor(targ)
        return dp, targ

train_ds = CustDF(X1_train, y1_train)
test_ds = CustDF(X1_test, y1_test)
</code></pre>
<p>Still not getting how to transform it so it works as in your example. I’m using this Kaggle Dataset <a href="https://www.kaggle.com/competitions/digit-recognizer/data" class="inline-onebox" rel="noopener nofollow ugc">Digit Recognizer | Kaggle</a>.</p>
<p>Thanks a lot ptrblck!</p>

ptrblck: <p>Yes, the issue might come from your custom <code>CustDF</code> dataset. Print the shapes of the data and target inside the <code>__getitem__</code> before returning them to see what’s happening and where the shape mismatch comes from.</p>

nfreitas: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/3/632338806f965b06d875a649d508aa1692f8f2e2.png" data-download-href="https://discuss.pytorch.org/uploads/default/632338806f965b06d875a649d508aa1692f8f2e2" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/3/632338806f965b06d875a649d508aa1692f8f2e2.png" alt="image" data-base62-sha1="e90JezqoiP3XxFR207wlCwuuYoO" width="578" height="499" data-dominant-color="F3F3F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">673×582 12 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Did you also check the shape of the target as suggested?</p>

nfreitas: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>!<br>
I checked but I found no errors!<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/9/196ddec9998357523423cde7b5d2b3fb0ad2acba.png" data-download-href="https://discuss.pytorch.org/uploads/default/196ddec9998357523423cde7b5d2b3fb0ad2acba" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/9/196ddec9998357523423cde7b5d2b3fb0ad2acba.png" alt="image" data-base62-sha1="3CXkCXpZaVDLMP0OMTDbnVoJ122" width="409" height="499" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">498×608 11.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I’m still trying to figure out what I’m doing wrong!<br>
The dataset is composed of 784 features and one target. the 784 features are pixels from images of 28x28.</p>
<p>I transformed the tensor using the .view so it has 28x28 as expected instead of 1x784.</p>

ptrblck: <p>Your screenshot still doesn’t show the shape of the <code>targ</code> tensor, but just of <code>dp</code> before and after the <code>view</code> operation.</p>

nfreitas: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/b/9bd59b04a1deacdb36d114934e19ac95841b540a.png" data-download-href="https://discuss.pytorch.org/uploads/default/9bd59b04a1deacdb36d114934e19ac95841b540a" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/b/9bd59b04a1deacdb36d114934e19ac95841b540a.png" alt="image" data-base62-sha1="mezEEqetgfJqaIxEWWuyOId6aKu" width="450" height="499" data-dominant-color="F3F3F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">601×667 12.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
like this you mean?</p>
<p>I was sure the issue was on the dp. But now I saw this strange outcome. as if the tensor was empty. but it’s not. I can’t add another print to this post, but this tensor exists and is not empty</p>

nfreitas: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/5/053cf6415323028f0b21d5f6ad2a659a7d30d9b1.png" data-download-href="https://discuss.pytorch.org/uploads/default/053cf6415323028f0b21d5f6ad2a659a7d30d9b1" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/0/5/053cf6415323028f0b21d5f6ad2a659a7d30d9b1.png" alt="image" data-base62-sha1="KkZR8qIMyrZXNiTZBdkqGVN5dL" width="690" height="188" data-dominant-color="F4F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">755×206 3.88 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
this tensor(8) is the one that is returning torch.<code>Size([])</code>. I have to sleep now. I’ll check it later. Tks a lot.</p>

nfreitas: <p>Issue solved my dear <a class="mention" href="/u/ptrblck">@ptrblck</a>!<br>
The problem was on the CustDF class as expected.</p>
<p>The CustDF that makes it works is this:</p>
<pre><code class="lang-auto">class CustDF(torch.utils.data.Dataset):
    def __init__(self, features, target):
        self.features = features
        self.target = target
    
    def __len__(self):
        return self.features.shape[0]
    
    def __getitem__(self, idx):
        dp = self.features.iloc[idx].values
        targ = self.target[idx]
        dp = torch.from_numpy(dp)
        dp = dp.view(1,28,28)
        targ = torch.tensor(targ)
        return dp, targ
</code></pre>
<p>the view must be (1,28,28) instaed of just (28,28). This is because conv2d expects [50, 1, 28, 28] in this case. (50=batch size, 1=number of channels which in this case is 1 B&amp;W imabe and the 28x28 from the image size)</p>
<p>Thaks a lot for your help and patience. It’s amazing to a newbie to have help from this community.</p>