Target size (torch.Size([32])) must be the same as input size (torch.Size([32, 24]))

Abdelrahman_Mohamed: <pre><code class="lang-auto">data_transforms = {
    'train': transforms.Compose([
        transforms.ToPILImage(),
        transforms.RandomRotation(15),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        
]),
    
    'valid': transforms.Compose([
        transforms.ToPILImage(),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        
    ]),
}
class Dataset_load(Dataset):
    
    def __init__(self, file_path, transform=None):
        self.data = pd.read_csv(file_path)
        self.transform = transform
        
    def __len__(self):
        return len(self.data)
    
    def __getitem__(self, index):
     
        image = self.data.iloc[index, 1:].values.astype(np.uint8).reshape((28,28,1))
        label = self.data.iloc[index, 0]
        
        if self.transform is not None:
            image = self.transform(image)
        
            
        return image , label

train_dataset=Dataset_load(train_dir,transform=data_transforms['train'])

train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=32, shuffle=True)

test_dataset=Dataset_load(test_dir,transform=data_transforms['valid'])

test_loader =torch.utils.data.DataLoader(test_dataset, batch_size=32, shuffle=True)


import torch.nn.functional as F


class Net(nn.Module):
  
  def __init__(self):
    
    super(Net,self).__init__()
    
    #input depth , output depth , kernel size(filter)x
    
    self.conv1=nn.Conv2d(1,16,kernel_size=(3, 3),padding=(1, 1),stride=(1, 1))
    
    self.conv2=nn.Conv2d(16,32,kernel_size=(3, 3),padding=(1, 1),stride=(1, 1))
    
    self.conv3=nn.Conv2d(32,64,kernel_size=(3, 3),padding=(1, 1),stride=(1, 1))
    
    #padding for last conv layer 
    self.adapt = nn.AdaptiveMaxPool2d((4,4))  
    
    #padding layer
    self.pool=nn.MaxPool2d(2,2)
    
    #dropout layer
    self.drop=nn.Dropout(p=0.2)
    
    #fc layers 
    self.fc1=nn.Linear(64*4*4,512)
    
    self.fc2=nn.Linear(512,256)
    self.fc3=nn.Linear(256,128)
    self.fc4=nn.Linear(128,24)
    
  def forward(self,x):
    
    x=self.pool(F.leaky_relu(self.conv1(x)))
    
    x=self.pool(F.leaky_relu(self.conv2(x)))
    
    x=self.adapt(F.leaky_relu(self.conv3(x)))
    
    
    #flatten Images
    x = x.view(x.size(0), -1)
    
    x=self.drop(x)
    
    x=F.leaky_relu(self.fc1(x))
    
    x=self.drop(x)
    x=F.leaky_relu(self.fc2(x))
    
    x=self.drop(x)
    x=F.leaky_relu(self.fc3(x))
    
    x=self.drop(x)
    
    x=self.fc4(x)
    
    return F.log_softmax(x)


model = Net()
optimizer = optim.Adam(model.parameters(), lr=0.001)
criterion = nn.BCEWithLogitsLoss()
exp_lr_scheduler = lr_scheduler.StepLR(optimizer, step_size=4, gamma=0.1)
if torch.cuda.is_available():
  
    model = model.cuda()
    criterion = criterion.cuda()



def train(n_epochs=100):
  
  

  for epoch in range(1, n_epochs+1):
    

    # keep track of training and validation loss
    train_loss = 0.0
    
   #train
    model.train()
    for data, target in train_loader:
        print(data.shape)
        print(target.shape)
        
        if torch.cuda.is_available():
            data, target = data.cuda(), target.cuda()
        # clear the gradients of all optimized variables
        optimizer.zero_grad()
        # forward pass: compute predicted outputs by passing inputs to the model
        output = model(data)
        
        # calculate the batch loss
        loss = criterion(output, target)
        # backward pass: compute gradient of the loss with respect to model parameters
        loss.backward()
        # perform a single optimization step (parameter update)
        optimizer.step()
        # update training loss
        train_loss += loss.item()*data.size(0)
    
    train_loss = train_loss/len(train_loader)
    
        
    # print training/validation statistics 
    print('Epoch: {} \tTraining Loss: {:.6f}}'.format(
        epoch, train_loss))
    
train(10)
</code></pre>
<pre><code class="lang-auto">torch.Size([32, 1, 28, 28])
torch.Size([32])
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:61: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
&lt;ipython-input-73-c813bd3a9aca&gt; in &lt;module&gt;()
      1 from torch.autograd import Variable
----&gt; 2 train(10)

3 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in binary_cross_entropy_with_logits(input, target, weight, size_average, reduce, reduction, pos_weight)
   2159 
   2160     if not (target.size() == input.size()):
-&gt; 2161         raise ValueError("Target size ({}) must be the same as input size ({})".format(target.size(), input.size()))
   2162 
   2163     return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)

ValueError: Target size (torch.Size([32])) must be the same as input size (torch.Size([32, 24]))
</code></pre>

ptrblck: <p>If you are dealing with a multi-class classification use case and your <code>target</code> tensor contains class indices, you should use <code>nn.NLLLoss</code> as your criterion.<br>
Alternatively, you could also remove the <code>F.Log_softmax</code> from your model and use <code>nn.CrossEntropyLoss</code> (identical to <code>F.log_softmax</code> + <code>nn.NLLLoss</code>).</p>
<p>However, if you are dealing with a multi-label classification (each sample can have more than a single active class), you should remove the <code>F.log_softmax</code> and would have to provide a target for each class output.</p>

Abdelrahman_Mohamed: <p>thanks for replying<br>
I  removed log_softmax and used CrossEntropyLoss but I got this Error</p>
<pre><code class="lang-auto">Assertion `cur_target &gt;= 0 &amp;&amp; cur_target &lt; n_classes' failed.  at /pytorch/aten/src/THNN/generic/ClassNLLCriterion.c:92
</code></pre>

ptrblck: <p>Your target should contain class indices in the range <code>[0, nb_classes-1]</code>.<br>
In your case this would be <code>[0, 23]</code>.</p>
<p>To debug this issue, you could add a <code>print</code> statement and check, which target batch fails this assumption.</p>

mathematics: <p>Please could you tell about doing it on <strong>nn.BCEWithLogitsLoss()</strong></p>
<pre><code class="lang-auto">def real_loss(D_out):
   labels = torch.ones(D_out.size(0)) * 0.9 #smoothing by default
    criterion = nn.BCEWithLogitsLoss()
    if train_on_gpu:
        labels = labels.cuda()
    loss = criterion(D_out.squeeze(), labels)
    return loss
</code></pre>
<p>It is giving error<br>
<code>ValueError: Target size (torch.Size([32])) must be the same as input size (torch.Size([32, 4096]))</code></p>
<p>But i tried to debug it, printed results and found that error were coming from using <code>criterion(D_out.squeeze(), labels)</code></p>
<p>I printed out  shape of D_out found that <code>torch.Size([32, 4096])</code></p>
<p>I hit trialed any  possible combination to accept criterion but it throws  same error.</p>
<p>Please could you give me any idea why this kind of error is occuring in my code?<br>
Thank you for helping us <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

ptrblck: <p><code>nn.BCEWihtLogitsLoss</code> expects the model output and target to have the same shape.<br>
Based on your provided shapes, it seems that your target might have the shape <code>[batch_size]</code> and might contain the class indices for each sample?<br>
If that’s the case, you are dealing with a multi-class classification use case and should use <code>nn.CrossEntropyLoss</code> instead.</p>
<p>Based on your model output you might also be dealing with a multi-label classification use case, where each sample might belong to zero, one, or multiple classes. If that’s the case, your target should contain values in the range <code>[0, 1]</code> and have the shape <code>[batch_size, nb_classes]</code>.</p>

mathematics: <p>Thanks explaining , after changing to <code>torch.Size([32, 1])</code> error was i was returning 4096 in discriminator.</p>

Agatha: <p>Greetings,</p>
<p>I’m working on multi-label classification with 8 labels and BCEWithLogitsLoss as a loss function. My target is in the range <code>[0, 7]</code>.<br>
I’m getting this error and I can’t figure out what I should change…</p>
<p>Any help is appreciated.</p>
<pre><code class="lang-auto">
     56             #labels = labels.unsqueeze(1)
---&gt; 57             loss = loss_fn(logits, labels)
     58              

~/anaconda3/envs/env/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--&gt; 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

~/anaconda3/envs/env/lib/python3.8/site-packages/torch/nn/modules/loss.py in forward(self, input, target)
    626 
    627     def forward(self, input: Tensor, target: Tensor) -&gt; Tensor:
--&gt; 628         return F.binary_cross_entropy_with_logits(input, target,
    629                                                   self.weight,
    630                                                   pos_weight=self.pos_weight,

~/anaconda3/envs/env/lib/python3.8/site-packages/torch/nn/functional.py in binary_cross_entropy_with_logits(input, target, weight, size_average, reduce, reduction, pos_weight)
   2536 
   2537     if not (target.size() == input.size()):
-&gt; 2538         raise ValueError("Target size ({}) must be the same as input size ({})".format(target.size(), input.size()))
   2539 
   2540     return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)

ValueError: Target size (torch.Size([16])) must be the same as input size (torch.Size([16, 8]))

</code></pre>

ptrblck: <aside class="quote no-group" data-username="Agatha" data-post="8" data-topic="55376">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/agatha/48/12756_2.png" class="avatar"> Agatha:</div>
<blockquote>
<p>I’m working on multi-label classification with 8 labels and BCEWithLogitsLoss as a loss function. My target is in the range <code>[0, 7]</code>.</p>
</blockquote>
</aside>
<p>This sounds rather like a multi-class classification (one active class per sample) and you could use <code>nn.CrossEntropyLoss</code> instead.<br>
<code>nn.BCEWithLogitsLoss</code> is used for a multi-label classification (zero, one, or multiple active classes per sample) and expects the model output and target in the same shape.</p>

Agatha: <p>It’s multi-label, with more than one class per sample.</p>

ptrblck: <p>How does the target value range of <code>[0, 7]</code> fit this use case, i.e. if the target contains an integer the the sample, how would you select e.g. two active classes?</p>

Agatha: <p>Its text classification using BertModel, where one text can have multiple labels.<br>
The idea of [0, 7] comes because I use PyTorch and the labels have to range from [0- n_labels-1].</p>
<p>Now I’m stuck because it’s said on Multilabel, nn.BCEWithLogitsLoss is applicable, but when I tried I get the above error…</p>

ptrblck: <aside class="quote no-group" data-username="Agatha" data-post="12" data-topic="55376">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/agatha/48/12756_2.png" class="avatar"> Agatha:</div>
<blockquote>
<p>The idea of [0, 7] comes because I use PyTorch and the labels have to range from [0- n_labels-1].</p>
</blockquote>
</aside>
<p>That’s true for a multi-class classification use case using <code>nn.CrossEntropyLoss</code> or <code>nn.NLLLoss</code>, but wrong for a multi-label classification using <code>nn.BCEWithLogitsLoss</code>.</p>
<p>For the latter use case, your targets should have the same shape as the model outputs and be multi-hot encoded, so that you can set multiple classes to “active” or “inactive” (<code>1</code> and <code>0</code>).</p>

Agatha: <p>Thanks for explaining,<br>
I that case, what should I change to get the same shape?</p>

ptrblck: <p>Your targets would have to be already multi-hot encoded. If your current target shape is <code>[batch_size]</code> and the targets contain values in <code>[0, nb_classes-1]</code>, then they are multi-class classification targets and you could use <code>nn.CrossEntropyLoss</code> as described before.</p>
<p>However, if you want to use these targets for a multi-label classification for specific reasons, you could use <code>F.one_hot</code> to create the expected shape for <code>nn.BCEWithLogitsLoss</code>.</p>

Phuc_Pham: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> , How about I used criterion = nn.BCEWithLogitsLoss() in binary classification and get the Error: Target size (torch.Size([1, 2])) must be the same as input size (torch.Size([1, 1000]))</p>

ptrblck: <p>The posted shaped don’t match a binary classification case, as the input seems to have the shape <code>[batch_size=1, nb_classes=1000]</code>, while the target has the shape <code>[batch_size=1, nb_classes=2]</code>. Based on the target shape it seems you are working on a multi-label classification. However based on the model output shape it seems you might have forgotten to replace the last linear layer of a pretrained classification model, so that the 1000 ImageNet class logits will be returned.</p>
<p>The error is raised, since <code>nn.BCEWithLogitsLoss</code> expects the model output and target so have the same shape.</p>