CrossEntropyLoss: Index Error (Target 3 is out of bounds)

feiland: <p>Hi there,</p>
<p>I am using code from a CIFAR classification problem (num_classes = 10) and want to use the code for my dataset (CheXpert with num_classes = 3). Therefore, I changed the num_classes in the ResNet model from 10 to 3.</p>
<pre><code class="lang-auto">class ResNet(FitModule):
    def __init__(self, block, num_blocks, num_classes=3):    #changed from 10 to 3
        super(ResNet, self).__init__()
        self.in_planes = 64
        self.conv1 = conv3x3(3, 64)
        self.bn1 = nn.BatchNorm2d(64)                                          
        self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1)      
        self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2)    
        self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2)    
        self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2)     

        self.ap = nn.AdaptiveAvgPool2d(1)
        self.mp = nn.AdaptiveMaxPool2d(1)

        self.linear = nn.Linear(1024 * block.expansion, num_classes)        
   ...
    def forward(self, x):
        x = x.float()                                               
        out = F.relu(self.bn1(self.conv1(x).float()).float())       
        out = self.layer1(out)                                      
        out = self.layer2(out)                                      
        out = self.layer3(out)                                      
        out = self.layer4(out)                                      
        out = torch.cat([self.mp(out), self.ap(out)], 1)           
        out = out.view(out.size(0), -1)                                   
        out = self.linear(out)
        return out
</code></pre>
<p>Then the <strong>IndexError: Target 3 is out of bounds</strong> occurs in my <strong>fit-methode</strong> when using CrossEntropyLoss.<br>
10 pictures of size 3x32x32 are given into the model. That’s why X_batch has size [10, 3, 32, 32], after going through the model, y_batch_pred has size [10, 3] as I changed num_classes to 3.<br>
When using the CrossEntropyLoss with y_batch_pred [10, 3] and the initial labels y_batch [10] the IndexError occurs.</p>
<pre><code class="lang-auto">    def fit(self, X, y, batch_size=32, epochs=10, verbose=1, validation_split=0.,             
              validation_data=None, shuffle=True, initial_epoch=0, seed=None,
              loss=CrossEntropyLoss(), optimizer=partial(SGD, lr=0.001, momentum=0.9),
              metrics=None):
        
        ...
            # Run batches
            for batch_i, batch_data in enumerate(train_data):
                # Get batch data
                X_batch = Variable(batch_data[0], requires_grad=True).float()
                y_batch = Variable(batch_data[1], requires_grad=True).long()
                # Backprop
                opt.zero_grad()
                y_batch_pred = self(X_batch).float()             # picture goes through model
                batch_loss = loss(y_batch_pred, y_batch)    # IndexError !!
                batch_loss.backward()
                opt.step()
                # Update status
                epoch_loss += batch_loss.item()
                for param in self.parameters():
                    param.requires_grad = True
                log['loss'] = float(epoch_loss) / (batch_i + 1)
                if verbose:
                    pb.bar(batch_i, log_to_message(log))
            ...
        return logs
</code></pre>
<p>There does not seem to be a big difference to between the original CIFAR problem with 10 classes and my problem with only 3 classes. When running my images on a 10 classes classification problem everything works just fine.</p>
<p>Can anyone help?</p>
<p>Thanks in advance,<br>
Best,<br>
Fabian</p>

ptrblck: <p>For 3 classes, your target should have the shape <code>[batch_size]</code> (which seems to be correct) and contain the values <code>[0, 1, 2]</code>. It seems you are using a value of 3, which will raise this error, as this would mean you are using 4 classes (note that the class index starts at 0).</p>

feiland: <p>Perfect, when importing my labels I changed them to 0,1,2 (before they were 1,2,3) and now it seems to work.<br>
Thanks plenty!</p>

jmrmontano: <p>Hi, That works for me too.</p>
<p>Thanks a lot.</p>

pdc_87: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>. I am dealing with the same issue in a multilabel problem.<br>
train_data[0][0].shape torch.Size([3, 519, 521])<br>
print((train_loader.dataset.class_to_idx)) {‘0’: 0, ‘1’: 1, ‘2’: 2, ‘3’: 3}<br>
print(train_loader.dataset.classes) [‘0’, ‘1’, ‘2’, ‘3’]</p>
<p>The error kicks in the loss function, CrossEntropyLoss.<br>
**—&gt; loss = loss_func(output, target)</p>
<p>tensor([[0.6857, 0.3143]], grad_fn=) tensor([1])<br>
tensor([[0.5454, 0.4546]], grad_fn=) tensor([3])</p>
<p>Targets 0,1 passed but when moves to 2 and 3 no. It is expecting a binary target although the loader has 4.</p>
<p>Thanks in advance</p>

ptrblck: <aside class="quote no-group" data-username="pdc_87" data-post="5" data-topic="83054">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pdc_87/48/54318_2.png" class="avatar"> pdc_87:</div>
<blockquote>
<p>Targets 0,1 passed but when moves to 2 and 3 no. It is expecting a binary target although the loader has 4.</p>
</blockquote>
</aside>
<p>This would be the root cause since a binary target would contain only two values (0 and 1) so could you check why the target contains 4 values?</p>

pdc_87: <p>Thanks! Already solved. The loader is correct. The Net architectures retrieved a binary output.That was the problem</p>