Problems with number of classes, adjust CrossEntropyLoss?

titoniubo: <p>Hello,</p>
<p>When running multi-class classification, I have to set the number of classes as <strong>n+1</strong>, as setting the number of classes as <strong>n</strong> returns error.</p>
<p>I attempted to modify CrossEntropyLoss as per instructions on this <a href="https://discuss.pytorch.org/t/ignore-index-in-the-cross-entropy-loss/25006">post</a>, but I must have done something wrong.</p>
<p>I also read this <a href="https://discuss.pytorch.org/t/solved-assertion-cur-target-0-cur-target-n-classes-failed/8034">post</a> , but I could not find out a solution.</p>
<p>I have 10 variables split as (X_train,X_test) and 1 target variable split as (Y_train, Y_test). My target/label is a 3 class (1,2,3)</p>
<p>Here’s the code:</p>
<pre><code class="lang-auto">class NeuralNet(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super(NeuralNet, self).__init__()
        self.layer1 = nn.Linear(input_size, hidden_size)
        self.relu = nn.ReLU()
        self.layer2 = nn.Linear(hidden_size, num_classes)
        
    def forward(self, x):
        out = self.layer1(x)
        out = self.relu(out)
        out = self.layer2(out)
        return out
</code></pre>
<pre><code class="lang-auto">tr_latent_X = data_utils.TensorDataset(train_x, train_y)
te_latent_X = data_utils.TensorDataset(test_x, test_y)

x_train_label = Y_train
x_test_label = y_test

train_loader_X = torch.utils.data.DataLoader(dataset=tr_latent_X,
                                             batch_size=batch_size,
                                             shuffle=False)

test_loader_X = torch.utils.data.DataLoader(dataset=te_latent_X,
                                             batch_size=batch_size,
                                             shuffle=False)
</code></pre>
<p>If I set the number of dimensions as 3(instead of 4)  here later I obtain the error:</p>
<pre><code class="lang-auto">vae = NeuralNet(10,2,3)


loss_function = nn.CrossEntropyLoss(reduction='mean',ignore_index=-1)

optimizer = torch.optim.SGD(vae.parameters(), lr = 0.01)

def train(epoch):
    vae.train()

    train_loss = 0

    for batch_idx, (data,label) in enumerate(train_loader_X):
        
        optimizer.zero_grad()

        out = vae(data)
        loss = loss_function(out, label)

        loss.backward()
        train_loss += loss.item()
        optimizer.step()

        if batch_idx % 5000 == 0:
            print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
                epoch, batch_idx * len(data), len(train_loader_X.dataset),
                100. * batch_idx / len(train_loader_X), loss.item() / len(data)))
    print('====&gt; Epoch: {} Average loss: {:.4f}'.format(epoch, train_loss / len(train_loader_X.dataset)))

for epoch in range(1, 10):
    train(epoch)
</code></pre>
<p>This is the error I obtained (last part of the long sequenece)</p>
<pre><code class="lang-auto">4 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
   1836                          .format(input.size(0), target.size(0)))
   1837     if dim == 2:
-&gt; 1838         ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
   1839     elif dim == 4:
   1840         ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)

RuntimeError: Assertion `cur_target &gt;= 0 &amp;&amp; cur_target &lt; n_classes' failed.  at /pytorch/aten/src/THNN/generic/ClassNLLCriterion.c:97
</code></pre>
<p>If instead of setting 3 classes, I set 4, then it runs well:</p>
<pre><code class="lang-auto">vae = NeuralNet(10,2,4)
loss_function = nn.CrossEntropyLoss(reduction='mean',ignore_index=-1)
optimizer = torch.optim.SGD(vae.parameters(), lr = 0.01)
</code></pre>
<p>If anyone could shed some light on this, I would be very grateful.</p>
<p>Sincerely,</p>

Eta_C: <p><code>label</code> -&gt; (0, 1, 2)<br>
Specifically, <code>label</code> will be used as index.<br>
<code>0 &lt;= label &lt; num_classes</code>.</p>

titoniubo: <p>Thanks for your reply <a class="mention" href="/u/eta_c">@Eta_C</a> but I am not sure how to implement your proposed solution. Where should I place this code?</p>
<p>I’d be grateful if you could elaborate on this.</p>

Eta_C: <p><code>train_y -= 1</code><br>
<code>test_y -= 1</code></p>

titoniubo: <p>Thanks, it works!</p>
<p>Should I keep loss function as it is?</p>
<p><code>loss_function = nn.CrossEntropyLoss(reduction='mean',ignore_index=-1)</code></p>
<p>or I should remove this part?</p>
<p><code>reduction='mean',ignore_index=-1</code></p>

Eta_C: <p>Actually, it depends on you. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

titoniubo: <p>well, code runs well with and without it.</p>
<p>Train results look almost alike, so I guess it makes no sense to keep it.</p>
<p>Thanks for your time and help.</p>