Cross Entropy Loss delivers wrong classes

username: titoniubo
cooked: <p>Hello,</p>
<p>When using torch.argmax(output, dim=1) to see the predicted classes, I get to see the values 0, 1, 2 when the expected ones are 1,2,3.</p>
<p>I assume there may be an when implementing my code. It’s a multi-class prediction, with an input of 10 variables to predict a target (y). The target has 3 class: 1,2 and 3.</p>
<p>I would appreciate if someone could have a look and let me know what I may be doing wrong.</p>
<p>Here’s my code:</p>
<pre><code class="lang-auto">class NeuralNet(nn.Sequential):
    def __init__(self, input_size, hidden_size, num_classes):
        super(NeuralNet, self).__init__()
        self.layer1 = nn.Linear(input_size, hidden_size)
        self.relu = nn.ReLU()
        self.layer2 = nn.Linear(hidden_size, hidden_size)
        self.relu = nn.ReLU()
        self.layer3 = nn.Linear(hidden_size, num_classes)

    def forward(self, x):
        out = self.layer1(x)
        out = self.relu(out)
        out = self.layer2(out)
        out = self.relu(out)
        out = self.layer3(out)
        return out
</code></pre>
<p>the training</p>
<pre><code class="lang-auto">vae = NeuralNet(10,6,3)

#device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

#vae.to(device)

loss_function = nn.CrossEntropyLoss()

optimizer = torch.optim.SGD(vae.parameters(), lr = 0.005)

train_y -= 1
test_y -= 1

def train(epoch):
    vae.train()
    train_loss = 0

    for batch_idx, (data,label) in enumerate(train_loader_X):
        #data= data.to(device)
        #label= label.to(device)

        optimizer.zero_grad()

        out = vae(data)
        loss = loss_function(out, label)

        loss.backward()
        train_loss += loss.item()
        optimizer.step()

        if batch_idx % 500 == 0:
            #data= data.to(device)    
            print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
                epoch, batch_idx * len(data), len(train_loader_X.dataset),
                100. * batch_idx / len(train_loader_X), loss.item() / len(data)))
    print('====&gt; Epoch: {} Average loss: {:.4f}'.format(epoch, train_loss / len(train_loader_X.dataset)))
</code></pre>
<p>and the prediction</p>
<pre><code class="lang-auto">output = vae(test)
predicted = torch.argmax(output,dim =1)
predicted
</code></pre>
<p>As a result I obtain this:</p>
<p>tensor([2, 1, 0,  …, 1, 1, 0])</p>
<p>As mentioned, expected values in above tensor would be 1, 2, 3.</p>
<p>I would be grateful if anyone could shed some light on this.</p>

username: beaupreda
cooked: <p>Hello,</p>
<p>This is because indexing in PyTorch is zero based. You simply have to map the <code>predicted</code> indices (0, 1, 2) to your classes (1, 2, 3). If you think about it, the classes you want to predict could be <em>cat</em>, <em>dog</em> and <em>fish</em> and <code>torch.argmax(output, dim=1)</code> would still give you numbers 0, 1, 2.</p>
<p>Let me know if it is not clear!</p>

username: titoniubo
cooked: <p><a class="mention" href="/u/beaupreda">@beaupreda</a>, I understand your point and thank you for the explanation.</p>
<p>I would appreciate if you could kindly guide me on how to map the predicted indices (0,1,3) to the expected classes (1,2,3)</p>
<p>Sincerely</p>

username: beaupreda
cooked: <p>I forgot about that, but if you look at the <code>CrossEntropyLoss</code> <a href="https://pytorch.org/docs/1.3.0/nn.html#crossentropyloss" rel="nofollow noopener">docs</a>, it expects for the <code>target</code> tensor to be in range <code>[0, C - 1]</code>, where <code>C</code> is the number of classes you want to predict (3 in your case). So what you have to do is in your preprocessing of data, map your classes (1, 2, 3) so they fit in the range (0, 1, 2). In your case, you could simply substract 1 to each class so the <code>target</code> tensor is in the correct range. Afterwards, simply add 1 to the predicted classes to get back to the (1, 2, 3) classes.</p>

username: titoniubo
cooked: <p><a class="mention" href="/u/beaupreda">@beaupreda</a>, thanks for your feedback.</p>
<p>Following your feedback I proceed to change the values from Y (train and test) before these variables would be converted into tensor:</p>
<pre><code class="lang-auto">data.damage_grade.replace([1,2,3],[0,1,2], inplace=True)
</code></pre>
<p>I also tried following this [post] the(<a href="https://discuss.pytorch.org/t/how-to-represent-class-to-idx-map-for-custom-dataset-in-pytorch/37510/2" class="inline-onebox">How to represent class_to_idx map for custom dataset in Pytorch</a>) the following code, but I guess I do not know where to link it to.</p>
<pre><code class="lang-auto">idx_to_class = {
    ‘1’: “0”,
    ‘2’: “1”,
    ‘3’: “2”,
    }
sample_class = idx_to_class[label+1]
</code></pre>
<p>Unfortunately though, with either option, when I run the whole code the model would not train and return the following error:</p>
<p><code>RuntimeError: Assertion cur_target &gt;= 0 &amp;&amp; cur_target &lt; n_classes' failed.  at /pytorch/aten/src/THNN/generic/ClassNLLCriterion.c:97</code></p>
<p>I’d appreciate if you could let me know what your thoughts are.</p>

username: beaupreda
cooked: <p>Could you post a small example of your dataset so I could try to execute your code?</p>

username: Eta_C
cooked: <p>Hi</p>
<p>In my opinion, you do not need to create a mapping from index to label. Since your <code>label</code> is a tensor, just simply change your code to</p>
<pre><code class="lang-python">for batch_idx, (data, label) in enumerate(train_loader_X):
    ...
    loss = loss_function(out, label - 1)
   ...
</code></pre>
<p>and</p>
<pre><code class="lang-python">output = vae(test)
predicted = torch.argmax(output, dim=1) + 1
</code></pre>

username: titoniubo
cooked: <p><a class="mention" href="/u/beaupreda">@beaupreda</a>, thanks for your help. Sorry I have no time to prepare this. Your suggestion to remmap helped me understand what the issue was and fix using remapping.</p>

username: titoniubo
cooked: <p><a class="mention" href="/u/eta_c">@Eta_C</a> many thanks for your valuable feedback.</p>
<p>Your proposed solution works.</p>