Nll_loss, in crossEntropyloss ,Target -1 is out of bounds error

flipflop: <p>I am pretty new to deep learning and pytorch API , When I try to build a ResNet 50 and train the image attributes with binary (1 or -1) class ,It gives me nll_loss, in crossEntropyloss ,Target -1 is out of bounds error , and I have followed the tutorial that my resNet 50 structure is correctly built ,tested with a random input and it give me the correct output which is tensor size 2.</p>
<p>Here is my code for training -<br>
def train_model(epoch):<br>
model_net.train()<br>
for batch_index,(input,labels) in enumerate(train_loader):<br>
labels=labels.view(batch_size)<br>
input,labels=input.to(device),labels.to(device)<br>
outputs=model_net(input)<br>
print(input.shape)<br>
print(outputs.shape)<br>
print(labels.shape)<br>
loss =lostFunction(outputs,labels)<br>
if batch_index %2==0 or batch_index==len(train_loader)-1:<br>
print(‘epoch {}  batch {}/{}  loss {:.3f}’.format(<br>
epoch, batch_index, len(train_loader)-1, loss.item()))<br>
optimizer.zero_grad()  # Set gradients to zero<br>
loss.backward()        # From the loss we compute the new gradients<br>
optimizer.step()</p>
<hr>
<p>this is the output -&gt;<br>
torch.Size([20, 3, 218, 178])<br>
torch.Size([20, 2])<br>
torch.Size([20])</p>
<p>IndexError                                Traceback (most recent call last)<br>
 in ()<br>
----&gt; 1 train_model(0)</p>
<p>4 frames<br>
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)<br>
2216                          .format(input.size(0), target.size(0)))<br>
2217     if dim == 2:<br>
-&gt; 2218         ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)<br>
2219     elif dim == 4:<br>
2220         ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)</p>
<p>IndexError: Target -1 is out of bounds.</p>
<hr>
<p>this is the testing data format -&gt;<br>
(tensor([[[-0.6794, -0.7137, -0.8164,  …, -1.0048, -1.0048, -0.9705],<br>
[-0.6794, -0.7137, -0.8164,  …, -1.0048, -1.0048, -0.9705],<br>
[-0.6794, -0.7137, -0.8164,  …, -1.0048, -1.0048, -0.9705],<br>
…,<br>
[ 0.3994,  0.2111,  0.0741,  …,  1.6667,  1.6667,  1.4954],<br>
[ 0.2453,  0.2453,  0.2967,  …,  1.5639,  1.6838,  1.7694],<br>
[ 0.2624,  0.2624,  0.3652,  …,  1.5639,  1.6838,  1.7694]],</p>
<pre><code>    [[-0.6176, -0.6527, -0.7752,  ..., -1.1604, -1.1604, -1.1253],
     [-0.6176, -0.6527, -0.7752,  ..., -1.1604, -1.1604, -1.1253],
     [-0.6176, -0.6527, -0.7752,  ..., -1.1604, -1.1604, -1.1253],
     ...,
     [ 0.2052,  0.0126, -0.4601,  ...,  1.8333,  1.8333,  1.6583],
     [ 0.0476,  0.0476, -0.2500,  ...,  1.7108,  1.8158,  1.9034],
     [ 0.0476,  0.0476, -0.2150,  ...,  1.7108,  1.8158,  1.9034]],

    [[-0.5147, -0.5495, -0.6018,  ..., -1.0550, -1.0550, -1.0201],
     [-0.5147, -0.5495, -0.6018,  ..., -1.0550, -1.0550, -1.0201],
     [-0.5147, -0.5495, -0.6018,  ..., -1.0550, -1.0550, -0.9853],
     ...,
     [ 0.3219,  0.1302, -0.2881,  ...,  2.1868,  2.2217,  2.0474],
     [ 0.1476,  0.1476, -0.1138,  ...,  2.0648,  2.2217,  2.3088],
     [ 0.1476,  0.1476, -0.0615,  ...,  2.0648,  2.2217,  2.3088]]]), tensor([-1])) 
</code></pre>
<hr>
<p>I really cant figure out why this happen , I have checked that all the training data ,label tensor ,batch number ,output feature number are all correct , can someone help me to debug?</p>

Caruso: <p>Hi,</p>
<p><a href="https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss" rel="noopener nofollow ugc">CrossEntropyLoss</a> expects your labels to be in range <code>[0, C-1]</code>, where <code>C</code> denotes the number of classes - so in your case <code>[0, 1]</code> and not -1 or 1.<br>
PS: If its a single-label binary classification, you can also have a look at <a href="https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html#torch.nn.BCEWithLogitsLoss" rel="noopener nofollow ugc">BinaryCrossEntropy</a>.</p>
<p>Greetings.</p>

flipflop: <p>Thank you ! ,this is a really stupid mistake , another question , since we output tensor with [value1.value2] , which is the prob of binary class , how do we know which value is representing 0 and which value representing 1. I am still a bit confused about this .</p>

KFrank: <p>Hi Chriseven!</p>
<aside class="quote no-group" data-username="flipflop" data-post="3" data-topic="98904" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/f/f6c823/48.png" class="avatar"> flipflop:</div>
<blockquote>
<p>since we output tensor with [value1.value2] , which is the prob of binary class , how do we know which value is representing 0 and which value representing 1.</p>
</blockquote>
</aside>
<p>The output of your model, <code>[value1, value2]</code>, means whatever you<br>
trained your model for it to mean.</p>
<p>As I understand it, you have structured your model as a two-class<br>
multi-class classifier.  (Your model outputs two values, and you use<br>
<code>CrossEntropyLoss</code>.)  Conceptually (although not in implementation)<br>
this is the same as a binary-classification problem.  (One output value,<br>
and <code>BCEWithLogitsLoss</code>.)</p>
<p>The following post answers your question, but in the language of a<br>
binary-classification problem:</p>
<aside class="quote quote-modified" data-post="10" data-topic="45709">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-to-interpret-the-probability-of-classes-in-binary-classification/45709/10">How to interpret the probability of classes in binary classification?</a> <a class="badge-category__wrapper " href="/c/nlp/8"><span data-category-id="8" data-drop-close="true" class="badge-category " title="Topics related to Natural Language Processing"><span class="badge-category__name">nlp</span></span></a>
  </div>
  <blockquote>
    Hello Shaun! 

In short, “class ‘1’” means whatever you trained you model for it 
to mean. 
To explain this, let me go back to one of your earlier posts: 

You talk about x_test and y_test (and y_pred).  I assume 
that y_test is a vector of length the number of test samples, 
with each value in y_test being the number 0, meaning that 
this sample is in class “0”, or the number 1, meaning class “1”. 
(And x_test is the input data whose classes you are trying 
to predict.) 
You don’t mention it, b…
  </blockquote>
</aside>

<p>Best.</p>
<p>K. Frank</p>

flipflop: <p>Thank you for your answer Frank .it helps!  btw do you have any experience on google colab cuda out og memory issues. I try to train around 150,000 images (7kb each)  , it seems I dunt have enough resources on GPU , but train using cpu really really too slow .</p>