DAN / IndexError: Target 1 is out of bounds

username: Asonjay
cooked: <p>Hi, I am trying to implement deep averaging network with GloVe embedding. My code keeps having problem with loss function. I tried to check both y and log_probs shape, it doesn’t seem to have problems. Could you please help me look into this?</p>
<blockquote>
<p>num_epoch = 100<br>
emb_size = 128<br>
num_classes = 1</p>
</blockquote>
<pre><code>DAN = NeuralSentimentClassifier(word_embeddings.get_embedding_length(), emb_size, num_classes, word_embeddings)
optimizer = optim.Adam(DAN.parameters(), lr=0.1)
loss_function = nn.NLLLoss()

for epoch in range(num_epoch):
    #total_loss = 0.0
    for se in train_exs:
        #1# Prepare for inputs
        x = []
        for word in se.words:
            x.append(word.lower())
        y = torch.from_numpy(np.asarray(se.label)).long()
        #2# Zero the weights             
        DAN.zero_grad()
        #3# Forward the embedding
        log_probs = DAN.forward(x)
        # print(y_onehot)
        #4# Calculate Loss
        loss = loss_function(log_probs, y)
        #5# Backprop and update loss
        loss.backward()
        optimizer.step()
</code></pre>

username: anantguptadbl
cooked: <p><a class="mention" href="/u/asonjay">@Asonjay</a>  what is the exact error that you are getting?</p>

username: Asonjay
cooked: <p>Thank you so much for your reply! The error comes from</p>
<blockquote>
<p>loss = loss_function(log_probs, y)</p>
</blockquote>
<p>It prompts that:</p>
<blockquote>
<p>Traceback (most recent call last):<br>
File “C:\Users\Jason\Dropbox\Academic\SP2022\CSE5525\A2\a2-distrib\neural_sentiment_classifier.py”, line 106, in <br>
model = train_deep_averaging_network(args, train_exs, dev_exs, word_embeddings)<br>
File “C:\Users\Jason\Dropbox\Academic\SP2022\CSE5525\A2\a2-distrib\models.py”, line 112, in train_deep_averaging_network<br>
loss = loss_function(log_probs, y)<br>
File “C:\Users\Jason\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py”, line 1102, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “C:\Users\Jason\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\loss.py”, line 211, in forward<br>
return F.nll_loss(input, target, weight=self.weight, ignore_index=self.ignore_index, reduction=self.reduction)<br>
File “C:\Users\Jason\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\functional.py”, line 2532, in nll_loss<br>
return torch._C._nn.nll_loss_nd(input, target, weight, _Reduction.get_enum(reduction), ignore_index)<br>
IndexError: Target 1 is out of bounds.</p>
</blockquote>

username: ptrblck
cooked: <p><code>nn.NLLLoss</code> is used for a multi-class classification (or segmentation), such that <code>num_classes = 1</code> wouldn’t make sense, since your model is only predicting a single class (class0).<br>
Since your target tensor contains (at least) the index 1, you are dealing with (at least) 2 classes and would need to use <code>num_classes=2</code> such that the model output has the shape <code>[batch_size, nb_classes=2]</code>.</p>

username: Asonjay
cooked: <p>Thank you so much for your reply! I definitely need to check the signature of the loss function clearly. Plus, I am using imported GloVe data to train my model. How does nn.Embedding.from_pretrained work? Can I just feed the GloVe data (word, embedding…) directly into this?</p>

username: ptrblck
cooked: <p><code>from_pretrained</code> expects the <code>weight</code> tensor in the shape <code>[num_embeddings, embedding_dim]</code>. Afterwards you can pass the work indices to this layer and will get the dense outputs.</p>