RuntimeError: shape '[-1, 2]' is invalid for input of size 9, please help me figure out this error

ravindra: <pre><code class="lang-auto">import torch
import torch.optim as optim

optimizer = optim.AdamW(model.parameters(), lr = 2e-5)  # lr = 0.00002
criterion = torch.nn.CrossEntropyLoss()

num_epochs = 4
for epoch in range(num_epochs):
  optimizer.zero_grad()
  output = model(input_ids, attention_mask = attention_mask, labels = torch.tensor(sentiment_labels))
  loss = output.loss
  loss.backward()
  optimizer.step()
</code></pre>
<p>My sentiment lables is 0, 1, 2 with respect to positive, negative, and neutral</p>
<pre><code class="lang-auto">sentiment_labels = [sentiments.index(sentiment) for sentiment in sentiments]
print(sentiment_labels)
print(sentiments)
</code></pre>

srishti-git1110: <p>Hi Ravindra,<br>
The code that you have posted most probably does not contain the line that’s producing the error <code>RuntimeError: shape ‘[-1, 2]’ is invalid for input of size 9</code>.<br>
You must be trying to reshape/view a tensor somewhere that’s causing this error. The number of elements in your initial tensor (9) aren’t compatible with the shape that you’re trying to achieve (some rows * 2 columns).</p>
<p>Could you please post the relevant part of the code? I can help debug further.</p>

ravindra: <p>this is my model</p>
<pre><code class="lang-auto">from transformers import DistilBertTokenizer, DistilBertForSequenceClassification
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
model = DistilBertForSequenceClassification.from_pretrained('distilbert-base-uncased')
</code></pre>
<p>This is my basic text data</p>
<pre><code class="lang-auto">texts = ['I loved the movie. It was great!',
         'The food was terrible.',
         'The weather is okay.']
sentiments = ['positive', 'negative', 'neutral']
</code></pre>
<p>Tokenize the text samples</p>
<pre><code class="lang-auto">encoded_texts = tokenizer(texts, padding = True, truncation = True, return_tensors = 'pt')
input_ids = encoded_texts['input_ids']
decoding = tokenizer.decode(input_ids[2])
</code></pre>
<p>Attention Mask<br>
<code>attention_mask =  encoded_texts['attention_mask']</code></p>
<p><code>sentiment_labels = [sentiments.index(sentiment) for sentiment in sentiments]</code></p>
<pre><code class="lang-auto">import torch.nn as nn
num_classes = len(set(sentiment_labels))
classification_head = nn.Linear(model.config.hidden_size, num_classes)
model.classifier = classification_head
</code></pre>
<p>import torch<br>
import torch.optim as optim</p>
<pre><code class="lang-auto">
optimizer = optim.AdamW(model.parameters(), lr = 2e-5)  # lr = 0.00002
criterion = torch.nn.CrossEntropyLoss()

num_epochs = 3
for epoch in range(num_epochs):
  optimizer.zero_grad()
  output = model(input_ids, attention_mask = attention_mask, labels = torch.tensor(sentiment_labels))
  loss = output.loss
  loss.backward()
  optimizer.step()
</code></pre>
<p>The above code is simple but I didn’t able to figure out where I am doing mistake.<br>
Thank you <a class="mention" href="/u/srishti-git1110">@srishti-git1110</a></p>

srishti-git1110: <p>Thanks for the code!<br>
I looked into hugging face’s source code and found that it’s indeed a <code>view</code> operation that’s causing this error. Specifically, it is -</p>
<pre data-code-wrap="python"><code class="lang-python">loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
</code></pre>
<p>This line is present in the <code>forward</code> method and hence <code>self</code> (the currently calling object) refers to the <code>model</code>. If you further try to do <code>print(model.num_labels)</code>, it indeed gives 2 rather than 3 which explains the 2 in your error - <code>RuntimeError: shape ‘[-1, 2]’ is invalid for input of size 9.</code></p>
<p>To get rid of this, you basically need to change only one line of code which is:</p>
<pre><code class="lang-auto">model = DistilBertForSequenceClassification.from_pretrained('distilbert-base-uncased', num_labels=3)
</code></pre>