Expected target size in loss

username: Bolanus
cooked: <p>Hi everyone,</p>
<p>I’m used to other deep learning frameworks and it’s been a while since I’ve used PyTorch. Having a problem with my forward method. I viewed the other topics which were suggested but I couldn’t quite find anything helpful.</p>
<p>This is the error, 32 is the batch size in the dataloader:</p>
<pre><code class="lang-auto">return torch._C._nn.nll_loss_nd(input, target, weight, _Reduction.get_enum(reduction), ignore_index)

RuntimeError: Expected target size [32, 3], got [32]
</code></pre>
<p>This is the model:</p>
<pre><code class="lang-auto">X = pad_sequence([torch.LongTensor(s) for s in df['Tweets']], batch_first=True, padding_value=0)
y = torch.tensor(df['Sentiments'].values)

trainCount = int(.7 * TotalObs)
validCount = int(.2*TotalObs)
testCount = TotalObs-trainCount-validCount
dataset = TensorDataset(X,y)
trainDataset, validDataset, testDataset = torch.utils.data.random_split(dataset, (trainCount, validCount, testCount))
BATCH_SIZE=32
train_dataset_loader = DataLoader(trainDataset, batch_size=BATCH_SIZE, shuffle=True)  
valid_dataset_loader = DataLoader(validDataset, batch_size=BATCH_SIZE, shuffle=True) 
test_dataset_loader  = DataLoader(testDataset , batch_size=BATCH_SIZE, shuffle=False)
embedding_dim = 5709
hidden_dim=128
num_labels=3

class LSTMNetwork(nn.Module):
    def __init__(self, embedding_dim, hidden_dim, num_labels):
        super(LSTMNetwork, self).__init__()
        self.lstm = nn.LSTM(embedding_dim, hidden_dim)
        self.label = nn.Linear(hidden_dim*20, num_labels)
    
    def forward(self,tweet):
        h_n, c_n = self.lstm(tweet)
        x =self.label(h_n.view(len(tweet),1,-1))
        return x

model = LSTMNetwork(embedding_dim, hidden_dim, vocab_size, num_labels).to(device)
optimizer = optim.SGD(model.parameters(), lr=1e-3)
loss_fn=NLLLoss()

def train(dataloader, model, loss_fn, optimizer):
    size = len(dataloader.dataset)
    model.train()
    count = 0
    for batch, (X,y) in enumerate(dataloader):
        X, y = X.to(device), y.to(device)
        pred=model(X)
        loss = loss_fn(pred,y)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
</code></pre>
<p>The result of:</p>
<pre><code class="lang-auto">x,y= next(iter(train_dataset_loader))
x.shape
torch.Size([32, 20, 128])
y.shape
Out[220]: torch.Size([32])
</code></pre>
<p>Am I right in assuming that the 3 dimensions expected are because it’s a 32x3 tensor with the predictions for each class? If so, how do I correct that? Let me know if you need more information.</p>
<p>Thanks again everyone!</p>

username: KFrank
cooked: <p>Hi Bolanus!</p>
<aside class="quote no-group quote-modified" data-username="Bolanus" data-post="1" data-topic="146991" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bolanus/48/47761_2.png" class="avatar"> Bolanus:</div>
<blockquote>
<pre><code class="lang-auto">RuntimeError: Expected target size [32, 3], got [32]
</code></pre>
<p>…</p>
<pre><code class="lang-auto">        loss = loss_fn(pred,y)
</code></pre>
</blockquote>
</aside>
<p>Perhaps the most likely cause of your error would be an unexpected<br>
singleton dimension sneaking into <code>pred</code>.</p>
<p>Less likely would be <code>NLLLoss</code> getting confused by <code>y</code> having the wrong<br>
type (but such a scenario wouldn’t give you the error message you quote<br>
in the latest stable version of pytorch, 1.11.0).</p>
<p>Try printing out the <code>shape</code> and <code>dtype</code> of <code>pred</code> and <code>y</code> just before you<br>
call <code>loss_fn (pred, y)</code>.  Also, tell us what version of pytorch you are<br>
using.</p>
<p>Best.</p>
<p>K. Frank</p>