RuntimeError: Expected floating point type for target with class probabilities, got Long when using Cross Entropy Loss

coder_456: <p>I am getting this error when using CrossEntropyLoss().<br>
I am only training a linear layer and use a pre-trained model.<br>
I also tried sending output through softmax but still got the same error.</p>
<pre><code class="lang-auto">cross_entropy_loss = nn.CrossEntropyLoss()
    optimizer = torch.optim.Adam(linear_layer.parameters(), lr=3e-4)

    for epoch_idx in range(args.linear_max_epoch):
        linear_layer.train()
        loss_sum = 0
        total, correct = 0, 0

        for data, gt_labels in train_loader:
            data, gt_labels = data.to(device).float(), gt_labels.to(device)
            # data shape  torch.Size([512, 512])  [batch size, features]
            gt_labels = fn.one_hot(gt_labels.long(), num_classes = 5)
            # target shape  torch.Size([512, 5])
            output = linear_layer(data)   
            # tried output = linear_layer(data).softmax(dim = 1) and still got error
            # output shape  torch.Size([512, 5])
            # output tensor contains probabilities (output from a linear_layer)
            loss = cross_entropy_loss(output, gt_labels)   # THIS IS WHERE I GET THE ERROR
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            loss_sum += loss.detach().cpu().numpy()

            _, predicted = torch.max(output.data, 1)
            total += gt_labels.size(0)
            correct += (predicted == gt_labels).sum().item()
</code></pre>
<p>The error is,</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "tools/linear_eval.py", line 156, in &lt;module&gt;
    linear_eval_train(1)
  File "tools/linear_eval.py", line 104, in linear_eval_train
    loss = cross_entropy_loss(output, gt_labels)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/loss.py", line 1166, in forward
    label_smoothing=self.label_smoothing)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py", line 3014, in cross_entropy
    return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)
RuntimeError: Expected floating point type for target with class probabilities, got Long
</code></pre>

ptrblck: <p><code>nn.CrossEntropyLoss</code> expects <code>LongTensor</code>s as the target containing class indices or (in newer PyTorch releases) <code>FloatTensor</code>s in case you are using probabilities. Since you are one-hot encoding the tensor (I don’t know why you would need it) you should pass it as <code>FloatTensor</code>s.</p>
<p>Also,</p>
<aside class="quote no-group" data-username="coder_456" data-post="1" data-topic="167046">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/coder_456/48/55225_2.png" class="avatar"> coder_456:</div>
<blockquote>
<p>I also tried sending output through softmax</p>
</blockquote>
</aside>
<p>this is wrong, since <code>nn.CrossEntropyLoss</code> expects raw logits.</p>