Repository: pytorch
Issue: CrossEntropyLoss target shape expectation is inconsistent with BCEWithLogits
Labels: triaged,module: shape checking

## 🐛 Bug

This isn't necessarily a bug, but...

When working with most losses in pytorch, including BCEWithLogits, the expectation is that the number of dimensions in the input and target match, i.e. input is (N, *) and target is (N, *). In comparison, CrossEntropyLoss the input is expected to be size (N, k, *) and the target is expected to be (N, *).

This leads to some potentially confusing behavior for an end-user where binary classification tasks require shaping the target tensor as (N, 1) to match the implicit neural network channel dimension, while multiclass classification does not.  Further, the error message "1D target tensor expected, multi-target not supported" is somewhat confusing, if nothing else a more direct error message for when an (N, 1, *) target is passed with an (N, k, *) input such as, "RuntimeError: CrossEntropyLoss expects an (N, *) shaped target when given an (N, k, *) shaped input" would likely be helpful.

## To Reproduce

Steps to reproduce the behavior:

A user likely starts with a binary classification problem where they had the following correct code
```
simple_net = nn.Linear(3,1)
loss_bce = nn.BCEWithLogitsLoss()
input = torch.randn(3, 3)
pred = simple_net(input)
# This is correct, size needs to be (3,1) because Linear doesn't squeeze
target = torch.empty(3, 1).random_(2)
output = loss_bce(pred, target)
```

The user might then try to adapt that code for multi-classification, which is consistent with the BCE code, but is actually wrong.
```
simple_net = nn.Linear(3,5)
loss_ce = nn.CrossEntropyLoss()
input = torch.randn(3, 3, requires_grad=True)
pred = simple_net(input)
# This is wrong, should only be size = (3,) not (3,1)
target = torch.empty(3, 1 dtype=torch.long).random_(5)
output = loss_ce(pred, target)
```

```
  File "/.../torch/nn/modules/loss.py", line 961, in forward
    return F.cross_entropy(input, target, weight=self.weight,
  File "/.../torch/nn/functional.py", line 2468, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/.../torch/nn/functional.py", line 2264, in nll_loss
    ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: 1D target tensor expected, multi-target not supported

```
## Expected behavior

BCEWithLogitsLoss and CrossEntropyLoss would have consistent behavior or the error would identify the problem more clearly.

## Environment
PyTorch version: 1.7.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 10.13.6 (x86_64)
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.11.3

Python version: 3.8 (64-bit runtime)
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.1
[pip3] torch==1.7.1
[pip3] torchvision==0.8.2
[conda] blas                      1.0                         mkl  
[conda] mkl                       2019.4                      233  
[conda] mkl-service               2.3.0            py38h9ed2024_0  
[conda] mkl_fft                   1.3.0            py38ha059aab_0  
[conda] mkl_random                1.1.1            py38h959d312_0  
[conda] numpy                     1.20.1                   pypi_0    pypi
[conda] numpy-base                1.19.2           py38hcfb5961_0  
[conda] torch                     1.7.1                    pypi_0    pypi
[conda] torchvision               0.8.2                    pypi_0    pypi]

## Additional context

If anyone finds themselves here due to this error, you just have an extra dimension in your target, and you have to do target.squeeze(1) as I show here:
```
simple_net = nn.Linear(3,5)
loss_ce = nn.CrossEntropyLoss()
input = torch.randn(3, 3, requires_grad=True)
pred = simple_net(input)
# This is wrong, should only be size = (3,) not (3,1)
target = torch.empty(3, 1 dtype=torch.long).random_(5)
# Adding a squeeze here fixes the issue of having the wrong shape
target.squeeze(1)
output = loss_ce(pred, target)
```

From discussion pages, this does seem to trip people up, and the confusion gets combined with confusion over encodings.  For example discussions on the "1D Target Tensor expected error" at https://discuss.pytorch.org/t/runtimeerror-multi-target-not-supported-newbie/10216/19, https://discuss.pytorch.org/t/runtimeerror-multi-target-not-supported-newbie/10216/23