Unintentional(?) interface change on loss function in models didn't work well for single-column regression
Author: yoshitomo-matsubara
Pull request: no

The recent PR #11012 changed the interface of forward function for `labels` in regression tasks as it skips `.view(-1)`  in loss function like [this](https://github.com/huggingface/transformers/pull/11012/files#diff-a48ba7f6444ca4954a58f1ac3e66c7941a2bbc4615649d56b182aeac8cc36d9cL1523). 
As shown below, that causes `UserWarning: Using a target size (torch.Size([32])) that is different to the input size (torch.Size([32, 1]))` with BERT in glue example code, and it looks like this change was applied **not only to BERT but also a lot of models in #11012**

To resolve it, the current example glue code needs `if` statement that transform `labels` variable before `forward` function only for regression task.
But if the interface change in the PR was unintentional, I think we should revert `.view(-1)` in loss function.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.6.0
- Platform: Google Colab
- Python version: 3.7
- PyTorch version (GPU?): 1.8.1
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@abhi1thakur @sgugger @LysandreJik from #11012

<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.


Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @Rocketknight1

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...): `bert-base-uncased`

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run 
```
mkdir /tmp/stsb/ -p

python transformers/examples/pytorch/text-classification/run_glue_no_trainer.py \
  --model_name_or_path bert-base-cased \
  --task_name stsb \
  --max_length 128 \
  --per_device_train_batch_size 32 \
  --learning_rate 2e-5 \
  --num_train_epochs 3 \
  --output_dir /tmp/stsb/
```

2. We will see `UserWarning: Using a target size (torch.Size([32])) that is different to the input size (torch.Size([32, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.` which didn't appear with the previous version of transformers like 2-3 weeks ago.  

```
05/19/2021 22:39:45 - INFO - __main__ -   ***** Running training *****
05/19/2021 22:39:45 - INFO - __main__ -     Num examples = 5749
05/19/2021 22:39:45 - INFO - __main__ -     Num Epochs = 3
05/19/2021 22:39:45 - INFO - __main__ -     Instantaneous batch size per device = 32
05/19/2021 22:39:45 - INFO - __main__ -     Total train batch size (w. parallel, distributed & accumulation) = 32
05/19/2021 22:39:45 - INFO - __main__ -     Gradient Accumulation steps = 1
05/19/2021 22:39:45 - INFO - __main__ -     Total optimization steps = 540
  0% 0/540 [00:00<?, ?it/s]/usr/local/lib/python3.7/dist-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([32])) that is different to the input size (torch.Size([32, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)
 33% 178/540 [00:24<00:47,  7.66it/s]/usr/local/lib/python3.7/dist-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([21])) that is different to the input size (torch.Size([21, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)
 33% 180/540 [00:24<00:43,  8.26it/s]/usr/local/lib/python3.7/dist-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([8])) that is different to the input size (torch.Size([8, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)
/usr/local/lib/python3.7/dist-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([4])) that is different to the input size (torch.Size([4, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)
05/19/2021 22:40:13 - INFO - /usr/local/lib/python3.7/dist-packages/datasets/metric.py -   Removing /root/.cache/huggingface/metrics/glue/stsb/default_experiment-1-0.arrow
05/19/2021 22:40:13 - INFO - __main__ -   epoch 0: {'pearson': 0.40341441213742524, 'spearmanr': 0.41749739006146336}
 66% 359/540 [00:52<00:24,  7.26it/s]05/19/2021 22:40:40 - INFO - /usr/local/lib/python3.7/dist-packages/datasets/metric.py -   Removing /root/.cache/huggingface/metrics/glue/stsb/default_experiment-1-0.arrow
05/19/2021 22:40:40 - INFO - __main__ -   epoch 1: {'pearson': 0.4407148954008369, 'spearmanr': 0.4550002378117188}
100% 539/540 [01:20<00:00,  7.42it/s]05/19/2021 22:41:08 - INFO - /usr/local/lib/python3.7/dist-packages/datasets/metric.py -   Removing /root/.cache/huggingface/metrics/glue/stsb/default_experiment-1-0.arrow
05/19/2021 22:41:08 - INFO - __main__ -   epoch 2: {'pearson': 0.4408745967619131, 'spearmanr': 0.43830345183360847}
Configuration saved in /tmp/stsb/config.json
Model weights saved in /tmp/stsb/pytorch_model.bin
100% 540/540 [01:24<00:00,  6.42it/s]
```

As a result, this gave me a pretty bad performance `epoch 2: {'pearson': 0.4408745967619131, 'spearmanr': 0.43830345183360847}` while they were both around 0.87 with the previous version.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
The following warning (actually should be a bug) should not appear, and the validation performance pearson and spearmanr should be around 0.87 with the parameters given in the example command.

`UserWarning: Using a target size (torch.Size([32])) that is different to the input size (torch.Size([32, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.`
<!-- A clear and concise description of what you would expect to happen. -->


Comments:
Yes the change was unintentional to enable multi-label regression. I think the old
```
loss = loss_fct(logits.view(-1), labels.view(-1))
```
will work in the case of one or several labels but might not give a clear error message if we have multiple labels but a shape error (if we have 5 possible lables but the model was configured with 4, we would see, with a batch size of 8, an error saying shape incompatibility between a tensor of size 32 and a tensor of size 40).

Something that would give a nicer error message is probably:
```
if self.num_labels == 1:
    loss = loss_fct(logits.squeeze(), labels.squeeze())
else:
    loss =  loss_fct(logits, labels)
```
which would take care of this problem and show a clear error message.

I can implement that change quickly and we should do a patch release but want to check the fix seems ok. What do you think @LysandreJik and @abhi1thakur ?

Thank you @sgugger for your prompt response!
I also found #11583 reports a weird result with STS-B and could be fixed by the patch.

Indeed, I don't know why I couldn't reproduce the bad results earlier but this is definitely the same issue (I probably wasn't trying on the master branch.)