Repository: huggingface/transformers

Issue:
[Example] Using label_smoothing_factor raise error when evaluating model
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.2
- Platform: Ubuntu 20.04
- Python version: 3.8
- PyTorch version (GPU): 1.6.0

### Who can help

Library:
- pipelines: @LysandreJik

Examples:
- maintained examples (not research project or legacy): @sgugger, @patil-suraj

## Information

Model I am using BERT:

The problem arises when using:
* [x] the official example scripts: https://github.com/huggingface/transformers/blob/master/examples/legacy/token-classification/run_ner.py
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. I run the old script run_ner.py with default label_smoothing_factor = 0.0.  It works well.

2. I add label_smoothing_factor = 0.1 to JSON config file. 
`{
    "data_dir": "/home/dzungle/NER/data/",
    "train_file": "/home/dzungle/NER/data/train.csv",
    "validation_file": "/home/dzungle/data/dev.csv",
    "model_name_or_path": "emilyalsentzer/Bio_ClinicalBERT",
    "output_dir": "/home/dzungle/NER/models/",
    "label_smoothing_factor": 0.1,
    "max_seq_length": 256,
    "num_train_epochs": 1,
    "per_device_train_batch_size": 8,
    "gradient_accumulation_steps": 4,
    "per_device_eval_batch_size": 1,
    "save_steps": 1000,
    "eval_steps" : 50,
    "save_total_limit":1,
    "seed": 1,
    "do_train": true,
    "do_eval": true,
    "do_predict": true,
    "overwrite_output_dir" : true,
    "evaluate_during_training" : true
}`

3. I run the script and it works well for training but got an error when evaluating.

**Error:**
```
Traceback (most recent call last):
  File "run_ner.py", line 333, in <module>
    main()
  File "run_ner.py", line 282, in main
    result = trainer.evaluate()
  File "/home/dzungle/miniconda3/envs/hppi/lib/python3.8/site-packages/transformers/trainer.py", line 1604, in evaluate
    output = self.prediction_loop(
  File "/home/dzungle/miniconda3/envs/hppi/lib/python3.8/site-packages/transformers/trainer.py", line 1742, in prediction_loop
    loss, logits, labels = self.prediction_step(model, inputs, prediction_loss_only, ignore_keys=ignore_keys)
  File "/home/dzungle/miniconda3/envs/hppi/lib/python3.8/site-packages/transformers/trainer.py", line 1874, in prediction_step
    labels = nested_detach(tuple(inputs.get(name) for name in self.label_names))
  File "/home/dzungle/miniconda3/envs/hppi/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", line 111, in nested_detach
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/home/dzungle/miniconda3/envs/hppi/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", line 111, in <genexpr>
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/home/dzungle/miniconda3/envs/hppi/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", line 112, in nested_detach
    return tensors.detach()
AttributeError: 'NoneType' object has no attribute 'detach'
```
## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
As I known, label_smoothing_factor is a new feature of recent transformers version. I would expect that the script with label_smoothing_factor=0.1 works well as using default value 0.0.



Discussion:
Can reproduce locally, here is a short reproducer from the root of the repo:
```
python examples/token-classification/run_ner.py \
  --model_name_or_path bert-base-uncased \
  --train_file tests/fixtures/tests_samples/conll/sample.json \
  --validation_file tests/fixtures/tests_samples/conll/sample.json \
  --output_dir /tmp/test-ner \
  --overwrite_output_dir \
  --do_train \
  --do_eval \
  --label_smoothing_factor 0.1
```

Will look into it tomorrow.

Resolving patch:
diff --git a/src/transformers/trainer.py b/src/transformers/trainer.py
--- a/src/transformers/trainer.py
+++ b/src/transformers/trainer.py
@@ -1888,6 +1888,14 @@ def prediction_step(
             else:
                 ignore_keys = []
 
+        # labels may be popped when computing the loss (label smoothing for instance) so we grab them first.
+        if has_labels:
+            labels = nested_detach(tuple(inputs.get(name) for name in self.label_names))
+            if len(labels) == 1:
+                labels = labels[0]
+        else:
+            labels = None
+
         with torch.no_grad():
             if has_labels:
                 loss, outputs = self.compute_loss(model, inputs, return_outputs=True)
@@ -1918,13 +1926,6 @@ def prediction_step(
         if len(logits) == 1:
             logits = logits[0]
 
-        if has_labels:
-            labels = nested_detach(tuple(inputs.get(name) for name in self.label_names))
-            if len(labels) == 1:
-                labels = labels[0]
-        else:
-            labels = None
-
         return (loss, logits, labels)
 
     def floating_point_ops(self, inputs: Dict[str, Union[torch.Tensor, Any]]):


Test patch: