IndexError when finetuning barthez on summarization
Author: moussaKam
Pull request: no

## Environment info
- `transformers` version: 4.3.0.dev0
- Platform: Linux-4.4.0-197-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no



### Who can help
@LysandreJik  @patrickvonplaten 

## Information

Model I am using (Bert, XLNet ...): BARThez

## To reproduce

Steps to reproduce the behavior:

```
python finetune_trainer.py --learning_rate 3e-5 --fp16 --evaluation_strategy steps --predict_with_generate --model_name_or_path moussaKam/barthez --data_dir xsum --do_train --do_eval --output_dir welcome_back --per_device_train_batch_size 4 --task summarization --max_target_length 50 --overwrite_output_dir --eval_steps 50 --n_val 20
```

```
Traceback (most recent call last):
  File "finetune_trainer.py", line 373, in <module>
    main()
  File "finetune_trainer.py", line 303, in main
    train_result = trainer.train(
  File "/datadisks/datadisk1/transformers/src/transformers/trainer.py", line 942, in train
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch)
  File "/datadisks/datadisk1/transformers/src/transformers/trainer.py", line 1017, in _maybe_log_save_evaluate
    metrics = self.evaluate()
  File "/datadisks/datadisk1/transformers/src/transformers/trainer_seq2seq.py", line 96, in evaluate
    return super().evaluate(eval_dataset, ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix)
  File "/datadisks/datadisk1/transformers/src/transformers/trainer.py", line 1458, in evaluate
    output = self.prediction_loop(
  File "/datadisks/datadisk1/transformers/src/transformers/trainer.py", line 1617, in prediction_loop
    metrics = self.compute_metrics(EvalPrediction(predictions=preds, label_ids=label_ids))
  File "/datadisks/datadisk1/transformers/examples/seq2seq/utils.py", line 92, in summarization_metrics
    pred_str, label_str = decode_pred(pred)
  File "/datadisks/datadisk1/transformers/examples/seq2seq/utils.py", line 86, in decode_pred
    label_str = tokenizer.batch_decode(pred.label_ids, skip_special_tokens=True)
  File "/datadisks/datadisk1/transformers/src/transformers/tokenization_utils_base.py", line 3070, in batch_decode
    return [
  File "/datadisks/datadisk1/transformers/src/transformers/tokenization_utils_base.py", line 3071, in <listcomp>
    self.decode(
  File "/datadisks/datadisk1/transformers/src/transformers/tokenization_utils_base.py", line 3109, in decode
    return self._decode(
  File "/datadisks/datadisk1/transformers/src/transformers/tokenization_utils.py", line 711, in _decode
    filtered_tokens = self.convert_ids_to_tokens(token_ids, skip_special_tokens=skip_special_tokens)
  File "/datadisks/datadisk1/transformers/src/transformers/tokenization_utils.py", line 695, in convert_ids_to_tokens
    tokens.append(self._convert_id_to_token(index))
  File "/datadisks/datadisk1/transformers/src/transformers/models/barthez/tokenization_barthez.py", line 237, in _convert_id_to_token
    return self.sp_model.IdToPiece(index)
  File "/home/dascim/anaconda3/envs/transformers/lib/python3.8/site-packages/sentencepiece/__init__.py", line 501, in _batched_func
    return _func(self, arg)
  File "/home/dascim/anaconda3/envs/transformers/lib/python3.8/site-packages/sentencepiece/__init__.py", line 494, in _func
    raise IndexError('piece id is out of range.')
IndexError: piece id is out of range.
```

## Expected behavior

For some reason the tokenizer is trying to decode some -100 id. 


Comments:
Hmmmm the -100 id should be linked to the ignored values. It shouldn't try to decode this. Pinging @sgugger 

Not sure why there would be `-100` in the labels with the old script. Note that we are not maintaining that one anymore and will replace it with `run_seq2seq` which is almost ready for use (misses a few features you don't seem to be using in your command anyway).

If you really need the old one, you should add the line
```
labels = np.where(labels != -100, labels, tokenizer.pad_token_id)
```
in the `compute_metric` function to replace the -100s by the pad token id.

We also fixed this very recently (yesterday) in the model, see: https://github.com/huggingface/transformers/commit/74f16b82765a05eccee45e80d79370202a958873 => so you should also be able to run: 
```
python finetune_trainer.py --learning_rate 3e-5 --fp16 --evaluation_strategy steps --predict_with_generate --model_name_or_path moussaKam/barthez --data_dir xsum --do_train --do_eval --output_dir welcome_back --per_device_train_batch_size 4 --task summarization --max_target_length 50 --overwrite_output_dir --eval_steps 50 --n_val 20
```
on master now.

However, as @sgugger points out, we strongly recommend using the `run_seq2seq.py` script  from now on as we won't continue maintaining `finetune_trainer.py` anymore.