seq2seq examples can't handle DataParallel
Author: stas00
Pull request: no

### System Info

main

### Who can help?

@sgugger

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

This was originally reported here: https://github.com/pytorch/pytorch/issues/98102#issuecomment-1496173632

with 2+ gpus:

```
PYTHONPATH=src python examples/pytorch/translation/run_translation.py \
--model_name_or_path t5-base --do_train --do_eval --source_lang en \
--target_lang de --source_prefix 'translate English to German: ' \
--dataset_name stas/wmt14-en-de-pre-processed --output_dir \
/tmp/tst-translation --num_train_epochs 1 --per_device_train_batch_size=1 \
--max_train_samples 10 --overwrite_output_dir --seed 1137 \
--per_device_eval_batch_size 1 --predict_with_generate --fp16 \
--max_eval_samples 10
```
crashes:
```
[INFO|configuration_utils.py:575] 2023-04-04 09:20:48,136 >> Generate config GenerationConfig {
  "_from_model_config": true,
  "decoder_start_token_id": 0,
  "eos_token_id": 1,
  "pad_token_id": 0,
  "transformers_version": "4.28.0.dev0"
}

Traceback (most recent call last):
  File "examples/pytorch/translation/run_translation.py", line 664, in <module>
    main()
  File "examples/pytorch/translation/run_translation.py", line 605, in main
    metrics = trainer.evaluate(max_length=max_length, num_beams=num_beams, metric_key_prefix="eval")
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer_seq2seq.py", line 159, in evaluate
    return super().evaluate(eval_dataset, ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix)
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer.py", line 2990, in evaluate
    output = eval_loop(
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer.py", line 3171, in evaluation_loop
    loss, logits, labels = self.prediction_step(model, inputs, prediction_loss_only, ignore_keys=ignore_keys)
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer_seq2seq.py", line 280, in prediction_step
    gen_config = model.generation_config
  File "/home/stas/anaconda3/envs/py38-pt20/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1614, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'DataParallel' object has no attribute 'generation_config'
```

Using a workaround `CUDA_VISIBLE_DEVICES=0` overcomes this problem - so we aren't dealing with wrapping properly here.

But then it fails again inside eval:

```
[INFO|trainer.py:3126] 2023-04-04 09:28:07,548 >> ***** Running Evaluation *****
[INFO|trainer.py:3128] 2023-04-04 09:28:07,548 >>   Num examples = 10
[INFO|trainer.py:3131] 2023-04-04 09:28:07,548 >>   Batch size = 1
[INFO|configuration_utils.py:575] 2023-04-04 09:28:07,552 >> Generate config GenerationConfig {
  "_from_model_config": true,
  "decoder_start_token_id": 0,
  "eos_token_id": 1,
  "pad_token_id": 0,
  "transformers_version": "4.28.0.dev0"
}

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:02<00:00,  3.72it/s]Traceback (most recent call last):
  File "examples/pytorch/translation/run_translation.py", line 664, in <module>
    main()
  File "examples/pytorch/translation/run_translation.py", line 605, in main
    metrics = trainer.evaluate(max_length=max_length, num_beams=num_beams, metric_key_prefix="eval")
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer_seq2seq.py", line 159, in evaluate
    return super().evaluate(eval_dataset, ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix)
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer.py", line 2990, in evaluate
    output = eval_loop(
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer.py", line 3278, in evaluation_loop
    metrics = self.compute_metrics(EvalPrediction(predictions=all_preds, label_ids=all_labels))
  File "examples/pytorch/translation/run_translation.py", line 546, in compute_metrics
    decoded_preds = tokenizer.batch_decode(preds, skip_special_tokens=True)
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/tokenization_utils_base.py", line 3445, in batch_decode
    return [
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/tokenization_utils_base.py", line 3446, in <listcomp>
    self.decode(
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/tokenization_utils_base.py", line 3485, in decode
    return self._decode(
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/tokenization_utils_fast.py", line 549, in _decode
    text = self._tokenizer.decode(token_ids, skip_special_tokens=skip_special_tokens)
OverflowError: out of range integral type conversion attempted
```


Comments:
No, it is a wrapping problem.
@gante The Seq2SeqTrainer might need to do something to use the unwrapped model (which is the `self.model` attribute) instead of the model. I think just changing line 280 to use `self.model.config` instead of `model.config` will be enough.

and a new test please! Thank you!

This was caught by the last scheduled test, looking at the reports right now. It's just that Yih-Dar is off so didn't ping anyone on it :-)

oh, then all is perfect testing-wise!

In the interim perhaps before merging Trainer-related PRs those slow trainer-only tests could be run locally - would require 2 gpus I think.

------------------

and I'd imagine the subsequent crash was not detected by the test and it's not wrapping related it seems. (part 2 of my Issue)

I split off the 2nd issue into its own Issue https://github.com/huggingface/transformers/issues/22634 as they aren't really related

So closing this one as the first part has been resolved here https://github.com/huggingface/transformers/pull/22584