Fine tune with local model raised `torch.nn.modules.module.ModuleAttributeError: 'DataParallel' object has no attribute 'config'`
Author: Abbyyan
Pull request: no

# ❓ Questions & Help

I've fined tune distilgpt2 with `run_language_modeling.py` under my local `output_dir` and want to fine tune with the model in `output_dir` again, but it raised `torch.nn.modules.module.ModuleAttributeError: 'DataParallel' object has no attribute 'config'`. 

## Details
<!-- Description of your issue -->
I've fined tune distilgpt2 with `run_language_modeling.py` as follows:
```shell
python3 run_language_modeling.py     --output_dir=/home/xxx/gpt_model/transformers/examples/language-modeling/output_dir --model_type=gpt2  --model_name_or_path=distilgpt2 --per_device_train_batch_size=10     --do_train     --train_data_file=/home/xxx/gpt_model/data_info/data.txt --block_size=64 --save_steps=100 --overwrite_output_dir
```
It works fine and i can get the fine-tuned model after training with my data `data.txt`. Now i want to fine-tune in the `output_dir` so i run `run_language_modeling.py` as follows on my `new_data.txt`:
```shell
 python3 run_language_modeling.py     --output_dir=/home/xxx/gpt_model/transformers/examples/language-modeling/output_new_data --model_type=gpt2  --model_name_or_path=/home/xxx/gpt_model/transformers/examples/language-modeling/output_dir/  --per_device_train_batch_size=20  --do_train     --train_data_file=/home/xxx/gpt_model/data_info/new_data.txt --block_size=64 --save_steps=1000 --overwrite_output_dir
```
But it raise exception and exit. The stderr is as follows
```shell
/home/xxx/gpt_model/pytorch/pytorch/torch/optim/lr_scheduler.py:235: UserWarning: Please also save or load the state of the optimizer when saving or loading the scheduler.
  warnings.warn(SAVE_STATE_WARNING, UserWarning)
Traceback (most recent call last):
  File "run_language_modeling.py", line 320, in <module>
    main()
  File "run_language_modeling.py", line 284, in main
    trainer.train(model_path=model_path)
  File "/home/xxx/anaconda3/envs/transformers/lib/python3.6/site-packages/transformers/trainer.py", line 683, in train
    self.total_flos = getattr(model.config, "total_flos", 0)
  File "/home/xxx/gpt_model/pytorch/pytorch/torch/nn/modules/module.py", line 779, in __getattr__
    type(self).__name__, name))
torch.nn.modules.module.ModuleAttributeError: 'DataParallel' object has no attribute 'config'
```
##### (1) I'm training on multiple GPUs, but i did'n specify a specific gpu to run. So at first i think this may caused by multiple GPUs, and add the following code under https://github.com/huggingface/transformers/blob/52d250f6aa14844024806e5e4dd1c7882bbd8dd5/src/transformers/trainer.py#L641
```shell
if isinstance(model,torch.nn.DataParallel):
		model = model.module
```
It has no error but doesn't work. The output is as follows . After that, the process is killed and return code `echo $?` is `0`.
```shell
Epoch: 0it [00:00, ?it/s]
/home/lenajin/anaconda3/envs/transformers/lib/python3.6/site-packages/transformers/trainer.py:1087: FutureWarning: This method is deprecated, use `Trainer.is_world_process_zero()` instead.
  warnings.warn("This method is deprecated, use `Trainer.is_world_process_zero()` instead.", FutureWarning)
```
##### (2) Following with https://github.com/huggingface/transformers/issues/1991 , I try to train with following script to train with gpu whose id is `1`.
```shell
CUDA_VISIBLE_DEVICES=1 python3 run_language_modeling.py     --output_dir=/home/xxx/gpt_model/transformers/examples/language-modeling/output_dir --model_type=gpt2  --model_name_or_path=distilgpt2 --per_device_train_batch_size=20     --do_train     --train_data_file=/home/xxx/gpt_model/data_info/data.txt --block_size=64 --save_steps=100 --overwrite_output_dir
```
it return with return code `echo $?` =`0`. At first i don't know what's the problem, but now i changed `--per_device_train_batch_size=20    ` from 20 to 10. It seems all is well, and now my `GPU-Util` is about `98%`.  Maybe it return without training duing to my limit gpu CC？It's wired but at least it works now. Maybe there can give some error message about the return reason?


Comments:
I haven't used the trainer but I think maybe you just need to change the line `self.total_flos = getattr(model.config, "total_flos", 0)` to 

`_model = model.module if hasattr(model, 'module') else model `
`self.total_flos = getattr(_model.config, "total_flos", 0)`

This is a breaking change that was not announced - it broke our production script.

@sgugger might be interested in this.

@marrrcin what is the breaking change? This seems to be a bug rather than an intentional change.

I meat it's a bug that broke our usage of Trainer in production - we're using single-multigpu-machine to train our models and https://github.com/huggingface/transformers/blob/9e68d075a4100906509170498480823e7e61874a/src/transformers/trainer.py#L698 basically breaks it after upgrade to `3.2.0`.
Maybe this is enough for the fix: 
https://github.com/huggingface/transformers/pull/7384


I see! Thanks for opening a PR!

My bad, I think this bug slipped through with one of the style changes at code review on one of my PRs. Thanks for raising the issue!

Do you think that it will be possible to move this fix forward and release patch over the weekend? We don't want to have custom package installation in our CI.

We're having a new release (v3.3.0) on Monday, is that soon enough? The fix will be in it.

That would be great!

I am facing the same issue. Whenever I continues my training from certain checkpoint instead of training from scratch. I got the exact same error as OP. Should I just wait for the new release on tomorrow?

Hi,
I tested this PR, this does not fully solve the issue, I am getting these error during evaluation of seq2seq_trainer.


 File "finetune_t5_trainer.py", line 233, in <module>
    main()
  File "finetune_t5_trainer.py", line 188, in main
    result = trainer.evaluate(eval_datasets, compute_metrics_fn)
  File "/home/rabeeh/internship/seq2seq/t5_trainer.py", line 175, in evaluate
    prediction_loss_only=True if self.compute_metrics is None else None, # self.compute_metrics[eval_task]
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/trainer.py", line 1452, in prediction_loop
    metrics = self.compute_metrics(EvalPrediction(predictions=preds, label_ids=label_ids))
  File "/home/rabeeh/internship/seq2seq/metrics/metrics.py", line 112, in translation_metrics
    pred_str, label_str = decode_pred(pred)
  File "/home/rabeeh/internship/seq2seq/metrics/metrics.py", line 99, in decode_pred
    label_str = tokenizer.batch_decode(pred.label_ids, skip_special_tokens=True)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 2966, in batch_decode
    for seq in sequences
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 2966, in <listcomp>
    for seq in sequences
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 3002, in decode
    **kwargs,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/tokenization_utils.py", line 732, in _decode
    filtered_tokens = self.convert_ids_to_tokens(token_ids, skip_special_tokens=skip_special_tokens)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/tokenization_utils.py", line 716, in convert_ids_to_tokens
    tokens.append(self._convert_id_to_token(index))
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/tokenization_t5.py", line 245, in _convert_id_to_token
    token = self.sp_model.IdToPiece(index)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/sentencepiece/__init__.py", line 501, in _batched_func
    return _func(self, arg)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/sentencepiece/__init__.py", line 494, in _func
    raise IndexError('piece id is out of range.')
IndexError: piece id is out of range.
42it [01:31,  2.18s/it]



@rabeehk this seems like a completely different issue. Please open a new issue.