Training GPT2 does not use GPU
Author: jnehring
Pull request: no

Im using [run_clm.py](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_clm.py) to train GPT2. When using the [example from the documetnation](https://github.com/huggingface/transformers/tree/master/examples/language-modeling) it works fine and uses the GPU. But when I start it on my custom dataset it does not use any GPUs. Can you give me a tip on how to get it use the GPUs, or what might be wrong?

this works and uses GPUs:

```
python run_clm.py \
    --model_name_or_path gpt2 \
    --dataset_name wikitext \
    --dataset_config_name wikitext-2-raw-v1 \
    --do_train \
    --do_eval \
    --output_dir /netscratch/nehring/projects/opensubtitles/datadir/tmp \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2
```


this starts training but it does not use GPUs:

```
python run_clm.py  \
    --model_type gpt2 \
    --tokenizer_name gpt2 \
    --train_file $DATA_PATH/train.txt \
    --validation_file $DATA_PATH/valid.txt \
    --do_train \
    --do_eval \
    --output_dir /netscratch/nehring/projects/opensubtitles/datadir/models/gpt2-small \
    --per_device_train_batch_size 32 \
    --per_device_eval_batch_size 32 \
    --num_train_epochs 10
```

This is my environement as created by `transformers-cli env`. It says that I did not install tensorflow. But when I do `python -c 'import tensorflow as tf; print(tf.__version__)` then the commandline prints "1.15.0". 

```
- `transformers` version: 4.5.0.dev0
- Platform: Linux-5.4.0-65-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.8.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: this is the problem
- Using distributed or parallel set-up in script?: no
```

This here is part of the output of run_clm.py. it says `_n_gpu=6`. So the GPUs are detected but for some reason they are not used.

```
03/23/2021 18:34:14 - INFO - __main__ -   Training/evaluation parameters TrainingArguments(output_dir=/netscratch/nehring/projects/opensubtitles/datadir/models/gpt2-small, overwrite_output_dir=False, do_train=True, do_eval=True, do_predict=False, evaluation_strategy=IntervalStrategy.NO, prediction_loss_only=False, per_device_train_batch_size=32, per_device_eval_batch_size=32, gradient_accumulation_steps=1, eval_accumulation_steps=None, learning_rate=5e-05, weight_decay=0.0, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=10.0, max_steps=-1, lr_scheduler_type=SchedulerType.LINEAR, warmup_ratio=0.0, warmup_steps=0, logging_dir=runs/Mar23_18-34-14_graz, logging_strategy=IntervalStrategy.STEPS, logging_first_step=False, logging_steps=500, save_strategy=IntervalStrategy.STEPS, save_steps=500, save_total_limit=None, no_cuda=False, seed=42, fp16=False, fp16_opt_level=O1, fp16_backend=auto, fp16_full_eval=False, local_rank=-1, tpu_num_cores=None, tpu_metrics_debug=False, debug=False, dataloader_drop_last=False, eval_steps=500, dataloader_num_workers=0, past_index=-1, run_name=/netscratch/nehring/projects/opensubtitles/datadir/models/gpt2-small, disable_tqdm=False, remove_unused_columns=True, label_names=None, load_best_model_at_end=False, metric_for_best_model=None, greater_is_better=None, ignore_data_skip=False, sharded_ddp=[], deepspeed=None, label_smoothing_factor=0.0, adafactor=False, group_by_length=False, report_to=['tensorboard'], ddp_find_unused_parameters=None, dataloader_pin_memory=True, skip_memory_metrics=False, _n_gpu=6)
```

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.