issue in pretraining language model with checkpoint 
Author: 008karan
Pull request: no
Labels: name: wontfix

# 🐛 Bug

## Information

I am pre-training albert from scratch and it was going fine.(8 v100)
But when I am training using a checkpoint its using single gpu and that also 1gb/32gb GPU ram.



## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
 `transformers` version: 2.10.0

launching script with:
```
python transformers/examples/language-modeling/run_language_modeling.py   --train_data_file text.txt  --output_dir albert_model --model_type albert --mlm --config_name test --tokenizer_name test --do_train --line_by_line --learning_rate 5e-5 --num_train_epochs 3 --save_total_limit 50 --save_steps 5000 --per_gpu_train_batch_size 150 --seed 42 --overwrite_output_dir --max_steps 200000 --fp16 --model_name_or_path albert_model/checkpoint-200000
```
It seems that its starting it properly with global step but not using gpu properly which really weird
```
/language_model/lm/lib/python3.6/site-packages/torch/optim/lr_scheduler.py:218: UserWarning: Please also save or load the state of the optimzer when saving or loading the scheduler.
  warnings.warn(SAVE_STATE_WARNING, UserWarning)
Selected optimization level O1:  Insert automatic casts around Pytorch functions and Tensor methods.

Defaults for this optimization level are:
enabled                : True
opt_level              : O1
cast_model_type        : None
patch_torch_functions  : True
keep_batchnorm_fp32    : None
master_weights         : None
loss_scale             : dynamic
Processing user overrides (additional kwargs that are not None)...
After processing overrides, optimization options are:
enabled                : True
opt_level              : O1
cast_model_type        : None
patch_torch_functions  : True
keep_batchnorm_fp32    : None
master_weights         : None
loss_scale             : dynamic
Warning:  multi_tensor_applier fused unscale kernel is unavailable, possibly because apex was installed without --cuda_ext --cpp_ext. Using Python fallback.  Original ImportError was: ModuleNotFoundError("No module named 'amp_C'",)
06/08/2020 11:25:09 - INFO - transformers.trainer -   ***** Running training *****
06/08/2020 11:25:09 - INFO - transformers.trainer -     Num examples = 28236463
06/08/2020 11:25:09 - INFO - transformers.trainer -     Num Epochs = 43
06/08/2020 11:25:09 - INFO - transformers.trainer -     Instantaneous batch size per device = 150
06/08/2020 11:25:09 - INFO - transformers.trainer -     Total train batch size (w. parallel, distributed & accumulation) = 1200
06/08/2020 11:25:09 - INFO - transformers.trainer -     Gradient Accumulation steps = 1
06/08/2020 11:25:09 - INFO - transformers.trainer -     Total optimization steps = 1000000
06/08/2020 11:25:09 - INFO - transformers.trainer -     Continuing training from checkpoint, will skip to saved global_step
06/08/2020 11:25:09 - INFO - transformers.trainer -     Continuing training from epoch 8
06/08/2020 11:25:09 - INFO - transformers.trainer -     Continuing training from global step 200000
06/08/2020 11:25:09 - INFO - transformers.trainer -     Will skip the first 11752 steps in the first epoch
Epoch:   0%|                                                                                                                               | 0/35 [00:00<?, ?it/s]

Iteration:  42%|███████████████████████████████████████████▋                                                             | 9781/23531 [1:20:59<1:53:51,  2.01it/s]
```
can any one suggest something here?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.