Repository: huggingface/transformers

Issue:
RuntimeError: expected device cpu but got device cuda:0
I am traing a roberta model and running the script examples/run_language_modeling.py
The following error occurs when i am trying to resume training.
Traceback (most recent call last):
  File "examples/run_language_modeling.py", line 284, in <module>
    main()
  File "examples/run_language_modeling.py", line 254, in main
    trainer.train(model_path=model_path)
  File "/home/socian-pc1/anaconda3/envs/XformerEnv/lib/python3.6/site-packages/transformers/trainer.py", line 326, in train
    optimizer.step()
  File "/home/socian-pc1/anaconda3/envs/XformerEnv/lib/python3.6/site-packages/torch/optim/lr_scheduler.py", line 67, in wrapper
    return wrapped(*args, **kwargs)
  File "/home/socian-pc1/anaconda3/envs/XformerEnv/lib/python3.6/site-packages/transformers/optimization.py", line 155, in step
    exp_avg.mul_(beta1).add_(1.0 - beta1, grad)
RuntimeError: expected device cpu but got device cuda:0

My config
python examples/run_language_modeling.py \
    --train_data_file $TRAIN_FILE \
    --eval_data_file $TEST_FILE \
    --output_dir ./MyRobertaOutput \
    --model_name_or_path ./MyRoBERTa/checkpoint-570000 \
    --config_name ../xformer_output \
    --tokenizer_name ../xformer_output \
    --mlm \
    --do_train \
    --do_eval \
    --line_by_line \
    --learning_rate 1e-5 \
    --num_train_epochs 2 \
    --save_total_limit 20 \
    --save_steps 5000 \
    --per_gpu_train_batch_size 6 \
    --warmup_steps=10000 \
    --logging_steps=100 \
    --gradient_accumulation_steps=4 \
    --seed 666 --block_size=512






Discussion:
Try initialize the model in the trainer script from transformers with self.model = model.cuda()
I am getting the same error. Is there work around ? 
What @tebandesade  mentioned didnt work out for.
I faced the same problem with RoBERTa pretraining, however inserting line model = model.cuda() before trainer in run_language_modeling.py file helped me
@tebandesade, Thank you!

Hello! I'm having trouble reproducing that on master. Do you mind installing from source and letting me know if you still have the issue? Thank you
Hi @LysandreJik, installing from source doesn't fix the issue, though @tebandesade's suggestion works fine. 

@octalpixel try editing this line, it shall work;
https://github.com/huggingface/transformers/blob/62427d0815825436fa55b43725f44776e94abb65/src/transformers/trainer.py#L145
I am getting this error too.

I think the issue is optimizers are [setup](https://github.com/huggingface/transformers/blob/3e0f06210646a440509efa718b30d18322d6a830/src/transformers/trainer.py#L242) from `self.model` which is in cpu, but the model is [moved to device](https://github.com/huggingface/transformers/blob/3e0f06210646a440509efa718b30d18322d6a830/src/transformers/trainer.py#L338) afterwards. Which is why `self.model = model.cuda()` fixes the error. 


Resolving patch:
diff --git a/src/transformers/trainer.py b/src/transformers/trainer.py
--- a/src/transformers/trainer.py
+++ b/src/transformers/trainer.py
@@ -188,7 +188,7 @@ def __init__(
             prediction_loss_only:
                 (Optional) in evaluation and prediction, only return the loss
         """
-        self.model = model
+        self.model = model.to(args.device)
         self.args = args
         if data_collator is not None:
             self.data_collator = data_collator
@@ -393,7 +393,6 @@ def train(self, model_path: Optional[str] = None):
             scheduler.load_state_dict(torch.load(os.path.join(model_path, "scheduler.pt")))
 
         model = self.model
-        model.to(self.args.device)
         if self.args.fp16:
             if not is_apex_available():
                 raise ImportError("Please install apex from https://www.github.com/nvidia/apex to use fp16 training.")
@@ -726,7 +725,6 @@ def _prediction_loop(
         prediction_loss_only = prediction_loss_only if prediction_loss_only is not None else self.prediction_loss_only
 
         model = self.model
-        model.to(self.args.device)
         # multi-gpu eval
         if self.args.n_gpu > 1:
             model = torch.nn.DataParallel(model)


Test patch: