CTRL - RuntimeError: Expected object of backend CUDA but got backend CPU for argument #3 'index'
Author: GuyTevet
Pull request: no

## 🐛 Bug

Hi, 
The error 
`RuntimeError: Expected object of backend CUDA but got backend CPU for argument #3 'index'`

arise while running CTRL using examples/run_generation.py

Model I am using (Bert, XLNet....): **CTRL**

Language I am using the model on (English, Chinese....): **English**

The problem arise when using:
running CTRL using run_generation.py

`python examples/run_generation.py  --model_type ctrl --model_name ctrl --temperature 0.2 --repetition 1.2`

Full trace:

```
Traceback (most recent call last):
  File "examples/run_generation.py", line 236, in <module>
    main()
  File "examples/run_generation.py", line 222, in main
    repetition_penalty=args.repetition_penalty,
  File "/media/disk1/guytevet/venvs/py3/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 43, in decorate_no_grad
    return func(*args, **kwargs)
  File "/media/disk1/guytevet/transformers/src/transformers/modeling_utils.py", line 744, in generate
    effective_batch_size,
  File "/media/disk1/guytevet/transformers/src/transformers/modeling_utils.py", line 775, in _generate_no_beam_search
    outputs = self(**model_inputs)
  File "/media/disk1/guytevet/venvs/py3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/media/disk1/guytevet/transformers/src/transformers/modeling_ctrl.py", line 520, in forward
    inputs_embeds=inputs_embeds,
  File "/media/disk1/guytevet/venvs/py3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/media/disk1/guytevet/transformers/src/transformers/modeling_ctrl.py", line 388, in forward
    inputs_embeds = self.w(input_ids)
  File "/media/disk1/guytevet/venvs/py3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/media/disk1/guytevet/venvs/py3/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 118, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "/media/disk1/guytevet/venvs/py3/lib/python3.6/site-packages/torch/nn/functional.py", line 1454, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected object of backend CUDA but got backend CPU for argument #3 'index'

```
## Environment

* OS: Ubuntu 18.04.2
* Python version: 3.6
* PyTorch version: 1.0.1.post2
* PyTorch Transformers version (or branch): master, installed from source
-e git+https://github.com/huggingface/transformers.git@f75bf05ce6a05ef316363de129c29f2e00cacd7b#egg=transformers
* Using GPU ? Yes
* Distributed of parallel setup ? 



Comments:
Upgrading torch to 1.3.1 solves the issue

I have the same problem even with torch==1.3.1

I think this should be re-opened

I have the same issue for generating with gpt2 

Here is the error log:
```
  File "run_generation.py", line 236, in <module>
    main()
  File "run_generation.py", line 222, in main
    repetition_penalty=args.repetition_penalty,
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 49, in decorate_no_grad
    return func(*args, **kwargs)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/transformers/modeling_utils.py", line 744, in generate
    effective_batch_size,
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/transformers/modeling_utils.py", line 775, in _generate_no_beam_search
    outputs = self(**model_inputs)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/transformers/modeling_gpt2.py", line 589, in forward
    inputs_embeds=inputs_embeds,
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/transformers/modeling_gpt2.py", line 456, in forward
    inputs_embeds = self.wte(input_ids)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 114, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/torch/nn/functional.py", line 1484, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected object of device type cuda but got device type cpu for argument #3 'index' in call to _th_index_select
```

I have torch 1.3.0 installed.

@ehsan-soe check my last PR #2377, solves the issue. 



@alberduris Thanks 👍 

This seems to be an issue with transformers 2.3.0, as I was able to run the generation code successfully by checkout tag v2.2.2

add device and assign the model to it
```
...
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
...
```
assign also the tensor to the device
```
...
sentence = 'Today, scientists confirmed the worst possible outcome: the massive asteroid will collide with Earth'
context_tokens = tokenizer.encode(sentence, add_special_tokens=False)
context = torch.tensor(context_tokens, dtype=torch.long)
context = context.to(device)
...
```