RuntimeError:  Invalid index in gather at ../aten/src/TH/generic/THTensorEvenMoreMath.cpp:469  (GPT2DoubleHeadsModel)
Author: tonyhqanguyen
Pull request: no

## 🐛 Bug


Model I am using (Bert, XLNet....): GPT2DoubleHeadsModel

Language I am using the model on (English, Chinese....): English

The problem arise when using:
* [x] the official example scripts: Trying out documentation
* [ ] my own modified scripts: 

## To Reproduce

Steps to reproduce the behavior:
```python
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2DoubleHeadsModel.from_pretrained('gpt2')
choices = ["Hello, my dog is cute [CLS]", "Hello, my cat is cute [CLS]"]  # Assume you've added [CLS] to the vocabulary
input_ids = torch.tensor([tokenizer.encode(s) for s in choices]).unsqueeze(0)  # Batch size 1, 2 choices
mc_token_ids = torch.tensor([-1, -1]).unsqueeze(0)  # Batch size 1
outputs = model(input_ids, mc_token_ids)
lm_prediction_scores, mc_prediction_scores = outputs[:2]
```
This is from the documentation of [GPT2DoubleHeadsModel](https://github.com/huggingface/pytorch-transformers/blob/f2b300df6bd46ad16580f0313bc4b30ddde8515d/pytorch_transformers/modeling_gpt2.py#L617)

The error:
> Traceback (most recent call last):
  File "<input>", line 6, in <module>
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytorch_transformers/modeling_gpt2.py", line 718, in forward
    mc_logits = self.multiple_choice_head(hidden_states, mc_token_ids).squeeze(-1)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytorch_transformers/modeling_utils.py", line 774, in forward
    output = hidden_states.gather(-2, token_ids).squeeze(-2) # shape (bsz, XX, hidden_size)
RuntimeError: Invalid index in gather at ../aten/src/TH/generic/THTensorEvenMoreMath.cpp:469
Using cls_token, but it is not set yet.
Using mask_token, but it is not set yet.
Using pad_token, but it is not set yet.
Using sep_token, but it is not set yet.
Using unk_token, but it is not set yet.
## Environment

* OS: MacOS Mojave 10.14.4
* Python version: 3.7
* PyTorch version: 1.2.0
* PyTorch Transformers version (or branch): latest
* Using GPU ? No
* Distributed of parallel setup ? No
* Any other relevant information:


Comments:
Indeed, there seems to be a problem here. I'll look into it.

Thanks for the report, the error was in the docstring, we cannot use `-1` as the index for the last token, it has to be the positive index of the CLS token (in the case of the example `9`.

The fix seems to have led to other issues. I'm getting the error:  

----> 1 outputs = model(input_ids, mc_token_ids)
      2 lm_prediction_scores, mc_prediction_scores = outputs[:2]

/opt/anaconda/anaconda3/envs/huggingface_env/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/opt/anaconda/anaconda3/envs/huggingface_env/lib/python3.7/site-packages/pytorch_transformers/modeling_gpt2.py in forward(self, input_ids, mc_token_ids, lm_labels, mc_labels, token_type_ids, position_ids, past, head_mask)
    710                 position_ids=None, past=None, head_mask=None):
    711         transformer_outputs = self.transformer(input_ids, position_ids=position_ids, token_type_ids=token_type_ids,
--> 712                                                past=past, head_mask=head_mask)
    713         hidden_states = transformer_outputs[0]
    714 

/opt/anaconda/anaconda3/envs/huggingface_env/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/opt/anaconda/anaconda3/envs/huggingface_env/lib/python3.7/site-packages/pytorch_transformers/modeling_gpt2.py in forward(self, input_ids, position_ids, token_type_ids, past, head_mask)
    493         position_ids = position_ids.view(-1, position_ids.size(-1))
    494 
--> 495         inputs_embeds = self.wte(input_ids)
    496         position_embeds = self.wpe(position_ids)
    497         if token_type_ids is not None:

/opt/anaconda/anaconda3/envs/huggingface_env/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/opt/anaconda/anaconda3/envs/huggingface_env/lib/python3.7/site-packages/torch/nn/modules/sparse.py in forward(self, input)
    112         return F.embedding(
    113             input, self.weight, self.padding_idx, self.max_norm,
--> 114             self.norm_type, self.scale_grad_by_freq, self.sparse)
    115 
    116     def extra_repr(self):

/opt/anaconda/anaconda3/envs/huggingface_env/lib/python3.7/site-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1465         # remove once script supports set_grad_enabled
   1466         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 1467     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1468 
   1469 

RuntimeError: index out of range: Tried to access index 50257 out of table with 50256 rows. at /pytorch/aten/src/TH/generic/THTensorEvenMoreMath.cpp:237

What exact series of command did you use to get this error (maybe open a new issue with more details)