Training DistilBert - RuntimeError: index out of range at /pytorch/aten/src/TH/generic/THTensorEvenMoreMath.cpp:237
Author: alexandrabenamar
Pull request: no

## 🐛 Bug

Hello,

<!-- Important information -->

Model I am using (Bert, XLNet....): DistilBert

Language I am using the model on (English, Chinese....): French

The problem arise when using:
* [ ] the official example scripts: examples/distillation/train.py

The tasks I am working on is:
* [ ] the official training DistilBert from scratch task 

## To Reproduce

I followed the required steps to train distil* from scratch : 

```bash
python ./scripts/binarized_data.py \
        --file_path ./data/dataset.txt \
        --tokenizer_type bert \
        --tokenizer_name bert-base-multilingual-cased \
        --dump_file ./data_output/binarized_text &
```

The only modification I made was to increase the vocab_size, otherwise I had a bug:
```bash
python ./scripts/token_counts.py \
    --data_file ./data/binarized_text.bert-base-multilingual-cased.pickle \
    --token_counts_dump ./data/token_counts.bert-base-multilingual-cased.pickle \
    --vocab_size 65536
```

Then, I launched the training with the following : 
```bash
python train.py \
    --student_type distilbert \
    --student_config ./training_configs/distilbert-base-uncased.json \
    --teacher_type bert \
    --teacher_name bert-base-multilingual-cased \
    --alpha_ce 5.0 --alpha_mlm 2.0 --alpha_cos 1.0 --mlm \
    --dump_path ./serialization_dir/my_first_training \
    --data_file ./data/binarized_text.bert-base-multilingual-cased.pickle \
    --token_counts ./data/token_counts.bert-base-multilingual-cased.pickle \
    --force
```

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

Error message : 
```bash
-Iter:   0% 0/586181 [00:00<?, ?it/s]Traceback (most recent call last):
  File "train.py", line 289, in <module>
    main()
  File "train.py", line 284, in main
    distiller.train()
  File "/dds/work/distil/transformers/examples/distillation/distiller.py", line 339, in train
    self.step(input_ids=token_ids, attention_mask=attn_mask, lm_labels=lm_labels)
  File "/dds/work/distil/transformers/examples/distillation/distiller.py", line 369, in step
    s_logits, s_hidden_states = self.student(input_ids=input_ids, attention_mask=attention_mask)     # (bs, seq_length, voc_size)
  File "/dds/miniconda/envs/dds/lib/python3.5/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/dds/miniconda/envs/dds/lib/python3.5/site-packages/transformers/modeling_distilbert.py", line 528, in forward
    head_mask=head_mask)
  File "/dds/miniconda/envs/dds/lib/python3.5/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/dds/miniconda/envs/dds/lib/python3.5/site-packages/transformers/modeling_distilbert.py", line 461, in forward
    embedding_output = self.embeddings(input_ids)   # (bs, seq_length, dim)
  File "/dds/miniconda/envs/dds/lib/python3.5/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/dds/miniconda/envs/dds/lib/python3.5/site-packages/transformers/modeling_distilbert.py", line 92, in forward
    word_embeddings = self.word_embeddings(input_ids)                   # (bs, max_seq_length, dim)
  File "/dds/miniconda/envs/dds/lib/python3.5/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/dds/miniconda/envs/dds/lib/python3.5/site-packages/torch/nn/modules/sparse.py", line 114, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "/dds/miniconda/envs/dds/lib/python3.5/site-packages/torch/nn/functional.py", line 1467, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: index out of range: Tried to access index 61578 out of table with 30521 rows. at /pytorch/aten/src/TH/generic/THTensorEvenMoreMath.cpp:237
-Iter:   0% 0/586181 [00:00<?, ?it/s]
```

## Environment

* OS: Debian
* Python version: 3.5
* PyTorch version: 1.2.0
* PyTorch Transformers version (or branch): 1.2.0
* Using GPU ? No
* Distributed of parallel setup ?
* Any other relevant information:

Thanks in advance !


Comments:
I re-downloaded PyTorch 1.2.0 and the problem was fixed for some reason...