Error when training with distributed training on 4/8 Nvidia v100.
Author: timsoraro
Pull request: no

# 🐛 Bug

## Information

I'm getting the following error while training the official implementation in [examples/run_language_modeling.py](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py) on [WikiText-2 dataset](https://blog.einstein.ai/the-wikitext-long-term-dependency-language-modeling-dataset/) using multi-gpu setting (4/8 Nvidia GPU's).

```python
Traceback (most recent call last):
  File "run_language_modeling.py", line 976, in <module>
    main()
  File "run_language_modeling.py", line 926, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer)
  File "run_language_modeling.py", line 513, in train
    outputs = model(inputs, masked_lm_labels=labels) if args.mlm else model(inputs, labels=labels)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/distributed.py", line 457, in forward
    self.reducer.prepare_for_backward(list(_find_tensors(output)))
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by (1) passing the keyword argument `find_unused_parameters=True` to `torch.nn.parallel.DistributedDataParallel`; (2) making sure all `forward` function outputs participate in calculating loss. If you already have done the above two steps, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's `forward` function. Please include the loss function and the structure of the return value of `forward` of your module when reporting this issue (e.g. list, dict, iterable). (prepare_for_backward at /pytorch/torch/csrc/distributed/c10d/reducer.cpp:518)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x33 (0x7f66375cb273 in /usr/local/lib/python3.6/dist-packages/torch/lib/libc10.so)
frame #1: c10d::Reducer::prepare_for_backward(std::vector<torch::autograd::Variable, std::allocator<torch::autograd::Variable> > const&) + 0x734 (0x7f66822b09e4 in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #2: <unknown function> + 0x691a4c (0x7f668229fa4c in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #3: <unknown function> + 0x1d3ef4 (0x7f6681de1ef4 in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #4: _PyCFunction_FastCallDict + 0x35c (0x5674fc in /usr/bin/python)
frame #5: /usr/bin/python() [0x50abb3]
frame #6: _PyEval_EvalFrameDefault + 0x449 (0x50c5b9 in /usr/bin/python)
frame #7: /usr/bin/python() [0x508245]
```
Model I am using (Bert, XLNet ...): GPT-2

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: [examples/run_language_modeling.py](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: [WikiText-2 dataset](https://blog.einstein.ai/the-wikitext-long-term-dependency-language-modeling-dataset/)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run [examples/run_language_modeling.py](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py) Using this command:
```
python -m torch.distributed.launch --nproc_per_node 4 run_language_modeling.py --output_dir=./output/ --model_type=gpt2 --model_name_or_path=gpt2 --do_train --train_data_file=./data/wiki.train.raw --per_gpu_train_batch_size 2 --num_train_epochs 10 --fp16
```

## Expected behavior

Should run distributed training without any errors.

## Environment info

I'm using this docker:
```
docker pull deepspeed/deepspeed:latest
```
     
- `transformers` version: 2.6.0
- Platform: Linux
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?: Yes


Comments:
Out of curiosity, does the same issue arise without fp16?

@BramVanroy Yes.

Someone's help, please?

Have look here: https://github.com/pytorch/pytorch/issues/22436

@BramVanroy I'm sorry, the repo is working fine with distributed training. I found the error comes from adding special tokens:
```python
SPECIAL_TOKENS_DICT = {'additional_special_tokens': ['token1', 'token2']}
tokenizer.add_special_tokens(SPECIAL_TOKENS_DICT)
```
It's weird because I didn't get any error with only 1 GPU.
I solved it by doing:
```python
model.resize_token_embeddings(len(tokenizer))
```
@BramVanroy Can you please confirm it's the right way?

Yes, after modifying the vocabulary of the tokenizer, you also need to propagate those changes to the model's embeddings.

If your problem is fixed, please close this topic.

Great, thanks!