[s2s test] examples/seq2seq/test_finetune_trainer.py::TestFinetuneTrainer::test_finetune_trainer_slow fails on GPU
Author: stas00
Pull request: no

This works (cpu / any pytorch):
```
CUDA_VISIBLE_DEVICES="" RUN_SLOW=1 pytest -sv examples/seq2seq/test_finetune_trainer.py::TestFinetuneTrainer::test_finetune_trainer_slow
```
This fails torch-1.5/gpu or 1.6, or nightly:
```
CUDA_VISIBLE_DEVICES="0" RUN_SLOW=1 pytest -sv examples/seq2seq/test_finetune_trainer.py::TestFinetuneTrainer::test_finetune_trainer_slow
```
Same with pytorch-nightly. same with py37 and py38.


Error:
```
{'eval_loss': 5223.2333984375, 'eval_bleu': 0.0, 'eval_gen_len': 1.0, 'epoch': 1.0}                                                                   
{'eval_loss': 5064.154296875, 'eval_bleu': 0.0, 'eval_gen_len': 1.0, 'epoch': 2.0}                                                                    
{'eval_loss': 4966.837890625, 'eval_bleu': 0.0, 'eval_gen_len': 3.8, 'epoch': 3.0}                                                                    
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:03<00:00,  1.55it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  4.69it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:01<00:00,  3.04it/s]FAILED                                                                                                                                                

====================================================================== FAILURES ======================================================================
___________________________________________________ TestFinetuneTrainer.test_finetune_trainer_slow ___________________________________________________

self = <seq2seq.test_finetune_trainer.TestFinetuneTrainer testMethod=test_finetune_trainer_slow>

100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:01<00:00,  2.78it/s]
    @slow
    def test_finetune_trainer_slow(self):
        # There is a missing call to __init__process_group somewhere
        output_dir = self.run_trainer(eval_steps=2, max_len="128", model_name=MARIAN_MODEL, num_train_epochs=3)
    
        # Check metrics
        logs = TrainerState.load_from_json(os.path.join(output_dir, "trainer_state.json")).log_history
        eval_metrics = [log for log in logs if "eval_loss" in log.keys()]
        first_step_stats = eval_metrics[0]
        last_step_stats = eval_metrics[-1]
    
>       assert first_step_stats["eval_bleu"] < last_step_stats["eval_bleu"]  # model learned nothing
E       AssertionError: assert 0.0 < 0.0

examples/seq2seq/test_finetune_trainer.py:36: AssertionError
```
env:
```
- `transformers` version: 3.4.0
- Platform: Linux-4.15.0-118-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.9
- PyTorch version (GPU?): 1.5.0 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

@sshleifer 

Comments:
I would fix this by training on much more data (like 1000 obs) and getting the loss down much further.

I used more iterations - 6 was enough for 1 gpu, 10 for 2, so I went with 10.

This issue will be resolved by https://github.com/huggingface/transformers/pull/7965