BERT model gets fairly random results
Author: dorooddorood606
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.5.0
- Platform: linux
- Python version: 3.8
- PyTorch version (GPU?): 1.8
- Tensorflow version (GPU?): - 
- Using GPU in script?: - 
- Using distributed or parallel set-up in script?: - 

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @Rocketknight1

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

@LysandreJik, @sgugger

## Information

Model I am using BERT-base-uncased model with run_glue.py on MRPC dataset, there are substantial differences in the results each time I run the codes, sometimes it reaches 5% percent. Too much variation makes the results not reliable, and this is quite a big issue. Thanks for your help on this. This might be a bug in the trainer/or the model itself.

first run:
```
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   epoch                        =                3.0
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   eval_average_metrics         = 0.8300182784978398
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   eval_mem_cpu_alloc_delta     =                0MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   eval_mem_cpu_peaked_delta    =                2MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   eval_mem_gpu_alloc_delta     =                0MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   eval_mem_gpu_peaked_delta    =              264MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   mrpc_eval_accuracy           =              0.799
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   mrpc_eval_combined_score     =               0.83
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   mrpc_eval_f1                 =              0.861
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   mrpc_eval_loss               =             0.4643
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   mrpc_eval_runtime            =         0:00:00.38
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:03:22,711 >>   mrpc_eval_samples_per_second =            529.617
```
second run:
```
[INFO|trainer_pt_utils.py:722] 2021-04-26 10:02:59,294 >> ***** test metrics *****
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   epoch                        =                3.0
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   eval_average_metrics         = 0.8090236094437775
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   eval_mem_cpu_alloc_delta     =                0MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   eval_mem_cpu_peaked_delta    =                2MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   eval_mem_gpu_alloc_delta     =                0MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   eval_mem_gpu_peaked_delta    =              264MB
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   mrpc_eval_accuracy           =             0.7745
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   mrpc_eval_combined_score     =              0.809
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   mrpc_eval_f1                 =             0.8435
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,294 >>   mrpc_eval_loss               =             0.4631
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,295 >>   mrpc_eval_runtime            =         0:00:00.35
[INFO|trainer_pt_utils.py:727] 2021-04-26 10:02:59,295 >>   mrpc_eval_samples_per_second =            567.515
```

## To reproduce

Steps to reproduce the behavior:

Please run run_glue.py default script on MRPC.


## Expected behavior

The model needs to reproduce the same results each time it runs. 

Comments:
Please give us the whole command you are running as we can't reproduce without it.
Are you properly setting the seed? Depending on the seed used, the results differ a lot on MRPC, since it's a tiny dataset. This is known and there have been [published papers](https://arxiv.org/pdf/2002.06305.pdf) on this.

Hi,
thank you the issue resolved with moving the codes to version 4.6.dev
thanks