XLM-R has extremely low accuracy after fine-tuning on MNLI
Author: Lollipop
Pull request: no

### System Info

- `transformers` version: 4.25.1
- Platform: Linux-5.4.0-113-generic-x86_64-with-glibc2.17
- Python version: 3.8.15
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.13.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker @younesbelkada 
about `xlm-roberta-large` performance on GLUR/MNLI

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Run below command from official example (run_glue.py for text classification):
```bash
seed=42
epochs=3
lr=2e-5
max_length=128
batch_size=32

# Parameters for AltCLIP
MODEL_NAME_OR_PATH=xlm-roberta-large
device=0

for task_name in mnli
do
    if [ $task_name = "mrpc" ]; then
        epochs=5
    fi

    if [ $task_name = "stsb" ]; then
        metric=spearmanr
    elif  [ $task_name = "qqp" ] || [ $task_name = "mrpc" ]; then
        metric=f1
    else
        metric=accuracy
    fi

    model_name=${MODEL_NAME_OR_PATH##*/}
    output_dir=evaluation/$model_name/glue/$task_name/$seed
    if [ ! -d "$output_dir" ]; then
        mkdir -p $output_dir
    else
        echo "$output_dir does exist"
    fi

    CUDA_VISIBLE_DEVICES=$device python glue.py \
        --model_name_or_path $MODEL_NAME_OR_PATH \
        --task_name $task_name \
        --cache_dir cache/$model_name \
        --overwrite_cache \
        --do_train \
        --overwrite_output_dir \
        --do_eval \
        --do_predict \
        --max_seq_length $max_length \
        --per_device_train_batch_size $batch_size \
        --per_device_eval_batch_size $batch_size \
        --evaluation_strategy steps \
        --learning_rate $lr \
        --num_train_epochs $epochs \
        --save_total_limit 2 \
        --load_best_model_at_end \
        --metric_for_best_model eval_$metric \
        --greater_is_better true \
        --seed $seed \
        --output_dir $output_dir > $output_dir/log.txt 2>&1
done

```
2. The accuracy is quite low:
eval_mnli/acc: 35.44%
eval_mnli-mm/acc:35.22%
























































































### Expected behavior

Higher performance on MNLI. Running script with `bert-base` or `roberta-base` instead yields around 85+ point.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I think this kind of question is more suited to the `forum`, it is a discussion rather than a bug.