text: 
  
  TITLE
  XLM-RoBERTa model for QA seems not properly work
  
  COMMENTS
  3
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  ## Environment info
  <!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
       Don't forget to fill out the missing fields in that output! -->
       
  - `transformers` version: 3.3.1 (and I've also tried installing Transformers from `master`, see details below)
  - Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
  - Python version: 3.6.9
  - PyTorch version (GPU?): 1.6.0+cu101 (False)
  - Tensorflow version (GPU?): 2.3.0 (False)
  - Using GPU in script?: Yes
  - Using distributed or parallel set-up in script?: No
  
  ### Who can help
  albert, bert, GPT2, XLM: @LysandreJik
  
  ## Information
  
  Model I am using (Bert, XLNet ...): 
  [deepset/xlm-roberta-large-squad2](https://huggingface.co/deepset/xlm-roberta-large-squad2) 
  
  The problem arises when using:
  * [x] the official example scripts: [run_squad.py](https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad.py)
  * [ ] my own modified scripts: (give details below)
  
  The tasks I am working on is:
  * [x] an official GLUE/SQUaD task: **SQuAD 2.0 dev set evaluation**
  * [ ] my own task or dataset: (give details below)
  
  ## To reproduce
  
  Steps to reproduce the behavior:
  ```
  ! wget https://raw.githubusercontent.com/rajpurkar/SQuAD-explorer/master/dataset/dev-v2.0.json
  ! python transformers/examples/question-answering/run_squad.py \
    --model_type xlm-roberta \
    --model_name_or_path 'deepset/xlm-roberta-large-squad2' \
    --do_eval \
    --do_lower_case \
    --predict_file 'dev-v2.0.json' \
    --output_dir 'output' \
    --overwrite_output_dir \
    --version_2_with_negative
  ```
  
  ## Expected behavior
  There are some values mismatch between:
  1. values reported in the model card [here](https://huggingface.co/deepset/xlm-roberta-large-squad2#performance)
  2. values obtained when Transformers is installed using `pip install transformers` 
  3. values obtained when Transformers is installed from master
  
  In particular:
  - Reported metrics in the model card: `"exact": 79.45759285774446, "f1": 83.79259828925511`
  - Transformers installed from pip: `'exact': 64.67615598416576, 'f1': 77.27580544355429`
  - Transformers installed from master: `'exact': 60.11959090373114, 'f1': 76.13129575803934`
  