text: 
  
  TITLE
  Poor f1 score when validating existing models
  
  COMMENTS
  1
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  I have been attempting to validate the results on several models, but have been unable to match them with the results posted on the model cards.
  
  For example, I took an Albert model: https://huggingface.co/ahotrod/albert_xxlargev1_squad2_512
  and I run:
  ```
  python run_squad.py 
  --model_type albert 
  --model_name_or_path ahotrod/albert_xxlargev1_squad2_512 
  --do_eval 
  --predict_file ../squd/dev-v2.0.json 
  --per_gpu_eval_batch_size 8  
  --max_seq_length 512 
  --doc_stride 128 
  --output_dir ../squd/output/albert 
  --overwrite_output_dir 
  --threads 16 
  --verbose 
  --version_2_with_negative
  ```
  
  I got: 
  ```
  exact: 77.39408742525058
  f1: 81.6576936707378
  total: 11873
  HasAns_exact: 71.60931174089069
  HasAns_f1': 80.14875117285251
  HasAns_total': 5928
  NoAns_exact': 83.16232127838519
  NoAns_f1': 83.16232127838519
  NoAns_total': 5945
  best_exact': 77.38566495409754
  best_exact_thresh': 0.0
  best_f1': 81.64927119958456
  best_f1_thresh': 0.0
  ```
  
  While on the model card stated:
  ```
  exact: 86.11134506864315
  f1: 89.35371214945009
  total': 11873
  HasAns_exact': 83.56950067476383
  HasAns_f1': 90.06353312254078
  HasAns_total': 5928
  NoAns_exact': 88.64592094196804
  NoAns_f1': 88.64592094196804
  NoAns_total': 5945
  best_exact': 86.11134506864315
  best_exact_thresh': 0.0
  best_f1': 89.35371214944985
  best_f1_thresh': 0.0
  ```
  
  I did not alter any code, I was simply trying to validate the results. What am I overlooking here?
  
  (the same diff also found using roberta models, bert, and others)