[Bug Fix] Fix run_squad.py evaluation code doesn't use probabilities
Author: elronbandel
Pull request: yes

Modification of run_squad.py fine tuning example so it will use the answer correctness probabilities the models produce while evaluating the model and calculating the best thresholds.

Evaluation was done without the evaluated model probabilities but rather with default zero values. It corrupted the evaluation results and the best thresholds (which evaluated always as 0.0).

**Notice: many squad models were evaluated without the probabilities, therefore, the results published in their model cards are possibly wrong.**

Example: [ahotrod/electra_large_discriminator_squad2_512](https://huggingface.co/ahotrod/electra_large_discriminator_squad2_512)

The results with current evaluation script:
```
"exact": 87.09677419354838,
  "f1": 89.98343832723452,
  "total": 11873,
  "HasAns_exact": 84.66599190283401,
  "HasAns_f1": 90.44759839056285,
  "HasAns_total": 5928,
  "NoAns_exact": 89.52060555088309,
  "NoAns_f1": 89.52060555088309,
  "NoAns_total": 5945,
  "best_exact": 87.09677419354838,
  "best_exact_thresh": 0.0,
  "best_f1": 89.98343832723432,
  "best_f1_thresh": 0.0
```

The results after the fix:
```
'exact': 87.00412701086499,
 'f1': 89.77725380276271,
 'total': 11873,
 'HasAns_exact': 83.80566801619433,
 'HasAns_f1': 89.35987422405582,
 'HasAns_total': 5928,
 'NoAns_exact': 90.19343986543313,
 'NoAns_f1': 90.19343986543313,
 'NoAns_total': 5945,
 'best_exact': 87.34102585698643,
'best_exact_thresh': 0.09882385462915344,
 'best_f1': 90.07804792988485,
'best_f1_thresh': 0.09882385462915344
```







Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7319?src=pr&el=h1) Report
> Merging [#7319](https://codecov.io/gh/huggingface/transformers/pull/7319?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e4b94d8e581e547eaf9e47b76fd1a6497e911905?el=desc) will **decrease** coverage by `2.73%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/7319/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/7319?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #7319      +/-   ##
==========================================
- Coverage   81.59%   78.85%   -2.74%     
==========================================
  Files         174      174              
  Lines       33671    33671              
==========================================
- Hits        27474    26552     -922     
- Misses       6197     7119     +922     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/7319?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_longformer.py](https://codecov.io/gh/huggingface/transformers/pull/7319/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9sb25nZm9ybWVyLnB5) | `16.37% <0.00%> (-82.31%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_funnel.py](https://codecov.io/gh/huggingface/transformers/pull/7319/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9mdW5uZWwucHk=) | `18.53% <0.00%> (-75.51%)` | :arrow_down: |
| [src/transformers/modeling\_longformer.py](https://codecov.io/gh/huggingface/transformers/pull/7319/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19sb25nZm9ybWVyLnB5) | `19.71% <0.00%> (-51.90%)` | :arrow_down: |
| [src/transformers/configuration\_longformer.py](https://codecov.io/gh/huggingface/transformers/pull/7319/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX2xvbmdmb3JtZXIucHk=) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [src/transformers/activations\_tf.py](https://codecov.io/gh/huggingface/transformers/pull/7319/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9hY3RpdmF0aW9uc190Zi5weQ==) | `54.16% <0.00%> (-20.84%)` | :arrow_down: |
| [src/transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/7319/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19yb2JlcnRhLnB5) | `81.17% <0.00%> (-15.39%)` | :arrow_down: |
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/7319/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `72.25% <0.00%> (-10.00%)` | :arrow_down: |
| [src/transformers/generation\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/7319/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3V0aWxzLnB5) | `96.92% <0.00%> (-0.28%)` | :arrow_down: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/7319/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `83.08% <0.00%> (+0.24%)` | :arrow_up: |
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/7319/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `89.88% <0.00%> (+0.38%)` | :arrow_up: |
| ... and [14 more](https://codecov.io/gh/huggingface/transformers/pull/7319/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/7319?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7319?src=pr&el=footer). Last update [e4b94d8...be00f57](https://codecov.io/gh/huggingface/transformers/pull/7319?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Opened Issue: [[BUG] Wrong Scores for many SQUAD models ](https://github.com/huggingface/transformers/issues/8710) #8710

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.