run_squad with roberta
Author: erenup
Pull request: yes

Hi, @julien-c  @thomwolf  this PR is based on #1386 and #1984. 

- This PR modified run_squad.py and models_roberta to support Roberta. 

- This PR also made use of multiple processing to accelerate converting examples to features. (Converting examples to feature needed **15minus before and 34 seconds now** with 24 cpu cores' acceleration. The threads number is 1 by default which is the same as the original single processing's speed).

- The result of Roberta large on squad1.1:  
`{'exact': 87.26584673604542, 'f1': 93.77663586186483, 'total': 10570, 'HasAns_exact': 87.26584673604542, 'HasAns_f1': 93.77663586186483, 'HasAns_total': 10570, 'best_exact': 87.26584673604542, 'best_exact_thresh': 0.0, 'best_f1': 93.77663586186483, 'best_f1_thresh': 0.0}`, which is sighltly lower than #1386 in a single run. 
Parameters are `python ./examples/run_squad.py
    --model_type roberta
    --model_name_or_path roberta-large
    --do_train
    --do_eval
    --do_lower_case \
    --train_file data/squad1/train-v1.1.json
    --predict_file data/squad1/dev-v1.1.json
    --learning_rate 1.5e-5
    --num_train_epochs 2
    --max_seq_length 384
    --doc_stride 128
    --output_dir ./models_roberta/large_squad1
    --per_gpu_eval_batch_size=3 
    --per_gpu_train_batch_size=3 
    --save_steps 10000 --warmup_steps=500 --weight_decay=0.01`. Hope this gap can be improved by `add_prefix_space=true' . I will do this comparasion in the next days. 

- The result of Roberta base is '{'exact': 80.65279091769158, 'f1': 88.57296806525736, 'total': 10570, 'HasAns_exact': 80.65279091769158, 'HasAns_f1': 88.57296806525736, 'HasAns_total': 10570, 'best_exact': 80.65279091769158, 'best_exact_thresh': 0.0, 'best_f1': 88.57296806525736, 'best_f1_thresh': 0.0}'. Roberta-base was also tested since it's more easy to be reproduced.

- The results of bert-base-uncased is `{'exact': 79.21475875118259, 'f1': 87.13734938098504, 'total': 10570, 'HasAns_exact': 79.21475875118259, 'HasAns_f1': 87.13734938098504, 'HasAns_total': 10570, 'best_exact': 79.21475875118259, 'best_exact_thresh': 0.0, 'best_f1': 87.13734938098504, 'best_f1_thresh': 0.0}'. This is tested for the multiple processing's influence on other models. This result is the same with bert-base-uncased result without multiple processing. 

- Hope that someone else can help to reproduce my results. thank you! I will continue to find if three is some ways to improve the roberta-large's results.
- Squad1 model on google drive [roberta-large-finetuned-squad](https://drive.google.com/drive/folders/1BZJeOeri_cKGUG_cRI5OCmkWC5deQqcc?usp=sharing): 


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2173?src=pr&el=h1) Report
> Merging [#2173](https://codecov.io/gh/huggingface/transformers/pull/2173?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/7bd11dda6f43656cf0a3891b7f61a67196d233b4?src=pr&el=desc) will **decrease** coverage by `1.35%`.
> The diff coverage is `9.09%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2173/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2173?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #2173      +/-   ##
==========================================
- Coverage   80.79%   79.43%   -1.36%     
==========================================
  Files         113      113              
  Lines       17013    17067      +54     
==========================================
- Hits        13745    13558     -187     
- Misses       3268     3509     +241
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2173?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [transformers/data/metrics/squad\_metrics.py](https://codecov.io/gh/huggingface/transformers/pull/2173/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL2RhdGEvbWV0cmljcy9zcXVhZF9tZXRyaWNzLnB5) | `0% <0%> (ø)` | :arrow_up: |
| [transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/2173/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX3JvYmVydGEucHk=) | `53.2% <21.21%> (-18.57%)` | :arrow_down: |
| [transformers/data/processors/squad.py](https://codecov.io/gh/huggingface/transformers/pull/2173/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL2RhdGEvcHJvY2Vzc29ycy9zcXVhZC5weQ==) | `14.75% <5.5%> (+0.56%)` | :arrow_up: |
| [transformers/modeling\_tf\_pytorch\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2173/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX3RmX3B5dG9yY2hfdXRpbHMucHk=) | `9.72% <0%> (-85.42%)` | :arrow_down: |
| [transformers/tests/modeling\_tf\_common\_test.py](https://codecov.io/gh/huggingface/transformers/pull/2173/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX3RmX2NvbW1vbl90ZXN0LnB5) | `80.51% <0%> (-16.42%)` | :arrow_down: |
| [transformers/modeling\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/2173/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX3hsbmV0LnB5) | `72.21% <0%> (-2.33%)` | :arrow_down: |
| [transformers/modeling\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/2173/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX2N0cmwucHk=) | `94.27% <0%> (-2.21%)` | :arrow_down: |
| [transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/2173/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX29wZW5haS5weQ==) | `80.13% <0%> (-1.33%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2173?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2173?src=pr&el=footer). Last update [7bd11dd...805c21a](https://codecov.io/gh/huggingface/transformers/pull/2173?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Really nice job!
Here are my results of RoBERTa-large on SQuAD using this PR:
`Results: {'exact': 84.52792049187232, 'f1': 88.0216698977779, 'total': 11873, 'HasAns_exact': 80.90418353576248, 'HasAns_f1': 87.9017015344667, 'HasAns_total': 5928, 'NoAns_exact': 88.1412952060555, 'NoAns_f1': 88.1412952060555, 'NoAns_total': 5945, 'best_exact': 84.52792049187232, 'best_exact_thresh': 0.0, 'best_f1': 88.02166989777776, 'best_f1_thresh': 0.0}`
The hyper-parameters are as follows:
`python ./examples/run_squad.py \
    --model_type roberta \
    --model_name_or_path roberta-large \
    --do_train \
    --do_eval \
    --do_lower_case \
    --train_file data/squad2/train-v2.0.json \
    --predict_file data/squad2/dev-v2.0.json \
    --learning_rate 2e-5 \
    --num_train_epochs 2 \
    --max_seq_length 384 \
    --doc_stride 128 \
    --output_dir ./models_roberta/large_squad2 \
    --per_gpu_eval_batch_size=6   \
    --per_gpu_train_batch_size=6   \
    --save_steps 10000 --warmup_steps=500 --weight_decay=0.01 --overwrite_cache --overwrite_output_dir --threads 24 --version_2_with_negative`


Really nice, thanks a lot @erenup