Fix tokenization in SQuAD for RoBERTa, Longformer, BART
Author: tholor
Pull request: yes

Originating from this discussion: https://github.com/huggingface/transformers/pull/4615#issuecomment-697725357

**Issue:** 
Tokenization of context in `squad_convert_example_to_features()` for RoBERTA-like tokenizers is not preserving whitespace, because we call the tokenizer on previously splitted, individual words.

**Example:**
Q =  Who was Jim Henson?
Context = Jim Henson was a nice puppet
Expected Tokens: ['< s>', 'who', 'Ġwas', 'Ġj', 'im', 'Ġhen', 'son', '?', '
', '
', 'Ġj', 'im', 'Ġhen', 'son', 'Ġwas', 'Ġa', 'Ġnice', 'Ġpuppet', '
']
Actual Tokens:         ['< s>', 'who', 'Ġwas', 'Ġj', 'im', 'Ġhen', 'son', '?', '
', '
', 'j', 'im', 'hen', 'son', 'was', 'a', 'nice', 'p', 'uppet', '
']
Decoded string: Who was Jim Henson?JimHensonwasanicepuppet

**Why a problem?**
- Inconsistency: The question gets tokenized incl. whitespace while the context doesn't. If we have the same word in question and context, we will encode them to different ids.
- Model performance: Eval metrics of `deepset/roberta-base-squad2` on SQuAD 2 dev are significantly lower than originally (F1: 69.6 vs. 81.7). After this fix, it's back to normal (F1: 81.91).  

Evalated via: 
```
run_squad.py  \
    --model_type roberta   \
    --model_name_or_path deepset/roberta-base-squad2  \
    --output_dir results/deepset-roberta-base-squad2 \
    --data_dir .   \
    --predict_file dev-v2.0.json \
    --do_eval \
    --version_2_with_negative \
    --per_gpu_eval_batch_size 16   \
    --max_seq_length 384   \
    --doc_stride 128 \
    --seed 42 \
    --threads 12 \
```

**Fix:** 
Enable `add_prefix_space` for RoBERTa-like tokenizers

**Limitations:** 
- not the most elegant solution
- not sure if there are more tokenizers with similar behavior that we should add

**Related to:** 
https://github.com/huggingface/transformers/issues/7249

@patrickvonplaten @patil-suraj 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7387?src=pr&el=h1) Report
> Merging [#7387](https://codecov.io/gh/huggingface/transformers/pull/7387?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/2dd652d757132d97e43173fb048849685ecccb68?el=desc) will **increase** coverage by `2.39%`.
> The diff coverage is `50.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/7387/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/7387?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #7387      +/-   ##
==========================================
+ Coverage   76.92%   79.32%   +2.39%     
==========================================
  Files         181      181              
  Lines       35721    35726       +5     
==========================================
+ Hits        27480    28339     +859     
+ Misses       8241     7387     -854     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/7387?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/data/processors/squad.py](https://codecov.io/gh/huggingface/transformers/pull/7387/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL3Byb2Nlc3NvcnMvc3F1YWQucHk=) | `28.61% <50.00%> (+0.47%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/7387/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl94bG0ucHk=) | `58.52% <0.00%> (-34.74%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/7387/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9ncHQyLnB5) | `65.38% <0.00%> (-29.59%)` | :arrow_down: |
| [src/transformers/modeling\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/7387/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ194bG0ucHk=) | `78.81% <0.00%> (-12.50%)` | :arrow_down: |
| [src/transformers/tokenization\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/7387/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fcm9iZXJ0YS5weQ==) | `83.11% <0.00%> (-10.39%)` | :arrow_down: |
| [src/transformers/tokenization\_utils\_base.py](https://codecov.io/gh/huggingface/transformers/pull/7387/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHNfYmFzZS5weQ==) | `86.33% <0.00%> (-7.31%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/7387/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `79.94% <0.00%> (-6.27%)` | :arrow_down: |
| [src/transformers/tokenization\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/7387/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fb3BlbmFpLnB5) | `82.57% <0.00%> (-1.52%)` | :arrow_down: |
| [src/transformers/tokenization\_utils\_fast.py](https://codecov.io/gh/huggingface/transformers/pull/7387/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHNfZmFzdC5weQ==) | `90.47% <0.00%> (-1.37%)` | :arrow_down: |
| [src/transformers/tokenization\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/7387/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fYmVydC5weQ==) | `91.07% <0.00%> (-0.45%)` | :arrow_down: |
| ... and [6 more](https://codecov.io/gh/huggingface/transformers/pull/7387/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/7387?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7387?src=pr&el=footer). Last update [2dd652d...a3b11d3](https://codecov.io/gh/huggingface/transformers/pull/7387?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Also pinging @mfuntowicz here

@mfuntowicz @sgugger Is there anything else you want to tackle before merging?