Integrate fast tokenizers library inside transformers
Author: mfuntowicz
Pull request: yes

Integrate the BPE-based tokenizers inside transformers.

- [x] Bert (100% match)
- [x] DistilBert (100% match)
- [x] OpenAI GPT (100% match)
- [x] GPT2 (100% match if no trailing \n)
- [x] Roberta (100% match if no trailing \n)
- [x] TransformerXL
- [x] CTRL (No binding will be provided).

Added priority for Tokenizer with fast implementation in `AutoTokenizer` this is done through a new mapping (name: class) -> (name: Tuple[class, class]) which represents both the Python and Rust implementation classes. if no Rust implementation is available, it is simply set to None. AutoTokenizer will try to pick the Rust class if not None, otherwise it defaults to the Python one.

Added some matching tests which basically checks that there is a huge % of element wise matching tokens. This is set arbitrary to 0.05 (5%) _[i.e. at max, 5% of differences between Python and Rust]_.

Added parameter `return_offsets_mapping=False` over encoding methods which will return the offset mapping if using a Rust tokenizer. If using a Python tokenizer, a warning message is displayed through the module logger and the argument is discarded.



Comments:
only took a superficial look, but looks very clean 👍 

Excited to use fast tokenizers by default!

Current CI issues are real and "normal" we need to release the next version of tokenizers lib which will bring all the dependencies.

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2674?src=pr&el=h1) Report
> Merging [#2674](https://codecov.io/gh/huggingface/transformers/pull/2674?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/20fc18fbda3669c2f4a3510e0705b2acd54bff07?src=pr&el=desc) will **increase** coverage by `0.29%`.
> The diff coverage is `83.01%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2674/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2674?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##           master   #2674      +/-   ##
=========================================
+ Coverage      75%   75.3%   +0.29%     
=========================================
  Files          94      94              
  Lines       15288   15424     +136     
=========================================
+ Hits        11467   11615     +148     
+ Misses       3821    3809      -12
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2674?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/2674/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9fX2luaXRfXy5weQ==) | `98.87% <100%> (ø)` | :arrow_up: |
| [src/transformers/tokenization\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/2674/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fcm9iZXJ0YS5weQ==) | `100% <100%> (ø)` | :arrow_up: |
| [src/transformers/tokenization\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/2674/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fYmVydC5weQ==) | `96.92% <100%> (+0.3%)` | :arrow_up: |
| [src/transformers/pipelines.py](https://codecov.io/gh/huggingface/transformers/pull/2674/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9waXBlbGluZXMucHk=) | `70.88% <100%> (+0.14%)` | :arrow_up: |
| [src/transformers/tokenization\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/2674/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fZGlzdGlsYmVydC5weQ==) | `100% <100%> (ø)` | :arrow_up: |
| [src/transformers/tokenization\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/2674/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fZ3B0Mi5weQ==) | `96.85% <100%> (+0.58%)` | :arrow_up: |
| [src/transformers/tokenization\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/2674/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fYXV0by5weQ==) | `97.22% <100%> (+0.25%)` | :arrow_up: |
| [src/transformers/tokenization\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/2674/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdHJhbnNmb194bC5weQ==) | `37.91% <51.42%> (+5.04%)` | :arrow_up: |
| [src/transformers/tokenization\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/2674/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fb3BlbmFpLnB5) | `82.27% <81.57%> (+0.46%)` | :arrow_up: |
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2674/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `90.08% <87.23%> (+3.98%)` | :arrow_up: |
| ... and [30 more](https://codecov.io/gh/huggingface/transformers/pull/2674/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2674?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2674?src=pr&el=footer). Last update [20fc18f...56748e8](https://codecov.io/gh/huggingface/transformers/pull/2674?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).