Fix byte-level BPE decoding error when using added tokens
Author: thomwolf
Pull request: yes

This PR fixes a mismatch between regular unicode added tokens and byte-level BPE tokens when doing decoding.

Wrong behavior reported in #1133.

Also adds regression tests.

Comments:
Great addition! There was just a small issue regarding the way the special tokens were joined if they were not at the beginning of the sentence. I fixed it with my commit.

Before, the following code:
```py
tok = GPT2Tokenizer.from_pretrained("gpt2")
tok.add_tokens(["there my", "name is"])
print(tok.decode(tok.encode("Hi there my name is Lysandre")))
```
would output:
```
 Hithere myname is Lysandre
```

Now it outputs:
```
 Hi there my name is Lysandre
```

# [Codecov](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174?src=pr&el=h1) Report
> Merging [#1174](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/pytorch-transformers/commit/b6cd856b08e3860e59cc126be86b901ccab4f193?src=pr&el=desc) will **increase** coverage by `0.17%`.
> The diff coverage is `94.73%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #1174      +/-   ##
==========================================
+ Coverage   80.67%   80.84%   +0.17%     
==========================================
  Files          46       46              
  Lines        7859     7874      +15     
==========================================
+ Hits         6340     6366      +26     
+ Misses       1519     1508      -11
```


| [Impacted Files](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...h\_transformers/tests/tokenization\_tests\_commons.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvdGVzdHMvdG9rZW5pemF0aW9uX3Rlc3RzX2NvbW1vbnMucHk=) | `100% <100%> (+2.88%)` | :arrow_up: |
| [pytorch\_transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvdG9rZW5pemF0aW9uX3V0aWxzLnB5) | `80.47% <93.33%> (-0.13%)` | :arrow_down: |
| [pytorch\_transformers/tokenization\_transfo\_xl.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvdG9rZW5pemF0aW9uX3RyYW5zZm9feGwucHk=) | `34.17% <0%> (+0.28%)` | :arrow_up: |
| [pytorch\_transformers/tokenization\_gpt2.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvdG9rZW5pemF0aW9uX2dwdDIucHk=) | `96.69% <0%> (+0.82%)` | :arrow_up: |
| [pytorch\_transformers/file\_utils.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvZmlsZV91dGlscy5weQ==) | `71.83% <0%> (+1.4%)` | :arrow_up: |
| [pytorch\_transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvbW9kZWxpbmdfdXRpbHMucHk=) | `90.02% <0%> (+1.45%)` | :arrow_up: |
| [...orch\_transformers/tests/tokenization\_utils\_test.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvdGVzdHMvdG9rZW5pemF0aW9uX3V0aWxzX3Rlc3QucHk=) | `96% <0%> (+4%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174?src=pr&el=footer). Last update [b6cd856...31d3373](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1174?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Great thanks! Merging now.