Tokenization differs for different intepreter instances
Author: pglock
Pull request: no

## 🐛 Bug

<!-- Important information -->
Tokenization of `" "` changes for each python interpreter instance.

## To Reproduce

```python
from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
for i in range(5):
    print(tokenizer.encode(" "))
```

## Environment

* Python version: 3.7.2
* PyTorch version: 1.3.1
* PyTorch Transformers version (or branch): 2.2.0

Comments:
By using **Python 3.6.9**, the results is the following:

```
> from transformers import BertTokenizer
> tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
> for i in range(5):
    print(tokenizer.encode(" "))
>>> [101, 100, 102]
[101, 100, 102]
[101, 100, 102]
[101, 100, 102]
[101, 100, 102]
```

I don't understand your question very much. Are you saying that with Python v.X the output of the code you've posted is different from that with Python v.Y?

> ##  Bug
> Tokenization of `" "` changes for each python interpreter instance.
> 
> ## To Reproduce
> ```python
> from transformers import BertTokenizer
> 
> tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
> for i in range(5):
>     print(tokenizer.encode(" "))
> ```
> 
> ## Environment
> * Python version: 3.7.2
> * PyTorch version: 1.3.1
> * PyTorch Transformers version (or branch): 2.2.0

No, if I execute this code in a script, the output differs each time.
For example:
```
# first run
[101, 0, 102]
[101, 0, 102]
[101, 0, 102]
[101, 0, 102]
[101, 0, 102]

# second run
[101, 100, 102]
[101, 100, 102]
[101, 100, 102]
[101, 100, 102]
[101, 100, 102]
```

I've noticed this "bug" right now.
Sometimes, the whitespace character " " is encoded with token with ID=100, other times with token with ID= 103. After looking "into" the `tokenizer.vocab` variable, I've seen that:
- token with ID = 0 is **'[PAD]'**
- token with ID = 100 is **'[UNK]'**
- token with ID = 103 is **'[MASK]'**

**1st run:**
```
> from transformers import BertTokenizer
> tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
> for i in range(5):
...     print(tokenizer.encode(" "))
>>> [101, 100, 102]
[101, 100, 102]
[101, 100, 102]
[101, 100, 102]
[101, 100, 102]
```

**2nd run:**
```
> from transformers import BertTokenizer
> tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
> for i in range(5):
...     print(tokenizer.encode(" "))
>>> [101, 103, 102]
[101, 103, 102]
[101, 103, 102]
[101, 103, 102]
[101, 103, 102]
```

> No, if I execute this code in a script, the output differs each time.
> For example:
> 
> ```
> # first run
> [101, 0, 102]
> [101, 0, 102]
> [101, 0, 102]
> [101, 0, 102]
> [101, 0, 102]
> 
> # second run
> [101, 100, 102]
> [101, 100, 102]
> [101, 100, 102]
> [101, 100, 102]
> [101, 100, 102]
> ```

Feel like this is a non-deterministic behavior arising from encoding an empty sentence.

Do you have a real-world use case for encoding empty sentence?

I guess we could catch this case before using the model, but a deterministic behaviour would still be neat. 

If `" "` is the same as `""`, the tokenizer should just return an empty list right?


fixed in #2081