Check in PreTrainedTokenizer can cause incorrect tokenization
Author: codedecde
Pull request: no

## Environment info


- `transformers` version: 4.5.1
- Platform: Linux-5.4.0-1047-azure-x86_64-with-glibc2.10
- Python version: 3.8.8
- PyTorch version (GPU?): 1.9.0a0+2ecb2c7 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: NA
- Using distributed or parallel set-up in script?: NA

### Who can help
@LysandreJik


## Information
[This check](https://github.com/huggingface/transformers/blob/7fcee113c163a95d1b125ef35dc49a0a1aa13a50/src/transformers/tokenization_utils.py#L336) in `PreTrainedTokenizer` can cause incorrect tokenization (and subsequent encoding) for space only sequences (or sequences with leading and trailing spaces).  This can be problematic for byte only models (byT5 etc.), can  cause inconsistent tokenizations between `Tokenzer` and `TokenizerFast` classes and can cause issues wherever the code assumes non-destructive behaviour of a tokenizer.

## To reproduce

Steps to reproduce the behavior:
```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("roberta-base", use_fast=False)
tokenizer_fast = AutoTokenizer.from_pretrained("roberta-base")
# Correct Tokenization
out = tokenizer_fast.tokenize(' ')
# The above results in ['Ġ'], which is correct
# Incorrect Tokenization
out = tokenizer.tokenize(' ')
# The above results in [], which is incorrect

# Example 2.
assert ' ' == tokenizer.decode(tokenizer.encode(' ', add_special_tokens=False))  # This will fail, since '' != ' '

```


## Expected behavior
Leading and trailing spaces should be considered during tokenization, especially for non-destructive tokenizers. 

## Proposed Solution
Changing the check from
```python
if not text.strip():
    return []
```
To
```python
if len(text) == 0:   # or if not text:
    return []
```
should be okay.
Alternatively, having a flag (eg: remove_extra_whitespaces), and enabling the current behaviour only for the case when the flag is passed as True would also work.
 

Comments:
May be of interest to @SaulLu 

Thank you very much for the detailed issue @codedecde !

This check had been integrated to solve non-deterministic tokenization problems and I think this solution had been retained because we did not see a use case at the time to tokenize a sentence containing only spaces (see [issue](https://github.com/huggingface/transformers/issues/2027) and [PR](https://github.com/huggingface/transformers/pull/2081)). 

Could you please explain in which case you need to tokenize a sentence containing only a space? Thank you very much in advance!

Hi @SaulLu.
Thank you for responding, and really sorry for the late response. 
My use-case is a little niche. I am training byte level encoder models. In order to do the masking, I am using a BPE tokenizer with dropout, and remapping it back to the byte level.
Eg: 
```[python]
tokenized = tokenizer.tokenize("Huggingface is awesome")
# ['Hug', 'ging', 'face', 'Ġ', 'is', 'Ġawesome']
inputs_with_mask, masked_tokens = mask_function(tokenized)
# ['Hug', 'ging', <mask>, **<mask>**, 'is', 'Ġawesome'], [<pad>, <pad>, 'face', **'Ġ',** <pad>, <pad>]
# The marked 'Ġ' token will get destroyed later because of the issue
decoded_text = byte_tokenizer.decode(inputs_with_mask)
# Hugging<mask><**mask>**is awesome
model_inputs, model_outputs = byte_tokenizer.encode(decoded_text, masked_tokens)
# ['H', 'u', 'g', 'g', 'i', 'n', 'g', <mask>, <mask>, <mask>, <mask>, **<mask>**, 'i', 's', ' ', 'a', 'w', 'e', 's', 'o', 'm', 'e']
# model_outputs = [<pad>,<pad>,<pad>,<pad>,<pad>,<pad>,<pad>, 'f', 'a', 'c', 'e', **''**, <pad>, ...]
```
In the above example, the mask inclosed between ** ** and its associated label are impacted by the problem mentioned. 
Since it is a niche use-case, having this as a kwarg flag enabled behaviour would be quite helpful (eg: by default, trailing and leading spaces are always stripped out, except when the flag is set to true ). 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.