Roberta Tokenizer cannot handle inputs with `<mask>` token
Author: liyucheng09
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.5.1
- Platform: macOS-11.1-arm64-arm-64bit
- Python version: 3.9.1
- PyTorch version (GPU?): 1.8.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @Rocketknight1

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Description

It'a bizarre bug. When I encode a string with a `<mask>` token and decode it immediately, the space in front of the `<mask>` token disappears. 

```
>>> from transformers import AutoTokenizer
>>> tokenizer=AutoTokenizer.from_pretrained('roberta-base')
>>> tokenized_inputs=tokenizer('I <mask> you')['input_ids']
>>> tokenizer.decode(tokenized_inputs)
'
I<mask> you
'
>>> 
```

It will lead to many problems. Such as `pipeline('fill-mask')` cannot provides valid results.

```
>>> from transformers import pipeline
>>> nlp=pipeline('fill-mask')
>>> nlp.tokenizer
PreTrainedTokenizerFast(name_or_path='distilroberta-base', vocab_size=50265, model_max_len=512, is_fast=True, padding_side='right', special_tokens={'bos_token': '
', 'eos_token': '
', 'unk_token': '<unk>', 'sep_token': '
', 'pad_token': '<pad>', 'cls_token': '
', 'mask_token': AddedToken("<mask>", rstrip=False, lstrip=True, single_word=False, normalized=False)})
>>> nlp('app <mask>')
{'score': 0.09366267919540405,
 'sequence': 'appeal',
 'token': 18696,
 'token_str': 'eal'}
```

It seems that the mask-filling process omits the space, which is not what we expected (we expect the token to be filled in mask is another word rather than a sub-word, since there is a space as a separator.) 

Does anyone notice the same issue?

Comments:
Hi, the tokenizer of RoBERTa is a byte-level BPE tokenizer. It is a subword tokenizer.

In your example with `app ` it completes the word, but there also exists full words within it's dictionary, as you can see when the input sequence is appropriate:

```py
>>> nlp('Hello, how are you <mask> sir?')
[
    {'sequence': 'Hello, how are you doing sir?', 'score': 0.6784416437149048, 'token': 608, 'token_str': ' doing'}, 
    {'sequence': 'Hello, how are you feeling sir?', 'score': 0.08236288279294968, 'token': 2157, 'token_str': ' feeling'}, 
    {'sequence': 'Hello, how are you, sir?', 'score': 0.06469670683145523, 'token': 6, 'token_str': ','}, 
    {'sequence': 'Hello, how are you looking sir?', 'score': 0.04527667537331581, 'token': 546, 'token_str': ' looking'}, 
    {'sequence': 'Hello, how are you going sir?', 'score': 0.02970985323190689, 'token': 164, 'token_str': ' going'}]
```

@LysandreJik Thanks for your reply! I think this behavior is proper during the pretraining process: roberta completes or predicts the next token when we give the following input `app<mask>`. But I believe this is not expected when we call `fill-mask` pipeline.

Take `app <mask>` as example, there is a space in `app <mask>`, which means the `app` token is a complete token and roberta should predict the next token. However, after tokenizing, the space vanished. To realize what I said, maybe the only way is to rewrite the predict function to find those token with a space in front of it.

P.S. add an extra space `app  <mask>` do not bring expected results as well.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.