Do we need to add [CLS] and [SEP] for BertForMaskedLM ?
Author: yaolu
Pull request: no

## ❓ Questions & Help

https://github.com/huggingface/transformers/blob/cc7968227e08858df4a5c618c739e1a3ca050196/transformers/modeling_bert.py#L837-L841

Seems like the example is wrong?

Comments:
like this ? 
```
input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute", add_special_tokens=True)).unsqueeze(0)  # Batch size 1 
```

The output **without** `add_special_tokens=True`:
```
import torch
from transformers import BertTokenizer, BertForMaskedLM
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
text = 'Hello, my dog is cute'
input_ids = torch.tensor(tokenizer.encode(text)).unsqueeze(0)
>>> tensor([[  101,  7592,  1010,  2026,  3899,  2003, 10140,   102]])
```
If you inspect the vocabulary built by BertTokenizer (accessible through `tokenizer.vocab`), you can see that the token [CLS] and [SEP] have ID 101 and 102, respectively. So, `tokenizer.encode` already add these two tokens at the start and at the end of each encoded sentence.

The output **with** `add_special_tokens=True`:
```
import torch
from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', add_special_tokens=True)
text = 'Hello, my dog is cute'
input_ids = torch.tensor(tokenizer.encode(text)).unsqueeze(0)
>>> tensor([[  101,  7592,  1010,  2026,  3899,  2003, 10140,   102]])
```

As you can see, **the output obtained is the same**.

Moreover, this night Transformers passed from 2.1.1 to 2.2.0 version, and reading [here](https://github.com/huggingface/transformers/releases) we can see the statement **Tokenizers now add special tokens by default.**.

> ##  Questions & Help
> https://github.com/huggingface/transformers/blob/cc7968227e08858df4a5c618c739e1a3ca050196/transformers/modeling_bert.py#L837-L841
> 
> Seems like the example is wrong?

Yes, my version v2.1.1 will not set add_special_tokens to True by default. Thanks for your comment.

> like this ?
> 
> ```
> input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute", add_special_tokens=True)).unsqueeze(0)  # Batch size 1 
> ```

> The output **without** `add_special_tokens=True`:
> 
> ```
> import torch
> from transformers import BertTokenizer, BertForMaskedLM
> tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
> text = 'Hello, my dog is cute'
> input_ids = torch.tensor(tokenizer.encode(text)).unsqueeze(0)
> >>> tensor([[  101,  7592,  1010,  2026,  3899,  2003, 10140,   102]])
> ```
> 
> If you inspect the vocabulary built by BertTokenizer (accessible through `tokenizer.vocab`), you can see that the token [CLS] and [SEP] have ID 101 and 102, respectively. So, `tokenizer.encode` already add these two tokens at the start and at the end of each encoded sentence.
> 
> The output **with** `add_special_tokens=True`:
> 
> ```
> import torch
> from transformers import BertTokenizer
> tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', add_special_tokens=True)
> text = 'Hello, my dog is cute'
> input_ids = torch.tensor(tokenizer.encode(text)).unsqueeze(0)
> >>> tensor([[  101,  7592,  1010,  2026,  3899,  2003, 10140,   102]])
> ```
> 
> As you can see, **the output obtained is the same**.
> 
> Moreover, this night Transformers passed from 2.1.1 to 2.2.0 version, and reading [here](https://github.com/huggingface/transformers/releases) we can see the statement **Tokenizers now add special tokens by default.**.
> 
> > ## Questions & Help
> > https://github.com/huggingface/transformers/blob/cc7968227e08858df4a5c618c739e1a3ca050196/transformers/modeling_bert.py#L837-L841
> > 
> > Seems like the example is wrong?

create pull request #1971  to make this less ambiguous across different versions.