mT5 additional_special_tokens seems not work
Author: PiggyFan
Pull request: no

I want add some special tokens such as <POS>  <CON_START> . But T5tokenizer/MT5tokenizer both can't tokenize correctly after using additional_special_tokens parameter. It still split these special tokens to subwords.
<img width="1015" alt="截圖 2021-01-22 下午5 18 00" src="https://user-images.githubusercontent.com/26171212/105473581-3d88d100-5cd8-11eb-8568-6fedd19513e2.png">


It works when using OpenAIGPTTokenizer additional_special_tokens parameter. It's clear that after declare additional_special_tokens parameter, OpenAIGPTTokenizer tokenize <POS> as one word rather split it.
<img width="979" alt="截圖 2021-01-22 下午5 54 57" src="https://user-images.githubusercontent.com/26171212/105475970-049e2b80-5cdb-11eb-8470-576fd8f38999.png">
<img width="697" alt="截圖 2021-01-22 下午5 55 10" src="https://user-images.githubusercontent.com/26171212/105475992-0962df80-5cdb-11eb-9cae-205b57818e95.png">




The version of transformers is 4.2.2
And I'm not sure this problem is related with [issue624](https://github.com/google-research/text-to-text-transfer-transformer/issues/624) in T5 which talk about SentencePiece extra vocab.

Thank you for your feedback

Comments:
Hi! Could you either post a link to your notebook or your code as actual code? Images makes it impossible to copy/paste or for others to search with similar issues. Thanks.

> Hi! Could you either post a link to your notebook or your code as actual code? Images makes it impossible to copy/paste or for others to search with similar issues. Thanks.

Here is colab notebook link. Thanks for your reply.

https://colab.research.google.com/drive/1fbp7VvnUvbf5r8CSitOg2pDZyM47y9xj?usp=sharing

Hi! Indeed, this is a bit misleading. Special tokens are considered as those that were in the pre-training, that is: unknown tokens, bos tokens, eos tokens, etc.

If you want to use special tokens that you use as special tokens, I would argue it is better to define them as simple tokens. Therefore doing the following:

```py
>>> from transformers import MT5Tokenizer, MT5ForConditionalGeneration
... import torch
... special_tokens = ['<POS>', '<NEG>','<CON_START>','<START>','<END>'] # Set the special tokens
... mt5_add_tokenizer = MT5Tokenizer.from_pretrained("google/mt5-small")
... mt5_add_tokenizer.add_tokens(special_tokens)
... print(mt5_add_tokenizer.tokenize("<POS> <CON_START> the biscuits and gravy were . <START>"))
```

You'll get the following output:
```out
['<POS>', '<CON_START>', '▁the', '▁b', 'iscuit', 's', '▁and', '▁grav', 'y', '▁were', '▁', '.', '<START>']
````

Let me know if that makes sense.

Thank you very much, it works well.
I was confused by [issue5940](https://github.com/huggingface/transformers/issues/5940) which mentioned "special tokens are carefully handled by the tokenizer (they are never split)". If use add_token() method, the additional simple tokens may still split?
In addition, the code in Colab notebook as above link, OpenaiTokenizer should use add_tokens method rather than add_special_tokens (define them as a simple tokens) ?

For tokens that cannot be identified as being either:
- `eos_token`
- `bos_token`
- `cls_token`
- `unk_token`
- `pad_token`
- `sep_token`
- `mask_token`

Then I would recommend using the `add_token` method. These tokens shouldn't split either:
```py
>>> from transformers import BertTokenizer
>>> tokenizer = BertTokenizer.from_pretrained("bert-base-cased")
>>> tokenizer.add_tokens(["<CON_START>", "<CON_ST"])
2
>>> tokenizer.tokenize("<CON_START>")
['<CON_START>']
>>> tokenizer.tokenize("<CON_STAR>")
['<CON_ST', 'AR', '>']
```

Thank for your help and recommend.

> For tokens that cannot be identified as being either:
> 
> * `eos_token`
> * `bos_token`
> * `cls_token`
> * `unk_token`
> * `pad_token`
> * `sep_token`
> * `mask_token`
> 
> Then I would recommend using the `add_token` method. These tokens shouldn't split either:
> 
> ```python
> >>> from transformers import BertTokenizer
> >>> tokenizer = BertTokenizer.from_pretrained("bert-base-cased")
> >>> tokenizer.add_tokens(["<CON_START>", "<CON_ST"])
> 2
> >>> tokenizer.tokenize("<CON_START>")
> ['<CON_START>']
> >>> tokenizer.tokenize("<CON_STAR>")
> ['<CON_ST', 'AR', '>']
> ```

Hi it seems `add_tokens` not working with `AutoTokenizer` , but work with specific-defined tokenizer like `BertTokenizer`, 
```python
special_tokens = ["-Title-"]
#tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
tokenizer.add_tokens(special_tokens)
tokenizer.tokenize("-Title-")
# ['-', 'title', '-']
```

He! Thanks for reporting. This should be fixed by #23909