[`WhisperTokenizer`]  Allow encoding timestamp tokens
Author: ArthurZucker
Pull request: yes

# What does this PR do?

Adresses #20225. Openai recently changed their tokenizer to allow encoding timestamp tokens as is (instead of splitting them). This is a breaking change because you can't encode them by splitting anymore, it will fail with the following error: 
```ptyhon 
ValueError: Encountered text corresponding to disallowed special token '<|7.86|>'.
If you want this text to be encoded as a special token, pass it to `allowed_special`, e.g. `allowed_special={'<|7.86|>', ...}`.
If you want this text to be encoded as normal text, disable the check for this token by passing `disallowed_special=(enc.special_tokens_set - {'<|7.86|>'})`.
To disable this check for all special tokens, pass `disallowed_special=()`.
```

This PR will have to wait before being merge. This is because the models on the hub need to be updated first otherwise the tests will be red. 
Moreover, `add_tokens` has to be fixed before that! 
Snipper showing why: 
```python 
from transformers import WhisperTokenizer, WhisperTokenizerFast, AddedToken
timestamps = [AddedToken("<|%.2f|>" % (i * 0.02), lstrip=False, rstrip=False) for  i in range(1500 + 1)]

from whisper.tokenizer import get_tokenizer
openai_tok = get_tokenizer(multilingual=True, language="en", task="transcribe")
model_path =f"openai/whisper-tiny"
slow = WhisperTokenizer.from_pretrained(model_path)
fast = WhisperTokenizerFast.from_pretrained(model_path)
slow.bos_token = AddedToken(slow.eos_token, lstrip=False, rstrip=False)
fast.bos_token = AddedToken(slow.eos_token, lstrip=False, rstrip=False)
slow.add_tokens(timestamps)
fast.add_tokens(timestamps)
```

The output from slow and fast is different. Fast matches the original implementation (not stripping spaces on the rigth and left) while slow does not. 

```python 
>>> openai_tok.encode("<|7.86|> Hey", allowed_special=set(openai_tok.special_tokens.keys()))
[50757, 1911]
>>> fast.encode('<|7.86|> Hey', add_special_tokens = False)
[50757, 1911]
>>> slow.encode('<|7.86|> Hey', add_special_tokens = False)
[50757, 7057]
```

script to update all models :
```python
from transformers import WhisperTokenizer, WhisperTokenizerFast, AddedToken
timestamps = [AddedToken("<|%.2f|>" % (i * 0.02), lstrip=False, rstrip=False) for  i in range(1500 + 1)]
models_ids = ["tiny","small","medium","base","large"]

from whisper.tokenizer import get_tokenizer
openai_tok = get_tokenizer(multilingual=True, language="en", task="transcribe")


openai_tok.encode("<|1.00|>", allowed_special=set(openai_tok.special_tokens.keys()))

for id in models_ids:
    model_path =f"openai/whisper-{id}"
    slow = WhisperTokenizer.from_pretrained(model_path)
    fast = WhisperTokenizerFast.from_pretrained(model_path)
    slow.bos_token = AddedToken(slow.eos_token, lstrip=False, rstrip=False)
    fast.bos_token = AddedToken(slow.eos_token, lstrip=False, rstrip=False)
    slow.add_tokens(timestamps)
    fast.add_tokens(timestamps)
    slow.push_to_hub(model_path, create_pr = True)
    fast.push_to_hub(model_path, create_pr = True)

    if id == "large":
        exit(0)

    model_path += '.en'
    slow = WhisperTokenizer.from_pretrained(model_path)
    fast = WhisperTokenizerFast.from_pretrained(model_path)
    slow.bos_token = AddedToken(slow.eos_token, lstrip=False, rstrip=False)
    fast.bos_token = AddedToken(slow.eos_token, lstrip=False, rstrip=False)
    slow.add_tokens(timestamps)
    fast.add_tokens(timestamps)
    slow.push_to_hub(model_path, create_pr = True)
    fast.push_to_hub(model_path, create_pr = True)


```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

cc @sanchit-gandhi 

In order to keep backward compatibility / follow the original behaviour, I'll add a `encode_special_token` to whisper tokenizer. Not sure we can have 100% backward on this, because all specials tokens will be affected. 

Closing this as #25081 adds `split_special_tokens` and the timestamp tokens will be manually added! 

Just to clarify - we'll only need to update the tokenizer vocabs on the Hub following #25081?

yes! 

Cool! Happy to open the Hub PRs!

Just to clarify, it looks like the slow tokenizer still doesn't _quite_ give the expected behaviour when new special tokens are added:
```python
from transformers import WhisperTokenizer, AddedToken

tokenizer = WhisperTokenizer.from_pretrained("openai/whisper-tiny")

timestamps = [AddedToken("<|%.2f|>" % (i * 0.02), lstrip=False, rstrip=False) for  i in range(1500 + 1)]
tokenizer.add_tokens(timestamps)

print(tokenizer.decode(tokenizer("<|0.00|> But like mobile phones have screens and they're cheap.<|2.60|>", split_special_tokens=False).input_ids))
```
**Print Output:**
```
"

<|0.00|>But like mobile phones have screens and they're cheap.<|2.60|>
"
```

=> we loose the space between a special token and the adjacent token, e.g. `<|0.00|> But` goes to `<|0.00|>But`


Yep, will be fixed by #23909 😉 

Cool! And the inconsistency between the slow and fast tokenizer too? Is this related to the add tokens?
```python
from transformers import WhisperTokenizer, WhisperTokenizerFast

tokenizer = WhisperTokenizer.from_pretrained(f"openai/whisper-tiny")
tokenizer_fast = WhisperTokenizerFast.from_pretrained(f"openai/whisper-tiny")

print(tokenizer.encode("<|0.00|> hey"))
print(tokenizer_fast.encode("<|0.00|> hey"))
```
**Print Output:**
```
[50258, 50363, 50364, 17230, 50257]
[50258, 50363, 50364, 4177, 50257]
```

Yep, when you add the token add them as `AddedToken` with rstrip = True and `lstrip=True` if you want the same behaviour