AutoTokenizer tokenization issue
Author: satpalsr
Pull request: no

### System Info

On colab, transformers==4.38.2

### Who can help?

@ArthurZucker @younesbelkada

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import GemmaTokenizer, AutoTokenizer

gemma_tokenizer = GemmaTokenizer.from_pretrained("google/gemma-7b")
gemma_auto_tokenizer = AutoTokenizer.from_pretrained("google/gemma-7b")

print(gemma_tokenizer.tokenize("<start_of_turn>"))
print(gemma_auto_tokenizer.tokenize("<start_of_turn>"))
```
The result is
```
['<start_of_turn>']
['<', 'start', '_', 'of', '_', 'turn', '>']
```



### Expected behavior

Both tokenization result should be same.

Comments:
Hey! Thanks for the report. I can reproduce (note that this model does not use this token as it is not the finetuned one so not super important IMO). cc @xenova seems like all of these would have needed to be manually added! 😓 
The fix is to update the `convert_slow` part to make sure all the user-defined tokens are added as `AddedToken`. 
I'll open a PR soon 

cc @pcuenca we should probably update the repos on the hub as well for that! 

There's this another model where this is problematic.
The Llama tokenizer here was expanded to include Hindi language tokens.
```
from transformers import LlamaTokenizer, AutoTokenizer
auto_tokenizer = AutoTokenizer.from_pretrained("sarvamai/OpenHathi-7B-Hi-v0.1-Base")
llama_tokenizer = LlamaTokenizer.from_pretrained("sarvamai/OpenHathi-7B-Hi-v0.1-Base")

# sentence = "हिंदी में एक वाक्य"
sentence = "दक्षिण एशिया में स्थित भारतीय उपमहाद्वीप का सबसे बड़ा देश है।"
print(len(auto_tokenizer.tokenize(sentence)), auto_tokenizer.tokenize(sentence))
print( len(llama_tokenizer.tokenize(sentence)), llama_tokenizer.tokenize(sentence))
```
When using LlamaTokenizer it splits perfectly, but not with AutoTokenizer
Result:
```
20 ['▁दक्ष', 'ि', 'ण', '▁एशिया', '▁में', '▁स्', 'थ', 'ित', '▁भारतीय', '▁उप', 'महा', 'द्', 'वी', 'प', '▁का', '▁सबसे', '▁बड़ा', '▁देश', '▁है', '।']
16 ['▁दक्षिण', '▁एशिया', '▁में', '▁स्थित', '▁भारतीय', '▁उप', 'महा', 'द्व', 'ी', 'प', '▁का', '▁सबसे', '▁बड़ा', '▁देश', '▁है', '।']
```

Was the tokenizer trained from the `sentencepiece` repo with user defined symbols? 

Though I haven't trained this tokenizer myself, I believe it must have been done in similar fashion to these scripts:
1. [Tokenzier train](https://github.com/abhinand5/tamil-llama/blob/main/scripts/train/sentencepiece/train.py)
2. [Merge Tokenizer](https://github.com/abhinand5/tamil-llama/blob/main/scripts/train/sentencepiece/merge_tokenizer.py)

Here's an example of tokenizer created with these scripts:
```
from transformers import LlamaTokenizer, AutoTokenizer
auto_tokenizer = AutoTokenizer.from_pretrained("abhinand/tamil-llama-7b-base-v0.1")
llama_tokenizer = LlamaTokenizer.from_pretrained("abhinand/tamil-llama-7b-base-v0.1")

sentence = "அவன் ஒரு புத்தகம் படித்து கொண்டிருக்கிறான்"
print(len(auto_tokenizer.tokenize(sentence)), auto_tokenizer.tokenize(sentence))
print( len(llama_tokenizer.tokenize(sentence)), llama_tokenizer.tokenize(sentence))
```
Result:
```
10 ['▁அவன்', '▁ஒரு', '▁புத்தகம்', '▁படித்து', '▁கெ', 'ா', 'ண்டி', 'ரு', 'க்கிற', 'ான்']
7 ['▁அவன்', '▁ஒரு', '▁புத்தகம்', '▁படித்து', '▁கெ', 'ாண்ட', 'ிருக்கிறான்']
```

Hey! I opened a PR to fix the `gemma` issue, but for Llama it is not related to `user_defined_symbols`. 
You have defined your custom way of converting the tokenizer, so it's not related to `tokenizers` nor `transformers`.