[BUG] RecursionError: maximum recursion depth exceeded when loading LLaMA-1 but success loading LLaMA-2
Author: SingL3
Pull request: no

### System Info

- `transformers` version: 4.31.0
- Platform: Linux-4.18.0-425.3.1.el8.x86_64-x86_64-with-glibc2.31
- Python version: 3.10.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: 0.21.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker @younesbelkada 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Running (Open-Assistant)[https://github.com/LAION-AI/Open-Assistant] script for (SFT)[https://github.com/LAION-AI/Open-Assistant/blob/main/model/model_training/trainer_sft.py]
When loading LLaMA-1, got this error:
```
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1155, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
RecursionError: maximum recursion depth exceeded
```
Seems like go into a infinite loop, and success loading LLaMA-2.

### Expected behavior

Success loading LLaMA-1 and LLaMA-2

Comments:
I set a small recursion limit to 50 and I now got this trace back for loading LLaMA-1:
```
  File "/mnt/home//Open-Assistant/model/model_training/trainer_sft.py", line 481, in <module>
    main()
  File "/mnt/home//Open-Assistant/model/model_training/trainer_sft.py", line 333, in main
    tokenizer = get_tokenizer(training_conf)
  File "/mnt/home//Open-Assistant/model/model_training/utils/utils.py", line 214, in get_tokenizer
    tokenizer = transformers.AutoTokenizer.from_pretrained(tokenizer_name, cache_dir=conf.cache_dir)
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 702, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1841, in from_pretrained
    return cls._from_pretrained(
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2004, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 126, in __init__
    self.update_post_processor()
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 136, in update_post_processor
    bos_token_id = self.bos_token_id
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1136, in bos_token_id
    return self.convert_tokens_to_ids(self.bos_token)
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 250, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/mnt/data/conda/envs/oasst/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 257, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
...
```
Until it exceeded the limit.

FYI, I can load LLaMA-1 using `transformers==4.28.1`

Hey! Thanks for reporting, this is very similar to #22415, and #22414.
Having a minimal reproducer would be better, I cannot reproduce this out-of-the-box, have no idea what `tokenizer_config` they are using or what not. 

From the traceback, I created a minimal reproducer:
```python 
>>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("huggyllama/llama-7b", unk_token = "<endoftext>", bos_token="<endoftext>", eos_token="<endoftext>")
```
In this case, the `<endoftext>` token does not exist, and since there are a few issues with adding tokens when initializing, cf #23909 after calling `super().__init__()` the token is still not part of the vocab. 


It works with  `transformers==4.28.1`, because the tokenizer did not have the `self.update_post_processor()`. 

There are no real quick fixes appart from downgrading for now, but I'll probably either remove the call in the init, or fixing token addition will make sure the token is properly added after calling `super().__init__()`. 

@ArthurZucker 
To reproduce:
`tokenizer_config.json`:
```json
{"bos_token": "", "eos_token": "", "model_max_length": 1000000000000000019884624838656, "tokenizer_class": "LlamaTokenizer", "unk_token": ""}
```
This is an old version of LLaMA-1, I think it is from [here](https://huggingface.co/decapoda-research/llama-7b-hf/tree/main)(need to edit `LLaMATokenizer` to `LlamaTokenizer`).

Transformers versions 4.31.0
This also hits the same issue

```
from tokenizers.trainers import BpeTrainer
from tokenizers.pre_tokenizers import Whitespace
from tokenizers import Tokenizer
from tokenizers.models import BPE
from datasets import load_dataset
from transformers import LlamaConfig, LlamaTokenizer, LlamaForCausalLM, LlamaTokenizerFast, PreTrainedTokenizerFast

dataset = load_dataset("tiny_shakespeare")
tokenizer = Tokenizer(BPE(unk_token="[UNK]"))
trainer = BpeTrainer(vocab_size=1024)
tokenizer.pre_tokenizer = Whitespace()
tokenizer.train_from_iterator(dataset['train']['text'], trainer=trainer)

# tokenizer.encode("hello world")
tokenizer.save('tiny.json')
t_tok1 = PreTrainedTokenizerFast(tokenizer_file='tiny.json') # works
t_tok2 = LlamaTokenizerFast(tokenizer_file='tiny.json') # fails
```

```
trainer = BpeTrainer(special_tokens=["<unk>", "
", "
"], vocab_size=1024)
```

Got it to work by doing this

Thanks for debugging! 

@ArthurZucker I just ran into this issue today when doing:
```py
from transformers import pipeline
pipeline('text-generation', 'JackFram/llama-160m')
```

it does indeed look like the problem is what you mentioned. Here's the [tokenizer_config.json](https://huggingface.co/JackFram/llama-160m/blob/main/tokenizer_config.json).

Edit: Author merged by PR to fix this [here](https://huggingface.co/JackFram/llama-160m/discussions/9#64f3df59da084738feffa4ae), but the error still remains in transformers.


That is expected, the[ special tokens map file is still wrong](https://huggingface.co/JackFram/llama-160m/blob/main/special_tokens_map.json) 😉 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Will close ass #23909 should have resolved it

Using `transformers=4.34.1` and python 3.9.18 I still encounter the the recursion depth limit which should have been fixed with #23909 if I understood correctly?
Here is my code:
``` python
import transformers
import sys
sys.setrecursionlimit(100)
print(transformers.__version__)
tokenizer = transformers.LlamaTokenizer.from_pretrained('baffo32/decapoda-research-llama-7B-hf')
print("Done LlamaTokenizer")
tokenizer = transformers.AutoTokenizer.from_pretrained('baffo32/decapoda-research-llama-7B-hf')
print("Done AutoTokenizer")
```
This is the `tokenizer_config.json`:
``` json
{"bos_token": "", "eos_token": "", "model_max_length": 1000000000000000019884624838656, "tokenizer_class": "LlamaTokenizer", "unk_token": ""}
```
The error I am getting is:
``` shell
4.34.1
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thouroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565
Done LlamaTokenizer
Traceback (most recent call last):
  File "/home/r403k/Projects/huggingface/minimal_working_example.py", line 10, in <module>
    tokenizer = transformers.AutoTokenizer.from_pretrained('baffo32/decapoda-research-llama-7B-hf', use_local_files=False)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 751, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 2017, in from_pretrained
    return cls._from_pretrained(
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 2249, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 134, in __init__
    self.update_post_processor()
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 147, in update_post_processor
    bos_token_id = self.bos_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1172, in bos_token_id
    return self.convert_tokens_to_ids(self.bos_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 329, in convert_tokens_to_ids
    return self._convert_token_to_id_with_added_voc(tokens)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 336, in _convert_token_to_id_with_added_voc
    return self.unk_token_id
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1191, in unk_token_id
    return self.convert_tokens_to_ids(self.unk_token)
  File "/home/r403k/miniconda3/envs/openflamingo/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1057, in unk_token
    return str(self._unk_token)
RecursionError: maximum recursion depth exceeded while getting the str of an object
```

Hey! It was indeed fixed for other models, but Llama is a bit specific, we call `self.update_post_processor()` which makes sure that the eos token is added. This should work without a bos token, but you can't have no `unk_token`, this is pretty much a requirement for all our tokenizers. Not sure I can fix this as raising an error would not be BC :/ would recommend doing this: 
```python 
>>> tokenizer = transformers.LlamaTokenizer.from_pretrained('baffo32/decapoda-research-llama-7B-hf', unk_token="<unk>") 
```

Hey Arthur!
thank you for the quick reply! I only just now saw [this](https://github.com/huggingface/transformers/issues/24318#issuecomment-1596801322) where you solved my issue as well.
Setting the `unk_token` in the `tokenizer_config.json` was sufficient as well.