Slow tokenizer not loading mixtral 8x22b tokenizer correctly
Author: Yangqing
Pull request: no

### System Info

- `transformers` version: 4.41.1
- Platform: Linux-6.5.0-28-generic-x86_64-with-glibc2.35
- Python version: 3.10.13
- Huggingface_hub version: 0.23.2
- Safetensors version: 0.4.1
- Accelerate version: 0.25.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.3.0+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Run the following command:
```
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
                    "mistralai/Mixtral-8x22B-Instruct-v0.1",
                    trust_remote_code=True,
                    use_fast=False,
                )
```
And one will encounter an error that looks like this:
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[2], line 1
----> 1 tokenizer = AutoTokenizer.from_pretrained(
      2                     "mistralai/Mixtral-8x22B-Instruct-v0.1",
      3                     trust_remote_code=True,
      4                     use_fast=False,
      5                 )

File ~/anaconda3/envs/lepton/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:880, in AutoTokenizer.from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)
    876     if tokenizer_class is None:
    877         raise ValueError(
    878             f"Tokenizer class {tokenizer_class_candidate} does not exist or is not currently imported."
    879         )
--> 880     return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
    882 # Otherwise we have to be creative.
    883 # if model is an encoder decoder, the encoder tokenizer class is used by default
    884 if isinstance(config, EncoderDecoderConfig):

File ~/anaconda3/envs/lepton/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:2110, in PreTrainedTokenizerBase.from_pretrained(cls, pretrained_model_name_or_path, cache_dir, force_download, local_files_only, token, revision, trust_remote_code, *init_inputs, **kwargs)
   2107     else:
   2108         logger.info(f"loading file {file_path} from cache at {resolved_vocab_files[file_id]}")
-> 2110 return cls._from_pretrained(
   2111     resolved_vocab_files,
   2112     pretrained_model_name_or_path,
   2113     init_configuration,
   2114     *init_inputs,
   2115     token=token,
   2116     cache_dir=cache_dir,
   2117     local_files_only=local_files_only,
   2118     _commit_hash=commit_hash,
   2119     _is_local=is_local,
   2120     trust_remote_code=trust_remote_code,
   2121     **kwargs,
   2122 )

File ~/anaconda3/envs/lepton/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:2336, in PreTrainedTokenizerBase._from_pretrained(cls, resolved_vocab_files, pretrained_model_name_or_path, init_configuration, token, cache_dir, local_files_only, _commit_hash, _is_local, trust_remote_code, *init_inputs, **kwargs)
   2334 # Instantiate the tokenizer.
   2335 try:
-> 2336     tokenizer = cls(*init_inputs, **init_kwargs)
   2337 except OSError:
   2338     raise OSError(
   2339         "Unable to load vocabulary from file. "
   2340         "Please check that the provided vocabulary is accessible and not corrupted."
   2341     )

File ~/anaconda3/envs/lepton/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama.py:169, in LlamaTokenizer.__init__(self, vocab_file, unk_token, bos_token, eos_token, pad_token, sp_model_kwargs, add_bos_token, add_eos_token, clean_up_tokenization_spaces, use_default_system_prompt, spaces_between_special_tokens, legacy, add_prefix_space, **kwargs)
    167 self.add_eos_token = add_eos_token
    168 self.use_default_system_prompt = use_default_system_prompt
--> 169 self.sp_model = self.get_spm_processor(kwargs.pop("from_slow", False))
    170 self.add_prefix_space = add_prefix_space
    172 super().__init__(
    173     bos_token=bos_token,
    174     eos_token=eos_token,
   (...)
    185     **kwargs,
    186 )

File ~/anaconda3/envs/lepton/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama.py:196, in LlamaTokenizer.get_spm_processor(self, from_slow)
    194 tokenizer = spm.SentencePieceProcessor(**self.sp_model_kwargs)
    195 if self.legacy or from_slow:  # no dependency on protobuf
--> 196     tokenizer.Load(self.vocab_file)
    197     return tokenizer
    199 with open(self.vocab_file, "rb") as f:

File ~/anaconda3/envs/lepton/lib/python3.10/site-packages/sentencepiece/__init__.py:961, in SentencePieceProcessor.Load(self, model_file, model_proto)
    959 if model_proto:
    960   return self.LoadFromSerializedProto(model_proto)
--> 961 return self.LoadFromFile(model_file)

File ~/anaconda3/envs/lepton/lib/python3.10/site-packages/sentencepiece/__init__.py:316, in SentencePieceProcessor.LoadFromFile(self, arg)
    315 def LoadFromFile(self, arg):
--> 316     return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)

TypeError: not a string
```

It seems that the vocab_file is resolved to be None.

### Expected behavior

The tokenizer should be properly loaded - interesting enough, the 8x7b model loads correctly. This can be verified if you run
```
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
                    "mistralai/Mixtral-8x22B-Instruct-v0.1",
                    trust_remote_code=True,
                    use_fast=False,
                )
```
instead. Note that the problem also does not show up if one uses use_fast=True. However, in some cases, we do need to use the slow tokenizer for other reasons, which is why we discovered this issue.

Comments:
I did a bit of search and found https://github.com/huggingface/transformers/issues/29137 - seems to be a mixtral model issue itself. Closing.

Thanks for updating