`AutoTokenizer.from_pretrained` raise error when another same filename tokenizer imported
Author: tongyifan
Pull request: no

### System Info

- `transformers` version: 4.28.1
- Platform: Linux-4.19.91-009.ali4000.alios7.x86_64-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.16.2
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@sgugger

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. bug reproduction
```python
from transformers import AutoTokenizer

# import one tokenizer from local (/home/admin/notebook/THUDM/chatglm2-6b/tokenization_chatglm.py)
tokenizer = AutoTokenizer.from_pretrained("/home/admin/notebook/THUDM/chatglm2-6b/", trust_remote_code=True)

# import another version tokenizer from another directory, but same filename (/home/admin/notebook/THUDM/chatglm-6b/tokenization_chatglm.py)
tokenizer = AutoTokenizer.from_pretrained("/home/admin/notebook/THUDM/chatglm-6b/", trust_remote_code=True)
```

raise exception 
```text
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[1], line 10
      6 from transformers import AutoTokenizer
      8 tokenizer = AutoTokenizer.from_pretrained("/home/admin/notebook/THUDM/chatglm2-6b/", trust_remote_code=True)
---> 10 tokenizer = AutoTokenizer.from_pretrained("/home/admin/notebook/THUDM/chatglm-6b/", trust_remote_code=True)

File /usr/local/lib/python3.8/dist-packages/transformers/models/auto/tokenization_auto.py:702, in AutoTokenizer.from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)
    698     if tokenizer_class is None:
    699         raise ValueError(
    700             f"Tokenizer class {tokenizer_class_candidate} does not exist or is not currently imported."
    701         )
--> 702     return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
    704 # Otherwise we have to be creative.
    705 # if model is an encoder decoder, the encoder tokenizer class is used by default
    706 if isinstance(config, EncoderDecoderConfig):

File /usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py:1811, in PreTrainedTokenizerBase.from_pretrained(cls, pretrained_model_name_or_path, *init_inputs, **kwargs)
   1808     else:
   1809         logger.info(f"loading file {file_path} from cache at {resolved_vocab_files[file_id]}")
-> 1811 return cls._from_pretrained(
   1812     resolved_vocab_files,
   1813     pretrained_model_name_or_path,
   1814     init_configuration,
   1815     *init_inputs,
   1816     use_auth_token=use_auth_token,
   1817     cache_dir=cache_dir,
   1818     local_files_only=local_files_only,
   1819     _commit_hash=commit_hash,
   1820     **kwargs,
   1821 )

File /usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py:1965, in PreTrainedTokenizerBase._from_pretrained(cls, resolved_vocab_files, pretrained_model_name_or_path, init_configuration, use_auth_token, cache_dir, local_files_only, _commit_hash, *init_inputs, **kwargs)
   1963 # Instantiate tokenizer.
   1964 try:
-> 1965     tokenizer = cls(*init_inputs, **init_kwargs)
   1966 except OSError:
   1967     raise OSError(
   1968         "Unable to load vocabulary from file. "
   1969         "Please check that the provided vocabulary is accessible and not corrupted."
   1970     )

File ~/.cache/huggingface/modules/transformers_modules/tokenization_chatglm.py:69, in __init__(self, vocab_file, padding_side, **kwargs)
     66 def _get_text_tokenizer(self):
     67     return self.text_tokenizer
---> 69 @staticmethod
     70 def get_blank_token(length: int):
     71     assert length >= 2
     72     return f"<|blank_{length}|>"

File /usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils.py:347, in PreTrainedTokenizer.__init__(self, **kwargs)
    346 def __init__(self, **kwargs):
--> 347     super().__init__(**kwargs)
    349     # Added tokens - We store this for both slow and fast tokenizers
    350     # until the serialization of Fast tokenizers is updated
    351     self.added_tokens_encoder: Dict[str, int] = {}

File /usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py:1534, in PreTrainedTokenizerBase.__init__(self, **kwargs)
   1530 self.deprecation_warnings = (
   1531     {}
   1532 )  # Use to store when we have already noticed a deprecation warning (avoid overlogging).
   1533 self._in_target_context_manager = False
-> 1534 super().__init__(**kwargs)

File /usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py:828, in SpecialTokensMixin.__init__(self, verbose, **kwargs)
    826     setattr(self, key, value)
    827 elif isinstance(value, (str, AddedToken)):
--> 828     setattr(self, key, value)
    829 else:
    830     raise TypeError(f"special token {key} has to be either str or AddedToken but got: {type(value)}")

AttributeError: can't set attribute
```

In this traceback, I found the exception raised from a abnormal line 69 `@staticmethod`, which pointed to the correct file `/home/admin/notebook/THUDM/chatglm-6b/tokenization_chatglm.py`
![image](https://github.com/huggingface/transformers/assets/25120867/1762dd64-8d92-49f4-963e-5ad26dece4d5)

but the exception was raised from the wrong file (/home/admin/notebook/THUDM/chatglm2-6b/tokenization_chatglm.py)
![image](https://github.com/huggingface/transformers/assets/25120867/dbaf37e0-3a1a-4795-8613-4016332818bb)

2. a hard fix
After edit one filename of these two tokenizers, the bug disappear.

### Expected behavior

import the tokenizer with no exception

Comments:
I think you should report the issue in the repositories where this code comes from, as the fix you suggest is in that code (and not in the Transformers library).

hmmm that's an ad-hoc solution to validate the problem, and I'm not sure whether this import error is expected by Transformers?

I don't see an import error in the code and traceback you pasted. I see the custom code of this tokenizer failing to execute.

Here's a reproduce code
[transformers_issue_25194.tar.gz](https://github.com/huggingface/transformers/files/12257984/transformers_issue_25194.tar.gz)
```
transformers_issue_25194
├── main.py
├── test_a
│   ├── tokenization.py
│   └── tokenizer_config.json
└── test_b
    ├── tokenization.py
    └── tokenizer_config.json

2 directories, 5 files
```

`test_a` is a tokenizer with param `a` and `test_b` is another tokenizer with param `b`.

When running the `main.py`, the exception raised when loading `test_b`, but the error is `__init__() missing 1 required positional argument: 'a'`, which is the required param from `test_a`.

In this line `b = AutoTokenizer.from_pretrained("./test_b/", trust_remote_code=True)`, I think it should use `./test_b/tokenization.py`, which contains param `b` instead `a`

```text
model_a
test_a tokenizer init
TestTokenizer(name_or_path='./test_a/', vocab_size=1, model_max_length=1000000000000000019884624838656, is_fast=False, padding_side='right', truncation_side='right', special_tokens={}, clean_up_tokenization_spaces=True)
Traceback (most recent call last):
  File "/Users/tongyifan/PycharmProjects/transformers_issue_25194/main.py", line 8, in <module>
    b = AutoTokenizer.from_pretrained("./test_b/", trust_remote_code=True)
  File "/Users/tongyifan/PycharmProjects/transformers_issue_25194/venv/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 689, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/Users/tongyifan/PycharmProjects/transformers_issue_25194/venv/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1841, in from_pretrained
    return cls._from_pretrained(
  File "/Users/tongyifan/PycharmProjects/transformers_issue_25194/venv/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2004, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
TypeError: __init__() missing 1 required positional argument: 'a'
```

There is a conflict in the names of your tokenizer files basically. This is feature is called "code on the Hub" for a reason, it doesn't work as well for local models ;-) 
Basically you need to name the two tokenization files differently when coding locally like this.

Thanks for your reply!