potential mismatch between `save_pretrained` and `from_pretrained` for `AutoTokenizer`
Author: sosuneko
Pull request: no
Labels: name: WIP

## Environment info

- `transformers` version: 4.5.1
- Platform: Linux-4.15.0-48-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.3
- PyTorch version (GPU?): 1.6.0+cu101 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


### Who can help

Since it is related to tokenizer, I guess @LysandreJik could help to have a look.


## Information

Model I am using (Bert, XLNet ...): `AutoTokenizer`

I tried to save the official pretrained tokenizer `bert-base-multilingual-cased` into a local folder so I can load it from this folder next time.

I got the official `bert-base-multilingual-cased` tokenizer:
```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
    pretrained_model_name_or_path='bert-base-multilingual-cased',
    use_fast=True,
)
```
The tokenizer looks like this:
```
PreTrainedTokenizerFast(name_or_path='bert-base-multilingual-cased', vocab_size=119547, model_max_len=512, is_fast=True, padding_side='right', special_tokens={'unk_token': '[UNK]', 'sep_token': '[SEP]', 'pad_token': '[PAD]', 'cls_token': '[CLS]', 'mask_token': '[MASK]'})
```

Tried to save it locally:
```python
tokenizer.save_pretrained('bert-base-multilingual-cased-good')
```
The folder structure after saving:
```
bert-base-multilingual-cased-good/
├── special_tokens_map.json
├── tokenizer_config.json
└── vocab.txt
```

However, when I tried to load the tokenizer from the local folder, I got this error:
```python
fail_tokenizer = AutoTokenizer.from_pretrained(
    pretrained_model_name_or_path='bert-base-multilingual-cased-good',
    use_fast=True,
)
```
error message:
```
OSError: Can't load config for 'bert-base-multilingual-cased-good'. Make sure that:

- 'bert-base-multilingual-cased-good' is a correct model identifier listed on 'https://huggingface.co/models'
- or 'bert-base-multilingual-cased-good' is the correct path to a directory containing a config.json file
```
Seems the code was looking for `config.json` under this folder. :thinking: 
According to the codes below, I guess the assumption should be correct:
- https://github.com/huggingface/transformers/blob/04ab2ca639ee6fd1002ce0a498d892245f0c9093/src/transformers/configuration_utils.py#L456
- https://github.com/huggingface/transformers/blob/04ab2ca639ee6fd1002ce0a498d892245f0c9093/src/transformers/file_utils.py#L218

After googling, I found an answer from stackoverflow that we could just rename the `tokenizer_config.json` as `config.json`. That sounded nice, so I just tried it, now my folder structure became: (I use a new folder `bert-base-multilingual-cased-bad` just for showcase)
```
bert-base-multilingual-cased-bad
├── config.json
├── special_tokens_map.json
└── vocab.txt
```
Then I tried to load from it:
```python
bad_tokenizer = AutoTokenizer.from_pretrained(
    pretrained_model_name_or_path='bert-base-multilingual-cased-bad',
    use_fast=True,
)
```
It works! :100: However, now the tokenizer became very weird:
```
PreTrainedTokenizerFast(name_or_path='bert-base-multilingual-cased-bad', vocab_size=119547, model_max_len=1000000000000000019884624838656, is_fast=True, padding_side='right', special_tokens={'unk_token': '[UNK]', 'sep_token': '[SEP]', 'pad_token': '[PAD]', 'cls_token': '[CLS]', 'mask_token': '[MASK]'})
```
Well, the `model_max_len` became `1000000000000000019884624838656` instead of `512`. After further checking other attributes that were included in the original `tokenizer_config.json`, I found even `do_lower_case` now becomes `True`, scary... :fearful: 

I bet the `bad_tokenizer` didn't take into account the settings from `config.json`, it might just initialize with the default settings instead. So I tried to copy `config.json` and renamed it as `tokenizer_config.json`, now my folder structure looks like this: (I got both `config.json` and `tokenizer_config.json` although they are just the same file with different names)
```
bert-base-multilingual-cased-work
├── config.json
├── special_tokens_map.json
├── tokenizer_config.json
└── vocab.txt
```
Although it looks weird, it works as expected: (I use a new folder `bert-base-multilingual-cased-work` just for showcase)
```python
work_tokenizer = AutoTokenizer.from_pretrained(
    pretrained_model_name_or_path='bert-base-multilingual-cased-work',
    use_fast=True,
)
```
Now the tokenizer seems fine: 
```
PreTrainedTokenizerFast(name_or_path='bert-base-multilingual-cased-work', vocab_size=119547, model_max_len=512, is_fast=True, padding_side='right', special_tokens={'unk_token': '[UNK]', 'sep_token': '[SEP]', 'pad_token': '[PAD]', 'cls_token': '[CLS]', 'mask_token': '[MASK]'})
```

### Why I think this could be severe?
Initially, I just renamed `tokenizer_config.json` as `config.json` and deployed my NLP model (that's why I need to save the tokenizer locally) without checking the tokenizer in detail. However, I found the results from the deployed model didn't match with the local ones. So I did some investigation and found that this issue. 
If you also encountered similar issues, maybe you can try to check your tokenizer.

I think it doesn't make sense that the code looks for `config.json` in function `from_pretrained` while there is no `config.json` generated when function `save_pretrained` is called (https://github.com/huggingface/transformers/blob/04ab2ca639ee6fd1002ce0a498d892245f0c9093/src/transformers/tokenization_utils_base.py#L110). It's better to unify the config naming system, otherwise, it might cause severe issues that are very hard to spot. 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Faced with the same issue

Hi! This is close to something @SaulLu is working on. Thanks for opening this issue, we're keeping an eye on it and will resolve it.

> Hi! This is close to something @SaulLu is working on. Thanks for opening this issue, we're keeping an eye on it and will resolve it.

Thank you! 🤩

This is fixed on main! Closing 😉 

This issue still persists.
Downloaded files from https://huggingface.co/bert-base-multilingual-cased/tree/main and tried to create tokenizer using from_pretrained from local files

<img width="947" alt="Screenshot 2023-06-01 at 17 26 29" src="https://github.com/huggingface/transformers/assets/22324507/943d1b83-8801-4b61-8bd0-ae7ef639a926">


Hey! 
If you issue is about potential missmatch (but not about the files) please open a seperate issue with a reproducing script so that we can help you.