fix: Fix incorrent loading config in AutoTokenizer.from_pretrained
Author: zsaladin
Pull request: yes

# What does this PR do?
The description in argument `subfolder` of `AutoTokenizer.from_pretrained` doesn't work properly. 
> subfolder (str, optional) — In case the relevant files are located inside a subfolder of the model repo on huggingface.co (e.g. for facebook/rag-token-base), specify it here.

The code the error occurs
```py
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("facebook/rag-token-base", subfolder="generator_tokenizer")
```

The error message
```
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Traceback (most recent call last):
  File "/Users/daehee/Workspace/Projects/transformers/main.py", line 4, in <module>
    tokenizer = AutoTokenizer.from_pretrained("facebook/rag-token-base", subfolder="generator_tokenizer")
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daehee/Workspace/Projects/transformers/src/transformers/models/auto/tokenization_auto.py", line 657, in from_pretrained
    config = AutoConfig.from_pretrained(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daehee/Workspace/Projects/transformers/src/transformers/models/auto/configuration_auto.py", line 922, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daehee/Workspace/Projects/transformers/src/transformers/configuration_utils.py", line 574, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daehee/Workspace/Projects/transformers/src/transformers/configuration_utils.py", line 629, in _get_config_dict
    resolved_config_file = cached_file(
                           ^^^^^^^^^^^^
  File "/Users/daehee/Workspace/Projects/transformers/src/transformers/utils/hub.py", line 404, in cached_file
    raise EnvironmentError(f"Could not locate {full_filename} inside {path_or_repo_id}.")
OSError: Could not locate generator_tokenizer/config.json inside facebook/rag-token-base.
```

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23098). All of your documentation changes will be reflected on that endpoint.

No this fix is incorrect. The problem lies in the checkpoint you are using, which does not specify the tokenizer class in the `tokenizer_config.json` present in the subfolder. If this was done properly, this path would not be executed.

The current function finds `config.json` in subfolder not root if tokenizer class in `tokenizer_config.json` is not specified. So `AutoTokenizer.from_pretrained("facebook/rag-token-base", subfolder="generator_tokenizer")` fails since [facebook/rag-token-base/generator_tokenizer](https://huggingface.co/facebook/rag-token-base/tree/main/generator_tokenizer) doesn't have `config.json`.

This PR makes the function find `config.json` in root not subfolder to run the example code described in the document. Or 
`config.json` has to be put in [facebook/rag-token-base/generator_tokenizer](https://huggingface.co/facebook/rag-token-base/tree/main/generator_tokenizer)?

As I said above, the function should not even attempt to find a `config.json`. It only does so because the tokenizer config is not right.

Then there are several problems in this.

1. Docs of [AutoTokenizer.from_pretrained](https://huggingface.co/docs/transformers/v4.28.1/en/model_doc/auto#transformers.AutoTokenizer.from_pretrained) and [PreTrainedTokenizerBase.from_pretrained](https://huggingface.co/docs/transformers/v4.28.1/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.from_pretrained) should be fixed. `subfolder` in the docs is decsribed with incorrect example [facebook/rag-token-base](https://huggingface.co/facebook/rag-token-base/tree/main/generator_tokenizer). It may cause confusion.

2. Correct error message has to be shown like "tokenizer class is not specified in tokenizer_config.json" instead of finding `config.json`

Do I understand your comment properly?

No, there is one problem and it is that the tokenizer config file in that repo is wrong and should be fixed. There is nothing to change after that.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.