AutoTokenizer - add `from_model_name` method
Author: patrickvonplaten
Pull request: yes

This PR adds a new `.from_model_name(...)` class method to `AutoTokenizer`. 

In some example scripts we would like to train the tokenizer from scratch or create it from scratch - e.g. see: https://github.com/huggingface/transformers/pull/13620. Depending on which model is used the corresponding tokenizer class should be loaded. I think the cleanest way to pick the corresponding tokenizer class is to go over the **model names**, such as `"bert"`, `"gpt2"` etc...

Comments:
> ```python
> tokenizer = AutoTokenizer.from_config(config)
> ```

Hmm, I think `from_model_name(....)` is the better option here because:

- The use cases of `AutoTokenizerr.from_config(...)` was mainly intended for example scripts where the tokenizer is trained/created in the script itself. In such a case, we can't leverage the model config since it's the creation of the tokenizer that defines `vocab_size`, `pad_token_id`, .... => e.g. I would like to enable to create a generic ASR tokenizer in such a script: https://github.com/huggingface/transformers/blob/da8beaaf762c0ea4eecb150039be63949fe5cf94/examples/research_projects/wav2vec2/run_common_voice.py#L358. Note that in this script the vocab file is created in the example. 
- We can't do `AutoTokenizer.from_config(...)` as we also will have to pass a vocab_file, merges file, etc... -> so the function cannot have the same API as `AutoModel`
- I think we were planning on seperating the model config and the tokenizer config instead of having a single config => I think it can be confusing then to do `AutoTokenizer.from_config(...)` as one doesn't know whether a tokenizer config or model config should be passed here. Essentially I was aiming for such an API:


```python
config = AutoConfig.from_pretrained("facebook/wav2vec2-pretrained")  # <- pretrained speech models have no information about vocab size, etc.... yet

# ... create tokenizer vocab files

tokenizer = AutoTokenizer.from_model_name(config.model_type, vocab_file="path/to/vocab/file")

config["vocab_size"] = len(tokenizer)

model = AutoModelForCTC(config)  # <- now create model
```

I haven't thought too much about the API for the "NLP" tokenizers in this case, but couldn't one use `tokenizers` to train a tokenizer and then load it into a `transformers` tokenizer class via `AutoTokenizer.from_model_name("roberta", tokenizer_file="path/to/tokenizer")`? Or does the API doesn't make much sense here?

Keen to hear your thoughts here @LysandreJik @SaulLu 

I understand your points. I agree that it is confusing to have `AutoTokenizer.from_config(...)` rely on the model configuration.


A last comment: `from_model_name` implies that a tokenizer is 1-to-1 linked with a model, but that's not the case. For example, what of BERTweet, which has a `BertweetTokenizer` but its model is a RoBERTa, or GPT-Neo/GPT-J/iBERT that use a `GPT2Tokenizer` under the hood? I'd opt for having `from_name`, or `from_type` if the goal is to always leverage `config.model_type` instead.

After talking to Lysandre offline a better solution might actually be to leverage the `AutoTokenizer.from_pretrained(...)` method directly. 

The main reason for this PR is to make **fine-tuning of pretrained speech recognition models easier**. I want to be able to write a fine-tuning script just by using `Auto....` classes and no `Wav2Vec2...`.

To give a bit more context: 

In speech recognition, a pretrained speech model (Wav2Vec2, HuBERT) has the following files defined after pretraining:

1) the **pretrained model**, which can be loaded with `AutoModelForCTC.from_pretrained(...)`
2) the **feature extractor**, which can be loaded with `AutoFeatureExtractor.from_pretrained(...)` 
3) the **configuration file**, which needs to be adapted before being used to load the model

**Note** that no tokenizer is needed for pretraining and has to be defined on-the-fly for fine-tuning.

In fine-tuning the workflow is then as follows:

```python
# 1. load the config
config = AutoConfig.from_pretrained("facebook/wav2vec2-base")

# 2. create the vocabulary
characters = training_data["unique_chars"]
vocab_dict = {v, k for k,v in enumerate(characters)}

# 3. save the config locally to be able to create the tokenzier
with open("vocab.json", "w") as f:
     f.write(json.dump(vocab_dict))

# 4. IMPORTANT: Now I want to instantiate a new tokenizer using the just created `"vocab.json"` and passing the model type.
# The tokenizer will be uploaded to the model repo after fine-tuning. that will be uploaded to the new model that uses `vocab.json`

# <- so possible solutions to this are
# 1. AutoTokenizer.from_name(config.model_type)
# 2. AutoTokenizer.from_config(config)
# 3. AutoTokenizer.from_pretrained("./", config=config)
# 4. AutoTokenizer.from_pretrained("./", model_type=config.model_type)
```

Case # 1. was the proposal of this PR. 
I'm not a fan of # 2. as it's not clear to me whether `config` is a tokenizer config or a model config
Case # 3. actually already works out of the box and wouldn't require any changes. Given that the API is already there I think it's fine to use it. However, here I'm also a bit worried that people will assume that the `config` will actually overwrite also attributes in the tokenizer config. E.g. I could see people setting the `pad_token_id`, `vocab_size` in the config and then assume the tokenizer will have the same `vocab_size` and `pad_token_id` after calling `AutoTokenizer.from_pretrained("./", config=config)`
Case # 4. would require some minimal changes to `AutoTokenizer` (just adding a kwargs.pop("model_type")) essentially


=>  Having talked to @LysandreJik a bit more about it, I think it does indeed not necessary make sense to create a whole new API of the `AutoTokenizer` and that it should be enough to just use the already existing `.from_pretrained(...)`. 

Also it seems like in the case of training a tokenizer from scratch, one would simply always make use of `PreTrainedTokenizer(...)` instead of using the `AutoTokenizer` class. 

@SaulLu - I very much agree that we should not assume that the user knows how to spell the model name. Rather, it should be extracted from the configuration file.

I would be in favor of Case #4 as I think it's the cleanest approach, but given the API for Case #3 already exist I guess I would also be fine with using this one. Still think it's quite confusing though to pass the model config to the AutoTokenizer taking into consideration that we have a tokenizer config with `tokenizer_config.json`... Think `AutoTokenizer.from_pretrained(..., config=config)` is also not really used, no?

What do you guys think?

(Sorry for the long discussions here!) 

Thanks for the write-up @patrickvonplaten. I think case 4 is the cleanest approach too, even if I still dislike `model_type` - how about `tokenizer_type`?

This is a nitpick though, happy to go with either.

I don't mind adding the API for case 4, but I share Lysandre's view on the name of the argument. `tokenizer_type` would be better indeed.

Superseded by #13668