Can't load tokenizer for 'microsoft/wavlm-base' when using Wav2Vec2Processor as in docs
Author: pszemraj
Pull request: no

# overview

Thanks for adding `WavLM` guys! I wanted to try it out and ran into some issues which I am reporting (hopefully clearly) here.

the transformers docs say to use `Wav2Vec2Processor` as the processor/tokenizer for `WavLM`. Transformers 4.15.0 tells me it can't find a suitable tokenizer when running the following line `processor = Wav2Vec2Processor.from_pretrained('microsoft/wavlm-base')` verbatim from the [docs](https://huggingface.co/docs/transformers/model_doc/wavlm#transformers.WavLMForCTC). I received the error originally when trying to add to [a branch in my repo](https://github.com/pszemraj/vid2cleantxt/tree/model-updates) and replicated the issue [on Google Colab](https://colab.research.google.com/gist/pszemraj/ec39b013e14e5ec6ddcda74cc1741edb/transformers-4-15-0-wavlm-test.ipynb).

## Environment info


- `transformers` version: 4.15.0
- Platform: Linux-5.4.144+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.12
- PyTorch version (GPU?): 1.10.0+cu111 (False)
- Tensorflow version (GPU?): 2.7.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No
- 

### Who can help

@patrickvonplaten , @anton-l, @sgugger 

## Information

I am using `wavLM` which seems to inherit some things from `wav2vec2`.

The problem arises when using:
* [x] the official example scripts: (give details below)


The tasks I am working on is:

automatic speech recognition - ASR

## To reproduce

Steps to reproduce the behavior:

- run the code listed under [WavLMForCTC](https://huggingface.co/docs/transformers/model_doc/wavlm#transformers.WavLMForCTC) after a clean install.
- this is replicated in a colab gist [here](https://colab.research.google.com/gist/pszemraj/ec39b013e14e5ec6ddcda74cc1741edb/transformers-4-15-0-wavlm-test.ipynb)

### copy paste of error

in case this saves time:

```
OSError: Can't load tokenizer for 'microsoft/wavlm-base'. Make sure that:

- 'microsoft/wavlm-base' is a correct model identifier listed on 'https://huggingface.co/models'
  (make sure 'microsoft/wavlm-base' is not a path to a local directory with something else, in that case)

- or 'microsoft/wavlm-base' is the correct path to a directory containing relevant tokenizer files



During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py in from_pretrained(cls, pretrained_model_name_or_path, *init_inputs, **kwargs)
   1734                 msg += f"- or '{revision}' is a valid git identifier (branch name, a tag name, or a commit id) that exists for this model name as listed on its model page on 'https://huggingface.co/models'\n\n"
   1735 
-> 1736             raise EnvironmentError(msg)
   1737 
   1738         for file_id, file_path in vocab_files.items():

OSError: Can't load tokenizer for 'microsoft/wavlm-base'. Make sure that:

- 'microsoft/wavlm-base' is a correct model identifier listed on 'https://huggingface.co/models'
  (make sure 'microsoft/wavlm-base' is not a path to a local directory with something else, in that case)

- or 'microsoft/wavlm-base' is the correct path to a directory containing relevant tokenizer files

```

## Expected behavior

I want it to load wavLM and transcribe spoken audio. 

---

thanks for all your work and let me know if I can provide any more info!


Comments:
Hey @pszemraj,

I'm sorry the documentation is really bad here :-/ I'll fix this asap. 

`wavlm-base` is just the pretrained model of wavlm. It has no character predicition (CTC) head and therefore it also cannot have a tokenizer. 

If you want to try out a fine-tuned wavlm checkpoint you could try out one of those:
https://huggingface.co/models?other=wavlm_libri_finetune 

*e.g.*

```python
>>> from transformers import Wav2Vec2Processor, WavLMForCTC
>>> from datasets import load_dataset
>>> import torch

>>> dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation")
>>> sampling_rate = dataset.features["audio"].sampling_rate

>>> processor = Wav2Vec2Processor.from_pretrained('patrickvonplaten/wavlm-libri-clean-100h-base-plus')
>>> model = WavLMForCTC.from_pretrained('patrickvonplaten/wavlm-libri-clean-100h-base-plus')

>>> # audio file is decoded on the fly
>>> inputs = processor(dataset[0]["audio"]["array"], sampling_rate=sampling_rate, return_tensors="pt")
>>> logits = model(**inputs).logits
>>> predicted_ids = torch.argmax(logits, dim=-1)

>>> # transcribe speech
>>> transcription = processor.batch_decode(predicted_ids)

>>> # compute loss
>>> with processor.as_target_processor():
...     inputs["labels"] = processor(dataset[0]["text"], return_tensors="pt").input_ids

>>> loss = model(**inputs).loss
```

@anton-l @sgugger @LysandreJik @NielsRogge  - besides the bad documentation on my part, this error seems to happen quite a bit:
- https://github.com/microsoft/UniSpeech/issues/15
- https://github.com/huggingface/transformers/issues/14214

I suspect the general problem to be a bit as follows. In NLP, `transformers` users know that no matter what "head" of a model is chosen (whether it's `BertModel`, `BertForPreTraining` or `BertForMaskedLM`) - one always needs a tokenizer. So for every BERT repo the following works:
`BertTokenizer.from_pretrained(...)`.

Now the problem for speech is that actually only the `...ForCTC` head requires **both** a feature extractor and a tokenizer, all other classes `...ForPreTraining`, `...Model`, etc... **only** need the feature extractor and calling `Wav2Vec2Processor` on any class other than `...ForCTC` actually fails. `....ForCTC` is however the most used class, so it's not an edge case.

This doesn't seem to be a good solution for now - it's not intuitive. The reason why `Wav2Vec2Processor` was added back then was mainly so that the API fits better with the general `transformers` API which was always 1 model object and 1 processor object for nlp. To force ASR to also have 1 model object and 1 processor object (instead of 2 - being tokenizer and feature extractor), `Wav2Vec2Processor` was created. Besides all the problems with `Wav2Vec2Processor` (see discussion here: https://github.com/huggingface/transformers/pull/14881), I still think forcing a 1 model, 1 processor API is a very strong argument to have a `Wav2Vec2Processor` class and it's probably difficult to remove it now anyways. 

I see two options here: 
1) We allow to load a `Wav2Vec2Processor` from a repo without a tokenizer and only throw an error when `batch_decode` or `decode` is called. This way, people can happily use all wav2vec2 models with the `Wav2Vec2Processor` and we can throw a nice error message when users try to transcribe with pretrained models only. The problem here is that it defeats a bit the purpose of having an independent `Wav2Vec2FeatureExtractor` class as one could just always use `Wav2Vec2Processor`....

2) We make it somehow crystal clear that only fine-tuned Wav2Vec2 speech recognition modes should use `Wav2Vec2Processor` and that all other Wav2Vec2 models should use the `...FeatureExtractor` object. The big drawback is here is obviously that users have to remember when to use `...Processor` and when to use `...FeatureExtractor` which is not ideal IMO

So I tend to 1). 

Curious to hear if you guys have other (better) ideas


BTW, I can very well see the same problem for vision where one model type, *e.g.* ViT only needs a feature extractor for vision classification, but a feature extractor and tokenizer for image captioning 

@patrickvonplaten I agree with option 1, it would significantly streamline our examples and tests.
Also, this PR that I drafted a while back could add some clarity (gotta change other loaders for consistency to merge it): https://github.com/huggingface/transformers/pull/14519

For me there are two separate issues:
- the detailed doc of the model is using the wrong class:
  For a `BertModel` we show the use of a `BertTokenizer`, not `AutoTokenizer`. So here the code samples should use the appropriate class. The `XxxFeatureExtractor` when the model only has one modality, and the `XxxProcessor` when it has two

- the user should be able to use the same `AutoClass` for all the models of an architecture, which corresponds to your option 1.
  I agree this require users to be able to load a processor without a tokenizer, and it's fine to enable this when some of the models can function without it.

@patrickvonplaten chiming in on the specifics: I confirm that 
`processor = Wav2Vec2Processor.from_pretrained('patrickvonplaten/wavlm-libri-clean-100h-base-plus')` 
(also tried the `large`) works, thanks for the heads up. 

FWIW as a user I think option 1 integrates into my workflow(s) / testing more intuitively. 

Thanks again + happy holidays!