Use real tokenizers if tiny version(s) creation has issue(s)
Author: ydshieh
Pull request: yes

# What does this PR do?

When creating tiny versions of a model loaded with real checkpoints, we might have some trouble converting the tokenizer(s) to the tiny version. One situation is: **we have a fast tokenizer with its vocabulary shrunk then saved to a local path, but the slow tokenizer loaded from this path have the original (large) vocabulary size.** This happens for at least `XLNet` and `MBart`. See the code snippet at the end.

As discussed offline, **we decide to keep the original tokenizer(s) in such problematic situation(s)**. See the quote at the end. This PR therefore implements this logic. Some existing tiny models on the Hub have been updated.

### Code snippet (to show one of the problematic situations)

```python
from transformers import MBartTokenizer, MBartTokenizerFast
from datasets import load_dataset

TARGET_VOCAB_SIZE = 1024

ds = load_dataset("wikitext", "wikitext-2-raw-v1")
training_ds = ds["train"]
testing_ds = ds["test"]

# ckpt = "hf-internal-testing/tiny-random-MBartModel"
ckpt = "facebook/mbart-large-cc25"

original_fast_tokenizer = MBartTokenizerFast.from_pretrained(ckpt)
new_fast_tokenizer = original_fast_tokenizer.train_new_from_iterator(training_ds["text"], TARGET_VOCAB_SIZE, show_progress=False)

local_ckpt = "tiny-random-MBartModel"
new_fast_tokenizer.save_pretrained(local_ckpt)
new_slow_tokenizer = MBartTokenizer.from_pretrained(local_ckpt)

print(f"original_fast_tokenizer.vocab_size: {original_fast_tokenizer.vocab_size}")
print(f"new_fast_tokenizer.vocab_size: {new_fast_tokenizer.vocab_size}")
print(f"new_slow_tokenizer.vocab_size: {new_slow_tokenizer.vocab_size}")
```
This gives:
```bash
original_fast_tokenizer.vocab_size: 250027
new_fast_tokenizer.vocab_size: 1054
loaded_slow_tokenizer.vocab_size: 250027
```


### Offline discussion

> **(from Lysandre)** From a development perspective it seems to me that having the highest number of use-cases available from tiny models would be optimal, even if we sacrifice performance. So in that case, if we would need to choose between "Lightweight tokenizer, but only the fast tokenizer file" and "Normal tokenizer, with both fast and slow files", then I would personally choose to go with the latter even if it's heavier. I'd aim for both files to be the same though, so both files as heavy, rather than one small and one large (otherwise we run in the discrepancies that Nico was describing).
Going with the first one means that we might run in situations where we want to test the slow file but we can't for some specific models (leading to model-specific code), meaning that we have to find a workaround for something that would have otherwise worked.
(This wouldn't be necessary if we were trying to have 100% parity between slow and fast, but that's not the case, so keeping slow files is all the more relevant, imo)
However I don't feel strongly (and I won't be the one fixing tests/issues :grin:) so if you all think it'd be better to go with just the fast file in these use-cases, then I'll trust you with it.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Since this PR also creats more slow tokenizers (the real one), we get more test failures for some model types. I will have to deal with them (by skipping them first)

Ready to be reviewed 🚀 !

I'm not sure I fully understand if this PR is desirable: if we push the original tokenizer on the Hub, we end up with a model that won't be tiny anymore since it will have a very big vocab size (most tokenizers have a vocab of 30k/50k tokens...) so then all the pipeline tests for this model will become significantly slower.

I think it's best to not push the tokenizer built automatically in those cases and either try to build one manually or fix the conversion script so the issue does not appear or just not test the last models for which we can't get a small tokenizer if they are not heavily downloaded.

> won't be tiny anymore since it will have a very big vocab size

Yes, this is true. However, it's not as large as the original model(s) in terms of number of layers, hidden dimension etc.
The embedding matrix will be large, but getting the embeddings from input ids should be fast. The only part that will be slow down is the logits computation at the end.

I can perform some measurement of running time.

>  build one manually or fix the conversion script so the issue 

In some cases, there is no way to fix: the example I showed in the PR description involves sentencepiece based tokenizers for which we are not going to get the slow tokenizer from the (converted) fast tokenizer. (not an expert here, but that's what I understand/hear).

Regarding building one manually - one disadvantage is that if we ever find another new issue for that tokenizer, we will have to adding some fix(es) to that manual process. This brings some questions about: who develop that manual way for that tokenizer, where to save that manual process (in another script[s]?) etc.


Also see the end in the PR description (**Offline discussion**) if that part is skipped.

Thanks for the context. I looked at an extreme case with the "tiny" transfo-xl models (vocab size of 225k) and it's true the final model stays under a reasonable size and goes fast.

Thank you for checking on your side. I was running it too but you already done checking and approved 😃