Use tiny models for ONNX tests
Author: lewtun
Pull request: yes
Labels: name: WIP

# What does this PR do?

Uses tiny random models for the ONNX tests to speed up the test suite. Closes #18819 

## TODO

- [x] Add tiny model for `deepmind/language-perceiver`
- [x] Add tiny model for `deepmind/vision-perceiver-conv`
- [x] Add tiny model for `hustvl/yolos-tiny`
- [x] Add tiny model for  `nvidia/segformer-b0-finetuned-ade-512-512`
- [x] Add tiny model for `google/long-t5-local-base`
- [ ] Ensure slow tests pass

### Hub PRs that need merging to ensure slow test pass

- [x] https://huggingface.co/hf-internal-testing/tiny-random-beit/discussions/1
- [x] https://huggingface.co/hf-internal-testing/tiny-random-deit/discussions/1
- [x] https://huggingface.co/hf-internal-testing/tiny-random-deit/discussions/2
- [x] https://huggingface.co/hf-internal-testing/tiny-random-clip/discussions/1
- [ ] https://huggingface.co/hf-internal-testing/tiny-random-clip/discussions/2
- [x] https://huggingface.co/hf-internal-testing/tiny-random-convbert/discussions/1
- [x] https://huggingface.co/hf-internal-testing/tiny-random-xlm-roberta/discussions/1
- [x] https://huggingface.co/hf-internal-testing/tiny-random-xlm-roberta/discussions/2
- [x] https://huggingface.co/hf-internal-testing/tiny-random-ibert/discussions/1
- [x] https://huggingface.co/hf-internal-testing/tiny-random-ibert/discussions/2
- [x] https://huggingface.co/hf-internal-testing/tiny-random-blenderbot-small/discussions/1
- [x] https://huggingface.co/hf-internal-testing/tiny-random-blenderbot-small/discussions/2
- [x] https://huggingface.co/hf-internal-testing/tiny-random-mt5/discussions/1
- [x] https://huggingface.co/hf-internal-testing/tiny-random-mt5/discussions/2

Comments:
Thanks, @lewtun 

Let's run the scheduled CI manually (for ONNX tests) before merge :-)

> Thanks, @lewtun
> 
> Let's run the scheduled CI (for ONNX tests) before merge :-)

Yes, this is still WIP because I discovered some slow tests fail with the new tiny models. Will debug and fix on the model side where necessary :)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18924). All of your documentation changes will be reflected on that endpoint.

I left a few comments on Hub PRs. In general, we also like to have small vocab size. But if the current issues persist, no problem for  me to use the tokenizers config or files from the original model checkpoint.

> I left a few comments on Hub PRs. In general, we also like to have small vocab size. But if the current issues persist, no problem for me to use the tokenizers config or files from the original model checkpoint.

Thanks! 

As discussed offline, using a small vocab size `v` in the model config requires that `len(tokenizer) == v`. Otherwise the model cannot run inference because the tokenizer will generate out of vocab input IDs and throw an `index out of range` error.

AFAIK the only way to handle this is to train a tokenizer from scratch on a tiny "corpus" and use the resulting vocab size in the model config. This is simple for fast tokenizers, but somewhat painful for slow ones that don't have the `train_from_iterator()` method. 

In the end it may not be entirely necessary to optimise the model size this way if the resulting "tiny" models are fast enough for our test suite

As discussed internally, we'll revert the changes to the model repos and create dedicated `tiny-random-onnx-x` repos for the ONNX tests

Stable bot begone!

@gante told me we can use `wip` label to avoid this bot 😃 

Closing in favour of https://github.com/huggingface/transformers/pull/20333