:rotating_light: :rotating_light: Setup -> setupclass conversion
Author: Rocketknight1
Pull request: yes

This PR tries replacing expensive `setUp()` methods, which are run once per test, with `setUpClass()` methods that are run only once for the whole suite. It should improve test times and reduce instances of connection failures / DOSing the Hub.

This seems to reduce test runtime a lot for me, but there's a risk that it will cause issues in the slow nightly CI, so we shouldn't merge it before a major release and we should watch for issues.

Comments:
Simple error

request a review from me once ready 🙏 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37282). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @ydshieh sorry for the ping earlier, this should be ready for review now!

@ydshieh according to the [unittest docs](https://docs.python.org/3/library/unittest.html#class-and-module-fixtures): 

> When the test suite encounters a test from a new class then tearDownClass() from the previous class (if there is one) is called, followed by setUpClass() from the new class.

and

> Skipped classes will not have [setUpClass()](https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUpClass) or [tearDownClass()](https://docs.python.org/3/library/unittest.html#unittest.TestCase.tearDownClass) run. Skipped modules will not have setUpModule() or tearDownModule() run.

I think this means that we can add `setUpClass()` and `tearDownClass()`, and they will not run unless a test in that class runs. Therefore, they will be ignored for slow tests in the live CI (but will run for slow tests in the nightly CI).


> followed by setUpClass() from the new class.



> @ydshieh according to the [unittest docs](https://docs.python.org/3/library/unittest.html#class-and-module-fixtures):
> 
> > When the test suite encounters a test from a new class then tearDownClass() from the previous class (if there is one) is called, followed by setUpClass() from the new class.
> 
> and
> 
> > Skipped classes will not have [setUpClass()](https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUpClass) or [tearDownClass()](https://docs.python.org/3/library/unittest.html#unittest.TestCase.tearDownClass) run. Skipped modules will not have setUpModule() or tearDownModule() run.
> 
> I think this means that we can add `setUpClass()` and `tearDownClass()`, and they will not run unless a test in that class runs. Therefore, they will be ignored for slow tests in the live CI (but will run for slow tests in the nightly CI).

I actually ran

tests/models/qwen2_vl/test_modeling_qwen2_vl.py

before posting the comment, and despite all tests are being decorated as @slow, the `setupClass` is still called.

But you are right here, it's because the class `Qwen2VLIntegrationTest` itself is not decorated as slow: only tests are decorated. That is why the `setupClass` is still running.

But this means we have to double check the integration tests you have changed to make sure this won't happens. At this moment, `Qwen2VLIntegrationTest.setupClass` is called on CircleCI.

Another point that keep me not to do it is: if we create the objects in `setupClass`, there is a risk of some tests modifying it. In tokenization tests, I have to use `copy.deepcopy`

https://github.com/huggingface/transformers/blob/ba859fe83df355952c1d8383fd6cfb46f420377e/tests/test_tokenization_common.py#L92

to avoid such cases (it happens!).

During the review, I can't really be sure that we don't have such race condition however.

Maybe create a branch based on this one, keep the changes in Integration tests in that branch so you can create a new PR later.
Then in this branch, just keep the changes that are not belonging to integration tests?

cc @ydshieh I removed the changes from `IntegrationTest` classes, so hopefully this PR should be cleaner and less risky now!

Feel free  to merge once CI is ✅ . My concerns are all addressed. My last nit is simply a comment (but yeah you went forward 👍 )

Hope we see even reduced Hub failure!

Merged!