Fix missing test in `torch_job`
Author: ydshieh
Pull request: yes

# What does this PR do?

Currently we have

```
@pytest.mark.generate
class GenerationTesterMixin:
```
and
```
class Mamba2ModelTest(ModelTesterMixin, GenerationTesterMixin, PipelineTesterMixin, unittest.TestCase)
```
(or any model test class)
plus
```
torch_job = CircleCIJob(
    "torch",
    docker_image=[{"image": "huggingface/transformers-torch-light"}],
    marker="not generate",
    parallelism=6,
    pytest_num_workers=8
)
```
in CircleCI config.


So `torch_job` won't run tests which is marked as `generate`, which are *all* tests as any model test class inherits from  `GenerationTesterMixin`.

This PR fixes it

Comments:
cc @ArthurZucker for reference

(the failing test seems related to #33533 cc @zucchini-nlp )

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33593). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Those are a bit flaky, in no-cache settings. Since the weights are random, we can generate image tokens (it's not oov anymore) and then at some point fail to get enough image embeddings. Do you think we should overwrite those for VLMs for be always with cache? @gante 

imo, not a big deal, for me it never failed locally until I got to CI runs

TL;DR
- @ydshieh the failing test is flaky, I've retriggered the job to make the CI green. No success, I think I need to fix the flakiness first 👀 (in `tests/models/pegasus/test_modeling_pegasus.py::PegasusStandaloneDecoderModelTest::test_generate_from_inputs_embeds_decoder_only`)
- @zucchini-nlp a short-term patch is 100% needed, I can trigger the error locally with `CUDA_LAUNCH_BLOCKING=1 py.test tests/models/video_llava/test_modeling_video_llava.py::VideoLlavaForConditionalGenerationModelTest::test_sample_generate_dict_output --flake-finder --flake-runs=10`. If we don't patch it, we will have many red CIs out there. I'd suggest whichever solution is the quickest to implement, since the actual fix has longer dependencies (see below) :D
EDIT: see https://github.com/huggingface/transformers/pull/33623 

_________________________________________________

@zucchini-nlp If I got it right, the error is caused by a generation-time behavior that doesn't exist in pre-trained models. This reminds me of Whisper, which has a bunch of `PreTrainedConfig` fields to ensure certain tokens are never generated out of position. They are `PreTrainedConfig` fields, and not `GenerationConfig` fields, because `GenerationConfig` didn't exist back then.

The correct long-term fix should then be to parameterize the model (as in the model class, not the tester) to have a `GenerationConfig` such that the bad generation behavior never happens -- in the test and outside it. This is also related to a chat I had with @Cyrilvallez today, where we identified that a model should be able to specify its own default cache class (and, because caching is a property of `generate`, it belongs in `GenerationConfig`). However, we can't define a default `GenerationConfig` for a model at the moment! I will add to my tasks adding this functionality, so that we can start parameterizing a default `GenerationConfig` for each model, instead of relying exclusively on `PreTrainedConfig` to set the model default behavior.



@ydshieh #33602 should fix one of the tests frequently flaking in the CI runs (`tests/models/pegasus/test_modeling_pegasus.py::PegasusStandaloneDecoderModelTest::test_generate_from_inputs_embeds_decoder_only`)

OK, I will rebase once #33602 is merged.

Should I also wait for a fix for 

```bash
tests/models/video_llava/test_modeling_video_llava.py::VideoLlavaForConditionalGenerationModelTest::test_sample_generate_dict_output
```
?


> Should I also wait for a fix for `tests/models/video_llava/test_modeling_video_llava.py::VideoLlavaForConditionalGenerationModelTest::test_sample_generate_dict_output`

@ydshieh working on it now (cc @zucchini-nlp, who is off for the next few days 🤗 )

@ydshieh https://github.com/huggingface/transformers/pull/33623 :D 

@ydshieh rebasing now should get rid of the red CI 🙏