Add generation tests for multimodal generative models
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

It was found that none of the vision LLMs have `GenerationTesterMixin` which is why we had many blind spots, which were not being tested. This PR adds the Mixin to all VLMs and fixes where possible. Some tests are still failing, I am working on it. 

I hope the tests will be much cleaner when we refactor `generate`, because right now I had to add some hacks for tests in models with custom generation.  

EDIT:
Apparently, idefics does not work right now without enabling cache. To not forget in the upcoming rework of `generate`: the problem of idefics is in the special treatment of image attn mask [here](https://github.com/huggingface/transformers/blob/7eb3ba82241c927053689270a0751f4ff5d33c54/src/transformers/models/idefics/modeling_idefics.py#L1565). 

cc @gante 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29853). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Should be ready for review now. All tests (new added and old ones) are passing for all models. VipLlava will be fixed after rebasing the mentioned PR. 

The biggest problem with multimodal models was the use of different main input names in their custom generate. This behavior should be fixed after the `generate` rework is done.

@gante 

1. Yes, I found that some model's use custom `generate` and the main input name in those cases is either pixel_values. So if we leave it as it is and pass in "input_ids" as first main arg and others as model kwargs, those custom calls complain about incorrect arguments. It would be out of box I believe if we called the main `generate` directly, as I see that we check model forward args and main input names there. However now we cannot get rid of this. Another workaround would be to add separate `_get_inputs_and_config` for each VLMs test, similar to what is happening in the audio models. That way we can hardcode the method to return (`main_input_tensor`, `other_kwargs_in_dict`, `config`, `max_length`) in their specific model testers. But I think it is same level of complication as we have now :(
2. Actually `generate` raises error for unused model kwargs in [this line](https://github.com/huggingface/transformers/blob/fa2c49b00baadd4e7c509c7f241015f6deeb7664/src/transformers/generation/utils.py#L1125). And since some VLMs have required args only for that model, trying to get args from `prepare_config_and_inputs_for_common` by name seemed to be infeasible to generalize, even though it is working for text generative models. 
3. Agreed, we can get rid of it. I just encountered a bunch of new fails on text models, just because of changing seq length so decided to leave it for now. That maybe requires another PR to be merged before this one, to just "remove dividing seq length to two" in text models



@zucchini-nlp let's open a separate PR that addresses the 3rd point (get rid of the `//2` in `sequence_length = input_ids.shape[-1] // 2`), and chat about VLM tests during our next meeting. I really want to move in the direction of less complexity, even if that implies structural changes in `generate` :) 

@gante Yes, there is a PR already that you approved ahaha (https://github.com/huggingface/transformers/pull/30016)
Just waiting for a reply from @ArthurZucker on some comments to merge

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.