Add Blip2 model in VQA pipeline
Author: jpizarrom
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Add Blip2ForConditionalGeneration model in VisualQuestionAnsweringPipeline.
Fixes part of #21110 and is based on #23348 #21227 .


## Who can review?

Hi @NielsRogge  what do you think of this??
Thanks!

## TODOs
- [x] add Blip2 model in VQA pipeline 
- [x] use require_torch_gpu in test
- [x] use can_generate in vqa pipeline for Blip2ForConditionalGeneration
- [x] use float16 in the test_large_model_pt_blip2
- [ ] check if it is necessary to cast the input in torch.float16 inside _forward

Comments:
cc @amyeroberts and @younesbelkada 

Hi @amyeroberts and @younesbelkada, this PR is ready for review. Could you please take a look? Thanks :) 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25532). All of your documentation changes will be reflected on that endpoint.

> LGTM.
> 
> I'm not a big fan of using the names of the configs directly to detect if generative or not, I feel like using the `ForXXX` should be a better hint.
> 
> We also used `model.can_generate()` as a hint in other pipelines.
> 
> Pinging @ylacombe who used that flag. (Just FYI no need to do anything).

@Narsil, Thanks a lot for your feedback :)

At the moment model.can_generate() return False for Blip2ForConditionalGeneration, that is the reason why I was following the proposal of this non merged PR https://github.com/huggingface/transformers/pull/23348/files#diff-620bada7977c3d0040ed961581379598e53a9ef02fdbb26c570cac738c279c0eR64

Maybe could it be expected that can_generate method returns True for Blip2ForConditionalGeneration? if this is the case, we could use it. (i will take a look on it)

can_generate returns True for another model, does it make sense to do this in Blip2ForConditionalGeneration, or it could affect something else?
https://github.com/huggingface/transformers/blob/50573c648ae953dcc1b94d663651f07fb02268f4/src/transformers/models/speecht5/modeling_speecht5.py#L2782-L2787






I'm not the best person to comment on how `can_generate` works and what should or shouldn't be done.

The main thing about pipeline:

- They should try to be model agnostic as much as possible so when newer models come in they work out of the box.

But the current code is acceptable.

Generalizable solution here ☝️ 

Feel free to tweet/linkedin about it @jpizarrom and we'll amplify :)

I use the newest library of transformers,but it still reports "The model 'Blip2ForConditionalGeneration' is not supported for vqa. Supported models are ['ViltForQuestionAnswering'].",so how can I use blip2 in pipeline to deal with the vqa task?Are there any test codes of BLIP2 in pipeline?

Hi @RainyLayx i was able to run this sample with _transformers==4.37.1_

```python
from transformers import pipeline
import requests
from PIL import Image

vqa_pipeline = pipeline("visual-question-answering", model="Salesforce/blip2-opt-2.7b")

image =  Image.open(requests.get("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png", stream=True).raw)
question = "Question: Is there a parrot? Answer:"

print(vqa_pipeline(image, question, top_k=1))
```