RuntimeError when running batched inference for Salesforce/blip2-opt-2.7b VQA
Author: KheoAB
Pull request: no

### System Info

- `transformers` version: 4.34.0
- Platform: Linux-4.14.322-246.539.amzn2.x86_64-x86_64-with-glibc2.10
- Python version: 3.8.3
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.3
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.0.0+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

@gante 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

# Code To Reproduce
```python
from PIL import Image
import requests
from transformers import Blip2Processor, Blip2ForConditionalGeneration
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = Blip2Processor.from_pretrained("Salesforce/blip2-opt-2.7b")
model = Blip2ForConditionalGeneration.from_pretrained(
    "Salesforce/blip2-opt-2.7b", torch_dtype=torch.float16
)
model.to(device)
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

# ---------------------- Change made here ---------------------- #
# Passing `images=[image, image]` instead of `images=image` for testing batched inference
# And adding a text to the input
# inputs = processor(images=[image, image], return_tensors="pt").to(device, torch.float16)
prompt = "Question: What is a dinosaur holding? Answer:"
inputs = processor(images=[image, image], text=prompt, return_tensors="pt").to(device, torch.float16)
# ---------------------------------------------------------------- #

generated_ids = model.generate(**inputs)
```

# Error Stack Trace
```
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[8], line 7
      4 inputs = processor(images=[image, image], text=prompt, return_tensors="pt").to(device, torch.float16)
      5 # ---------------------------------------------------------------- #
----> 7 generated_ids = model.generate(**inputs)

File /opt/conda/lib/python3.8/site-packages/torch/utils/_contextlib.py:115, in context_decorator.<locals>.decorate_context(*args, **kwargs)
    112 @functools.wraps(func)
    113 def decorate_context(*args, **kwargs):
    114     with ctx_factory():
--> 115         return func(*args, **kwargs)

File /opt/conda/lib/python3.8/site-packages/transformers/models/blip_2/modeling_blip_2.py:1874, in Blip2ForConditionalGeneration.generate(self, pixel_values, input_ids, attention_mask, **generate_kwargs)
   1872 if attention_mask is None:
   1873     attention_mask = torch.ones_like(input_ids)
-> 1874 attention_mask = torch.cat([language_attention_mask, attention_mask.to(language_attention_mask.device)], dim=1)
   1876 # concatenate query embeddings with prompt embeddings
   1877 inputs_embeds = self.get_input_embeddings()(input_ids)

RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 2 but got size 1 for tensor number 1 in the list.
```

### Expected behavior

I'd expect being able to run batched inference for a batch size > 1 with a text prompt for VQA 

Comments:
#21578 I already looked at this issue, but mine is with vqa.

I've fixed the issue it's running fine could you guide me to the folder so that I can pull requests 


Hey, there. I don't understand what you want from me. Can you re-phrase or explain me more ?

yaa I fixed ur issue but I'm not able to find the repository where I'm required to push this code

The repository is transformers : the one i'm writting this issue from

Hi,

You're preparing 2 images but only one text:
```
inputs = processor(images=[image, image], text=prompt, return_tensors="pt").to(device, torch.float16)
```
hence you need to do:
```
inputs = processor(images=[image, image], text=[prompt, prompt], return_tensors="pt").to(device, torch.float16)
```

Thanks for the answer! It doesn't seem logical at first glance because the argument is text without an 's'.