[BLIP-2] Batch generation error in inference
Author: Kim-DKyu
Pull request: no

### System Info

transformers            4.38.2
torch                   1.13.1+cu116
torchvision             0.14.1+cu116


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

sorce code
```
from transformers import Blip2Processor, Blip2ForConditionalGeneration
import torchvision.transforms.functional as TF
from PIL import Image

image (Batch_size x 3 x 224 x 224)

prompt_list = []
for i in word_list:
    blip_prompt = f"Question: Explain the advantages of {i}? Answer:"
    prompt_list.append(blip_prompt) ->  len(prompt_list) : Batch_size

pil_images = []
for img_tensor in image:
    pil_image = TF.to_pil_image(img_tensor)
    pil_images.append(pil_image)

ERROR ❱ inputs = self.processor_blip(images=pil_images, text=prompt_list, return_tensors="pt").to(device, torch.float16)
generated_ids = model.generate(**inputs) #, max_length=256)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0].strip()
```

```
File "src/models/adapter/lp.py", line 59, in forward
    inputs = self.processor_blip(images=pil_images, text=prompt_list, return_tensors="pt").to(device, torch.float16)
  File ".local/lib/python3.8/site-packages/transformers/models/blip_2/processing_blip_2.py", line 108, in __call__
    text_encoding = self.tokenizer(
  File ".local/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2829, in __call__
    encodings = self._call_one(text=text, text_pair=text_pair, **all_kwargs)
  File ".local/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2915, in _call_one
    return self.batch_encode_plus(
  File ".local/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 3106, in batch_encode_plus
    return self._batch_encode_plus(
  File ".local/lib/python3.8/site-packages/transformers/tokenization_utils_fast.py", line 552, in _batch_encode_plus
    return BatchEncoding(sanitized_tokens, sanitized_encodings, tensor_type=return_tensors)
  File ".local/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 223, in __init__
    self.convert_to_tensors(tensor_type=tensor_type, prepend_batch_axis=prepend_batch_axis)
  File ".local/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 764, in convert_to_tensors
    raise ValueError(
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (`input_ids` in this case) have excessive nesting (inputs type `list` where type `int` is expected).
```

First, thank you for sharing your wonderful work as an open source. I appreciate your work.
I intend to perform inference using the BLIP-2 model with multiple images and prompts.
Could you tell me what might be causing the error message described above?
I'd also like to inquire whether a batch size of 512 is too large during inference.
Your response would be greatly appreciated. Thank you.

### Expected behavior

No error

Comments:
cc @zucchini-nlp 

@Kim-DKyu hey! As the error message says, you have to pad the inputs if you want a tensor format. See [here](https://huggingface.co/docs/transformers/en/pad_truncation) for more details 🤗 

`inputs = self.processor_blip(images=pil_images, text=prompt_list, padding=True, return_tensors="pt").to(device, torch.float16)`

Thank you for the quick response. It was helpful