Batch inference of GIT model
Author: JosephChenHub
Pull request: no

### System Info

- `transformers` version: 4.26.1            
- Platform: Linux-5.4.0-125-generic-x86_64-with-glibc2.27 
- Python version: 3.8.0
- Huggingface_hub version: 0.12.1
- PyTorch version (GPU?): 1.9.1+cu111 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA) 
- Jax version: not installed 
- JaxLib version: not installed
- Using GPU in script?:  True
- Using distributed or parallel set-up in script?:  No


When I use GIT model to do image caption, it throws an exception :
```
 File "/usr/local/lib/python3.8/dist-packages/transformers/models/git/modeling_git.py", line 1272, in forward  
hidden_states = torch.cat((projected_visual_features, embedding_output), dim=1)            
RuntimeError: Sizes of tensors must match except in dimension 0. Got 1 and 0 (The offending index is 0)
```


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

to reproduce this error, use the following core code:
```
processor = AutoProcessor.from_pretrained("microsoft/git-base-coco")
model = AutoModelForCausalLM.from_pretrained("microsoft/git-base-coco").to(device)
model.eval()

...
pixel_values = processor(images=images, return_tensors="pt").pixel_values.to(device) 
print(pixel_values) # (10, 3, 224, 224) 
output_ids = model.generate(pixel_values=pixel_values, max_length=50)
preds = processor.batch_decode(output_ids, skip_special_tokens=True)[0]
```

I have checked the source file "modeling_git.py" line 1272,  
![image](https://user-images.githubusercontent.com/22444830/220228879-cae18b8f-f7f9-4d76-9ef8-f7d715011eba.png)

due to **embedding_output.size(0)** is 1 but visual features size(0) is 10,  1//10 = 0,  
![image](https://user-images.githubusercontent.com/22444830/220229180-cfee16ac-5e65-4aec-b04e-1ce778268706.png)

so these two features cannot be concatenated. 


### Expected behavior

support batch input 

Comments:
cc @amyeroberts and @gante 

Hi @JosephChenHub 👋 We have been working on similar problems since the release of v4.26. Can you please confirm that the problem still exists in `main`?

(`pip install --upgrade git+https://github.com/huggingface/transformers.git`)

> pip install --upgrade git+https://github.com/huggingface/transformers.git

Hi @gante,  I use this command 

> Hi @JosephChenHub 👋 We have been working on similar problems since the release of v4.26. Can you please confirm that the problem still exists in `main`?
> 
> (`pip install --upgrade git+https://github.com/huggingface/transformers.git`)

Hi @gante , I use the source code of `main` branch (transformers==4.27.0.dev0), the issue still exists. 

Hey @JosephChenHub 👋 Thank you for your confirmation :) 

I was able to track down the root cause -- #21738 fixes it! After it is merged, you can install from `main` again and it should work 🚀 

(@JosephChenHub It should be working now, let me know if it is not!)