Not implement resize_token_embeddings in blip2-itm-vit-g
Author: pspdada
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.45.2
- Platform: Linux-5.15.0-120-generic-x86_64-with-glibc2.35
- Python version: 3.10.15
- Huggingface_hub version: 0.25.2
- Safetensors version: 0.4.5
- Accelerate version: 1.0.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.4.0+cu124 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: None

### Who can help?

@ArthurZucker @itazap

### Information

- [] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I tried to use the model `Salesforce/blip2-itm-vit-g`, but encountered a warning. 

> Expanding inputs for image tokens in BLIP-2 should be done in processing. Please follow instruction here (https://gist.github.com/zucchini-nlp/e9f20b054fa322f84ac9311d9ab67042) to update your BLIP-2 model. Using processors without these attributes in the config is deprecated and will throw an error in v4.47.

The code that need to be added is:
```
# Load your model and processor and run the following to update BLIP-2 model
# It will update file in your repo by adding new args in configs and resizing embedding layer
# Then you'll be able to run BLIP-2 without warnings/errors
from transformers import AddedToken

processor.num_query_tokens = model.config.num_query_tokens
image_token = AddedToken("<image>", normalized=False, special=True)
processor.tokenizer.add_tokens([image_token], special_tokens=True)

model.resize_token_embeddings(len(processor.tokenizer), pad_to_multiple_of=64) # pad for efficient computation
model.config.image_token_index = len(processor.tokenizer) - 1

model.push_to_hub("YOUR-REPO")
processor.push_to_hub("YOUR-REPO")
```

After adding the code mentioned there, I received a `NotImplementedError` when calling the function `resize_token_embeddings`. What should I do?

The code of mine:
```
def create_blip2_retrieval() -> tuple[Blip2Processor, Blip2ForImageTextRetrieval]:
    """
    Create a BLIP2 image-text retrieval model and processor.
    Size: 1.17B params
    """
    model_name = "Salesforce/blip2-itm-vit-g"
    processor: Blip2Processor = AutoProcessor.from_pretrained(
        model_name,
        cache_dir=model_dir,
    )
    model: Blip2ForImageTextRetrieval = Blip2ForImageTextRetrieval.from_pretrained(
        model_name,
        cache_dir=model_dir,
        torch_dtype=torch.float16,
        device_map="auto",
        low_cpu_mem_usage=True,
    ).to(device)

    # Following https://gist.github.com/zucchini-nlp/e9f20b054fa322f84ac9311d9ab67042#file-update_blip-py
    processor.num_query_tokens = model.config.num_query_tokens
    image_token = AddedToken("<image>", normalized=False, special=True)
    processor.tokenizer.add_tokens([image_token], special_tokens=True)

    model.resize_token_embeddings(len(processor.tokenizer), pad_to_multiple_of=64)  # pad for efficient computation
    model.config.image_token_index = len(processor.tokenizer) - 1

    return processor, model
```
Then I tried the model `Salesforce/blip2-flan-t5-xl` and saw the same warning. I added the additional code and it worked. No warning left.

### Expected behavior

Implement resize_token_embeddings on the model  `Salesforce/blip2-itm-vit-g`?
Maybe

Comments:
looking into it