Update InstructBLIP & Align values after rescale update
Author: amyeroberts
Pull request: yes

# What does this PR do?

After #25174 the integration tests for Align and InstructBLIP fail. 


### InstructBLIP
The difference in the output logits is small. Additionally, when debugging to check the differences and resolve the failing tests, it was noticed that the InstructBLIP tests are not independent. Running 

```
RUN_SLOW=1 pytest tests/models/instructblip/test_modeling_instructblip.py::InstructBlipModelIntegrationTest::test_inference_vicuna_7b
```

produces different logits than running: 
```
RUN_SLOW=1 pytest tests/models/instructblip/test_modeling_instructblip.py::InstructBlipModelIntegrationTest
```

The size differences between these two runs was similar to the size of differences seen with the update in `rescale`. Hence, I decided that updating the logits was OK. 


### Align

The differences in align come from the model's image processor config values. Align uses EfficientNet's image processor. By default, [EfficientNet has `rescale_offset` set to `False`](https://github.com/huggingface/transformers/blob/0fd8d2aa2cc9e172a8af9af8508b2530f55ca14c/src/transformers/models/efficientnet/image_processing_efficientnet.py#L92) and [`rescale_factor` set to `1 / 255`](https://github.com/huggingface/transformers/blob/0fd8d2aa2cc9e172a8af9af8508b2530f55ca14c/src/transformers/models/efficientnet/image_processing_efficientnet.py#L91). Whereas Align has it set to `True` e.g. for [this config](https://huggingface.co/kakaobrain/align-base/blob/e96a37facc7b1f59090ece82293226b817afd6ba/preprocessor_config.json#L25) and the [`rescale_factor` set to `1 / 127.5`](https://huggingface.co/kakaobrain/align-base/blob/e96a37facc7b1f59090ece82293226b817afd6ba/preprocessor_config.json#L24). 

In #25174, the `rescale` logic was updated so that if `rescale` is called with `offset=True`, the image values are rescaled between by `scale * 2`.  This was because this was I was working from the EfficientNet and ViVit `rescale_factor` values which were both 1/255, so assumed the intention was to have this adjust if `rescale_offset` was True. 

There's three options for resolving this: 

1. Update Align Config
Update the values in the align checkpoint configs so that `rescale_factor` is `1 / 255` instead of `1 /127.5`. 
* ✅  Rescale behaviour and config flags consistent across image processors
* ❌ Remaining unexpected behaviour for anyone who has their own checkpoints of this model. 

2. Update rescale and ViVit config
Update the values in the ViVit model config. Revert the rescale behaviour so that `rescale_offset` and `rescale_factor` are independent. 

* ✅  Rescale behaviour and config flags consistent across image processors
* ❌ Remaining unexpected behaviour for anyone who has their own checkpoints of this model.  
* 🟡 No magic behaviour (adjusting `rescale_factor`) but relies on the user correctly updating two arguments to rescale between `[-1, 1]`

3. Revert EfficientNet's rescale method to previous behaviour.  
* ✅ Both models fully backwards compatible with previous rescale behaviour and config values
* ❌ Rescale behaviour and config flags not consistent across image processors
 

I think option 2 is best. ViVit is a newly added model, it keeps consistent behaviour between Align / EffiicentNet and ViVit and the `rescale` method isn't doesn't anything magic to make the other arguments work. @sgugger @ydshieh I would be good to have your opinion on what you think is best here. 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Agreed with your plan!

I also prefer 2., but I am a bit confused

> Update rescale and ViVit config

So this only changes `ViVit` config and its `rescale`. And Align uses `EfficientNet` image processor. So when we change  something in `ViVitf`, how this fixes the CI failing ... 🤔 ?

> So this only changes ViVit config and its rescale. And Align uses EfficientNet image processor. So when we change something in ViVitf, how this fixes the CI failing ... 🤔 ?

@ydshieh  Sorry, it wasn't super clear. The reason the CI is failing is because:
* Align doesn't have its own image processor - it uses EfficientNet's
* EfficientNet and ViVit both have the option to 'offset' when rescaling i.e. centering the pixel values around 0. 
* As both EfficientNet and ViVit's image processors have a rescale_factor of `1/255` by default, their docstrings mention setting `rescale_offset=True` rescales between `[-1, 1]` and they offset before rescaling, I assumed that then intention was to optionally rescale by `2 * rescale_factor` if `rescale_offset=True` for both
* This was true for ViVit.
* Align image processor config value are actually already updated so `rescale_factor` is `2 * (1 / 255) = 1 / 127.5`
* Therefore, the resulting pixel values from Align's image processor weren't in the range `[-1, 1]` when rescale was changed. 

Updating something in ViVit doesn't fix the CI directly. I'll also have to update `rescale` for both the methods to use Align's intended logic. 



@ydshieh I've made the updates for option 2: 

* Reverted to the previous `rescale` behaviour for EfficientNet: 7c3b3bb
* Same behaviour is copied across to ViVit, also in 7c3b3bb
* Made PRs to update the rescale values in ViVit models - `rescale_factor` 1/255 -> 1/127.5
    - https://huggingface.co/google/vivit-b-16x2/discussions/1#64c92542c96a10fa85bbca0b
    - https://huggingface.co/google/vivit-b-16x2-kinetics400/discussions/2#64c9253aaf935d3927ec1409



Oh I know why I get confused now 

> Update the values in the ViVit model config. Revert the rescale behaviour so that rescale_offset and rescale_factor are independent.

I thought only ViVit would be changed in this PR, but actually you mean both ViVit and `EfficientNet` (but the revert to before #25174).

Thanks for the update!