Image Preprocess in Clip
Author: JayceWen
Pull request: no
Labels: name: Vision

### System Info

- `transformers` version: 4.37.1
- Platform: Linux-5.15.0-102-generic-x86_64-with-glibc2.31
- Python version: 3.10.14
- Huggingface_hub version: 0.22.2
- Safetensors version: 0.4.2
- Accelerate version: 0.21.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): 2.15.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.7.5 (cpu)
- Jax version: 0.4.20
- JaxLib version: 0.4.20
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

l dont know

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

When l try to using the  class **CLIPImageProcessor** to preprocess my images like this 
```
image = self.processor.preprocess(image, return_tensors='pt', do_normalize=False, do_rescale=False, do_center_crop=True, crop_size={'height':324,'width':324})['pixel_values']
```
l found that in the function preprocess as follows, which means the func **first resize** the image to the target shape(such as 336*336) which is corresponding to clip vit. Then **center crop** the image,which cause the image to be the shape of (324,324) in my case,and it's mismatch the target size. And need to call twice the preprocess function to bypass this problem which l thought there is a more efficient way to solve it.
```
        if do_resize:
            images = [
                self.resize(image=image, size=size, resample=resample, input_data_format=input_data_format)
                for image in images
            ]

        if do_center_crop:
            images = [
                self.center_crop(image=image, size=crop_size, input_data_format=input_data_format) for image in images
            ]
```

### Expected behavior

may be add a post resize function after center crop to ensure the images are in the target shape or simply change the order of resize and center crop may be ok.

Comments:
Hi @lesjie-wen, thanks for opening this issue! 

The CLIPImageProcessor defines a series of steps which will prepare raw images in the format expected by the CLIP model. It follows the preprocessing steps done for the original model. 

The first resize will resize according to `processor.size`. You can modify the output size by passing `size={"shortest_edge: x"}` or `size={"height": h, "width": w}` when calling the image processor e.g. `processor(images, size={"height": h, "width": w})`. 

The reason the image is `(324, 324)` when output, is because this is the crop size specified in the example provided:

```
image = self.processor.preprocess(image, return_tensors='pt', do_normalize=False, do_rescale=False, do_center_crop=True, crop_size={'height':324,'width':324})['pixel_values']
```

If you want a size that matches the target size you can either: 
* Change `crop_size` when calling the processor to match your needs i.e crop_size is the target size. This is the most common behaviour when processing images. 

```
image = self.processor.preprocess(image, return_tensors='pt', do_normalize=False, do_rescale=False, do_center_crop=True, crop_size={'height': target_height, 'width': target_width})['pixel_values']
```

* Call `processor.resize` on the output images
```
from transformers.image_processing_utils import BatchFeature

images = self.processor.preprocess(image, do_normalize=False, do_rescale=False, do_center_crop=True, crop_size={'height': target_height, 'width': target_width})['pixel_values']
images = [self.processor.resize(image, size={"height": target_height, "width": target_width}) for image in images]
images = BatchFeature({"pixel_values": images}, tensor_type="pt")
```

Thank you very much @amyeroberts, l change my code to resize images bigger than the target size and then crop to target size. And it works well.