CLIP image processor fails when resizing a 1x1 image
Author: justinpinkney
Pull request: no

### System Info

- `transformers` version: 4.26.1
- Platform: Linux-4.19.0-23-cloud-amd64-x86_64-with-debian-10.13
- Python version: 3.7.12
- Huggingface_hub version: 0.12.0
- PyTorch version (GPU?): 1.12.1+cu113 (True)
- Tensorflow version (GPU?): 2.11.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help?

@amyeroberts

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run the following:

```python
from PIL import Image
import requests
from transformers import AutoProcessor, CLIPModel

model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32")
processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32")

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
image = image.resize((1,1))

print(image.mode, image.size)

inputs = processor(
    text=["a photo of a cat", "a photo of a dog"], images=image, return_tensors="pt", padding=True
)

outputs = model(**inputs)
logits_per_image = outputs.logits_per_image  # this is the image-text similarity score
probs = logits_per_image.softmax(dim=1)  # we can take the softmax to get the label probabilities
```

The issue appears to be caused by `infer_channel_dimension_format(image)` which for a numpy array of shape 1x1x3 corresponding to a 1x1 rgb image, the return value is `<ChannelDimension.FIRST: 'channels_first'>` for the input data which is incorrect in this case.

Gives the error:
```
Traceback (most recent call last):
  File "hf_bug.py", line 15, in <module>
    text=["a photo of a cat", "a photo of a dog"], images=image, return_tensors="pt", padding=True
  File "/some/path/.venv/lib/python3.7/site-packages/transformers/models/clip/processing_clip.py", line 102, in __call__
    image_features = self.image_processor(images, return_tensors=return_tensors, **kwargs)
  File "/some/path/.venv/lib/python3.7/site-packages/transformers/image_processing_utils.py", line 446, in __call__
    return self.preprocess(images, **kwargs)
  File "/some/path/.venv/lib/python3.7/site-packages/transformers/models/clip/image_processing_clip.py", line 327, in preprocess
    images = [self.normalize(image=image, mean=image_mean, std=image_std) for image in images]
  File "/some/path/.venv/lib/python3.7/site-packages/transformers/models/clip/image_processing_clip.py", line 327, in <listcomp>
    images = [self.normalize(image=image, mean=image_mean, std=image_std) for image in images]
  File "/some/path/.venv/lib/python3.7/site-packages/transformers/models/clip/image_processing_clip.py", line 211, in normalize
    return normalize(image, mean=mean, std=std, data_format=data_format, **kwargs)
  File "/some/path/.venv/lib/python3.7/site-packages/transformers/image_transforms.py", line 334, in normalize
    raise ValueError(f"mean must have {num_channels} elements if it is an iterable, got {len(mean)}")
ValueError: mean must have 1 elements if it is an iterable, got 3
```

### Expected behavior

the 1x1 input image should be resized to 224x224

Comments:
Thanks for raising this issue @justinpinkney and for the detailed snippet and trackback! 

Indeed, you're right, the issue is arising from trying to infer the image channel dimension. As it's possible to have images with a single channel, and images with 3 channels, an image with shape `(1, 1, 3)` could be either a 1x3 single channel image or 1x1 3 channel image. This ambiguity in dimensions causes many issues and it's one that I'm currently trying to address.

Depending on the input data format you're feeding to the image processor (torch/pil/tf/np/jax and batched/single image/list of images), the fastest way around this would be tiling the pixels to create a compatible shape e.g. 2x2x3 image, as this will result in the same image after resizing as the original 1x1. However, this is quite hacky and the bug will still persist in cases when the dimensions cannot be confidently inferred e.g.  a 3x3x3 image. 

I'll make sure to keep this issue updated with changes to the code to address this. 

I was hoping I could specify the data format using the `data_format` argument, but that turned out to be just for the output images, not specifying the inputs. In my case these 1xn and 1x1 images were just bad samples, so I could filter them out in the data loading pipeline.

Thanks for the quick response though!

Yes, at the moment it just controls the output format. I think being able to specify the input data format is a good solution however! I'll draft something up :) 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.