Retaining computational graph after using AutoImageProcessor
Author: YinniKun
Pull request: no
Labels: name: Feature request

### Feature request

Right now `AutoImageProcessor` would detach the tensor from any gradient, which is totally fine if the input is obtained from DataLoader and have no gradient anyway. But would it be possible to retain the computational graph after using the preprocessing of `AutoImageProcessor` (similar to `torchvision.transforms.v2`)? 

### Motivation

I discovered this when I was trying to build a model that translates an image before using some other models to do downstream task. So something like this:

image -> my translator -> image processor for pre-trained model -> pre-trained model -> downstream

And I discovered that it is impossible to train end-to-end with the image processor in the middle because it would detach the gradient completely. So this feature would be helpful if we want to train something before using some existing models, we can let the gradient flow through during the training. 

### Your contribution

I haven't looked too deeply into the source code of ImageProcessor, but I suppose I can draft up a PR if there is interest in implementing this. I think most transforms in `torchvision.transforms.v2` would not cut off the gradient, so theoretically implementing the transformations in `torchvision.transforms.v2` would allow this feature

Comments:
Hi @YinniKun, thanks for the feature request! Do you have any specific model/image processor in mind? Not sure all ops in image processor are differentiable, I suppose @yonigozlan can tell more about it

Hi @qubvel , thanks for the prompt reply! I was trying to use the image processor of [owkin/phikon-v2](https://huggingface.co/owkin/phikon-v2/blob/main/preprocessor_config.json). I ended up just looked at this json file and implemented the transformations myself in `torchvision.transforms.v2` to keep the gradient flowing through. 

Hi @YinniKun ! Using slow image processor surely won't work because they use PiL/numpy functions for preprocessing, however fast image processors should do what you're describing, as they perform all ops on tensors, if the inputs are tensors! Just set `use_fast` to True when instantiating the image processor with `AutoImageProcessor`.
As @qubvel said, it will depend on the ops in the image processor, but AFAIR Bit has only typical ops (resizing/rescaling/normalizing) so it should work.

Hi @yonigozlan , thanks for the comment! I tried `use_fast=True` but I think it will still detach the tensor once I get it from `['pixel_values']`, despite the backend is with `torchvision`. I checked the source code but I couldn't find the culprit of this behaviour, and all ops in [pikon-v2](https://huggingface.co/owkin/phikon-v2/blob/main/preprocessor_config.json) aren't too fancy so `torchvision` shouldn't break it. The code block looks something like this:

```python
image_processor = AutoImageProcessor.from_pretrained("owkin/phikon-v2", use_fast=True)

dummy_input = torch.randn(8, 3, 224, 224)
x = my_converter(dummy_input) #I verify that the tensor has grad here
x = image_processor(x, return_tensors="pt", do_rescale=False)["pixel_values"] #the tensor does not have grad anymore
```