processor = Owlv2Processor.from_pretrained("google/owlv2-base-patch16-ensemble") may have some problem
Author: syyxtl
Pull request: no
Labels: name: Good First Issue

name: Good First Documentation Issue

![image](https://github.com/huggingface/transformers/assets/66545456/c979a108-4e14-44da-9767-f27b2faf325b)

if the Owlv2Processor have the "padding" operator, the post_process_object_detection may can not directly the following:
    boxes = boxes * scale_fct[:, None, :]


I found the y-axis have some shift

Comments:
cc @NielsRogge 

my ori image size=[1080, 1920]
after preprocess size=[960, 960]

exp1:
target_sizes = torch.Tensor([[1080, 1920]])
results = processor.post_process_object_detection(outputs=outputs, target_sizes=target_sizes, threshold=0.2)
![image](https://github.com/huggingface/transformers/assets/66545456/ace8fb36-e49a-4a7f-b3cc-cf86a1d74f6c)

exp2:
target_sizes = torch.Tensor([[1920, 1920]])
results = processor.post_process_object_detection(outputs=outputs, target_sizes=target_sizes, threshold=0.2)
![image](https://github.com/huggingface/transformers/assets/66545456/71d5b6dd-ba54-42e0-aaf4-e089d49e65a0)


Hi,

Do you have a fully reproducible script? Note that the target sizes should be (height, width) rather than (width, height).

> Hi,
> 
> Do you have a fully reproducible script? Note that the target sizes should be (height, width) rather than (width, height).

@NielsRogge 
 I use the official script:
https://huggingface.co/docs/transformers/main/model_doc/owlv2
![image](https://github.com/huggingface/transformers/assets/66545456/aa89d62b-e461-4560-93f1-dcc337922dc3)

In the above example, I may have written the order backwards

> Hi,
> 
> Do you have a fully reproducible script? Note that the target sizes should be (height, width) rather than (width, height).

@NielsRogge 
i may change the script below
![image](https://github.com/huggingface/transformers/assets/66545456/7c9a6b3b-0f0a-4247-bfb5-0f9453fb71d9)


Hi, noticed the same issue and looked into the cause:

The problem is coming from padding the image in the preprocessor. After the padding the model returns the coordinates within padded square. Thus, always when padding was used in preprocessor, the target sizes for post_process functions should be [(max(W, H), max(W,H))] as noted above. Please add this note to the documentation and/or the example as even the sample image bounding boxes are off due to this.  

Hi, is this change already done? If not, can I take this up?

No not yet, you can take it up!

@itsadarshms @NielsRogge As this is a recently added model, the fix for this should be to update the processor to accept target sizes in `(h, w)` rather than updating the documentation. `(h, w)` is the standard format for our image processing. 

It already does accept `target_sizes` in (height, width) format, see [here](https://github.com/huggingface/transformers/blob/35551f9a0f66a22de4971b4a51b3c172d3b87f95/src/transformers/models/owlv2/image_processing_owlv2.py#L505). There's nothing wrong with the `post_process_object_detection` method, see my [notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/OWLv2/Zero_and_one_shot_object_detection_with_OWLv2.ipynb) for an illustration.

The only reason, as pointed out by @michal-stary, is that the preprocessor internally pads the image. Hence visualizations need to be shown on the padded image, rather than the original image. See also my notebook for an illustration of that.

Hence to fix this, one needs to update the code snippet to set the `target_sizes` based on the padded image rather than the original one. I've opened a PR to address this.