Owlv2 seems to have a really poor performance in image guided detection and the example of owlv2 on Huggingface indicated that.
Author: c1ekrt
Pull request: no

### System Info

- `transformers` version: 4.35.0.dev0
- Platform: Windows-10-10.0.22621-SP0
- Python version: 3.10.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.2
- Accelerate version: 0.22.0
- Accelerate config:    - compute_environment: LOCAL_MACHINE
        - distributed_type: NO
        - mixed_precision: no
        - use_cpu: False
        - debug: False
        - num_processes: 1
        - machine_rank: 0
        - num_machines: 1
        - gpu_ids: all
        - rdzv_backend: static
        - same_network: True
        - main_training_function: main
        - downcast_bf16: no
        - tpu_use_cluster: False
        - tpu_use_sudo: False
        - tpu_env: []
- PyTorch version (GPU?): 2.0.1+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help?

@ amyeroberts if not busy

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Copy paste the example of class transformers.Owlv2ForObjectDetection image_guided_detection and run it
https://huggingface.co/docs/transformers/main/en/model_doc/owlv2#transformers.Owlv2ForObjectDetection

### Expected behavior

In OwlVit model the image guided detection works out fine, successfully capture the reference image and identify two cats in the picture, however the new owlv2 seems to fail the given task of identifying the picture, which returns multiple boxes that don't make sense.

Detected similar object with confidence 0.938 at location [327.31, 54.94, 547.39, 268.06]
Detected similar object with confidence 0.959 at location [5.78, 360.65, 619.12, 366.39]
Detected similar object with confidence 0.902 at location [2.85, 360.01, 627.63, 380.79]
Detected similar object with confidence 0.985 at location [176.97, -29.45, 672.69, 182.83]
Detected similar object with confidence 1.0 at location [6.53, 14.35, 624.87, 470.82]
Detected similar object with confidence 0.998 at location [579.98, 29.14, 615.49, 489.05]
Detected similar object with confidence 0.985 at location [206.15, 10.53, 247.74, 466.01]
Detected similar object with confidence 0.947 at location [18.62, 429.72, 646.5, 457.72]
Detected similar object with confidence 0.996 at location [523.88, 20.69, 586.84, 483.18]
Detected similar object with confidence 0.998 at location [3.39, 360.59, 617.29, 499.21]
Detected similar object with confidence 0.969 at location [4.47, 449.05, 614.5, 474.76]
Detected similar object with confidence 0.966 at location [31.44, 463.65, 654.66, 471.07]
Detected similar object with confidence 0.924 at location [30.93, 468.07, 635.35, 475.39]

![image](https://github.com/huggingface/transformers/assets/40287606/afddb70f-5d25-48cf-9dd2-6dccabbe11e7)



Comments:
Hi,

Yes that documentation example requires an update. I've noticed that running the image guided detection with OWLv2 is pretty sensitive to the threshold values. Using `threshold=0.9`, `nms_threshold=0.3` in the `post_process_image_guided_detection`  method gives better results. See the bottom of my [demo notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/OWLv2/Zero_and_one_shot_object_detection_with_OWLv2.ipynb).

Could you open a PR to update the threshold values, as well as the expected values [here](https://github.com/huggingface/transformers/blob/main/src/transformers/models/owlv2/modeling_owlv2.py#L1604-L1623)?

@NielsRogge Thank you for your fast reply. In fact, parameter between Huggingface document and your demo notebook are the same, and the only difference is the query image you use. After changing the query image, the outcome Huggingface document produced is the same as in demo notebook. It seems that the image guided detection is quite sensitive.
Furthermore, tweaking the parameter in documentation example can't yield satisfying result if the original image is input as reference. 

Hi @c1ekrt,

I've opened a PR to address this, and I've also updated my [demo notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/OWLv2/Zero_and_one_shot_object_detection_with_OWLv2.ipynb) to illustrate the image guided detection in more detail. Hope that helps!