Fixed VitDet for non-squre Images
Author: cjfghk5697
Pull request: yes
Labels: name: Vision

# What does this PR do?

Fixes #35280  (issue)

The VitDet model doesn't support tuple-type inputs for image_size and patch_size. I’ve fixed it so that tuples can now be used.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ x ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ x ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ x ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@qubvel @amyeroberts


Comments:
Hey @cjfghk5697! Thanks for opening a PR 🤗 Feel free to ping me when it's ready for review

@qubvel Thanks for your response! Would you like me to provide a mock code to check the tuple case?

@qubvel, thank you for reviewing it🤗. I've finished adding the test code. Could you please check it again?

@qubvel I’ve finished running `make modified_only_fixup`—thanks for your help!

hm... something goes wrong, probably you have older/newer version of ruff? Can you please check and update accordingly?

> hm... something goes wrong, probably you have older/newer version of ruff? Can you please check and update accordingly?

I tried running it with `ruff==0.5.1`, but the results look really strange. Could you let me know which version you're using?


It's ruff 0.5.1 in my env, it might be that you are running with no `pyproject.toml` configuration?

It's better to delete 3 last commits, apply a modified fixup, and then force push

@qubvel Sorry about that! I created a new environment and deleted some commits, which ended up modifying the fixup. Let me know if anything needs to be adjusted.

@qubvel Can you review it again?

Hey @cjfghk5697 can you please fix quality issues? most probably `make modified_only_fixup` will resolve the issue

@qubvel Thank you for your help. I checked and discovered that there was an issue with my environment. I've applied the fix!🤗


Hey @cjfghk5697, with the recent modification we have a bunch of tests failed, can you please check what should be fixed?

```
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_attention_outputs - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_backbone - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_batching_equivalence - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_can_use_safetensors - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_correct_missing_keys - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_determinism - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_from_pretrained_no_checkpoint - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_gradient_checkpointing_backward_compatibility - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_gradient_checkpointing_enable_disable - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_hidden_states_output - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_initialization - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_inputs_embeds_matches_input_ids - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_load_save_without_tied_weights - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_model - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_model_get_set_embeddings - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_model_is_small - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_model_outputs_equivalence - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_model_weights_reload_no_missing_tied_weights - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_peft_gradient_checkpointing_enable_disable - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_retain_grad_hidden_states_attentions - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_save_load - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_save_load_fast_init_from_base - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_save_load_fast_init_to_base - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_save_load_keys_to_ignore_on_save - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_tie_model_weights - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_tied_weights_keys - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetModelTest::test_torch_save_load - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetBackboneTest::test_backbone_common_attributes - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetBackboneTest::test_backbone_outputs - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetBackboneTest::test_backbone_stage_selection - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetBackboneTest::test_channels - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetBackboneTest::test_create_from_modified_config - TypeError: 'int' object is not subscriptable
FAILED tests/models/vitdet/test_modeling_vitdet.py::VitDetBackboneTest::test_forward_signature - TypeError: 'int' object is not subscriptable
```

I think I got why tests failed, I hope you don't mind I've pushed the fix 🤗 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35969). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @yonigozlan can you take a look

@qubvel I sincerely appreciate your help in successfully wrapping up this task🤗