Add D-FINE Model into Transformers
Author: VladOS95-cyber
Pull request: yes
Labels: name: New model

name: Vision

# What does this PR do?

This PR add D-FINE into the Transformers library. There is a new thing in transformers called modular, which adds new models by creating a modeling_modelname.py file. Since D-FINE updates several RT-DETR arch parts while keeping the rest of the model unchanged, it serves as an ideal use case for this modular approach.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. Link: https://github.com/huggingface/transformers/issues/35283
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@qubvel @Rocketknight1 @ArthurZucker 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @ylacombe, @eustlb
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
cc @qubvel @NielsRogge 

Model implementation is done. Unit tests are still missing. I am working on it

Hi @qubvel! Thank you for your comment! So, I added some integration tests, refactored modules and changed conversion script according to the latest standards. 

Hi @qubvel! I fixed all failing tests except repository consistency, because during resnet modular conversion, the script add additional resnet postfix into configuration import path in modeling_d_fine_resnet file like **from .configuration_d_fine_resnet_resnet import DFineResNetConfig**. I manually fixed that, but honestly, I have no idea why modular conversion script behave like that..

Hey @VladOS95-cyber, thanks a lot for the update! I have a sprint, but I will try to review it as soon as have any bandwidth!
Regarding modular conversion, it might have some issues, it's still in active development.

cc @Cyrilvallez re `modular` issues

Hi @qubvel! I resolved your comments and did some refactoring and cleanup

Hi @qubvel! Thank you for your comments. I resolved most of them, for several ones I provided my comments

Hi @qubvel! Sorry for the mess with commits, just unlucky rebase..., I hope it is not an obstacle for review. To sum up, I cleaned up modulars according to your comments as much as possible. On this step we have two unresolved problems: 1. Wrong namespace import for **modeling_d_fine_resnet.py** where conversion script provides **from .configuration_d_fine_resnet_resnet**. 2. DFine config conversion using modular does not work properly because it provides a call `super().__init__(is_encoder_decoder=is_encoder_decoder, **kwargs)` too eraly in the very beginning of __init__ and before all self.args assignments, which is wrong as well.