Add EoMT with DINOv3 backbone
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds EoMT with a DINOv3 backbone. The authors of EoMT released new checkpoints which swap the DINOv2 backbone by the newer DINOv3: https://github.com/tue-mps/eomt/blob/master/model_zoo/dinov3.md

## Disclaimer

This PR was implemented using OpenAI Codex and further improved by me.

## Who can review?

@yonigozlan 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41212). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@yonigozlan I've addressed your comments. I've updated `modular_eomt_dinov3.py` to inherit from `EomtForUniversalSegmentation` instead. As can be seen, the difference with `modeling_eomt.py` is quite minimal. But we cannot add a self.backbone attribute to `EomtForUniversalSegmentation` as that would be a breaking change with respect to the DINOv2 backbone.

Het @NielsRogge ! I'm ok to merge on the eomt dinov3 part, but it looks like there might have been an issue with branch merging or something?😅 

All checkpoints have been converted: https://huggingface.co/models?library=transformers&other=eomt_dinov3&sort=trending.

@molbap failing tests are unrelated, all checkpoints have been transferred :)

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=41212&sha=3db7f7

Thanks for your reviews, addressed all comments.

#43556 is now merged so inheritance should properly work now with modular re the rope mixin

@vasqu I have used it, however I cannot have the config inherit from `EomtConfig`, I had to inherit from `PreTrainedConfig`, as modular otherwise introduces additional attributes - seems like a bug

run-slow: eomt, eomt_dinov3

This comment contains `run-slow`, running the specified jobs: 

models: ["models/eomt", "models/eomt_dinov3"]
quantizations: []

run-slow: eomt, eomt_dinov3, vit

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/21523533237)

⚠️ No test being reported (jobs are skipped or cancelled)!


run-slow: eomt, eomt_dinov3, vit

This comment contains `run-slow`, running the specified jobs: 

models: ["models/eomt", "models/eomt_dinov3", "models/vit"]
quantizations: []

This comment contains `run-slow`, running the specified jobs: 

models: ["models/eomt", "models/eomt_dinov3", "models/vit"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/21523990708)

### Model CI Report

#### ❌ Failed tests

- [eomt_dinov3](https://github.com/huggingface/transformers/actions/runs/21523990708/job/62023079155):
    tests/models/eomt_dinov3/test_modeling_eomt_dinov3.py::EomtDinov3ForUniversalSegmentationTest::test_sdpa_can_dispatch_on_flash
    tests/models/eomt_dinov3/test_modeling_eomt_dinov3.py::EomtDinov3ForUniversalSegmentationIntegrationTest::test_inference_fp16
    tests/models/eomt_dinov3/test_modeling_eomt_dinov3.py::EomtDinov3ForUniversalSegmentationIntegrationTest::test_semantic_segmentation_inference



@NielsRogge I've fixed the last few smaller things, can you check the failing tests at https://github.com/huggingface/transformers/pull/41212#issuecomment-3824812558 

Moreso the integration tests, if that's a GPU diff (A10) lmk I can update as well in that case - sdpa dispatch one is probably since we are forced to use a mask --> suffices to skip with a comment

run-slow: eomt_dinov3

This comment contains `run-slow`, running the specified jobs: 

models: ["models/eomt_dinov3"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/21588455140)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [2228231d](https://github.com/huggingface/transformers/commit/2228231d1aa9dab02dfc796947d46d2f3df68754) | merge commit |
| PR | [8dfe9736](https://github.com/NielsRogge/transformers/commit/8dfe973665a471908d46b1324ad1ec5c1249adea) | branch commit |
| main | [e4b22baa](https://github.com/huggingface/transformers/commit/e4b22baa9b14560dc37657bd0109563a9bf3a4e3) | base commit |

### Model CI Report

❌ **[3 new failed tests from this PR](https://huggingface.co/datasets/hf-internal-testing/transformers_pr_ci/raw/b4f2c1220670fec6fc6a2fd05c32d3d9d255e950/2026-02-02/runs/27961-21588455140/ci_results_run_models_gpu/new_failures_with_bad_commit.json)** 😭

- [eomt_dinov3](https://github.com/huggingface/transformers/actions/runs/21588455140/job/62203016894):
    tests/models/eomt_dinov3/test_modeling_eomt_dinov3.py::EomtDinov3ForUniversalSegmentationTest::test_sdpa_can_dispatch_on_flash
    tests/models/eomt_dinov3/test_modeling_eomt_dinov3.py::EomtDinov3ForUniversalSegmentationIntegrationTest::test_inference_fp16
    tests/models/eomt_dinov3/test_modeling_eomt_dinov3.py::EomtDinov3ForUniversalSegmentationIntegrationTest::test_semantic_segmentation_inference



@vasqu feel free to handle the integration tests, I assume this is because of the GPU difference

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, eomt, eomt_dinov3, vit

run-slow: eomt, eomt_dinov3, vit

This comment contains `run-slow`, running the specified jobs: 

models: ["models/eomt", "models/eomt_dinov3", "models/vit"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/21597122473)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [f9483f5f](https://github.com/huggingface/transformers/commit/f9483f5fe6ef70e5f78481c19660b8a8bfb41150) | merge commit |
| PR | [56155dd2](https://github.com/NielsRogge/transformers/commit/56155dd2086bc6048347bff2b1feb10179a6ed81) | branch commit |
| main | [645db400](https://github.com/huggingface/transformers/commit/645db400b69e66867f0c011c0ab900042fc0ac6d) | base commit |

✅ No failing test specific to this PR 🎉 👏 !


Merging now, just fixed the last few tests due to GPU diffs resulting in slightly higher atols - updated to a10 numbers where needed (2 tests)