Repository: huggingface/transformers
Pull request: fix(models): Forward timm model kwargs to timm.create_model for OmDet-Turbo

### What does this PR do?

The following issue was identified and fixed in this PR:

→ [This PR (🚨 Delete duplicate code in backbone utils)](https://github.com/huggingface/transformers/pull/43323) structured config loading to use [BackboneMixin.consolidate_backbone_kwargs_to_config](https://github.com/huggingface/transformers/blob/main/src/transformers/backbone_utils.py#L273). For the DETR-family; the current state works correctly because [timm_default_kwargs](https://github.com/huggingface/transformers/blob/main/src/transformers/models/detr/configuration_detr.py#L95-L100) only contains keys that map to [TimmBackboneConfig.init](https://github.com/huggingface/transformers/blob/main/src/transformers/models/timm_backbone/configuration_timm_backbone.py#L52-L60). There might be others; but OmDet-Turbo is a model [that passes kwargs meant for timm.create_model itself](https://github.com/huggingface/transformers/blob/main/src/transformers/models/omdet_turbo/configuration_omdet_turbo.py#L150-L154), and which are not `TimmBackboneConfig` params and were dropped.
→ From the prev PR's diff, before the refactor, the implementation forwarded these params via `**kwargs` to `timm.create_model` and it worked before, but after the refactor they were stored as attributes on `PreTrainedConfig` and never forwarded, and parameters like `img_size` are ignored.

Fixes #44610.

cc: @zucchini-nlp

**[CI Failures](https://github.com/huggingface/transformers/actions/runs/22934851265)**

**Before the fix (feel free to cross-check; these errors are reproducible):**

<img width="500" height="500" alt="Image" src="https://github.com/user-attachments/assets/6ae9830c-2968-4f16-958f-a125dbac8d57" /><br>

**After the fix (feel free to cross-check):**

<img width="500" height="500" alt="2" src="https://github.com/user-attachments/assets/f5c855ae-e5cd-4212-ace9-e1fdfd088f45" />

### Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you fix any necessary existing tests?

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Changes timm backbone construction and argument forwarding; while scoped to an optional `timm_model_kwargs` dict, it can affect any model relying on `TimmBackbone` if those kwargs are set or collide with existing `**kwargs`.
> 
> **Overview**
> Fixes OmDet-Turbo’s timm backbone initialization by storing timm-only parameters (e.g. `img_size`, `always_partition`) under `backbone_config.timm_model_kwargs` instead of top-level config fields.
> 
> Updates `TimmBackbone` to forward `config.timm_model_kwargs` into `timm.create_model`, and adds a backward-compatibility shim to migrate older hub configs that had `img_size`/`always_partition` as direct attributes.
> 
> <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 414ee30d2c42077e07d0f0b1d582b65b25606f09. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Changed files:
filename: src/transformers/models/omdet_turbo/configuration_omdet_turbo.py
patch: @@ -161,6 +161,13 @@ def __init__(
             **kwargs,
         )
 
+        # Extract timm.create_model kwargs; TimmBackbone doesn't forward arbitrary config attrs to timm
+        timm_kwargs = {}
+        if getattr(backbone_config, "model_type", None) == "timm_backbone":
+            for attr in ("img_size", "always_partition"):
+                if hasattr(backbone_config, attr):
+                    timm_kwargs[attr] = getattr(backbone_config, attr)
+
         if text_config is None:
             logger.info("`text_config` is `None`. Initializing the config with the default `clip_text_model`")
             text_config = CONFIG_MAPPING["clip_text_model"]()
@@ -212,8 +219,14 @@ def __init__(
         self.eval_size = eval_size
         self.learn_initial_query = learn_initial_query
         self.cache_size = cache_size
+        self.timm_kwargs = timm_kwargs
 
         super().__init__(is_encoder_decoder=is_encoder_decoder, **kwargs)
 
+    def to_dict(self):
+        output = super().to_dict()
+        output.pop("timm_kwargs", None)
+        return output
+
 
 __all__ = ["OmDetTurboConfig"]

filename: src/transformers/models/omdet_turbo/modeling_omdet_turbo.py
patch: @@ -26,7 +26,6 @@
 
 from ... import initialization as init
 from ...activations import ACT2CLS, ACT2FN
-from ...backbone_utils import load_backbone
 from ...integrations import use_kernel_forward_from_hub
 from ...masking_utils import create_bidirectional_mask
 from ...modeling_layers import GradientCheckpointingLayer
@@ -39,7 +38,7 @@
     logging,
     torch_compilable_check,
 )
-from ..auto import AutoModel
+from ..auto import AutoBackbone, AutoModel
 from .configuration_omdet_turbo import OmDetTurboConfig
 
 
@@ -279,7 +278,7 @@ class OmDetTurboVisionBackbone(nn.Module):
     def __init__(self, config: OmDetTurboConfig):
         super().__init__()
         self.apply_layernorm_after_vision_backbone = config.apply_layernorm_after_vision_backbone
-        self.vision_backbone = load_backbone(config)
+        self.vision_backbone = AutoBackbone.from_config(config.backbone_config, **getattr(config, "timm_kwargs", {}))
         self.layer_norms = nn.ModuleList(
             [nn.LayerNorm(in_channel_dim, eps=config.layer_norm_eps) for in_channel_dim in config.encoder_in_channels]
         )