`sliding_window` issue with Qwen3-MoE models
Author: saattrupdan
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 5.0.1.dev0
- Platform: Linux-5.15.186.el8-x86_64-with-glibc2.39
- Python version: 3.11.5
- Huggingface_hub version: 1.3.4
- Safetensors version: 0.7.0
- Accelerate version: 1.12.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.1+cu128 (CUDA)
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: Yes
- GPU type: NVIDIA H100 80GB HBM3

### Who can help?

@ArthurZucker @Cyrilvallez

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

To reproduce:

```python
from transformers import AutoConfig
AutoConfig.from_pretrained("Qwen/Qwen3-Omni-30B-A3B-Thinking")
```

Getting this error:

```bash
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/work/.venv/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1405, in from_pretrained
    return config_class.from_dict(config_dict, **unused_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/.venv/lib/python3.11/site-packages/transformers/configuration_utils.py", line 711, in from_dict
    config = cls(**config_dict)
             ^^^^^^^^^^^^^^^^^^
  File "/work/.venv/lib/python3.11/site-packages/transformers/models/qwen3_omni_moe/configuration_qwen3_omni_moe.py", line 1142, in __init__
    self.talker_config = Qwen3OmniMoeTalkerConfig(**talker_config)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/.venv/lib/python3.11/site-packages/transformers/models/qwen3_omni_moe/configuration_qwen3_omni_moe.py", line 913, in __init__
    self.code_predictor_config = Qwen3OmniMoeTalkerCodePredictorConfig()
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/.venv/lib/python3.11/site-packages/transformers/models/qwen3_omni_moe/configuration_qwen3_omni_moe.py", line 592, in __init__
    self.sliding_window = sliding_window if self.use_sliding_window else None
                                            ^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/.venv/lib/python3.11/site-packages/transformers/configuration_utils.py", line 164, in __getattribute__
    return super().__getattribute__(key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Qwen3OmniMoeTalkerCodePredictorConfig' object has no attribute 'use_sliding_window'. Did you mean: 'sliding_window'?
```

Seems like it's all because [this line](https://github.com/huggingface/transformers/blob/83163c6cf65d875178478467b3235cfaee0bf5b0/src/transformers/models/qwen3_omni_moe/configuration_qwen3_omni_moe.py#L584C1-L584C45) should be `self.use_sliding_window = use_sliding_window`, or simply removed, and then the `self.sliding_window` attribute should use `use_sliding_window` instead of `self.sliding_window`. But given that `use_sliding_window` have been removed as an argument to the initialiser, I suppose all mention of it should be removed in the initialiser as well.

### Expected behavior

No errors.

Comments:
I experience the same issue and I agree with the diagnostic done by @saattrupdan .

I used this temporary monkey-patch to get around it:

```
from transformers.models.qwen3_omni_moe.configuration_qwen3_omni_moe import Qwen3OmniMoeTalkerCodePredictorConfig
original_init = Qwen3OmniMoeTalkerCodePredictorConfig.__init__

def patched_init(self, *args, **kwargs):
    # Set use_sliding_window before calling original init to avoid the AttributeError
    # Extract it from kwargs if present, otherwise use default (False)
    use_sliding_window = kwargs.get('use_sliding_window', False)
    self.use_sliding_window = use_sliding_window
    original_init(self, *args, **kwargs)

Qwen3OmniMoeTalkerCodePredictorConfig.__init__ = patched_init
```

It indeed seems to be an unused attribute, open for PRs to remove the irrelevant line (along some small test)