[Bug] Qwen3OmniMoe Loading fails in transformers v5
Author: HuiyingLi
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 5.0.0.dev0
- Platform: Linux-5.15.0-1063-nvidia-x86_64-with-glibc2.39
- Python version: 3.12.3
- Huggingface_hub version: 1.2.1
- Safetensors version: 0.6.2
- Accelerate version: 1.11.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.0a0+50eac811a6.nv25.09 (CUDA)
- Using distributed or parallel set-up in script?: no
- Using GPU in script?: yes
- GPU type: NVIDIA H100 80GB HBM3

transformers commit: a7f29523361b2cc12e51c1f5133d95f122f6f45c

### Who can help?

@zucchini-nlp @ArthurZucker @CyrilVallez

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Script to reproduce:
```
from transformers import AutoModelForMultimodalLM
model = AutoModelForMultimodalLM.from_pretrained(
    "Qwen/Qwen3-Omni-30B-A3B-Instruct",
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)
```

Error:
```
Traceback (most recent call last):
  File "/users/huiyingl/auto/Automodel-omni/transformers5rc1/load_omni.py", line 6, in <module>
    model = AutoModelForMultimodalLM.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/users/huiyingl/auto/Automodel-omni/transformers5rc1/src/transformers/models/auto/auto_factory.py", line 373, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/users/huiyingl/auto/Automodel-omni/transformers5rc1/src/transformers/modeling_utils.py", line 3951, in from_pretrained
    model, missing_keys, unexpected_keys, mismatched_keys, offload_index, error_msgs = cls._load_pretrained_model(
                                                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/users/huiyingl/auto/Automodel-omni/transformers5rc1/src/transformers/modeling_utils.py", line 4118, in _load_pretrained_model
    model.mark_tied_weights_as_initialized()
  File "/users/huiyingl/auto/Automodel-omni/transformers5rc1/src/transformers/modeling_utils.py", line 4437, in mark_tied_weights_as_initialized
    param = self.get_parameter(tied_param)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 839, in get_parameter
    mod: torch.nn.Module = self.get_submodule(module_path)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 725, in get_submodule
    raise AttributeError(
AttributeError: Qwen3OmniMoeTalkerForConditionalGeneration has no attribute `lm_head`
```


### Errors
Error 1: AttributeError: Qwen3OmniMoeTalkerForConditionalGeneration has no attribute 'lm_head'

Error 2: (after bypassing Error 1) AttributeError: 'Qwen3OmniMoeTalkerConfig' object has no attribute 'initializer_range'

### Root Cause Analysis
Issue 1: Incorrect _tied_weights_keys in Qwen3OmniMoeTalkerForConditionalGeneration
Talker model does not have `lm_head` and is not tied weight. But the modeling file: https://github.com/huggingface/transformers/blob/a7f29523361b2cc12e51c1f5133d95f122f6f45c/src/transformers/models/qwen3_omni_moe/modeling_qwen3_omni_moe.py#L3045

Issue 2: Missing initializer_range in composite config classes
The following config classes are missing initializer_range: `Qwen3OmniMoeTalkerConfig`, `Qwen3OmniMoeCode2WavConfig`, `Qwen3OmniMoeConfig`
When _initialize_missing_keys() runs during loading, it calls initialize_weights() on modules without _is_hf_initialized. This eventually calls _init_weights() which accesses self.config.initializer_range, causing the error.


### Expected behavior

Model loads successfully

Comments:
i want to workaround this bug. 

reviewed the linked PR!

PR is merged we can close