lm_head parameters missing from named_parameters() in Qwen2.5-VL-3B-Instruct model
Author: Buhua-Liu
Pull request: no
Labels: name: bug

### System Info

```
- `transformers` version: 4.49.0
- Platform: Linux-6.8.0-40-generic-x86_64-with-glibc2.35
- Python version: 3.10.16
- Huggingface_hub version: 0.27.1
- Safetensors version: 0.5.0
- Accelerate version: 1.0.1
- Accelerate config:    - compute_environment: LOCAL_MACHINE
        - distributed_type: DEEPSPEED
        - use_cpu: False
        - debug: False
        - num_processes: 8
        - machine_rank: 0
        - num_machines: 1
        - rdzv_backend: static
        - same_network: True
        - main_training_function: main
        - enable_cpu_affinity: False
        - deepspeed_config: {'deepspeed_config_file': 'LLaMA-Factory/examples/deepspeed/ds_model_parallel_config.json', 'zero3_init_flag': True}
        - downcast_bf16: no
        - tpu_use_cluster: False
        - tpu_use_sudo: False
        - tpu_env: []
- DeepSpeed version: 0.15.4
- PyTorch version (GPU?): 2.5.1+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: No
- GPU type: NVIDIA H200
```



### Who can help?

## 🐛 Bug Description

When loading the **Qwen2.5-VL-3B-Instruct** model from Hugging Face, the `lm_head` parameters (`lm_head.weight` and `lm_head.bias`) **do not appear** in `named_parameters()`, although they correctly appear in `state_dict()`.

This behavior differs from other Qwen-2.5-VL models (**Qwen2.5-VL-7B-Instruct**, **Qwen2.5-VL-72B-Instruct**), creating inconvenience during fine-tuning, optimizer setup, and parameter freezing tasks.

@amyeroberts, @qubvel 

---

## 📌 Additional Context

- It appears the issue is related to how `lm_head` is registered within the model structure.
- Manually accessing `model.lm_head` works, but this is inconsistent with standard practice.

---

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import Qwen2_5_VLForConditionalGeneration

model_name = "Qwen/Qwen2.5-VL-3B-Instruct"
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
        model_name,
        torch_dtype="auto",
        device_map="auto"
    )

# Check named_parameters for lm_head
has_lm_head_in_named_params = any("lm_head" in name for name, _ in model.named_parameters())
print(f"lm_head in named_parameters(): {has_lm_head_in_named_params}")

# Check state_dict for lm_head
has_lm_head_in_state_dict = any("lm_head" in key for key in model.state_dict().keys())
print(f"lm_head in state_dict(): {has_lm_head_in_state_dict}")
```

### Output:

```bash
lm_head in named_parameters(): False
lm_head in state_dict(): True
```


### Expected behavior

The `lm_head` parameters should appear in both `named_parameters()` and `state_dict()` outputs consistently, similar to other Qwen-2.5-VL models.

Example expected output:

```bash
lm_head in named_parameters(): True
lm_head in state_dict(): True
```

Comments:
cc @zucchini-nlp 

Hi Buhua @Buhua-Liu , after reading the source code and checking some documentations, I noticed this behavior may actually be expected. The [implementation](https://github.com/huggingface/transformers/blob/fc8764c9a618add64c33e83720f974750bcd0978/src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py#L1511) of `Qwen2_5_VLForConditionalGeneration` model uses weight tying to tie the output layer weights (lm_head) with the input embedding layer (embed_tokens) to improve efficiency:
```python
class Qwen2_5_VLForConditionalGeneration(Qwen2_5_VLPreTrainedModel, GenerationMixin):
    _tied_weights_keys = ["lm_head.weight"]
    config_class = Qwen2_5_VLConfig
    _no_split_modules = ["Qwen2_5_VLDecoderLayer", "Qwen2_5_VLVisionBlock"]

    def __init__(self, config):
        super().__init__(config)
        ...
```
You may try the following script to verify:
```python
# Check named_parameters for embed_token
embed_token_in_named_params = any("model.embed_tokens.weight" in name for name, _ in model.named_parameters())
print(f"embed_token weights in amed_parameters(): {embed_token_in_named_params}")

# Check embed_token is equal to lm_head
embed_token_eq_lm_head = torch.all(model.state_dict()['model.embed_tokens.weight'] == model.state_dict()['lm_head.weight'])
print(f"embed_token weights equal to lm_head weights: {embed_token_eq_lm_head}")
```
Hope this helps.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.