Let's break Qwen-VL 🚨  
Author: zucchini-nlp
Pull request: yes
Labels: name: for_v5?

# What does this PR do?

As per title, break the cycle. Qwen will no longer have access to text attributes with `config.vocab_size` which solves the duplicated attribute issue we had in TRL. A config should not have two different values for the same key

Tested with the TRL issue that there is no regression and ran slow test locally

Comments:
run-slow: qwen2_5_vl, qwen2_vl

This comment contains `run-slow`, running the specified jobs: 

models: ["models/qwen2_5_vl", "models/qwen2_vl"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42420). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/19702531268)

✅ No failing test specific to this PR 🎉 !


run-slow: qwen2_5_vl, qwen2_vl

This comment contains `run-slow`, running the specified jobs: 

models: ["models/qwen2_5_vl", "models/qwen2_vl"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/19703491970)

### Model CI Report

#### ❌ Failed tests

- [qwen2_5_vl](https://github.com/huggingface/transformers/actions/runs/19703491970/job/56445857245):
    tests/models/qwen2_5_vl/test_modeling_qwen2_5_vl.py::Qwen2_5_VLIntegrationTest::test_small_model_integration_test
    tests/models/qwen2_5_vl/test_modeling_qwen2_5_vl.py::Qwen2_5_VLIntegrationTest::test_small_model_integration_test_batch
    tests/models/qwen2_5_vl/test_modeling_qwen2_5_vl.py::Qwen2_5_VLIntegrationTest::test_small_model_integration_test_batch_wo_image
    tests/models/qwen2_5_vl/test_modeling_qwen2_5_vl.py::Qwen2_5_VLIntegrationTest::test_small_model_integration_test_expand

- [qwen2_vl](https://github.com/huggingface/transformers/actions/runs/19703491970/job/56445857187):
    tests/models/qwen2_vl/test_modeling_qwen2_vl.py::Qwen2VLIntegrationTest::test_small_model_integration_test
    tests/models/qwen2_vl/test_modeling_qwen2_vl.py::Qwen2VLIntegrationTest::test_small_model_integration_test_batch
    tests/models/qwen2_vl/test_modeling_qwen2_vl.py::Qwen2VLIntegrationTest::test_small_model_integration_test_batch_different_resolutions
    tests/models/qwen2_vl/test_modeling_qwen2_vl.py::Qwen2VLIntegrationTest::test_small_model_integration_test_batch_wo_image
    tests/models/qwen2_vl/test_modeling_qwen2_vl.py::Qwen2VLIntegrationTest::test_small_model_integration_test_expand



run-slow: qwen2_5_vl, qwen2_vl

This comment contains `run-slow`, running the specified jobs: 

models: ["models/qwen2_5_vl", "models/qwen2_vl"]
quantizations: []

run-slow: qwen2_5_vl, qwen2_vl




run-slow: qwen2_5_vl, qwen2_vl




run-slow: qwen2_5_vl, qwen2_vl



## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/19717108197)

⚠️ No test being reported (jobs are skipped or cancelled)!


This comment contains `run-slow`, running the specified jobs: 

models: ["models/qwen2_5_vl", "models/qwen2_vl"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/19718024830)

✅ No failing test specific to this PR 🎉 !


Thanks for addressing this issue, @zucchini-nlp. :hugs: 

For completeness, here is the original issue we opened in TRL that tracked this problem:
- https://github.com/huggingface/transformers/issues/41020
  - More concretely, this comment: https://github.com/huggingface/transformers/issues/41020#issuecomment-3330098710

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen2_5_vl, qwen2_vl