Qwen3-VL hidden_states are not the same used to compute logits
Author: LautaroEst
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.57.1
- Platform: Linux-5.15.0-161-generic-x86_64-with-glibc2.35
- Python version: 3.10.19
- Huggingface_hub version: 0.36.0
- Safetensors version: 0.6.2
- Accelerate version: 1.12.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.1+cu128 (NA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: No

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Run the following self-contained script (no access-key required):

```Python

MODEL_NAME = "Qwen/Qwen3-VL-2B-Instruct"

# Load model + processor
processor = AutoProcessor.from_pretrained(MODEL_NAME, trust_remote_code=True, cache_dir=os.environ["TRANSFORMERS_CACHE"])
model = AutoModelForImageTextToText.from_pretrained(
    MODEL_NAME,
    dtype=torch.float16,
    trust_remote_code=True,
)
model.eval()
processor = AutoProcessor.from_pretrained(MODEL_NAME)

# Create input tensors
dummy_prompt = "This is a sample prompt."
messages = [{"role": "user", "content": [{"type": "text", "text": dummy_prompt,}]}]
inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt"
)
inputs = inputs.to(model.device)

# Inference
outputs = model(**inputs, return_dict=True, output_hidden_states=True)
logits = model.lm_head(outputs.hidden_states[-1])
logits2 = outputs.logits

# Check if outputs are equal
assert torch.equal(logits, logits2)
```

### Expected behavior

Hello. 

I've noticed that the implementation of the [`forward`](https://github.com/huggingface/transformers/blob/v5.0.0rc0/src/transformers/models/qwen3_vl/modeling_qwen3_vl.py#L1325) method of the `Qwen3VLForConditionalGeneration` module do not explicitely outputs the `hidden_states` tuple while the [`forward`](https://github.com/huggingface/transformers/blob/v5.0.0rc0/src/transformers/models/qwen2_vl/modeling_qwen2_vl.py#L1294) method of the `Qwen2VLForConditionalGeneration` module does.

This is causing a difference between the expected logits output by the model. Specifically, the above code produce an assertion error, but when I change the `MODEL_NAME` variable to `Qwen/Qwen2-VL-2B-Instruct` the error goes away.

I believe that this can be solved by explicitly adding the `outputs.hidden_states` tuple to the [`Qwen3VLCausalLMOutputWithPast`](https://github.com/huggingface/transformers/blob/v5.0.0rc0/src/transformers/models/qwen3_vl/modeling_qwen3_vl.py#L1413) instance, exactly as it is done for the [`Qwen2VLCausalLMOutputWithPast`](https://github.com/huggingface/transformers/blob/v5.0.0rc0/src/transformers/models/qwen2_vl/modeling_qwen2_vl.py#L1399) instance. Is there a reason why they are implemented differently?

Comments:
Yeah, it seems like we're just not populating `hidden_states` in `Qwen3VLForConditionalGeneration`! @zucchini-nlp I assume that's just a bug, but checking in case there's a reason for it

Hm, might be smth wrong with `check_model_inputs`. Usually we don't use it for multimodals because multomodals don't have access to backbone class names before init


@zucchini-nlp it seems like they're just not passed to the output object!

<img width="935" height="383" alt="Image" src="https://github.com/user-attachments/assets/283efc08-8def-4750-8a1b-b9a0fb591f22" />

We can probably just add an `output_hidden_states` argument and pass `hidden_states` to `Qwen3VLCausalLMOutputWithPast`?

Yeah, right, and replace `check_model_inputs` with `can_return_tuple`. I'm quite sure they were omitted with assumption that the decorator will handle everything automatically. But it did not

Thanks to the detailed issue and the helpful discussion, I've opened a PR hoping it's what you wanted to do.