Qwen3VL input_ids = input_ids[attention_mask[i] == 1]
Author: bent1e
Pull request: no
Labels: name: bug

name: Vision

### System Info

`transformers==4.57.1`

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
[rank0]:   File "/root/miniconda3/lib/python3.11/site-packages/transformers/generation/utils.py", line 2787, in _sample
[rank0]:     outputs = model_forward(**model_inputs, return_dict=True)
[rank0]:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:   File "/root/miniconda3/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
[rank0]:     return self._call_impl(*args, **kwargs)
[rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:   File "/root/miniconda3/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1845, in _call_impl
[rank0]:     return inner()
[rank0]:            ^^^^^^^
[rank0]:   File "/root/miniconda3/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1793, in inner
[rank0]:     result = forward_call(*args, **kwargs)
[rank0]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:   File "/root/miniconda3/lib/python3.11/site-packages/transformers/utils/generic.py", line 1064, in wrapper
[rank0]:     outputs = func(self, *args, **kwargs)
[rank0]:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:   File "/root/miniconda3/lib/python3.11/site-packages/transformers/models/qwen3_vl/modeling_qwen3_vl.py", line 1344, in forward
[rank0]:     outputs = self.model(
[rank0]:               ^^^^^^^^^^^
[rank0]:   File "/root/miniconda3/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
[rank0]:     return self._call_impl(*args, **kwargs)
[rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:   File "/root/miniconda3/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
[rank0]:     return forward_call(*args, **kwargs)
[rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:   File "/root/miniconda3/lib/python3.11/site-packages/transformers/utils/generic.py", line 1064, in wrapper
[rank0]:     outputs = func(self, *args, **kwargs)
[rank0]:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:   File "/root/miniconda3/lib/python3.11/site-packages/transformers/models/qwen3_vl/modeling_qwen3_vl.py", line 1201, in forward
[rank0]:     position_ids, rope_deltas = self.get_rope_index(
[rank0]:                                 ^^^^^^^^^^^^^^^^^^^^
[rank0]:   File "/root/miniconda3/lib/python3.11/site-packages/transformers/models/qwen3_vl/modeling_qwen3_vl.py", line 949, in get_rope_index
[rank0]:     input_ids = input_ids[attention_mask[i] == 1]
[rank0]:                 ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: IndexError: The shape of the mask [719] at index 0 does not match the shape of the indexed tensor [1] at index 0
```
### Expected behavior

error

Comments:
Can you provide a tiny reproducer please, it will help a lot to find the root issue?

> Can you provide a tiny reproducer please, it will help a lot to find the root issue?

It's hard for me to provide a reproduction example. I encountered this bug during the training of GRPO, but I found that if I set gradient_checkpointing to false, the error disappeared

Ah, oke, sounds reasonable if the issue is in gradient_checkpointing because GC calls `forward` many times for the same inputs. Do you know if the `config.use_cache` is set to `False` when training? It cannot be `True` with GC otherwise it is going to fail

So Easy.

```python
model.eval()
```

> 啊，好吧，如果问题出在gradient_checkpointing上，听起来很合理，因为GC会`forward`针对相同的输入调用多次。你知道训练时是否`config.use_cache`设置了`False`？不能`True`和GC一起使用，否则会失败。

I followed your suggestion and set use_cache to false, but then the following error appeared:
/root/miniconda3/lib/python3.11/site-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be Nonewarnings.warn(

cc @qgallouedec for vis, I think this has something to do with how TRL internally uses GC and cache in GRPO. Might be also related to the internal slack discussion in https://huggingface.slack.com/archives/C01N44FJDHT/p1760725474636279

Sadly there is no reproducer, but maybe you have already encountered the error

> Ah, oke, sounds reasonable if the issue is in gradient_checkpointing because GC calls `forward` many times for the same inputs. Do you know if the `config.use_cache` is set to `False` when training? It cannot be `True` with GC otherwise it is going to fail



> So Easy.
> 
> model.eval()

I tried your method and it did work, but I'm not sure if it will cause any other problems?

<img width="947" height="190" alt="Image" src="https://github.com/user-attachments/assets/af7122b9-2811-474d-9f62-565084b7dad9" />

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I have encounted the exactly same bug.