RuntimeError with use_remove_padding=True in GRPO training for Qwen2.5-VL due to tensor shape mismatch in transformers==4.53.3
Author: zhihaofang1017
Pull request: no
Labels: name: bug

### System Info

NPU & Ascend Driver:

NPU: 16x Ascend 910B2 64GB 

CANN Version: 8.2.RC1

Python Version: Python 3.11.9

Key Libraries:

verl: git+[https://github.com/volcengine/verl@8e1fc24](https://www.google.com/search?q=https://github.com/volcengine/verl%408e1fc24&authuser=1)

transformers: 4.53.3

torch: 2.5.1+cpu

torch-npu: 2.5.1

accelerate: 1.10.0

bytedray:2.10.0.32

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

https://github.com/volcengine/verl/blob/main/examples/grpo_trainer/run_qwen2_5_vl_7b_npu.sh

### Expected behavior

I am trying to run the GRPO training example for the Qwen2.5-VL model using the script in VeRL(https://github.com/volcengine/verl/blob/main/examples/grpo_trainer/run_qwen2_5_vl_7b_npu.sh). 

The core error message is:
(TaskRunner pid=1205760, ip=[2605:340:cd51:4900:f78:21b8:4971:440b])   File "/home/tiger/.local/lib/python3.11/site-packages/transformers/masking_utils.py", line 704, in _preprocess_mask_arguments
(TaskRunner pid=1205760, ip=[2605:340:cd51:4900:f78:21b8:4971:440b])     position_ids = position_ids.expand(batch_size, -1)
(TaskRunner pid=1205760, ip=[2605:340:cd51:4900:f78:21b8:4971:440b])                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(TaskRunner pid=1205760, ip=[2605:340:cd51:4900:f78:21b8:4971:440b]) RuntimeError: expand(npuLongType{[3, 1, 717]}, size=[1, -1]): the number of sizes provided (2) must be greater or equal to the number of dimensions in the tensor (3)

<img width="2288" height="970" alt="Image" src="https://github.com/user-attachments/assets/bc6752a1-f83b-4490-8462-a652f4c2502d" />

However, the training process fails at step 0 with a RuntimeError related to a tensor dimension mismatch inside the model's forward pass.

Through further debugging, I've discovered a key finding: this error only occurs when use_remove_padding=True is set in the configuration. If I set use_remove_padding=False, the training proceeds without this specific error.





Comments:
We can't really debug issues in VERL, unfortunately! We won't be able to identify and fix any bug unless there's a reproducer script in pure Transformers that shows the problem

Might be fixed by https://github.com/huggingface/transformers/pull/40490 if the inputs are not packed tensors (can you checkout to branch and verify). Otherwise +1 to @Rocketknight1's comment, without a reproducer we can't help

> We can't really debug issues in VERL, unfortunately! We won't be able to identify and fix any bug unless there's a reproducer script in pure Transformers that shows the problem

Downgrading transformers worked for me，the error disappears when I use transformers: 4.52.4

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.