apply_rotary_pos_emb_flashatt failed during triton jit compilation 'constexpr' object has no attribute 'bit_length'
Author: hebiao064
Pull request: no
Labels: name: bug

### System Info

Hi,

I am trying to run FSDP Training with `Qwen/Qwen2.5-VL-3B-Instruct`, and I hit the error like this:


```
File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py", line 856, in forward
    output = self._fsdp_wrapped_module(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/transformers/utils/generic.py", line 969, in wrapper
    output = func(self, *args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py", line 1908, in forward
    outputs = self.model(
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py", line 1661, in forward
    image_embeds = self.get_image_features(pixel_values, image_grid_thw)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py", line 1614, in get_image_features
    image_embeds = self.visual(pixel_values, grid_thw=image_grid_thw)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py", line 530, in forward
    hidden_states = blk(hidden_states, cu_seqlens=cu_seqlens_now, position_embeddings=position_embeddings)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py", line 856, in forward
    output = self._fsdp_wrapped_module(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py", line 341, in forward
    hidden_states = hidden_states + self.attn(
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py", line 189, in forward
    q, k = apply_rotary_pos_emb_flashatt(q.unsqueeze(0), k.unsqueeze(0), cos, sin)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py", line 156, in apply_rotary_pos_emb_flashatt
    q_embed = apply_rotary_emb(q.float(), cos.float(), sin.float()).type_as(q)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/flash_attn/layers/rotary.py", line 121, in apply_rotary_emb
    return ApplyRotaryEmb.apply(
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/autograd/function.py", line 575, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/flash_attn/layers/rotary.py", line 51, in forward
    out = apply_rotary(
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/flash_attn/ops/triton/rotary.py", line 159, in apply_rotary
    torch.library.wrap_triton(rotary_kernel)[grid](
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/_higher_order_ops/triton_kernel_wrap.py", line 1812, in __call__
    return tracing_triton_hopifier_singleton.call_triton_kernel(
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/_higher_order_ops/triton_kernel_wrap.py", line 1670, in call_triton_kernel
    return self.call_HOP(variable, grids, combined_args_raw, tx)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/_higher_order_ops/triton_kernel_wrap.py", line 1766, in call_HOP
    return triton_kernel_wrapper_mutation(
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/_higher_order_ops/triton_kernel_wrap.py", line 783, in __call__
    return super().__call__(
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/_ops.py", line 471, in __call__
    return wrapper()
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/_ops.py", line 467, in wrapper
    return self.dispatch(
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/_ops.py", line 455, in dispatch
    return kernel(*args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/torch/_higher_order_ops/triton_kernel_wrap.py", line 886, in triton_kernel_wrapper_mutation_dense
    kernel[grid_fn](*args, **kwargs, **constant_args)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/triton/runtime/jit.py", line 347, in <lambda>
    return lambda *args, **kwargs: self.run(grid=grid, warmup=False, *args, **kwargs)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/triton/runtime/jit.py", line 569, in run
    kernel = self.compile(src, target=target, options=options.__dict__)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/triton/compiler/compiler.py", line 278, in compile
    module = src.make_ir(options, codegen_fns, module_map, context)
  File "/home/jobuser/sglang/venv/lib/python3.10/site-packages/triton/compiler/compiler.py", line 81, in make_ir
    return ast_to_ttir(self.fn, self, context=context, options=options, codegen_fns=codegen_fns,
triton.compiler.errors.CompilationError: at 32:28:
    # Meta-parameters
    # We want ROTARY_DIM to be constexpr, otherwise the triton compiler doesn't know that
    # the mask is constant every 8 elements, and it will generate LDG.16 instead of LDG.128
    ROTARY_DIM: tl.constexpr,
    IS_SEQLEN_OFFSETS_TENSOR: tl.constexpr,
    IS_VARLEN: tl.constexpr,
    INTERLEAVED: tl.constexpr,
    CONJUGATE: tl.constexpr,
    BLOCK_H: tl.constexpr,
    BLOCK_M: tl.constexpr,
):
    BLOCK_K: tl.constexpr = triton.next_power_of_2(ROTARY_DIM)
                            ^
AttributeError("'constexpr' object has no attribute 'bit_length'")
```


This is my env:
```
transformers                       4.52.3
flash-attn                         2.8.0.post2
flashinfer-python                  0.2.6.post1
torch                              2.7.1
```

Wonder if you have any insight about this weird compilation issue?

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

.

### Expected behavior

Model forward should work without issues

Comments:
Fall back fa to 2.7.4.post1 can fix it in my program

> Fall back fa to 2.7.4.post1 can fix it in my program

I need to use torch 2.7 due to other software's limitation, so i have to use flash-attn >= 2.8.0.post2 

:(

https://github.com/Dao-AILab/flash-attention/issues/1644#issuecomment-2899396361

Flash attn 2.7.4's build is compatible with pytorch 2.7

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.