Add ApplyRoPE and RMSNorm kernels written in OpenAI Triton
#6
by
wangzihan99
- opened
No description provided.
wangzihan99
changed pull request title from
Add fused ApplyRoPE and RMSNorm kernels written in OpenAI Triton
to Add ApplyRoPE and RMSNorm kernels written in OpenAI Triton
This PR add kernels of ApplyRoPE and RMSNorm written in OpenAI Triton. These kernels offer better performance, support a wider range of GPU architectures (including V100 and T4), and requires no pre-compilation of kernels compared with flash-attn. They are enabled automatically if Triton is installed (usually bundled with PyTorch 2.x).
wangzihan99
changed pull request status to
open
wangzihan99
changed pull request status to
closed