GitHub repository: Bruce-Lee-LY/decoding_attention
Primary language: C++
Description: Decoding Attention is specially optimized for multi head attention (MHA) using CUDA core for the decoding stage of LLM inference.