GitHub repository: mtanghu/Additive-Attention-Is-All-You-Need
Primary language: Jupyter Notebook
Description: Additive Attention for causal language modeling with linear sequence complexity, O(1) path length, and O(1) inference