GitHub repository: simudt/Ring-Attention-Jax
Primary language: Python
Description: Packaged Ring Attention with Blockwise Transformers for Near-Infinite Context implemented in Jax + Flax.