GitHub repository: superckid/Ring-Attention-Jax
Primary language: Python
Description: Packaged Ring Attention with Blockwise Transformers for Near-Infinite Context implemented in Jax + Flax.