GitHub repository: simudt/ring-attention-jax
Primary language: Python
Created: 2024-03-05T17:08:55Z
Description: Packaged Ring Attention with Blockwise Transformers for Near-Infinite Context implemented in Jax.