GitHub repository: lucaslingle/mu_transformer
Primary language: Python
Description: Transformer with Mu-Parameterization, implemented in Jax/Flax. Supports FSDP on TPU pods.