GitHub repository: sriprasanna/swarm-jax
Primary language: Python
Description: Swarm training framework using Haiku + JAX + Ray for layer parallel transformer language models on unreliable, heterogeneous nodes