GitHub repository: kingoflolz/swarm-jax
Primary language: Python
Created: 2020-12-22T01:15:29Z
Description: Swarm training framework using Haiku + JAX + Ray for layer parallel transformer language models on unreliable, heterogeneous nodes