GitHub repository: xrsrke/pipegoose
Primary language: Python
Created: 2023-06-14T06:14:52Z
Description: Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*