GitHub repository: Conjecture-Inc/parallel-pytorch
Primary language: Python
Description: A (WIP) lightweight implementation of DeepSpeed/Megatron-ML style 3D parallelism, along with some models and helpful utilities.