GitHub repository: 6un9-h0-Dan/gpt-neo
Description: An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.