GitHub repository: yspkm/pretraining-llm
Primary language: Python
Description: This repository implements a scalable language model training pipeline. It includes modules for data processing, model definition, training loop, and logging, serving as a study for pretraining llm.