GitHub repository: 0xCarbon/Sophia
Description: The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”