Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo Paper • 2503.09799 • Published 19 days ago • 13
Eager Updates For Overlapped Communication and Computation in DiLoCo Paper • 2502.12996 • Published Feb 18 • 7
Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch Paper • 2501.18512 • Published Jan 30 • 28
Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2 Paper • 2408.05147 • Published Aug 9, 2024 • 39
BOND: Aligning LLMs with Best-of-N Distillation Paper • 2407.14622 • Published Jul 19, 2024 • 19
WARP: On the Benefits of Weight Averaged Rewarded Policies Paper • 2406.16768 • Published Jun 24, 2024 • 23
Asynchronous Local-SGD Training for Language Modeling Paper • 2401.09135 • Published Jan 17, 2024 • 11
DiLoCo: Distributed Low-Communication Training of Language Models Paper • 2311.08105 • Published Nov 14, 2023 • 15