ZClip: Adaptive Spike Mitigation for LLM Pre-Training Paper • 2504.02507 • Published 19 days ago • 76
ZClip: Adaptive Spike Mitigation for LLM Pre-Training Paper • 2504.02507 • Published 19 days ago • 76
A Refined Analysis of Massive Activations in LLMs Paper • 2503.22329 • Published 25 days ago • 14
A Refined Analysis of Massive Activations in LLMs Paper • 2503.22329 • Published 25 days ago • 14
Variance Control via Weight Rescaling in LLM Pre-training Paper • 2503.17500 • Published Mar 21 • 5
Variance Control via Weight Rescaling in LLM Pre-training Paper • 2503.17500 • Published Mar 21 • 5
Running 2.5k 2.5k The Ultra-Scale Playbook 🌌 The ultimate guide to training LLM on large GPU Clusters
view article Article Falcon 2: An 11B parameter pretrained language model and VLM, trained on over 5000B tokens tokens and 11 languages By Quent-01 and 9 others • May 24, 2024 • 25