OLMo-2-1B Midtrained
Collection
Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting
Ishaan Watts*, Catherine Li*, Sachin Goyal, Jacob Mitchell Springer, Aditi Raghunathan • 3 items • Updated
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
OLMo-2-1B checkpoint midtrained with SAM optimizer with rho = 5e-2.
Paper:"Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting" Ishaan Watts*, Catherine Li*, Sachin Goyal, Jacob Mitchell Springer, Aditi Raghunathan ICML2026 https://arxiv.org/abs/2605.02105
license: cc-by-4.0 datasets: - allenai/dolmino-mix-1124 base_model: - allenai/OLMo-2-0425-1B
- Downloads last month
- 17
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support