precision-optimizer-30M

Precision x optimizer study: 30M OLMo-core models trained with AdamW or Muon, in BF16 or with 6-bit fake quantization (naive integer and QuEST-style variants), on Dolma CC.

Layout

One directory per training run, named after the run:

• <run>/final-unsharded/model.pt: final model state dict (OLMo-core format).
• <run>/final-unsharded/config.json: model config.
• <run>/launch.yaml: the full training config.
• <run>/*.json: per-run measurements (final loss, Gaussian weight-perturbation sweeps, post-training quantization, etc.). Field names follow the project code.

Run names encode model size (30M, ...), training tokens (D<tokens>B), training precision (bf16, w<bits> weight-only, f<bits> weights + activations), learning rate and seed. Training checkpoints and training data are not included.