Slayer1B 1.01 — training checkpoints
Full trainer checkpoints of the run that produced Slayer1B 1.01: continued training of
Slayer1B 1.00 from its last checkpoint before the learning-rate decay
(ckpt_1050.pt), with a new total budget of twice the
tokens per parameter of 1.00. Each file holds the model weights and the optimizer state.
| Start | 1.00 ckpt_1050.pt (step 1050, constant lr 6e-4) |
| Steps | 2860 in total × 1 048 576 tokens = 2 998 927 360 tokens (≈ 2.8 tokens per parameter; 1.00: 1430 steps) |
| Schedule | constant lr 6e-4 through step 2289, then 1 − sqrt decay over steps 2290–2860 to lr / 10 |
| Data | steps 1051–1801: the same pack-00 stream as 1.00 (same windows); steps 1802–2860: pack-01 of SlayerLab/slayer-pl-8x3b (fresh web slice; shared components disjoint from those already used) |
| Last checkpoint before the decay | ckpt_2250.pt — starting point for further continued training |
| Training curve and evaluations during training | https://track.fabryka.ai/run/4934429a-d393-42e0-8aac-03eae6b71ff7 |
Checkpoints every 150 steps from 1200 to 2850, plus the final step 2860. Released model (weights in safetensors) and model card: SlayerLab/Slayer1B-1.01.
File format
torch.save dictionary of the trainer train_gpt_ref_r23.py (in the 1.00 model repository): model, opt (Muon +
AdamW), step, config. Load with torch.load(path, map_location="cpu", weights_only=True).
Run record (run/)
metrics.jsonl (training loss and learning rate every 10 steps), train.log, train.out, args.txt (trainer
arguments), checkpoints_SHA256SUMS (written on the node at save time), data manifests and sums of both token streams
(wsad_v2_*: pack-00 stream, wsad_p01_*: pack-01 stream), 13gram_p01.json (13-gram overlap of the pack-01 stream
with the evaluation sets), pip_freeze.txt (software environment).
SHA-256
Recorded off the training node at the moment each checkpoint was written, before upload; files are downloaded back and checked against these sums.
3637f1d90f60c2151724ad88b4b7c52925c87b7330a7680974b9093e8c1b4021 ckpt_1200.pt
d6d75584ffbd7261ed23f489ed4927d3e33fc3a6fef3bd9d3ad46b309b63daf4 ckpt_1350.pt
3b2af30f8f4331f7bf549b1e7f0b2643ac2ec8ad3369a42b579d578930ba2934 ckpt_1500.pt
db254bfb77e302fc618f9cc7e3c4e23f1e3d6923386e54ad5fb930ff78a04bc4 ckpt_1650.pt
9591dfa0d41f68b9e5cc40db2caf6f601b102c31b37c76d2ddeffa9f26cec49e ckpt_1800.pt
6b8037e19b3fc42bb1c70599754749cdbb6a507a27fa4e2c3e316d676e969555 ckpt_1950.pt
b46f5e02f74dab2db1055fa009668c784411f403e01198dfff36deae9a0200f5 ckpt_2100.pt
aeddfb023abb11c8654b47c728c9680c232d1b6568697e3a1eee8128efd37054 ckpt_2250.pt
616df6070fbc6e338e106d4b535030d06767063168e80494b67f508005904b73 ckpt_2400.pt
82fd3d07914308411a2ddec965ce6881f296b8bb6956369096534bdc2c45b0c8 ckpt_2550.pt
e2b5b29f05d3113c0bc95a5a4d74fe07fee057031068bc82844f129a341c09f0 ckpt_2700.pt
decd0f02487fa15c65aaaaf9e2c2320f196571efa0337c8c588705915f7fd315 ckpt_2850.pt
b0a288798ae614c38e37ff8176a293f6650cec2221d59e46d5116afac7c31eba ckpt_2860.pt
License
Apache-2.0.