Slayer1B 1.02 — training checkpoints
Full trainer checkpoints of the run that produces Slayer1B 1.02: continued training of
Slayer1B 1.01 from its last checkpoint before the learning-rate decay
(ckpt_2250.pt, sha256 aeddfb02…), with a new total
budget of twice the tokens of 1.01. Each file holds the model weights and the optimizer state.
| Start | 1.01 ckpt_2250.pt (step 2250, constant lr 6e-4) |
| Steps | 5720 in total × 1 048 576 tokens = 5 997 854 720 tokens (≈ 5.6 tokens per parameter; 1.01: 2860 steps) |
| Schedule | constant lr 6e-4 to the end (no final decay). The run started with a planned 1 − sqrt decay from step 4578; at step 3300, still in the constant phase, it was resumed with the decay removed (training loss at step 3310 before and after the resume equal to four decimal places). The decay is deferred to the end of the whole Slayer1B line. |
| Data | steps 1–1801: pack-00 stream (as 1.00 and 1.01); 1802–3594: pack-01 stream (as 1.01, continued); 3595–5389: pack-02; 5390–5720: pack-03 — all from SlayerLab/slayer-pl-8x3b revision d50df04a; shared components of each pack disjoint from those already used |
| Checkpoints | every 150 steps from 2400, step 4576 and the final step 5720; every checkpoint is a constant-lr state and can start further continued training |
| Trainer | train_gpt_ref_r24.py = the 1.00/1.01 trainer plus a chain of data streams and extra checkpoint steps; batches up to step 3594 are identical to those of 1.01's data order |
| Training curve and evaluations during training | https://track.fabryka.ai/run/21ae1234-82c2-42b7-887b-cbcc69901ae1 |
The run finished at step 5720 (all 25 checkpoints listed below). Released model and model card: SlayerLab/Slayer1B-1.02.
Run record (run/)
metrics.jsonl (training loss and learning rate every 10 steps), train.log, train.out, args.txt (trainer arguments
after the change at step 3300), args_before_step_3300.txt and train_before_step_3300.log (the same run before the
decay was removed), checkpoints_SHA256SUMS (written on the node at save time), batch_check_before_start.txt
(sha256 of training batches at steps 2251, 3593, 3594, 5388 and 5389 from the new trainer, compared with the 1.01
trainer before the start).
File format
torch.save dictionary of the trainer: model, opt (Muon + AdamW), step, config. Load with
torch.load(path, map_location="cpu", weights_only=True). Model class: train_gpt_ref_r23.py in the 1.00/1.01 model
repositories (the architecture is unchanged).
SHA-256
Recorded off the training node at the moment each checkpoint is written, before upload; files are downloaded back and checked against these sums.
289186300fa010c6918f4f6fd1fcf4e3054707f4d616200f52944d11d58109f8 ckpt_2400.pt
14cd38933fcc2d6565fe9664d735cfc790c19aafb24d2d673f39dcca90a2ad6f ckpt_2550.pt
7b6b7089f203825d413af43cbd06e8ef46f66e66bbf08267b60edd74dc48c25a ckpt_2700.pt
c6883b8589a03f8859003f67cd65cf6ee82f185f3a52008380dcfa54b83c283e ckpt_2850.pt
147fec8fca88dc3aa9e9dfd458cb5f95cba9b2352cafef77ad359d84168a7874 ckpt_3000.pt
5f6f224c418953547de6fb65e12ecfbf3ca6fa803bb713396f4d315608e1e98e ckpt_3150.pt
c8d6873290cffdf089cbdacc3700f5bda3ae5dd974d6442c13d430681057a225 ckpt_3300.pt
df68a26dd8f2b0b22d3acea4cb464d6fe1459a1ab2a2c3d720483e995a87b08c ckpt_3450.pt
d22c2b63c1d8e676cf166637df477b6ccb9d2ba61de906d4da2418fa81834cbe ckpt_3600.pt
b387d6d732b52a02a7b45b00a9853489467200996cfc335d7de6167234ba7d64 ckpt_3750.pt
1330992a4745a4ab72062ae052d3bdc867fd3682ecb1d181cc66b42b9d7cd8f4 ckpt_3900.pt
84e9c06efe7fce214a9ffd4d676eb0ec777fc541838033503fc697b3a77f47a1 ckpt_4050.pt
f7517c553bae4c25131c0db8f3c3841ea218dcd8ad2eaf27d50ec5ec49a02925 ckpt_4200.pt
07d27aa81a056dd081493698cebaf4234759b40004555cad1bf07089374c9710 ckpt_4350.pt
642ec154019ea18bce4dae447e4eaad62bb34304e1be411657fa799148d3d84a ckpt_4500.pt
423f51091c057336b3cfce80e6e64109c634c4efc742c503ca0588b7915ddff2 ckpt_4576.pt
7f3f03f5c63b9bfb655e16652e9685059fa766b14b3d161629a29e78f1cb35f0 ckpt_4650.pt
91f737dc34e91724fa9e4b25a1b1e743c4940dc000123baf77c0e3c2fd34334a ckpt_4800.pt
fa447bce4f93835752c2330fa251c38d12810452cf91e333a3dd96971addc77a ckpt_4950.pt
a519d20a0cc409c715f2a4ebdca841f6f3bc0f3b0ee663600d00ee466b043e51 ckpt_5100.pt
f6faafb27bfbbcfa1122cc904edc4b3dd4a9e890702722af2f2b2eae7163d923 ckpt_5250.pt
7b2f6b18d9b1137c55db98a18f97b46b42f7ef0eb9d72eaeb513e72505e45626 ckpt_5400.pt
896d27baf2f60ebd46710aa420358500355d3d8b74281c1a689f698fbf189ade ckpt_5550.pt
5a830fb09e3374caac8482198fb5ee7c1a3bea7fe8fd7d329cae5df38faa300d ckpt_5700.pt
6b2a9b11df4c1063476d5235475c98d35abbab3be4282a83d9f3613bbcbc805a ckpt_5720.pt
License
Apache-2.0.