Slayer1B 1.04 — training checkpoints
Full trainer checkpoints of continued pre-training of Slayer1B 1.03
from its final checkpoint (ckpt_7150.pt, sha256
d7cad967…). Each file holds the model weights and the optimizer state.
Status (28 October 2026): complete. Release Slayer1B 1.04 is the
state at step 10010 (ckpt_10010.pt). This repository holds every checkpoint of the 1.04 run (7200–10010, 58 files)
and, because the run continued without a break, also the checkpoints of the next two releases (10050–13848, 78 files):
| release | steps | files | per-release repository |
|---|---|---|---|
| 1.04 | 7151–10010 | 58 | this repository |
| 1.05 | 10011–12870 | 58 | Slayer1B-1.05-checkpoints (same files) |
| 1.06 | 12871–13848 | 20 | Slayer1B-1.06-checkpoints (same files) |
| Start | 1.03 ckpt_7150.pt (step 7150, constant lr 6e-4) |
| Steps | 7151 → 10010, 1 048 576 tokens per step (256 sequences × 4096 tokens) |
| Schedule | constant lr 6e-4, no decay |
| Data | SlayerLab/slayer-pl-8x3b revision d50df04a, continuing the single pass over the corpus: tail of pack-03 (steps 7151–7192), pack-04 from step 7193, pack-05 from step 8991 |
| Leak-scan exclusion | from pack-04 on (step 7193), the 5133 documents with n-gram overlap with BoolQ / MMLU / ARC test items are excluded (list sha256 84caed78…); steps up to 7192 used the unfiltered data |
| Hardware | 4 × NVIDIA L40S (48 GB), DDP. Training paused (budget) at step 8300 and resumed from ckpt_8300.pt |
| Checkpoints | every 50 steps from 7200, and 10010; all are constant-lr states |
| Trainer | train_gpt_ref r24.1 (sha256 881cb055…); model definition train_gpt_ref_r23.py (unchanged). Steps 10011–13848: train_gpt_ref r24.2 (sha256 46f83b51…) on 8 × L40S |
| Training curve and evaluations | https://track.fabryka.ai/run/f056bb5c-f1da-56d5-9776-cc33063f10a5 |
SHA-256 (steps 7200–10010)
Each value equals the LFS object id of the file in this repository (checked anonymously after upload).
| step | sha256 |
|---|---|
| 7200 | dcff686a2b719ff16d5b8bc625b5f77a2d45d207b7d5fbea35cdf411bd32f5fd |
| 7250 | dcecd2d043eae4d6474ec040a61c293846446b9a041970e6ced0f3c5a33f723f |
| 7300 | b59643ebb8e2c68784de6140aa34f1c46d6d05d3e381ec3bc46c60d4af40d249 |
| 7350 | cd0a91338aa512ca399a5df05c0e0e5946cd9f993cdb75ae93745f3c2f3e9bba |
| 7400 | 7d97c13b3c72e1b023626d5253ff030603764dd47c6af259c9c11dd0b6a92b6b |
| 7450 | cbc2b17fa6a716bc970c6e6303396f40f1ab7561595e12ace976e5833edbfe0a |
| 7500 | 20bb66250b63a43ac54adbab63493ffac56b41e919463e13e274908f32781867 |
| 7550 | 0b19d6e08d8b9b7d07bc2a0f0ee86ea0057916b8aaf84d39aec47e0515568e6e |
| 7600 | 37b547147ea6ec0856cb23bc78bec3f78319ac9c45ec63480f833c5335601d59 |
| 7650 | 0cf517a74b8d15ae226dee131c2185053cb0dbde8e5ba8d84a9558b45b0f5ab0 |
| 7700 | 6de388d452035069608033eacd262ce44edc6f4b8e819512320118e3b68989f0 |
| 7750 | 8d8ade244783dded8335a9f5fbe000b215e5af6e8f39d6493016f90ec5946ab0 |
| 7800 | 473893a4c4d1386be504501ac41692359aaf66817837e08bb775e0973b9b9f2c |
| 7850 | c0ec371d48741888918040b88906d3c8077e6213d276fb076047bbe0eb33902b |
| 7900 | 7a5beaa46a025c1bd9b42b43faaae0a6362692a1a66913cc989b73b590fc4cac |
| 7950 | 8f8fb65da465e410495382a7b9f401349cd3d1af9c864bc61b02058288ffb7a3 |
| 8000 | b4b4aa12a8bd7f9b4949226b49e953ed88dded2fddd984a080f1d02c9c1712df |
| 8050 | 2bdc077d7f71ff91e74fbc0deb120c24d41cd63cddcb079100f2fe3ad7b4ecb4 |
| 8100 | 5e2e4dd12bfc5a8c5fa690b442b0062a2b4b559064a57dcb1830e68357548908 |
| 8150 | 05dcdcf26b4d5cbf86b7fd56aaedc3bc7dafe03143a2eb98ee0c3dc6df1e5a19 |
| 8200 | 39d7f609e88cb7f9363fa2920c29d0c13daa948c9911c1f7c5bcd55d619d72dc |
| 8250 | b2c9fa7c9f83b5e4209b1d4c01267fcdf33459fd85255dd21c986c033ff21eef |
| 8300 | 1487a2673b64e26c241ff46a7e98222f9cc6a059a7c509ebd9cf3581bc3c68d6 |
| 8350 | 66f4c2e7946379a9e372be1b2865c788b0008a97afcb73c0b5c9d4e341caa9f3 |
| 8400 | 112c3cf9878f65cdd4904dd7fd4387ff8b188192ba744b0523c74d0aedfb82e8 |
| 8450 | 5ac0a07ec745fc71e6f9fd136fcb4f3824df07bff3a7568bf570bd7ff269665e |
| 8500 | 3083a4eb82c32eb6c12a0a8b481b1ee3deaf70294d9ec728371a9bfb614d58d5 |
| 8550 | 73b425383f706df43f2f4b482186e1ba733c0e47ce6d832e95c53b6a9142eeaa |
| 8600 | daa4308f0502c5c6562a9b96a33ab5c80dd051a8a4d61e541f262655be9c62b7 |
| 8650 | 38147d42c718faeaf34b861827a2502689aac857c2781a398a018af817176b2b |
| 8700 | a7849393b913cd9a7ed07602e623110edc3fb12c5f96529f7d639bd26acb96db |
| 8750 | 90a0f8a261839aa615481204c3d7f9286579f50df1f4f8bbb01925ff25e88b2f |
| 8800 | 24b648b0ed510e2958121d1c51c07731b605c8c09a64154a11dcb03ecdc7a191 |
| 8850 | d4a7476c44b7f9ff9dc111acc0dd51c8041c93a31c7555c4a08de46ec1d8bbc5 |
| 8900 | 68d0c316249adf620256ef90e5becbddd06920667a3fa858d0028ec2f5d17e7a |
| 8950 | 9c47394c72b08c4da14b32504fdf1eb1711c11fa13d2f49a862379954029ed9c |
| 9000 | d8ad7edec7e27425f5be4ed0c7d6b1cf32d84ea00e33ebd77fbde00c598dca64 |
| 9050 | cca1ab10de62de41eef5038e87f3b3ffcd17876c2cea284ef9dfd83e6c048028 |
| 9100 | 1968f7969b207d9740fdbf4a4efdf0dfebedc17c9fc1b8e7d2cdbc20c43048f5 |
| 9150 | 9798249cfdbce9cebb687df9b7cad1c8c87a30ecde243a3ba0853f75007b5bb6 |
| 9200 | 9a3a7dd1aaa9a57df1fd68d6a549f0a92e519394e04b5e4371c68200a3aaabab |
| 9250 | 209c1afd3360955e51c4b8fde7ea48789ad62ce2f31f0e839e32627ca6f9fd66 |
| 9300 | 4540338c3bad2642b1d8f45ea093fb25ae809f29b520a6787e034dab2995dd45 |
| 9350 | 2bb43e35480e82be7b509fcb8cc6b796897c7e1203f50840665f4d23ab7753ba |
| 9400 | 2783ac61aa374ce936ff59c5f3e2df821afefecd774979478d88a3b84e7acad7 |
| 9450 | 93601b632ae2e1ae9272ed0a9ae792c0b33b825f867e2dc7b454e8615d6a7cb8 |
| 9500 | 14ee41b7b04165ecbc3dea3e1db88aca070df14a85732e21133c17a976db5312 |
| 9550 | 4af10fe35c5d087b223b3af96852483aabbc876dda3729601fa0fe3a62ad7fca |
| 9600 | a5fc866ef4fe196cb959b6a5d9aa34f236bb15fa8294c56da27b3c6b2038e896 |
| 9650 | 8f3f58e37fe7b2c51b10e4e8569041f73457d198a1fdf6c53f2b63c38a1e4a28 |
| 9700 | 75de6f614ea667790fb4209a8b7ee9fe281cea7b03da9eb307263f4c37b291d3 |
| 9750 | c42785a68715e7dff90e2471aef433a77cf9b01b7c20c323c39a29ef148993c9 |
| 9800 | c9fc88ae437d6c5c0d4532d1225ff3af1e56b2709280f96f864c1695f0552c3d |
| 9850 | 6d7bdbcdb035a8445dc079ee011feac4ead3aeb5eaa15b1950db4f7965b316c8 |
| 9900 | e0bb1cc460798d8eaed3260550a0dd874b53dbbfba6f86c7c66c2ff2c366fa2f |
| 9950 | b8249f8f20b3a55c992d0ec3ae2d44ad27a93b769cd25926f432dcd611536192 |
| 10000 | 37fb6d1893978dfce3a2f87a6bc177e069f4af7ec7b7581249d2c908acfa8ba0 |
| 10010 | 04438959d7b317d1a8d972b264e9610a06fd8addde7349c6e9f458ee56dc1e51 |
Steps 10050–13848: see the per-release repositories above (identical files, SHA-256 listed there).
File format
torch.save dictionary of the trainer: model, opt (Muon + AdamW), step, config. Load with
torch.load(path, map_location="cpu", weights_only=True). Model class: train_gpt_ref_r23.py in the model
repositories (1.00–1.06). Every file is 8 860 287 151 bytes.
License
Apache-2.0 for the checkpoints. The training data includes CC BY-SA sources (shared/core_sa, shared/legal_sa of
slayer-pl-8x3b); see the dataset card for the licenses of the data.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support