khazic/spec-b300

NGram Candidates: completed 3-epoch GB200 run

This repository archives only ngram-candidates-3epoch-b300-20260910: six complete draft checkpoints, all ten successful training-segment logs, two failed-initialization logs, vLLM logs and provenance, local W&B journals, and the frozen source snapshot.

Training completed 3 epochs / 256,706 optimizer steps on one node: GPU0 ran one vLLM engine; GPU1-3 ran DDP training.

Directory	Global step	Validation loss	Role
checkpoints/0	85,568	1.358138	Epoch 1 end
checkpoints/1	171,135	1.273758	Epoch 2 end
checkpoints/2	256,706	1.227255	Final and best checkpoint
checkpoints/.previous-0	84,744	—	Previous epoch-1 generation
checkpoints/.previous-1	170,312	—	Previous epoch-2 generation
checkpoints/.previous-2	255,879	—	Previous epoch-3 generation

Each checkpoint includes model.safetensors, optimizer_state_dict.pt, scheduler_state_dict.pt, training_state.json, model configuration and train_command.txt. Epoch-end checkpoints additionally contain validation metrics. The model file is the draft model, not the 35B target weights. All checkpoint directories contain real files, not the original local symlinks.

Logs and reproducibility

• training_logs/segments: per-allocation training, vLLM and W&B logs; configuration, resume position and command provenance.
• training_logs/manifest.json: run configuration, source/image revisions and repair history.
• training_logs/training_complete.json: exact completion audit. Packed epoch lengths are 85,568 / 85,567 / 85,571; the older nominal scheduler horizon of 256,704 is not the actual completed count.
• source: frozen source revision cd5091edb48f3de74eee315e508161f494ef3cb1, without Git object storage or generated bytecode. Historical source documents describe earlier intermediate states; the completion audit is authoritative.
• ASSET_MANIFEST.json and SHA256SUMS: complete payload inventory and checksums.
• W&B run and final W&B artifact.

Final validation: loss 1.2272551174, candidate recall 0.9703120655, proposal EAL 4.9589815238. These are training-validation metrics, not a target-only serving speedup benchmark for the final checkpoint.

For inference/evaluation, use checkpoints/2 with the compatible Speculators source and the separately obtained target model. Resume requires the supplied continuous/resume-safe wrapper and the corresponding training recipe, optimizer and scheduler states; it is not a generic Transformers causal-LM checkpoint.

Credentials, other runs, the target model, training datasets, the sqsh image, Git metadata and reusable compilation caches are not included. Historical logs retain original cluster paths as provenance. Duplicate directory symlinks, stale PID/lock markers and caches are listed as exclusions in the manifest.