ajaxdavis/alpha-60m-training-checkpoints

Alpha 60M native training checkpoints

This repository is the off-box recovery archive for Alpha 60M Chat. It contains Alpha's native binary checkpoints, including model parameters, AdamW optimizer tensors and step, RNG state, tokenizer artifacts, and training step. These files are for exact continuation with Alpha; use the standard Transformers export in the model repository for ordinary inference.

The terminal SFT checkpoint failed the predeclared chat-quality gates. It is preserved for reproducibility and future research, not represented as a usable assistant. The sealed evaluation produced 92 empty responses among 100 prompts, six degenerate loops, two unusable fragments, and 0/200 closed-book QA answers.

Recovery points

Path	Step	SHA-256	Purpose
checkpoints/base-pretrain-step-61036.alph	61,036	08e14fa9604bf1b46ebcd5df37933c84d2496c1d05d9e4b32ebad98792cc6049	Completed 1,000,013,824-token base pretrain
checkpoints/sft-best-retained-step-29000.alph	29,000	03eaac3e7be06e8fb5720415a334b36d7ef5019fcff72ca9227636b84011a7f3	Best held-out loss among surviving full SFT checkpoints: 1.6412250
checkpoints/sft-terminal-step-30322.alph	30,322	6c279d086d8c0679495e38ebec8a473ac23d16bfb3b93516e144712963fecbc8	Exact one-epoch terminal SFT state and default continuation point

The lower validation observation at step 28,500 (1.3334526) was a metric-only half-checkpoint; no checkpoint was written there. This archive does not misrepresent it as recoverable.

Integrity and provenance

• Training source commit: c333bf247fbe87b85d01f3d34789b46615dd1034
• Base-pretrain source: e561f66
• Architecture: Llama-form 16 layers, width 512, 8 attention heads, 57,688,576 parameters
• Tokenizer: Alpha byte-level BPE, vocabulary 12,288
• SFT corpus manifest SHA-256: ffad0a376c7eac2e0ec91f0901ec1ff87cba67cc298222828ce3df1a3e60b3fb
• SFT: one deterministic assistant-only masked-loss epoch, 30,322 steps
• Terminal Alpha/Transformers parity: 2/2 top-1, maximum logit difference 7.153e-06

CHECKSUMS.sha256 binds every published file except itself. The reports directory preserves the execution contracts, complete metric stream, native audits, terminal analyzer, and both failed frozen evaluation reports. The failure evidence is part of the artifact.

Resume on a future RunPod

1. Download the repository and run sha256sum -c CHECKSUMS.sha256 before uploading anything to a pod.
2. Bootstrap the pod with scripts/runpod_bootstrap.sh from the Alpha repository. Preserve the proven settings: float32, HELIOS_DISABLE_COOP_MAT=1, --no-fallback=true, and no CPU training.
3. Restore the tokenizer and input audits from inputs/, the selected checkpoint, and the relevant contract/metrics from reports/; verify the same hashes remotely.
4. Use sft-terminal-step-30322.alph by default. Branch from step 29,000 only as a separately named, explicitly documented experiment.
5. Do not reuse the original one-epoch launcher unchanged: it intentionally stops at step 30,322. Define a new continuation contract with a higher target, retain the old run as immutable input, and prove resume behavior before authorizing GPU spend.

No continuation run is currently authorized or running.