Qwen2.5-1.5B-Instruct — DeepMath RAFT (Direct-OPD exp3, Part B)
pi_post^RAFT for the Direct-OPD mechanism experiment (pre-registration section F1–F4).
Rejection-sampling SFT: Qwen/Qwen2.5-1.5B-Instruct @ 989aa7980e4cf806f80c7fef2b1adb7bc71aa306 fine-tuned on its own verified
completions — k=4 samples at T 0.7 / top_p 0.95 per prompt over the pilot's sft_train split,
math-verify-checked against DeepMath-103K ground truth, at most one correct sample kept per prompt.
The point of the pair (pi_pre, this model) is that it is SFT in mechanics but
pi_post ~ pi_pre * exp(advantage) in distribution, so the Direct-OPD shift reward
log pi_post - log pi_pre should behave like the RL pairs, not like the corpus-SFT pairs.
| base | Qwen/Qwen2.5-1.5B-Instruct @ 989aa7980e4cf806f80c7fef2b1adb7bc71aa306 |
| data | cmpatino/direct-opd-exp3-raft-data @ 5cd1dea4b139b22f2db98b56a734db34cb469743 :: raft_sft.parquet (4187 kept samples) |
| loss | assistant-masked causal LM, token-weighted |
| epochs / steps | 2 / 124 |
| batch | global 64 (2 x 32) |
| lr / schedule | 1e-05, cosine decay, 6 warmup |
| precision | fp32 master weights, bf16 autocast, bf16 checkpoints |
| val loss | step 0 0.1688 -> final 0.1617 |
| checkpoints | checkpoint-25pct, checkpoint-50pct, checkpoint-75pct, checkpoint-100pct (+ final at root) |
Checkpoint config.json, generation_config.json, tokenizer.json and tokenizer_config.json
are byte-identical to the base snapshot; lm_head.weight is omitted because the embeddings are
tied, exactly as in the base checkpoint.
Private research artefact for the Direct-OPD SFT-transfer study. Not a general-purpose model.
- Downloads last month
- 14