YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Rank-5 SVD Reconstruction — Qwen2.5-Math-1.5B (RELEX rebuttal artifact)
Oracle rank-5 reconstruction of the RLVR weight-delta trajectory:
theta_recon = theta_0 + V_5 V_5^T (theta_step - theta_0), where V_5 are the top-5
right singular vectors of the full 500-step observed delta trajectory (SVD, --mode reconstruct --rank 5).
Subfolders step100/ .. step500/ = reconstruction of the real RLVR checkpoint at that step.
Base model = Qwen/Qwen2.5-Math-1.5B.
Eval config (MUST match to compare with the RLVR / rank-1 rows)
Greedy MATH, temperature=0.0, top_p=0.95, top_k=-1, num_generation=1, max_tokens=4096,
dataset TianHongZXY/MATH split test (5000 problems). Accuracy = mean of the boolean label field.
Reference rows (same config, already graded):
- RLVR (real): 100=67.96 200=70.14 300=70.78 400=71.38 500=71.54
- Rank-1 recon: 100=67.20 200=70.62 300=70.68 400=71.54 500=71.06
Expected rank-5 ~= rank-1 (>=, within ~0.5pp) -> confirms components 2-5 add ~nothing with oracle coefficients.
Load: AutoModelForCausalLM.from_pretrained("weizhepei/Rank5-Reconstruction-Qwen2.5-Math-1.5B", subfolder="step100")