MDM Sudoku / 3-SAT reproducibility rerun (R2D / TCT)
Checkpoints for the reproducibility rerun of the small-model (6.85M-parameter, Qwen2-style masked-diffusion) experiments on Sudoku-Hard and 3-SAT (SATLIB uf20-91) for the paper "Random Remasking Scales: A Unified View of Masked Diffusion Inference."
All runs were trained from scratch on the cleaned codebase, 3 seeds each (s0/s1/s2), with
checkpoints saved throughout training (for accuracy-vs-training-step curves) and never deleted.
Contents
12 runs = {task} x {training method} x {seed}, each a directory of step<N>.pt snapshots:
| Directory | Task | Training | Steps saved | Final |
|---|---|---|---|---|
sudoku_standard_s{0,1,2} |
Sudoku-Hard | Standard MDM | every 25k | step500000 |
sudoku_tct_s{0,1,2} |
Sudoku-Hard | TCT (K=8) | every 25k | step500000 |
sat_standard_s{0,1,2} |
3-SAT | Standard MDM | every 5k | step100000 |
sat_tct_s{0,1,2} |
3-SAT | TCT (K=5) | every 5k | step100000 |
Batch size 128 (Sudoku) / 512 (3-SAT). Each .pt holds model_state_dict (fp32) + the training
config (no optimizer state) -> ~27 MB.
Model
Qwen2-style MDMTransformer: hidden 256, intermediate 768, 8 layers, 8 heads, RMSNorm.
Vocab 11 / max_position 162 / mask_id 10 (Sudoku); vocab 42 / max_position 384 / mask_id 0 (3-SAT).
Load
import torch
ck = torch.load("sudoku_tct_s0/step500000.pt", map_location="cpu")
state_dict = ck["model_state_dict"]
cfg = ck["config"] # OmegaConf-dumped training config
Inference/eval code (R2D, ReMDM, standard decoding) is in the accompanying r2d-markovian-expt repo.