c3-mdlm-110m

Masked diffusion language model (MDLM objective, absorbing mask, SUBS), trained from scratch for the C3 project (committed-confidence calibration in diffusion LMs).

The plan-v2 110M (run train110m-v3, W&B train110m-v3-oct12): the model the C1 traces and the T9 probe use. Gate G2 passed (artifacts/gate_g2_train110m-v3.json).

parameters	123.9M (tied embeddings)
training tokens	9,995,032,224
steps	169055
weights	ema
config hash	53f70713a312b3b2
code commit	aa41680fa279a302d8d04456e5b318c40e45ec97
data	FineWeb-Edu sample-10BT, document-aligned 256-token windows (no packing), evaluation documents excluded by id / text hash / first-256-token hash
tokenizer	GPT-2 BPE + one [MASK] token (id 50257)
held-out NELBO	3.6175 nats/token (ppl bound 37.25, 2000 documents)
GPT-2 small, same documents	ppl 28.49