liangyue36/minimind-stop-token-study

MiniMind 64M stop-token study checkpoints

PyTorch state dicts (.pth) from a paired pretraining experiment: does the document-end token used during pretraining affect whether a model stops after LoRA SFT? Code and write-up: github.com/cassieliang6709/stop-token-study. Load with the MiniMind model code in that repo.

File	What
pretrain_A.pth	pretrained from scratch, <|im_end|> as document end
pretrain_B.pth	pretrained from scratch, <|endoftext|> as document end
sft_*	run 1 SFT (LoRA / LoRA + embeddings)
sft2_*	run 2 SFT, training data filtered to ≤ 300 tokens (the reported protocol)
*_loss.jsonl	training loss logs

Result (run 2, 200 held-out prompts): runaway generation 26.0% (52/200) for A + LoRA versus 46.5% (93/200) for B + LoRA (z = 4.4); B + LoRA + embeddings 39.0% (78/200), not significantly different from B + LoRA.