Techno03/illada-8b-sudoku-lora

iLLaDA-8B Sudoku LoRA

LoRA adapter for GSAI-ML/iLLaDA-8B-Base fine-tuned to solve 9×9 Sudoku puzzles formatted as space-separated row tokens (Input: R1: 5 3 . . 7 . . . . | R2: ...).

Results (100 held-out validation puzzles, pct_blank = % of originally-blank cells correct)

Difficulty	Baseline (zero-shot)	Fine-tuned
easy	~35.8%*	92.7%
medium	~8.2%*	69.6%
hard	~0.0%*	27.4%
overall	—	60.3%

* baseline estimates on a small n; see the repo README for caveats.

For comparison, gemma-4-12B-it fine-tuned with the same recipe reaches 95.7 / 38.9 / 8.9 (easy / medium / hard) — the diffusion model degrades far more gracefully as difficulty rises.

Training

• Base: GSAI-ML/iLLaDA-8B-Base, LoRA r=16, alpha=32, dropout=0.05, targets q/k/v/o_proj
• Masked cross-entropy over randomly-masked solution tokens only (prompt left intact)
• 5,000 optimizer steps on an H100; checkpoint selected by generation score (best gen pct_blank 85.96 @ step 4250), not eval loss
• Data: 50k synthetic puzzles, difficulty uniform in [0.2, 0.8]

Usage

from transformers import AutoTokenizer
from peft import PeftModel
import torch

tok = AutoTokenizer.from_pretrained("GSAI-ML/LLaDA-8B-Base", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "GSAI-ML/LLaDA-8B-Base", torch_dtype=torch.bfloat16,
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, "<this-repo-id>")

Then run confidence-ordered remasking diffusion sampling over the masked solution canvas — see the SudokuDiffusion repo for a full sampler implementation.

metrics.json and training_meta.json are included for provenance.