tawer12/qwen3-4b-sudoku-missing16-20-binary-rl

Qwen3-4B Sudoku: 16–20 missing cells, binary-reward RL

LoRA adapter for Qwen/Qwen3-4B-Instruct-2507, trained to solve Sudoku by emitting linear traces with row updates.

The adapter weights are 11,815,792 bytes (11.82 MB). The tokenizer is included. Loading this adapter also requires the separate base model. The preceding SFT adapter is already incorporated into these updated adapter weights; it does not need to be downloaded separately.

Training details

• Checkpoint: rl_train_missing16_20_k600_linear_row_markdown_binary_rl.
• Saved: 2026-04-10 17:12:06 UTC, based on the original weight-file timestamp.
• Initialization: the local trivial_missing1_15_linear_row_markdown_sft adapter.
• RL dataset: 600 training puzzles with 16–20 missing cells, from rl_train_missing16_20_k600.jsonl.
• Completed: 1 epoch, 600 samples, 75 optimizer updates, confirmed by the training log.
• Reward: 1 when the extracted solution matches the reference solution, otherwise 0.
• Learning rate: 5e-6; gradient accumulation: 8; seed: 42.
• Training generation: temperature 0.4, top-p 1.0, maximum 1,536 new tokens.
• LoRA: rank 16, alpha 32, dropout 0.05; targets q_proj and v_proj.
• Adapter tensors: BF16.

The full saved training configuration is in resolved_linear_binary_rl_config.json. Its relative paths identify the original local training setup. Training data and the base model weights are not bundled.

Run a Sudoku example

Install the dependencies in a Python environment. A GPU is recommended for running the 4B base model.

pip install torch 'transformers>=5.4.0' 'peft>=0.18.1' accelerate huggingface_hub
hf download tawer12/qwen3-4b-sudoku-missing16-20-binary-rl --local-dir sudoku-adapter
python sudoku-adapter/inference.py

To supply your own board, pass --puzzle followed by 81 digits in row-major order, using 0 for each empty cell.

The included script downloads the base model, loads this adapter with PeftModel.from_pretrained, and uses the original training prompt. The input board is a markdown table with rows A–I, columns 1–9, and X for empty cells. The expected output uses STATE, FILL, Updated Row, SOLVED, and FINAL_STATE. Prompts are tokenized directly, without applying a chat template, matching training and evaluation.

Load from Python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "tawer12/qwen3-4b-sudoku-missing16-20-binary-rl"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-4B-Instruct-2507", dtype="auto", device_map="auto"
)
model = PeftModel.from_pretrained(base, repo_id).eval()

Use linear_row_markdown_trace_instruction.txt to construct the prompt; inference.py shows the complete formatting and generation steps.

Evaluation

This upload documents the saved training run. It does not report a completed held-out evaluation for this checkpoint. Generated solutions should be checked against Sudoku rules and the original clues.