ISB369/shellminator-qwen05b-dpo-selfplay

shellminator — Qwen2.5-Coder-0.5B + DPO self-play

Natural language → single bash command. 0.5B params, 78% exec accuracy, runs on CPU.

IBM nl2bash-eabench bash_1 (50 prompts, single-pass greedy): 78% — +6 over its SFT base, +30 over a 270M first-gen. Trained for ~$5 of GPU.

---

What this model is

A full fine-tune of Qwen2.5-Coder-0.5B-Instruct for translating natural language into one bash command, polished with DPO against its own execution-verified failures. Ships as safetensors and a Q4_K_M GGUF (served by llama.cpp or ollama).

The user-facing launcher that turns this into sm "<question>" lives here: github.com/ISB333/shellminator.

Quick start

Zero-setup (recommended):

curl -fsSL https://raw.githubusercontent.com/ISB333/shellminator/main/install.sh | bash
sm "show me the 5 largest files in /var"

Plain ollama:

huggingface-cli download ISB369/shellminator-qwen05b-dpo-selfplay \
  shellminator-qwen05b-dpo-selfplay-Q4_K_M.gguf --local-dir .
# note: FROM needs an ABSOLUTE gguf path (ollama rejects relative ones)
printf 'FROM %s\nPARAMETER temperature 0\nPARAMETER num_predict 200\nPARAMETER stop "<|im_end|>"\n' \
  "$(pwd)/shellminator-qwen05b-dpo-selfplay-Q4_K_M.gguf" > Modelfile
ollama create shellminator -f Modelfile
ollama run shellminator "count hidden files in the current directory"

Transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer

SYSTEM = ("You are a helpful assistant that translates natural language to bash commands.\n"
          "Context: cwd=/home/user, system=Linux x86_64, shell=bash.\n"
          "Reply with a single bash command only. No explanation, no markdown fences.")
tok = AutoTokenizer.from_pretrained("ISB369/shellminator-qwen05b-dpo-selfplay")
model = AutoModelForCausalLM.from_pretrained("ISB369/shellminator-qwen05b-dpo-selfplay").cuda()

prompt = "show kernel name, release and version"
text = tok.apply_chat_template(
    [{"role": "system", "content": SYSTEM},
     {"role": "user", "content": f"Generate single Bash command: {prompt}"}],
    tokenize=False, add_generation_prompt=True)
ids = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=200, do_sample=False,
                     eos_token_id=tok.convert_tokens_to_ids("<|im_end|>"))
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))

Benchmark

Model	IBM nl2bash-eabench bash_1 · 50 prompts · single-pass greedy · exec-verified
Qwen2.5-Coder-0.5B-Instruct (stock)	44%
gemma-3-270M fine-tune (first-gen shellminator)	48%
+ SFT on 105K exec-filtered rows (shellminator-qwen05b-sft105k)	72%
+ DPO self-play — 804 verified pairs (this model)	78%

Protocol: each prompt's command runs in a per-test docker sandbox and is graded by the IBM eabench verifier (exit code + workspace state). One greedy pass, no retries, no self-correction, temperature 0.

What the +6 points bought (fail classes fixed end-to-end): find-count-without-self (find . -mindepth 1 -type d | wc -l), df -i overlay inode read-outs, uname flag selection. All six fail classes the SFT missed are represented in the 804 pairs (find_count 263, uname_flags 144, sed_insert_top 144, mv_glob_dir 77, …).

How it was made (exec-verified self-play)

Three data levers were measured at +0 against the 72% SFT plateau: +12K rows, teacher distillation DPO (synthetic rejections), and 949 targeted rows. The lever that worked:

1. Build a 9,219-prompt pool weighted toward the residual fail classes, keeping 105K provenance (training/build_dpo_pool.py).
2. Harvest the model's greedy outputs on Modal A100 (~$2): the gold command AND the generated candidate run in fresh per-row sandboxes; exit code, stdout, stderr and the post-state file tree must match, gold first. Divergent rows where gold itself broke or was unreproducible were dropped — including a two-run self-consistency probe that kills free/df-style drift noise (without it, ~8% of harvest smoke pairs punished equally-correct answers).
3. One DPO epoch (TRL, lr 5e-7, beta 0.1 → 26 s of A100): chosen = gold, rejected = the model's own wrong command. Eval accuracies 0.90 / margins 1.29.

Full recipes and scripts: github.com/ISB333/shellminator → training/. A chained round 2 (477 pairs from the upgraded champion) scored 74% — an ablation kept private; preference tuning past the first big-class dose overfits recent pairs.

Limitations

• Single command by design — no multi-line scripts, no heredocs; the launcher's refine loop is the intended workflow for chaining.
• Residual fails: sed -i '1i …'-style top inserts inside for-loops (the model can overwrite the file), comm/diff count variants, mv dir1/* dir2/ arg order. These survived +12K rows, synthetic DPO, targeted SFT, and two rounds of self-play.
• Trained on Linux-x86_64-conventional fixtures; commands referencing exotic mounts (overlay, xvda) inherit sandbox assumptions.
• The benchmark's protocol (single-pass greedy, exec-verified) is documented in the repo; numbers are not comparable to token-match nl2bash scores.

Training config

Stage	Value
Base	Qwen2.5-Coder-0.5B-Instruct
SFT	105K conversational rows, 3 epochs, lr 2e-5, assistant-only loss, fp32 + bf16 autocast
Pool	9,219 fail-class prompts (provenance: part + line)
Harvest	batch 64, bf16, greedy, 200 tokens, exec-verify both sides, 24 workers
DPO	804 pairs, 1 epoch, lr 5e-7, beta 0.1, 48 steps, A100
GGUF	convert_hf_to_gguf f16 → Q4_K_M (398 MB)

Citation

@misc{shellminator2026,
  title  = {shellminator: exec-verified self-play for a 0.5B bash coder},
  author = {ISB333},
  year   = {2026},
  url    = {https://github.com/ISB333/shellminator}
}