sahellx/kitty-bash-llm

🐱 Kitty Bash LLM

A 0.5B parameter shell assistant that runs on a laptop CPU.

Kitty Bash LLM turns plain English into bash commands, repairs commands that just failed, and completes half-typed ones. It is deliberately small — the 4-bit GGUF is 398 MB and generates a command in well under a second on an ordinary CPU, so it can sit behind a shell integration without a GPU, an API key, or a network round trip.

It is a specialist, not a chatbot. Ask it the capital of France and you will get nonsense. Ask it to find every file over 100 MB and it will tell you.

---

What it does

Three behaviours, selected by the system prompt. One model, one file, flat memory.

task	you give it	it returns
nl2cmd	find all files larger than 100MB under /var	find /var -type f -size +100M
fixcmd	$ gerp -r 'TODO' . bash: gerp: command not found	grep -r 'TODO' .
complete	tar -czf backup.	$(date +%F).tar.gz /path/to/dir

System prompts

NL2CMD   = ("You are a bash command generator. Given a natural language request, "
            "output only the bash command that accomplishes it. "
            "No explanation, no markdown fences.")

FIXCMD   = ("You fix broken bash commands. Given a failed command and its error "
            "output, output only the corrected command. No explanation.")

COMPLETE = ("You complete partially typed bash commands. Given a command prefix, "
            "output only the text that completes it. No explanation.")

---

Quick start

llama.cpp / GGUF (recommended — this is what it was built for)

llama-cli -m kitty-bash-llm-q4_k_m.gguf -st -t 4 --no-display-prompt -n 64 --temp 0 \
  -p "<|im_start|>system
You are a bash command generator. Given a natural language request, output only the bash command that accomplishes it. No explanation, no markdown fences.<|im_end|>
<|im_start|>user
show which process is listening on port 8080<|im_end|>
<|im_start|>assistant
"
# -> lsof -i :8080

For anything interactive, run a persistent server so the system prompt is prefilled once (see Performance — this is the single most important implementation detail):

llama-server -m kitty-bash-llm-q4_k_m.gguf -c 1024 -t 4 --port 8080

transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("sahellx/kitty-bash-llm")
tok   = AutoTokenizer.from_pretrained("sahellx/kitty-bash-llm")

SYSTEM = ("You are a bash command generator. Given a natural language request, "
          "output only the bash command that accomplishes it. "
          "No explanation, no markdown fences.")

msgs = [{"role": "system", "content": SYSTEM},
        {"role": "user",   "content": "compress the logs folder into a tar.gz archive"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=64, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
# -> tar -czvf logs.tar.gz logs/

Greedy decoding (do_sample=False / --temp 0) is recommended. You want the most likely command, not a creative one.

---

Evaluation

Measured against the untuned base model on a held-out split, with identical prompts, identical f16 GGUF format, and the same llama.cpp engine — the only variable is the fine-tuning. 100 examples per task.

nl2cmd

metric	base	Kitty	Δ
exact match	7.00	23.00	+16.00 (3.3×)
first-utility accuracy	61.00	79.00	+18.00
token F1	39.35	59.77	+20.42
bash -n valid	93.00	94.00	+1.00
ShellCheck clean	77.00	82.00	+5.00

fixcmd

metric	base	Kitty	Δ
exact match	0.00	82.00	+82.00
first-utility accuracy	1.00	99.00	+98.00
token F1	45.20	96.71	+51.51
ShellCheck clean	63.00	83.00	+20.00

⚠️ Read this number sceptically. The base model scores ~0 largely because it ignores the output format and explains the error in prose rather than emitting a command — so much of the gap is format compliance, not repair skill. The test set also uses synthetic corruptions of three kinds the model trained on. On real shell errors, hands-on testing put it closer to 1-in-3. Treat 82% as an upper bound.

complete

metric	base	Kitty	Δ
token F1	2.22	29.88	+27.66 (13×)
exact match	0.00	7.00	+7.00
prefix+completion valid	—	99.29	—

Exact match is a poor metric here — many completions are valid for any prefix. bash -n on the completion fragment alone is meaningless (a fragment like --rm ubuntu is not valid standalone bash); the meaningful figure is prefix + completion, at 99.29%.

Comparison with other shell-focused models

Alongside the base-model comparison above, an informal benchmark was run against four other publicly available fine-tunes for shell/linux command generation, all at Q4_K_M (except where a repo ships only f16), through llama.cpp on the same 2-thread CPU, using the same 80 held-out prompts.

model	params	size	utility acc	token F1	s/cmd
Kitty Bash LLM	0.5B	398 MB	78.75	59.54	1.68
louisguthmann/qwen3.5-2b-shellcommand-linux	2.0B	1274 MB	63.75	32.02	15.39
vitali87/shell-commands-qwen2-1.5b-extended	1.5B	3094 MB	57.50	33.93	9.48
mecha-org/linux-command-generator-llama3.2-1b	1.0B	808 MB	52.50	34.21	2.78
chamibuddhika/linux-commands-0407-00	—	1709 MB	—	—	—

⚠️ Please read this before citing the table

This is indicative, not a controlled evaluation. Four specific caveats, all of which favour this model:

1. The test set is drawn from this model's own training distribution. The split is properly held out with zero leakage, but it comes from the same corpora and shares their formatting conventions. The other models never saw that style. This inflates exact match in particular, which is why exact match is omitted above.
2. All models received this model's system prompt. Peers trained with a different prompt format are being evaluated off-distribution, which alone could account for a large part of the gap.
3. n = 80. The 95% confidence interval is roughly ±9 points; the gap to the runner-up is significant only marginally (p ≈ 0.03).
4. chamibuddhika/linux-commands-0407-00 returned empty output for every prompt and is reported as no-result rather than zero — the cause may well be the harness rather than the model. qwen3.5-shellcommand emits </think> reasoning tags that the harness did not strip, so its syntax-validity score was invalid and has been omitted here.

A properly controlled comparison would use a neutral test set none of the models trained on, each model's own prompt format, and n ≥ 500. Treat the ordering above as a hint worth verifying, not a settled result.

The base-model comparison in the previous section does not carry these caveats: same architecture, same data, same harness, only the fine-tuning differs.

ShellCheck is used because bash -n only proves a command parses. ShellCheck catches genuine defects — unquoted expansions (SC2086), word splitting (SC2046) — that parse fine and then break on a filename with a space.

---

Training

base	Qwen/Qwen2.5-Coder-0.5B-Instruct
method	LoRA r=64, alpha=64, all attention + MLP projections, merged
precision	fp16 (Tesla T4, no bf16)
epochs / lr	1 / 2e-4 cosine, 3% warmup
batch	16 × 2 grad-accum = 32 effective
max seq len	512
loss	response-only (masked so loss lands on the command, never the prompt)
runtime	41 min on a single T4
final train / eval loss	0.5579 / 0.5919

The base model was chosen by measurement, not assumption: a bake-off against Qwen2.5-0.5B-Instruct under an identical recipe gave 84.98 vs 84.21 utility accuracy, so the code-pretrained variant won.

---

Dataset

~60,000 examples across the three tasks, pooled from seven sources, deduplicated, rebalanced, and decontaminated.

split	nl2cmd	fixcmd	complete	total
train	22,573	19,891	17,528	59,992
val	1,254	1,238	1,456	3,948
test	1,254	1,219	1,409	3,882

Sources: AnishJoshi/nl2bash-custom, neulab/tldr, emirkaanozdemr/bash_command_data_6K, huytd189/command-line-suggestions, aelhalili/bash-commands-dataset, Jawajawa/command-linux-bash-balanced-sft, PocketDoc/Dans-Toolmaxx-ShellCommands.

Decontamination — please read this if you benchmark on nl2bash

The most widely used NL2Bash split on the Hub leaks badly. Measured directly:

test  prompts also present in train : 71.0 %
dev   prompts also present in train : 71.8 %
duplicate rows within train         : 39.5 %  (19,658 rows → 11,890 unique)

Training and evaluating on those splits measures memorisation. Every published number here comes from splits rebuilt from scratch: all rows pooled, deduplicated on the normalised prompt, re-split 90/5/5 with a fixed seed, and asserted in code to have zero train↔test overlap.

Two further corrections:

• find rebalanced 54% → 30%. In the source corpus more than half of all targets were find, which made the model a find specialist that produced nonsense for tar and du. Capping it fixed both.
• Gold targets filtered through bash -n. Broken targets teach broken bash.
• fixcmd and complete derived after the split, from within each split only — so a command in test can never appear in train under a different task label.

---

Limitations

Findings from hands-on testing, not guesses.

Weak on multi-step logic. It picks the right tool but can get the reasoning wrong:

"print the 3rd column of a csv where the 1st column equals ERROR"
  → cut -d',' -f3 data.csv | grep ERROR          ✗ filters the wrong column
  correct: awk -F, '$1=="ERROR" {print $3}'

"show the 10 biggest directories under /home sorted by size"
  → find /home -type d | sort -n -r | head -10   ✗ sorts names, not sizes
  correct: du -h /home/* | sort -rh | head -10

Uneven across utilities. Strong on find, tar, docker, sed, grep, lsof. Weaker on less common subcommands (git branch --merged came out as gibberish).

fixcmd is much weaker on real errors than the benchmark suggests — see the warning above.

complete may duplicate a prefix boundary. Given docker run -it -- it can return --rm .... Strip the overlap between prefix and completion in your client.

Catastrophic forgetting — by design. General ability is gone. This is a bash tool, not an assistant.

No safety behaviour whatsoever. Asked to delete the root filesystem it emits the command immediately, with no warning. See below.

---

⚠️ Safety

This model will generate destructive commands without hesitation.

"delete every file in the root filesystem"  →  find / -exec rm {} \;

It has no refusal training and no notion of danger. Any tool built on it must:

1. Never auto-execute. Print the command; require an explicit keypress.
2. Pattern-match destructive commands — rm -rf, dd, mkfs, > /dev/sd*, chmod -R 777, curl … | sh — and warn loudly.
3. Treat output as untrusted. It is a suggestion from a 0.5B model, not an authority.

Verify commands before running them. Especially ones touching /.

---

Performance

Measured with llama.cpp, Q4_K_M, on 2 CPU threads (deliberately weak):

prompt processing	42.8 tok/s
generation	17.1 tok/s
40-token system prompt prefill	935 ms
~12-token command generation	701 ms
cold request	1636 ms
warm (KV cache reused)	701 ms

Implication: run a persistent server. Over half of a cold request is spent re-processing the same system prompt. Spawning a process per invocation costs 1.6 s; a warm llama-server costs ~700 ms. On a typical 8-core machine expect roughly 3–4× faster (200 ms).

For inline autosuggestion, that is still too slow to run on every keystroke. Debounce (~250 ms), fire asynchronously, and cancel in-flight requests — the Copilot pattern. History-based matching should handle the common case; use the model for what history has never seen.

---

Files

file	size	use
kitty-bash-llm-q4_k_m.gguf	398 MB	recommended — CPU inference
kitty-bash-llm-q5_k_m.gguf	420 MB	slightly higher quality
kitty-bash-llm-q8_0.gguf	531 MB	near-lossless
kitty-bash-llm-f16.gguf	994 MB	full precision GGUF
model.safetensors	988 MB	transformers / further fine-tuning

---

Reproducing this

Every step is a script, not a notebook cell: data pooling and decontamination, the base-model bake-off, training, GGUF conversion, and the evaluation harness (including the ShellCheck pass). The find rebalancing and the leakage assertions are the parts worth copying if you build on the NL2Bash corpora.

Citation

@misc{kitty-bash-llm,
  title  = {Kitty Bash LLM: a 0.5B shell assistant for CPU inference},
  author = {sahellx},
  year   = {2026},
  url    = {https://huggingface.co/sahellx/kitty-bash-llm}
}

Built on Qwen2.5-Coder (Apache-2.0), trained with Unsloth, quantized with llama.cpp.