terllm

Small local models that complete the shell command you are typing (fish / zsh-autosuggestions style ghost text), fine-tuned from SmolLM2-135M. Runs locally through llama.cpp (~25 ms per keystroke on CPU, Q4_K_M). Code, data generator, zsh plugin and daemon: https://github.com/partum55/terllm.

Prompt format (v3; v2 is the same without <|fs|>)

Plain text with added special tokens; the model continues the typed prefix and ends with <|end|>:

<|cwd|>~/proj<|ls|>README.md src<|git|>main -> origin/main | ?? x.py<|env|>VIRTUAL_ENV=.venv
<|hist|>$ git clone ...
$ cd proj<|cand|>git checkout main<|fs|>src/ scripts/<|cmd|>cd s

• <|hist|> last commands (with exit codes), <|err|>/<|out|> previous output (optional), <|nl|> natural-language request (optional)
• <|cand|> older history commands that extend the typed prefix (what zsh-autosuggestions would offer)
• <|fs|> (new in v3) up to 8 tab-completion candidates for the word being typed: files, directories, PATH commands. The model decides whether to copy one, write a new name (mkdir, branch names) or stop (command already complete)

The reference implementation of the prompt is datagen/render.py in the repository; use the daemon (runtime/terllmd.py) rather than building prompts by hand. Show only the confident prefix of the completion (all tokens with probability >= 0.8); the policy is in runtime/policy.py.

Models in this repo

folder	what	files
v3/	current: adds the `<	fs
v2/	previous: history candidates (`<	cand

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("partum/terllm", subfolder="v3")
model = AutoModelForCausalLM.from_pretrained("partum/terllm", subfolder="v3")

Evaluation (honest numbers)

	v2	v3
chars saved on a real zsh history (150 commands, cwd unknown, threshold 0.8)	15.8%	18.4% (with `<
exact match, synthetic path probe (fresh names)	33%	73.5%
exact match, synthetic val (no `<	fs	>` blocks)

For reference, history-prefix matching (zsh-autosuggestions) saves 0.4% on the same commands. The real-history sample is small (about +-1-2 points) and context-poor; it has not been validated on live usage yet. On command/tree shapes outside the training scenarios, path suggestions are noticeably weaker, and the model sometimes ignores a single obvious candidate. Treat it as a suggestion engine: it never executes anything.

Training data

Synthetic terminal sessions executed in a shell/git simulator (so exit codes, stderr and state are consistent), seed scenarios plus LLM-proposed sessions (Qwen2.5-Coder, every command re-executed by the simulator), and text-to-command pairs derived from tldr-pages (CC BY 4.0, "tldr-pages team and contributors") and NL2Bash (MIT). Details and licenses: data/SOURCES.md in the repository. No personal shell history was used for training.