manuelgscom/dale-2b-GGUF

dale-2b (GGUF)

A full fine-tune of Qwen/Qwen3.5-2B that turns a natural-language request, in Spanish or English, into one bash command for Linux, answered as a small JSON object. It was trained with the same data as dale-0.8b. It is meant for anyone who wants to build their own application on top of llama-server, on CPU or GPU. The dale CLI does not download it: it uses dale-0.8b on the CPU and dale-4b or dale-9b on the GPU.

• Author: Manuel García Sanz
• Website: https://manuelgs.com/en/dale/
• Source code of dale: https://github.com/manuelgscom/dale
• Format: GGUF, Q4_K_M (imatrix) · 1,274.4 MB
• Test v2 accuracy: 81.7 % (385/471)
• Not downloaded by the dale CLI, which uses dale-0.8b (CPU) and dale-4b / dale-9b (GPU); dale --model can still run it. About 0.9 s per request and 1.8 GiB of VRAM on a GPU (details)
• The dale family: four sizes of the same model, all in one collection

The dale CLI live, with dale-0.8b on a desktop CPU (real, unedited recording): the same contract dale-2b answers, here wrapped in dale's risk verifier and confirmation. Your own application can follow the same pattern.

Resumen en español

dale-2b es un ajuste completo de Qwen3.5-2B que convierte una petición en castellano o en inglés en un único comando bash para Linux, con una explicación breve. Se entrenó con los mismos datos que dale-0.8b. En el test v2 (471 casos) acierta 385 de 471 (81,7 %), frente al 62,8 % de Qwen3.5-9B sin ajustar. Está pensado para quien quiera montar su propia aplicación con llama-server, en CPU o en GPU. La CLI dale no lo descarga: usa dale-0.8b en la CPU y dale-4b o dale-9b en la GPU. Usa el mismo contrato que dale-0.8b (prompt fijo, razonamiento desactivado y respuesta en JSON). Fuera de dale no hay verificador de riesgo: revisa siempre el comando antes de ejecutarlo.

The dale family

The same task, data, contract and prompt in four sizes. Scores on the public test set dale-bash-test-v2 (471 cases), all in Q4_K_M with llama-server.

Model	Base	GGUF (Q4_K_M)	Test v2	In the dale CLI	Repository
dale-0.8b	Qwen3.5-0.8B	529.3 MB	79.4 %	CPU (default and fallback)	manuelgscom/dale-0.8b-GGUF
dale-2b	Qwen3.5-2B	1,274.4 MB	81.7 %	not used (llama-server only)	manuelgscom/dale-2b-GGUF (this one)
dale-4b	Qwen3.5-4B	2,708.8 MB	87.0 %	GPU, ≥ 4 GiB free VRAM	manuelgscom/dale-4b-GGUF
dale-9b	Qwen3.5-9B	5,629.1 MB	89.2 %	GPU, ≥ 8 GiB free VRAM, if you choose it	manuelgscom/dale-9b-GGUF

Which one?

 the dale CLI, first run:
   dedicated GPU, ≥ 8 GiB free VRAM ─► you choose: 1) dale-4b (Enter, faster)  2) dale-9b (more accurate)
   dedicated GPU, ≥ 4 GiB free VRAM ─► dale-4b
   anything else (CPU, integrated GPU) ─► dale-0.8b on the CPU

 your own application (llama-server, CPU or GPU) ─► any of the four: pick by memory and accuracy

What it is and intended use

your app ─► system context ─► dale-2b ──────────► your own review ─► show / run
            (os, distro,      (llama-server,       (no risk verifier
             package manager,  CPU or GPU:          outside dale:
             installed tools)  one bash command,    you add it)
                               refuse or unsure)

• Same contract as dale-0.8b (section below): a drop-in replacement wherever you call that model through llama-server.
• Honest when unsure. The model answers refuse when a request is not a shell task and unsure when it is ambiguous, instead of inventing a command.
• The model does not rate risk. In dale, a deterministic verifier decides the risk of every command and dangerous commands need a typed confirmation. Outside dale there is none: your application must review commands before running them.

The model was trained only for this contract. It is not a general chat model.

Speed on a GPU

dale does not download this model, but dale --model dale-2b-1.0.0-Q4_K_M.gguf runs it like the others, on the GPU for example with device = gpu in its configuration. Measured that way, with Vulkan: 0.86 s median per request (0.91 s p90) and 1.8 GiB of VRAM.

Times are for the whole dale process per request (start, load, prompt, generation) on one reference desktop with a 16 GB GPU: 30 requests, three warm runs each. They depend on the machine.

Files

File	Quantization	Size (bytes)	SHA-256
dale-2b-1.0.0-Q4_K_M.gguf	Q4_K_M, with imatrix	1,274,397,216	407fa38a07d0c03dbf247ae43c4624dec5cf744505bfc62cfa3baca2037ef146
LICENSE	Apache-2.0

sha256sum dale-2b-1.0.0-Q4_K_M.gguf

The GGUF carries general.* metadata (name, author, URL, version, license, base model). The tensors are identical to the file evaluated below; only the metadata was added.

Using it with llama-server (CPU or GPU)

Follow the contract (version 2) exactly; the model was trained on this format and nothing else. Reasoning is always disabled.

Prompt

The prompt is Qwen3.5's chat template with thinking disabled (apply_chat_template(..., add_generation_prompt=True, enable_thinking=False)), with two turns only: a fixed system prompt and the user message. No few-shot examples, no reasoning.

System prompt (exactly two lines, no trailing newline):

Turn the request into one bash command for the Linux system described in the first line of the user message.
Reply with only JSON {"type":"command|refuse|unsure","command":"...","explanation":"..."}: "command" is "" unless type is command; "explanation" is one short sentence in the language of the request.

User message: a context line, a newline, then the request (whitespace collapsed to single spaces, at most 1000 bytes):

os=linux distro=ubuntu-24.04 pm=apt tools=curl,docker,fdfind,git,jq,python3,rg,rsync,ssh,systemctl
busca en este directorio los ficheros que contienen la palabra TODO

Context key	Value
os	linux
distro	ID-VERSION_ID from os-release, lowercase (or just ID, or unknown)
pm	apt, dnf, yum, pacman, zypper, apk or none
tools	optional tools found in PATH, alphabetical, comma-separated, or none

The tools list dale checks: fd fdfind rg bat batcat eza tree ncdu fzf jq yq zip unzip 7z zstd xz unrar curl wget ip ss netstat nmcli dig nc nmap ssh rsync iptables nft ufw firewall-cmd systemctl journalctl lsof htop sudo doas hostnamectl timedatectl loginctl setfacl getfacl docker podman kubectl apt dnf yum pacman zypper apk flatpak snap git python3 make gcc ffmpeg convert magick exiftool pdftotext. Near-universal tools (coreutils, find, grep, sed, awk, tar, gzip) are not listed.

Full prompt string (line breaks shown for readability; \n are real newlines):

<|im_start|>system\n{system}<|im_end|>\n<|im_start|>user\n{context}\n{request}<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n

The prompt ends with an empty <think> block, so the model starts writing the JSON straight away. Tokenize with special tokens parsed and without BOS.

Answer

Exactly one JSON object with exactly these three string keys:

{"type":"command|refuse|unsure","command":"...","explanation":"..."}

• type is command, refuse (not a shell task) or unsure (ambiguous request).
• command is one bash command (steps joined with && or |), no newlines; it is "" unless type is command.
• explanation is one short sentence in the language of the request.
• Nothing before or after the object. Treat anything else as invalid and run nothing.

Decode greedily and cap at 200 tokens.

Example

1. Install llama.cpp: a prebuilt binary from the releases, brew install llama.cpp, or the Docker image ghcr.io/ggml-org/llama.cpp:server (:server-cuda or :server-vulkan for a GPU).

2. Start the server. It downloads the model from this repository the first time:

   llama-server -hf manuelgscom/dale-2b-GGUF:Q4_K_M -ngl 99 --port 8080
   

   Or download the file yourself, check it and point to it:

   curl -LO https://huggingface.co/manuelgscom/dale-2b-GGUF/resolve/main/dale-2b-1.0.0-Q4_K_M.gguf
   sha256sum dale-2b-1.0.0-Q4_K_M.gguf     # must match the SHA-256 in Files
   llama-server -m dale-2b-1.0.0-Q4_K_M.gguf -ngl 99 --port 8080
   

   -ngl 99 puts every layer on the GPU; on CPU only, leave it out. Any llama.cpp backend works (CPU, CUDA, Vulkan, Metal…); the dale CLI uses Vulkan.

3. Send a request with one of the snippets below (Python 3, standard library only).

Raw prompt with /completion (recommended: you control the exact string):

import json, urllib.request

SYSTEM = (
    "Turn the request into one bash command for the Linux system described in the first line of the user message.\n"
    'Reply with only JSON {"type":"command|refuse|unsure","command":"...","explanation":"..."}: '
    '"command" is "" unless type is command; "explanation" is one short sentence in the language of the request.'
)
context = "os=linux distro=ubuntu-24.04 pm=apt tools=curl,docker,fdfind,git,jq,python3,rg,rsync,ssh,systemctl"
request = "busca en este directorio los ficheros que contienen la palabra TODO"

prompt = (
    f"<|im_start|>system\n{SYSTEM}<|im_end|>\n"
    f"<|im_start|>user\n{context}\n{request}<|im_end|>\n"
    "<|im_start|>assistant\n<think>\n\n</think>\n\n"
)
body = {"prompt": prompt, "n_predict": 200, "temperature": 0, "top_k": 1}
req = urllib.request.Request("http://127.0.0.1:8080/completion",
                             data=json.dumps(body).encode(),
                             headers={"Content-Type": "application/json"})
print(json.load(urllib.request.urlopen(req))["content"])

Chat API with /v1/chat/completions: the server applies the chat template stored in the GGUF, so you must turn reasoning off with chat_template_kwargs (with an older llama.cpp build, also start the server with --jinja):

body = {
    "messages": [
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": f"{context}\n{request}"},
    ],
    "chat_template_kwargs": {"enable_thinking": False},
    "max_tokens": 200, "temperature": 0, "top_k": 1,
}
req = urllib.request.Request("http://127.0.0.1:8080/v1/chat/completions",
                             data=json.dumps(body).encode(),
                             headers={"Content-Type": "application/json"})
print(json.load(urllib.request.urlopen(req))["choices"][0]["message"]["content"])

The chat template in this GGUF is Qwen3.5-2B's, unchanged, and it already disables reasoning by default. Pass "enable_thinking": false anyway, so the prompt does not depend on that default.

Valid answers look like:

{"type":"command","command":"rg -l TODO","explanation":"Lista los ficheros que contienen TODO"}
{"type":"refuse","command":"","explanation":"This is a general question, not a shell task."}
{"type":"unsure","command":"","explanation":"No está claro qué hay que arreglar; indica el fichero o el problema."}

Never run the command blindly. Outside dale there is no risk verifier: add your own review step.

Evaluation

Test v2: 471 cases (339 command requests, 101 out-of-scope, 31 ambiguous; 257 in Spanish, 214 in English). Published as manuelgscom/dale-bash-test-v2.

All models in Q4_K_M, reasoning disabled, greedy decoding, each in its own llama-server. Base models use contract v1 (the system prompt plus 8 few-shot examples, which suits a model that was not fine-tuned); the dale models use contract v2 (above).

Model	Params	GGUF	Correct	Accuracy	95 % CI (Wilson)
Qwen3.5-0.8B (base)	0.8 B	532.5 MB	70	14.9 %	11.9–18.4 %
Qwen3.5-2B (base)	2 B	1,280.8 MB	152	32.3 %	28.2–36.6 %
Gemma 4 E2B-it	2 B effective	3,106.7 MB	254	53.9 %	49.4–58.4 %
Qwen3.5-4B (base)	4 B	2,740.9 MB	257	54.6 %	50.0–59.0 %
Gemma 4 E4B-it	4 B effective	4,977.2 MB	296	62.8 %	58.4–67.1 %
Qwen3.5-9B (base)	9 B	5,680.5 MB	296	62.8 %	58.4–67.1 %
dale-0.8b	0.8 B	529.3 MB	374	79.4 %	75.5–82.8 %
dale-2b	2 B	1,274.4 MB	385	81.7 %	78.0–85.0 %
dale-4b	4 B	2,708.8 MB	410	87.0 %	83.7–89.8 %
dale-9b	9 B	5,629.1 MB	420	89.2 %	86.0–91.7 %

MB = 10^6 bytes.

Against dale-0.8b, case by case on the same server (McNemar): dale-2b gets 32 cases right that dale-0.8b misses, and misses 18 that dale-0.8b gets right (p = 0.065).

Where the errors are

Metric	Qwen 9B	Gemma E4B	dale-0.8b	dale-2b
Invalid JSON (of 471)	3	17	0	0
Correct refusals (out-of-scope, of 101)	101	101	101	101
False refusals (command cases, of 339)	17	13	0	2
Correct unsure (ambiguous, of 31)	3	4	31	31
Wrong type	45	40	0	2
Wrong command	127	118	97	84

Fine-tuning removes format errors and the "when not to answer" errors (refusals, ambiguity). Of the 86 remaining errors, 84 are wrong commands.

By language: 80.9 % in Spanish (208/257) and 82.7 % in English (177/214).

Per-category table (correct / cases)

Category	Cases	Qwen3.5-9B	Gemma 4 E4B-it	dale-0.8b	dale-2b
ambiguous	31	3	4	31	31
compression	23	14	17	16	18
disks	22	10	6	13	18
docker	19	16	14	14	14
files	42	19	21	26	28
out of scope	101	101	101	101	101
git	28	14	18	21	20
misc	25	15	16	18	20
packages	28	15	20	24	24
permissions	28	18	20	23	23
processes	30	13	12	21	20
network	25	17	14	21	20
system	29	20	24	22	23
text	40	21	9	23	25

Methodology

• Independent test. Test v2 was written by a different, independent LLM from a written specification, validated automatically and frozen (SHA-256 fdb5564af3c153804b8780b36f9b22326340e1a57a45907765a45ab4f39991f3). No data-generation step ever read it, and the training data was decontaminated against it.
• Double judge with debate. Format and type are checked by rules. Each proposed command is judged by two independent LLM judges; if they disagree, they debate one round, and if they still disagree, the answer counts as wrong.
• Same conditions for everyone. Each model ran in its own llama-server, with greedy decoding, seed 0 and a 200-token cap. The larger dale models were measured on a later run of the same server, with dale-0.8b as a control (371/471 there, 374 in the main run).
• Verdicts have no manual review. With 471 cases the 95 % interval is about ±3 points for this model, and much wider per category.

Training

Full fine-tune of all weights of Qwen/Qwen3.5-2B, with the same recipe as dale-0.8b. Same dataset as dale-0.8b (48,841 training examples). The training data and the training recipe are not published; see the dale-0.8b card for how the data was built and the tldr-pages attribution. Quantized to Q4_K_M with an importance matrix computed on a sample of the training data.

Limitations

• bash on Linux only. No zsh, fish, PowerShell or cmd; no macOS. On Windows, use it through WSL 2.
• It can be wrong: it gets 385 of the 471 test cases right. A wrong command may look plausible. Always review a command before running it.
• It is not a security tool. The model can propose a destructive command and does not classify risk. In dale, the verifier and the confirmations do that; outside dale, your application must.
• Trained for one task with one fixed prompt. Other prompts, chat use or reasoning mode are unsupported.
• Tool choice depends on the context line: with no or wrong context it may suggest tools that are not installed.

Changes from the base model

• All weights fine-tuned for the contract above (natural language → one bash command as JSON, with refuse and unsure).
• Tokenizer and chat template unchanged from Qwen3.5-2B.
• Converted to GGUF and quantized to Q4_K_M with imatrix; general.* metadata added.

License

Apache-2.0, the same license as the base model Qwen3.5-2B. See LICENSE. The training seeds from tldr-pages are CC BY 4.0, attributed in the dale-0.8b card.