fpenin/coyote-low

Coyote Low

A 3B on-device router that decides, in a single JSON response, whether a coding-agent request can be handled locally or must be escalated to a frontier model.

Coyote Low is the local tier of the Coyote model family: a small model runs on the developer's machine and handles mechanical requests itself; anything that needs real reasoning is escalated to a large cloud model. In the author's measurements, running this two-tier setup inside a modified version of OpenCode that routes between both models cut large-model token usage by ~75%.

Why

Most of the messages a developer sends to a coding agent are mechanical: "run the tests", "git status", "show me package.json", "build the binary". Sending them to a frontier model wastes tokens, money and latency. Coyote Low intercepts every message, resolves the mechanical ones locally with a single shell command, and forwards only the requests that need code generation, debugging, reasoning, design or review.

                     ┌───────────────────────────────┐
 developer message ─▶│  Coyote Low (3B, local, ~2GB) │
 + workspace context │  temperature 0 → one JSON obj │
                     └───────────────┬───────────────┘
                                     │
            can_resolve: true        │        can_resolve: false
       ┌─────────────────────────────┴─────────────────────────────┐
       ▼                                                           ▼
 run shell command locally                              escalate to the large model
 (0 cloud tokens)                                       with a typed reason

Highlights

• Cascade routing for coding agents: the local model acts as a gate in front of a frontier LLM, so cloud tokens are only spent on requests that actually need them (~75% fewer tokens in the author's OpenCode setup).
• Runs fully on-device: Q4_K_M GGUF, 1.9 GB, served with Ollama / llama.cpp on a laptop. No GPU required.
• Deterministic, machine-readable output: greedy decoding and a strict JSON contract, so the agent can act on the answer without parsing free text.
• Typed escalation: when the model declines, it says why (requires_code_generation, requires_debugging, requires_reasoning, requires_design, requires_review, ambiguous_intent), which lets the harness pick the right downstream model or prompt.
• Stack-aware commands: uses the workspace context (language, package manager, branch, current file) to emit the right command per ecosystem, e.g. a "type check" becomes tsc --noEmit in TypeScript, mypy . in Python and cargo check in Rust.
• Bilingual and informal: trained on English, Spanish (including Rioplatense), Spanglish and typos (gti status, tirá los tests, dale un type check).
• Adversarially trained against false positives: includes requests that look mechanical but are not ("commit the good parts", "delete the failing tests", "skip the slow tests"), so the router escalates instead of guessing.

Model details

Base model	Qwen2.5-Coder-3B-Instruct (3.1B params, 36 layers, GQA 16 query / 2 KV heads)
Method	QLoRA (4-bit base) with Unsloth + TRL SFTTrainer
LoRA	r = 32, α = 64, dropout 0, on all attention and MLP projections (q,k,v,o,gate,up,down)
Optimization	AdamW 8-bit, lr 2e-4, cosine schedule, 30 warmup steps, effective batch 8 (2 × 4 grad. accumulation), bf16, gradient checkpointing
Sequence length	4,096 tokens
Model selection	evaluation every 50 steps, best checkpoint by validation loss
Compute	NVIDIA A10G on Modal (serverless GPU)
Export	LoRA merged into 16-bit weights → llama.cpp convert_hf_to_gguf (F16) → Q4_K_M quantization
Serving	Ollama, ChatML template, temperature 0, top_p 1.0, 4k context

Training data

A purpose-built synthetic dataset of developer requests paired with the target routing decision, generated per subcategory and checked by a schema and count validator before training.

• Resolvable locally: navigation and search, git operations, command execution (tests, build, lint), file operations, state queries, formatting and tooling.
• Must escalate: code generation, explanation and refactoring, debugging, design, code review, and ambiguous intent.
• Identity: the model presents itself as Coyote Low instead of its base model.
• Coverage: TypeScript, Python (Django and FastAPI), Go, Rust, Java and PHP workspaces with their package managers (npm, pnpm, pip, poetry, go, cargo, maven, composer).
• Split: 80 / 10 / 10 train / validation / test (seed 42).

Usage

Ollama

hf download fpenin/coyote-low coyote-low-q4_k_m.gguf Modelfile --local-dir coyote-low
cd coyote-low
ollama create coyote-low -f Modelfile

Prompt format

[TASK: resolve]
cwd=/Users/dev/api lang=typescript pm=npm branch=main file=src/main.ts
Message: tirá los tests

Output contract

Resolved locally:

{"can_resolve": true, "command": "npm test", "response": "Running the test suite."}

Escalated:

{"can_resolve": false, "reason": "requires_code_generation"}

Identity questions ("¿quién sos?", "what model are you?") return "command": "respond:identity" with a short self-introduction.

Integration tips

• Call it on every user message before the frontier model; on can_resolve: false, forward the original message unchanged.
• Auto-execute only read-only commands (git status, git log, ls, cat, test runners) and ask for confirmation on anything that writes.
• Block destructive patterns outright (rm -rf, sudo, git push --force, git reset --hard).

Limitations

• It routes; it does not write code. Multi-step tasks should be escalated.
• Trained on synthetic data; commands for uncommon toolchains may need review.
• Always review a command before executing it with side effects.

License

Derived from Qwen2.5-Coder-3B-Instruct and distributed under the Qwen Research License.

Author

Built by Federico Penin (@fpenin).

@misc{penin2026coyotelow,
  title  = {Coyote Low: an on-device router for cascaded coding agents},
  author = {Penin, Federico},
  year   = {2026},
  url    = {https://huggingface.co/fpenin/coyote-low}
}