howdo-4b-v4

howdo-4b-v4 is a 4.0B parameter language model fine-tuned for single-line natural-language-to-shell-command translation (POSIX and Bash).

The model is designed for fast, local inference with llama.cpp and powers the howdo CLI tool.

Model Description

• Base Architecture: Qwen3.5-4B (language backbone with vision encoder weights removed to minimize memory footprint and inference latency).
• Target Task: Converting natural language terminal requests into precise, single-line POSIX and Bash commands.
• Quantization: Q4_K_M GGUF (2.57 GB).
• Context Length: 512 tokens (optimized for fast single-turn terminal execution).
• License: Apache-2.0.

Training Details

The model was fine-tuned using LoRA (rank 32, alpha 64) on 69,000 verified command generation examples, with particular focus on:

• Scope boundary enforcement (e.g. non-recursive -maxdepth 1 constraints).
• Option terminators for leading dashes (e.g. -- for filenames beginning with -).
• Safe pipeline patterns (e.g. find -print0 | xargs -0).
• Flag arity and modern CLI replacements (rg, fd, jq, sed, awk, tar).

Usage

With howdo CLI

Install the howdo tool:

git clone https://github.com/Mahsery/howdo.git
cd howdo
cargo install --path .
howdo setup

Generate commands:

howdo find all empty directories in the current folder only

With llama.cpp

Run the GGUF model directly using llama-cli:

llama-cli -m howdo-4b-v4-Q4_K_M.gguf \
  -p "<|im_start|>system\nYou are a shell command generator. Output exactly one line: a single POSIX/bash command that accomplishes the user's request. No prose, no markdown fences, no explanation.<|im_end|>\n<|im_start|>user\nfind all empty directories in the current folder only<|im_end|>\n<|im_start|>assistant\n" \
  -n 64 --temp 0.0

Prompt Format

The model expects the ChatML format with a strict single-line instruction system prompt:

<|im_start|>system
You are a shell command generator. Output exactly one line: a single POSIX/bash command that accomplishes the user's request. No prose, no markdown fences, no explanation.<|im_end|>
<|im_start|>user
[Your request in plain English]<|im_end|>
<|im_start|>assistant