termite-command-4b 0.2

This model turns a request into shell commands, and refuses hostile requests. It was trained for termite, which checks every command before anything can run.

Do not run its output without a check. This model reduces how often a hazard is written. It does not stop it. It is one layer, and termite's rules are the layer that decides.

Files

File	Size	SHA-256
termite-command-4b-0.2-Q8_0.gguf	4.28 GB	31971d21762b56e611fa932cb5684feef35344e5f3dd77b19521cc6d4b960408
termite-command-4b-0.1-Q8_0.gguf	4.28 GB	f274aaf4b7dbfe6977f645902391104d340047755bc615df2f675d49d1381c9e

Use it

llama-server -hf tanepiper/termite-command-4b -hff termite-command-4b-0.2-Q8_0.gguf --jinja -c 8192 --port 8080

Then run tmite setup, or set model_url and model in termite's config. The model expects termite's system prompt and replies in termite's JSON format. With any other prompt, the scores below don't apply.

Score card

Version 0.2 was rebuilt using only training data with clear licences. It writes fewer hazards and offers more alternatives than 0.1, but it refuses more requests that it should answer. All results below come from fixed test sets, and every version gets the same tests.

Measure	0.1	0.2	0.2 mark	Result
Hostile requests with a hazard	3 of 100	1 of 100	3 or fewer	Pass
Everyday answers with two or more commands	5 of 40	17 of 40	30%	Pass
Outside everyday answers with a hazard	8 of 58	5 of 51	4 or fewer	Miss
Everyday commands that ran, of 40 requests	30	30	33 or more	Miss
Everyday requests answered usefully	40 of 40	38 of 40		Worse
Outside prompt injection requests answered	2 of 30	1 of 30		Better
Outside harmful requests refused	30 of 30	30 of 30		Same
Outside everyday requests answered	58 of 60	51 of 60		Worse

The hazard that went through on a hostile request stopped for confirmation in termite. It did not go straight in.

• Hazard means that termite's rules rated a command medium severity or above.
• Useful means that the first program in a command is one that the request's test case expects. This is a weak measure: it doesn't show that the command is right.
• The outside requests were never trained on. They come from public sets of everyday, harmful and prompt injection requests.

Commands that run

Everyday commands ran in a throwaway Linux machine with no network card. These are the best results for each request. Requests the model refused are not in the counts.

	Requests	Ran	Tool not installed	File not there	Failed or broken
0.1, our test	40	30	5	2	3
0.2, our test	39	30	5	1	3
0.1, outside set	58	27	18	8	5
0.2, outside set	51	22	14	6	9
0.1, second everyday set	50	38	2	2	8
0.2, second everyday set	50	40	0	1	9

What changed in each release compares each version with the one before it.

Why some scores went down

• It refuses more. It declined 9 of 60 outside everyday requests. Five of those were fair requests, and three of them touch system folders. Three had no real job in them, and one was right to refuse. It also refused 3 of 21 risky requests that a person might really make, and one harmless request in our everyday test. The cause is probably that the new training data has too few allowed jobs near system folders. This isn't confirmed yet.
• The everyday score lost 2. One was that refusal. The other answer does use an expected program, but not first, so this measure can't see it.
• Fewer outside commands ran. It answered fewer requests, and 2 of its commands used options that don't exist. Version 0.1 had none of those on this set.

What the next version will try

1. Add allowed jobs near system folders, and for risky jobs on a person's own files, to reduce refusals.
2. Find out why it writes options that don't exist.
3. Train each data set more than once, and report the spread.

Known limits

• It can write a hazard for an everyday request. On the outside set, 5 of 51 everyday answers were rated hazards.
• It refuses some fair requests. See above.
• It can be wrong about risk. Its risk label is the model's opinion. Never use it as a safety check.
• It can make up a program or an option. Expect more of this with rare tools.
• It is not a chat model. Outside termite's prompt it can answer badly, make up results or refuse harmless requests. termite sends each request on its own.
• It sometimes answers what it should refuse. It wrote a command for 1 of the 30 outside prompt injection requests. It was not rated a hazard.
• Answers change a little between runs. The tests use a temperature of 0.2 with no fixed seed. The grading of each answer is fixed, but the same test can differ by a row or two.
• The main test has been read closely. It was used to choose between runs, so its scores are likely to flatter the model. The outside results are stronger evidence, but they are small samples.
• The full tests ran on macOS. The commands themselves ran in a Linux test machine.
• English only. A few refusal rows are in German and French. Nothing else was tested.

Training

• Base: Qwen/Qwen3-4B-Instruct-2507, revision cdbee75f17c01a7cc42f958dc650907174af0554, Apache 2.0.
• Method: LoRA, rank 16, alpha 32, on all linear layers, then merged. Two passes, learning rate 0.0001, batch size 8.
• Data: 6,538 rows, which are not published: 5,716 from tldr-pages and 822 written for this project.
• Filtering: every target passed termite's rules. Rows that matched a test set were removed.
• Refusals: a hostile example teaches refusal. No harmful command is a training target.
• Conversion: llama.cpp convert_hf_to_gguf.py, Q8_0.

Training data sources

Version 0.2 was rebuilt from sources that were checked back to their origin. It uses no data from the sets that trained 0.1.

Source	Licence
tldr-pages, commit b9bf7d8308486b5a0361126f1984436f4d709904	CC BY 4.0, © the tldr-pages team and contributors
Rows written for this project	Apache 2.0

The tldr-pages text was changed: placeholders were filled in, and pages were turned into requests and commands. No source data is published here.

Licence

Apache 2.0, the same as the base model. This is a fine-tuned version of the base model.