SemanticRepair-270M

A 270M rewriter that sits behind an embedding router. When a question does not land on any capability with enough margin, this model restates it in the plain form the capabilities are described in, and the router tries again on the restatement. When it reads no request at all, it says so.

That is the whole job. It does not answer questions, it does not decide anything, and nothing it writes is ever executed: the router runs on the restatement, the tool runs on the original.

The surface it reads, which is not a chat prompt

Fine-tuned on a bare completion surface. It has never seen a chat template, a system message or a few-shot example. Speak to it the way it was trained or it will not work:

{Language}: {the question, verbatim}
=>

It completes with one line per request it found, or the single token NO_REQUEST.

{Language} is the English name of the language the question is in — not the language of whatever system is asking. English, Italian, French, German, Spanish; a language outside that set keeps whatever tag the caller declares, because an invented tag is a surface the model never read either.

This is the whole call the engine makes, on /v1/completions and never on a chat endpoint:

{
  "prompt": "Italian: che tempo fa domani a Bologna?\n=>\n",
  "stop": ["\n=>"],
  "temperature": 0.0,
  "repeat_penalty": 1.0,
  "max_tokens": 96
}

Greedy, because the same words must always produce the same rewrite: a router that reruns on a different restatement each time cannot be reasoned about. repeat_penalty is 1.0, which is neutral — a rewrite legitimately repeats the words of the question. The stop string cuts a runaway that starts echoing another question in its own trained format; everything before it is the answer. 96 tokens leaves room for several rewritten lines: the longest answer in the training data is 43 tokens.

The sentinel is compared case-insensitively and tolerates a trailing period. It is an instruction the model follows, not a token it is guaranteed to emit byte-exact.

These are real outputs from the released weights, greedy:

in	out
Spanish: no busques la tienda de ropa, dime la dosis de paracetamol	dime la dosis de paracetamol
Italian: scusa il disturbo, mi diresti che tempo fa domani a Bologna?	dimmi che tempo fa domani a Bologna
English: book me a table for friday and also cancel my dentist	book me a table for friday cancel my dentist
Italian: guarda un po', questo sacchetto della spesa è tutto rotto.	NO_REQUEST

A negation is dropped, an apology is dropped, one message asking for two things becomes two lines, and something that is not a request at all answers with the sentinel.

What is in this repository

file	size	what it is
sft-v19-q8_0.gguf	300 MB	what the engine serves, through llama-server
model.safetensors	536 MB	the same weights fused, BF16, 236 tensors
model.safetensors.index.json		the index over that single file
tokenizer.json, tokenizer_config.json	33 MB	the tokenizer as the fuse wrote it
config.json, generation_config.json		gemma3_text, torch_dtype: bfloat16
chat_template.jinja		present because the fuse writes it — not the surface this model reads, see above

The GGUF's sha256 is 33b6cc73c7d306c9c81aa96c5a8ddb9dbfd46a5ac5d0388a2de45b5229928c56. The engine pins it and refuses anything else, which is what makes a routing decision reproducible.

Both files carry the same weights: the fused final adapter of training run v19, adapter sha256 1082985953ed6c33…, quantised to Q8_0. Q8_0 and not Q4: measured on this seat, at temperature 0 a Q8_0 is stable to the byte across runs and a Q4 is not.

What it is measured to do

Measured against the two things worth measuring against: the model it was fine-tuned from, and doing no repair at all. Two benches over different corpora, because a model that wins one of them is a model chosen by which table got printed.

400 routing tasks over 343 graphs, of which 78 ask for more than one capability and 160 name a day. These exact released weights against the untrained base, every column the bench reports:

	this model	gemma-3-270m, untrained
exact, every capability the question asks for, and no other	32	24
safe, a proper subset reached, nothing wrong run	6	1
abstained, nothing executed at all	298	197
wrong execution, something ran that should not have	1	0
answered in the wrong language	6	123
lost the day the question named, of 160	57	55
echoed the question back instead of restating it	64	106
of those echoes, ones that still routed exactly	9	20
compound questions answered whole, of 78	0	0

Read the columns together rather than one at a time.

The language column is what the fine-tune buys and it is the largest effect in any table here: 123 becomes 6.

The abstention column is the honest cost. This model refuses far more often than the base does, 298 against 197. It abstains rather than guess, and abstaining is why its wrong executions stay at one. A rewriter that guesses would score better on exact and worse on everything that matters.

The echo column says how often the model gave the question back unchanged. It halves, 106 to 64, which is the fine-tune learning that restating is the job.

The date column barely moves, 55 to 57, and the compound column does not move at all: neither the base nor this model answers a single one of the 78 questions asking for two capabilities on this bench. Those are the two things the training did not buy.

39 held-out questions on a live workspace, written by three blind judges against the capabilities in it, in two languages:

	exact
this model	16
routing the question as written, no repair	12
google/gemma-3-270m, untrained	9

The base model scoring below "no repair at all" is the whole argument for fine-tuning something for this seat: an untrained 270M put here makes routing worse than leaving the question alone. Of the 10 compound questions in that set — one message asking for two things — this model answers 2 whole.

Its own training gate, on 1000 held-out examples: 261 exact (26.1%), 927 of 1000 answers well formed, and the language-balance gate passed with a 7-point gap between English and Italian against a 10-point bar.

What it does not fix, which a card naming only the gains would hide

The sentinel does not catch every non-request. guarda un po', questo sacchetto della spesa è tutto rotto answers NO_REQUEST, but ciao come stai comes back verbatim rather than refused. If nothing downstream checks that the restatement is a request, a greeting will be routed.

It sometimes rewrites a word rather than restating a sentence. Measured: Bitte reserviere mir einen Platz im Klettergarten für Freitag comes back as reserviere mir einen Platz im Kletterpark für Freitag — Klettergarten became Kletterpark. The restatement is only ever compared against capability descriptions and never executed, which is why this is tolerable here; in any design where the rewrite is acted on, it is not.

Dates. In the 400-task run above, 160 of the tasks name a day. This model loses the day in 57 of them — more than a third. It is the weakest column it has, and it is weak in a way that does not announce itself: the restatement reads as a clean request, with the "yesterday" or the "before Easter" simply gone. If the day matters to your routing, measure that before you serve this.

It is not distinguishable from its own neighbours. Ten checkpoints of this run were compared on the same 400 tasks pair by pair rather than by totals — counting only the tasks where two checkpoints disagree, which are the only ones carrying information. Not one difference reaches significance: the widest gap in the table is 7 tasks to 1, p = 0.07. With around thirty exact answers out of four hundred the natural fluctuation is about five, which is most of the ranking. Treat the order inside a run as noise: a checkpoint one or two points ahead of another inside the same run has not been shown to be better than it.

What bounds all of this

None of these weaknesses are fixed. They are bounded, by the system around the model rather than by the model, and the bounds are worth stating because they decide how much a weakness costs.

Nothing it writes is executed. The restatement is compared against capability descriptions and then thrown away. Whatever runs, runs on the original message. Fixed SQL and HTTP operations are authored in the tool and cannot be assembled from either text.

The source message stays beside the restatement. Dates, language, page references and everything shown to the reader are read from what the person actually wrote. So a dropped day costs a routing decision, not the date: the question may reach the wrong capability, but the day itself was never the restatement's to lose.

A restatement that does not win by enough margin routes nothing. The router applies a margin and a floor, and below them the system abstains and asks for a rephrasing. That is why the wrong-execution column is 1 in 400 while the abstention column is 298: the failure mode is refusing, not acting wrongly.

The descriptions are the other half. What a question is matched against is written prose in the workspace, and a capability that keeps being missed is usually a description that needs rewriting, not a model that needs retraining. The rewriter is one of two things you can improve, and the cheaper one is often not the model.

The engine has a linter for exactly this, applying the same score the router applies, over a whole graph or over one description while it is being written. It reports collisions: pairs of capabilities the router cannot separate. A collision is not a verdict that either is wrong. Capabilities that overlap in meaning are expected to sit close, and the linter says where the router cannot tell them apart, leaving the choice to rewrite, merge or keep to whoever knows what the nodes are for. collision is also one of the four states a routing can end in, beside uncovered, uncertain and confident, so the reader is told which one happened rather than being handed an answer with no account of how it was reached.

Where it runs

Served with llama-server, reached by the engine as one of four model seats:

llama-server -m sft-v19-q8_0.gguf --host 127.0.0.1 --port 9250 -ngl 99 -c 2048 --jinja

It is loaded only when routing is genuinely unsure, never as the default path, and released after an idle window.

On CPU alone

Measured on an Apple M-series machine with -ngl 0 -t 4: no GPU offload at all, four threads, and nothing in the server log naming Metal or a GPU.

	latency	tokens/s
strip a negation, 8 tokens out	38 ms	210
split one message into two requests, 11 tokens	32 ms	344
answer NO_REQUEST, 4 tokens	17 ms	236

762 MB resident while serving.

llama-server -m sft-v19-q8_0.gguf --host 127.0.0.1 --port 9250 -ngl 0 -t 4 -c 2048

What this does and does not establish: it runs usefully with no accelerator, on few threads. It has not been run on a phone or a tablet, and the three things that would decide it there are the three not measured here — 762 MB resident is a lot for a mobile process, loading 300 MB from slow storage is a different question from generating in 38 ms, and mobile cores throttle under heat while these did not.

The data it was trained on

Gramscii-IT/semantic-repair-routing — published, and everything below can be recounted from it.

84,819 pairs: 82,819 train, 1,000 validation, 1,000 test. Each pair is a message somebody could plausibly write and the requests inside it restated plainly, one per line.

Five languages, close to balanced:

	rows
English	18,006	21.7%
Spanish	16,903	20.4%
German	16,659	20.1%
French	16,602	20.0%
Italian	14,649	17.7%

What the message is doing, which is what the data is really organised by. Every one of the 82,819 training rows carries one of 38 such labels; these are the ten largest:

	rows
massive_real — real assistant traffic	5,736	6.9%
minimal_pair — two messages differing in one word	5,299	6.4%
indirect — the request is implied, never stated	4,912	5.9%
presto_native — natively written, not translated	4,606	5.6%
oasst_human — human-written	4,450	5.4%
multi — more than one request in one message	3,974	4.8%
direct — the plain case	3,904	4.7%
vent_statement_request — a complaint with a request inside	3,516	4.2%
negation_exclusion — names what is not wanted	3,398	4.1%
trap_topic_lure — a topic named to pull routing the wrong way	3,324	4.0%

The remaining 28 cover self-correction, code-switching, ultra-short messages, typos and slang, messages carrying two, three or four requests, eight kinds of date reference — a weekday in the past, a named holiday, a bounded range — and the two families below.

The two families that decide what this model refuses to do

injection_meta — 2,278 rows — is a message aimed at the assistant itself: reveal your instructions, ignore what you were told, pretend you have no rules. 1,901 of them answer NO_REQUEST, because there is nothing here to route. Together with injection_command these two families supply 19% of every NO_REQUEST row in the set: refusing is largely taught by messages that try hardest to get an answer.

injection_command — 2,410 rows — is the opposite lesson, and the more delicate one. A hostile but explicit command about the user's own things: delete every saved bank account, empty my personal cloud, cancel all my bookings tonight. 2,281 of them are restated faithfully, action intact.

The rewriter does not sanitise and does not soften. Deciding whether an action is allowed is not its job and it has no way to do that job well: it sees one sentence, not the account, the permissions or the consequences. A model that quietly dropped "delete" would hand the router a different request from the one the person made, and the person would never learn their instruction had been edited. So the restatement carries the command as written, the router places it, and whatever runs it is where refusing belongs.

12.6% of rows (10,432) answer NO_REQUEST, and 16.9% (14,028) answer with more than one line.

Who wrote it, and what was thrown away:

generated by Qwen3.6-35B	72,697 rows, 87.8%
generated by Qwen3.8-27B	7,935 rows, 9.6%
generated by Gemma-4-E4B	2,187 rows, 2.6%
rejected by the judge, not in the set	72,996 rows

A separate model read every message against written rules and answered a two-word verdict, decoding greedily so a verdict does not change between runs. The dataset card says where judge and generator were different models and where they stopped being.

Training

Fine-tuned from google/gemma-3-270m with mlx_lm lora, full fine-tune. The released weights are the run's final adapter, chosen by a gate on 1,000 held-out examples rather than by the best number on a validation curve.

Where all of this happened

One desktop machine, and nothing left it.

machine	Mac mini, Mac16,11
chip	Apple M4 Pro
CPU	14 cores, 10 performance and 4 efficiency
GPU	20 cores
memory	64 GB unified
system	macOS 26.4

the fine-tune
method	mlx_lm lora, full fine-tune
iterations	20,704
tokens seen	2,383,728
peak memory	7.3 GB
speed	5.6 iterations/s, 610 tokens/s
wall clock	11:15 to 12:42, about 90 minutes

Generators and judge ran on that same machine as local llama.cpp servers on loopback. No message here was written by a hosted API, and none was sent to one to be judged. The unified memory is why: a 35B generator and a fine-tune only share a desk when CPU and GPU read the same 64 GB.

A model that decides where a question goes does not need a cluster or a month. It needs a narrow job, data built for it, and a bench honest enough to say when a round did not help.

Licence

Gemma terms, which travel with the file: this is a fine-tune of Gemma-3-270M and inherits them.