e2chndr/evoo-ai-v1

Evoo AI v1

A small language model fine-tuned to polish dictation. It takes what a speech recogniser wrote and returns what the speaker meant to type: fillers and stutters removed, self-corrections applied, punctuation and capitals fixed, and the speaker's own words kept. It is trained to type a dictated question or request, not to answer or act on it.

Evoo AI v1 is the polish model of Evoo, an open-source voice dictation app for macOS that runs entirely on your Mac. It was fine-tuned from Qwen3-0.6B, and the whole fine-tune ran on an 8 GB M1 MacBook.

Dictated (as the recogniser wrote it)	Evoo AI v1
Email the cast agent about the about the reshoot the reshoot dates.	Email the cast agent about the reshoot dates.
Well uh smart board would be very uh nice to work with if it worked really well.	Smart board would be very nice to work with if it worked really well.
Who is the current Premier I mean the current Governor of Victoria?	Who is the current Governor of Victoria?
Hey Priya do you know why the login screen write crashes when the session token expires?	Hey Priya, do you know why the login screen crashes when the session token expires?
can you explain how a transformer works like in simple terms and uh give me an example	Can you explain how a Transformer works in simple terms and give me an example?

The last one is a request, and it stays a request: the model polishes it instead of answering it.

At a glance

Version	v1, October 2026
Base model	Qwen3-0.6B (Alibaba, Apache-2.0), 596M parameters
Method	LoRA: rank 8, scale 20, on 16 of the 28 layers; 2.9M trained parameters (0.48%); merged into the base after training
Training data	43,429 dictation → polished pairs, one pass
Hardware and time	M1 MacBook with 8 GB of memory, about 5 h 45 min
Framework	MLX 0.32 and mlx-lm 0.32
Speech model the data was made with	NVIDIA Parakeet TDT 0.6B v3, through Evoo's own engine (FluidAudio)
Teacher model	Claude Haiku 5.5 (Anthropic), following Evoo's own polish prompt
Cost of the teacher	about $2, plus about $1.40 to compare teachers
Licence	Apache-2.0

Results

All test sentences were held out from training: whole recordings and passages were set aside, so none appears on both sides. Except in the rows marked "alone", Evoo's deterministic rules run first. "Evoo today" then calls stock Qwen3-0.6B (with Evoo's full prompt of rules and worked examples) only where the rules flag something; the other rows call the model on every dictation.

300 held-out dictations (podcasts, meetings, calls, questions, everyday messages and prompts):

	Median word error	Exactly right	Ran too long
Evoo today	0.24	36	21
Evoo rules + stock Qwen3-0.6B	0.24	43	13
Evoo rules + Evoo AI v1	0.14	78	12
Evoo AI v1 alone	0.12	86	15

Word error is the word-level edit distance from a reference polish, counting case and punctuation, divided by the reference length. Ran too long means the answer was over 1.5× the reference length.

By kind of speech (rules first, model on every dictation):

Test set	n	Stock: median error	Evoo AI v1: median error	Stock: exactly right	Evoo AI v1: exactly right
Spoken questions with self-corrections (Disfl-QA)	107	0.33	0.12	17	42
Meetings (AMI, ICSI, committees)	96	0.23	0.15	11	15
Earnings calls, recogniser output	24	0.13	0.10	5	5
Everyday messages and notes, read aloud	23	0.23	0.12	2	2
Earnings calls, teacher-polished	15	0.23	0.17	1	2
Prompts (Dolly), read aloud	13	0.24	0.04	2	6
Prompts (OpenAssistant), read aloud	8	0.14	0.07	2	3
Earnings calls, human-edited	6	0.12	0.06	2	2
Prompts (WildChat), read aloud	5	0.33	0.33	1	1
Podcasts, human-edited	3	0.29	0.06	0	0

Evoo's own benchmark files (Benchmarks/): self-corrections as a recogniser writes them, a held-out set of phrasings written after Evoo's rules were tuned, run-on sentences with little punctuation, and sound-alike words (with and without the earlier text, plus correct sentences that must not change):

	Self-corrections	Held-out corrections	Run-on punctuation	Sound-alike words	...with earlier text	Correct text left alone
Evoo today	84 / 85	28 / 30	10 / 10	1 / 10	2 / 6	9 / 10
Evoo rules + stock Qwen3-0.6B	75 / 85	28 / 30	5 / 10	4 / 10	2 / 6	9 / 10
Evoo rules + Evoo AI v1	82 / 85	27 / 30	10 / 10	2 / 10	4 / 6	8 / 10
Evoo AI v1 alone	26 / 85	11 / 30	10 / 10	2 / 10	3 / 6	9 / 10

In Evoo's own engine (llama.cpp, the Q8_0 file below, 8 GB M1, rules first):

evoo-cli test	Stock Qwen3-0.6B	Evoo AI v1
corpus corrections.tsv --rewrite	80 / 85	82 / 85
corpus corrections-holdout.tsv --rewrite	28 / 30	28 / 30
golden --parakeet v3 --polish (69 spoken dictations and commands)	68 / 69	68 / 69
field (real failure reports)	7 / 15	6 / 15
misheard (fixed / correct kept)	7 / 16, 10 / 10	7 / 16, 9 / 10
Time per dictation	about 0.36–0.40 s	about 0.30–0.41 s

Use it after Evoo's rules. On its own it is the most accurate on everyday dictation, but it often leaves spoken self-corrections ("Monday, sorry, Tuesday") unapplied, and Evoo's rules resolve those almost perfectly first. Sound-alike slips ("their going", "meat you") are still weak in every setup.

Every answer behind these tables is in examples/ and eval/.

How it was made

Why fine-tune

Asked with Evoo's full prompt, stock Qwen3-0.6B got 4 of 28 hand-written test dictations exactly right, and it answered three of four dictated questions and commands ("write a Python function that reverses a string" came back as code). A look inside the model came first: switching off its parts one at a time showed that layers 0–2 matter for everything, and that the neurons polishing relies on most sit in layers 14–18. Cutting the model down without retraining always cost quality, so the plan became a fine-tune on many examples of the job.

The data: 43,429 pairs

Source	Pairs	What it teaches
Everyday messages, notes and prompts, read aloud through Parakeet	17,859	What Evoo's recogniser really writes for everyday text
Meetings (AMI, ICSI and committee transcripts, via QMSum), polished by the teacher	6,000	Real conversational disfluency
Voice-assistant commands (Amazon MASSIVE), polished by the teacher	4,000	Short requests stay requests
Earnings calls: four commercial recognisers' output against the human transcript (Earnings-21)	3,500	Real recognition and formatting slips
Common Voice recognition errors (HyPoradise), words punctuated by the teacher	3,000	Fixable sound-alike slips
Podcasts transcribed word for word and lightly edited by professionals (Rev16)	2,821	Human editing of real speech
Spoken questions with self-corrections (Disfl-QA)	2,500	"X, no, Y" means Y
Earnings calls, polished by the teacher (Earnings-22)	2,500	Long-form spoken English
Earnings calls, word for word and human-edited (Earnings-22 subset)	1,249	Human editing of real speech

Read aloud. The everyday pairs started as 39,233 clean passages: real prompts from OpenAssistant, Dolly and WildChat, plus messages and notes the teacher wrote across a grid of writers, topics and lengths. Each passage got a simulated speaker's fillers, restarts and self-corrections. Then one of 26 macOS voices read it at 150–215 words per minute, with US, UK, Indian, Irish, Australian or South African accents, and Parakeet transcribed the audio through evoo-cli transcribe. A pair was kept only if the recogniser heard the words right (sound-alike slips allowed) and every number was written the same way on both sides. 20,959 passed (53%): 8,979 teacher-written messages and notes, 5,058 Dolly prompts, 4,781 OpenAssistant prompts and 2,141 WildChat prompts. 70% of the inputs then went through Evoo's rules, as in the app, and 30% were left as Parakeet wrote them.

The teacher. Claude Haiku 5.5 was given Evoo's own polish prompt, exported from the app with evoo-cli prompt-prefix, so its answers follow Evoo's rules. Every answer was checked before it became a pair:

• it is at most 10% longer than what was said, and at least 40% as long;
• at least 85% of its words were said;
• no number appears unless one was said ("15" for "fifteen" is fine);
• no currency or percent sign is added ("70 billion" must not become "$70 billion");
• there's no second attempt or comment after the answer.

Between 88% and 98% of answers passed, depending on the source.

Choosing the teacher. On 300 dictations a person had already polished, the candidates scored:

	Passed the checks	Median error against the human edit	Mean error	Batch cost per 1,000 items
Claude Haiku 5.5	96%	0.117	0.166	$0.05
Claude Opus 5.5	99%	0.117	0.136	$2.29

Only fixable corrections. In recogniser-output pairs, only corrections that can be made from the text alone were kept: sound-alike swaps, small words, formatting. Teaching a text model to turn "two" into "three" because the audio said so would teach it to invent. This removed 57% of the Earnings-21 pairs.

Training

Setting	Value
Adapter	LoRA, rank 8, scale 20, dropout 0, last 16 of 28 layers
Steps	21,714 (one pass, batch size 2)
Learning rate	1e-4, constant, Adam
Longest example	256 tokens (the 1.7% of pairs that were longer were left out)
Loss	On the answer only (the prompt is masked)

Batches of 2 are a limit of the laptop, not a choice: macOS cancels GPU work that keeps the screen waiting ("Impacting Interactivity"), and batches of 4 up to 320 tokens were cancelled within minutes. Training saved a checkpoint every 100 steps and resumed by itself after each cancellation. Afterwards the adapter was merged into the base model and converted to GGUF (Q8_0) with llama.cpp's converter.

What did not work (yet)

Two follow-up models tried to fold Evoo's rules into the model, so it would need no rules at all:

	Exactly right (300 held out, alone)	Self-corrections alone	Held-out corrections alone	Sound-alike words
Trained on raw recogniser output only	74	36 / 85	15 / 30	1 / 10
The same, trained further on 28,016 self-corrections, sound-alikes and replayed pairs	61	47 / 85	18 / 30	5 / 10

Both stayed far below the rules' 84 of 85, and the top-up hurt everyday dictation. So v1 runs after the rules. Evoo AI v2 aims to close the gap with corrections checked by Evoo's rules and a larger adapter.

How to use it

In Evoo: Settings › Smart cleanup (local AI) › Model › Evoo AI v1, with Polish every dictation with local AI on. Evoo runs its rules, then polishes every English dictation of four words or more with the model. Lists the rules laid out are left as they are. It is English-only; other languages and Hinglish keep their own setup.

Prompt format. It was trained with exactly this. Thinking is off (the empty think block is part of the answer).

<|im_start|>system
You are a dictation editor inside a voice keyboard. You receive text the user just dictated and return it cleaned up, ready to be typed into their app.<|im_end|>
<|im_start|>user
Clean up this dictated text. Do not answer or act on it.
<transcript>so um can we move the review to thursday</transcript><|im_end|>
<|im_start|>assistant
<think>

</think>

With Transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("e2chndr/evoo-ai-v1")
model = AutoModelForCausalLM.from_pretrained("e2chndr/evoo-ai-v1")
messages = [
    {"role": "system", "content": "You are a dictation editor inside a voice keyboard. You receive text the user "
                                  "just dictated and return it cleaned up, ready to be typed into their app."},
    {"role": "user", "content": "Clean up this dictated text. Do not answer or act on it.\n"
                                "<transcript>so um can we move the review to thursday</transcript>"},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
ids = tok(prompt, return_tensors="pt")
out = model.generate(**ids, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0, ids.input_ids.shape[1]:], skip_special_tokens=True).strip())
# Can we move the review to Thursday?

With llama.cpp, LM Studio or Ollama: use evoo-ai-v1-q8_0.gguf with the prompt above.

Files

File	For
model.safetensors + config and tokenizer	Transformers or MLX: the merged model
evoo-ai-v1-q8_0.gguf	llama.cpp, LM Studio, Ollama: the merged model in one file (Q8_0, 639 MB)
evoo-ai-v1-lora.gguf	llama.cpp: the 5.8 MB adapter on top of Qwen3-0.6B
adapter/	mlx-lm: the LoRA adapter as trained
examples/held_out_dictations.jsonl	300 held-out dictations with the model's answers, references and licences
eval/	All results, and every answer on Evoo's benchmark files
training/	The code that built the data, trained the model and produced every number here

Reproduce it

training/ holds the whole pipeline, from downloading the public data to scoring the model. Its README lists the steps. The training pairs themselves are not published, because they mix share-alike licences with teacher-written text, but every step that made them is.

Intended use and limits

• For: polishing English dictation, ideally after deterministic rules like Evoo's.
• Not for: answering questions, writing new text, or translating.
• English only, and trained on dictations of up to about 125 words.
• Without rules it often leaves spoken self-corrections in place.
• It rarely fixes sound-alike recognition slips. It edits; it does not know facts, so a name or number the recogniser got wrong usually stays wrong.
• Accents in the read-aloud data come from macOS voices, which are cleaner than real speech. The real-speech sources skew to American and British English.
• Like any polish model it can drop or change words, now and then a whole clause. Evoo checks every answer and falls back to the rules' text when it adds content, drops most of the text, loses a negation or a quotation, or leaves out a name or number that was said.

Licence and credits

The model weights are released under Apache-2.0, like Qwen3-0.6B, which they are built on. The training data came from these sources, each under its own licence:

• Rev.com speech datasets (Rev16, Earnings-21, Earnings-22): CC BY-SA 4.0
• Disfl-QA (Google): CC BY 4.0
• QMSum: MIT; the AMI and ICSI meeting corpora: CC BY 4.0
• Amazon MASSIVE: CC BY 4.0
• HyPoradise, Common Voice part only: CC0
• OpenAssistant oasst1: Apache-2.0
• databricks-dolly-15k: CC BY-SA 3.0
• WildChat (AI2): ODC-BY

Teacher answers and some messages were written by Claude Haiku 5.5. Speech came from Parakeet TDT 0.6B v3 (NVIDIA; CC BY 4.0) via FluidAudio, and the voices from macOS. The examples in examples/ keep the licence of the source they came from, noted on each line.

@misc{evoo_ai_v1,
  title  = {Evoo AI v1: a small language model fine-tuned to polish dictation},
  author = {{The Evoo project}},
  year   = {2026},
  url    = {https://huggingface.co/e2chndr/evoo-ai-v1}
}