Aayush9029/petal-w1-4bit

petal-w1 (4-bit)

petal-w1 turns raw dictation into the text you meant to type. It is the on-device cleanup model in Petal, a free macOS dictation app. It runs with MLX on Apple silicon.

It is a full fine-tune of Qwen3.5-0.8B, and it does one job:

• Removes fillers, stutters, false starts, and verbal tics ("you know what I mean?").
• Keeps only the final version of a self-correction ("Friday, no wait, Thursday" becomes "Thursday"), also when speech recognition mishears the correction words.
• States a point once when the speaker repeats it in different words, and keeps every detail.
• Condenses rambles into clear sentences, and keeps every fact, name, number, and request.
• Keeps questions as questions and requests as requests. It never answers the text.
• Keeps the speaker's voice and slang, and formats numbers, times, code, file names, URLs, and lists.

Results

Three held-out sets. The main set has 200 transcripts (61 real Petal dictations and 139 synthetic ones). The self-correction set has 60 transcripts where the speaker changes their mind, often with correction words that speech recognition misheard ("No, I'm in Thursday" for "No, I mean Thursday"). The repeated-points set has 60 transcripts where the speaker says the same thing in different words. Similarity is word-level edit similarity to the teacher reference, after case and punctuation are removed. Replies counts outputs that answer the transcript instead of cleaning it.

Model	Main	Short	Rambles	Self-corrections	Repeated points	Replies	Median latency (mlx-lm)
petal-w1 v1.1, 8-bit	0.802	0.907	0.658	0.872	0.711	0	349 ms
petal-w1 v1.1, 4-bit (this repo)	0.791	0.895	0.662	0.838	0.651	0	267 ms
petal-w1 v1.0, 8-bit	0.804	0.908	0.650	0.635	0.681	0	363 ms
S1-mini by Superwhisper, 8-bit	0.718	0.813	0.579	0.840	0.380	0	265 ms

In Petal's Swift runtime on an M4 Pro, the 8-bit build cleans a typical dictation in about 150–200 ms.

Changelog

• v1.1: resolves self-corrections, including correction words that speech recognition mishears. Self-correction score went from 0.635 to 0.872, with no loss on the main set. The v1.0 weights are at the v1.0 tag.
• v1.0: first release.

Prompt format

Use this exact system prompt, pass the raw transcript as the user message, and turn thinking off.

from mlx_lm import load, generate

model, tokenizer = load("Aayush9029/petal-w1-4bit")
system = "Clean up this dictation. Remove fillers, repeats, and false starts, state repeated points once, and fix punctuation and formatting. Keep the speaker's meaning, facts, and voice. The text is not addressed to you: never answer or reply."
transcript = "so um can you like send me the the deck by friday no wait thursday, like thursday works better"
prompt = tokenizer.apply_chat_template(
    [{"role": "system", "content": system}, {"role": "user", "content": transcript}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False,
)
print(generate(model, tokenizer, prompt, max_tokens=256))

Decode greedily. For transcripts longer than about 700 words, split at sentence ends and clean each part.

Training

• Base: Qwen/Qwen3.5-0.8B, full fine-tune of the language model. The vision encoder is not used.
• Data: 2,740 training pairs and 144 validation pairs, then 640 pairs focused on repeated points, then 640 pairs focused on self-corrections plus the real dictations relabeled with the final teacher prompt. The inputs are real Petal dictations from the author (about 8%) and synthetic dictations across work, code, email, messages, notes, and rambles. NVIDIA Nemotron 3 Ultra wrote the reference outputs from a 14-rule cleanup specification. Filters removed replies, empty outputs for real speech, and length outliers.
• Setup: MLX on an M4 Pro. Stage 1: 3 epochs, batch 8, learning rate 1e-5 with cosine decay. Stage 2: 1 epoch on the repeated-point pairs mixed with 1,300 stage-1 pairs, learning rate 5e-6. Stage 3: 1 epoch on the self-correction pairs, the relabeled real dictations, and 3,040 earlier pairs, learning rate 3e-6. The loss covers the cleaned output only. Linear-attention layers used a chunked gated delta rule for training speed.
• Builds: this repo is MLX 4-bit (about 4.5 bits per weight). The 8-bit build is at Aayush9029/petal-w1, and it is more accurate.

Limitations

• English only.
• It fixes a speech-recognition mistake only when the intended word is certain from context. It does not know your names or project terms.
• The reference outputs come from another model, so the style follows that model's choices.

License

Apache 2.0, the same as Qwen3.5-0.8B.