Aayush9029/s1-mini-mlx-8bit

S1-mini by Superwhisper (MLX 8-bit)

This is an MLX 8-bit build of S1-mini by Superwhisper, for Apple silicon. S1-mini is a 0.6B text normalizer for speech-to-text output. Petal uses this build for on-device transcript cleanup.

Changes from the original

• The BF16 weights were quantized with mlx_lm.convert -q --q-bits 8 --q-group-size 64 (affine).
• The model, the tokenizer, the chat template, and the prompt format are otherwise unchanged.

Usage

Use the exact system prompt and control line from the original model card, and set enable_thinking=False.

from mlx_lm import load, generate

model, tok = load("Aayush9029/s1-mini-mlx-8bit")
system = (
    "You are a text normalizer for speech-to-text transcripts. The input begins "
    "with a control line specifying the styling, structure, and context settings; "
    "clean the transcript to match those settings and output only the cleaned text."
)
user = "[Styling: semi-formal] [Structure: prose] [Context: general]\nso um send the report by uh friday"
prompt = tok.apply_chat_template(
    [{"role": "system", "content": system}, {"role": "user", "content": user}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False,
)
print(generate(model, tok, prompt, max_tokens=128))

Performance

Measured on an M4 Pro with greedy decoding:

Build	Output identical to BF16 (16 cases)	Prefill	Decode	Peak memory
BF16	16/16	~2,600 tok/s	—	1.4 GB
8-bit (this repo)	15/16 (one comma)	~5,600 tok/s	~240 tok/s	0.85 GB
4-bit	10/16	~5,600 tok/s	~340 tok/s	0.55 GB

The 4-bit build dropped words in some cases, so this repo ships 8-bit only.

License

S1-mini is released under Apache 2.0 plus a naming term: it must keep its name, "S1-mini" by "Superwhisper". See LICENSE and NOTICE.