iky1e/punctuation-fullstop-truecase-english-mlx-q8

English punctuation, capitalization, and segmentation MLX — Q8

This repository contains 8-bit affine weight quantization with group size 64 and FP16 residual tensors for Apple-silicon inference with MLX.

It is converted from 1-800-BAD-CODE/punctuation_fullstop_truecase_english at revision b26fd1c40e88678859048898218ea4edcc24c84a. The original model restores punctuation and capitalization and predicts sentence boundaries for lowercase English text. Its source ONNX SHA-256 is dd922d459da618cd324280889740608b76fb3e9e61d3f402291be1251f91421b.

MLX variant comparison

Variant	Repository	Weight file	Size vs ONNX FP32	Character agreement	Word agreement	Formatter inference
ONNX FP32 source	1-800-BAD-CODE/punctuation_fullstop_truecase_english	209.5 MB	100.0%	100.0000%	100.0000%	2.044s
FP16	iky1e/punctuation-fullstop-truecase-english-mlx-fp16	104.7 MB	50.0%	99.9916%	99.9559%	0.417s
Q8 (this repository)	iky1e/punctuation-fullstop-truecase-english-mlx-q8	55.8 MB	26.6%	99.9536%	99.7575%	0.260s
Q6	iky1e/punctuation-fullstop-truecase-english-mlx-q6	42.8 MB	20.4%	99.8777%	99.3681%	0.346s
Q5	iky1e/punctuation-fullstop-truecase-english-mlx-q5	36.2 MB	17.3%	99.6066%	97.9941%	0.350s
Q4	iky1e/punctuation-fullstop-truecase-english-mlx-q4	29.7 MB	14.2%	99.2899%	96.4879%	0.330s

Agreement is measured against the original ONNX FP32 model's formatted output, not against a human transcript. Character and word agreement are 100 − normalized Levenshtein distance. The input was Granite Q8's raw transcript of a 6,118.72-second (101m58.72s) Stanford CME295 lecture: 69,168 input characters and 71,116 ONNX-formatted characters. Formatter time excludes process startup and model loading. Peak RSS from the Python harness includes Python and framework overhead.

Q8 is the recommended default: it is substantially smaller than FP16 while remaining very close to ONNX FP32 and was the fastest measured MLX variant on this machine.

Granite-MLX usage

Formatted output is intended to be the Granite-MLX default:

granite-mlx recording.mp4 --punctuation-model iky1e/punctuation-fullstop-truecase-english-mlx-q8

Use --no-punctuate when exact raw Granite CTC text or minimum memory usage is required.

Files

• model.safetensors: MLX weights
• mlx_config.json: architecture, precision, quantization, and source metadata
• tokenizer.json and tokenizer_config.json: native-compatible SentencePiece Unigram tokenizer
• spe_32k_lc_en.model: original SentencePiece model for parity/reference runtimes
• config.yaml: source labels and sequence configuration

Reproducing conversion

uv run python Scripts/convert_punctuation.py \
  /path/to/punctuation_fullstop_truecase_english \
  /path/to/output-q8 \
  --bits 8

The converter and complete benchmark artifacts are maintained in the Granite-MLX project.

License

These converted model weights retain the original model's Apache 2.0 license. Granite-MLX is separate software that downloads and runs a user-selected checkpoint.