per2021/whisper-large-v3-ca-punctuated-mlx

whisper-large-v3-ca-punctuated-3370h, converted to MLX

BSC-LT/whisper-large-v3-ca-punctuated-3370h converted to the MLX format so it runs on the GPU of Apple silicon through mlx-whisper. The weights are the same model, in float16; nothing was retrained or pruned.

Its transcripts come out with question marks, commas and casing, rather than normalised. The original model card describes the training data as punctuated.

Use

pip install mlx-whisper

import mlx_whisper

result = mlx_whisper.transcribe(
    "audio.wav",
    path_or_hf_repo="per2021/whisper-large-v3-ca-punctuated-mlx",
    language="ca",
)
print(result["text"])

Why it exists

There was no MLX build of this model, so anyone on a Mac had to download the PyTorch weights and convert them by hand — fiddlier here than usual, because it ships as sharded checkpoints that Apple's converter cannot read directly.

For accuracy, see the evaluation in the original model card, measured on Catalan test sets.

Being a full large-v3, it is slower than whisper-large-v3-turbo. Whisper always processes a 30-second window, so the cost per utterance does not depend on how long you speak.

Conversion

Apple's converter reads a single model.safetensors, while this model publishes pytorch_model-0000N-of-00002.bin shards, so they have to be merged first. Both steps are scripted in voice-agent/scripts — the gist:

# 1. merge the shards into one safetensors file (float16 keeps it at ~3 GB)
# 2. run Apple's converter
curl -sL -o convert.py \
  https://raw.githubusercontent.com/ml-explore/mlx-examples/main/whisper/convert.py
python convert.py --torch-name-or-path <merged-dir> --mlx-path out --dtype float16
# 3. mlx-whisper looks for weights.safetensors, the converter writes model.safetensors
mv out/model.safetensors out/weights.safetensors

Credits and licence

• Fine-tune: Barcelona Supercomputing Center.
• Base model: openai/whisper-large-v3.
• Licence: Apache 2.0, inherited from the original. This derivative only changes the weight format.

Related

• per2021/whisper-bsc-large-v3-cat-mlx: a sibling fine-tune whose transcripts come out normalised, also converted to MLX.