aitestinguserpg/whisper-large-v3-turbo-baps-coreml

Atlas BAPS Whisper (Core ML)

whisper-large-v3-turbo with its decoder tuned for English speech with Gujarati words and names in it, as heard in BAPS Swaminarayan satsang talks. It writes the Gujarati words in Latin letters, in the spelling the Atlas app uses (sakhi samajanma ghanu sukh che), rather than in Gujarati script, and keeps names and terms (Bhagwan Swaminarayan, Mahant Swami Maharaj, Vachanamrut, satsang) intact.

This is the Core ML build Atlas downloads and runs on the Mac, through WhisperKit. Nothing about it needs a server: the audio never leaves the device.

Files

whisper-large-v3-turbo-baps/
  AudioEncoder.mlmodelc/   MelSpectrogram.mlmodelc/   TextDecoder.mlmodelc/
  tokenizer.json, config.json, generation_config.json, …

About 1.5 GB in 16-bit precision. Apple silicon only in practice (the first load compiles the model for the Neural Engine, which takes a few minutes once; later loads take about a second).

How it was made

• Base: openai/whisper-large-v3-turbo (MIT). The audio encoder is unchanged.
• Tuning: only the 4-layer text decoder was trained, at a low learning rate, on 558 clips (about 4 hours) from 6 satsang talks, to write each clip's transcript in Atlas's Latin spelling. Transcripts were produced by Gemini from the audio and then converted to that spelling; they have not been checked by a person. The recordings are not included.
• Conversion: to Core ML with whisperkittools; the decoder weights were mapped onto the original checkpoint's and verified tensor by tensor.
• Prompting: Atlas uses it as it is: English, no timestamps, no prompt.

How well it works

Held-out test: 207 clips from 14 talks the model never saw. Word error rate against the Gemini transcripts, with "um/uh" ignored and number words read as digits:

	All	English only	A few Gujarati words	Mixed	Mostly Gujarati
Parakeet TDT 0.6B (English) + repair	14.0%	6.4%	10.9%	26.6%	48.1%
This model, in Atlas	7.3%	3.9%	6.3%	10.8%	30.4%

Treat these as indicative. The reference transcripts are machine-made and unchecked, so some of the remaining "errors" are the references' own; there were only 9 mostly-Gujarati test clips; and the talks are clean recordings, not microphone dictation.

Limits

• It writes Latin letters, not Gujarati script, by design.
• Over music (the opening chants of a talk), or past what it can follow, it can loop on a word, drop words or write a sentence in another script. Atlas checks for this and has Parakeet read the clip instead (2 of 207 test clips).
• English first: it is not a general Gujarati recogniser, and a long stretch of Gujarati read aloud is its weakest case.
• Spelling of Gujarati words varies in romanised text; this model follows one convention.

Licence and credit

MIT, as the base model. Built on OpenAI's Whisper, with Core ML conversion tooling from Argmax (whisperkittools, WhisperKit).