whisper-heb-ipa

Fine-tuned Whisper model that transcribes Hebrew speech into IPA phonemes.

• Architecture: Whisper large-v3-turbo
• Base model: ivrit-ai/whisper-large-v3-turbo
• Pipeline: IPA pretrain → fine-tune (whisper_ipa_finetune/best)
• Training precision: bf16

Benchmark results

Scored with normalize_ipa(strip_punctuation=True) (ASCII model output → Unicode IPA).

ILSpeech full (data/ilspeech-v2/metadata_full.csv, 1530 samples)

Metric	Value
CER	1.54%
WER	7.67%
SER	5.91%
VER	1.47%
Exact match	743 / 1530 (48.6%)

Michael Gold v1 (data/michael-gold-v1, 561 samples)

Metric	Value
CER	3.46%
WER	16.55%
SER	12.26%
VER	2.42%
Exact match	65 / 561 (11.6%)

Full per-sample reports are in benchmarks/.

Usage

from transformers import pipeline

pipe = pipeline(
    "automatic-speech-recognition",
    model="notmax123/whisper-heb-ipa",
    generate_kwargs={"language": "he", "task": "transcribe"},
)
print(pipe("audio.wav")["text"])

CLI from this repo:

uv run src/infer.py audio.wav --model notmax123/whisper-heb-ipa

Training data (fine-tune stage)

• data/large-ipa-dataset (segments.tsv)
• data/imaginary-jail-clean-v2 train split
• Eval holdout: imaginary-jail 30m (metadata_eval_30m.csv)