xeus-yi-ear — a Hasidic Yiddish phone recognizer

PhoneticXeus (575 M, CTC over 428 universal IPA symbols) fine-tuned into a 35-way CTC head over the closed Hasidic Yiddish inventory of the phonikud-yi engine: 11 vowels (a aː ɛ ə i u ɔ ej aj ɔj oʊ), 23 consonants (b d f ɡ h j k l m n p r s t v z x ʃ ʒ ʦ ʧ ʤ ŋ), no stress. Demo: Yiddish Ear.

Files

file	what
ckpt/inner.safetensors	the fine-tuned XEUS encoder (frontend + blocks 0–5 frozen, the rest tuned)
ckpt/yi_head.pt	the Yiddish CTC head, warm-started from the pretrained rows of each phone's symbols
ckpt/meta.json, ckpt/eval.md	validation numbers of this checkpoint
dictionary.json	the 412 words whose readings a native Hasidic reader verified, with every accepted variant
menu.json	per-word heard-reading counts from the lattice evaluation

Load with xeus_yi_decode.load_finetuned(ckpt_dir) from the Phonikud-yi scripts (or the Space's copy): it builds the pretrained model with trust_remote_code, loads the encoder state and the head. Needs transformers==4.56.2, huggingface_hub<1.0, typeguard, pyyaml.

Training

Run 2 of Phonikud-yi/docs/xeus_finetune.md: 79k clips (21 h) cut from one speaker's episodes by forced alignment, containing only words whose readings were verified (nothing the G2P guessed), gold variant per clip chosen by CTC likelihood, speed/gain/noise augmentation, 6 epochs on a 3090, ≈ $1.

Results (held out, paired on identical clips)

split	pretrained + fold map	this
unseen word types (3,600 clips), PER	0.547	0.336
unseen episodes (3,600), PER	0.548	0.272 (exact 4% → 35%)
unseen episodes at 15 dB SNR	0.723	0.301
85 unseen speakers (crowd recordings), PER	—	0.242
ʦ / aj / ɔj / oʊ recall	0.10 / 0 / 0 / 0	0.67 / 0.79 / 0.64 / 0.39

The pretrained model cannot output a diphthong at all and hears ʦ as s. Known weaknesses: word-final ə is often dropped, oʊ vs ɔj is discriminated at about a coin flip on unseen words (the phone is 0.04% of training frames). Every continued-training attempt so far (schwa oversampling, blank penalty, aux frame loss, attested-label curriculum, extra speakers) improved something and lost oʊ — this checkpoint stays the reference ear.

Licence

Weights CC BY 4.0. The training audio is one host's public podcast; the model is for speech research and for verifying pronunciations in Yiddish text-to-speech work, not for voice cloning.