You need to agree to share your Hugging Face information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
Access to this model is by request and is governed by the Hebrish terms of use (https://hebrish.app/terms). Non-commercial use is free; commercial use needs a commercial licence from access-requests@hebrish.app. Please also complete the attestation form: https://hebrish.app/legal/hebrish-terms-of-use.pdf
Log in or Sign Up to review the conditions and access this model content.
Jachnun: English-Hebrew code-switching speech recognition
Jachnun is the Hebrish model: a fine-tune of Parakeet TDT 0.6B v2 for English with Hebrew words in it,
named after the Yemenite pastry. This repository holds Jachnun v1.0 (run r6-en30, 2026-10-19), the
first stable release; later retrains will be v1.1, v1.2, and so on.
The proof-of-concept checkpoints (PoC-1 to PoC-5) and the first copy of v1.0 were published at
danielrosehill/Parakeet-TDT-0.6B-v2-English-Hebrew-Codeswitching
(DOI 10.57967/hf/10995); cite that DOI for the proof of concept. Revision
hashes in the version history below refer to that repository. GGUF, ONNX, sherpa-onnx and MLX builds of
Jachnun v1.0 are in the Hebrish organisation.
A fine-tune of nvidia/parakeet-tdt-0.6b-v2
for English speech with Hebrew words dropped in, the way English speakers in Israel
talk: "pick up bamba from the makolet", "I have an appointment at Misrad Hapnim
tomorrow". The Hebrew words are written in a single romanized spelling (the v2
tokenizer has no Hebrew characters), and the output keeps v2's casing and punctuation.
Website: hebrish.app: get the Hebrish keyboard for Android and the desktop apps, see the results, and find every model, dataset and demo in one place. Android updates through F-Droid: add the Hebrish F-Droid repository (https://fdroid.hebrish.app/repo; steps).
Access and terms of use. Use of this model and every converted or quantised build of it (GGUF, ONNX, sherpa-onnx, MLX) is governed by the Hebrish terms of use. Access is by request: use the button on this page, or write to access-requests@hebrish.app. Non-commercial use (personal, research, education, open-source projects) is free; please complete the attestation form. Commercial use, including telephony and call centres, needs a commercial licence: access-requests@hebrish.app. Versions released under CC-BY-4.0 (PoC-1 to PoC-5, v1.0) stay under that licence; the terms do not restrict rights it grants.
A real result: the speaker recorded this in the demo Space (2026-10-15):
Transcript Stock Parakeet v2 I want to go to the Micollette today to buy my groceries. This fine-tune I want to go to the makolet today to buy my groceries. Makolet (מכולת) is the Israeli corner grocery shop. Stock v2 has never heard the word, so it guesses an English-looking name; the fine-tune writes the word itself.
Which file should I download?
All links below are the latest version, Jachnun v1.0 (2026-10-19); they always point to the newest release. Earlier versions, with permanent links, are under Version history.
| You want to… | Download | Run it with |
|---|---|---|
| Dictate on an Android phone | Q8_0 GGUF (730 MB) | Offline Voice Input |
| Dictate on an Android phone with little memory (≤ 4 GB RAM), or if Q8_0 is too slow | Q4_K_M GGUF (475 MB) | Offline Voice Input |
| Dictate on a Linux desktop | Q8_0 GGUF (730 MB) | Handy |
| Dictate on Android, Ubuntu or Windows with the project's own app, Hebrish | Nothing: the app downloads this model and keeps it updated | F-Droid repo · Android APK · Ubuntu .deb · Windows .zip |
| Transcribe in Python, on a GPU, or fine-tune further | .nemo (2.47 GB, full precision) |
NVIDIA NeMo |
If unsure, take Q8_0. It is the closest to full accuracy that still runs on a phone, and on desktop CPUs it was also faster than Q4_K_M: about 3× realtime against 1.4× on 8 cores. On five held-out test clips, Q8_0 and Q4_K_M both matched the full-precision model word for word, except that Q4_K_M lowercased one name (v1.0, 2026-10-19: Q8_0 5/5, Q4_K_M 4/5; v1.0 was published before its benchmark, so there were no PyTorch transcripts to compare with, and the full-precision ONNX export stood in for them). Speed on phones has not been measured yet.
Other formats (separate model repos)
| Repo | Format | For |
|---|---|---|
| …-GGUF | transcribe.cpp GGUF: Q8_0, Q4_K_M, F16 | Offline Voice Input (Android), Handy, transcribe-cli; F16 for GPU |
| …-ONNX | onnx-asr layout, fp32 + int8 | Voxtype push-to-talk on Linux, onnx-asr |
| …-sherpa-onnx-int8 | sherpa-onnx transducer, int8 | Hebrish (Android, Ubuntu, Windows), other sherpa-onnx apps |
| …-mlx | MLX, fp32 safetensors | Macs with Apple Silicon: parakeet-mlx, mlx-audio |
The same files also stay in this repo under gguf/, onnx/ and sherpa-onnx/, so existing
links keep working.
Files
| File | Use |
|---|---|
Parakeet-TDT-0.6B-v2-English-Hebrew-Codeswitching.nemo (2.47 GB) |
Original NeMo checkpoint: NeMo / PyTorch, further fine-tuning |
gguf/…-Q8_0.gguf (730 MB) |
transcribe.cpp GGUF for on-device use. Recommended for phones. |
gguf/…-Q4_K_M.gguf (475 MB) |
Smaller GGUF, for phones short on memory |
gguf/…-F16.gguf (1.2 GB) |
GGUF for GPU inference (transcribe.cpp with Vulkan, HIP or CUDA) |
onnx/fp32/, onnx/int8/ |
onnx-asr / Voxtype layout; see the -ONNX repo |
sherpa-onnx/int8/ |
sherpa-onnx transducer; see the -sherpa-onnx-int8 repo. The two apps below download from this exact path and check it for updates (with version.json), so keep it here |
The GGUFs were converted with transcribe.cpp v0.1.3, the version the Android app below links. On five held-out test clips, Q8_0 gave the same transcripts as the full-precision ONNX export on 5/5 and Q4_K_M on 4/5 (v1.0, 2026-10-19; v1.0 was published before its benchmark, so there were no PyTorch transcripts to compare with, and the full-precision ONNX export stood in for them).
On Android:
- Install Offline Voice Input (source).
- Import the Q8_0 GGUF and select it.
- Use it as the system voice input, or through a keyboard whose mic key opens it (SwiftKey, HeliBoard, FlorisBoard).
On Linux: Handy loads the same GGUF from
~/.local/share/com.pais.handy/models/.
Own app: Hebrish (2026-10-19; first released as Parlance). One repo for the Android and desktop apps, with a choice of model: stock Parakeet TDT 0.6B v2 or a fine-tune, including this one (it replaces the earlier Voice To Hebrish and Parakeet-Type-Ubuntu apps). Both run the sherpa-onnx int8 build on-device. They download it on first run and offer newer versions when the weights here change, so a new fine-tune needs no new app.
- Android: a voice keyboard that works as the companion voice input for HeliBoard
(Android 9+, arm64). Download the APK (v0.2.2), or get updates automatically
through the Hebrish F-Droid repository: on the phone, open this link with
F-Droid, or add
https://fdroid.hebrish.app/repounder F-Droid's Settings → Repositories with fingerprintEF8C2D63D994A10CD489A978585510B47E47452EC09248591CB2C0F39C2BC1FA. Same APKs and signing key as the direct download. Steps: hebrish.app/download#fdroid. - Ubuntu / Debian: a tray app. Dictate with any hotkey or foot pedal (tap to toggle, or hold to talk) and it pastes at the cursor; the Transcribe tab records or opens audio files for long-form transcripts. Download the .deb (v0.2.2, Ubuntu 26.04 amd64).
- Windows: the same desktop app, portable. Download the .zip (v0.2.2). Not yet tested on real hardware.
All releases: https://github.com/Hebrish-ASR/Hebrish-Dictation/releases.
Jachnun v1.0 is the first release the project stands behind. The five checkpoints before it (PoC-1 to PoC-5, published 2026-10-15 to 2026-10-18 as "v1" to "v5") were the proof of concept: each added recordings, and word error rate moved the right way at every benchmarked step. v1.0 is the sixth checkpoint, trained on almost six hours of code-switched speech. Their pinned files are under Version history.
It is still a single-speaker model. It was trained on one speaker (an English speaker living in Jerusalem) reading prompts on an Android phone, and will help most for that voice.
Try it and tell us how it does on your voice. The next step is real people using it: English speakers in Israel who dictate and are tired of every model mangling makolet, misrad hapnim and tachles. Install one of the apps below or try the demo Space, then send what worked and what did not to https://hebrish.app or info@hebrish.app.
Project tracker: https://github.com/Hebrish-ASR/English-Hebrew-Codeswitching-Parakeet-Finetune
Results (benchmarked 2026-10-19)
Jachnun v1.0 (run r6-en30) was scored on 2026-10-19 in two ways: on the public benchmark,
and on the held-out split of its own training dataset. One speaker's voice: all training
and test audio is the same speaker, so these numbers rank models on that speaker only. Gains
on other voices are untested.
Public benchmark. Hebrish/English-Hebrew-Codeswitching-ASR-Benchmark,
test set v1, clean subset: 84 clips, 172 term occurrences, 139 distinct terms. Round 3 on the
leaderboard Space,
scored 2026-10-19.
| Model | Term recall | Unseen-term recall | WER | English control WER |
|---|---|---|---|---|
| Jachnun v1.0 (this model) | 73.3% | 70.1% | 3.3% | 2.2% |
| PoC-3 (published as v3) | 67.4% | 63.2% | 4.0% | 2.5% |
| PoC-2 (published as v2) | 65.7% | 60.7% | 4.2% | 2.4% |
| PoC-1 (published as v1) | 62.2% | 61.5% | 4.6% | 2.5% |
| Nemotron streaming, checkpoint 2 (scored offline, not streaming) | 62.2% | 54.7% | 4.7% | 2.5% |
| Whisper Hebrish v1 | 54.1% | 56.4% | 6.5% | 4.7% |
| Whisper Large V3 | 45.9% | 46.2% | 7.5% | 2.4% |
| Whisper Large V3 Turbo | 40.1% | 39.3% | 8.9% | 2.4% |
| Parakeet TDT 0.6B v2 (stock) | 32.6% | 32.5% | 9.5% | 2.7% |
| Parakeet TDT 0.6B v3 (stock) | 31.4% | 31.6% | 10.5% | 2.7% |
A 2 to 3 point recall difference is about 4 to 5 hits, within noise. Contamination check: none
of the benchmark's 145 clips (by id or by text) is in the train.jsonl of v1.0 or of the
Nemotron checkpoint.
Held-out split. Read-Aloud-Voice-Sentences
at revision 550bfcaa0f84: 387 code-switched test clips, with 293 occurrences of 118 held-out
terms that are never in training and 278 occurrences of test terms that are also in training;
73 clips of the speaker's own English; 200 LibriSpeech test-clean utterances; 15 long-form files.
Normalised WER.
| Test set | Stock v2 | PoC-3 | Jachnun v1.0 | Nemotron checkpoint 2 (offline) |
|---|---|---|---|---|
| Code-switching WER (387 clips) | 11.3% | 6.0% | 5.1% | 5.8% |
| Held-out terms correct | 25.9% | 45.7% | 47.8% | 44.4% |
| Seen test terms correct | 27.7% | 63.3% | 72.7% | 68.0% |
| Own English WER (73 clips) | 3.1% | 2.3% | 1.7% | 3.3% |
| LibriSpeech test-clean WER (200) | 1.2% | 1.0% | 1.2% | 2.0% |
| Long-form WER (15 files) | 9.6% | 5.8% | 5.0% | 5.8% |
- v1.0's raw (cased, punctuated) code-switching WER is 7.22%.
- The held-out gain over PoC-3 (+2.1 points, about 6 of 293) is within noise; the seen-term gain (+9.4 points) is not.
- LibriSpeech unrounded: 1.20% for v1.0 against 1.23% for stock v2, within noise. English did not get worse.
- Nemotron drops all 8 number tokens. It has no forgetting verdict, because stock Nemotron was not re-scored on this revision.
- Known caveat: two terms on the held-out list (mandatim, Beit Safafa) may have leaked into training; this is being checked.
How v1.0 was trained. Run r6-en30: full fine-tune from stock
nvidia/parakeet-tdt-0.6b-v2 on 339.2 min of code-switched speech, 95.3 min of the speaker's
own English and 48.5 min of LibriSpeech replay (29.8% English), 20 epochs, best epoch 18,
validation WER 0.0497, 41.0 min on one L40S. The benchmark round was six Modal L40S jobs of
about 2.5 min wall time each: under 15 GPU-minutes, about US$0.50 at most.
More: hebrish.app/benchmarks · hebrish.app/datasets · white paper (PDF).
Earlier results: PoC-3, run r3-en30, on test revision b47310621f79
Everything in this section is PoC-3 (published as v3), on the older test revision
b47310621f79, and is not comparable with the550bfcaa0f84numbers above. PoC-4 and PoC-5 have not been benchmarked.
Both models were evaluated with the same NeMo harness (bf16 GPU, greedy decoding) on dataset
revision b47310621f79. WER is shown normalised (lowercase, no punctuation); raw
cased-and-punctuated WER is in evals/*.json.
| Test set | Stock v2 | PoC-3 |
|---|---|---|
| Code-switching, held-out test split (219 clips) | 10.9% | 5.3% |
| Hebrew terms never seen in training: share transcribed correctly | 27% | 48% |
| Hebrew terms seen in training, in test sentences: share correct | 29% | 66% |
| Speaker's own plain English (33 clips) | 3.5% | 2.9% |
| LibriSpeech test-clean, 200 utterances | 1.2% | 1.0% |
| Long-form, about 3-minute files, 9 files | 9.0% | 5.0% |
- The test split is term-disjoint: 242 Hebrew terms are held out of training entirely, and the "never seen" row scores only those terms.
- A term counts as correct when it appears in the hypothesis on a word boundary, case-insensitively; hyphen and space are treated as equal.
- The test split grew from 121 to 219 clips for PoC-3, so these numbers are not comparable with the ones published for PoC-2. The version history below re-scores every version on the new split.
Version history
Every version is re-scored on the same test split as the newest benchmark (revision
b47310621f79, the PoC-3 benchmark), so the rows below compare like with like. PoC-4, PoC-5
and v1.0 have not been scored on this split. v1.0, PoC-3 and stock v2 were scored on the v1.0
test split (387 clips) on 2026-10-19: see Results. PoC-1 to PoC-5 were published as v1 to v5. The files at the top of this
page are always the latest version; the links in the last row stay pinned to each
version's files forever.
| Stock v2 | PoC-1 | PoC-2 | PoC-3 | PoC-4 | PoC-5 | Jachnun v1.0 (latest) | |
|---|---|---|---|---|---|---|---|
| Released | 2026-10-15 | 2026-10-15 | 2026-10-18 | 2026-10-18 | 2026-10-18 | 2026-10-19 | |
| Code-switching training audio | 77 min | 120 min | 196 min | 269 min | 294 min | 339 min | |
| Code-switching WER | 10.9% | 6.0% | 5.8% | 5.3% | pending | pending | pending |
| … phone recordings | 10.7% | 6.0% | 5.6% | 5.1% | pending | pending | pending |
| … desktop-mic recordings (30 clips) | 13.8% | 7.1% | 8.6% | 8.6% | pending | pending | pending |
| Hebrew terms seen in training, correct | 29% | 57% | 60% | 66% | pending | pending | pending |
| Hebrew terms never seen in training, correct | 27% | 47% | 47% | 48% | pending | pending | pending |
| Speaker's plain English WER | 3.5% | 3.2% | 2.6% | 2.9% | pending | pending | pending |
| LibriSpeech test-clean WER | 1.2% | 1.4% | 1.1% | 1.0% | pending | pending | pending |
| Long-form WER | 9.0% | 5.9% | 5.6% | 5.0% | pending | pending | pending |
| Pinned files | Q8_0 · Q4_K_M · .nemo | Q8_0 · Q4_K_M · .nemo | Q8_0 · Q4_K_M · .nemo | Q8_0 · Q4_K_M · .nemo | Q8_0 · Q4_K_M · .nemo | Q8_0 · Q4_K_M · .nemo |
What v1.0 changed: 45 more minutes of code-switched audio (294 → 339 min, 3,224 clips) and 13 more minutes of the speaker's own English (83 → 95 min). Same recipe as PoC-5: one run, full fine-tune from stock v2, 30% English. Trained below the next data checkpoint by choice. It is named v1.0 because it closes the proof of concept, not because of a recipe change. Test-set results: see Results (benchmarked 2026-10-19).
What PoC-5 (v5) changed: 25 more minutes of code-switched audio (269 → 294 min, 2,829 clips) and 10 more minutes of the speaker's own English (73 → 83 min). Same recipe as PoC-4: one run, full fine-tune from stock v2, 30% English. This is a smaller step than the usual 1.5×, released by choice, so expect a small change. Test-set results are pending (see Results).
What PoC-4 (v4) changed: 73 more minutes of code-switched audio (196 → 269 min, 2,622 clips) and nearly twice the speaker's own English (43 → 73 min), which brought the training mix to its 70 / 20 / 10 target for the first time. Same recipe as PoC-3: one run, full fine-tune from stock v2, 30% English. Test-set results are pending (see Results).
What PoC-3 (v3) changed: 76 more minutes of code-switched audio (120 → 196) lowered
code-switching WER from 5.8% to 5.3% and long-form WER from 5.6% to 5.0%, and Hebrew words
with recordings went from 60% to 66% correct. Words the model has never heard barely moved
(47% → 48%, one hit out of 176): as with PoC-2, more data improves the words it has takes of,
not generalisation to new ones. Plain English moved 2.6% → 2.9% on 33 clips (about two
words), which is within noise, and stays below stock v2. Full per-utterance results:
evals/r3-*.json.
Training data
Everything v1.0 was trained on, by dataset. The training job pins one revision of
Read-Aloud-Voice-Sentences (550bfcaa0f84) and adds public English replay at train time.
| Dataset | Whose | What v1.0 used from it | Licence |
|---|---|---|---|
Read-Aloud-Voice-Sentences |
Mine | The training source. 3,224 code-switching clips (339.2 min): 2,785 takes from the capture app, mostly on the phone (312.4 min), and 439 recordings imported from the desktop-mic round below (26.8 min). Plus 751 clips of my own monolingual English (95.3 min). | CC-BY-4.0 |
English-Hebrew-Codeswitching-Sentences |
Mine | The code-switching phone takes, published on their own. Same audio as above, not extra data. | CC-BY-4.0 |
English-Hebrew-Mixed-Sentences |
Mine | An earlier desktop-mic round (2025-11-17). 439 of its recordings are imported into Read-Aloud-Voice-Sentences with fixed transcripts, and counted there. |
MIT |
LibriSpeech train-clean-100 with LibriSpeech-PC transcripts |
Public | 48.5 min of English replay, so the model keeps general English. LibriSpeech-PC supplies the cased, punctuated transcripts. test-clean (200 utterances) is used only for evaluation. |
CC-BY-4.0 |
Evaluation uses the held-out test split of Read-Aloud-Voice-Sentences (387 code-switching
clips at v1.0's revision, term-disjoint, 362 held-out terms) and is never trained on.
Training
- Data:
Read-Aloud-Voice-Sentencesat revision550bfcaa0f84, made up of:- 339 min of code-switched speech (3,224 clips; phone recordings plus an earlier desktop-mic set);
- 95 min of the speaker's own monolingual English;
- 49 min of LibriSpeech replay:
train-clean-100audio with the cased, punctuated LibriSpeech-PC transcripts.
- Training mix, by audio duration: 70.2% code-switching, 19.7% the speaker's own English, 10.0% LibriSpeech replay (target 70 / 20 / 10). This follows the code-switching literature: fine-tuning on code-switched data alone degrades monolingual English, and about 10% replay of original-domain data limits forgetting. The ratio is still untuned: PoC-3 to v1.0 were each released as a single 30% run. Comparing English shares is optional and runs only on request, to keep retrains cheap.
- Full fine-tune from stock v2 (not continued from v2) with NeMo 2.7.3:
- AdamW, lr 3e-5, cosine schedule with 10% warmup;
- 20 epochs, batch 16, bf16, on one L40S, in 41 minutes;
- the best checkpoint by validation WER was from epoch 18.
- Training config and result:
training/r6-en30/.
Usage
import nemo.collections.asr as nemo_asr
model = nemo_asr.models.ASRModel.restore_from("Parakeet-TDT-0.6B-v2-English-Hebrew-Codeswitching.nemo")
print(model.transcribe(["audio_16k_mono.wav"])[0].text)
Roadmap
This is a living model: the speaker keeps recording, and a new version is trained when enough new data has come in. Each version replaces the files here; earlier versions stay in the repo history.
When the next version is trained. Retraining is triggered by the amount of code-switching training audio, in steps of about 1.5×, because smaller steps make a difference the test set cannot measure:
| Checkpoint | Code-switching train audio | Status |
|---|---|---|
| PoC-1 (published as v1) | 77 min | Released 2026-10-15 |
| PoC-2 (published as v2) | 120 min | Released 2026-10-15 |
| PoC-3 (published as v3) | 196 min (checkpoint 180) | Released 2026-10-18 |
| PoC-4 (published as v4) | 269 min (checkpoint 270) | Released 2026-10-18; benchmark pending |
| PoC-5 (published as v5) | 294 min (off-schedule, before checkpoint 360) | Released 2026-10-18; benchmark pending |
| Jachnun v1.0 (this model) | 339 min (off-schedule, before checkpoint 360) | Released 2026-10-19; benchmarked 2026-10-19 |
| v1.1 | 450 min | Collecting; retraining is paused for a while after v1.0 |
A change to the transcripts themselves (for example a new spelling convention for a Hebrew sound) also triggers a retrain, since the model learns whichever spelling it sees.
Planned for v1.1
- Benchmark PoC-4 and PoC-5 on the v1.0 test split (v1.0, PoC-3 and stock v2 were scored on it on 2026-10-19).
- Settle the English share (optional). Runs from stock v2 at 70/30, 80/20, 90/10 and 100/0 code-switching / English, two seeds each. PoC-1 to v1.0 used about 30% without the comparison.
- Unseen words. Three versions show the same pattern: words with recordings improve, words without barely move. Recording favours terms stock v2 never gets right and terms with only one or two takes.
- Numbers written as digits ("DeepSeek 4.1"): sentences are being recorded; the test split had none at PoC-3's revision, so they are not scored yet.
Beyond v1.1
- Measure speed and accuracy on the phone itself (the GGUF in Offline Voice Input).
Not planned: tech vocabulary (a separate fine-tune), and filler-word removal and paragraph formatting (other projects). Other speakers' voices are not in the current plan; see Open questions and ideas below.
Open questions and ideas
On 2026-10-18 the speaker recorded an 18-minute voice note of open questions and ideas for the project (audio, transcript). This section is what it raises. Nothing here is settled.
Data to collect
- Messier acoustic conditions. Most takes so far were recorded at home, but real dictation happens elsewhere: in the street with traffic and construction drilling, in cafés, at the shuk, with a baby crying nearby. A few takes with a crying baby are already recorded. More takes outside the home are wanted, even with no new vocabulary in them.
- More microphones. Every capture-app take so far comes from one phone (OnePlus Nord 3 5G). Desktop-microphone takes will follow once the desk is set up again after a move.
- Pronunciation variants. Where a Hebrew word has two common pronunciations, such as the Diaspora (Ashkenazi) and the Israeli Rosh Hashanah, takes are recorded in both, so that the model writes one spelling for either.
- Other speakers, later. All training takes are still the speaker's own. Wanted later: female voices; other English accents common in Israel (South African, Australian, British, American; an Irish accent is a small minority); and Sephardi as well as Ashkenazi pronunciations of Hebrew. A contributor login for the capture app could follow once the project has some momentum.
- Vocabulary from other people's blind spots. Ask dictation users from other walks of life, for example more religious friends, which words their tools always get wrong. Gaps being filled now: smaller towns, moshavim and kibbutzim, street names in Jerusalem and Tel Aviv, public figures, politicians and new party names.
- TTS errors as a vocabulary signal. When a text-to-speech voice mispronounces an Israeli name (seen with a new political party in the speaker's AI-generated podcast), the word is missing from common lexicons, so it is a candidate for the sentence bank.
Tooling
- Tagging after the fact. Recording conditions (noise, crying, speaker) are tagged on the phone, which is awkward and often skipped. Ideas: a web tool to correct the tags afterwards, and an audio classifier that learns from a few tagged clips (a child crying, say) and finds the untagged ones.
Open questions
- How many takes does a word need? Hafnaya (a medical referral) comes up all the time. Are 6 sentences with it enough, or 12? Should take counts be weighted by how often a word is actually used, so that everyday words get more takes than once-a-year ones?
- Should the general fixes go into every fine-tune? Numbers written as digits and homophones target general weaknesses of stock v2, not code-switching. Should those sentences join the code-switching and tech training mixes, or would they dilute them?
- Speech that is not the speaker's. In a café or the street there are often other conversations nearby, in Israel frequently in several languages at once. Can a fine-tune learn to ignore them, from loudness for example, or does that need diarization or a separate target-speaker model? Not a code-switching target, but a real one.
- Language of origin. Some words used in English in Israel are Yiddish rather than
Hebrew (mamish beside the Hebrew mamash), and many Hebrew words come from other
languages. Since 2026-10-19 the dataset tags rows with a Yiddish word as
yiddish(its own subset,Yiddish-Sentences), on top ofcode_switchwhere the word is also Hebrew. Training does not treat them differently: this is a dictation tool, and its users mix words from several languages of origin.
Direction
- Separate, targeted fine-tunes rather than one that covers everything: code-switching first (the core of the project), then tech vocabulary and accent, possibly a third for general fixes. Training a model on a domain it does not need risks overfitting and forgetting for no gain. The main dataset stays one growing clip library that each fine-tune draws its own mix from.
- Slower, steady updates. PoC-1 to PoC-5 were the proof of concept; v1.0 closes it. From here, about 20 to 30 minutes of recording a week, and daily use on the phone as the real test.
- Use cases beyond dictation: Israeli place names spoken into navigation apps (Waze) in the car, medical words for patients rather than doctors, notes to self.
- Audience: English speakers living in Israel who already dictate and notice that every model, Whisper included, fails on Hebrew words, rather than local-AI specialists. Once it holds up in daily use, share it with immigrant organisations such as Nefesh B'Nefesh.
- Collaboration with the Israeli Hebrew NLP community (for example Dicta), who work on Hebrew LLMs, TTS and ASR. First contact has been made.
- Apple Silicon: done, as the MLX build (2026-10-18).
Pronunciation handling
One of the less obvious facets of this kind of code-switching is that the words switched into English do not come from one language with one pronunciation. Some are Modern Hebrew, some are Yiddish, and even within Hebrew there is more than one pronunciation tradition. Modern Israeli Hebrew uses a Sephardi-based pronunciation (Shabbat, Sukkot, brit); many English-speaking Jews use the Ashkenazi pronunciation of the same words, especially religious ones (Shabbos, Sukkos, bris), and one speaker can move between the two in a single conversation. Add the speaker's own accent in English, and this is a harder target than code-switching with a single donor language.
Methodology. I record some Hebrew words deliberately in more than one pronunciation, so the training data has the same term said in different ways. The transcript follows the sentence's spelling: "Shabbat" read the Ashkenazi way is still written "Shabbat", so the model is taught to map both pronunciations to one spelling. Only a sentence that spells the Ashkenazi form ("Shabbos") is transcribed that way.
Tagging. I think the variants need to be tagged properly for the best performance: without a
tag, the model cannot be scored per pronunciation and the mix cannot be balanced. Since 2026-10-19
the dataset has a hebrew_pronunciation column (israeli, ashkenazi, mixed, or null for not
tagged), set per take in the capture app. Older takes are null apart from the few whose transcript
already spells an Ashkenazi form, and the deliberate variants recorded before then are still to be
retro-tagged. Versions up to v1.0 were trained without the tag, and their results are not broken
down by pronunciation.
Limitations
- One speaker. Gains on other voices are untested.
- About four and a half hours of code-switched audio, with most Hebrew terms recorded only a few times. Recognition of unseen terms was under 50% at PoC-3, and still is at v1.0 (47.8% on the held-out split).
- Spellings follow one convention (the Hebrew-Latin-Token-Classifier lexicon), so the model may prefer it over other romanizations.
- On-device accuracy is not yet measured beyond the five-clip GGUF check above.
Licence and attribution
CC-BY-4.0, inherited from the base model. Base model © NVIDIA,
nvidia/parakeet-tdt-0.6b-v2 (CC-BY-4.0).
LibriSpeech-PC transcripts: CC-BY-4.0.
- Downloads last month
- -