Papers
arxiv:2609.13498

Building a Production Greek-English Speech Recognizer

Published on Sep 11
· Submitted by
Ayoub Kirouane
on Sep 15
Authors:
,
,
,
,

Abstract

A production bilingual Greek-English speech recognition system was engineered through iterative training, data filtering, ablation studies, and model ensembling to meet strict quality gates and achieve competitive benchmark results.

We report a multi-month engineering program to build Sophea, a production bilingual Greek-English automatic speech recognition system. We evaluate the system against nine production gates covering Greek and English word error rate, language identification, and hallucinations on non-speech audio. Across twenty-three training iterations and two model architectures, no training-data composition passed all nine gates simultaneously. Meeting the Greek noisy-environment target required about 1,500 steps of dense domain exposure, while preserving English language identification tolerated only about 250 steps, or about 1,250 with a rebalanced mix that reduced Greek accuracy. We describe a six-stage data pipeline in which calibrating an audio-quality filter against in-domain anchors reduced the discarded share of scored Greek audio from 98.7 percent to 10.6 percent. A pre-registered ablation isolated a hallucination defect to one training-data package. A three-model ROVER ensemble increased gate coverage from 4-7 of 9 for individual models to 9 of 9 and reduced overlapping-speech WER from 53.35 percent to 37.87 percent, a 29 percent relative improvement. A separate learned per-clip arbiter over two models is listed as sophea/asr-k1 (preview) on the public Open ASR Leaderboard, with 4.26 percent average WER across eight public English test sets, and reaches 25.88 percent WER on live Greek noisy-environment traffic. We also document five cases in which a measurement tool produced a plausible but incorrect result and seven substantial approaches that were evaluated but not shipped. No model weights or training data are released; we report methodology and quantitative results only.

Community

Paper author Paper submitter

Hi folks, author here.

This is the story of getting Sophea ASR model, our Greek-English ASR, into production. We set nine quality gates (Greek WER, English WER, language ID, hallucination on silence, etc.) and wouldn't ship until all passed.

No single model ever did. Best of 23 training runs across two architectures was 7 of 9. Greek in noise wanted ~1,500 steps of in-domain data; English language ID broke after ~250. A three-model ROVER ensemble got us to 9 of 9 and cut overlapping-speech WER from 53% to 38%.

Two things worth your time: calibrating our audio filter against in-domain anchors dropped discarded Greek audio from 98.7% to 10.6%. And we document five times a measurement tool gave a plausible wrong answer, plus seven things we built and didn't ship.

Paper author Paper submitter
edited about 3 hours ago

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2609.13498
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2609.13498 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2609.13498 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2609.13498 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.