Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Models (2025)

Abstract: Speech foundation models trained at a massive scale, both in terms of model and data size, result in robust systems capable of performing multiple speech tasks, including automatic speech recognition (ASR). These models transcend language and domain barriers, yet effectively measuring their performance remains a challenge. Traditional metrics like word error rate (WER) and character error rate (CER) are commonly used to evaluate ASR performance but often fail to reflect transcription quality in critical contexts, particularly when detecting fabricated outputs. This phenomenon, known as hallucination, is especially concerning in high-stakes domains such as healthcare, legal, and aviation, where errors can have severe consequences. In our work, we address this gap by investigating hallucination in ASR models. We examine how factors such as distribution shifts, model size, and model architecture influence the hallucination error rate (HER), a metric we introduce to quantify hallucinations. Our analysis of over 20 ASR models reveals key insights: (1) High WERs can mask low hallucination rates, while low WERs may conceal dangerous hallucinations. (2) Synthetic noise, both adversarial and common perturbations like white noise, pitch shift, and time stretching, increase HER. (3) Distribution shift correlates strongly with HER ( \alpha = 0.91 ). Our findings highlight the importance of incorporating HER alongside traditional metrics like WER to better assess ASR model performance, particularly in high-stakes domains.

## introduction automatic speech recognition (asr) systems have become fundamental to various applications, including personal assistants, automated customer service, and transcription tools used in fields such as education, healthcare, and law @xcite . these systems have seen remarkable improvements in recent years @xcite @xcite , with state-of-the-art models demonstrating their capabilities across diverse datasets and languages @xcite . however, the evaluation of asr performance remains largely dependent on word and character error rate (wer/cer). the primary limitation of wer and cer is their dependence on token-level overlapping, which focuses on matching individual words or characters without considering the overall semantic aspect of the transcription. this could result in misleading evaluations, as a high wer/cer does not necessarily indicate poor outputs in all cases. in addition, these metrics fall short in capturing more subtle semantic failures which are not typically caught without human verification, such as hallucinations. hallucinations in asr systems mirror perceptual experiences in neuroscience-plausible outputs generated without grounding in input stimuli @xcite , deviating phonetically or semantically from source speech @xcite . like natural neural perceptions, asr hallucinations arise when models prioritize distributional patterns over fidelity to audio input, fabricating text unlinked to reference content @xcite . these errors are uniquely hazardous in high-stakes domains (williamson and prybutok, 2024), as they evade wer/cer detection while distorting meaning, similar to how clinical hallucinations disconnect from reality. hallucination in domains such as medical and legal can have serious consequences, including life-threatening outcomes and distorted testimonies or contracts, and may disproportionately affect marginalized groups @xcite @xcite @xcite . hence, detecting and mitigating hallucination is crucial for ensuring the reliability of asr systems in sensitive environments. the existing literature on hallucination detection in asr models is confined to a single model @xcite @xcite @xcite or test setting @xcite , highlighting a significant research gap for a systematic investigation across different supervision paradigms, test domains, and conditions. in this work, we address this critical research gap and make the following key contributions: the significance of this work lies in how it redefines the evaluation and improvement of asr systems. by emphasizing hallucination detection, we aim to enhance the reliability of asr models, particularly in domains where accuracy and precision are non-negotiable. outline. in section 2, we review prior work related to asr evaluation and hallucination detection. section 3 outlines our proposed methodology. section 4 presents our experimental setup. finally, section 5 discusses the results and implications of our findings, outlining directions for future research. we conclude our work in section 6 and provide limitations in section 7. ## related work the use of asr systems in high-stakes domains, including healthcare @xcite @xcite , legal proceedings @xcite , and finance @xcite , has heightened the necessity for ensuring their robustness. conventionally, the performance of these models is assessed using metrics such as wer and cer @xcite . @xcite ; @xcite they show that when these metrics are used in isolation, they exhibit notable limitations. recent studies have extensively investigated hallucination in text generated by large language models (llms), identifying it as a prevalent phenomenon @xcite @xcite @xcite @xcite @xcite . this issue has also been observed in audio foundation models @xcite . furthermore, research suggests that pretraining language models for predictive accuracy inherently predisposes them to hallucination, even under ideal conditions where the training data is entirely factual @xcite . however, few studies explore hallucination evaluation and detection in automatic speech recognition (asr) systems, with most research focusing on whisper, a semi-supervised model. for instance, @xcite analyzes the aphasia dataset and reports that while whisper's overall hallucination rate is 1%, 40% of these hallucinations contain violent or harmful content. similarly, @xcite demonstrates that whisper hallucinates at significantly higher rates under low signal-to-noise ratio (snr) conditions, and observes a 20% increase in hallucinations at -4 db and -2 db snrs. prior work by @xcite proposes augmenting models with hallucinated transcripts to improve performance, while @xcite introduces a perturbationbased evaluation method using automatic metrics such as word error rate (wer), perplexity, and cosine similarity. @xcite develops a filtered bag of hallucinations (boh) for detection, and reveals that hallucinations in whisper correlate strongly with training data biases (e.g., phrases like "thank you for watching" linked to youtube content). ## methodology we examine transcription quality using the standard metrics wer and cer, and we assess the occurrence of hallucination errors to provide a comprehensive view of model performance. our testing environment is characterized by both natural and synthetic distribution shifts. furthermore, we investigate the deterioration of error rates when transitioning from a controlled source domain (lib-rispeech) to various target domains, with a particular focus on both wer and the hallucination error rate (her). in addition, we introduce noise to the input data and analyze its effect on error rate degradation. this multifaceted strategy offers valuable insights into the challenges encountered by asr systems in real-world scenarios. we provide additional details about each step subsequently. ## experiments in this section, we present the experimental details of our work. we examine the models' tendencies to produce hallucinated outputs, using llm-based evaluation as described in 3. the results provide insights into the reliability of different asr systems across various real-world and adversarial conditions. ## results in our experiments, we use an llm-based pipeline to classify asr errors across various evaluation setups, validating it against human evaluation and heuristic baselines. we then explore the effects of natural and synthetic distribution shifts on error metrics, specifically examining how domain variations and input perturbations impact word and hallucination error rates. additionally, we analyze the influence of model architecture and scale through a comparison of whisper variants and other architectures. this approach provides insights into the complex interactions between error types, data conditions, and model characteristics. in this section, we present our findings. ## conclusion in our work, we introduce the hallucination error rate (her) as a crucial complement to traditional asr evaluation metrics like wer, especially in high-risk applications where model reliability is critical. by developing a robust llm-based hallucination detection framework, we present a comprehensive evaluation of asr models across both synthetic and natural distribution shifts, highlighting the specific challenges asr systems face under real-world conditions. our findings emphasize the importance of incorporating her into standard asr evaluation practices, particularly for applications in safety-critical domains such as healthcare, legal, and aviation. through detailed analysis, we show that traditional metrics like wer can mask significant hallucinations, emphasizing the need for more holistic evaluation methods. our work lays the ground for future work in asr model reliability, aiming to ensure that asr systems not only produce accurate transcriptions but also avoid generating misleading, harmful, and unfaithful to input speech transcriptions. in future work, we plan to expand our evaluation to cover additional evaluation setups, ensuring a comprehensive set of assessments. additionally, we aim to explore mitigation strategies, which are critical for enhancing the reliability of asr systems.