ASR Under Noise: Exploring Robustness for S undanese and J avanese (2025)

Abstract: We investigate the robustness of Whisper-based automatic speech recognition (ASR) models for two major Indonesian regional languages: Javanese and Sundanese. While recent work has demonstrated strong ASR performance under clean conditions, their effectiveness in noisy environments remains unclear. To address this, we experiment with multiple training strategies, including synthetic noise augmentation and SpecAugment, and evaluate performance across a range of signal-to-noise ratios (SNRs). Our results show that noise-aware training substantially improves robustness, particularly for larger Whisper models. A detailed error analysis further reveals language-specific challenges, highlighting avenues for future improvements.

## introduction automatic speech recognition (asr) systems have made remarkable progress in recent years, especially for high-resource languages like english. while modern asr handles diverse accents @xcite and noise @xcite in high-resource languages, it remains unreliable for low-resource ones. indonesia, with 284m people and over 700 languages, is among the world's most linguistically diverse countries @xcite @xcite . yet, both remain underrepresented in asr research and resources. these languages exhibit high dialectal variation and are spoken daily in uncontrolled, noisy settings, which makes them difficult for standard asr models, which are mostly trained on indo-european data @xcite . figure 1 right illustrates how background noise severely degrades transcription quality, even with advanced models like whisper. this demonstrates the vulnerability of current asr systems to real-world acoustic challenges. amid the growing use of large-scale speechlanguage models, whisper has emerged as a strong multilingual asr system @xcite . unlike prior models such as wav2vec 2.0 and xls-r, whisper demonstrates superior robustness and generalization, particularly in noisy and lowresource scenarios @xcite . these strengths make whisper an ideal foundation for exploring asr robustness in javanese and sundanese. in this work, we present the first systematic study of asr robustness to noise in these languages using over 60 hours of training data. our key takeaways are: (1) evaluating whisper models across clean and noisy test conditions; (2) exploring training strategies like specaugment and noise-aware fine-tuning; (3) analyzing language-specific transcription errors; and (4) releasing our training and evaluation pipeline for reproducibility. this is the first work to benchmark asr robustness to noise in these languages systematically. noise robustness ensuring asr robustness in noisy environments is a well-recognized challenge @xcite @xcite . prior work addresses this through data augmentation techniques such as synthetic noise injection and room impulse re-pre-trained whisper clean audio specaugment noisy audio time warp frequency masking time masking base audio openslr (train split) + base audio background noise sampled from audioset fine-tune evaluation wer / cer (clean audio) openslr (test set) audioset held-out classes sponses. among these, specaugment (park et al., 2019) has gained popularity as a simple and effective method. other approaches include noise-aware training @xcite and denoising frontends @xcite . in our work, we independently evaluate specaugment and noise-aware finetuning, using noise samples from audioset (gemmeke et al., 2017), as two distinct strategies to improve asr robustness. ## limitations this study has three main limitations. first, the openslr corpora were only from limited regions, which may not reflect spontaneous or dialectal variation in javanese and sundanese. second, the noisy conditions are synthetic and cannot fully capture real-world environments such as conversational overlap or varied recording devices. third, our experiments focus only on whisper-based models with a small set of fine-tuning strategies. these factors constrain the generalizability of the findings but also motivate directions for improvement. ## conclusion we evaluated whisper-based asr models on javanese and sundanese under noisy conditions. while clean audio performance was strong, wer degraded by 2-3× in low-snr scenarios without noise-aware training. both specaugment and synthetic noise improved robustness, with noisetrain consistently outperforming other methods on average across models and languages. error analysis showed sundanese struggled with vowel confusion and name errors, while javanese had more digraph and consonant issues, resulting in higher wer. future work includes dialect-aware fine-tuning and speech enhancement for better realworld robustness.