Human-Evaluated U rdu- E nglish Speech Corpus: Advancing Speech-to-Text for Low-Resource Languages (2025)

Abstract: This paper presents our contribution to the IWSLT Low Resource Track 2: ‘Training and Evaluation Data Track’. We share a human-evaluated Urdu-English speech-to-text corpus based on Common Voice 13.0 Urdu speech corpus. We followed a three-tier validation scheme which involves an initial automatic translation with corrections from native reviewers, full review by evaluators followed by final validation from a bilingual expert ensuring reliable corpus for subsequent NLP tasks. Our contribution, CV-UrEnST corpus, enriches Urdu speech resources by contributing the first Urdu-English speech-to-text corpus. When evaluated with Whisper-medium, the corpus yielded a significant improvement to the vanilla model in terms of BLEU, chrF++, and COMET scores, demonstrating its effectiveness for speech translation tasks.

## introduction speech translation (st) is a key area of speech and natural language processing that involves translating spoken content across languages @xcite . it typically integrates automatic speech recognition (asr), machine translation (mt), and text-to-speech (tts) capabilities in a pipeline. early research adopted a cascade paradigm, where asr, mt, and tts operated in separate stages @xcite . however, recent progress has shifted the focus toward end-to-end architectures that unify these components into a single, trainable model, reducing latency and error propagation between modules @xcite @xcite . speech-to-text translation (s2t), a specialized form of end-to-end st, involves converting speech signal in the source language to textual output in the target language @xcite @xcite . the success of s2t systems critically depends on the quality, size, and linguistic diversity of the training corpus, which underpins model generalization and robustness @xcite . historically, s2t corpora have evolved from task-specific datasets to large-scale multilingual resources that are essential for building performant translation systems @xcite @xcite @xcite . despite this evolution, corpus creation for low-resource languages remains severely underdeveloped due to challenges such as dialectal diversity, limited written resources, and high annotation costs @xcite . while, these advances have accelerated progress in st for high-resource languages, lowresource languages continue to face substantial challenges @xcite @xcite . asr, tts, and mt models have shown impressive gains in well-resourced settings, but the lack of well-annotated, parallel speech-text corpora has hindered similar progress for underrepresented languages like urdu. this data scarcity is a fundamental bottleneck not only for st but also for downstream tasks like cross-lingual retrieval and multilingual dialogue systems @xcite @xcite . urdu remains a low-resource language for speech translation, with only a few domainspecific corpora available @xcite . urdu-english is a moderately resourced language pair with existing corpora for tts @xcite , asr @xcite and machine translation (abdul rauf et al., 2020 we worked on an urdu subset of mozilla common voice 13.0 @xcite . though, mozilla common voice provides open-source urdu speech, yet its original transcriptions are unvalidated and susceptible to crowd-sourced inconsistencies @xcite . since, the urdu transcriptions underwent comprehensive human validation, they can be used as a gold-standard foundation for asr tasks. in addition to transcription integrity, we ensured precise english translations that carefully preserve idiomatic structures, named entities, cultural references, and semantic intent. this positions the corpus as a valuable parallel text resource for urdu-english machine translation and cross-lingual nlp applications. we followed a three-tier validation scheme. firstly, initial translations were generated automatically using the google translate api. this was followed by an expert correction phase, where two native urdu-english bilinguals manually refined the translations to eliminate syntactic, semantic, and contextual errors. finally, an extended review and final validation was performed. this multi-phase pipeline ensures high inter-annotator reliability, contextual fidelity, and translation accuracy, improving the datasets suitability for both speech-to-text and text-to-text modeling. ## related work recent advances in multilingual low-resource speech datasets have led to innovative data collection and transcription strategies. for instance, @xcite introduced gigaspeech 2, an asr corpus for thai, indonesian, and vietnamese via automated web crawling, transcription, and iterative refinement. @xcite focused on marathi asr and emphasized diversity by sourcing speech from 36 speakers across rural and urban communities, yielding a 109-hour corpus that captures dialectal variance. community-driven initiatives are also central to low-resource dataset development. butryna et al. (2020) the availability of urdu-english speech-totext corpora remains sparse compared to betterresourced language pairs. while, efforts like the urdu-english parallel corpus for speech translation offer foundational bilingual resources, they often lack rigorous human evaluation and speech alignment @xcite . mozilla common voice provides opensource urdu speech, yet its original transcriptions are unvalidated and susceptible to crowd-sourced inconsistencies @xcite . in machine translation, domain-specific corpora have contributed meaningfully. for example, the legal-uqa dataset addresses the legal q&a domain using constitutional texts @xcite , while the urdu-english religious domain corpus offers 18,426 sentence pairs for theological texts @xcite . although these datasets advance text-to-text mt, they lack paired audio components necessary for s2t applications. ## corpus preparation pipeline our corpus comprises of approximately 7k sentence pairs from the urdu subset of mozilla common voice 13.0 @xcite . the focus was on creating high-quality, human-validated translations rather than maximizing scale. common voice was chosen for its open license, reproducibility, and established use in speech research. future expansions will consider integrating other open-access urdu speech resources, contingent on annotation capacity. common voice 13.0 features a communitydriven validation system, where users vote on the correctness of audio-transcription pairs. while, this process ensures surface-level alignment, it does not address deeper syntactic or semantic inconsistencies common in urdu, a morphologically rich language. examples of such issues are presented in ## model building to establish a performance reference, we finetuned openais whisper-medium foot_0 model, a transformer-based encoder-decoder pretrained on multilingual speech data for direct speech-to-text translation. training was performed on a google colab a100 gpu using the adamw optimizer with a learning rate of 1 × 10 -5 with cosine annealing. the batch size of 16 was used and early stopping was based on the improvements in the bleu score on the development set. all audio inputs were resampled to 16 khz and converted into 80-bin log-mel spectrograms. zero-padding ensured uniform sequence lengths within batches. we used bleu (token-level accuracy), chrf++ (character-level fluency), and comet (semantic adequacy) as evaluation metrics. bleu scores measures token-level overlap with reference translations and reflects surface-level accuracy. chrf++ captures character-level fluency and recall, it is especially robust to morphological variation, whereas comet evaluates semantic similarity using neural metrics, higher scores indicate better meaning preservation. ## conclusion this study contributes a human evaluated urduto-english speech-to-text corpus designed to advance nlp research in under-resourced linguistic domains. by integrating automated translation with systematic human validation, we address critical gaps in handling idiomatic and culturally specific content, producing translations which retain the cultural aspects.