NSR _ LT - EDI -2025 Automatic speech recognition in T amil (2025)

Abstract: Automatic Speech Recognition (ASR) technology can potentially make marginalized communities more accessible. However, older adultsand transgender speakers are usually highly disadvantaged in accessing valuable services due to low digital literacy and social biases. In Tamil-speaking regions, these are further compounded by the inability of ASR models to address their unique speech types, accents, and spontaneous speaking styles. To bridge this gap, the LT-EDI-2025 shared task is designed to develop robust ASR systems for Tamil speech from vulnerable populations. Using whisper based models, this task is designed to improve recognition rates in speech data collected from older adults and transgender speakers in naturalistic settings such as banks, hospitals and public offices. By bridging the linguistic heterogeneity and acoustic variability among this underrepresented population, the shared task is designed to develop inclusive AI solutions that break communication barriers and empower vulnerable populations in Tamil Nadu.

## introduction speech is the most natural form of human communication. as speech technologies and ai advance rapidly, automatic speech recognition (asr) systems have become essential for human-computer interactions, offering convenience especially in multilingual countries like india. however, a significant gap exists in how these technologies reach vulnerable communities.but there is a big disconnect in how these technologies get to communities that are at risk. elderly people and transgender communities are marginalized groups that struggle to access essential services in tamil-speaking areas. while transgender communities, who are frequently the targets of discrimination in society, encounter obstacles in education and digital literacy, older adults struggle with age-related disabilities that make interacting with digital interfaces more difficult. the most dependable method of expressing needs for both groups is still face-to-face interaction; however, current asr systems that were trained on mainstream data are unable to accurately transcribe their distinct speech patterns. developing reliable asr systems for these groups involves multiple challenges. elderly speakers show acoustic variability due to physiological changes like decreased vocal cord elasticity and altered speech rhythm. transgender speakers may present varied vocal characteristics influenced by hormonal treatments or voice therapy. tamil itself poses linguistic challenges as a dravidian language with complex morphology, multiple dialects, and code-mixing tendencies. compounding these issues is the severe shortage of high-quality speech corpora from older and transgender tamil speakers. available resources typically focus on standard varieties with minimal coverage of marginalized voices. to address these gaps, the lt-edi 2025 shared task on speech recognition for vulnerable speakers in tamil(b. @xcite aims to stimulate research in this underexplored area. the task focuses on developing asr systems to transcribe spontaneous tamil speech from older and transgender speakers, providing curated datasets and leveraging models like whisper to tackle the unique challenges these populations present. this collaborative initiative primarily seeks to bridge the technological divide and enhance digital inclusion for underserved communities, empowering vulnerable tamil speakers with improved access to essential services through accurate speech recognition. more details about the shared task can be found at 1 . ## related works recent work in asr has gained interest in calibrating systems to function proficiently with underrepresented and non-standard speech categories. (s and b, 2022) introduced a transformer-based tamil conversational asr for the speech of older and trans women. training their system with actual audio captured from public spaces, they confronted natural speech variations and obtained a wer of 39.65%, divulging the sophistication in creating accessible asr. in a similar vein, (r et al., 2024) developed asr for vulnerable tamil speakers based on fine-tuned whisper and xls-r models. finetuned on lt-edi@eacl2024 data, the whisper model performed with improved robustness with a wer of 24.452, proving its ability to deal with age, gender, and social background-caused variability. @xcite presented a large-scale method by training asr models on 680,000 hours of weakly labeled web audio. their zero-shot models saw success across several benchmarks and demonstrated nearly human-level performance and extreme generalizability over languages and tasks. @xcite , in contrast, investigated asr performance on child speech, a field usually overlooked because of the scarcity of data and pitch/articulation contrasts. benchmarking six endto-end models, they highlighted the limitation of adult-trained asr systems when applied to child speech. @xcite suggested the application of weighted finite-state transducers (wfsts) to integrate acoustic, lexical, and language models within a probabilistic and efficient decoding pipeline. as opposed to end-to-end neural systems, wfsts are modular and flexible and thus effective for structured asr frameworks. overall, these studies highlight the requirement of asr systems that are robust, flexible, and inclusive, capable of accommodating linguistic, acoustic, and demographic variation. ## dataset the dataset was distributed by the shared task organisers of speech recognition for vulnerable individuals in tamil -lt-edi@ldk 2025 (b et al., 1 https://codalab.lisn.upsaclay.fr/competitions/ 21879 2022) @xcite . the audio samples are collected from individuals whose mother tongue is tamil and was presented in a .wav format. the total audio length is 7 hours and 30 minutes and is divided into 5.5 hours(approx.) ## methodology a speech recognition model (whisper-v3-large) @xcite is trained using a specially created audio-text dataset. the suggested methodology makes use of two folders, one of which contains audio recordings and the other of which contains the text transcripts that go with them. every audio file has a corresponding text file with the same name but a different extension. first, each transcript's encoding is automatically identified to guarantee accurate text file reading. because text files can be saved in a variety of formats and incorrect reading can result in errors or misinterpreted characters, this step is crucial. after being identified, the transcript is read and cleared of any extraneous words before being paired with the matching audio file in a structured list. following the collection of all legitimate audiotranscript pairs, the information is transformed into a specific format that facilitates effective audio data handling. the way the audio column is handled enables the system to directly load and process waveform data, which makes it appropriate for speech recognition model training. the dataset is divided into two sections, usually in a 90-10 ratio, one for training and one for testing, in order to accurately assess model performance. other components are initialized to get this data ready for model training. these comprise tools for converting text into numerical form, extracting significant features from unprocessed audio, and combining the two to expedite input processing. in addition, a pre-trained model architecture is loaded, which is intended to convert spoken language into written formis loaded. all of these elements are set up to function exclusively with a selected language, in this example tamil, guaranteeing that the input and output match the features of that language. when combined, the dataset and these tools form a comprehensive pipeline that is prepared for training a model to comprehend and record spoken tamil. ## experimental inference in this proposed methodology, a pre-trained whisper model from hugging face is leveraged to perform automatic speech recognition (asr) @xcite . the script randomly selects a sample from the test set of a preloaded dataset. the sample contains an audio file and its corresponding transcribed text. the audio file is extracted from the sample, along with its sampling rate, which is important for the subsequent feature extraction process. the audio data, in the form of a raw waveform, is passed to the whisper processor, which is responsible for converting the raw audio into features that the whisper model can understand. then, the waveform is transformed into a spectrogram, a 2d representation of the audio, which is the input format the model expects. once the audio has been transformed into input features, it is sent to the model which takes the processed audio features as input and outputs a sequence of predicted token ids that represent the transcription of the speech. these token ids are converted from the numerical token ids back into readable text, skipping any special tokens that may have been used during training (such as padding or end-of-sequence markers). finally, the original and machine-predicted scripts are compared, which allows an evaluation of the models performance, offering insight into how well the whisper model has learned to transcribe speech. by printing both the original and predicted text, users can directly observe how accurately the model has transcribed the audio sample, which is the primary goal of the asr process. with the same method used to obtain inference on the test data, we secured 2nd rank in this task. ## conclusion this paper presented the results of the shared task addresses a challenging area in automatic speech recognition: vulnerable old-aged and transgender people in tamil. many elderly people do not know how to use the equipment available to them. speech is the only medium that could help transgender people meet their needs because they are denied access to primary education due to societal prejudice. data on spontaneous speech are collected from elderly and transgender individuals who cannot take advantage of these resources.