SSNCSE @ LT - EDI -2025:Speech Recognition for Vulnerable Individuals in T amil (2025)

Abstract: Speech recognition is a helpful tool for accessing technology and allowing people to interact with technology naturally. This is especially true for people who want to access technology but may encounter challenges interacting with technology in traditional formats. Some examples of these people include the elderly or people from the transgender community. This research presents an Automatic Speech Recognition (ASR) system developed for Tamil-speaking elderly and transgender people who are generally underrepresented in mainstream ASR training datasets. The proposed work used the speech data shared by the task organisers of LT-EDI2025. In the proposed work used the fine-tuned model of OpenAI’s Whisper model with Parameter-Efficient Fine-Tuning (P-EFT) with Low-Rank Adaptation (LoRA) along with SpecAugment, and used the AdamW optimization method. The model’s work led to an overall Word Error Rate (WER) of 42.3% on the untranscribed test data. A key feature of our work is that it demonstrates potential equitable and accessible ASR systems addressing the linguistic and acoustic features of vulnerable groups.

## introduction automatic speech recognition (asr) has made great advancements in the form of multilingual models like openai's whisper that demonstrate solid performance across a variety of languages and acoustic conditions. nevertheless, these systems apply in a generic sense and perform poorly when a marginalized population, such as elderly and transgender speakers, is in a low-resource language setting like tamil. these populations may have unique vocal characteristics, vocabulary, or fluency differences that are not represented in the standard asr model training data. to address this disparity, we must improve current asr systems to better understand the linguis-tic and phonetic diversity of the group. recent research describes how asr systems can introduce significant bias, especially when the target user does not fit the general speaker types used when training a model @xcite . there are even greater disparities present in languages such as tamil because there is rarely an opportunity to directly account for acoustic variability across speaker demographics at scale. to this end, we explore a parameter-efficient fine-tuning (peft) method with a pretrained version of openai's whisper model, a state-ofthe-art multilingual asr system @xcite . peft approaches like low-rank adaptation (lora) allow efficient fine-tuning by only fine-tuning a small subset of model parameters for computational efficiency while also allowing for the performance to be maintained @xcite . this also makes peft approaches well-suited for customizing asr systems for underrepresented user groups in low-resource settings. we will use this technique to transcribe tamil speech from old and transgender speakers. we wish to assess inclusive and effective options for whisper and contribute to equitable and accessible speech technologies. the rest of the paper is organized as follows: section 2 analyzes the related works done in the previous research, and section 3 discusses the speech corpus used in the current work. section 4 contains a detailed discussion of the proposed model. section 5 explains the experimental results. section 6 discusses the limitations. section 7 concludes the paper. ## related work automatic speech recognition (asr) for underrepresented and marginalized communities, such as elderly individuals and the transgender community, is an area that is still difficult and underdeveloped, particularly in lower-resource languages such as tamil. (b. @xcite provides an overview of the fifth shared task on speech recognition for vulnerable individuals in tamil. in recent years, there have been increasing efforts aimed at developing asr systems that are more inclusive for such populations -the lt-edi shared tasks @xcite have been very helpful in gathering and sharing annotated datasets of tamil speech data, with elderly and transgender speakers, and setting benchmarks to evaluate models. the paper @xcite provides an overview of this shared task on speech recognition for vulnerable people. dynamic models such as wav2vec 2.0 and whisper have been refined in several experiments to consider these distinct speech patterns. a study conducted by @xcite , tamil datasets and vulnerable speakers were optimized on whisper and xls-r models, and reported a word error rate (wer) of 24. 45%. the wav2vec 2.0 large-xls-r300m-tamil model provided a wer of 29.30% when using speech data that included elderly speakers @xcite . other works comparing traditional transformer-based models (bert, roberta) with wav2vec reported wers ranging from 37.71% to 40.55% @xcite . beyond the tamil and the lt-edi shared tasks, international work has been done to develop asr for underrepresented groups across languages. the authors @xcite used a domain-adapted self-supervised model (beginning with), wav2vec 2.0 as a feature extractor for the tdnn and conformer asr systems, finding lower wers in dysarthric speech and elderly englishspeaking speech corpora. the research @xcite pursued a different approach by finetuning asr systems over speech data from people with parkinson's disease (i.e., subglottic), and enhanced models as a multitask learning architecture to produce more accurate transcriptions, as well as prediction of symptom severity. a new approach to adaptation, spectrotemporal adaptation, was used by @xcite , where the models were specifically fine-tuned with the speech data of dysarthric and elderly speech, which outperformed all baseline adaptation strategies. an improvement in wer for disordered speech recognition was demonstrated by @xcite ) by hyperparameter tuning with a set of conformer models. in low-resource contexts (e.g., elderly frisian speakers), when the authors used data augmentation and fine-tuned wav2vec 2.0 xls-r (and trained the models with several learning rate adjustments), the findings indicated a 20% relative wer improvement when each audio sample was 50 seconds or longer. finally, (chen and asgari, 2020) used transfer learning to train asr on socially isolated seniors and approached the problem with attention mechanisms to improve robustness given the inevitable variability (e.g., cognitive and acoustic) with this population. in general, the studies discussed in this section promoted the importance of language, domain-aware, and inclusive adaptation approaches in the construction of asr systems for marginalized groups of people. in this work, our approach proposed an effective automatic speech recognition (asr) system for elderly and transgender tamil speakers by applying parameter-efficient fine-tuning (peft) on the whisper model. unlike prior work that relies on full model fine-tuning, we proposed that only a small and hopefully manageable subset of parameters is adapted, which provides a cost-effective option while maintaining similar performance to the unwieldy fine-tuning of 176 million parameters. this approach provides a good way forward as asr can scale to lower resource settings and thus provide an avenue for developers to address inclusive asr. overall, this work offers a simple and low-cost pragmatic approach to providing access to communities to the speech landscape. ## speech corpus description the dataset is comprised of spontaneous speech data collected to support automatic speech recognition (asr) for vulnerable tamil-speaking communities (especially elders and transgender people) @xcite . people in their older years commonly attend primary points such as banks, hospitals, and administrative offices, which address their needs in their daily lives, where communication is essential. this dataset includes 7 hours and 30 minutes of spontaneous tamil speech from people whose mother tongue is tamil. recordings were done in controlled environments to ensure clarity of audio quality with no background noise or overlap. all files are in .wav format. we have approximately 5.5 hours of the audio transcribed, and as the training set, 2 hours is used for testing, and is un-transcribed. ## proposed methodology this study presents a tamil automatic speech recognition (asr) system specifically for elderly and transgender speakers and realizes the differences in linguistic characteristics and phonetic uniqueness that exist in their spontaneous speech. the above speaker groups are noticeably underrepresented in speech data, which can lead to poor or no performance by general asr systems. to overcome this aspect, the methodology is proposed to adapt a large, pre-trained speech model by way of a parametrically efficient methodology. a fine-tuned version of ope-nai's whisper model, known as yaygomii/fyp_whisper_peft_tamil @xcite , is used as the base system. this model is adapted using low-rank adaptation (lora), a technique under the parameter-efficient fine-tuning (peft) framework. lora allows a model to learn domain-specific features by introducing trainable low-rank matrices into pre-existing attention layers while freezing all but a small number of parameters. this reduces the computational burden and memory use in the training phase, making it a great option for low-resource scenarios. the model was fine-tuned using 5.5 hours of tamil speech, manually transcribed from elderly and transgender speakers. given the relatively small dataset size, this methodology incorporates data augmentation to improve model generalizability. use of specaugment by using time and frequency masking of the input spectrograms is included to simulate variability in speech patterns that may be present in actual recordings from the populations of interest. background noise is imported into the training dataset to also help improve robustness for the model to handle real environmental conditions. the goal of these augmentation strategies was to mitigate overfitting and enhance the model's generalization to denoting expected acoustic variation in spontaneous conversation and speech. once the model was fine-tuned, transcript files underwent post-processing to improve their linguistic and contextual accuracy. linguistic 'ripe' corrections pertinent to tamil used domain-specific dictionaries and language model-based rules and corrected graphemic inconsistencies, removed common spelling mistakes while keeping contextual consistency in mind for the sentence; this is very important for a morphologically rich language like tamil. in addition, decoder prompts can be forced during the decoding stage, so that the model remains consistent with the tamil language, particularly in multilingual settings. the proposed strategy focuses not only on improving recognition performance for a minority group but also illustrates a commitment to the ethical and inclusive design of speech technologies. by adjusting the system to the phonetic, lexical, and syntactic tendencies of marginalized communities, the resulting template is intended to make asr systems more equitable and representative. the method leaves room for future improvement by proposing the use of speaker embeddings, for greater personalization and context, and the use of semi-supervised learning, to utilize the rest of the untranscribed half of the dataset. these directions continue to align with the larger purpose of developing robust and accessible asr systems to serve many speaker populations with integrity. ## experimental results the effectiveness of the proposed tamil asr system was evaluated, and a 2-hour test set was developed subsequently using the recordings of spontaneous elderly and transgender speech that had not been transcribed. the recordings reflected the same demographic and acoustic characteristics of the data used for training. word error rate (wer) is the main evaluation metric in this study, which is the percentage of substitutions, insertions, and deletions required to match the system's output to the true output. after decoding the output and performing postprocessing with tamil-specific corrections and dictionaries for the reference, the model was evaluated, and the wer was 42.3% over the test set. fig. 1 demonstrates the target and predicted sentences. the source code for the proposed approach and found here foot_0 the model has shown evidence of transcribing tamil speech from older adult and transgender participants while showing respect to dialectal and morphological ambiguities. overall, the model's syntax and intention worked very well within the speech, but rare forms were difficult and therefore not transcribed well with this original achievement. the model achieved this level of performance in a resource-constrained language environment with lora tuning of the pre-trained model, data augmentation, and tamil-specific post-processes. ## limitations the current work faced several hurdles, including: this work presents an inclusive tamil automatic speech recognition (asr) system tailored for elderly and transgender individuals, developed using a parameter-efficient fine-tuning (peft) approach on openai's whisper model. the system leverages a range of techniques, including data augmentation, specaugment, lora, and tamilspecific post-processing, achieving a word error recognition/blob/main/automatic_speech_detection. ipynb rate (wer) of 42.3% on spontaneous speech from marginalized communities. despite limited resources, this work demonstrates the feasibility of building equitable, speech-based technologies in low-resource and underrepresented settings. future directions include integrating speaker embeddings, employing semi-supervised learning to utilize untranscribed speech, and reducing recognition errors to further enhance accessibility for speakers with diverse speech patterns.