DRAVIDIAN LANGUAGE @ LT - EDI 2024:Pretrained Transformer based Automatic Speech Recognition system for Elderly People (2024)

Abstract: In this paper, the main goal of the study is to create an automatic speech recognition (ASR) system that is tailored to the Tamil language. The dataset that was employed includes audio recordings that were obtained from vulnerable populations in the Tamil region, such as elderly men and women and transgender individuals. The pre-trained model Rajaram1996/wav2vec2- large-xlsr-53-tamil is used in the engineering of the ASR system. This existing model is finetuned using a variety of datasets that include typical Tamil voices. The system is then tested with a specific test dataset, and the transcriptions that are produced are sent in for assessment. The Word Error Rate is used to evaluate the system’s performance. Our system has a WER of 37.733.

## introduction the goal of this research project is to create an automatic speech recognition (asr) system that is specifically designed to serve vulnerable populations in the tamil-speaking community, such as the elderly and transgender people. this shared task's scope entails improving speech recognition skills to enable certain demographic groups to access facilities including banks, hospitals, and shopping centers. the problem is closing the information gap between the elderly, who might not know how to use the tools at their disposal, and the transgender population, which experiences educational inequalities as a result of prejudice from society. it is highlighted how important speaking is to these groups as their main form of communication because it is essential to meeting their everyday requirements. the study's dataset is made up of spontaneous speech samples that were obtained from transgender and elderly people who have trouble using different facilities. a two-hour speech dataset is provided for testing, and a training set of 5.5 hours of transcribed speech is also accessible. even while technological developments have made it easier for people to access electronic devices in a wider range of industries, educational barriers still pose a barrier for the elderly and transgender people. even though these people use electronics, the fact that they primarily rely on audio messages highlights the need for better speech recognition models in order to accommodate their distinct speech patterns and deliver accurate responses. these people provide audio input to the method described in this paper, which converts it into corresponding tamil transcripts. the word error rate (wer) is used to measure the accuracy of these transcripts, taking into account the inherent difficulty that comes with different accents. the structure of the report offers a thorough understanding of the research process. in section 2, relevant literature is examined in detail, emphasizing gaps in the field and current knowledge. the dataset is thoroughly described in section 3, along with its makeup and significance to the goals of the research. while section 5 describes the implementation procedure, section 4 covers the technique used in the development of the asr system. key findings from the research are summarized in section 6, which also offers insights on the study's performance and difficulties. a thorough discussion is started in section 7 by interpreting the findings and placing them within the larger context of the study goals. the paper's main conclusions are summarized in section 8, which also highlights the ongoing development of asr systems for vulnerable populations and moves into a discussion of possible directions for further research. the last section, section 9, offers a thorough summary of the academic background that underpinned this project and is devoted to the reference articles that were examined during the research pro-cess. essentially, this work lays the groundwork for future advancements and improvements in this crucial area by advancing asr technologies that meet the particular communication demands of vulnerable groups. ## related work voice recognition has advanced significantly over the past ten years, mostly due to the quick development of deep learning methods. ten years ago, most researchers concentrated on using deep learning to extract audio data. subsequently, these data were combined with hidden markov models, which were popular at the time. but things have changed, and modern approaches now use more advanced recurrent neural networks (rnns), like long short-term memory (lstm) networks, in place of more conventional gaussian mixture models. notably, large models with higher parameterization, such contextnet and conformer, have shown improved voice recognition accuracy. convolutional neural networks (cnns) are used by the authors of a study titled "speech rate control for improving elderly speech recognition of smart devices" to overcome the difficulties in geriatric speech recognition on smart devices. in a different study, "transformertransducer: end-to-end voice recognition with self-attention" @xcite , the authors want to use transformer networks in the neural transducer architecture to create an end-to-end speech recognition model. in order to incorporate positional information and reduce frame rates, their method integrates vggnet with causal convolution, maximizing computing efficiency through reduced self-attention. the preprocessing of data is essential to building reliable models. the authors stress the significance of labeled data for training models and discuss methods that entail using both labeled and unlabeled speech data to train large-scale models. an example of self-supervised learning is wav2vec @xcite , which uses contrastive learning for feature learning and unlabeled speech data for training. the effectiveness of deep learning in machine translation and speech recognition is recognized, highlighting its natural language comprehension abilities. this impact is being felt in a variety of domains, as scientists are investigating neural methods to comprehend code semantics and spot weak points. notably, studies on low-resource speech recognition methods for minority languages have been conducted, and attempts have been made to improve accuracy by means of data augmentation. the rajaram1996/wav2vec-large-xlsr-53-tamil transformer model is used by the authors of @xcite , @xcite to identify tamil speech utterances made by susceptible individuals. to calculate the word mistake rate, the model takes into account variables like the number of utterances and the quality of the.wav file. this concept is primarily intended to promote inclusivity for marginalized people by increasing accessibility to regional languages. in this work, audio files are transcribed using a pre-trained xlsr model, and word error rates are computed as a result. the focus is on using cuttingedge deep learning techniques to improve speech recognition systems' inclusivity and accuracy, especially when it comes to disadvantaged populations. the following parts provide a thorough analysis of relevant literature, dataset specifications, methodology, implementation details, outcomes that have been seen, and a thorough discussion. a few suggestions for future directions in this ever-evolving field of study are included in the paper's conclusion. the reference section lists the articles that were consulted for this research project, giving the provided conclusions a strong basis. ## dataset description the dataset given to this shared task @xcite is an tamil conversational speech recorded from the elderly people whose average age is around 61 for male, 59 for female and 30 for transgender people .a total of 6 hours and 42 minutes is collected from the elderly people. 46 audio files were recorded and each audio file is split into many subsets as transformer model does not support the large audiofiles. the speech is recorded with a sampling rate of 16khz. the audio files from audio -audio-10, to audio-35 are used for training (duration is approximately 5.5 hours) @xcite and audio -37 to audio -48 are used for testing (duration is approximately 2 hours). ## proposed work to construct their automatic speech recognition system, the researchers used the rajaram1996/wav2vec2-large-xlsr-53-tamil pre-trained transformer model. this model is an advanced speech recognition system designed by facebook ai specifically for the tamil language, and it is based on the wav2vec2 architecture. wav2vec2 is a self-supervised learning technique that builds representations that capture important information about audio features by utilizing massive amounts of unlabeled voice data. the model's fundamental architecture is based on transformers, which have proven to be incredibly useful in a variety of natural language processing applications. transformers improve the model's ability to do nuanced analysis by allowing it to effectively capture long-range dependencies in audio inputs. the model is trained by subjecting it to a sizable corpus of tamil speech-containing monolingual and multilingual data. pre-training enables a thorough grasp of the fundamental structure and characteristics of speech data by teaching the model to predict masked or distorted chunks of the input audio. after pre-training, the model is fine-tuned utilizing labeled data customized for particular downstream tasks, like tamil keyword detection or transcription. through this process of fine-tuning, the model can be made to adjust to the specifics of a given speech recognition task. the model gains from the multilingual character of its pre-training data even though it was particularly trained on tamil. because of the large corpus of words it has been trained on, it can process a wide variety of words and phrases, which makes it ideal for tasks like transcription or speech recognition. the training set of the model and its fine-tuning methodology are purposefully created to capture the unique phonetic, phonological, and grammatical characteristics of the tamil language. this careful process improves the model's capacity to identify and translate tamil speech. business-wise, the use of cutting-edge models such as rajaram1996/wav2vec2-large-xlsr-53tamil highlights a dedication to utilizing cuttingedge technologies in voice recognition system development. the model is able to extract meaningful representations from unlabeled data thanks to the innovative use of a self-supervised learning strategy. applying transformers, which are well-known for their effectiveness in natural language processing, shows a deliberate architectural decision to improve the model's analytical powers. in line with industry best practices, the focus on fine-tuning for certain downstream applications guarantees that the model is tuned for the subtleties of tamil speech recognition. given that the model can accommodate a wide vocabulary, it can be used as a flexible solution for transcription or speech recognition tasks that need to cover a wide range of languages. to sum up, the researchers' careful selection of the model, training process, and fine-tuning technique shows a dedication to creating a reliable and adaptable automatic speech recognition system that is suited to the nuances of the tamil language. this strategy, which is based on cutting-edge technologies and best practices from the industry, presents the system as a useful tool for companies looking for precise and flexible voice recognition solutions. the ongoing development of these models has implications for various applications in various industries and offers potential for the area of natural language processing as a whole. ## implementation we have harnessed an efficient model, leveraging a pre-trained transformer-based architecture named rajaram1996/wav2vec2-large-xlsr-53tamil. this particular model, a derivative of facebook/wav2vec2-large-xlsr-53, is specialized for tamil and fine-tuned using the common voice dataset. to operate seamlessly, this model mandates a 16 khz sampling rate for voice input. our assessments have utilized lt-edi's dataset to evaluate the model's efficacy. the core functionality revolves around loading voice utterances into the library, storing them as variables, and tokenizing them via a dedicated tokenizer. this transformation pipeline is instrumental in converting audio signals into textual representations. our meticulous approach involves a thorough comparison between these transcribed texts and the original audio transcripts. this critical alignment allows us to calculate the word error rate (wer), a metric that reflects the fidelity of voice recognition thereby used to quantify the accuracy and precision of the model's voice recognition capabilities. this approach, rooted in the xlsr (cross-lingual speech representation) framework, extends its capabilities to cross-lingual speech data, showcasing the model's adaptability across languages. the derived wer provides a robust assessment of the model's proficiency in voice-to-text transcription. by using the wer as our benchmark, we gain deeper insights into the model's performance, and efficiency for affirming its prowess in transforming spoken words into accurate text. ## evaluation of results the general difficulty of measuring performance lies in the fact that the recognized word sequence can have a different length from the reference word sequence. the wer is derived from the levenshtein distance, working at the word level instead of the phoneme level. the task's evaluation measure is based on the wer (word error rate) computed between the original transcriptions of the given audio and the transcribed text. wer ( word error rate) = ( s + d + i) / n where, @xmath0. of substitutions @xmath1. of deletions @xmath2. of insertions @xmath3. of words in the reference transcription ## observation the name of the speech data and its wer value are included in the result. similar to this, the same procedure is used for all audio files. the number of subgroups into which each audio file is divided is also listed in the table . ## discussion the number of test speech utterances are 295. from the total number of 295 audio subset files from 10 audio files which is given for testing and the wer measured is 37.73. we ranked fourth position in shared task competition. ## conclusions conversational speech data is utilised to improve the speech recognition system's capacity to detect elderly people. a trained model is used to construct an automatic speech recognition system. a dataset collection is focusing on older adults and transgender people who use tamil as their first language. the dataset's utterance was taken during a conversation in a major site in tamil. because the system's pre-trained model was enhanced using a common speech dataset, the model might be trained using our own dataset and tested in the future, which could improve performance.