ASR _ SSN _ CSE @ LTEDI - 2023: Pretrained Transformer based Automatic Speech Recognition system for Elderly People (2023)

Abstract: Submission of the paper for the result submitted in Shared Task on Speech Recognition for Vulnerable Individuals in Tamil- LT-EDI-2023. The task is to develop an automatic speech recognition system for Tamil language. The dataset provided in the task is collected from the elderly people who converse in Tamil language. The proposed ASR system is designed with pre-trained model. The pre-trained model used in our system is fine-tuned with Tamil common voice dataset. The test data released from the task is given to the proposed system, now the transcriptions are generated for the test samples and the generated transcriptions is submitted to the task. The result submitted is evaluated by task, the evaluation metric used is Word Error Rate (WER). Our Proposed system attained a WER of 39.8091%.

## introduction this shared task tackles a difficult area in tamil automatic speech recognition system for vulnerable elderly and transgender individuals. to take care of their daily necessities, elderly people go to important places including banks, hospitals, and administrative offices. many elderly folks are not aware of how to use the tools provided to help people. similar to how transgender persons lack access to basic schooling due to societal discrimination, speech is the only channel that can help them meet their demands. the data on spontaneous speech is collected from elderly and transgender people who are unable to take benefit of these amenities @xcite . 2 hours of speech data will be made available for testing, while the speech corpus containing 5.5 hours of transcribed speech will be made available for the training set. recently, the majority of people have begun using various electronic devices to access the internet. in this situation, the elderly people have also started using smart phones to access the internet @xcite . some elderly people attempt to acquire information from the internet using their audio message because they are not well-versed in technology. an acoustic model must be created to handle these types of audio messages from elderly individuals; the model will identify their speech and extract the output of the speech data. as a result, a text file will be the output. the speech's output will be used to determine the wer value. the wer number demonstrates how accurately the model predicted the outcome . no other corpus for elderly people is larger than the japanese newspaper article sentences (jnas), japanese newspaper article sentences read speech corpus of the aged (s-jnas), and corpus of spontaneous japanese (csj) corpora @xcite . it has been determined that automatic speech recognition using some standard models has not achieved a good performance @xcite . it is challenging to identify conversational speech in public settings since each person may have their own accent and pronunciation. additionally, the methodology for identifying standard speech cannot be applied to the conversational speech corpus because it raises wer. a transformer model technique is utilised to treat this type of older people's conversational speech. the paper is organised as follows: section 2 discusses the examination of related literature, section 3 describes the data set, section 4 discusses the methodology, section 5 describes the implementation, section 6 describes the observations, and section 7 discusses the discussion. section 8 of the essay concludes with a section on future research. ## related work there have been numerous studies on recognising the speech of elderly persons using the adaption acoustic model for csj corpus @xcite , which yields the lowest wer values. the performance of continuous word identification and phoneme recognition is measured from the two distinct age groups, and the corpus is collected in bengali @xcite . prosodic and spectral properties are retrieved for senior people speech.the exploration of additional features @xcite , such as the speech's volume level, sampling frequency, fundamental frequency, and sentence segmentation, is also possible. other measurements were locating the pause in the sentence and calculating how long it lasted @xcite . low number of utterances is a sign of inadequate performance. if the recorded voice quality is poor @xcite , the wer value rises. by integrating the transformers for a broad context @xcite , the e2e asr transformer can perform encoding and decoding in a hierarchical manner. the wer is decreased by 25.4% via transfer learning when using the hybridbased lstm transformer @xcite . additionally, the lstm decoder lowers wer by 13%. self-attention and a multi-head attention layer @xcite can be used to carry out the encoding and decoding of transformer models. the transformer model is utilised for ctc/attention based end-to-end asr, and it produces a wer of 23.66% @xcite . transformers for streaming asr are the foundation of the end-to-end asr system, where an output must be produced quickly after each spoken word. time-restricted self-attention is employed for the encoder, and prompted attention is used for the encoder-decoder attention mechanism. the innovative fusion attention technique results in a wer reduction of 16.7% on the wall street journal test compared to the nonfusion standard transformer and 12.1% compared to other authors' transformer-based benchmarks. findings of the automatic speech recognition for vulnerable individuals are given in @xcite , have used transformer models used for transformer based asr for vulnerable individuals in tamil. ## dataset description tamil conversational speech data is collected from the elderly people. the speech corpus contains a total of 6 hours and 42 minutes of speech data @xcite . the recorded speech of elderly people contains how the elderly people communicate in primary locations like market, bank, shop, public transport and hospitals. it includes both male and female utterances and also this speech data is collected from the transgender people. the below figure 1 . shows the sample prediction for the given corpus. 4 proposed work in our proposed system, the pretrained transformer model anuragshas/wav2vec2-xlsr-53-tamil foot_0 is used. the pretrained model "anuragshas/wav2vec2-xlsr-53-tamil" is based on the wav2vec2 architecture and specifically trained for the tamil language. wav2vec2 2 is a state-of-the-art speech recognition model developed by facebook ai. it utilizes a selfsupervised learning approach, where it learns from large amounts of unlabeled speech data to build representations that capture meaningful information about the audio. the model is based on the transformer architecture, which has proven to be highly effective for various natural language processing tasks. transformers enable the model to efficiently capture long-range dependencies in the audio input. the model is pretrained on a large corpus of multilingual and monolingual data containing tamil speech. during pretraining, it learns to predict masked or distorted portions of the input audio, which helps it understand the underlying structure and features of the speech data. after pretraining, the model undergoes fine-tuning using labeled data for specific downstream tasks. fine-tuning allows the model to adapt to a particular speech recognition task, such as transcription or keyword spotting, in this case, for tamil language. although the model is specifically trained for tamil, it benefits from the multilingual nature of its pretraining data. it can understand and process speech from various languages, making it useful for multilingual applications or tasks involving code-switching. the model has been trained on a vast vocabulary, enabling it to handle a wide range of words and phrases. this makes it suitable for tasks that involve transcribing or recognizing speech with diverse vocabulary. the model's training data and fine-tuning procedure focus on capturing the unique characteristics of the tamil language, including its phonetics, phonology, and syntax. this enhances its ability to accurately recognize and transcribe tamil speech. ## implementation to create an efficient acoustic model based on a pre-trained transformer model. there are many publicly accessible transformer-based pre-trained models. here, the "anuragshas/wav2vec2-xlsr-53tamil" pretrained model for handling tamil speech corpus is used. this pretrained model is fine-tuned from "facebook/wav2vec2-large-xlsr-53" 3 by common voice dataset in tamil. the model can be used directly and only accepts input if the voice data is sampled at 16 khz. it is independent of any language model. the model for creating the wav2vec uses the xslr (cross-lingual speech representation), which additionally tests cross-lingual speech data. the quantization of latents, which is common to all languages, can be learned by xlsr. the voice utterance is loaded into the library, saved in a variable, and tokenized using the tokenizer. this process converts the audio to text, and the results are transcripts of the audio file that is loaded into the library. the transcripts are kept in a separate folder after the voice recognition process is complete. between the transcripts produced by the model and the actual transcripts of the audio 3 https://huggingface.co/facebook/wav2vec2-large-xlsr-53 written by humans, the wer (word error rate) is determined. the degree of voice recognition can be calculated using the wer value. ## evaluation of results the evaluation metric used by the task to test the results submitted by us is based on the wer computed between the asr hypotheses submitted by the participants and the ground truth of human speech transcription. wer ( word error rate) = ( s + d + i) / n where, @xmath0. of substitutions @xmath1. of deletions @xmath2. of insertions @xmath3. of words in the reference transcription ## observation the name of the speech data and its wer value are included in the result. similar to this, the same procedure is used for all audio files. the number of subgroups into which each audio file is divided is also listed in the ## conclusion conversational speech data is used to enhance the speech recognition system's ability to identify older persons. using a trained model, an automatic speech recognition system is created. older adults and transgender people with tamil as their mother tongue are the subjects of a dataset collection. the dataset's utterance was captured during a conversation in a primary site in tamil. as the system's pre-trained model was improved using a common speech dataset, in the future the model might be trained using our own dataset and utilised for testing, which could improve performance.