SANBAR @ LT - EDI -2023:Automatic Speech Recognition: vulnerable old-aged and transgender people in T amil (2023)

Abstract: An Automatic Speech Recognition systems for Tamil are designed to convert spoken lan- guage or speech signals into written Tamil text. Seniors go to banks, clinics and authoritative workplaces to address their regular necessities. A lot of older people are not aware of the use of the facilities available in public places or office. They need a person to help them. Like- wise, transgender people are deprived of pri- mary education because of social stigma, so speaking is the only way to help them meet their needs. In order to build speech enabled systems, spontaneous speech data is collected from seniors and transgender people who are deprived of using these facilities for their own benefit. The proposed system is developed with pretraind models are IIT Madras transformer ASR model and akashsivanandan/wav2vec2- large-xls-r-300m-tamil model. Both pretrained models are used to evaluate the test speech ut- terances, and obtainted the WER as 37.7144% and 40.55% respectively.

## introduction the earliest known old tamil inscriptions are found in adichanallur and date back to the period between 905 bc and 696 bc. these inscriptions provide valuable insights into the early stages of the tamil language.tamil employs an agglutinative grammar system. this means that suffixes are attached to words to convey various grammatical features, such as noun class, number, case, verb tense, and other categories. tamil's historical significance and linguistic features make it a fascinating language with a rich cultural heritage. now a days all the people are using internet for their everyday needs. especially old-aged and transgender who are deprived of education due to prejudice of social. so speech is only tool to do their own needs. in the present situation all people are using smartphones to do their daily needs without any direct visits to bank, hospital, etc. the integration of a voice and speech recognition system in tamil would be highly advantageous for native tamil users who encounter difficulties with default foreign languages on their smartphones @xcite . it would elevate their user experience, facilitate accessibility, boost productivity, uphold the language's preservation, and encourage the creation of localized applications. in @xcite , two approaches were employed to improve automatic speech recognition (asr) for gujarati, tamil, and telugu languages. the first approach involved data-pooling and phone mapping, which combined the data by mapping phones from the source languages to the target language. the second approach utilized a multi-task deep neural network (dnn) with a modified loss function to train the asr model using the pooled data. this technique achieved relative reductions in the wers. @xcite because of the impact of speech articulation and speaking tendencies, older individuals tend to exhibit slower speech rates, longer pauses between syllables, and slightly reduced speech clarity. smart devices are now not only extensively used by the younger generation but also by seniors, both indoors and outdoors throughout the day. thus, we concluded that implementing a speech-recognition interface within a smart device could enhance its user-friendliness for the elderly and transgender. this feature would be particularly valuable for senior citizens during critical scenarios, including emergencies or situations where they might be physically restricted due to a traumatic event. numerous seniors encounter uncertainty when attempting to utilize the provided devices meant to aid them. similarly, transgender individuals, due to societal discrimination, are often deprived of access to primary education, leaving speech as their sole means of addressing their requirements. the information pertaining to natural speech is gathered from elderly and transgender individuals who are unable to avail themselves of such assistance. the paper is organized as follows: the following section gives a detailed description of pretrained models used for our proposed work. section 2 discusses the related work previously done for our current work. section 3 detailed description of the data set used for this work. discusses on the recognition system and the various approaches used for this work. section 4 explains in detail the pretrained models used for the proposed work. section 6 discusses on the results. section 7 analysis on the performance of each pretrained model. section 8 concludes the paper and discusses the areas of further improvement. the asr system, which involves fine-tuning a pretrained wav2vec2.0 xlsr model with ctc (connectionist temporal classification), has been successfully developed. this system demonstrates the ability to recognize speech samples and provide accurate transcriptions. the average word error rate (wer) achieved by the system is 0.58, indicating a relatively low rate of transcription errors. similarly, the character error rate (cer) is 0.11 @xcite . in this paper, (rojathai and venkatesulu, 2014), have pac features were extracted from input speech samples,the extracted features are energy entropy, zero crossing rate and short time energy. the extracted pac features were trained by anfis system. the process of recognition performance is validated based on test words.this proposed method gave better results with different noise levels compared to previous methods. in this work @xcite show the features extraction based on english phonemes and language-independent inferred phones (ips). the tamil language-independent inferred phones (ips) are achieved better performance. but it has less number of speakers, so increasing number of speakers to the model training to avoid over-fitting. @xcite , were presented the deep neural network and membrane bio reactor design for tamil. a single vowel sound were linked with five different mono phones. the transcription was linked with lm (language model). so that it will decrease the impact of domain difference in am. the target of this work is preprocess the data in order to remove noise and improve brnn-som classification method scheme gains high-accuracy. sgf method was use for removing noise present in the speech samples. we achieve highest snr values. preprocessing technique @xcite . (s and n, 2017), show reduce the feature vector dimension reduction using linear discriminant analysis(lda). lda method is perform great when compared to all other well known dimensionality reduction methods like, pca, mds, lle. deep speech architecture is used by @xcite and achieves 24% wer. the accuracy depends on the training language model. tamil dialect recognition required to be regionally based spoken tamil data to build independent tamil recognition system. (nivetha s, 2020), explain the speech recognition system used random forest algorithm to identify isolated tamil word. both mfcc and lpc features are extracted from speech data. this algorithm gave better result and took less time for training model building. in this work, @xcite show the hmm based model used to build the speaker independent isolated tamil words recognition. its achieved 88% accuracy and 0.88 wer. but data set size is minimal only 2500 words are used in this recognition system. in this task, (chakravarthi, 2020) author used 20,198 tamil comments collected from youtube. its includes women in stem, lgbtiq issues, covid-19, india china war and affairs of dravidian from youtube comments. this data has two or more languages used by a single speaker. the data set is code mixed. naive bayes, knn, and svm, logistic regression used to train the model and use held-out test set to evaluate the trained model. evaluate the trained model. the result is shown using precision, recall, f1-score. findings of the automatic speech recognition for vulnerable individuals are given in @xcite . @xcite , have pretrained rajaram1996/wav2vec-large-xlsr-53-tamil transformer model used for transformer based asr for vulnerable individuals in tamil. in this pretained model gave 39.65% wer. ## data set description tamil speech utterances are collected from the oldaged people and transgender whose mother tongue is tamil. the recorded speech utterances of oldaged people and transgender contains how those people communicate in primary locations like bank, hospitals and administrative office.the data set contains 51 speakers of literates, illiterates elders and transgenders. people who have their primary edu-cation till sixth grade are considered literates while collecting data. the duration of corpus is 7 hours and 30 minutes. it is ensured that no audio recorded from an individual is less than 5 minutes. no interruption or overlap of other person voices in the audio other than the speaker's audio. the speech files in the directories are in the wav format. the sampling rate of the speech utterances are 44khz. the speech corpus with 5.5 hours of transcribed speech will be released for the training, and 2 hours of speech data will be released for testing. this file should include paths to of the audio and transcript files, as well as other parameters like sampling rate, the language of the text, etc, are dumped. the transformer model akashsivanandan/wav2vec2-large-xls-r-300mtamil model is a variant of the wav2vec2 architecture that has been fine-tuned specifically for the tamil language. the model has been trained in an unsupervised learning manner, it means the model doesn't need explicit transcriptions. the model was pre-trained using speech signal from multiple sources including babel, multilingual librispeech (mls), common voice, voxpopuli, and voxlingua107. the sampling rate of the original common voice dataset is 48khz. however, training the xlsr model, to ensure compatibility with the xlsr model's 16khz sampling rate, the common voice data was resampled and adjusted from its original 48khz sampling rate to the desired 16khz sampling rate. this downsampling process was implemented to align the data with the model's requirements. ## implementation the one of the pretrained model used in our proposed work is "akashsivanandan/wav2vec2-largexls-r-300m-tamil". this is model is fine tuned the xlsr model using the common voice tamil speech corpus. the other model used in the proposed system is "iit madras transformer asr model". the iit madras transformer asr model is created using espnet transformer model:e2eselfattention mechanism employs language model has a 12-layer encoder and a variable decoder. each encoder layer incorporates self-attention and 2048 units of a feed-forward neural network and relu activation. the decoder comprises six layers for indian languages, each featuring self-attention and a 2048-unit feed-forward network. for voxforge, there is a single layer with a 1024-unit feed-forwaproceedings of the second workshop on language technology for equality, diversity and inclusionrd network. 104.5 hours of tamil speech data used for training, and unified processor was used to generate lexicon. the model akashsivanandan/wav2vec2-largexls-r-300m-tamil pretrained model used for testing the spoken tamil speech data. this pretrained model is tuned by facebook/wav2vec2-large-xlsrr-300m-tamil using common voice tamil data. wav2vec2 uses a combination of convolutional neural networks (cnns) and self-attention mechanisms to learn speech representations from raw audio waveform. by cnns, it can extract local acoustic features, while self-attention mechanisms enable it to capture long-range dependencies in the audio data. it can cognizant and process tamil speech data efficacious. the model take advantage of a fusion of cnn and self-attention mechanisms to learn powerful representations from raw audio waveform. it has a larger capacity, allowing it to capture complex patterns in the audio data. in this model learning rate is 0.0003, training batch size and testing batch size are 16 and 8. optimizer: adam with @xmath0.9,0.999) and @xmath1. the speech corpus has 239 speech files for testing. the test speech data is given as input to the both the pretrained system. once the speech recognition is completed the output text transcription stored in a individual file. finally wer is calculated based on what transcription got from recognition and target transcription of the speech data which is used for testing. ## conclusions an automatic speech recognition system is developed with a pretrained fine tune models which is available publicly. the speech data is collected from old-aged people and transgender whose mother tongue is tamil. the speech data contains how the people access primary location in day to day life. evaluate the test speech samples using pretrained models and calculated wers. going forward, increase speech data and create our own training model with more region-based tamil speech data. this will enhance the performance of the proposed system.