SUH _ ASR @ LT - EDI - ACL 2022: Transformer based Approach for Speech Recognition for Vulnerable Individuals in T amil (2022)

Abstract: An Automatic Speech Recognition System is developed for addressing the Tamil conversational speech data of the elderly people andtransgender. The speech corpus used in this system is collected from the people who adhere their communication in Tamil at some primary places like bank, hospital, vegetable markets. Our ASR system is designed with pre-trained model which is used to recognize the speechdata. WER(Word Error Rate) calculation is used to analyse the performance of the ASR system. This evaluation could help to make acomparison of utterances between the elderly people and others. Similarly, the comparison between the transgender and other people isalso done. Our proposed ASR system achieves the word error rate as 39.65%.

## introduction in the recent days, most of the people have started using the internet through various electronic devices @xcite . in such a case, the elderly people have also started using the internet through smart phones. as some of the elderly people were not educated much about the technology, they try to retrieve the information from internet using their audio message. to handle such kind of audio messages of elderly people, an acoustic model has to be designed, the model will recognize the utterance of the elderly people and extracts the output of the speech data @xcite . therefore, the output will be a text file. based on the output of the speech, wer value will be calculated. the wer value shows the accuracy of the prediction by the model. it is identified that automatic speech recognition using some standard models have not achieved a good performance @xcite and also no other corpus for elderly people is larger than the japanese newspaper article sentences (jnas), japanese newspaper article sentences read speech corpus of the aged (s-jnas) and corpus of spontaneous japanese (csj) corpora @xcite . the earliest old tamil documents are small inscriptions in adichanallur dating from 905 bc to 696 bc. tamil has the oldest ancient non-sanskritic indian literature of any indian language. tamil uses agglutinative grammar, which uses suffixes to indicate noun class, number, case, verb tense, and other grammatical categories @xcite @xcite ). tamil's standard metalinguistic terminology and scholarly vocabulary is itself tamil, as opposed to the sanskrit that is standard for most aryan languages. tamil has many forms, in addition to dialects: a classical literary style based on the ancient language (cankattami), a modern literary and formal style (centami), and a current colloquial form (kotuntami) @xcite @xcite @xcite . these styles blend into one another, creating a stylistic continuity. it is conceivable, for example, to write centami using cankattami vocabulary, or to utilize forms connected with one of the other varieties while speaking kotuntami @xcite @xcite . tamil words are made up of a lexical root and one or more affixes. the majority of tamil affixes are suffixes. tamil suffixes are either derivational suffixes, which modify the part of speech or meaning of the word, or inflectional suffixes, which designate categories like as person, number, mood, tense, and so on. there is no ultimate limit to the length and scope of agglutination, which might result in large words with several suffixes, requiring many words or a sentence in english @xcite . likewise, the speech corpus of different languages are addressed by many people, those corpus contains either the male or the female speech and no corpus addressed the transgender speech. but, the speech corpus released by the shared task(lt-edi-acl2022)contains male, female and transgender utterances, which enhance the characteristic of the acoustic model, but the number of speech utterances collected are very less compared to other elderly speech corpus. the model also need to handle challenges faced in the corpus, as this shared task speech corpus contains conversational speech data in primary locations like bank, market, hospital and public transport. as the people may have their own accent and pronunciation for conversational speech in the primary places, it is difficult to recognize the speech and the model used for recognizing standard speech cannot be used for the conversational speech corpus because it increases the wer. to address this kind of conversational speech of the elderly people a transformer model approach is used. the follow of the paper is as follows: the review of the related work is discussed in section 2, data-set description is described in section 3, methodology used is discussed in section 4, followed by implementation, observations and discussion are described in section 5, 6 and 7 respectively. finally, the paper is concluded with future work in section 8. ## related work many studies have done on recognizing the elderly people speech corpus, using adaptation acoustic model for csj corpus which results in lowest wer values @xcite . the prosodic and spectral features are extracted for elderly people speech @xcite and the performance of the continuous word recognition and phoneme recognition is measured from the two different age groups and the corpus is collected in bengali language @xcite . additional feature analysis can also be done like loudness of the speech, sampling rate, fundamental frequency, and segmentation of the sentence. other measures were done by identifying the pause in the sentence and measuring the duration of the pause @xcite . insufficient performance is measured with low number of utterance @xcite . increase in wer value happens if the quality of recorded speech is low @xcite . e2e asr transformer can do encoding and decoding hierarchically by combining the transformers for large context @xcite . using the hybrid based lstm transformer, the wer is reduced with 25.4% by transfer learning. additionally, 13% wer is reduced by lstm decoder @xcite . transformer model encoding and decoding can be carried with self-attention and multi-head attention layer @xcite . for ctc/attention based end-to-end asr, the transformer model is used, which result 23.66% of wer @xcite . end-to-end asr system works based on transformers for streaming asr, where an output must be generated soon after each spoken word. for the encoder, the timerestricted self-attention is used, and for the encoderdecoder attention mechanism, prompted attention is used. on the wall street journal task, the novel fusion attention technique delivers a wer decrease of 16.7% compared to the non-fusion standard transformer and 12.1% compared to other authors transformer-based benchmarks. ## data-set description tamil conversational speech data is collected from the elderly people. the speech corpus contains a total of 6 hours and 42 minutes of speech data. the recorded speech of elderly people contains how the elderly people communicate in primary locations like market, bank, shop, public transport and hospitals. it includes both male and female utterances and also this speech data is collected from the transgender people. ## implementation to develop an effective acoustic model using a transformer based pre-trained model. there are various transformer based pre-trained model available publicly. here, the "rajaram1996/wav2vec2large-xlsr-53-tamil" pretrained model for handling tamil speech corpus is used. this pretrained model is fine-tuned from "facebook/wav2vec2-large-xlsr-53" foot_1 by common voice dataset in tamil. the model accepts the input only if the speech data is sampled at 16khz and it does not depend on any language model, instead it can be used directly. the xslr is used in the model for building the wav2vec and also it experiments the cross-lingual speech data. xlsr is capable of learning the quantization of latents which is shared across languages. the speech utterance is loaded in the librosa, then it is stored in a variable and it will be tokenized using the tokenizer, which converts the audio to text and the outputs are the transcripts of the audio file which is loaded in the librosa. once the speech recognition is done, the transcripts are stored in a separate folder. the wer(word error rate) is calculated between the transcripts generated by the model and the original transcripts of the audio created by the human. based on the wer value, the level of recognition of speech can be measured. our speech corpus contains total of 46 audio files where it is subdivided into 1147 audio files. from 46 audio files, 36 audio files were given for training with 908 subsets and 10 audio files for testing with ## observations the result contains the name of the speech data with its wer value. similarly, for all the audio files the same process is carried out. the table also includes the details about the number of subsets that each audio file is divided into. in in ## conclusion in order to improve the speech recognition system for recognizing the elderly people conversational speech data. an automatic speech recognition system is designed with a pre-trained model. a dataset is collected from the elderly people and transgender whose native language is tamil. the utterance of the dataset is a tamil language and recorded during a conversation in primary locations. as the pre-trained model used for the system is fine-tuned with common voice dataset, in future the model can trained with our own dateset and it can be used for testing, which can increase the performance.