Findings of the Shared Task on Speech Recognition for Vulnerable Individuals in T amil (2022)

Abstract: This paper illustrates the overview of the sharedtask on automatic speech recognition in the Tamillanguage. In the shared task, spontaneousTamil speech data gathered from elderly andtransgender people was given for recognitionand evaluation. These utterances were collected from people when they communicatedin the public locations such as hospitals, markets, vegetable shop, etc. The speech corpusincludes utterances of male, female, and transgender and was split into training and testingdata. The given task was evaluated using WER(Word Error Rate). The participants used thetransformer-based model for automatic speechrecognition. Different results using differentpre-trained transformer models are discussedin this overview paper.

## introduction there have been tremendous developments in smart technologies that continue to evolve and enhance human-machine interaction @xcite @xcite @xcite . one such recent technology is automatic speech recognition(asr) which has paved the way to a lot of voiced-based interfaces to many automated systems. many elderly and transgender people are unaware of of the technologies available that are facilitated to aid people in public places such as banks, hospitals and administrative offices. hence, speech is the only medium that could assist them in satisfying their needs @xcite . however, the usage of these asr systems by the elderly, transgender and less educated people are limited. the reason is most of the existing automated systems are enabled with voiced-based interfaces that are in english language. old aged and people in rural areas only feel comfortable to interact in their regional language. if the systems developed to aid people in public places are enabled with speech interfaces in the regional language, the aiding systems are benefited by all people. the spontaneous speech data in tamil language is gathered from old-aged and transgender people, who are bereft of using these facilities to their advantage. this task is organized to find an efficient asr model to handle the elderly people speech corpus. the speech corpus creation is represented in fig. 1 . the earliest old tamil documents are small inscriptions in adichanallur dating from 905 bc to 696 bc. tamil has the oldest ancient non-sanskritic indian literature of any indian language. tamil uses agglutinative grammar, which uses suffixes to indicate noun class, number, case, verb tense, and other grammatical categories. tamil's standard metalinguistic terminology and scholarly vocabulary is itself tamil, as opposed to the sanskrit that is standard for most aryan languages. tamil has many forms, in addition to dialects: a classical literary style based on the ancient language (cankattami), a modern literary and formal style (centami), and a current colloquial form (kotuntami) @xcite @xcite @xcite . these styles blend into one another, creating a stylistic continuity. it is conceivable, for example, to write centami using cankattami vocabulary, or to utilize forms connected with one of the other varieties while speaking kotuntami @xcite @xcite . tamil words are made up of a lexical root and one or more affixes. the majority of tamil affixes are suffixes. tamil suffixes are either derivational suffixes, which modify the part of speech or meaning of the word, or inflectional suffixes, which designate categories like as person, number, mood, tense, and so on. there is no ultimate limit to the length and scope of agglutination, which might result in large words with several suffixes, requiring many words or a sentence in english @xcite . initially, an asr system will extract the features from speech signals. further,acoustic models will be created with the features. finally, the language model will be created which captures the linguistic information from the text @xcite . the performance of the asr systems has to be evaluated for it to be used in real time applications. on large scale automatic speech recognition (asr) tasks, an end-to-end speech recognition system has showed promising performance, making it competitive with traditional hybrid systems. the end-toend system includes an acoustic model, lexicon, and language model that turns acoustic data into tag labels immediately @xcite . in the field of end-to-end speech recognition, two common frameworks are used. one is distinguished by frame synchronous prediction, which means that one target label is assigned to each input frame @xcite @xcite . the performance can also be measured in terms of phoneme recognition with different test feature vectors and different model parameters. the use of acoustic models for speech recognition, which are created using the voices of younger adults, may be a significant factor in the recognition of elderly speech @xcite @xcite . there are few acoustic models created to carry out the speech recognition task. some of the acoustic models are japanese newspaper article sentences (jnas), japanese newspaper article sentences read speech corpus of the aged (s-jnas) and corpus of spontaneous japanes (csj). in the literature, all the acoustic models are compared and found that the csj model achieves the lowest wer only after the adaptation of the elderly voices @xcite . similarly, dialect adaptation is also required so as to improve recognition accuracy @xcite . due to recent developments in large vocabulary continuous speech recognition (lvcsr) technologies, speech recognition systems have become widely used in a variety of fields @xcite . acoustic differences between speakers are thought to be one of the primary causes of the decline in speech recognition rates. for elderly speakers to use speech recognition systems trained using normal adult speech data, the acoustic difference between the speech of elderly speaker and that of a typical adult should be analyzed and adapted accordingly. instead, an acoustic model trained on the utterances of elderly speakers can reduce this degradation, as confirmed by a document retrieval system. high recognition accuracy can be obtained for speech reading a written text or similar by using cutting-edge speech recognition technology; however, the accuracy degrades for freely spoken spontaneous speech. the main reason for this issue is that acoustic and linguistic models used in speech recognition have been developed primarily using written language text or read speech. however, spontaneous speech and written language differ significantly both acoustically and linguistically @xcite . nowadays, developing asr systems recognizing elderly people speech data has became more common. due to the ageing population in modern society and the growth of smart devices, there is a need to improve speech recognition in smart devices so that information can be freely accessible to the elderly as well as the younger people @xcite @xcite . due to the impacts of speech articulation and speaking style, speech recognition systems are often optimised for an average adult's voice and have a lower accuracy rate when recognising an elderly person's voice. adapting the currently available speech recognition systems for handling the speech of senior users is certain to incur additional costs @xcite . ## task description this shared task tackles a difficult problem in automatic speech recognition: vulnerable elderly and transgender individuals in tamil. people in their senior years go to primary places such as banks, hospitals, and administrative offices to meet their daily needs. many elderly persons are unsure of how to use the devices provided to assist them. similarly, because transgender persons are denied access to primary education as a result of societal discrimination, speech is the only channel via which they may meet their needs. the data on spontaneous speech is collected from elderly and transgender people who are unable to take advantage of these services. for the training set, a speech corpus containing 5.5 hours of transcribed speech will be released, as well as 2 hours of speech data for testing test. ## related work when a model is fine-tuned on many languages at the same time, a single multilingual speech recognition model can be built that can compete with models that are fine-tuned on individual language speech corpus. speech2vec expands the text-based word2vec model to learn word embeddings directly from speech by combining an rnn encoder-decoder framework with skipgrams or cbow for training. acoustic models are designed at phoneme/syllable level to carry out the speech recognition task. initially, the acoustic models were created with jnas, s-jnas and csj speech corpus @xcite . later, the models were trained/fine-tuned with different speech corpus. to get a better performance and accuracy, backpropagation using the transfer learning was attempted in the literature. similar work was performed for other languages like bengali, japanese, etc. also, more speech corpus is collected from the young people for many languages @xcite . however, speaker fluctuation, environmental noise, and transmission channel noise all degrade asr performance. as the shared task is given with a separate training data set, an effective model has to be created during the training. therefore, hierarchical transformer based model for large context end to end asr can be used @xcite . in the recent era, the environment is changing with smart systems and is identified that there is a need for asr systems that are capable of handling speech of elderly people spoken in their native languages. to overcome this problem, the shared task is proposed for the research community to build an efficient model for recognizing the speech of elderly people and transgenders in tamil language. ## data-set description the dataset given to this shared task is an tamil conversational speech recorded from the elderly people whose average age is around 61 for male, 59 for female and 30 for transgender people which are tabulated in ## methodology the methodology used by the participants in shared task of speech recognition for vulnerable individuals in tamil is discussed in this section. different types of pre-trained transformer models used by the participants in this shared task are amrrs/wav2vec2-large-xlsr-53-tamil foot_0 @xcite akashsivanandan/wav2vec2-large-xslr-300mtamil-colab-final foot_1 @xcite nikhil6041/wav2vec2-large-xlsr-tamilcommonvoice foot_2 @xcite rajaram1996/wav2vec-large-xlsr-53-tamil foot_3 @xcite the above mentioned models are fine tuned on facebook/wav2vec-large-xlsr-53 foot_4 pre-trained model using multilingual common voice dataset. to fine-tune the model, they had a classifier representing the downstreams task's output vocabulary on top of it and train it with a connectionist temporal classification (ctc) loss on the labelled data. the models used are based on xlsr wav2vec model, this xlsr model is capable of learning cross-lingual speech data, where the raw speech ## evaluation of results the results submitted by the participants are evaluated based on the wer computed between the asr hypotheses submitted by the participants and the ground truth of human speech transcription. wer ( word error rate) = ( s + d + i) / n where, @xmath0. of substitutions @xmath1. of deletions @xmath2. of insertions @xmath3. of words in the reference transcription as discussed in the methodology, different average word error rate are measured using various pre-trained transformer based models. performance of the asr submitted by the participants are tabulated in ## conclusion this overview paper discusses the shared task for vulnerable speech recognition in tamil, where the speech corpus shared for this task is recorded from the elderly people. recognizing the speech elderly people with better accuracy is a challenging task. therefore, the collected speech corpus has been shared to participants to address the problem with their method to increase the accuracy and performance in recognizing the elderly people speech. totally, there were two participants who took part in this shared task and submitted the results as transcripts of the given data. the team has compared the result with the human transcripts and calculated the wer. both the participants have used different transformer based model for building their recognition systems. finally, the word error rates of the two participants are 39.4512 & 39.6487 respectively. based on the observations, it is suggested that the transformer based model can be trained with given speech corpus which could give a better accuracy than the pre-trained model, as the transformer based model used are trained with common voice dataset. also, a separate language model can also be created for this corpus.