Overview of the Second Shared Task on Speech Recognition for Vulnerable Individuals in T amil (2023)

Abstract: This paper manifest the overview of the shared task on Speech Recognition for Vulnerable individuals in Tamil(LT-EDI-ACL2023). Task is provided with an Tamil dataset, which is collected from elderly people of three different genders, male, female and transgender. The audio samples were recorded from the public locations like hospitals, markets, vegetable shop, etc. The dataset is released in two phase, training and testing phase. The partcipants were asked to use different models and methods to handle audio signals and submit the result as transcription of the test samples given. The result submitted by the participants was evaluated using WER (Word Error Rate). The participants used the transformer-based model for automatic speech recognition. The results and different pre-trained transformer based models used by the participants is discussed in this overview paper.

## introduction the earliest old tamil documents are small inscriptions in adichanallur dating from 905 bc to 696 bc. tamil has the oldest ancient non-sanskritic indian literature of any indian language. tamil uses agglutinative grammar, which uses suffixes to indicate noun class, number, case, verb tense, and other grammatical categories. tamil's standard metalinguistic terminology and scholarly vocabulary is itself tamil, as opposed to the sanskrit that is standard for most aryan languages. tamil has many forms, in addition to dialects: a classical literary style based on the ancient language (cankattami), a modern literary and formal style (centami), and a current colloquial form (kotuntami) @xcite . these styles blend into one another, creating a stylistic continuity. it is conceivable, for example, to write centami using cankattami vocabulary, or to utilize forms con-nected with one of the other varieties while speaking kotuntami @xcite . tamil words are made up of a lexical root and one or more affixes. the majority of tamil affixes are suffixes. tamil suffixes are either derivational suffixes, which modify the part of speech or meaning of the word, or inflectional suffixes, which designate categories like as person, number, mood, tense, and so on. there is no ultimate limit to the length and scope of agglutination, which might result in large words with several suffixes, requiring many words or a sentence in english. smart technologies have advanced significantly, and they are still developing and improving human-machine connection @xcite . one such modern technology is automatic speech recognition (asr), which has made it possible for many automated systems to have voice-based user interfaces. the technology that are facilitated to assist individuals @xcite in public spaces like banks, hospitals, and administrative offices are often unknown to many elderly and transgender persons. therefore, the only media that could help them meet their demands is communication. however, the elderly, transsexual, and less educated persons rarely use these asr systems. the majority of the automated systems in use today include voice-based interfaces that are available in english. people in rural areas and the elderly prefer to communicate in their own language. all people would benefit if the assistance systems created for use in public spaces could be equipped with speech interfaces in the local tongue. the data on spontaneous speech in tamil is collected from elderly and transgender individuals who are deprived of the opportunity to take benefit of these services.finding an effective asr model to handle the elderly persons speech corpus is the goal of this task. the representation of how the audio samples are collected is shown in fig:1 an asr system will initially extract the relevant features from speech signal. these extracted features will also be used to generate acoustic models. finally, the language model helps to turn these probabilities into words of coherent language. the language model, assigns probabilities to words and phrases based on statistics from training data @xcite . before using asr systems in realtime applications, their performance must be assessed. an end-to-end speech recognition system has demonstrated promising performance on large-scale automatic speech recognition (asr) tasks, placing it on par with conventional hybrid systems. the end-to-end system converts acoustic data into tag labels instantly using an acoustic model, lexicon, and language model @xcite . there are two widely used frameworks in the area of end-to-end speech recognition. frame synchronous prediction, which assigns one target label to each input frame, distinguishes one from the other @xcite @xcite . with alternative test feature vectors and model settings, the effectiveness can also be evaluated in terms of phoneme recognition. the ability to recognise senior speech may be significantly influenced by the use of acoustic models for speech recognition, which are produced using the voices of younger persons @xcite @xcite . few acoustic models have been developed to perform the voice recognition problem. japanese newspaper article sentences (jnas), japanese newspaper article sentences read speech corpus of the aged (s-jnas), and corpus of spontaneous japanes (csj) are a few examples of the acoustic models. all of the acoustic models are compared in the literature, and it is discovered that the csj model only obtains the lowest wer after the adaption of the elderly voices @xcite . the same goes for dialect adaptation, which is necessary to increase recognition accuracy @xcite . speech recognition systems are now widely used in a range of fields as a result of recent advancements in large vocabulary continuous speech recognition (lvcsr) technologies @xcite . one of the main reasons for the fall in speech recognition rates is assumed to be variances in the acoustics of different speakers. the acoustic differences between the speech of senior speakers and those of a typical adult should be examined and appropriately adjusted in order for older speakers to use speech recognition systems trained on normal adult speech data. instead, as demonstrated by a document retrieval system, an acoustic model improved using utterances of senior speakers can lessen this degradation. using cutting-edge voice recognition technology, high recognition accuracy can be achieved for speech reading a written text or anything similar; nevertheless, the accuracy declines for freely uttered spontaneous speech. the primary cause of this problem is that read speech or written language texts were predominantly used in the development of the acoustic and linguistic models used in speech recognition. however, both linguistically and acoustically, spontaneous speech and written language differ greatly @xcite . creating asr systems to recognise elderly people's voice data is becoming increasingly commonplace today. the need to improve voice recognition in smart devices has arisen as a result of the ageing population in contemporary society and the expansion of smart gadgets, allowing both the elderly and the younger generations to easily access information @xcite @xcite . speech recognition systems are frequently optimised for an average adult's voice and have a reduced accuracy rate when recognising an elderly person's voice due to the effects of speech articulation and speaking style. the cost of modifying the already existing voice recognition systems to handle the speech of older users will undoubtedly increases @xcite . ## task description this shared task tackles a difficult problem in automatic speech recognition: vulnerable elderly and transgender individuals in tamil. people in their senior years go to primary places such as banks, hospitals, and administrative offices to meet their daily needs. many elderly persons are unsure of how to use the devices provided to assist them. similarly, because transgender persons are denied access to primary education as a result of societal discrimination, speech is the only channel via which they may meet their needs. the data on spontaneous speech is collected from elderly and transgender people who are unable to take advantage of these services. for the training set, a speech corpus containing 5.5 hours of transcribed speech will be released, as well as 2 hours of speech data for testing test. the participants have to submit the text transcriptions for the test utterances in a separate text file. ## related work when a model is fine-tuned on many languages at the same time, a single multilingual speech recognition model can be built that can compete with models that are fine-tuned on individual language speech corpus. speech2vec expands the text-based word2vec model to learn word embeddings directly from speech by combining an rnn encoder-decoder framework with skipgrams or cbow for training. acoustic models are designed at phoneme/syllable level to carry out the speech recognition task. initially, the acoustic models were created with jnas, s-jnas and csj speech corpus @xcite . later, the models were trained/fine-tuned with different speech corpus. to get a better performance and accuracy, backpropagation using the transfer learning was attempted in the literature. similar work was performed for other languages like bengali, japanese, etc. also, more speech corpus is collected from the young people for many languages @xcite . however, speaker fluctuation, environmental noise, and transmission channel noise all degrade asr performance. as the shared task is given with a separate training data set, an effective model has to be created during the training. therefore, hierarchical transformer based model for large context end to end asr can be used @xcite . in the recent era, the environment is changing with smart systems and is identified that there is a need for asr systems that are capable of handling speech of elderly people spoken in their native languages. to overcome this problem, the shared task is proposed for the research community to build an efficient model for recognizing the speech of elderly people and transgenders in tamil language. findings of the automatic speech recognition for vulnerable individuals are given in @xcite , have used transformer models used for transformer based asr for vulnerable individuals in tamil. ## data-set description the dataset given to this shared task @xcite is an tamil conversational speech recorded from the elderly people whose average age is around 61 for male, 59 for female and 30 for transgender people which are tabulated in ## methodology the methodology used by the participants in shared task of speech recognition for vulnerable individuals in tamil is discussed in this section. three teams submitted their runs for this task. different types of pre-trained transformer models used by the participants in this shared task are as follows: the above mentioned second and third models are fine tuned on facebook/wav2vec-large-xlsr-53 foot_3 pre-trained model using multilingual common ## evaluation of results the results submitted by the participants are evaluated based on the wer computed between the asr hypotheses submitted by the participants and the ground truth of human speech transcription. wer ( word error rate) = ( s + d + i) / n where, @xmath0. of substitutions @xmath1. of deletions @xmath2. of insertions @xmath3. of words in the reference transcription as discussed in the methodology, different average word error rate are measured using various pre-trained transformer based models. performance of the asr submitted by the participants are tabulated in ## conclusion the shared challenge for vulnerable voice recognition in tamil is covered in this overview paper. the speech corpus shared for this job was recorded from elderly persons. getting older people's speech more accurately recognised is a difficult endeavour. in order to boost the accuracy and performance in recognising the elderly people's speech, the participants have been given access to the gathered speech corpus. there were two people that participated in this joint task and turned in their transcripts of the supplied data. the team estimated the wer and then compared the outcome to the human transcripts. both participants built their recognition systems using various transformer-based models. finally, the word error rates of the three participants are 37.7144, 39.8091 & 40.7462 respectively. based on the observations, it is suggested that the transformer based model can be trained with given speech corpus which could give a better accuracy than the pre-trained model, as the transformer based model used are trained with common voice dataset. also, a separate language model can also be created for this corpus.