Overview of the Third Shared Task on Speech Recognition for Vulnerable Individuals in T amil (2024)

Abstract: The overview of the shared task on speech recognition for vulnerable individuals in Tamil (LT-EDI-2024) is described in this paper. The work comes with a Tamil dataset that was gath- ered from elderly individuals who identify as male, female, or transgender. The audio sam- ples were taken in public places such as marketplaces, vegetable shops, hospitals, etc. The training phase and the testing phase are when the dataset is made available. The task required of the participants was to handle audio signals using various models and techniques, and then turn in their results as transcriptions of the pro- vided test samples. The participant’s results were assessed using WER (Word Error Rate). The transformer-based approach was employed by the participants to achieve automatic voice recognition. This overview paper discusses the findings and various pre-trained transformer- based models that the participants employed.

## introduction the earliest known examples of old tamil writing are tiny inscriptions found in adichanallur that date between 905 and 696 bc. of all the indian languages, tamil possesses the most ancient non-sanskritic literature. the grammar of tamil is agglutinative, meaning that noun class, number, case, verb tense, and other grammatical categories are indicated by suffixes. unlike other aryan languages, which use sanskrit as their standard language, tamil uses tamil for both its scholarly vocabulary and its metalinguistic terminology. together with dialects, tamil has multiple forms: cankattami, the classical literary style based on the ancient language; centami, the modern literary and formal style; and kotuntami, the present vernacular form. @xcite . there is a stylistic continuity created by these styles merging together. for instance, one may write centami using cankattami vocabulary, or one could speak kotuntami while using forms related to one of the other types. @xcite . a lexical root plus one or more affixes combine to form tamil words. suffixes make up the bulk of affixes in tamil. tamil suffixes fall into two groups: derivational suffixes, which change a word's meaning or part of speech, and inflectional suffixes, which identify certain categories like person, number, mood, tense, and so on. agglutination can lead to huge words with multiple suffixes, needing numerous words or a phrase in english. its length and scope are infinite. although smart technologies have come a long way, humanmachine interaction is still being developed and enhanced. @xcite . automatic speech recognition (asr) is one such recent technology that has enabled voice-based user interfaces for numerous automated systems. many elderly and transgender people are frequently unaware of the technology @xcite that is made available to help people in public places like banks, hospitals, and administrative offices. thus, communication is the only kind of media that can assist people in getting what they want. however, these asr systems are infrequently used by the elderly, transsexuals, and others with lower levels of education. english-language voice-based interfaces are a feature of most automated systems currently in use. elderly people and those living in rural areas prefer to speak in their native tongue. the provision of speech interfaces in the local language for help systems designed for public usage would be advantageous to all. information regarding spontaneous speech in tamil is gathered from transgender and elderly people who are not able to use these programs. the aim of this challenge is to find an efficient asr model to handle the elderly person's speech corpus. the pertinent features will first be extracted from the speech signal using an asr system. acoustic models will also be produced using these features that were retrieved. ultimately, the language model assists in converting these probabilities into grammatical words. the language model uses statistics from training data to assign probabilities to words and phrases @xcite . it is necessary to evaluate asr systems' performance prior to deploying them in real-time applications. on large-scale automatic speech recognition (asr) tasks, an end-to-end speech recognition system has shown promising performance, matching or surpassing that of traditional hybrid systems. using an acoustic model, lexicon, and language model, the end-to-end system quickly transforms audio data into tag labels @xcite . in the field of end-to-end voice recognition, there exist two extensively utilized frameworks. frame synchronous prediction separates one input frame from the other by giving each one a target label @xcite @xcite . phoneme identification can also be used to assess the efficacy using different test feature vectors and model settings. the use of acoustic models for speech recognition, which are created using the sounds of younger people, may have a substantial impact on the capacity to recognize elder speech @xcite @xcite . there aren't many acoustic models that can handle the voice detection task. among the acoustic models are japanese newspaper article sentences (jnas), japanese newspaper article sentences read speech corpus of the aged (s-jnas), and corpus of spontaneous japanes (csj). the csj model only achieves the lowest wer once the older voices are adjusted, according to a comparison of all the acoustic models in the literature @xcite . dialect adaptation is also required in order to improve recognition accuracy @xcite . recent advances in large vocabulary continuous speech recognition (lvcsr) technologies have led to the widespread use of speech recognition systems in several fields @xcite . variations in the acoustics of individual speakers are thought to be one of the primary causes of the decline in speech recognition rates. for elder speakers to use speech recognition systems trained on typical adult speech data, the acoustic discrepancies between their speech and that of an adult should be investigated and correctly adjusted. rather, this loss can be mitigated by an acoustic model enhanced by senior speakers' utterances, as shown by a document retrieval system. modern voice recognition technology can reach excellent recognition accuracy while speaking while reading a written text or something comparable; nevertheless, the accuracy decreases when speaking spontaneously and freely. the main reason for this issue is that the linguistic and acoustic models used in voice recognition were mostly developed using read-aloud or written language materials. however, there are significant linguistic and auditory differences between written language and spontaneous speech @xcite . currently, it is becoming more and more popular to create asr systems that can detect voice data from older persons. the aging population in modern society and the proliferation of smart devices, which make information freely accessible to both the young and the old, have led to a demand for improved voice recognition in smart devices @xcite @xcite . because of the influences of speech articulation and speaking style, speech recognition systems are often optimized for the voice of an average adult and have a lower accuracy rate when recognising the voice of an elderly person. it will surely become more expensive to adapt the current voice recognition systems to handle the speech of elderly users @xcite . ## related work when a model is fine-tuned on many languages at the same time, a single multilingual speech recognition model can be built that can compete with models that are fine-tuned on individual language speech corpus. speech2vec expands the text-based word2vec model to learn word embeddings directly from speech by combining an rnn encoder-decoder framework with skipgrams or cbow for training. acoustic models are designed at the phoneme/syllable level to carry out the speech recognition task. initially, the acoustic models were created with jnas, s-jnas and csj speech corpus @xcite . later, the models were trained/fine-tuned with different speech corpus. to get a better performance and accuracy, backpropagation using transfer learning was attempted in the literature. similar work was performed for other languages like bengali, japanese, etc. also, more speech corpus is collected from young people for many languages @xcite . however, speaker fluctuation, environmental noise, and transmission channel noise all degrade asr performance. as the shared task is given with a separate training data set, an effective model has to be created during the training. therefore, the hierarchical transformer-based model for large context end-to-end asr can be used @xcite . in the recent era, the environment is changing with smart systems and is identified that there is a need for asr systems that are capable of handling the speech of elderly people spoken in their native languages. to overcome this problem, the shared task is proposed for the research community to build an efficient model for recognizing the speech of elderly people and transgenders in tamil language. findings of the automatic speech recognition for vulnerable individuals are given in @xcite ("s and b, "2023") @xcite , have used transformer models used for transformer-based asr for vulnerable individuals in tamil. ## data-set description the dataset given to this shared task @xcite is an tamil conversational speech recorded from the elderly people whose average age is around 61 for male, 59 for female and 30 for transgender people which are tabulated in ## methodology the methodology used by the participants in the shared task of speech recognition for vulnerable individuals in tamil is discussed in this section. three teams submitted their runs for this task. all three teams have used the pre-trained models. the first team "cen_amrita" has used the whisper model, whisper is a pre-trained automatic speech recognition (asr) model trained on 680,000 hours of multilingual and multitask supervised data sourced from the web. this end-to-end transformer-based model adopts the encoder-decoder architecture.. the second team "asr_tamil_ssn" have used the transformer based model called 'akashsivanandan/wav2vec2large-xls-r-300m-tamil-colab-final'. the third team " have also used the transformer based pretrained model called 'rajaram1996/wav2vec2-large-xlsr-53-tamil'. ## evaluation of results the results submitted by the participants are evaluated based on the wer computed between the asr hypotheses submitted by the participants and the ground truth of human speech transcription. wer ( word error rate) = ( s + d + i) / n where, @xmath0. of substitutions @xmath1. of deletions @xmath2. of insertions @xmath3. of words in the reference transcription as discussed in the methodology, different average word error rates are measured using various pre-trained transformer-based models. the participating team's wer are shown in table . 2. ## conclusions the shared challenge for vulnerable voice recognition in tamil is covered in this overview paper. the speech corpus shared for this job was recorded from elderly persons. getting older people's speech more accurately recognised is a difficult endeavor. in order to boost the accuracy and performance in recognising elderly people's speech, the participants have been given access to the gathered speech corpus. there were a total of seven teams participated in this joint task and turned in their transcripts of the supplied data. the team estimated the wer and then compared the outcome to the human transcripts. three teams built their recognition systems using various whisper models and transformer-based models. finally, the word error rates of the three participants are 24.452, 29.297, 37.7333 respectively. based on the observations, it is suggested that the transformer-based model and whisper model can be trained with given speech corpus which could give better accuracy than the pre-trained model, as the transformer-based model and whisper model used are trained with a common voice dataset. also, a separate language model can also be created for this corpus.