Revisiting Automatic Speech Recognition for T amil and H indi Connected Number Recognition (2023)

Abstract: Automatic Speech Recognition and its applications are rising in popularity across applications with reasonable inference results. Recent state-of-the-art approaches, often employ significantly large-scale models to show high accuracy for ASR as a whole but often do not consider detailed analysis of performance across low-resource languages applications. In this preliminary work, we propose to revisit ASR in the context of Connected Number Recognition (CNR). More specifically, we (i) present a new dataset HCNR collected to understand various errors of ASR models for CNR, (ii) establish preliminary benchmark and baseline model for CNR, (iii) explore error mitigation strategies and their after-effects on CNR. In the due process, we also compare with end-to-end large scale ASR models for reference, to show its effectiveness.

## introduction automatic speech recognition is a wide variety with a majority of them claiming that these speech-to-text systems are able to deliver high accuracy on some of the well-established benchmarks @xcite . connected number recognition (cnr) is a subproblem of asr that focuses on recognizing spoken numbers that are connected in a continuous sequence. for example, tol . l . āyirattu aintu is a cnr speech sample representing number 905. extracting such numbers from the speech is helpful in multiple applications @xcite ranging from assisting senior citizens to make online purchases to simplification of complex banking functions. recently there are many works that further augment asr systems into low-resource languages such as tamil @xcite . the motivation for this research work lies in the use of ˚corresponding author asr techniques in the sectors (like finance etc.) where the frequency of speaking connected numbers and the importance of each utterance is very high. however many of these methods largely focus on reporting exclusively the overall word error rate (wer) of the whole without discerning application-specific results and corner cases. these results are not transferable across the subset of asr applications with different human perceptions @xcite . as such to advance applications of the asr further there is a need to understand the impact of input in realistic application settings on the final asr output across specific applications. moreover, such analysis of errors, will in turn help introduce better post-editing mechanisms that are dynamic and selectable for specific inputs, leading to improved effectiveness of such systems. specifically, benchmarking of cnr will economic progress by enabling technology for people across the spectrum. thus, in this work, we analyze asr systems for the problem of cnr in tamil and hindi languages. specifically, this work focuses on benchmarking asr systems for cnr in tamil, and hindi and the impact of input data-related errors on the final performance of cnr. for the former case, we study the performance of 4 different models that currently exist for asr. accordingly, we find that all the existing models show significant performance degradation for cnr in tamil and hindi. in the latter case, we find very few works to focus on input data-related errors of asr systems, with a majority of them concentrating on the english language and establishing wer and few of them on other languages @xcite , but not from the point of cnr. overall the contributions of this paper are as follows. the rest of the paper is organized as follows. in section 2, we present the existing literature, followed by 3 showing hcnr dataset used in this work. meanwhile in section 4, we discuss various methods, followed by results and key findings in section 5. we conclude with implications on future work in section 6. number recognition using speech samples, often limited to recognizing single digits with shallow analysis on few samples. notable of these include @xcite which identifies digits spoken in bangladesh, @xcite investigated the recognition of arabic digits from the speech signals using artificial neural network and attempts of @xcite , krishnamurthy and prasanna (2017) and @xcite for languages of hindi, malayalam, gujarati respectively. in this work, we focus on estab-lishing a comprehensive benchmark for connected number recognition using @xcite and @xcite . datasets often used to train these models are trained on large, clean, and very generic. few of the notable datasets for indian languages include @xcite for hindi, @xcite for bengali, @xcite for tamil, @xcite for emotion-based speech recognition and accented speech data by @xcite . however to date, there aren't any large datasets specifically developed for connected numbers, accordingly in this work, we create new dataset catering to cnr in tamil and hindi. the overall hcnr dataset characteristics are as shown in the dataset was separated into the train, validation, and test splits as shown in from the histograms, we can see that for the hindi dataset, around 45% of the total speech samples are having snr less than 40 db while for the tamil dataset, around 57% of the total speech samples are having snr less than 40 db. ## experimental setup our experimental setup is split into two parts which aim to establish a strong baseline for the problem of connected number recognition. more specifically, we assess wer for the baseline methods that are not end-to-end asrs, rather widely used hybrid deep learning lstm-tdnn model. following this, we understand various errors and relate them to the characteristic of the dataset in turn highlighting the strength and weaknesses of the said baseline methods. following this, we explore certain mitigation strategies to further ground the method so selected as a strong baseline candidate for connected number recognition. throughout this work, we employ word error rate (wer) and sentence error rate (ser) inline with @xcite . ## conclusion and future work overall in this work, we study the problem of cnr by creating a new hcnr dataset and report baseline results with the lt-kaldi model across tamil and hindi languages. in the process, we find that the baseline lt-kaldi shows wer of around 15% and 7% respectively across the languages. in the due process, we conjectured the sample characteristics might be the key reason leading to higher wer through analysis, for which we studied spectral gating, spectral subtraction, and diarization methods for further improvement. however, we could see that the overall results improved only by 2% for tamil and 1% for hindi cnr. most importantly, we could also see that compared to lt-kaldi, the pretrained models performed significantly worse, unlike prior works. however, we think this may be attributed to the case of outof-domain samples, needing further studies. in this regard, a possible question to explore includes evaluating the effect of training the e2e model with hcnr and the effect of in-domain data on cnr performance. additionally, we plan to explore other methods to remove negative sample characteristics and study their impact on overall cnr results.