A Systematic Exploration of Linguistic Phenomena in Spoken H indi: Resource Creation and Hypothesis Testing (2024)

Abstract: This paper presents a meticulous and well-structured approach to annotating a corpus of Hindi spoken data. We deployed 4 annotators to augment the spoken section of the EMILLE Hindi corpus by marking the various linguistic phenomena observed in spoken data. Then we analyzed various phonological (sound deletion), morphological (code-mixing and reduplication) and syntactic phenomena (case markers and ambiguity), not attested in written data. Code mixing and switching and constitute the majority of the phenomena we annotated, followed by orthographic errors related to symbols in the Devanagiri script. In terms of divergences from written form of Hindi, case marker usage, missing auxiliary verbs and agreement patterns are markedly distinct for spoken Hindi. The annotators also assigned a quality rating to each sentence in the corpus. Our analysis of the quality ratings revealed that most of the sentences in the spoken data corpus are of moderate to high quality. Female speakers produced a greater percentage of high quality sentences compared to their male counterparts. While previous efforts in corpus annotation have been largely focused on creating resources for engineering applications, we illustrate the utility of our dataset for scientific hypothesis testing. Inspired from the Surprisal Theory of language comprehension, we validate the hypothesis that sentences with high values of lexical surprisal are rated low in terms of quality by native speakers, even when controlling for sentence length and word frequencies in a sentence.

## introduction the availability of high-quality linguistic resources plays a pivotal role in the field of computational linguistics from both theoretical and application-oriented perspectives. hindi (indo-aryan language, indo-european language family) is considered a medium-resource language primarily spoken in the indian subcontinent. in the context of spoken language resources, the following datasets (inter-alia) can be considered as pioneering efforts in corpus creation: simulated emotion hindi speech corpus @xcite , indic speech database @xcite , ldc-il hindi raw speech corpus @xcite , and hindi-urdu treebank corpus @xcite , consisting of both written and spoken data. however, except for hutb, the majority of dataset developments in the indian context have predominantly focused on engineering applications with relatively less emphasis placed on scientific hypothesis testing and language processing research. in this work, we present our preliminary efforts to annotate the hindi spoken section within the publicly available emille corpus @xcite , with information pertaining to various linguistic phenomena. the key objectives of our corpus annotation project are summarized below: ## experiments this section provides an analysis of the quality ratings and a summary of various linguistic phenomena pertaining to phonology, morphology, and syntax that were observed and annotated by four native hindi speakers following our annotation manual. each linguistic annotation, along with their frequency, is presented in ## discussion and conclusions we present a hindi speech corpus created by augmenting the emille spoken hindi corpus using linguistically motivated annotations and sentence quality ratings obtained by deploying 4 annotators. code mixing and switching constitute the majority of the phenomena we annotated, followed by orthographic errors related to symbols in the devanagiri script. in terms of divergences from written form of hindi, case marker usage, missing auxiliary verbs and agreement patterns are markedly distinct for spoken hindi. our analysis of the quality ratings assigned by the annotators revealed that most of the sentences in the spoken data corpus are of moderate to high quality. female speakers produced a higher percentage of high-quality sentences compared to their male counterparts. our conjectures is that this is possibly due to their tendency to adhere to the prescriptive norms of the language very closely. our augmented hindi spoken corpus can be used for both engineering applications as well as for scientific hypothesis testing, in contrast to existing corpora which are exclusively oriented towards engineering applications. using this dataset, we validated our hypothesis that lexical surprisal is a significant predictor of quality ratings by humans. sentences with higher lexical surprisal are associated with lower values of quality ratings compared to their lower surprisal counterparts. sentences with longer lengths and more frequent words tend to reflect higher quality. our findings directly align with prior corpus-based sentence literature, which attests to the correlation between lexical surprisal estimates and human preference judgments in word order choices @xcite . lexical predictability effects in silent reading are known to occur at the initial stages of word processing, where readers activate a set of plausible upcoming words in the given context. predictability effects are also associated with the early prelexical stage in the lexical access process, pertaining to the visual properties of the script @xcite . a key limitation of our work is that we work on transcribed speech, and thus miss out on the acoustic features associated with spoken data. in future work, we plan to overcome this lacuna by creating a hindi corpus of transcribed speech data using the pipeline developed by @xcite . we intend to create 500 hours of speech with annotations of linguistic phenomena discussed in this work as well as disfluency annotation for two levels of annotation for each recording: 1. transcript level: marking disfluencies in the transcript obtained from automatic speech recognition for audio recording 2. signal level: the start and end times of each disfluency annotated at the transcript level will be noted. annotators will be employed to manually annotate all the audio files for 5 types of disfluencies such as filled pauses, prolongation, part-word repetition, word repetition and phrase repetition and assign a confidence score to the annotation. in future research, we plan to explore the impact of each type of linguistic phenomenon (viz., phonological, morphological, and syntactic discussed in the paper) on human preference ratings. we also plan to employ more annotators, compute the inter-annotator agreement, and develop a platinum-standard resource for both engineering applications and scientific inquiries. we also envisage a systematic comparison of text and speech modalities of language use in terms of their cognitive properties.