Prosody Labelled Dataset for H indi (2021)

Abstract: This study aims to develop an intonation labelled database for Hindi, for enhancing prosody in ASR and TTS systems, which is also helpful for building Speech to Speech Machine Translation systems. Although no single standard for prosody labelling exists in Hindi, researchers in the past have employed perceptual and statistical methods in literature to draw inferences about the behaviour of prosody patterns in Hindi. Based on such existing research and largely agreed upon intonational theories in Hindi, this study attempts to develop a manually annotated prosodic corpus of Hindi speech data, which can be used for training speech models for natural-sounding speech in the future. 500 sentences (2,550 words) for declarative and interrogative types have been labelled using Praat.

## introduction in order to produce natural sounding speech units, many automatic speech recognition (asr) and text-to-speech (tts) systems incorporate suprasegmental prosodic features, which generally apply to larger units of representation like phrases or the sentence. some of the intonational aspects that are covered through prosody include pitch accent, phrasing, duration etc. spoken in natural rhythm, sentences constitute grammatical breaks and accents which lend specific intonational contours to different sentence types. in general, words may contain lexical stress according to grammatical rules (some words in many languages are not stressed at all), at other times, words may be stressed to convey focus. when strung together, sentences spoken naturally, and impacted by extraneous factors such as speaker motivation, mood, speed etc serve to modify prosodic structure of spoken speech in ways that render it natural sounding to human perception. it is therefore important to be able to input these features, along with the orthography to phonemic conversions, into tts systems, in order to emulate human-like intelligible voices in building speech to speech machine translation (ssmt) systems. section 2 discusses previous studies in hindi intonation, with a focused perspective on the development of theories on pitch accent and phrase breaks in hindi sentences. the theories discussed in this section form the basis for the linguistic analysis and annotation of the declarative and interrogative sentences, discussed in later chapters. section 3 talks about the speech resource used in the building of this dataset, with a bried overview of the labelling framework discussed in section 4. sections 5 and 6 discuss the manual and automatic approaches used in the development of this dataset, with the results. ## background hindi belongs to the indo-european language family and has over 500 million speakers in india. a number of studies exist in literature on hindi intonation. one of the most pioneer of works was by @xcite , who analyzed hindi intonation in terms of three different segmental levels in hierarchical relation to each other: foot, measure and sentence. according to his theory, foot consists of one or more syllables in which pitch rises from beginning to end continuously. measure is the second level of phrasing in which a focused element is separated from the rest of the sentence. sentence is the topmost level, which encompasses the entire sentence intonation. @xcite makes an observation along similar lines in which he states that there is a rising pitch contour on content words, in which the low part of the rising contour is a low pitch accent and the high part is either a high trailing tone or boundary tone. the other level of phrasing is the sentence. @xcite talk about the rising pitch contour on each constituent of the data that they have considered and call it the prosodic phrase and relate it to the syntax of the sentence. @xcite , @xcite suggest, in their work, that hindi has lexical stress, such that every word has a particular syllable on which prominence is realized. @xcite , from their investigative studies, put forward the theory that hindi is an accentual phrase language and that the accentual phrase (ap) was the smallest tonally marked prosodic unit, characterised by a rising contour, the observation being similar to that proposed for bangladeshi standard bengali @xcite , a closely related indo european language. their research hypothesized that the intonation pattern of hindi sentences contained a series of aps, characterised by rising contours (which correlate to pitch patterns within the ap) and that the domain of each ap is marked by prosodic boundaries, which may or may not be equal to a single word. the final tone can be overridden by a falling tone in case of declaratives. the entire sentence constituted an ip (intonational phrase) comprised of many ip (intermediate phrase), characterized by silence junctures, and each ip contains one or more aps. @xcite , through exploratory investigation using non-expert and expert transcribers, concluded that prosodic phrasing was more consistently agreed on between non-expert transcribers amongst themselves and with the expert transcribers (measured by cohen's kappa coefficient). it was also observed that the degree of agreement in prominence (pitch accent) marking was lower, in both cases. ## automatic prosodic labelling the manually labelled sentences developed in the previous section has been used as training data to fine-tune au-tobi, an existing automatic prosody labelling toolkit widely available (rosenberg, 2010), by building newly trained models within their standard specifications. the study is a comparative analysis on the performance of accuracy between pre-existing and newly trained au-tobi models for this research. au-tobi was particularly selected for its adaptability to tobi, which had been used as the labelling conventions for the manually annotated data as well.