Synthesizing Audio for H indi W ord N et (2018)

Abstract: In this paper, we describe our work on the creation of a voice model using a speech synthesis system for the Hindi Language. We use pre-existing “voices”, use publicly available speech corpora to create a “voice” using the Festival Speech Synthesis System (Black, 1997). Our contribution is two-fold: (1) We scrutinize multiple speech synthesis systems and provide an extensive report on the currently available state-of-the-art systems. We also develop voices using the existing implementations of the aforementioned systems, and (2) We use these voices to generate sample audios for randomly chosen words; manually evaluate the audio generated, and produce audio for all WordNet words using the winner voice model. We also produce audios for the Hindi WordNet Glosses and Example sentences. We describe our efforts to use pre-existing implementations for WaveNet - a model to generate raw audio using neural nets (Oord et al., 2016) and generate speech for Hindi. Our lexicographers perform a manual evaluation of the audio generated using multiple voices. A qualitative and quantitative analysis reveals that the voice model generated by us performs the best with an accuracy of 0.44.

## introduction wordnets have proven to be a rich lexical resource for many nlp sub-tasks such as machine translation (mt) and cross-lingual in-formation retrieval @xcite . they are lexical structures composed of synsets and semantic relations @xcite . such a lexical knowledge base is at the heart of an intelligent information processing system for natural language processing and understanding. the first wordnet was built in english at princeton university foot_0 . then, followed the wordnets for european languages foot_1 @xcite , and then indoword-net foot_2 @xcite . indowordnet consists of 18 indian languages with an average of 27000+ synsets for all the languages and 40000+ for the hindi language. it uses hindi wordnet foot_3 @xcite as a pivot to link all these languages and contains more than 25000 linkages to the princeton wordnet. cognitive theories of multimedia learning @xcite indicate that audio cues are effective aids in a learning scenario, and also help in retaining the material learned @xcite . ## related work a significant amount of work has been done in the area of speech synthesis or text-to-speech conversion for english, japanese, chinese, russian @xcite @xcite @xcite . text-to-speech conversion systems for indian languages have also emerged in the recent past @xcite . although these systems, which are already available, do not produce the most "natural" sounding output, but they are usable to an extent. manual evaluations of the speech synthesis systems built for the hindi language show that there is still a need for better text processing and additional phonetic coverage @xcite . @xcite create an online context sensitive dictionary using princeton wordnet and implement a java based speech interface for the text-to-speech (tts) engine. @xcite automatically collect images for in-dowordnet and augment them to the web interface, but due to the lack of tagged images openly available for use, they do not collect enough images. to the best of our knowledge, there has been no other work specifically in the direction of synthesizing audio for wordnet words or synthesizing audio for indian language wordnets. ## our approach among the three main sub-types of concatenative synthesis, we choose to perform unit selection synthesis and build cluster units of the speech data recorded by a human voice. we use the festival system to create a synthetic voice for hindi. we followed the documentation of the festival framework along with festvox foot_4 implementation to train a voice on hindi speech corpora provided by the in-dictts consortium foot_5 for research purposes. figure 1 displays a generic speech synthesis system which uses the concatenative synthesis or unit selection corpus-based speech synthesis to generate speech given an input text. ## results & evaluation we accumulate 6 usable voice models and produce word audios and randomly sample word audios from them. among these models, the one which we successfully generated using unit selection based concatenative synthesis, sounded most natural in a brief overview. speech synthesis evaluation is a subjective issue. different speech voices are used to train various speech systems, and no agreed upon metric for the quality of such an output has been produced, yet. quality of production technique is another factor on which speech synthesis depends, and hence the evaluation of speech synthesis systems has been compromised by differences between such factors (production techniques, recording facilities etc.) speech synthesis systems require human annotators for evaluation of their output. the annotation is done based on naturalness and intelligibility of the output. a recent work proposes a novel approach that formulates objective intelligibility assessment as an utterance verification problem using hidden markov models, thereby alleviating the need for human reference speech @xcite . although nothing exists to assess the naturalness of a speech synthesis output. we generate word audios for approximately 4000 words using four best voice models. for evaluation of our synthesized data, we create an experiment vaguely based on turing test. we randomly choose 30 hindi words and also get audio recorded for them with the help of our lexicographers. we randomly chose 535 words and generate synthesized outputs from four best models; these outputs were presented to two lexicographers for further analysis. they used the following scale to report the output (i) unusable (#0): this rating corresponds to audio clips which are either distorted, or too noisy for the user to comprehend, (ii) usable (#1): this rating corresponds to audio clips which are moderately usable and suggests that the user can comprehend the underlying words. however, audio clips with this rating can be synthesized better, (iii) good (#2): this rating corresponds to audio clips that are really good and convey the words. for each of the 535 words, the lexicographers were also asked to mark which of the four synthesized clips they liked the most. the evaluation results are shown in a qualitative analysis of the synthesized clips highlighted the following issues, partic-ularly with respect to the clips that were marked "unusable": i) flap or tap sounds (ड़, ढ़) were pronounced incorrectly, ii) intonation of the audio for heavy syllables was at times incorrectly rendered and for words such as 'एकदम', the pronunciation had a specific stress pattern which should have ideally been neutral, thus making it sound unnatural, iii) there were also a few examples of unnecessary lengthening of a vowel. for example, in बीमारी (beemari, sickness), there was unnecessary stress on 'बी' and hence it was lengthened, iv) incorrect syllable breaks were observed in some words. for example, नापसं द (naapasand, non-favourite),was pronounced as नाप-सं द, which is incorrect, v) it was also noted that sometimes consonant clusters were mispronounced. e.g. कु त्ा -(kutta) -dog, was incorrectly pronounced as कु -ता or कु त-ता. eventually, we employ the best voice model for generating word, gloss, and example audios. we generated, using unit selection based concatenative synthesis, audios for 151831 words, and 40337 synset glosses/example sentence. ## conclusion and future work we present our work on generating voice models using the festival speech synthesis system. we also describe our efforts to use deep learning based implementations for generating such a model. a survey of the current state-of-theart techniques available for speech synthesis was also done. we download pre-generated voice models available for hindi and provide a detailed qualitative and quantitative analysis by comparing them with the voice model generated by us. we evaluate our model via crowd-sourcing and select the best voice model to generate audios for all words, glosses and example sentences for hindi wordnet. we believe our work will help students and language learners understand the hindi language, and help them pronounce it as well. in future, we plan to improve the voice model by analyzing the speech output and incorporate more data for training. we also plan to implement wavenet and other such neural network based techniques for raw audio generation and training models to produce speech for a given text.