Design and Development of Spoken Dialogue System in I ndic Languages (2021)

Abstract: Based on the modular architecture of a task-oriented Spoken Dialogue System (SDS), the presented work focussed on constructing all the system components as statistical models with parameters learned directly from the data by resolving various language-specific and language-independent challenges. In order to understand the research questions that underlie the SLU and DST module in the perspective of Indic languages (Hindi), we collect a dialogue corpus: Hindi Dialogue Restaurant Search (HDRS) corpus and compare various state-of-the-art SLU and DST models on it. For the dialogue manager (DM), we investigate the deep-learning reinforcement learning (RL) methods, e.g. actor-critic algorithms with experience replay. Next, for the dialogue generation, we incorporated Recurrent Neural Network Language Generation (RNNLG) framework based models. For speech synthesisers as a last component in the dialogue pipeline, we not only train several TTS systems but also propose a quality assessment framework to evaluate them.

## introduction recently, substantial improvements in speech recognition performance have enticed the research community to build natural conversational interfaces in the form of a spoken dialogue system (sds) @xcite . this paper is concerned broadly with designing a complete spoken dialogue system in an indic language scenario, i.e. hindi. no significant work has been done earlier to promote the research and development of a hindi spoken dialogue system. hence, it becomes critical for the current work to address the issues and challenges unveiled for the hindi language through introducing new datasets, methods and measures to build and evaluate all the integral modules of the hindi sds. a typical sds structure is based on a modular pipeline design connecting five principal components in a specific order @xcite : automatic speech recognition (asr), spoken language understanding (slu), dialogue manager (dm), natural language dialogue generation (nldg) and text-to-speech synthesiser (tts). the work presented in this paper demonstrates how these components are developed individually and integrated at the end to develop a real-world spoken dialogue system in hindi. in a statistical spoken dialogue system, the aim is to replace each of the aforementioned components with a statistical model with parameters estimated from data @xcite . the overall goal is to build a datadriven dialogue system with the ability to get improved over time and be perceived as behaving human-like by the users. the components of such systems are based on statistical methods, i.e. probabilistic distribution, neural network models, which allow them to handle uncertainty in both their inputs and outputs @xcite . as the hindi text contains lots of lexical/morphological ambiguities, therefore, it becomes a key challenge for dst and nldg models to appropriately detect the das, understand the utterances and generate natural responses. hindi is very rich in inflectional morphology. there is usually a limit of 8-9 inflected word forms of nouns in english @xcite , but in hindi, it is more than 40 @xcite . the way a language is spoken and written gets changed from place to place. it leads to the introduc-tion of variations where the meaning of a sentence is the same, but the way to express gets changed @xcite . other language-related challenges that a hindi sds have to deal with are code-mixing @xcite , hidden information @xcite , echo-words @xcite , etc. code-mixing is the mixing of more languages in the conversation. there are many cases in the corpus where the user had expressed some words from english during the conversation. (example: "मु झे कम रें ज वाले रे स्तरां क तलाश है ।" ("i am looking for low (cost) range restaurants.")). here the word "रें ज" (range) is an english word that gives an indication of the cost. ## contributions this research contributes at the following levels: ## silpa 1 : a hindi sds we design our hindi sds by dividing it into five modules in a pipeline architecture @xcite and connecting them in a specific order, as shown in figure 1 . the remainder of the section discusses the contributions specific to these modules. ## dialogue agent & web interface we incorporated and adapted the multidomain statistical dialogue system toolkit pydial-toolkit @xcite to build our dialogue agent "silpassistant". the agent is the main component responsible for the dialogue interaction. the general architecture of the dialogue system with a speech interface is shown in figure 2 . the agent can communicate to the user in both texts as well as speech. for the text-based interaction, texthub utility is provided, which simply connects the agent to a terminal. to enable speech-based dialogue, the dialogue-server works as an interface between the agent and the speech-client. ## conclusion & future studies the current work has examined the challenges of developing a conversational system built upon native indian languages for a real-world task. the original contributions of this thesis include: the development of an hdrs corpus on which various state-of-the-art slu and dst models, i.e. nbt, glad, gce, gsat, simple-bert and sumbt, are implemented and compared; the rnnlg models, i.e. h-lstm, sc-lstm, msc-lstm and enc-dec, have been experimented and used to train corpus-based nldg module on a self-collected corpus in an indic language hindi; construction of dialogue policy with rl based approaches, i.e. gp-sarsa, dqn, a2c (actor-critic), a2cer (proposed), on the user-system act pairs generated by a user simulator; proposing a novel framework lboe for quality assessment of a synthesised speech generated from various tts engines, i.e. uss, hmm, clu and dnn. in the current work, we have explored a unimodal natural-language based dialogue scenario. as the human-to-human conversation is multimodal, involving various linguistic forms and non-verbal signals @xcite , a multimodal human-to-computer conversation should therefore be more intuitive.