Domain Expert Platform for Goal-Oriented Dialog Collection (2021)

Abstract: Today, most dialogue systems are fully or partly built using neural network architectures. A crucial prerequisite for the creation of a goal-oriented neural network dialogue system is a dataset that represents typical dialogue scenarios and includes various semantic annotations, e.g. intents, slots and dialogue actions, that are necessary for training a particular neural network architecture. In this demonstration paper, we present an easy to use interface and its back-end which is oriented to domain experts for the collection of goal-oriented dialogue samples. The platform not only allows to collect or write sample dialogues in a structured way, but also provides a means for simple annotation and interpretation of the dialogues. The platform itself is language-independent; it depends only on the availability of particular language processing components for a specific language. It is currently being used to collect dialogue samples in Latvian (a highly inflected language) which represent typical communication between students and the student service.

## introduction modeling of human-computer interaction through dialogue systems and chatbots has raised a great interest among researchers and industry already since the time when the first chatbot eliza @xcite was created. this interest has become viral after the successful introduction of siri @xcite . today, virtual assistants, virtual agents and chatbots are present everywhere: on mobile devices, on different social networking platforms, on many websites and through smart home devices and robots. the virtual conversational agents are usually implemented as end-to-end neural network models, or their components are implemented through neural network architectures @xcite . such architectures require creation of datasets that represent various dialogue scenarios, as well as knowledge of the specific domain. this information has to be provided in specific data formats that in many cases are too complicated for domain experts. moreover, the required training datasets usually include various annotation layers, such as named entities, dialogue acts, intents, etc. the creation of such datasets is a complex task, and the datasets are not completely isolated and abstracted from the particular dialogue system. thus, domain experts that are involved in the creation of the datasets must have a high-level understanding of the overall structure of the dialogue system and its components, and how it is reflected in the annotated dialogue samples. this demonstration paper address this issue by presenting a web-based platform for the creation and maintenance of dialogue datasets foot_0 . the interface of the platform is very simple and high-level: it allows a domain expert without detailed technical knowledge of the underlying dialogue system to create and update a representative training dataset, as well as to maintain the underlying database of domain-and organisation-specific information that will be used for question answering. the platform provides tools for the creation of goal-oriented dialogue systems, in particular: our platform not only supports collection of dialogue scenarios, but also simulates prototypical interaction between human and computer. the tool has been successfully used for the creation of a dialogue dataset for the virtual assistant that supports the work of the student service in relation to three frequently asked topics: working hours and contacts of the personnel and structural units (e.g. libraries), issues regarding academic leave, as well as enrollment requirements and documents to be submitted @xcite . in the next chapters of this paper, we describe our motivation to develop the platform, its overall architecture and main components, and the domain expert interface and its main components. ## background and motivation for english and several other widely used languages, many publicly available dialogue datasets have been created and are reused for different research and development needs (e.g., @xcite al. (2018), zeng et al. (2020) ). however, in the case of less-resourced languages, only few or no training datasets are available @xcite . to our knowledge, there is no publicly available dataset for latvian, that could be used for goal-oriented dialogue system modelling. to overcome this obstacle, some research groups machinetranslate existing english datasets into the lowresourced languages, while others try to build training datasets from scratch. when possible, crowdsourcing, including gamification @xcite , is used as well. however, there in no best recipe for obtaining or collecting dialogue samples for a specific nlp task (in our case, dialogue modeling) for a less-resourced language with a relatively small number of speakers. the motivation of our work is the necessity to build virtual assistants in less-resourced settings. the practical use case to test the platform has been the everyday communication between students and the student service of the university of latvia. since this communication has never been intentionally recorded, we started with the analysis of data retrieved from an online student forum to identify the most common topics, question and answer templates, and the typical dialogue scenarios. for the demonstration purposes, we have chosen three common topics: working hours, academic leave, and enrollment requirements. elaborated and annotated sample dialogues constituting the training dataset have been specified by a domain expert using the dialogue management platform presented in this paper. since we focus on goal-oriented virtul assistants, the hybrid code networks (hcn) architecture has been selected for the implementation @xcite allowing us to combine recursive neural networks (rnn) with the domain-specific knowledge and action templates of the dialogue system. the concrete dialogue system is implemented within the deeppavlov framework foot_1 . ## overall architecture and components the platform presented in this paper is designed to support three use cases: figure 1 illustrates the architecture of the platform. apart from the domain expert user interface described in detail in section 4, key components of the platform are four databases for storing the dialogue scenarios, the relevant entities and their aliases for slot filling, the required external knowledge for question answering, and reusable templates for response generation. ## user interface in this section we present overall interface and constituents (sub-windows) of our dialogue data preparation platform: the window for dialogue collection, the action template editing window, the knowledge base preparation and management window, the window for intent definition and the window for creation and maintenance of slot filler aliases (see figure 2 ) 3 . the user interface for data editing is powered by python http backend that serves static files and api calls. the backend modifies all four databases directly and uses slotfiller to retrieve slots from user interface. frontend is written in javascript and vuejs, and is running inside a web browser. ## conclusion in this paper, we have presented a configurable platform for dialogue collection that supports synchronization of information necessary for building a goal-oriented dialogue system, besides specification of dialogue scripts. the presented platform is publicly available 4 . it has been used for creation of latvian-specific dataset of dialogues between students and a student service. following recommendations from reviewers the platform is currently demonstrated on prototypical english dialogue samples, demon-4 http://bots.ailab.lv/ strating its scalability to other languages. we will add a short walkthrough video demonstrating the main features of the platform. the next development tasks include simple export of dialogue data in commonly used formats to facilitate experiments with various neural dialogue system architectures, and support for a one-click re-training process which currently is implemented as separate background process.