Semantic Content Prediction for Generating Interviewing Dialogues to Elicit Users’ Food Preferences (2022)

Abstract: Dialogue systems that aim to acquire user models through interactions with users need to have interviewing functionality. In this study, we propose a method to generate interview dialogues to build a dialogue system that acquires user preferences for food. First, we collected 118 text-based dialogues between the interviewer and customer and annotated the communicative function and semantic content of the utterances. Next, using the corpus as training data, we created a classification model for the communicative function of the interviewer’s next utterance and a generative model that predicts the semantic content of the utterance based on the dialogue history. By representing semantic content as a sequence of tokens, we evaluated the semantic content prediction model using BLEU. The results demonstrated that the semantic content produced by the proposed method was closer to the ground truth than the semantic content transformed from the output text generated by the retrieval model and GPT-2. Further, we present some examples of dialogue generation by applying model outputs to template-based sentence generation.

## introduction traditionally, dialogue systems have been characterized in terms of whether they are task-or non-task-oriented. in task-oriented dialogue systems, such as an airline ticket reservation system @xcite , eliciting specific information from the user, such as the date, time, and destination of the flight, is an important functionality for completing the task. however, in non-task-oriented dialogue systems, the system does not have a clear goal of eliciting information from the user, and the content of the dialogue is free. in this study, as another type of dialogue system, we focus on interviewing systems, in which the goal is to acquire a user model through a flexible flow of dialogue. specifically, we propose a method for interviewing a user's preference for food. to generate such dialogues, the system must be able to generate appropriate questions to elicit the user's preferences for food while touching on various topics in the food domain, such as how to eat, how to cook, etc., without limiting the content of the dialogue as a task-oriented dialogue does. one possible approach for achieving the requirements discussed above is end-to-end neural network, where dialogue generation is the task of predicting the next utterance using dialogue history as input @xcite . this method is widely used to generate open-domain dialogues, such as chitchats. however, it requires a large amount of dialogue data to learn the model. otherwise, less informative and contextually inappropriate utterances are frequently generated. to overcome this drawback, we propose a method that first determines the intention and semantic content of the interviewer's next utterance and then combines these to generate questions from the interviewer. figure 1 shows the proposed approach. first, we trained two models. the first is a classification model that takes the dialogue history as input and determines the interviewer's intention for the next utterance. the second is a generator model, which also takes the dialogue history as input and outputs the semantic content of the utterance, including the target (e.g., dish or ingredient) mentioned in the utterance and its related information (e.g., taste or how to eat). next, a template for sentence generation is selected based on these two outputs, and they are applied to the selected template to generate sentences. compared to learning a model that directly generates a surface expression, the models for predicting the intent and semantic content of an utterance can be learned using a smaller amount of data. additionally, because the content of an utterance is determined based on the context obtained from the dialogue history, appropriate utterances that are related to the preceding utterances can be generated. the contributions of this study are as follows: ## related work task-oriented dialog systems are typically designed to collect information from users. for example, previous studies have proposed an airline ticket reservation system (tis) @xcite , a restaurant reservation system @xcite , and interview systems to collect information, such as public opinion polls and class evaluation interview systems @xcite . in these systems, the purpose of the dialogue is to obtain information to accomplish a predefined task. meanwhile, chitchat does not have a clear goal as a task-oriented dialogue does, but this type of dialogue has the potential to elicit a variety of information from the user. for example, the system asks follow-up questions such as "please tell me more about the keyword" by using a keyword from the user's preceding utterance. to improve such interviewing functionality, relevant topics and questions should be selected and the dialogue strategies should be modified. to address these issues, we propose a method to determine the target object and semantic content of the system response based on the dialogue context. previous studies on dialogue generation have proposed different techniques to generate task-and non-task-oriented dialogue. early studies on generating open-domain chitchat proposed dnn-based techniques to generate system responses by exploiting the data-driven approach @xcite @xcite . recent studies have proposed incorporating useful information (that is relevant to the domain) and responses into the model, thus improving the quality of generated responses @xcite . some studies have exploited word-based information, such as nouns extracted from the user's preceding utterances and a set of keywords predicted to be used in the response @xcite . other studies have used knowledge ontologies, including commonsense @xcite @xcite . however, these end-to-end methods, in which training models directly generate system responses, require a large amount of training data, and our corpus was not sufficiently large for this approach. in traditional task-oriented dialogue systems, the information required to achieve the dialogue goals is limited to the task domain. therefore, the internal state of the system is defined as a slot-value pair, and the system generates responses through the following modules: a) understanding the user's utterance, b) determining the system action (e.g., the intention and the slot-value as the utterance content) based on the internal state, and c) generating a response sentence from the system action. the action of the system is determined by rule-based, statistical-based @xcite , deep learning @xcite and reinforcement learning approaches @xcite . in this study, we exploited the approach described above, which represents the interviewer's utterance as structured semantic content composed of the intent of the utterance, the objects mentioned in the utterance, and their attributes and values. we created a machine learning model to predict these types of information and generate responses based on the determined actions. ## data collection and dataset making this study aims to generate interview dialogues that elicit information about users' food preferences. for this purpose, we collected role-play conversations between an interviewer and a customer and constructed a corpus from the collected conversations. ## models with the goal of building a dialogue system that generates the interviewer's appropriate questions to acquire the customer's food preferences, we present two machine learning models in this section for communicative function prediction and semantic content generation. ## conclusion in this study, we created a dialogue model to interview the food preferences of users. text-based dialogues between an interviewer and customer were collected, and the communicative function and semantic content of the interviewer's utterances were annotated. using this dataset, we created models to predict the communicative function of the interviewer's utterances and generate semantic content. the outputs of these two models were then applied to template-based response generation to produce a response. in the model evaluation for generating semantic content, the proposed model outperformed the two baseline models, retrieval and generative, in the automatic evaluation using bleu-4. as future work, we will improve the response generation mechanism to generate a variety of expressions because the current template-based response generation may not be sufficient in its expressiveness. for example, one of the ideas would be presenting candidates such as japanese, chinese, and italian when asking about preferences for a genre and asking the user to select one. it would also be useful to predict the user's preference based on the dialog history and user information and generate questions such as "do you prefer chinese to italian? thus, by using question content (e.g., genre) and related vocabulary and knowledge (chinese and italian as examples of genre), the question variation can be increased. another possibility is to automatically extract or determine the response templates through machine learning, but this is a challenging task. further, a user study should be conducted, as it is known that automatic evaluation using bleu does not always correlate with human evaluation @xcite . in the user study, users interact with the system, and then they evaluate the quality of the responses generated from the system, and judge whether the system effectively elicits information from the user.