Interactive Question Clarification in Dialogue via Reinforcement Learning (2020)

Abstract: Coping with ambiguous questions has been a perennial problem in real-world dialogue systems. Although clarification by asking questions is a common form of human interaction, it is hard to define appropriate questions to elicit more specific intents from a user. In this work, we propose a reinforcement model to clarify ambiguous questions by suggesting refinements of the original query. We first formulate a collection partitioning problem to select a set of labels enabling us to distinguish potential unambiguous intents. We list the chosen labels as intent phrases to the user for further confirmation. The selected label along with the original user query then serves as a refined query, for which a suitable response can more easily be identified. The model is trained using reinforcement learning with a deep policy network. We evaluate our model based on real-world user clicks and demonstrate significant improvements across several different experiments.

## introduction in real-world dialogue systems, a substantial portion of all user queries are ambiguous ones for which the system is unable to precisely identify the underlying intent. we observed that many such queries in our question answering (qa) system exhibited one of the following two characteristics. in our work, we thus instead investigate interactive clarification by providing the user with specific choices as options, such as intent options @xcite . unlike previous work, we propose an end-to-end model that suggests labels to clarify ambiguous questions. an example of this sort of approach is given in figure 1 . here, we consider a closed-domain qa system, where a typical method is to build an intent inventory to address high-frequency requests. in this setting, the set of unambiguous candidate labels for an ambiguous user utterance corresponds to a set of frequently asked questions covered by the intent inventory. in a closed domain, we consider the candidate set to be finite. for example, in figure 1 , there are three specific intents corresponding to the ambiguous question "how to apply". our approach induces phrase tags as labels for each intent. thus, we have a catalog of intents with corresponding labels that can be presented to the user. the challenge lies in selecting a suitable list of labels that can effectively clarify the ambiguous question. in our approach, the problem of finding the label sequence is formulated as a collection partitioning problem, where the objective is to cover as many elements as possible while distinguishing elements as clearly as possible. the task of question clarification thus amounts to obtaining a suitable set of labels. the main contributions of our work are: ## preliminaries system overview. in order to provide a more concrete picture of our approach, we first briefly describe our qa system, illustrated in figure 2 , as an example of how this approach can be instantiated. when the conversation exceeds a certain number of rounds or the user explicitly requests human service, the conversation is transferred to a human customer service agent. in this setting, our clarification method chiefly serves to reduce the workload of those human agents. in our real system, there are two stages: label clarification and intent retrieval as illustrated in figure 1 . the label clarification stage provides 6 labels for the user to confirm. upon selecting one of the suggested labels, the user question is concatenated with the selected label phrase as a new query input. the intent retrieval stage seeks to provide 3 relevant intents for the user to select according to the concatenated query. these additional labels can help clarify and improve the relevance. intent and label inventory. our system relies on a closed-domain intent and label inventory. the intents along with their corresponding answers are compiled by human experts. the set of labels is a collection of words or phrases that are manually constructed from intents by marking up keywords such as suitable predicates, subjects, or objects. as shown in figure 3 , there is a many-to-many relationship between intents and labels. note that there is substantial synonymy among the set of labels, which may result in numerous repetitive recommendation results. thus, ensuring the diversity of the results ought to be a factor in the design of the policy model. ## conclusion we present an end-to-end model to resolve ambiguous questions in dialogue by clarifying them using label suggestions. we cast the question clarification problem as a collection partition problem. in order to improve the quality of the interactive labels as well as reduce the semantic overlap of the labels and the user's question, we propose a novel reward based on recall of potential intents and information gain. we establish its effectiveness in a series of experiments, which suggest that this novel notion of clarification may as well be adopted for other kinds of disambiguation problems.