Injecting Comparison Skills in Task-Oriented Dialogue Systems for Database Search Results Disambiguation (2023)

Abstract: In task-oriented dialogue (TOD) systems designed to aid users accomplish specific goals in one or more domains, the agent retrieves entities that satisfy user constraints from the database. However, when multiple database search results exist, an ambiguity occurs regarding which results to select and present to the user. Existing TOD systems handle this ambiguity by randomly selecting one or few results and presenting their names to the user. However, in a real scenario, users do not always accept a randomly recommended entity, and users should have access to more comprehensive information about the search results. To address this limitation, we propose a novel task called Comparison-Based database search Ambiguity handling (CBA), which handles ambiguity in database search results by comparing the properties of multiple entities to enable users to choose according to their preferences. Accordingly, we introduce a new framework for automatically collecting high-quality dialogue data along with the Disambiguating Schema-guided Dialogue (DSD) dataset, an augmented version of the SGD dataset. Experimental studies on the DSD dataset demonstrate that training baseline models with the dataset effectively address the CBA task. Our dataset and code will be publicized.

## introduction task-oriented dialogue (tod) systems that help users accomplish specific goals in one or multiple domains have become a subject of great interest in both academics and industry @xcite @xcite . the system performs a database (db) search @xcite based on user-specified constraints, confirms the target entity, and then helps the user with the de- ## disambiguating schema-guided dialogue (dsd) dataset we introduce a new dataset, disambiguating schema-guided dialogue (dsd), along with an automatic framework for constructing high-quality dialogue datasets to solve the cba task. the dsd dataset includes (1) utterances that solve the ambiguity of database search results and (2) corresponding data frames including a new dialogue act compare. we generate comparison-based disambiguating utterances and replace a target turn with them; the target turn represents the turn where the agent recommends the randomly selected entity to the user to resolve db search ambiguity in the sgd dataset. the generated utterances are designed to connect seamlessly with the existing dialogue. specifically, as shown in figure 1 , dsd contains one modified utterance and two additional utterances for the cba task. the first is a modified system utterance that presents two entities along with additional information about the entities in the form of a trade-off. the second is an additional user utterance in which the user mentions the desired entity according to their preference. to make the dataset more challenging, the user does not directly mention the entity name but the user's preference for the characteristics of the desired entity. finally, a confirmation utterance is followed by the system that confirms the selected entity. ## experiment we present baseline experiments on the dsd dataset. for the three subtasks presented in section 3.2, we formulate the problems as dialogue generation (dg) @xcite , dialogue state tracking (dst) @xcite , and named entity recognition (ner) @xcite . experiments are conducted using the dsd and sgd datasets for the three subtasks. because dsd is created by transforming only the training dataset of sgd, the test set is formed by splitting the train- ## conclusion we propose a new task for tod, comparison-based database search ambiguity handling (cba). this task aims to handle database search result ambiguity by comparing the properties of multiple entities to allow users to choose according to their preferences, which has proven to be the preferred solution over existing methods. we also introduced a disambiguating schema-guided dialogue (dsd) dataset by augmenting the sgd dataset to solve our task. experimental studies have demonstrated that the dsd can effectively inject comparison skills into the tod systems.