Unsupervised Extraction of Dialogue Policies from Conversations (2024)

Abstract: Dialogue policies play a crucial role in developing task-oriented dialogue systems, yet their development and maintenance are challenging and typically require substantial effort from experts in dialogue modeling. While in many situations, large amounts of conversational data are available for the task at hand, people lack an effective solution able to extract dialogue policies from this data. In this paper, we address this gap by first illustrating how Large Language Models (LLMs) can be instrumental in extracting dialogue policies from datasets, through the conversion of conversations into a unified intermediate representation consisting of canonical forms. We then propose a novel method for generating dialogue policies utilizing a controllable and interpretable graph-based methodology. By combining canonical forms across conversations into a flow network, we find that running graph traversal algorithms helps in extracting dialogue flows. These flows are a better representation of the underlying interactions than flows extracted by prompting LLMs. Our technique focuses on giving conversation designers greater control, offering a productivity tool to improve the process of developing dialogue policies.

## introduction chatbots and virtual assistants have emerged as powerful tools for guiding users or automating specific tasks across different domains, from facilitating restaurant reservations @xcite to handling product returns on e-commerce platforms @xcite . most task-oriented dialogue systems (tods) nowadays use two key components: a natural language understanding (nlu) engine and a dialogue manager @xcite . the role of the nlu engine is to perform intent detection and slot 1 data and code can be found at https://github.com/ makeshn/flows_from_conversations . in the development of task-oriented assistants, it is common to have access to a corpus of preexisting conversations. recent research has shown considerable interest in harnessing these conversational corpora to construct tods. extracting intents directly from these dialogues has demonstrated significant potential in augmenting the efficiency of conversation designers in modeling the nlu component @xcite @xcite . however, the task of deriving dialogue policies from the same set of conversations presents a more complex challenge and requires a nuanced understanding of conversational dynamics and objectives. only a limited number of studies have ventured into this domain, exploring methodologies for automatic dialogue policy extraction @xcite @xcite . in this paper, we introduce a novel hybrid methodology ( §3) that combines large language models (llms) with graph-based algorithms for the automated extraction of dialogue policies from a corpus of task-specific conversations. to that end, we first translate the turns in each dialogue into canonical forms (sreedhar and parisien, 2022) using an llm. the canonical forms are then clustered together to smooth out minor variations, following which we construct a graph modelling the entire corpus of conversations. this graph is akin to a flow network, where nodes represent canonical forms of dialogue turns and edges signify the progression and connection between different turns. finally, we apply path-finding algorithms to this graph to extract dialogue policies. the proposed approach combines sequences of user and assistant canonical forms, that can be seen as dialogue trajectories, into a more complex dialogue policy. thus, the extracted policies can handle digressions that are expressed using a branching logic determined by the intent of a user message. as all trajectories in our policies, including digressions, are composed of sequences of (user and bot) canonical forms, we can express the dialogue policy extraction from a corpus of conversations as a compositional task that combines translation and multi-document summarization. this enables using automatic metrics (e.g. bleu, bertscore) for evaluating the quality of the generated policies ( §4). we also show that these metrics correlate very well with human evaluation ( §5). our findings indicate superior performance of our hybrid graph and llm-based approach over techniques that rely solely on prompting llms for policy generation. in addition to better quantitative performance, the graph-based methodology provides enhanced controllability, interpretability, and robustness. these qualities render it a practical and effective tool for aiding conversation designers, in contrast to the more opaque, black-box nature of prompt-based llm approaches. our main contributions are as follows: ## background task-oriented dialogue. most tools for building task-oriented chatbots and virtual assistants use two different components: nlu and a dialogue manager @xcite . these range from commercial solutions (e.g. google di-alogflow @xcite or oracle digital assistant @xcite ) to open-source tools like rasa @xcite or research-focused platforms such as convlab @xcite . dialogue policies can be modeled as sequences of user intents and bot actions, for example using stories in rasa @xcite or colang flows in nemo guardrails @xcite . our work is valuable for this modeling: the extracted dialogue policies can serve as starting points for conversation designers to refine. traditionally, the development of task-oriented dialogue systems (tods) required manual effort from experts. however, recent tools and methods aim to reduce this effort by leveraging large datasets for automatic intent discovery, with some addressing the challenge of dialogue policy generation. even end-to-end neural tods (hosseini-asl et al., 2020) that embed intents and policies in model weights can use the extracted humanreadable dialogue policies to enhance the explainability of the underlying opaque systems. ## method to extract dialogue policies (flows) from conversational data, we propose a pipeline comprising three key stages: intent identification, graph construction utilizing the identified intents, and the application of graph traversal algorithms for the extraction of dialogue flows. the functionality is depicted in fig. 1 : the input is a corpus of conversations on a given task and the output is the dialogue policy as a combination of sequences of canonical forms. ## results and analysis this section provides a quantitative comparison of graph-based methods and prompting-based techniques. additionally, a qualitative analysis, examining the variances in extracted dialogue flows, the effect of incorporating digressions into these flows, and the degree of flexibility and control provided by the graph-based approach is also presented. ## conclusion generating dialogue policies from a dataset of conversations can significantly reduce the effort required by conversation designers and domain experts to develop tods. we propose a novel hybrid llm and graph-based method to extract dialogue policies without relying on a predefined set of dialogue acts. our results are significant for three reasons. first, we demonstrate that dialogue policies can be computed using network flow in a graph of all possible conversations for a given task. second, modeling conversations as sequences of canonical forms enhances explainability and controllability. third, incorporating digressions as high-flow paths in the graph allows conversation designers to control the granularity of dialogue policies.