Using Domain Knowledge to Guide Dialog Structure Induction via Neural Probabilistic Soft Logic (2023)

Abstract: Dialog Structure Induction (DSI) is the task of inferring the latent dialog structure (i.e., a set of dialog states and their temporal transitions) of a given goal-oriented dialog. It is a critical component for modern dialog system design and discourse analysis. Existing DSI approaches are often purely data-driven, deploy models that infer latent states without access to domain knowledge, underperform when the training corpus is limited/noisy, or have difficulty when test dialogs exhibit distributional shifts from the training domain. This work explores a neural-symbolic approach as a potential solution to these problems. We introduce Neural Probabilistic Soft Logic Dialogue Structure Induction (NEUPSL DSI), a principled approach that injects symbolic knowledge into the latent space of a generative neural model. We conduct a thorough empirical investigation on the effect of NEUPSL DSI learning on hidden representation quality, few-shot learning, and out-of-domain generalization performance. Over three dialog structure induction datasets and across unsupervised and semi-supervised settings for standard and cross-domain generalization, the injection of symbolic knowledge using NEUPSL DSI provides a consistent boost in performance over the canonical baselines.

## introduction the seamless integration of prior domain knowledge into the neural learning of language structure has been an open challenge in the machine learning and natural language processing communities. in this work, we inject symbolic knowledge into the neural learning process of a two-party dialog structure induction (dsi) task @xcite . this task aims to learn a graph, known as the dialog structure, capturing * work done during internship at google. the potential flow of states occurring in a dialog dataset for a specific task-oriented domain, e.g., figure 1 represents a possible dialog structure for the goal-oriented task of booking a hotel. nodes in the dialog structure represent conversational topics or dialog acts that abstract the intent of individual utterances, and edges represent transitions between dialog acts over successive turns of the dialog. similar to the motivation described in @xcite , previous work in dsi has been split between supervised and unsupervised methods. in particular, traditional supervised methods @xcite relied on dialog structure hand-crafted by human domain experts. unfortunately, this process is labor-intensive and, in most situations, does not generalize easily to new domains. therefore, recent work attempts to overcome this limitation by studying unsupervised dsi; e.g., hidden markov models @xcite @xcite and more recently variational recurrent neural networks (vrnn) @xcite . however, being purely datadriven, these approaches have difficulty with limited/noisy data and cannot easily exploit domainspecific or domain-independent constraints on dialog that may be readily provided by human experts (e.g., greet utterances are typically made in the first couple of turns). in this work, we propose neural probabilistic soft logic dialogue structure induction (neupsl dsi). this practical neuro-symbolic approach improves the quality of learned dialog structure by infusing domain knowledge into the end-to-end, gradient-based learning of a neural model. we leverage probabilistic soft logic (psl), a wellstudied soft logic formalism, to express domain knowledge as soft rules in succinct and interpretable first-order logic statements that can be incorporated easily into differentiable learning @xcite . this leads to a simple method for knowledge injection with minimal change to the sgd-based training pipeline of an existing neural generative model. our key contributions are: 1) we propose ne-upsl dsi, which introduces a novel smooth relaxation of psl constraints tailored to ensure a rich gradient signal during back-propagation; 2) we evaluate neupsl dsi over synthetic and realistic dialog datasets under three settings: standard generalization, domain generalization, and domain adaptation. we show quantitatively that injecting domain knowledge provides a boost over unsupervised and few-shot methods; and 3) we comprehensively investigate the effect of soft logic-augmented learning on different aspects of the learned neural model by examining its quality in representation learning and structure induction. ## related work dialog structure induction (dsi) refers to the task of inferring latent states of a dialog without full supervision of the state labels. earlier work focus on building advanced clustering methods, e.g., topic models, hmm, gmm @xcite , which are later combined with pre-trained or task-specific neural representations @xcite @xcite . another line of work focuses on inferring latent states using neural generative models, most notably direct-discrete variational recurrent neural networks (dd-vrnn) @xcite , with later improvements including bert encoder @xcite , gnn-based latent-space model @xcite , structured-attention decoder @xcite , and database query modeling @xcite . finally, @xcite ; @xcite explored dsi in a semi-supervised and few-shot learning context. no work has explored dsi with domain knowledge as weak supervision or conducted a comprehensive evaluation of model performance across different generalization settings (i.e., unsupervised, few-shot, domain generalization, and domain adaptation). a related field of work, neuro-symbolic computing (nesy), is an active area of research that aims to incorporate logic-based reasoning with neural computation. this field contains a plethora of different neural symbolic methods and techniques. the methods that closely relate to our line of work seek to enforce constraints on the output of a neural network @xcite @xcite @xcite . for a more in-depth introduction, we refer the reader to these excellent recent surveys: @xcite . these methods, although powerful, are either: specific to the domain they work in, do not use the same soft logic formulation, have not been designed for unsupervised systems, or have not been used for dialog structure induction. finally, our method is most closely related to the novel nesy approaches of neural probabilistic soft logic (neupsl) @xcite , deepproblog (dpl) @xcite , and logic tensor networks (ltns) @xcite . ltns instantiate a model which forwards neural network predictions into functions representing symbolic relations with real-valued or fuzzy logic semantics, and deepproblog uses the output of a neural network to specify probabilities of events. the mathematical formulation of ltns and dpl differs from our underlying soft logic distribution. neupsl unites state-of-the-art symbolic reasoning with the low-level perception of deep neural networks through probabilistic soft logic (psl). our method uses a neupsl formulation; however, we introduce a novel variation to the soft logic formulation, develop theory for unsupervised tasks, introduce the whole system in tensorflow, and apply it to dialog structure induction. ## background our neuro-symbolic approach to dialog structure induction combines the principled formulation of probabilistic soft logic (psl) rules with a neural generative model. in this work, we use the state-ofthe-art direct-discrete variational recurrent neural network (dd-vrnn) as the base model @xcite . we start by introducing the syntax and semantics for dd-vrnn and psl. ## neural probabilistic soft logic dialogue structure induction in this section, we describe our approach to integrating domain knowledge and neural network-based dialog structure induction. our approach integrates an unsupervised neural generative model with dialog rules using soft constraints. we refer to our approach as neural probabilistic soft logic dialogue structure induction (neupsl dsi). in the following, we define the dialog structure learning problem, describe how to integrate the neural and symbolic losses, and highlight essential model components that address optimization and representation-learning challenges under gradientbased neuro-symbolic learning. problem formulation given a goal-oriented dialog corpus d, we consider the dsi problem of learning a graph g underlying the corpus. more formally, a dialog structure is defined as a directed graph @xmath0, p ), where @xmath1, . . . , s m } encodes a set of dialog states, and p a probability distribution p(s t |@xmath2. given the underlying dialog structure g, a dialog d @xmath3, . . . , x t } ∈ d is a temporallyordered set of utterances x t . assume x t is defined according to an utterance distribution conditional on past history p(x t |@xmath4, @xmath5, and the state s t is defined according to p(s t |@xmath6. given a dialog corpus @xmath7, the task of dsi is to learn a directed graphical model @xmath8, p ) as close to the underlying graph as possible. ## experimental evaluation we evaluate the performance of neupsl dsi on three task-oriented dialog corpuses in both unsupervised and highly constrained semi-supervised settings. further, we provide an extensive ablation on different aspects of the learned neural model. we investigate the following questions: q1) how does neupsl dsi perform in an unsupervised setting when soft constraints are incorporated into the loss? q2) when introducing few-shot labels to dd-vrnn training, do soft constraints provide a boost? q3) how do design choices such as log relaxation and re-weighted bag-of-words loss (introduced in section 4.2-4.3) impact performance? datasets these questions are explored using three goal-oriented dialog datasets: multiwoz 2.1 synthetic @xcite and two versions of the schema guided dialog (sgd) dataset; sgd-synthetic (where the utterance is generated by a template-based dialog simulator) and sgd-real (which replaces the machine-generated utterances of sgd-synthetic with its human-paraphrased counterparts) @xcite . for the sgdreal dataset, we evaluate over three unique data settings, standard generalization (train and test over the same domain), domain generalization (train and test over different domains), and domain adaptation (train on (potentially labeled) data from the training domain and unlabeled data from the test domain, and test on evaluation data from the test domain). appendix c describes further details. ## discussion and conclusions this paper introduces neupsl dsi, a novel neurosymbolic learning framework that uses differentiable symbolic knowledge to guide latent dialog structure learning. through extensive empirical evaluations, we illustrate how the injection of just a few domain knowledge rules significantly improves both correctness and hidden representation quality in this challenging unsupervised nlp task. while neupsl dsi sees outstanding success in the unsupervised settings, the introduction of additional labels highlights a potential limitation of ne-upsl dsi. if the domain knowledge introduced is weak or noisy (as in the sgd setting), when the model is provided with more substantial evidence, this additional noisy supervision can, at times, hurt generalization. therefore, enabling models to perform weight learning, where the model adaptively weights the importance of symbolic rules as stronger evidence is introduced, is an interesting future direction @xcite . @xcite . learning discourse-level diversity for neural dialog models using conditional variational autoencoders. in association for computational linguistics (acl).