Using Domain Knowledge to Guide Dialog Structure Induction via Neural Probabilistic Soft Logic
Year: 2023

Abstract:
Dialog Structure Induction (DSI) is the task of inferring the latent dialog structure (i.e., a set of dialog states and their temporal transitions) of a given goal-oriented dialog. It is a critical component for modern dialog system design and discourse analysis. Existing DSI approaches are often purely data-driven, deploy models that infer latent states without access to domain knowledge, underperform when the training corpus is limited/noisy, or have difficulty when test dialogs exhibit distributional shifts from the training domain. This work explores a neural-symbolic approach as a potential solution to these problems. We introduce Neural Probabilistic Soft Logic Dialogue Structure Induction (NEUPSL DSI), a principled approach that injects symbolic knowledge into the latent space of a generative neural model. We conduct a thorough empirical investigation on the effect of NEUPSL DSI learning on hidden representation quality, few-shot learning, and out-of-domain generalization performance. Over three dialog structure induction datasets and across unsupervised and semi-supervised settings for standard and cross-domain generalization, the injection of symbolic knowledge using NEUPSL DSI provides a consistent boost in performance over the canonical baselines.

heading: 1 Introduction
text: The seamless integration of prior domain knowledge into the neural learning of language structure has been an open challenge in the machine learning and natural language processing communities. In this work, we inject symbolic knowledge into the neural learning process of a two-party dialog structure induction (DSI) task (Zhai and Williams, 2014; Shi et al., 2019). This task aims to learn a graph, known as the dialog structure, capturing
∗Work done during internship at Google.
the potential flow of states occurring in a dialog dataset for a specific task-oriented domain, e.g., Figure 1 represents a possible dialog structure for the goal-oriented task of booking a hotel. Nodes in the dialog structure represent conversational topics or dialog acts that abstract the intent of individual utterances, and edges represent transitions between dialog acts over successive turns of the dialog.
Similar to the motivation described in (Shi et al., 2019), previous work in DSI has been split between supervised and unsupervised methods. In particular, traditional supervised methods (Jurafsky, 1997) relied on dialog structure hand-crafted by human domain experts. Unfortunately, this process is labor-intensive and, in most situations, does not generalize easily to new domains. Therefore, recent work attempts to overcome this limitation by studying unsupervised DSI; e.g., hidden Markov models (Chotimongkol, 2008; lan Ritter et al., 2010; Zhai and Williams, 2014) and more recently Variational Recurrent Neural Networks (VRNN) (Chung et al., 2015; Shi et al., 2019). However, being purely datadriven, these approaches have difficulty with limited/noisy data and cannot easily exploit domainspecific or domain-independent constraints on dialog that may be readily provided by human experts
7631
(e.g., Greet utterances are typically made in the first couple of turns).
In this work, we propose Neural Probabilistic Soft Logic Dialogue Structure Induction (NEUPSL DSI). This practical neuro-symbolic approach improves the quality of learned dialog structure by infusing domain knowledge into the end-to-end, gradient-based learning of a neural model. We leverage Probabilistic Soft Logic (PSL), a wellstudied soft logic formalism, to express domain knowledge as soft rules in succinct and interpretable first-order logic statements that can be incorporated easily into differentiable learning (Bach et al., 2017; Pryor et al., 2023). This leads to a simple method for knowledge injection with minimal change to the SGD-based training pipeline of an existing neural generative model.
Our key contributions are: 1) We propose NEUPSL DSI, which introduces a novel smooth relaxation of PSL constraints tailored to ensure a rich gradient signal during back-propagation; 2) We evaluate NEUPSL DSI over synthetic and realistic dialog datasets under three settings: standard generalization, domain generalization, and domain adaptation. We show quantitatively that injecting domain knowledge provides a boost over unsupervised and few-shot methods; and 3) We comprehensively investigate the effect of soft logic-augmented learning on different aspects of the learned neural model by examining its quality in representation learning and structure induction.

heading: 2 Related Work
text: Dialog Structure Induction (DSI) refers to the task of inferring latent states of a dialog without full supervision of the state labels. Earlier work focus on building advanced clustering methods, e.g., topic models, HMM, GMM (Zhai and Williams, 2014), which are later combined with pre-trained or task-specific neural representations (Nath and Kubba, 2021; Lv et al., 2021; Qiu et al., 2022). Another line of work focuses on inferring latent states using neural generative models, most notably Direct-Discrete Variational Recurrent Neural Networks (DD-VRNN) (Shi et al., 2019), with later improvements including BERT encoder (Chen et al., 2021), GNN-based latent-space model (Sun et al., 2021; Xu et al., 2021), structured-attention decoder (Qiu et al., 2020), and database query modeling (Hudeček and Dušek, 2022). Finally, Zhang et al. (2020); Wu et al. (2020) explored DSI in a semi-
supervised and few-shot learning context. No work has explored DSI with domain knowledge as weak supervision or conducted a comprehensive evaluation of model performance across different generalization settings (i.e., unsupervised, few-shot, domain generalization, and domain adaptation).
A related field of work, Neuro-Symbolic computing (NeSy), is an active area of research that aims to incorporate logic-based reasoning with neural computation. This field contains a plethora of different neural symbolic methods and techniques. The methods that closely relate to our line of work seek to enforce constraints on the output of a neural network (Hu et al., 2016; Donadello et al., 2017; Diligenti et al., 2017; Mehta et al., 2018; Xu et al., 2018; Nandwani et al., 2019). For a more in-depth introduction, we refer the reader to these excellent recent surveys: Besold et al. (2017) and De Raedt et al. (2020). These methods, although powerful, are either: specific to the domain they work in, do not use the same soft logic formulation, have not been designed for unsupervised systems, or have not been used for dialog structure induction.
Finally, our method is most closely related to the novel NeSy approaches of Neural Probabilistic Soft Logic (NeuPSL) (Pryor et al., 2023), DeepProbLog (DPL) (Manhaeve et al., 2021), and Logic Tensor Networks (LTNs) (Badreddine et al., 2022). LTNs instantiate a model which forwards neural network predictions into functions representing symbolic relations with real-valued or fuzzy logic semantics, and DeepProbLog uses the output of a neural network to specify probabilities of events. The mathematical formulation of LTNs and DPL differs from our underlying soft logic distribution. NeuPSL unites state-of-the-art symbolic reasoning with the low-level perception of deep neural networks through Probabilistic Soft Logic (PSL). Our method uses a NeuPSL formulation; however, we introduce a novel variation to the soft logic formulation, develop theory for unsupervised tasks, introduce the whole system in Tensorflow, and apply it to dialog structure induction.

heading: 3 Background
text: Our neuro-symbolic approach to dialog structure induction combines the principled formulation of probabilistic soft logic (PSL) rules with a neural generative model. In this work, we use the state-ofthe-art Direct-Discrete Variational Recurrent Neural Network (DD-VRNN) as the base model (Shi
et al., 2019). We start by introducing the syntax and semantics for DD-VRNN and PSL.

heading: 3.1 Direct Discrete Variational Recurrent Neural Networks
text: A Direct Discrete Variational Recurrent Neural Networks (DD-VRNN) (Shi et al., 2019) is an expansion to the popular Variational Recurrent Neural Networks (VRNN) (Chung et al., 2015), which constructs a sequence of VAEs and associates them with states of an RNN. The main difference between the DD-VRNN and a traditional VRNN is the priors of the latent states zt. They directly model the influence of zt−1 on zt, which models the transitions between different latent (i.e., dialog) states. To fit the prior into the variational inference framework, an approximation of p(zt|x<t, z<t) is made which changes the distribution to p(zt|zt−1):
p(x≤T , z≤T ) = T∏
t=1
p(xt|z≤t, x<t)p(zt|x<t, zt−1)
≈ T∏
t=1
p(xt|z≤t, x<t)p(zt|zt−1)
(1)
zt is modeled as zt ∼ softmax(ϕpriorτ (zt−1)) for a feature extraction neural network for the prior ϕpriorτ . Lastly, the objective function used in the DD-VRNN is a timestep-wise variational lower bound (Chung et al., 2015) augmented with a bagof-word (BOW) loss and Batch Prior Regularization (BPR) (Zhao et al., 2017, 2018), i.e.:
LV RNN = Eq(z≤T |x≤T )[log p(xt|z≤t, x<t)+ T∑
t=1
−KL(q(zt|x≤t, z<t)||p(zt|x<t, z<t))] (2)
so that the full objective function is
LDD−V RNN = LV RNN + λ ∗ Lbow (3)
where λ is a tunable weight and Lbow is the BOW loss. For further details on Lbow see Section 4.3 and Shi et al. (2019). To expand this to a semisupervised domain, the objective is augmented as:
LDD−V RNN = LV RNN + λ ∗ Lbow + Lsupervised (4)
where Lsupervised is the loss between the labels and predictions, e.g., cross-entropy.

heading: 3.2 Probabilistic Soft Logic
text: This work introduces soft constraints in a declarative fashion, similar to Probabilistic Soft Logic (PSL). PSL is a declarative statistical relational learning (SRL) framework for defining a particular probabilistic graphical model, known as a hingeloss Markov random field (HL-MRF) (Bach et al., 2017). PSL models relational dependencies and structural constraints using first-order logical rules, referred to as templates, with arguments known as atoms. For example, the statement “the first utterance in a dialog is likely to belong to the greet state" can be expressed as:
FIRSTUTT(U) → STATE(U, greet) (5)
Where (FIRSTUTT(U), STATE(U, greet)) are the atoms (i.e., atomic boolean statements) indicating, respectively, whether an utterance U is the first utterance of the dialog, or if it belongs to the state greet. The atoms in a PSL rule are grounded by replacing the free variables (such as U above) with concrete instances from a domain of interest (e.g., the concrete utterance ’Hello!’); we call these the grounded atoms. The observed variables and target/decision variables of the probabilistic model correspond to ground atoms constructed from the domain, e.g., FIRSTUTT(′Hello!′) may be an observed variable and STATE(′Hello!′, greet) may be a target variable.
PSL performs inference over soft logic constraints by allowing the originally Boolean-valued atoms to take continuous truth values in the interval [0, 1]. Using this relaxation, PSL replaces logical operations with a form of soft logic called Lukasiewicz logic (Klir and Yuan, 1995):
A ∧B = max(0.0, A+B − 1.0) A ∨B = min(1.0, A+B)
¬A = 1.0−A (6)
where A and B represent either ground atoms or logical expressions over atoms and take values in [0, 1]. For example, PSL will convert the statement from Equation 5, into the following:
min{1, 1− FIRSTUTT(U)+STATE(U, greet))} (7)
sinceA→ B ≡ ¬A∨B. In this way, we can create a collection of functions {ℓi}mi=1, called templates, that map data to [0, 1]. Using the templates, PSL defines a conditional probability density function
over the unobserved random variables y given the observed data x known as the Hinge-Loss Markov Random Field (HL-MRF):
P (y|x) ∝ exp(− m∑
i=1
λi · ϕi(y,x)) (8)
Here λi is a non-negative weight and ϕi a potential function based on the templates:
ϕi(y,x) = max{0, ℓi(y,x)} (9)
Then, inference for the model predictions y proceeds by Maximum A Posterior (MAP) estimation, i.e., by maximizing the objective function P (y|x) (eq. 8) with respect to y.

heading: 4 Neural Probabilistic Soft Logic Dialogue Structure Induction
text: In this section, we describe our approach to integrating domain knowledge and neural network-based dialog structure induction. Our approach integrates an unsupervised neural generative model with dialog rules using soft constraints. We refer to our approach as Neural Probabilistic Soft Logic Dialogue Structure Induction (NEUPSL DSI). In the following, we define the dialog structure learning problem, describe how to integrate the neural and symbolic losses, and highlight essential model components that address optimization and representation-learning challenges under gradientbased neuro-symbolic learning.
Problem Formulation Given a goal-oriented dialog corpus D, we consider the DSI problem of
learning a graph G underlying the corpus. More formally, a dialog structure is defined as a directed graph G = (S, P ), where S = {s1, . . . , sm} encodes a set of dialog states, and P a probability distribution p(st|s<t) representing the likelihood of transition between states (see Figure 1 for an example). Given the underlying dialog structure G, a dialog di = {x1, . . . , xT } ∈ D is a temporallyordered set of utterances xt. Assume xt is defined according to an utterance distribution conditional on past history p(xt|s≤t, x<t), and the state st is defined according to p(st|s<t). Given a dialog corpus D = {di}ni=1, the task of DSI is to learn a directed graphical model G = (S, P ) as close to the underlying graph as possible.

heading: 4.1 Integrating Neural and Symbolic
text: Learning under NEUPSL DSI
We now introduce how the NEUPSL DSI approach formally integrates the DD-VRNN with the soft symbolic constraints to allow for end-to-end gradient training. To begin, we define the relaxation of the symbolic constraints to be the same as described in Section 3.2. With this relaxation, we can build upon the foundations developed by Pryor et al. (2023) on Neural Probabilistic Soft Logic (NeuPSL) by augmenting the standard unsupervised DD-VRNN loss with a constraint loss. Figure 2 provides a graphical representation of this integration of the DD-VRNN and the symbolic constraints. Intuitively, NEUPSL DSI can be described in three parts: instantiation, inference, and learning.
Instantiation of a NEUPSL DSI model uses a set of first-order logic templates to create a set of
potentials that define a loss used for learning and evaluation. Let pw be the DD-VRNN’s predictive function of latent states with hidden parameters w and input utterances xvrnn. The output of this function, defined as pw(xvrnn), will be the probability distribution representing the likelihood of each latent class for a given utterance. Given a first-order symbolic rule ℓi(y,x), where y = pw(xvrnn) is the decision variable representing the latent state prediction from the neural model, and x represents the symbolic observations not incorporated in the neural features xvrnn, NEUPSL DSI instantiates a set of deep hinge-loss potentials of the form:
ϕw,i(xvrnn,x) = max(0, ℓi(pw(xvrnn),x)) (10)
For example, in reference to Equation 7, the decision variable y = pw(xvrnn) is associated with the STATE(x, greet) random variables, leading to:
ℓi(pw(xvrnn),x) =
min{1, 1− FIRSTUTT(U) + pw(xvrnn)} (11)
With the instantiated model described above, the NEUPSL DSI inference objective is broken into a neural inference objective and a symbolic inference objective. The neural inference objective is computed by evaluating the DD-VRNN model predictions with respect to the standard loss function for DSI. Given the deep hinge-loss potentials {ϕw,i}mi=1, the symbolic inference objective is the HL-MRF likelihood (Equation 8) evaluated at the decision variables y = pw(xvrnn):
Pw(y|xvrnn,x, λ) = (12)
exp ( − m∑
i=1
λi · ϕw,i(xvrnn,x) )
Under the NEUPSL DSI, the decision variables y = pw(xvrnn) are implicitly controlled by neural network weights w, therefore the conventional MAP inference in symbolic learning for decision variables y∗ = argminy P (y|xvrnn,x, λ) can be done simply via neural weight minimization argminw Pw(y|xvrnn,x, λ). As a result, NEUPSL DSI learning minimizes a constrained optimization objective:
w∗ = argmin w
[ LDD−V RNN + Lconstraint ]
(13)
where we define the constraint loss to be the loglikelihood of the HL-MRF distribution (13):
LConstraint = −logPw(y|xvrnn,x, λ). (14)

heading: 4.2 Improving soft logic constraints for gradient learning
text: The straightforward linear soft constraints used by the classic Lukasiewicz relaxation fail to pass back gradients with a magnitude and instead pass back a direction (e.g., ±1). Formally, the gradient of a potential ϕw(xvrnn,x) = max(0, ℓ(pw(xvrnn),x)) with respect to w is:
∂
∂w ϕw =
∂
∂w ℓ(pw,x) · 1ϕw>0
= [ ∂ ∂pw ℓ(pw,x) ] · ∂ ∂w pw · 1ϕw>0 (15)
Here ℓ(pw,x) = a · pw(xvrnn) + b where a, b ∈ R and pw(xvrnn) ∈ [0, 1], which leads to the gradient ∂∂pw ℓ(pw,x) = a. Observing the three Lukasiewicz operations described in Section 3.2, it is clear that a will always result in ±1 unless there are multiple pw(xvrnn) per constraint.
As a result, this classic soft relaxation leads to a naive, non-smooth gradient:
∂
∂w ϕw =
[ a1ϕw>0 ] · ∂ ∂w pw (16)
that is mostly consists of the predictive probability gradient ∂∂wpw. It barely informs the model of the degree to which pw satisfies the symbolic constraint ϕw (other than the non-smooth step function 1ϕw>0), thereby creating challenges in gradientbased learning.
In this work, we propose a novel log-based relaxation that provides smoother and more informative gradient information for the symbolic constraints:
ψw(x) = log ( ϕw(xvrnn,x) ) (17)
= log ( max(0, ℓ(pw(xvrnn),x)) )
This seemingly simple transformation brings a nontrivial change to the gradient behavior:
∂
∂w ψw =
1 ϕw · ∂ ∂w ϕw (18)
= [ a ϕw 1ϕw>0 ] · ∂ ∂w pw
As shown, the gradient from the symbolic constraint now contains a new term 1ϕw . It informs the model of the degree to which the model prediction
satisfies the symbolic constraint ℓ so that it is no longer a discrete step function with respect to ϕw. As a result, when the satisfaction of a rule ϕw is non-negative but low (i.e., uncertain), the gradient magnitude will be high, and when the satisfaction of the rule is high, the gradient magnitude will be low. In this way, the gradient of the symbolic constraint terms ϕi now guides the neural model to more efficiently focus on learning the challenging examples that don’t obey the existing symbolic rules. This leads to more effective collaboration between the neural and the symbolic components during model learning and empirically leads to improved generalization performance (Section 5).

heading: 4.3 Stronger control of posterior collapse via weighted bag of words
text: It is essential to avoid a collapsed VRNN solution, where the model puts all of its predictions in just a handful of states. This problem has been referred to as the vanishing latent variable problem (Zhao et al., 2017). Zhao et al. (2017) address this by introducing a bag-of-word (BOW) loss to VRNN modeling which requires a network to predict the bag-of-words in response x. They separate x into two variables: xo (word order) and xbow (no word order), with the assumption that they are conditionally independent given z and c:
p(x, z|c) = p(xo|z, c)p(xbow|z, c)p(z|c). (19)
Here, c is the dialog history: the preceding utterances, conversational floor (1 if the utterance is from the same speaker and 0 otherwise), and metafeatures (e.g., the topic). Let f be the output of a multilayer perception with parameters z, x, where f ∈ RV with V the vocabulary size. Then the BOW probability is defined as log p(xbow|z, c) = log ∏|x| t=1 efxt ∑V
j e fj
, where |x| is the length of x and
xt is the word index of the tth word in x. To impose robust regularization against the posterior collapse, we use a tf-idf-based re-weighting scheme using the tf-idf weights computed from the training corpus. Intuitively, this re-weighting scheme helps the model focus on reconstructing non-generic terms that are unique to each dialog state, which encourages the model to “pull" the sentences from different dialog states further apart in its representations space to minimize the weighted BOW loss better. In comparison, a model under the uniformly-weighted BOW loss may be distracted
by reconstructing the high-prevalence terms (e.g., "what is," "can I," and "when") that are shared by all dialog states. As a result, we specify the tf-idf weighted BOW probability as:
log p(xbow|z, c) = log |x|∏
t=1
wxie fxt ∑V j e fj , (20)
wherewxt = (1− α) N +αw′xt ,N is the corpus size, w′xt is the tf-idf word weight for the xt index, and α is a hyperparameter. In Section 5, we explore how this alteration affects the performance and observe if the PSL constraints still provide a boost.

heading: 5 Experimental Evaluation
text: We evaluate the performance of NEUPSL DSI on three task-oriented dialog corpuses in both unsupervised and highly constrained semi-supervised settings. Further, we provide an extensive ablation on different aspects of the learned neural model. We investigate the following questions: Q1) How does NEUPSL DSI perform in an unsupervised setting when soft constraints are incorporated into the loss? Q2) When introducing few-shot labels to DD-VRNN training, do soft constraints provide a boost? Q3) How do design choices such as log relaxation and re-weighted bag-of-words loss (introduced in Section 4.2-4.3) impact performance?
Datasets These questions are explored using three goal-oriented dialog datasets: MultiWoZ 2.1 synthetic (Campagna et al., 2020) and two versions of the Schema Guided Dialog (SGD) dataset; SGD-synthetic (where the utterance is generated by a template-based dialog simulator) and SGD-real (which replaces the machine-generated utterances of SGD-synthetic with its human-paraphrased counterparts) (Rastogi et al., 2020). For the SGDreal dataset, we evaluate over three unique data settings, standard generalization (train and test over the same domain), domain generalization (train and test over different domains), and domain adaptation (train on (potentially labeled) data from the training domain and unlabeled data from the test domain, and test on evaluation data from the test domain). Appendix C describes further details.
Constraints In the synthetic MultiWoZ setting, we introduce a set of 11 structural domain agnostic dialog rules. An example of one of these rules can be seen in Equation 5. These rules are introduced to
represent general facts about dialogs, with the goal of showing how the incorporation of a few expertdesigned rules can drastically improve generalization performance. For SGD settings, we introduce a single dialog rule that encodes the concept that dialog acts should contain utterances with correlated tokens, e.g., utterances containing ’hello’ are likely to belong to the greet state. This rule is designed to show the potential boost in performance a model can achieve from a simple source of prior information. Appendix C contains further details.
Metrics and Methodology The experimental evaluation examines two aspects: correctness of the learned latent dialog structure and quality of the learned hidden representation.
Structure Induction. To evaluate the model’s ability to correctly learn the latent dialog structure, we adapt the Adjusted Mutual Information (AMI) metric from clustering literature (see Appendix D.1 for details). AMI allows for a comparison between ground truth labels1 (e.g., "greet", "initial request",
1These labels were only used for final evaluation, not for
etc.) and latent state predictions (e.g., State1, · · ·, Statek).
Representation Learning. A standard technique for evaluating the quality of unsupervised representation is linear probing, i.e., train a lightweight linear probing model on top of the frozen learned representation, and evaluate the linear model’s generalization performance for supervised tasks (Tenney et al., 2019). To evaluate the quality of the learned DD-VRNN, we train a supervised linear classifier on top of input features extracted from the penultimate layer of the DDVRNN. We evaluate with both full supervision and few-shot supervision. Full supervision averages the class-balanced accuracy of two separate models that classify dialog acts (e.g., "greet", "initial request", etc.) and domains ("hotel", "restaurant", etc.) respectively. Few-shot averages the classbalanced accuracy of models classifying dialog acts with 1-shot, 5-shot, and 10-shot settings.
training or hyperparameter tuning.

heading: 5.1 Main Results
text: Table 1 summarizes the results of NEUPSL DSI and DD-VRNN in strictly unsupervised settings. NEUPSL DSI outperforms the strictly data-driven DD-VRNN on AMI by 4%-27% depending on the setting while maintaining or improving the hidden representation quality. To reiterate, this improvement is achieved without supervision in the form of labels, but rather a few selected structural constraints. Comparing AMI performance on SGDreal across different settings (standard generalization v.s. domain generalization/adaptation), we see the NEUPSL DSI consistently improves over DDVRNN, albeit with the advantage slightly diminished in the non-standard generalization settings.
To further understand how the constraints affect the model, we examine three highly constrained few-shot settings (1-shot, 3-shot, and proportional 1-shot) trained using the loss described in Equation 4. The 1-shot and 3-shot settings are given one and three labels per class, while proportional 1-shot is provided the same number of labels as 1-shot with the distribution of labels proportional to the class size (classes below 1% are not provided labels). The results in Figure 3 show that in all settings, the introduction of labels improves performance.
This demonstrates that the soft constraints do not overpower learning but enable a trade-off between generalizing to priors and learning over labels. In the SGD settings, however, as the number of labels increases, the pure data-driven approach performs as well or better than NEUPSL DSI.

heading: 5.2 Ablation Study
text: We provide an ablation on the SGD real dataset over three major method axes: parameterization of the constraint loss (linear v.s. log constraint loss, Section 4.2), weighting scheme for the bagof-words loss (uniform v.s. tf-idf weights, Section 4.3), and the choice of underlying utterance embedding (BERT (Devlin et al., 2019) v.s. GloVe (Pennington et al., 2014)) leading to a total of 23 = 8 settings (Appendix D.3 presents a further analysis for the MultiWoZ and SGD Synthetic datasets). Table 2 summarizes the results for the SGD data set. Highlighted in bold are the highest-performing setting or methods within a standard deviation of the highest-performing setting. For structure induction, using a BERT embedding and uniform bag-of-words-weights generally produces the best AMI performance, while there is no significant difference between linear and log constraints. How-
ever, when examining the hidden representation it is clear that the log relaxation outperforms or performs as well as its linear counterpart. Additionally, Figure 4 summarizes the few-shot training results for the SGD data settings when training with 1-shot, proportional 1-shot, and 3-shots. We see three methods generally on top in performance: uniform-log-bert, tf-idf-linear-bert, and uniformlinear-bert. There seems to be no clear winner between uniform/tf-idf and linear/log; however, all three of these settings use BERT embeddings.

heading: 6 Discussion and Conclusions
text: This paper introduces NEUPSL DSI, a novel neurosymbolic learning framework that uses differentiable symbolic knowledge to guide latent dialog structure learning. Through extensive empirical evaluations, we illustrate how the injection of just a few domain knowledge rules significantly improves both correctness and hidden representation quality in this challenging unsupervised NLP task.
While NEUPSL DSI sees outstanding success in the unsupervised settings, the introduction of additional labels highlights a potential limitation of NEUPSL DSI. If the domain knowledge introduced is weak or noisy (as in the SGD setting), when the model is provided with more substantial evidence, this additional noisy supervision can, at times, hurt generalization. Therefore, enabling models to perform weight learning, where the model adaptively weights the importance of symbolic rules as stronger evidence is introduced, is an interesting future direction (Karamanolakis et al., 2021).

heading: Acknowledgments
text: This work was partially supported by the National Science Foundation grant CCF-2023495.

heading: A Model Details
text: This section provides additional details on the NEUPSL DSI models for the Multi-WoZ and SGD settings. Throughout these subsections, we cover the symbolic constraints and the hyperparameters used. All unspecified values for the constraints or the DD-VRNN model were left at their default values. The code is under the Apache 2.0 license.

heading: A.1 SGD Constraints
text: The NEUPSL DSI model uses a single constraint for all SGD settings (synthetic, standard, domain generalization, and domain adaptation). Figure 5 provides an overview of the constraint, which contains the following two predicates:
1. STATE(Utt, Class) The STATE continuous valued predicate is the probability that an utterance, identified by the argument Utt, belongs to a dialog state, identified by the argument Class. For instance, the utterance hello world ! for the greet dialog state would create a predicate with a value between zero and one, i.e., STATE(hello world !greet) = 0.7.
2. HASWORD(Utt, Class) The HASWORD binary predicate indicates if an utterance, identified by the argument Utt, contains a known token for a particular class, identified by the argument Class. For instance if a known
# Token Constraint w1 : HASWORD(Utt, Class) → STATE(Utt, Class)
This token constraint encodes the prior knowledge that utterances’ are likely to belong to dialog states when an utterance contains tokens representing that state. For example, if a known token associated with the greet class is hello, then the utterance hello world ! is likely to belong to the greet state. The primary purpose of incorporating this constraint into the model is to show how even a small amount of prior knowledge can aid predictions. To get the set of tokens associated with each state, we trained a supervised linear classifier where the input is an utterance, and the label is the class. After training, every token is individually run through the trained model to get a set of logits over each class. These logits represent the relative importance that each token has over every class. Sparsity is introduced to this set of logits, leaving only the top 0.1% of values and replacing the others with zeros. This sparsity reduces the set of 261,651 logits to 262 non-zero logits.

heading: A.2 Multi-WoZ Constraints
text: The NEUPSL DSI model for the Multi-WoZ setting uses a set of dialog constraints, which can be broken into dialog start, middle, and end. Figure 6 provides an overview of the constraints, which contains the following 11 predicates:
1. STATE(Utt, Class) The STATE continuous valued predicate is the probability that an utterance, identified by the argument Utt, belongs to a dialog state, identified by the argument Class. For instance, the utterance hello world ! for the greet dialog state would create a predicate with a value between zero and one, i.e., STATE(hello world !greet) = 0.7.
2. FIRSTUTT(Utt) The FIRSTUTT binary predicate indicates if an utterance, identified by the argument Utt, is the first utterance in a dialog.
4. PREVUTT(Utt1, Utt2) The PREVUTT binary predicate indicates if an utterance, identified by the argument Utt2, is the previous utterance in a dialog of another utterance, identified by the argument Utt1.
5. HASGREETWORD(Utt) The HASGREETWORD binary predicate indicates if an utterance, identified by the argument Utt, contains a known token for the greet class. The list of known greet words is [′hello′,′ hi′].
6. HASINFOQUESTIONWORD(Utt) The HASINFOQUESTIONWORD binary predicate indicates if an utterance, identified by the argument Utt, contains a known token for the info question class. The list of known info question words is [′address′,′ phone′].
7. HASSLOTQUESTIONWORD(Utt) The HASSLOTQUESTIONWORD binary predicate indicates if an utterance, identified by the argument Utt, contains a known token for the slot question class. The list of known slot question words is [′what′,′ ?′].
8. HASINSISTWORD(Utt) The HASINSISTWORD binary predicate indicates if an utterance, identified by the argument Utt, contains a known token for the insist class. The list of known insist words is [′sure′,′ no′].
9. HASCANCELWORD(Utt) The HASCANCELWORD binary predicate indicates if an utterance, identified by the argument Utt, contains a known token for the cancel class. The list of known cancel words is [′no′].
10. HASACCEPTWORD(Utt) The HASACCEPTWORD binary predicate in-
dicates if an utterance, identified by the argument Utt, contains a known token for the accept class. The list of known accept words is [′yes′,′ great′].
11. HASENDWORD(Utt) The HASENDWORD binary predicate indicates if an utterance, identified by the argument Utt, contains a known token for the end class. The list of known end words is [′thank′,′ thanks′].
The dialog start constraints take advantage of the inherent structure built into the beginning of task-oriented dialogs. In the same order as the dialog start rules in Figure 6: 1) If the first turn utterance does not contain a known greet word, then it does not belong to the greet state. 2) If the first turn utterance contains a known greet word, then it belongs to the greet state. 3) If the first turn utterance does not contain a known greet word, then it belongs to the initial request state.
The dialog middle constraints exploit the temporal dependencies within the middle of a dialog. In the same order as the dialog middle rules in Figure 6: 1) If the previous utterance belongs to the greet state, then the current utterance belongs to the initial request state. 2) If the previous utterance does not belong to the greet state, then the current utterance does not belong to the initial request state. 3) If the previous utterance belongs to the initial request state, then the current utterance belongs to the second request state. 4) If the previous utterance belongs to the second request state
and it has a known info question token, then the current utterance belongs to the info question state. 5) If the previous utterance belongs to the second request state and it has a known slot question token, then the current utterance belongs to the slot question state. 4) If the previous utterance belongs to the end state and it has a known cancel token, then the current utterance belongs to the cancel state.
The dialog end constraints take advantage of the inherent structure built into the end of task-oriented dialogs. In the same order as the dialog end rules in Figure 6: 1) If the last turn utterance contains a known end word, then it belongs to the end state. 2) If the last turn utterance contains a known accept word, then it belongs to the accept state. 3) If the last turn utterance contains a known insist word, then it belongs to the insist state.

heading: B Additional Model Details

heading: B.1 Symbolic-rule Normalization in the Multi-class Setting
text: In the multi-class setting (e.g., multiple latent states), some soft logic operation on the model probability pw will lead to a probability that no longer normalizes to 1. For example, the negation operation on the probability vector pw will lead to !pw = 1 − pw; then in the multi-class setting, the norm of !pw is ∑|C| i (1 − pi) = |C| − 1 > 1, where |C| is the number of classes. To address the above concern, we re-normalize after every soft
logic operation:
fw(y,x) = fw(y,x)/||fw(y,x)||,
where fw(y,x) is the output of a soft logical operation.

heading: B.2 Model Hyperparameters
text: The DD-VRNN uses an LSTM (Hochreiter and Schmidhuber, 1997) with 200-400 units for the RNNs, and fully-connected highly flexible feature extraction functions with a dropout of 0.4 for the input x, the latent vector z, the prior, the encoder and the decoder. The input to the DD-VRNN is the utterances with a 300-dimension word embedding created using a GloVe embedding (Pennington et al., 2014) and a Bert embedding (Devlin et al., 2019). The maximum utterance word length was set to 40, the maximum length of a dialog was set to 10, and the tunable weight, γ (Equation 3), was set to 0.1. The total number of parameters is 26,033,659 for the model with GloVe embedding and 135,368,227 with Bert embedding. The experiments are run in Google TPU V4, and the total GPU hours for all finetuning are 326 GPU hours.

heading: C Datasets
text: This section provides additional information on the SGD, SGD synthetic, and MultiWoZ 2.1 synthetic datasets.

heading: C.1 SGD
text: The Schema-Guided Dialog (SGD) (Rastogi et al., 2020) is a task-oriented conversation dataset involving interactions with services and APIs covering 20 domains. There are overlapping functionalities
over many APIs, but their interfaces differ. One conversion may involve multiple domains. The train set contains conversions from 16 domains, with four held-out domains only present in test sets. This gives 34,308 in-domain and 5,441 outof-domain test examples. To evaluate the model’s generalization, we evaluate the model performance on both test sets. In specific, we establish three different evaluation protocols.
• SGD Standard Generalization We train the model using the SGD train set and evaluate it on the in-domain test set.
• SGD Domain Generalization We train the model using the SGD train set and evaluate it on the out-of-domain test set.
• SGD Domain Adaptation We train the model using the SGD train set and label-wiped indomain and out-of-domain test sets and evaluate it on the out-of-domain test set.

heading: C.2 SGD Synthetic
text: Using the template-based generator from the SGD developers Kale and Rastogi (2020), we generate 10,800 synthetic dialogs using the same APIs and dialog states as the official SGD data. We split the examples with 75% train and 25% test. The schema-guided generator code is under Apache 2.0 license: https://github.com/google-research/taskoriented-dialogue/blob/main/LICENSE.

heading: C.3 MulitWoZ 2.1 Synthetic
text: MultiWoZ 2.1 synthetic (Campagna et al., 2020) is a multi-domain goal-oriented dataset covering five domains (Attraction, Hotel, Restaurant, Taxi, and Train) and nine dialog acts
(greet, initial request, second request, insist, info question, slot question, accept, cancel, and end). Following Campagna et al. (2020), we generate 104 synthetic dialogs from a known ground-truth dialog structure. Figure 7 provides an overview of the ground truth dialog structure, which is based on the original MultiWoz 2.1 dataset (Eric et al., 2019), used through the generative process. These 104 synthetic dialogs are randomly sampled without replacement to create ten splits with 80% train, 10% test, and 10% validation. The MultiWoZ 2.1 synthetic code is under the MIT License: https://github.com/stanfordoval/zero-shot-multiwoz-acl2020. The MultiWoZ 2.1 code uses genie under the MIT License: https://github.com/stanford-oval/geniek8s/blob/master/LICENSE.

heading: D Extended Experimental Evaluation
text: In this section, we provide extended experimental evaluations on the NEUPSL DSI models for all settings. We split the extended evaluation into evaluation metrics, main results, ablation results, and additional experiments. Details describing changes to the models are provided in each subsection.

heading: D.1 Evaluation Metrics
text: Adjusted Mutual Information (AMI) - AMI evaluates dialog structure prediction by evaluating the correctness of the dialog state assignments. Let U∗ = {U∗1 , . . . , U∗C∗} be the ground-truth assignment of dialog states for all utterances in the corpus, and U = {U1, . . . , UC} be the predicted assignment of dialog states based on the learned dialog structure model. U∗ and U are not directly comparable because they draw from different base sets of states (U∗ from the ground truth set of states and U from the collection of states induced by the DDVRNN) that may even have different cardinalities. We address this problem using Adjusted Mutual Information (AMI), a metric developed initially to compare unsupervised clustering algorithms. Intuitively, AMI treats each assignment as a probability
distribution over states and uses Mutual Information to measure their similarity, adjusting for the fact that larger clusters tend to have higher MI. AMI is defined as follows:
AMI(U,U∗) =
MI(U,U∗)− E(MI(U,U∗)) Avg(H(U), H(U∗))− E(MI(U,U∗))
Where MI(U,U∗) is the mutual information score, E(MI(U,U∗)) is the expected mutual information over all possible assignments, and Avg(H(U), H(U∗)) is the average entropy of the two clusters (Vinh et al., 2010).
Purity - Let U∗ = {U∗1 , . . . , U∗C∗} be the ground-truth assignment of dialog states for all utterances in the corpus, and U = {U1, . . . , UC} be the predicted assignment of dialog states based on the learned dialog structure model. Each cluster is assigned to the class which is most frequent in the cluster. This assignment then calculates accuracy by summing together the total correct of each cluster and dividing by the total number of clusters. Purity is defined as follows:
Purity(U,U∗) = 1
N
K∑
k=1
Count(U,U∗, Ak)
whereK is the number of unique clusters predicted, N is the total number of predicted utterances,Ak is the most frequent underlying ground truth in cluster k, and Count(U,U∗, Ak) is the total number of correctly labeled utterances within that assigned cluster.

heading: D.2 Main Results
text: This section provides additional experimental results for the structure induction and hidden representation learning performance. Table 3 summarizes the extended evaluation of the main results for the NEUPSL DSI model and DD-VRNN baseline on an additional metric: purity. These values correlate with the reported results in Table 1, i.e.,
these are not the best-performing results but are calculated using the model that produced the best AMI results. Purity follows a similar trend as AMI, where NEUPSL DSI outperforms the DD-VRNN in most settings. In addition, Figure 8 summarizes the few-shot results for purity and each hidden representation setting (full and few-shot). Similar to the AMI, the introduction of labels improves performance across all settings.

heading: D.3 Ablation Results
text: This section provides an extended ablation for the SGD real setting and full ablations for the SGD synthetic and MultiWoZ datasets. Each ablation analysis studies structure induction and hidden representation learning (Section 5) over various neural settings.
Table 4 summarizes the unsupervised results for the MulitWoZ data setting over four major method
axes: parameterization of the constraint loss (linear v.s. log constraint loss, Section 4.2), weighting scheme for the bag-of-words loss (uniform v.s. tfidf weights, Section 4.3), constraint normalization (standard v.s. normalized, Section B.1), and the choice of underlying utterance embedding (BERT (Devlin et al., 2019) v.s. GloVe (Pennington et al., 2014)) leading to a total of 24 = 16 settings.
Table 5 summarizes the unsupervised results for the SGD synthetic data setting over two major method axes: parameterization of the constraint loss (linear v.s. log constraint loss, Section 4.2), and weighting scheme for the bag-of-words loss
(uniform v.s. tf-idf weights, Section 4.3) leading to a total of 22 = 4 settings.
Figure 9, Figure 10, and Figure 11 summarize the few-shot training results for the MultiWoZ, SGD synthetic, and SGD real (Standard Generalization, Domain Generalization, and Domain Adaptation) data settings when training with 1 shot, proportional 1 shot, and 3 shots.

heading: D.4 Additional Experiments
text: Throughout this section, we provide additional dialog structure experiments to understand further when injecting common-sense knowledge as struc-
tural constraints is beneficial. The additional experiments are broken into 1) A study of the sparsity introduced into the tokens in the SGD synthetic setting and 2) An exploration of an alternative soft logic formulation.

heading: D.4.1 Sparsity
text: In this experiment, we explore varying the sparsity introduced to the token weights, as described in Appendix A.1. Table 6 shows the performance for the hidden representation and structure induction tasks. When the percent of non-zero word weights is 100.00%, this implies the model is trained on full supervision, while the non-zero word weights at 0.00% represent the unsupervised DD-VRNN results. Surprisingly, we see substantial improvement in all data settings. Even when the non-zero word weight percentage is 0.02%, resulting in 54 nonzero weights, we still see approximately a 20% improvement to the AMI. Note 54 non-zero weights are equivalent to about two identifiable tokens per
class.

heading: E Alternative Soft Logic Approximations
text: This work is highly related to the active field of neuro-symbolic computing (NeSy) (d’Avila Garcez et al., 2002; Bader and Hitzler, 2005; d’Avila Garcez et al., 2009; Serafini and d’Avila Garcez, 2016; Besold et al., 2017; Donadello et al., 2017; Yang et al., 2017; Evans and Grefenstette, 2018; Manhaeve et al., 2021; d’Avila Garcez et al., 2019; De Raedt et al., 2020; Lamb et al., 2020; Badreddine et al., 2022). NeSy methods aim to incorporate logic-based reasoning with neural networks. As such, various principled soft/fuzzy logic formulations have shown to work quite well as knowledge used within the loss of a NeSy approach (van Krieken et al., 2022; Diligenti et al., 2016; Badreddine et al., 2022; Manhaeve et al., 2021). While this work primarily focuses on using Lukasiewicz logic (Klir and Yuan, 1995), we are interested in
how different soft/fuzzy logic formulations affect the learning process. We, therefore, explore an alternative soft logic formulation, Product Real logic, which is used in another principled NeSy framework called Logic Tensor Networks (Badreddine et al., 2022). Similar to the Lukasiewicz logic, Product Real logic approximates logical clauses with
linear inequalities:
A ∧B = A ∗B A ∨B = A+B −A ∗B
¬A = 1.0−A
where A and B are either ground atoms or logical expressions over atoms. In either case, they have values between [0,1].
Table 7 summarizes the unsupervised results for the MulitWoZ data setting over three major method axes: the soft logic approximation (Lukasiewicz v.s. Product Real), parameterization of the constraint loss (linear v.s. log constraint loss, Section 4.2), and weighting scheme for the bag-of-words loss (uniform v.s. tf-idf weights, Section 4.3) leading to a total of 23 = 8 settings. Surprisingly, in structure induction, Lukasiewicz logic outperformed Product Real logic by over 7%. Though interestingly, the hidden representation learning performance was roughly equivalent between the two soft logic formulations. These findings highlight the importance of selecting the appropriate soft/fuzzy logic formulation for a given problem. Further investigation into different soft logic formulations would be an interesting direction for future research to gain a more comprehensive understanding of their potential applications.
Figure 12 summarizes the few-shot training results for the MultiWoZ synthetic data settings when training with 1 shot, proportional 1 shot, and 3 shots. Noticeably, the Product Real logic closes the gap with the introduction of labels. However, the Lukasiewicz logic still has an edge when observing the largest semi-supervised setting.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: A1. Did you describe the limitations of your work?
Left blank.
A2. Did you discuss any potential risks of your work? Left blank.
A3. Do the abstract and introduction summarize the paper’s main claims? Left blank.
A4. Have you used AI writing assistants when working on this paper? Left blank.

heading: B Did you use or create scientific artifacts?
text: Left blank.
B1. Did you cite the creators of artifacts you used? Left blank.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Left blank.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.

heading: C Did you run computational experiments?
text: Left blank.
C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Left blank.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Left blank.
C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Left blank.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Left blank.

heading: D Did you use human annotators (e.g., crowdworkers) or research with human participants?
text: Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Left blank.

heading: Future Work
text: Further investigation into different soft logic formulations would be an interesting direction for future research to gain a more comprehensive understanding of their potential applications. Figure 12 summarizes the few-shot training results for the MultiWoZ synthetic data settings when training with 1 shot, proportional 1 shot, and 3 shots. Noticeably, the Product Real logic closes the gap with the introduction of labels. However, the Lukasiewicz logic still has an edge when observing the largest semi-supervised setting. ACL 2023 Responsible NLP Checklist