KG-CRuSE: Recurrent Walks over Knowledge Graph for Explainable Conversation Reasoning using Semantic Embeddings
Abstract
Knowledge-grounded dialogue systems utilise external knowledge such as knowledge graphs to generate informative and appropriate responses.
A crucial challenge of such systems is to select facts from a knowledge graph pertinent to the dialogue context for response generation.
This fact selection can be formulated as path traversal over a knowledge graph conditioned on the dialogue context.
Such paths can originate from facts mentioned in the dialogue history and terminate at the facts to be mentioned in the response.
These walks, in turn, provide an explanation of the flow of the conversation.
This work proposes KG-CR U SE, a simple, yet effective LSTM based decoder that utilises the semantic information in the dialogue history and the knowledge graph elements to generate such paths for effective conversation explanation.
Extensive evaluations showed that our model outperforms the stateof-the-art models on the OpenDialKG dataset on multiple metrics.
Introduction
Inducing factual information during response generation has garnered a lot of attention in dialogue systems research.
While language models (Zhao et al., 2020; Zheng et al., 2020) have been shown to generate responses akin to the dialogue history, they seldom contain factual information, leading to a bland conversation with the agent.
Knowledgegrounded dialogue systems focus on leveraging external knowledge to generate coherent responses.
Knowledge Graphs (KGs) are a rich source of factual information and can be combined with an utterance generator for a natural and informative conversational flow.
Zhou et al.
(2018) showed that utilising KGs in dialogue systems improves the appropriateness and informativeness of the conversation.
Augmenting utterances in a dialogue with the KG information
Could you recommend me movies similar to The Dark Knight ?
The sequel to Batman Begins is The Dark Knight .
Okay.
Who is the director of The Dark Knight and any other movies from him not related to Batman ?
Christopher Nolan was the director.
He also directed Insomnia and Inception .
Christopher Nolan
Inception
The Dark Knight
English IMAX
U2 3D
Batman Begins Batman
D i r ec t o r o f
D i r ec t e d B y L a ngu a g e
S e qu e l
FilmFormat is film format of S ub j ec t o f    guides the conversational agent to include relevant entities and facts in the response.
For example,
Figure 1 shows an example conversation where a user is interacting with a dialogue agent about movies.
The agent has access to a KG that aids in suggesting relevant facts during the dialogue flow.
When responding to utterance 3, the agent can utilise information from the KG and produce relevant facts about “Christopher Nolan”.
This information would be more engaging than responding with information about “Batman” or “Batman
Begins”.
While KGs have been used extensively to include relevant facts in a dialogue, the explicability of such systems is limited.
Naturally, this fostered research on developing models for explainable conversation reasoning.
Moon et al.
(2019) addressed this problem by inducing KG paths for conversation explainability.
They posited a dialogue-KG path aligned corpus wherein utterances are augmented with a KG path to denote fact transitions in the dialogue.
The KG paths emanate from entities or facts mentioned in the dialogue history and terminate at the entity to be mentioned in the response text.
Such paths form a sequence of entities and relations and aid the dialogue agent in introducing appropriate knowledge to the dialogue.
In addition to this, they proposed an attention-based recurrent decoder over the KG to generate entity paths.
Jung et al.
(2020) designed a novel dialogue-context infused graph neural network to propagate attention scores over the knowledge graph entities for KG path generation.
While such approaches have their inherent strengths, their limitations are manifold.
Given a dialogue context, it is desirable to generate paths that results in a natural dialogue flow.
Therefore it is essential to capture the semantic information in the dialogue context as well as the
KG elements.
Transformer based models (Devlin et al., 2019; Lan et al., 2020;
Liu et al., 2019a) have enabled the capture of contextual relationships between different words in a sentence.
Textual representations from such models have been successfully adapted for the dialogue conditioned
KG reasoning task (Jung et al., 2020).
However, prior works use the embedding of the [ CLS ] token to encode the dialogue history and the KG elements.
Reimers and Gurevych (2019) demonstrated that such sentence embeddings are sub-optimal and lead to degraded performance in downstream application tasks.
Sentence-transformers (Reimers and
Gurevych, 2019) are strong tools for capturing the semantic information of a sentence into a fixed-size vector.
As KG elements can be long phrases, KGCR U SE uses the Sentence-BERT (SBERT) model to encode both the dialogue history and the KG elements for capturing their semantic information.
As a result of the long tailed distribution of node neighbors in a KG, it can become difficult to generate relevant paths over the KG for explainable conversation.
Given the dialogue history, it is desirable to traverse paths that are semantically relevant.
KG-CR U SE utilises the rich sequential information in the dialogue history and the path history to sample the top-k semantically similar neighbors for extending its walk over the KG.
We show that our KG-CR U SE improves upon the current state-of-the-art on multiple metrics, demonstrating the effectiveness of KG-CR U SE for explainable conversation reasoning.
To summarise, our contributions are as follows:
• We propose a KG-CR U SE , a LSTM based decoder leveraging Sentence-Transformer (SBERT) embedding to reason KG paths for explainable conversation.
• We show the efficacy of our model by improving the current state-of-the-art performances over multiple metrics on the OpenDialKG (Moon et al., 2019)
dataset.
Additionally, we conduct extensive empirical analysis to emphasise the effectiveness of KG-CR U SE for the reasoning task.
• We release 1 our system and baseline systems as an open-source toolkit to allow reproducibility and future comparison on this task.
Related Work
The use of external knowledge in dialogue agents has become commonplace, owing to the rich heterogeneous information contained in them.
He et al.
(2017) addressed the knowledge-grounded conversation task by iteratively updating the knowledge base embeddings to generate informative responses.
Following this, knowledge-based dialogue systems have been studied extensively including the collection of new knowledge-grounded datasets (Ghazvininejad et al., 2018;
Qin et al., 2019; Zhang et al., 2018) and developing knowledge-centric dialogue systems (Liu et al., 2018; Parthasarathi and
Pineau, 2018a; Zhang et al., 2020).
Young et al.
(2018a) attempted to integrate a large scale KG into an end-to-end dialogue system.
Other similar works (Chen et al., 2019; Zhou et al., 2020; Sarkar et al., 2020) leveraged graph neural networks and KG embeddings to recommend relevant products in conversational recommender systems.
Though successful in retrieving suitable entities or facts from the KG, these systems fail to provide explainability to the recommendations.
Such limitations encouraged explainable conversation reasoning using external knowledge.
Liu et al.
(2019b) develop the problem as a Partially
Sentence Bert
Encoder
L S
T M h.
Sentence Bert
Encoder r 11 ;er 1m ;e 1m r 12 ;e[ r 11 ; e 11 ] [ r 12 ; e 12 ] [ r 11 ; e 1m ]
P path h
W 1 W 2 W 3 . . .
W n
Available Actions
Action Scores
Observable Markov Decision Process and use policy gradient for training the agent to generate KG paths. Moon et al. (2019) posited a KG pathparallel-dialogue corpus along with DialKG Walker (DKGW) model, a recurrent decoder model to generate the KG path for a response entity selection. Jung et al.
(2020) suggested the use of graph neural networks using attention flow to generate KG entity paths. While novel, DKGW does not explicitly utilise the graph structure during model training. On the other hand, the performance of AttnIO (Jung et al., 2020) relies on the node sampler during training. AttnIO becomes computationally expensive due to dialogue specific graph neural network (both during training and inference) as the model concatenates the dialogue embedding to the node embeddings while propagating attention scores.
To counter these issues, we design a very simple, lightweight, yet efficient LSTM network leveraging the dialogue and path history to extend the path over the KG.
While, DKGW uses TransE (Bordes et al., 2013) for encoding the elements of the KG, such translation embeddings have weak representation capacity.
On the other hand, Jung et al.
(2020) utilise the ALBERT (Lan et al., 2020) representation of sentence to encode the dialogue history and the KG elements.
They use the [ CLS ] token representation of the text sequence as the sentence representation.
Reimers and Gurevych (2019) suggested SentenceTransformers for encoding sentences.
We encode the dialogue history and the KG elements using
Sentence-Transformers to capture rich semantic similarities between the dialogue history and the
KG elements.
The processing of semantically rich sequential information using a lightweight LSTM model makes KG-CR U SE ideal for generating walks over a KG for explainable conversation.
Methodology
In the following sections, we begin with formally introducing the problem statement.
We then outline the embeddings used in KG-CR U SE.
Following this, we discuss the architecture of KG-CR U SE as illustrated in Figure 2.
Finally, we describe decoding process used by KG-CR U SE during the inference step.
3. Formal Problem Definition
We describe the problem statement similar to Moon et al.
The KG is defined as G
= V KG ×
R KG × V KG , where V KG is set of entities and R KG is set of relations in the KG.
Facts in the KG are denoted by triples, and each has the form ( e, r, e’ ) where e, e’ ∈ V KG are entities and r ∈ R KG is the relation connecting them.
In addition to the KG, each input contains a dialogue D ∈ D , represented as a sequence of utterances D = { s 1 , ..., s n }, and the set of entities x e = {x ( i ) e } occurring in the user’s last utterance s n , where x ( i ) e ∈ V KG .
The output is represented as y =
{ y e , y r } , where y e is a set of entity paths y e
=
{y ( i ) e }, with each element y ( i ) e = {y ( i ) e,t } Tt =1 denoting an entity path connecting x
( i ) e to the response entity y ( i ) e,T .
Likewise, y r = {y ( i ) r } is a set of relation paths, where y ( i ) r ∈ R KG .
The element y ( i ) r
= {y ( i ) r,t } Tt = is a sequence of relations from the KG connecting x ( i ) e and {x ( i ) e,t } Tt =1 .
3. Dialogue and KG Representation
Capturing the semantic information in the dialogue context is an important component of our model.
SBERT is a contextual sentence encoder that captures the semantic information of a sentence in a fixed-size vector.
We encode pieces of text using
Equation 1.
The text is first sent though a pretrained BERT model to obtain the contextual representation of its tokens.
The sentence embedding is computed by taking a mean-pool of the contextual token representations.
The dialogue context is constructed by concatenating a maximum of three previous utterances and is then passed through SBERT encoder to obtain a fixed-size contextual dialogue representation.
[Equation]
In order to align the semantic vector space of the dialogue representations and the KG representations, we use SBERT to encode the KG elements.
As KG entities and relations can be words or phrases, SBERT can effectively capture their semantic information.
We use the publicly available SBERT-BERT-BASE-NLI 2 model with meanpooling as our SBERT encoder.
3. KG-CR U SE Architecture
KG-CR U SE learns to traverse a path on the KG by learning a function π θ that calculates the probability of an action a t ∈
A t given the state s t .
The state s t contains the dialogue history and entities already traversed by KG-CR U SE while decoding the paths, while a t is the set of edges from the KG available to KG-CR U SE for extending its path.
[Figure]
The state s t at step t is defined as a tuple (D, ( r 1 , e 1 , ..., r t − 1 , e t − 1 )), where D is the dialogue context and r i , e i ( i < t ) are the relation and entity already decoded by KG-CR U SE at step i .
The initial state s 0 is denoted as ( D , ∅ ), where ∅ is the empty set.
At step t , an action has the form a t =
( r t , e t ) ∈
A t , where A t is the set of all possible actions available to the model at step t .
A t includes all outgoing edges of e t − 1 in the KG G , i.e. A t is the set of all the outgoing edges of the entity decoded by
KG-CR U SE at timestep t − 1 .
To let the agent terminate the search process, we add self-loop edges to every entity node in the graph denoting no operation ("self-loop").
The action a t is represented as a concatenation of the relation and entity embedding a t =
[ r t ; e t ] , where r ∈ R d r , e ∈ R d e and
R d e , R d r are the size of the entity embedding and relation embedding respectively.
At step 1, KGCR U SE chooses between the entities mentioned in s n for path traversal.
The relation associated with action at step 1 is the zero vector.
As mentioned, the state s t contains the dialogue context and action history (path history).
This sequential information in s t is modelled using an LSTM:
[Equation]
where D is the contextual dialogue embedding obtained using Equation 1 and W d is a learnable matrix that maps the dialogue embedding to the LSTM input dimension.
Given the hidden state representation h t at time t , KG-CR U SE assigns a probability to each action using Equation 6.
[Equation]
The hidden state representation h t is passed through a two-layered dense network with ReLU activation (Nair and Hinton, 2010) in the first layer.
The LSTM weights, W 2 ,θ ∈ R d h × d h and
W 3 ,θ ∈ R ( d r + d e ) × d h are the learnable parameters, and d h is the LSTM hidden representation size.
3.3. Model Learning
We train KG-CR U SE by minimising the crossentropy loss on the entities decoded at each timestep.
Additionally, we train the model using teacher forcing (Sutskever et al., 2014), wherein the model makes each action conditioned on the gold history of the target path.
To prevent overfitting, we add L 2 regularisation to the parameters of the model.
During training, we do not fine-tune the SBERT architectures, but back-propagate the gradients to the entity and relation embeddings.
3.3. KG-CR U SE Path Generation
Once the model is trained, KG-CR U SE takes the dialogue history and the entities mentioned in the current utterance as input, a horizon T and outputs a set of entity paths, relations paths of length T along with the probability score of each path.
During inference, we remove self-loops from the KG except for the self-loop with label "self-loop" introduced in section 3.3.
We do so to allow the agent traverse diverse paths rather than staying at entities mentioned in the dialogue history.
Experimental Setup
This section presents the dataset used, the baselines compared with and the description of the model settings of KG-CR U SE along with the metrics the models have been evaluated on.
4. Dataset
We evaluate our proposed framework on the OpenDialKG dataset (Moon et al., 2019).
The dataset has 91,209 turns spread over 15,673 dialogues in the form of either task-oriented dialogues (recommendations) or chit-chat conversations on a given topic.
Each turn is annotated with a KG path to represent fact transitions in the conversation.
The
KG is a subset of the Freebase KG (Bollacker et al., 2008), which has 1,190,658 fact triples, 100, entities and 1,358 relations.
Following Moon et al. (2019), we split the dataset randomly into 70%, 15% and 15% for training, testing and validation.
4. Baselines and Evaluation Metrics
We compare KG-CR U SE against the following baseline models suggested by Moon et al.
(2019) and Jung et al.
• Tri-LSTM (Young et al., 2018b):
The model encodes each utterance along with facts from the KG within 1-hop distance from the entities mentioned in the current utterance.
This is used to retrieve facts from the KG for dialogue explanation.
• Ext-ED (Parthasarathi and Pineau, 2018b):
Moon et al.
(2019) conditioned the response generation with external knowledge vector input to generate response entity token at the final softmax layer, without using the structural information from the KG.
• Seq2Path
(Jung et al., 2020): An attention based Seq2Seq model is modified to generate entity paths by masking out unreachable nodes at each decoding step.
• Seq2Seq: An LSTM based seq2seq (Sutskever et al., 2014) model where the decoder is modified to generate entity paths.
Similar to DKGW (Moon et al., 2019) model, we use modality attention as the output of the encoder.
Following Moon et al.
(2019), we replace the softmax layer in the decoder with a zero-shot learning layer in the KG embedding space.
• DKGW (Moon et al., 2019):
A model to generate KG paths using domain-agnostic, attention-based recurrent graph decoder reinforced with a zero-shot learning layer over the KG embedding space.
• AttnIO (Jung et al., 2020):
A dialogue conditioned KG path traversal leveraging attention flow using graph neural networks.
Since the authors of OpenDialKG and AttnIO have not released their implementations, we report their performance on our re-implementations.
We note that for most systems, our implementation is similar or better than the reported results.
Regarding AttnIO, we were not able to reproduce the results although we note that errors in the implementation of the node sampler or leakage of the test dataset into the training dataset can easily lead to overestimation of the accuracy.
The code and dataset used for re-implementation as well as our system is accessible at https://github. com/rajbsk/kg-cruse .
We evaluate our models on different recall@k metrics for entity and path retrieval.
Path@k measures if the ground-truth path is present in the top-k paths with the highest probability searched by the agent.
Similarly, tgt@k measures if the response entity is present in the top-k entities retrieved by the agent.
In situations where multiple paths point to the same response entity, we consider the path with the highest score for entity retrieval.
[Table]
4. Implementation Details
For the task, we set horizon T to 3.
The dialogue, entity and relation embeddings are encoded using SBERT into a 768 dimensional vector.
In
KG-CR U SE, we consider 3 LSTM layers with d h =
d e + d r = 1 , 536 .
To prevent the agent from overfitting on the dataset, we add L 2 regularisation with a weight decay parameter of 1e-3.
Similar to Jung et al.
(2020), we set the batch size to 8 and train the model with Adam optimiser (Kingma and Ba, 2015) with a learning rate of 1efor 20 epochs.
For models with re-implementations, we report the results on five different splits of the data. For Tri-LSTM and Ext-ED, we report the number reported by Moon et al. (2019), while for Seq2Path, the numbers are reported from the work of Jung et al.
As entity occurrences in a dialogue dataset is sparse, it is desirable to report the performance on five different splits of the data rather than an assessment of five models on one split.
Results and Discussion
We begin with performing a quantitative evaluation of the models.
Following this, we study the impact of our choice of sentence embeddings on the model performance.
Then we analyse the impact of beam-width at each decoding step during inference.
Finally, we provide insights of examples where the results of KG-CR U SE are not consistent with the ground truth paths.
5. Quantitative Analysis
In this section, we compare the performance of our proposed approach against the different baselines.
From Table 1, it can be observed that KGCR U SE performs better than the different baseline [Table] models on the OpenDialKG dataset.
For entity and path accuracy, AttnIO has the closest performance compared to our model, with the latter being 2.7% relatively better on both path@1 and entity@1 metrics.
On increasing k of recall@k, we find KG-CR U SE has at least 10% relative improvement over the baseline models.
It is interesting to notice that on increasing the value “k”, KGCR U SE performs relative better than other models.
KG-CR U SE identifies paths semantically relevant to the dialogue context although different from the gold-label paths as discussed in Section 5.6.
The huge gain on the path@25 metric advocates for this hypothesis.
It is worthwhile to notice that although AttnIO has the closest performance for path@ and entity@1 to KG-CR U SE, the performance degrades when “k” increases in path@k and entity@k.
This might be due to the fact that the beam-width reported by the authors is not expressive enough to capture semantically relevant paths or entities.
5. Effectiveness of Sentence Embeddings
In our framework, we utilise sentence SBERT embeddings to encode dialogue context and KG elements.
In this section, we conduct an ablation study on the efficacy of such embeddings.
We replace the SBERT embeddings with the [CLS] [Table] token representation of BERT 3 (Devlin et al., 2019) and ALBERT model 4 (Lan et al., 2020) in KG-CR U SE.
Additionally, we consider an instance wherein the elements are encoded using Sentence-RoBERTa (SRoBERTa) 5 (Reimers and Gurevych, 2019).
The results in Table 2 demonstrates the strength of our embedding choices wherein SBERT and SRoBERTa outperforms the BERT and ALBERT embeddings.
Both the sentence embedding models models are pre-trained on NLI datasets, which allows them to capture rich semantic information for textual similarity.
These embeddings have demonstrated strong performances in the task of semantic search using cosine-similarity (Reimers and Gurevych, 2019).
It should be noted that before the softmax layer in
KG-CR U SE, we compute the dot product of the
LSTM layer hidden representation with that of the relation-entity embeddings available at the given timestep.
As a result of this step, we expect the performance of SBERT and SRoBERTa embeddings to be better that BERT and ALBERT embeddings.
Additionally, we see from the Table 2 that the relation accuracy of different models is higher than path accuracy.
This is due to the outgoing edges of a node (from the dialogue history) sharing similar features if they are connected using the same relation.
Thus multiple entities can fit our choice of the response entity given the dialogue context.
5. Impact of KG Embedding Alignment and SBERT Fine-tuning
In this section [Table] we study the impact of encoding KG elements with SBERT embeddings.
Additionally, we analyse if fine-tuning the SBERT architecture used for encoding the dialogue history is beneficial for explainability. [Figure]
[Table] [Table] Table 3 outlines four situations, where in two cases we fine-tune the SBERT architecture used for encoding the dialogue history.
We also consider two cases where the embeddings of the KG elements are initialised with values drawn from a normal distribution with mean 0 and standard deviation 1, corresponding to the value “No” in the second column.
It should be noted that we never consider fine-tuning the SBERT architecture used for encoding the KG elements.
We see from the Table
that in cases when the
KG elements are not encoded with SBERT embeddings, their performance drops as compared to cases when we use SBERT embeddings.
Additionally, we find that fine-tuning SBERT leads to a decrease in the performance of KG-CR U SE.
This can be attributed to the change in semantic space of the dialogue embeddings and the KG embeddings during fine-tuning.
Hence, we do not finetune the SBERT architecture in the default setting of KGCR U SE.
5. Impact of Beam-Width on Path Reasoning
In this experiment, we study the influence of beamwidth at different timesteps on the model perfor- [Table] mance.
The first column of Table 4 lists the tuples ( K 1 , K 2 , K 3 ) where each K i denotes the top- K i edges sampled at timestep i .
We conduct this analysis on a single split of the dataset keeping all other parameters of the model constant.
We consider a diverse set of values for each K i .
From Table 4, we find that although the tuples (2, 5, 50), (2, 10, 25), (2, 25, 10) and (2, 50, 5) have an equal number of sampled paths, tuple (2, 25, 10) performs better than others.
Interestingly, we observe that the sampling sizes at the second timestep play a significant role in finding optimal paths for KG-CR U SE.
The first two sets of fact selection (i.e. during timesteps 1 and 2) largely determine the facts reachable by KG-CR U SE.
Sampling more samples during the initial timesteps enables the agent to explore diverse paths initially and KG-CR U SE then makes an optimal selection of facts dependent on the dialogue information.
5. Analysis of Computational Requirements
In this study, we conduct an analysis of the time required for training the model.
We also compare the performance of different architectures with regards to the inference speed.
Table 5 shows that while DKGW has a better train time per epoch than KG-CR U SE and
Seq2Seq has a better inference speed than KGCR U SE, we can observe from Table 1 that our model achieves better performance compared to these models.
It is worthwhile to mention that while AttnIO achieves the closest performance to
KG-CR U SE as shown in Table 1, it requires roughly six times more training time and is ten times slower during inference.
This clearly indicates the benefits of using KG-CR U SE for explainable conversation using KGs.
5. Qualitative Analysis
This section highlights three scenarios showcasing the underlying working of KG-CR U SE.
Tabledisplays three examples where KG-CR U SE generates paths different from the gold KG paths.
In the first example, it can be observed that KGCR U SE identifies a path that is not sufficient to answer the given question.
This can be due to the limited dialogue context provided.
Choosing this fact might lead to a dialogue with the agent, however, the user query is not answer with the path chosen by KG-CR U SE.
In the second example, the relation traversed by
KG-CR U SE is correct, however as the dialogue context is not specific, it decodes a path that might potentially fit the dialogue context but is different from the gold path.
However, in the third example, even with limited context, KG-CR U SE identifies a path relevant to the context, however the final entity differs from the gold path.
Such paths are admissible as all of them fit the dialogue context appropriately.
Conclusion
In this work, we propose KG-CR U SE, an LSTM based lightweight framework for explainable conversational reasoning.
We utilise SBERT embeddings to capture the rich semantic information in the dialogue history and the KG elements.
We conduct an extensive evaluation to demonstrate that our framework outperforms several baseline models on both explainability and response entity retrieval.
As annotating ground truth paths is expensive, we plan on extending this model to scenarios when ground truth paths are not available involving the generation of knowledge-conditioned dialogue.  Acknowledgements This work is supported by a grant from The
Government of Ireland Postgraduate Fellowship, Irish Research Council under project ID
GOIPG/2019/3480.
The work is also co-supported by Science Foundation Ireland under grant number
SFI/12/RC/2289 2 (Insight).