SlowGuess/NAACL_2022_OCR huggingface dataset:
Dialogue: Knowledge-Structure Aware Task-Oriented Dialogue Generation
Md Rashad Al Hasan Rony $^{1,3}$ , Ricardo Usbeck $^{2}$ , Jens Lehmann $^{1,3}$
<sup>1</sup>University of Bonn, <sup>2</sup>University of Hamburg, <sup>3</sup>Fraunhofer IAIS Dresden
{rashad. rony, Jens.lehmann}@iais.fraunhofer.de
{lehmann}@uni-bonn.de
ricardo.usbeck@uni-hamburg.de
Task-oriented dialogue generation is challenging since the underlying knowledge is often dynamic and effectively incorporating knowledge into the learning process is hard. It is particularly challenging to generate both human-like and informative responses in this setting. Recent research primarily focused on various knowledge distillation methods where the underlying relationship between the facts in a knowledge base is not effectively captured. In this paper, we go one step further and demonstrate how the structural information of a knowledge graph can improve the system's inference capabilities. Specifically, we propose DialogoKG, a novel task-oriented dialogue system that effectively incorporates knowledge into a language model. Our proposed system views relational knowledge as a knowledge graph and introduces (1) a structure-aware knowledge embedding technique, and (2) a knowledge graph-weighted attention masking strategy to facilitate the system selecting relevant information during the dialogue generation. An empirical evaluation demonstrates the effectiveness of DialogoKG over state-of-the-art methods on several standard benchmark datasets.
Traditional task-oriented dialogue systems are designed to achieve specific goals such as a restaurant reservation, hotel booking and car navigation. These systems are often empowered by external domain- or task-specific knowledge that enables them to generate informative dialogues (Eric et al., 2017; Wu et al., 2019; Qin et al., 2019; He et al., 2020b). The external knowledge in these systems is usually incorporated in the form of structured knowledge triples (Zhou et al., 2018; Liu et al., 2018a) or unstructured documents (Ye et al., 2020; Ghazvininejad et al., 2018). Figure 1 depicts a knowledge-grounded dialogue about reserving a hotel.
Knowledge Graph
Figure 1: An illustration of knowledge-based multi-turn dialogue where DialogKG models the knowledge base as a Knowledge Graph. The user utterance is denoted by  $\mathbf{Q}$ , the ground-truth response by Gold, and the words in orange are knowledge graph entries.
Turn 1	Q	Hi, have you heard of a hotel called worth house ?
Gold	Yes, ma'am. Can I help book a reservation for you ?
DialoKG	Worth house is a 4 star guest house. Would you like me to make a reservation for you ?
Turn 2	Q	Can i get the price range on it ?
Gold	Worth house falls in the cheap price range. Would you like me to book a room for you ?
DialoKG	It's a cheap guest house. How many nights will you be staying ?
Recent research primarily concentrated on various knowledge filtering methods for selecting relevant knowledge (Wen et al., 2018; Kim et al., 2020; Wu et al., 2020b). These approaches treat the knowledge triples independently and leverage Pointer Networks and copy mechanisms to generate knowledge-grounded dialogues (Vinyals et al., 2015; Gu et al., 2016; Wu et al., 2019; Sukhbaatar et al., 2015; Raghu et al., 2021; Chaudhuri et al., 2021). Typically, these systems generate a template or sketch-response during training and learn to fill in the slots with knowledge graph entries. Such systems face two issues when they try to generate dialogues in a multi-domain setting. Firstly, they are unable to capture the underlying semantics of a knowledge graph, such as the relationship between entity and relation. This leads frequently to in-
Findings of the Association for Computational Linguistics: NAACL 2022, pages 2557 - 2571
(a) System architecture.
(b) Weighted-graph computation.
Figure 2: A high-level overview of DialogKG is shown in Figure (a). Figure (b) depicts the input and output of the Graph Weight Computer module of DialogKG.
correct and inappropriate dialogue generation (Lin et al., 2020). Secondly, they lack the ability to encode dynamic knowledge in a multi-domain setting, resulting in noisy dialogues (Madotto et al., 2020). Generally, integrating a knowledge base into the learning process and generating correct and coherent dialogues at the same time is a challenging task.
In this paper, we propose a novel task-oriented dialogue system, named DialogKG that employs structural information of the knowledge graph into a language model (LM) for generating informative dialogues (see Figure 2a). For this purpose, we exploit GPT-2 (Radford et al., 2019) - a language model developed based on a stack of Transformer decoders (Vaswani et al., 2017). Specifically, we introduce a novel structure-aware multiple embedding layer-based knowledge embedding technique that constructively embeds the underlying relationship between the knowledge triples. DialogKG interprets the knowledge as a knowledge graph; therefore, separate embedding layers for word token, entity, triple and token type enable the system to capture the graph features (e.g., subject, relation and object). This enables the system to generate correct and human-like dialogues and prevents generating erroneous responses such as "4 miles is located at 792 Bedoin Street Starbucks away". Furthermore, the ability to correctly capture the relationship in the knowledge graph eliminates the need for template-based or sketch-based response generation.
In order to guide the decoder on relevant parts of the knowledge graph, we propose a new knowledge attention masking method. For constructing the
knowledge attention mask, in each dialogue turn, a weighted graph is computed in two steps: 1) Entity weights are computed using a pre-trained language model that estimates the importance of an entity for the given utterance, and 2) relation weights are computed based on the concept of graph convolution networks (GCN) (Kipf and Welling, 2017). Both steps take the user utterance into consideration, i.e., the obtained weighted graph is question specific. A set of triples is then selected based on the most relevant entities and relations of the weighted graph to construct a knowledge attention mask for the language model. This allows the masked language model to focus on relevant graph triples. We hypothesise that this leads to the generation of more accurate responses and enhance the model's capabilities of understanding the domain and task.
To assess the performance of DialogoKG, we conduct experiments on three public benchmarks: SMD (Eric et al., 2017), CamRest (Wen et al., 2017) and Multi-WOZ 2.1 (Budzianowski et al., 2018). We evaluate the system generated responses using both human and automatic metrics. Furthermore, we analyse impact of the individual components on the overall performance to verify the effectiveness. Our experimental results show that DialogoKG outperforms state-of-the-art models in knowledge-grounded dialogue generation and can generate human-like responses. We made our code publicly available<sup>1</sup>.
'https://github.com/rashad101/DialoKG
Figure 3: An illustration of knowledge and dialogue embedding techniques.
DiaoKG aims to generate informative responses given a dialogue history, a question and a knowledge base. We define the dialogue history  $\mathcal{H}$  as a set of turns between two speakers, such that  $\mathcal{H} = \{U_1, S_1,.., U_t, S_t\}$ , where  $U_i$  and  $S_i$  are the sequences of words in turn  $i$ . We assume that the knowledge is stored in a multi-relational knowledge graph  $\mathcal{G}$ . Here,  $\mathcal{G}$  is a set of triples  $\mathcal{T}$  such that  $\mathcal{T} \subseteq \mathcal{E} \times \mathcal{R} \times \mathcal{E}$ , where  $\mathcal{E}$  is the set of entities and  $\mathcal{R}$  the set of relations. A triple  $\mathcal{T} \in \mathcal{G}$  is denoted as  $(s, r, o)$  in which  $s \in \mathcal{E}$  and  $o \in \mathcal{E}$  denote the subject and object entities, respectively, and  $r \in \mathcal{R}$  is the relation between them. We use the terms "Knowledge Graph" and "Graph" interchangeably throughout this paper. Furthermore, we denote the user utterance of the current dialogue turn as  $\mathcal{Q}$ . A GPT-2 (Radford et al., 2019) language model is used in this paper to generate responses. However, any Transformer decoder-based LM can be used. Formally, the probability distribution of generating a response by the language model is defined as:
$$
p \left(S _ {t} \mid \mathcal {H}, \mathcal {Q}, \mathcal {G}\right) = \prod_ {i = 1} ^ {n} p \left(s _ {i} \mid s _ {1},.., s _ {i - 1}, \mathcal {H}, \mathcal {Q}, \mathcal {G}\right) \tag {1}
$$
Here,  $S_{t}$  is the generated response in turn  $t$  and  $n$  is the maximum length of the generated response.
2.2 Knowledge and Dialogue Embedding
DiaLoKG takes a knowledge graph  $\mathcal{G}$ , dialogue history  $\mathcal{H}$ , and the current user utterance  $\mathcal{Q}$  together as input and constructs a single input sequence as depicted in Figure 3. The first part of the sequence contains graph related information (i.e., subject, relation, and object) and the latter part dialogue specific information such as dialogue history  $(\mathcal{H})$  and the current user utterance  $(\mathcal{Q})$ . Knowledge Specific Embedding. To infuse structural information, DiaLoKG employs entity embedding, triple
embedding and type embedding, besides the usual word token and positional embedding. Such an embedding technique allows the system to encode the knowledge graph structure. To do this, knowledge graph triples are linearized into a sequence as input, as depicted in Figure 3. To facilitate order invariance of the knowledge embedding, we shuffle the order of the graph triples in the input sequence during training. In the token embedding layer [S], [R] and [O] are special tokens to separate subject, relation and object of a triple from each other in the sequence. Entity and triple embedding layers embed entity and triple-level information of the word token. For instance, ENT1 in the entity embedding layer indicates that the corresponding words in the token embedding layer are related to the first subject, which is starbucks in this case. Likewise, T1 and T2 in the triple embedding layer indicate that the corresponding words in the token embedding layer are related to the first and second triple, respectively. Finally, the type embedding indicates that the corresponding tokens are from the knowledge graph as opposed to the dialogue history.
Dialogue Specific Embedding. The dialogue specific part of the input sequence is separated from the knowledge specific part by a [SEP] token in the token embedding layer. Furthermore, the user utterance/question  $(\mathcal{Q})$  of the current turn is separated by a [Q] token from the dialogue history. The type embedding layer stores information about whether the corresponding utterance is from the user or system. This way, the decoder can use information about typical dialogue turn patterns.
The positional embedding in both knowledge and dialogue embeddings encodes the position of each word token in the sequence. Finally, embeddings from all five layers are summed up as depicted in Figure 3. Layer Normalization (Ba et al., 2016) is then applied to obtain the final embedding representation of the complete input sequence. It
Figure 4: For the graph in Figure (a) and the question "Find me the quickest route to the restaurant?" the computation of the relation weight is shown in Figure (b), where  $\hat{A} = A + I$ .
normalizes the embedding representation of layers, which restricts the weights of the learning network from exploding.
We argue that the proposed design pattern of forming a single sequence and specifying each item in the input sequence further with additional embedding layers can improve the system's understanding of the task and domain.
2.3 Knowledge Attention Mask Construction
To notify the decoder about the relevant KG triples for answering the current user question, a knowledge graph weighted-attention mask is constructed. Prior to the construction of the knowledge attention mask, a weighted-knowledge graph,  $\mathcal{G}_w$  is first computed by a Graph Weight Computer module, where the entity and relation weights are computed independently. We discuss the components of the Graph Weight Computer module below.
Entity Weight Estimator. A pre-trained language model RoBERTa (Liu et al., 2019), is used to compute the entity weights, similar to (Yasunaga et al., 2021). Each entity  $E_{i} \in \mathcal{E}$  of graph  $\mathcal{G}$  is concatenated with the user utterance  $\mathcal{Q}$  to obtain the probability score from the language model.
$$
E _ {i w} = L M _ {\text {h e a d}} \left(L M _ {\text {e n c}} \left(\left[ \mathcal {Q}; E _ {i} \right]\right)\right) \tag {2}
$$
In Equation 2,  $LM_{head} \circ LM_{enc}$  represents the probability of the entity  $E_i$  computed by the language model. We consider  $E_{iw}$  as the weight of the entity  $E_i$ , which represents the relevance of the entity for the given user utterance  $Q$ .
Relation Weight Estimator. We follow (Kipf and Welling, 2017; Vashisth et al., 2019) and leverage the concept of GCN to obtain the relation weight. In contrast to the previous works, our
proposed relation weight estimator transforms the input graph into an undirected graph  $\hat{\mathcal{G}}$ , where the relations are considered as nodes of a graph. This transformation technique allows the relation estimator to obtain a score for each relation. The graph transformation is demonstrated in Figure 4a. The relation weight is computed as follows:
$$
R _ {w} = \hat {H} M ^ {r}, \tag {3}
$$
$$
\hat {H} = D ^ {- 1} (A + I) X
$$
Here,  $D^{-1}(A + I)$  computes the row-normalized adjacency matrix, where  $D$  and  $A$  are respectively the degree matrix and adjacency matrix of the graph  $\hat{\mathcal{G}}$  as depicted in Figure 4b and  $I$  is the identity matrix. Let  $d_g = |\mathcal{E}| + |\mathcal{R}|$  be the total number of entities and relations in the graph  $\hat{\mathcal{G}}$ , then  $D, A, I \in \mathbb{R}^{d_g \times d_g}$ . A feature vector  $X \in \mathbb{R}^{d_g \times 1}$  is obtained by computing the cosine similarity between the embedding of knowledge graph entries (entities and relations) and the embedding of question. Furthermore, a relation mask  $M^r \in \mathbb{R}^{d_g \times 1}$  is constructed by setting a value of 1 and 0 to the positions that correspond to relations and entities, respectively, to attend to the values that correspond to the relations only. The feature vector  $X$  eliminates the need for additional learnable parameters to capture the relationships, unlike GCN. Finally, values that correspond to the entities in  $\hat{H}$  are masked out by multiplying with  $M^r$  to obtain final relation weights  $R_w \in \mathbb{R}^{d_g \times d_g}$ .
The computed weighted-graph assists the model to focus on the task by constructing a knowledge attention mask. We use the normalized score of  $R_{w}$  and  $E_{w}$  for constructing the knowledge attention mask. To filter-out irrelevant knowledge triples,
Dataset	#Dialogues	#Utterances	Avg. Length of Utt.	#Utt. with Entities	Avg. #Entities per Utt.
SMD (Eric et al., 2017)	3,031	15,928	9.22	4430	2.96
CamRest (Wen et al., 2017)	676	2,744	11.72	2366	2.43
MWOZ (Budzianowski et al., 2018)	2,877	19,870	16.68	6241	2.06
we obtain the top-  $k$  entities and relations from the weighted graph as denoted as  $\hat{\mathcal{E}}$  and  $\hat{\mathcal{R}}$  respectively. Here,  $k$  is a hyper-parameter which we chose from a range of [0,  $\max(|\mathcal{E}|, |\mathcal{R}|)$ ], based on the validation score. Finally, based on the selected  $\hat{\mathcal{E}}$  and  $\hat{\mathcal{R}}$ , the knowledge attention mask is constructed as follows:
$$
M _ {i, j} ^ {k g} = \left\{ \begin{array}{l l} 0, & \text {i f} \left((s _ {i} \lor o _ {i}) \in \hat {\mathcal {E}}\right) \wedge (r _ {i} \in \hat {\mathcal {R}}) \\ - \infty , & \text {o t h e r w i s e} \end{array} \right.
$$
Here  $r_i$ ,  $s_i$  and  $o_i$  corresponds to the relation, subject and object entity of triple  $\mathcal{T}_i$ . Any position that corresponds to the value of  $-\infty$  results in 0 after computing the softmax during the attention computation (discussed in the next sub-section). The final mask  $M \in \mathbb{R}^{n \times n}$  is obtained by appending the mask for the dialogue related sequence with the knowledge attention mask where  $n$  is sequence length. Padding is added to adjust the dimension of the metrics.
2.4 Decoder
A Transformer (Vaswani et al., 2017) based GPT-2 (Radford et al., 2019) model is used for generating the response. The attention, computed in each of GPT-2's heads is formalized as follows:
$$
\begin{array}{l} A t t n (Q, K, V) = \operatorname {s o f t m a x} \left(\frac {1}{\sqrt {d _ {k}}} \left(Q K ^ {T}\right) + M\right) V, \tag {4} \\ H _ {i} = A t t n \left(Q W _ {i} ^ {Q}, K W _ {i} ^ {K}, V W _ {i} ^ {V}\right) \\ \end{array}
$$
where,  $\text{Att}n(\cdot)$  computes the masked attention,  $H_{i}$  is the  $i$ -th head,  $d_{k} = d_{m} / h$ . Here,  $d_{m}$  is the dimension of the model where  $h$  the number of heads.  $Q$ ,  $K$  and  $V$  are query, key and value where  $W_{i}^{Q}, W_{i}^{K}, W_{i}^{V}$  are trainable parameters. The objective of the model is to minimize the negative log-likelihood  $\mathcal{L}$  for next-token prediction. For a dialogue dataset  $D = \{D_{1}, D_{2}, \dots, D_{j}\}$ , we formally define  $\mathcal{L}$  as follows:
$$
\mathcal {L} (D) = - \sum_ {j} ^ {| D |} \sum_ {i} ^ {n} \log p \left(s _ {i} ^ {j} \mid s _ {1} ^ {j},.., s _ {i - 1} ^ {j}, \mathcal {H} ^ {j}, \mathcal {Q} ^ {j}, \mathcal {G} ^ {j}\right), \tag {5}
$$
where  $n$  is the maximum response length and  $\mathcal{H}^j,\mathcal{Q}^j,\mathcal{G}^j\in D_j$  .Top-k sampling (Fan et al., 2018) decoding is used to generate the next word token at each time step, during the inference.
We evaluate DialogoKG on three publicly available knowledge-grounded and task-oriented dialogue datasets: Stanford Multi-Domain dataset (SMD) (Eric et al., 2017), CamRest (Wen et al., 2017) and Multi-WOZ 2.1 (MWOZ) (Budzianowski et al., 2018). SMD consists of three domains: weather, navigation, and calendar. MWOZ contains five domains: train, hotel, restaurant, taxi and attraction. We use the splits provided with the datasets for train, validation, and test. Each dialogue is provided with a knowledge base. Table 1 shows the statistics of the benchmark datasets.
3.2 Hyper-parameter Settings
Throughout this paper, we use the GPT-2 (Radford et al., 2019) model with 117M parameters. AdamW (Loshchilov and Hutter, 2019) with  $\epsilon = 1e - 8$  and learning rate of  $6.25e - 5$  is employed as optimizer. GELU (Hendrycks and Gimpel, 2016) is used as activation function. The best hyperparameters for each dataset were found using grid search and based on the results on the validation set. We run all experiments on a distributed training setting with 10 GPUs, each with 12 GB of memory. More implementation details can be found in Appendix A.
Automatic Metrics. Following the baseline models, we use BLEU (Papineni et al., 2002) and Entity F1 score (Eric et al., 2017) as automatic evaluation metrics. The Entity F1 score represents the model's capability of generating knowledge grounded responses. It computes the F1 score between the set of entities present in the ground truth and system-generated responses. Several studies (Novikova et al., 2017; Liu et al., 2016) on evaluation metrics suggest that word-overlap based metrics such as BLEU are insufficient for evaluating natural language generation (NLG) systems. Hence, we use MoverScore (Zhao et al., 2019) as
Model	SMD	CamRest	MWOZ
BLEU	MoverScore	Ent. F1	BLEU	MoverScore	Ent. F1	BLEU	MoverScore	Ent. F1
GLMP (Wu et al., 2019)	13.9	54.2	59.6	15.1	57.2	58.9	6.9	51.2	32.4
MLM (Gangi Reddy et al., 2019)	17.0	64.0	54.6	15.5	57.0	62.1	-	-	-
Ent. Const. (Qin et al., 2019)	13.9	53.8	53.7	18.5	65.9	58.6	-	-	-
GPT2+KE (Madotto et al., 2020)	17.4	66.4	59.8	18.0	65.8	54.9	15.0	60.9	39.6
TTOS (He et al., 2020a)	17.4	59.8	55.4	20.5	67.0	61.5	-	-	-
DF-Net (Qin et al., 2020)	14.4	56.3	62.7	-	-	-	9.4	54.2	35.1
EER (He et al., 2020c)	17.2	60.9	59.0	19.2	66.1	65.7	13.6	57.2	35.6
FG2Seq (He et al., 2020b)	16.8	60.2	61.1	20.2	66.6	66.4	14.6	58.4	36.5
CDNet (Raghu et al., 2021)	17.8	61.1	62.9	21.8	67.8	68.6	11.9	55.8	38.7
DialoKG	20.0	70.6	65.9	23.4	70.4	75.6	12.6	62.6	43.5
Table 2: Performance of DialoKG and baseline models on three benchmark datasets. Best scores in bold and second-best underlined.
addition metric to evaluate the semantic similarity between the system generated response and the ground truth. We compute both MoverScore and BLEU scores on the sentence level.
Human Evaluation. To assess the quality of the system-generated responses, we conduct a human evaluation based on the following criteria: 1) Naturalness: how human-like and fluent the generated responses are, and 2) Correctness: how correct the knowledge-grounded responses are. We asked three annotators (two from Computer Science (CS) and one from a non-CS background) who are not part of this research work to evaluate the quality of the system-generated responses. We randomly sampled 90 dialogues in total from the benchmark datasets and asked annotators to evaluate the system-generated responses given the ground truth response and the knowledge graph triples on a scale of [1,5] (higher is better). The inter-annotator agreement score (Cohen's kappa  $\kappa$ ) of the annotated data is 0.82. The human evaluation process is explained in detail in Appendix D.
3.4 Baselines
We compare DialogKG with the following state-of-the-art methods: GLMP (Wu et al., 2019), MLM (Gangi Reddy et al., 2019), Ent. Const. (Qin et al., 2019), DF-Net (Qin et al., 2020), CDNet (Raghu et al., 2021), GPT2+KE (Madotto et al., 2020), TTOS (He et al., 2020a) and EER (He et al., 2020c). Most of these approaches adopt memory networks to generate knowledge grounded dialogues, whereas GPT2+KE (Madotto et al., 2020) directly embeds the knowledge base into the model's parameters and TTOS (He et al., 2020a) proposed a reinforcement learning-based framework.
We conduct both quantitative and qualitative analyses to assess system-generated responses. Table 2 summarizes the performance of DialogoKG with respect to the baseline models. It is evident that DialogoKG outperforms the baseline models significantly in Entity F1 score on CamRest, which contains mostly knowledge-grounded dialogues about restaurant reservations. A high Entity F1 score of 75.6 on CamRest shows DialogoKG's ability to generate knowledge-grounded with high accuracy. Although DialogoKG achieves an improved Entity F1 score on the MWOZ dataset, it has a lower BLEU score since MWOZ often contains lengthy responses. However, the high MoverScore across all datasets demonstrates that DialogoKG can generate highly semantically similar responses. Domain-wise results are reported in Appendix B due to space limitation.
Figure 5: Distribution of human evaluation scores.
We obtain human evaluation scores (naturalness and correctness) for the closest three models. Re
Model	Naturalness	Correctness
EER (He et al., 2020c)	3.27	3.61
FG2Seq (He et al., 2020b)	3.33	3.87
CDNet (Raghu et al., 2021)	3.53	3.94
DialoKG	4.33	4.01
Table 3: Human evaluation results.
Approach	BLEU	Δ	Ent. F1	Δ
DiaLoKG (seq2seq)	14.5	-	59.4	-
+ Entity embedding	17.7	3.2↑	63.0	3.6↑
+ Triple embedding	19.2	1.5↑	67.8	4.8↑
+ Type embedding	20.1	0.9↑	68.4	0.6↑
+ Knowledge attention mask	23.4	3.3↑	75.6	7.2↑
sults in Table 3 show that our proposed dialogue system can generate more human-like responses. An improved score is also achieved in terms of correctness, reflecting DialoKG's ability to generate highly accurate dialogues. Furthermore, Figure 5 shows the distribution of human evaluation scores. The figure allows a better direct comparison of the individual score levels. Details about the annotation process are reported in Appendix D.