SlowGuess/NAACL_2022_OCR huggingface dataset:
Yu Li\*, Baolin Peng, Yelong Shen, Yi Mao, Lars Liden, Zhou Yu, Jianfeng Gao
<sup>1</sup>Microsoft Research, Redmond, WA
$^{2}$ Columbia University, New York, NY
{bapeng, yeshe, maoyi, laliden, jfgao}@microsoft.com
{yl5016，zy2461}@columbia.edu
Knowledge-grounded dialogue systems are challenging to build due to the lack of training data and heterogeneous knowledge sources. Existing systems perform poorly on unseen topics due to limited topics covered in the training data. In addition, it is challenging to generalize to the domains that require different types of knowledge sources. To address the above challenges, we present PLUG<sup>1</sup>, a language model that homogenizes different knowledge sources to a unified knowledge representation for knowledge-grounded dialogue generation tasks. We first retrieve relevant information from heterogeneous knowledge sources (e.g., wiki, dictionary, or knowledge graph); Then the retrieved knowledge is transformed into text and concatenated with dialogue history to feed into the language model for generating responses. PLUG is pre-trained on a largescale knowledge-grounded dialogue corpus. The empirical evaluation on two benchmarks shows that PLUG generalizes well across different knowledge-grounded dialogue tasks. It achieves comparable performance with state-of-the-art methods in the fully-supervised setting and significantly outperforms other approaches in zero-shot and few-shot settings.
Recent work has shown that conversational models can be trained in an end-to-end fashion (Gao et al., 2019; Roller et al., 2020; Zhang et al., 2019; Adiwardana et al., 2020). Though such models can generate coherent and natural responses consistent with conversation history, there is still a clear gap between conversational AI agents and humans. The primary reason is that existing dialogue systems lack knowledge of the subject and thus cannot deep dive into specific topics with humans.
Dataset	Knowledge	% Topics
Open-domain
Wizard of Wikipedia	articles	0.02%
CMU_DOG	articles	0.04%
Recommendation
REDIAL	tables	15.0%
OPENDIALKG	graph	7.5%
Table 1: Knowledge representation and topic coverage statistics of existing knowledge-grounded dialogue datasets. % Topics means the portion of topics or facts in the knowledge database covered by the dataset.
In order to better incorporate knowledge into dialogue, knowledge-grounded dialogue systems have become increasingly popular.
Knowledge-grounded dialogue generation aims to generate informative and meaningful responses based on both conversation context and external knowledge sources. Thus far, researchers have collected knowledge-grounded dialogues for various tasks using crowdsourcing platforms, for instance, open-domain dialogues (Dinan et al., 2019; Zhou et al., 2018) and conversational recommendation dialogues (Li et al., 2018; Moon et al., 2019; Hayati et al., 2020). Workers are asked to base their replies on knowledge from structured knowledge bases (Moon et al., 2019; Tuan et al., 2019) or unstructured documents (Dinan et al., 2019; Zhou et al., 2018; Feng et al., 2020). Taking advantage of recent advances in large-scale language models (Raffel et al., 2019; Lewis et al., 2020a; Guu et al., 2020), researchers have also built knowledge-grounded dialogue systems by fine-tuning such language models in an end-to-end fashion (Shuster et al., 2021; Zhao et al., 2020b; Li et al., 2021).
However, there are two critical challenges in these existing methods. First, it is expensive and time-intensive to collect knowledge-grounded dialogues. As shown in Table 1, most of the datasets
*Work was done when Yu Li was interning at MSR
$^{1}$ Pre-trained Language model with a Unified knowledge representation for knowledge-Grounded dialogues.
Human Language Technologies, pages 206 - 218
only cover a small portion of the knowledge base. Thus, systems which only fine-tune with small training sets generalize poorly on unseen topics in the same knowledge base. Additionally, the formats of knowledge sources vary in different tasks, making the approaches unable to transfer to other domains with different knowledge sources. For example, REDIAL (Li et al., 2018) adopts a movie database as the knowledge source to recommend movies. Techniques on this task exploit the graph structure. It is not easy to adapt such techniques to document-grounded conversation tasks like Wizard of Wikipedia (Dinan et al., 2019).
In this work, we present PLUG, a model that can unify different knowledge formats for knowledge-grounded dialogue generation. First, we convert different knowledge formats (e.g., knowledge graph, knowledge base, and passages) to unstructured text, each using a different retriever. Then we use a pre-trained language model to process them into a unified representation to incorporate the knowledge into dialogue generation. We pre-train PLUG on different knowledge-ground dialogue corpora, including a large-scale open-domain conversation dataset from Reddit. This allows PLUG to learn knowledge in various formats from different tasks, and thus transfer to any knowledge-grounded dialogue task with few-shot learning techniques.
We evaluate the effectiveness of PLUG by applying it to an open-domain knowledge-grounded dialogue benchmark, Wizard of Wikipedia (Dinan et al., 2019), and a knowledge-grounded conversational recommendation benchmark, REDIAL (Li et al., 2018). PLUG achieves results comparable to the state-of-the-art method under a fully-supervised setting. It outperforms other methods on both tasks under zero-shot and few-shot settings, demonstrating that PLUG can be grounded on world knowledge in different knowledge sources and generalize to different downstream tasks.
Our contributions are three-fold: (1) We propose a novel knowledge-based pre-trained language model, PLUG, that can be applied to any knowledge-grounded dialogue tasks; (2) Our model achieves slightly better results than state-of-the-art models in fully-supervised settings and shows promising improvements over the current state-of-the-art in zero-shot and few-shot settings; (3) We present extensive experiments to explore the bottlenecks of the task and the future direction of knowledge-grounded dialogues.
We describe our approach in this section. Figure 1 gives a diagram of our proposed method. We first introduce the background of knowledge-grounded dialogues and the backbone language model in Section 2.1. Then, we formalize the task and introduce the details of PLUG in Section 2.2. Finally, we explain the training process of our PLUG, which includes the pre-training dataset selection and the data pre-processing processes in Section 2.3.
2.1 Background: Knowledge-Grounded Pre-training
Traditional knowledge-grounded dialogue includes three steps: information extraction, knowledge prediction, and response generation. Previous work focuses on developing separate modules (Zhou et al., 2020b). Inspired by the recent success of applying a large-scale pre-trained language model on task-oriented dialogue systems (Peng et al., 2020; Hosseini-Asl et al., 2020), we explore the possibility of using a unified knowledge representation in a large-scale language model. In order to properly manage the task in a sequence-to-sequence setup, we choose T5 (Raffel et al., 2020) as our backbone.
T5 is a sequence-to-sequence pre-trained Transformer (Vaswani et al., 2017) model for transfer learning. It is trained by converting various language tasks into text-to-text tasks. After fine-tuning on a dialogue dataset, T5 can generate fluent and coherent responses. Nevertheless, responses are often too generic because they are not grounded on specific knowledge. PLUG is built on the T5 model but grounded on real-world knowledge during training, making it inherit T5's capability of producing good responses but include more knowledge.
2.2 PLUG
We formulate a knowledge-grounded dialogue as:
$$
\mathcal {D} = \{C, R, S \} \tag {1}
$$
where  $C = \{C_i\}_{i=1}^n$  is a dialogue context, and  $R = \{R_i\}_{i=1}^n$  is the response in a dialogue that has  $n$  turns.  $\mathcal{S}$  is the external knowledge source for task  $t$ . For each dialogue turn, we can formulate a knowledge-grounded dialogue generation task on a single domain  $d$  as  $p(R_i|C_i, \mathcal{S})$ .
As shown in Figure 1, each task has its own knowledge source (e.g., documents, databases, and knowledge graphs). In order to make all knowledge-grounded dialogue generation tasks
Figure 1: A diagram of PLUG. PLUG homogenizes different knowledge sources in different tasks to a unified knowledge representation. Then it learns to ground response generation on the unified knowledge representation.
able to fit in the text-to-text encoder-decoder framework, we follow T5 to feed each dialogue turn into the language model simply by concatenating the context  $C_i = \{c_1, c_2, \dots, c_m\}$ , and essential knowledge triples  $K_i = \{k_1, k_2, \dots, k_n\}$  as a token sequence. The essential knowledge is extracted from the knowledge source  $\mathcal{S}$  and represented as text of triples. We train the model to predict the responses token sequence  $R = \{r_1, r_2, \dots, r_k\}$ . The probability of the responses is formulated as:
$$
p \left(R _ {i} \mid C _ {i}\right) = \prod_ {t = 1} ^ {k} p \left(r _ {t} \mid C _ {i}, K _ {i}, r _ {1},..., r _ {t - 1}\right) \tag {2}
$$
We will explain how we select and process pretraining datasets in the following sections.
2.3 Model training process
We pre-trained the PLUG model using two datasets, Reddit Conversation (Galley et al., 2018) and OpenDialKG (Moon et al., 2019). We will first present the three-step data cleaning process of Reddit Conversation in Section 2.3.1, then we will introduce OpenDialKG in Section 2.3.2.
2.3.1 Reddit Conversation
Reddit Conversation Galley et al. (2018) is a large-scale open-domain conversation dataset. It extracts the conversation threads grounded on a document from the Reddit data. We only keep the conversations grounded on Wikipedia passages for pretraining to recognize better the knowledge used in the dialogue. Since vanilla document-based dialogue in the dataset does not have a knowledge
label for each dialogue turn, we apply a hierarchical information extraction method to obtain the essential knowledge in each turn. Our information extraction method includes three steps: knowledge retrieval, statistical ranking, and semantic ranking.
Knowledge Retriever. We use a knowledge retriever to retrieve all relevant knowledge in a single turn's response. We first extract the title of the grounding Wikipedia passage in the dialogue. Then, we retrieve knowledge triples from a large-scale knowledge graph, DBpedia (Lehmann et al., 2015). Specifically, we query the DBpedia via a public SPARQL endpoint<sup>3</sup> and then collect triples whose subject or object is in the Wikipedia passage in the dialogue. For example, we keep triples <Barack Obama, alma mater, Columbia University> and <Michelle Obama, spouse, Barack Obama> for the dialogue about Barack Obama. To carry sufficient knowledge to refine in the next step, we retrieve 500 triples for every passage.
Statistical Ranking. After retrieving adequate knowledge, we rank the corresponding triples to refine the knowledge. Specifically, we get the TF-IDF (term frequency-inverse document frequency) value for all the retrieved triples. To find the triples related to the context, we concatenate the dialogue history and the response as the query. Then we compute the cosine similarity between the query and every triple. Because every triple has the Wikipedia passage name as the subject or object, a higher cosine similarity score means the query has more similar text with the distinguished text in the triple.
2Reddit data dumps: https://files.pushshift.io/redis/
3https://dbpedia.org/sparql
We rank the query-document similarity score and only keep the top-50 triples in this step.
Semantic Ranking. The TF-IDF-based cosine similarity score only counts words overlapping between triples and the query. It will introduce triples whose overlapping words are not meaningful in the context and response. Additionally, the Reddit Conversation dataset is obtained from Reddit conversation threads. It involves many responses that are not grounded on any knowledge. In order to find the triples that have the best semantic similarity with the response and filter out ungrounded responses, in this step, we estimate the semantic similarity score with Sentence-Bert (Reimers and Gurevych, 2019). We rerank the 50 triples from the second step based on the score. Additionally, we abandon the dialogue turns whose best semantic similarity is lower than a threshold because the response cannot find proper knowledge, while we want to pre-train the model with knowledge-grounded turns.
2.3.2 OpenDialKG
To generalize our model to various tasks, we also employ OpenDialKG to enrich our pre-training dataset. OpenDialKG consists of two types of tasks, recommendations and chit-chat, across four domains. Unlike the Reddit Conversation dataset, which needs to find the knowledge grounding in every turn, the original OpenDialKG has a Knowledge graph path label for each dialogue, and a triple label for each dialogue turn. The response is grounded on the labeled triple during data collection. Thus, we use the triple in the dataset as the essential knowledge in our pre-training examples.
We demonstrate our approach on two different downstream tasks: open-domain knowledge-grounded dialogue and conversational recommendation. Besides the fully-supervised learning setting, we also explore the performance of our approach in few-shot and zero-shot settings. We describe our implementation details in Section A in Appendix.
3.1 Datasets and Knowledge Sources
We test our approach on Wizard of Wikipedia (WoW; (Dinan et al., 2019)) and REDIAL (Li et al., 2018). Basic dataset statistics are listed in Table 2.
Wizard of Wikipedia. This dataset (Dinan et al., 2019) is collected on Amazon Mechanical Turk.
Dataset	Train	Valid	Test
WoW	18,430	Seen - 981	965
Unseen - 967	968
REDIAL	8,004	1,001	1,001
Table 2: Number of conversations in Wizard of Wikipedia (WoW) and REDIAL
Each conversation happens between a "wizard" who has access to knowledge about a specific topic, and an "apprentice" who is interested in the topic. The wizard's response is grounded on a Wikipedia article in each turn. The data is split as a training set, a validation set, and a test set. The test set has two subsets: Test Seen and Test Unseen. Test Seen contains conversations whose topics are seen in the training set, while topics in Test Unseen are not seen in the training or validation set. To extract the essential knowledge in each dialogue turn, we first keep the top five passages retrieved by the TF-IDF retriever in Shuster et al. (2021). Then we use an Open Information Extraction (OpenIE) annotator<sup>4</sup> to extract the top three triples from the passages as our essential knowledge. The pre-processing is conducted with the code published on ParlAI.<sup>5</sup>
REDIAL. REDIAL (Li et al., 2018) is also collected on Amazon Mechanical Turk. Two crowdworkers, a "movie seeker" and "movie recommender," are randomly paired. The recommender has access to a movie database and can recommend movies based on movie information, such as actors and movie genres. There are 6,924 different movies mentioned in 51,699 movie slots in the dataset. We follow Li et al. (2018) to split the dataset into training, validation, and test sets. Since recommenders use movie-related knowledge when they recommend movies to seekers, we use it as the essential knowledge for a given turn in this dataset. We experiment with three knowledge sources: (1) We query the movie names mentioned in the dialogue context and retrieve similar movies from the knowledge graph DBpedia, mentioned in Section 2.3, and then input the similar movies in a triple format as the essential knowledge. (2) We query the movie names mentioned in the context and retrieve movie comments from MovieLens, then use the keywords in the comments as the essential knowledge. (3) We use the output of the recommender
<sup>4</sup>https://nlp.stanford.edu/software/openie.html
$^{5}$ https://github.com/facebookresearch/ParlAI
$^{6}$ https://grouplens.org/datasets/movielens/
module in KGSF (Zhou et al., 2020a), which is the state-of-the-art system on this dataset.
We compare the known best models from different datasets in the following experiments. For the Wizard of Wikipedia dataset, we choose the retrieval-augmented generation (RAG) model from Shuster et al. (2021). It retrieves wiki documents and generates responses based on the documents. We compare PLUG with this document-based generation method to see the impact of our essential knowledge format. We choose the RAG model also using T5 as the baseline for a fair comparison.
For the REDIAL dataset, we choose the current state-of-the-art: KBRD (Chen et al., 2019) and KGSF (Zhou et al., 2020a) as our baselines. Both use a recommender module to predict the recommendation item in the next turn and a generation model to generate the response. All baseline results are from Zhou et al. (2021). To investigate the best performance of our approach, We also use the recommender from KGSF as a knowledge source in our system and compare it with other knowledge sources we mentioned in Section 3.1. As an ablation study, we also explore the performance of vanilla T5 on both tasks to see the performance gain brought by our pre-training process.
For evaluation, we report the performance with standard automatic metrics: BLEU-4 (B4) (Papineni et al., 2002), ROUGE-L (RL) (Lin, 2004), and unigram overlap (F1) of the generated responses. Besides that, for the Wizard of Wikipedia dataset, we follow Shuster et al. (2021) to report the overlapping unigrams between the model's generation and the knowledge on which the human grounded during dataset collection (KF1), attempting to capture whether a model is speaking knowledgeably. On the other hand, for the REDIAL dataset, we follow previous work (Chen et al., 2019; Zhou et al., 2020a; Wang et al., 2021) to report distinct-n (Distn) at the sentence level to evaluate the diversity of the model's generation. We also evaluate whether the ground truth movie recommendation can be found in the generated response and report it as the recommendation item recall in responses (Rec).
3.4 Fully-Supervised Results
We first evaluate PLUG with all training examples in the training sets to compare its performance with
other state-of-the-art systems. Additionally, we experiment with using golden knowledge in the input to explore the upper bound of our method.
Table 3 shows the Wizard of Wikipedia Test Seen and Test Unseen results. We can see that PLUG with retrieved knowledge achieves better BLEU-4, ROUGE-L, and F1 scores than the RAG method and the model without adding knowledge in the input, on both seen and unseen topics. It suggests that our essential knowledge format helps the model generate responses to ground knowledge better. We also observe that PLUG outperforms the model without pre-training on all metrics, which means our pre-training can boost this task.
We list REDIAL's results in Table 4. We compare our approach to the state-of-the-art systems and T5-Large models without pre-training. Additionally, we include a comparison to models with different knowledge sources as described in Section 3.1. It shows that our best model (PLUG+KGSF) achieves the new state-of-the-art results on the recommendation item recall metric and distinct metrics. This result is understandable given that our approach is built upon pre-trained language models. Similarly, we also observe noticeable performance gains for the pre-training on this task. However, compared to systems with currently available knowledge sources, it is immediately apparent that the system with golden knowledge outperforms the current state-of-the-art on all metrics by a large margin. This huge gap implies that current knowledge retrievers are the main bottleneck for the conversational recommendation task. We will discuss more details in Section 3.7.
Overall, we observe noticeable improvement brought by pre-training on both tasks, but it is less significant than expected. It implies that the knowledge grounding pattern in the response is limited; a complete training set is more than enough for the T5-Large model to learn the generation task. We will discuss more details in zero-shot and few-shot settings in the following subsections.
3.5 Zero-Shot and Few-Shot Results
We focus on zero-shot and few-shot settings because it is more realistic to evaluate dialogue systems. Specifically, we randomly sample 10/50/500 dialogues with different topics from the training sets and observe performance on the complete test sets. We also evaluate under a zero-shot setting. We experiment with knowledge retrieved by exist-
Model	Test Seen	Test Unseen
BLEU4	ROUGE-L	F1	KF1	BLEU4	ROUGE-L	F1	KF1
RAG-T5-Large (Shuster et al., 2021)	3.8	22.1	21.9	25.9	2.8	20.4	20.5	21.9
T5-Large-w/o Knowledge	4.1	18.0	18.3	19.2	2.1	15.4	21.4	13.9
T5-Large-Retrieved Knowledge	5.8	21.8	25.8	22.6	3.4	19.2	22.7	17.6
T5-Large-Golden Knowledge	11.3	30.8	35.6	46.8	8.7	28.4	33.0	43.6
PLUG-Retrieved Knowledge	6.0	22.3	26.5	22.4	3.5	19.5	23.3	18.6
PLUG-Golden Knowledge	11.5	31.1	36.0	47.8	8.8	29.0	33.4	46.0
(c)
Figure 2: Zero-shot and few-shot results on Wizard of Wikipedia Test Seen and Test Unseen sets.
Table 3: Fully-supervised results on Wizard of Wikipedia Test Seen and Test Unseen Sets.
Model	B4	RL	DIST2	DIST4	Rec
KBRD	1.8	16.5	0.48	0.67	0.7
KGSF	2.3	13.1	0.49	1.28	0.9
T5-Large
+w/o KG	3.7	18.3	0.72	1.10	3.4
+Golden	10.4	32.7	1.17	1.60	83.5
+KGSF	3.7	17.4	1.13	2.02	4.7
PLUG
+w/o KG	3.9	19.6	0.78	1.31	3.7
+Golden	10.6	33.5	1.26	1.81	84.3
+DBpedia	3.3	18.3	0.45	0.66	0.8
+MovieLens	3.4	17.8	0.91	1.34	2.4
+KGSF	3.8	18.0	1.51	2.84	5.3
Table 4: Fully-supervised results on REDIAL.
ing retrievers on both tasks to set a realistic setting. We compare our models to those without pretraining to explore how our pre-training benefits the model's few-shot learning capability. Wizard of Wikipedia's experiments results are in Figure 2, and REDIAL's results are in Figure 3. Note that for Wizard of Wikipedia, topics in original Test Seen set may not be seen during training in this setting since we only use a small portion of data in the original training set. We use original Test Seen and Test Unseen sets to compare with fully-supervised results. As can be seen in Figure 2 (a)-(c), 3 (a)-(b), PLUG maintains a higher BLEU-4, ROUGE-L, and F1 scores on both tasks when training with less than 500 dialogues. It means PLUG obtains knowledge-grounded generation ability from pretraining and can generalize to different tasks.
Figure 2 (d) shows that models without pretraining achieve a higher knowledge F1 score under a zero-shot setting for the Wizard of Wikipedia
dataset. In contrast, it achieves a deficient performance on the language quality-related metrics, which implies that models only copy knowledge words but generate gibberish responses without training. Nevertheless, PLUG still generates knowledge-grounded responses with a lower knowledge F1 score out-of-the-box. This result also suggests that we should only consider knowledge F1 scores when the model has decent scores on language quality metrics.
For the REDIAL dataset, Figure 3 (d) shows that there is not as much improvement in recommendation item recall brought by pre-training when compared to BLEU-4 and ROUGE-L on a zero-shot setting. However, we observe a noticeable difference between PLUG and the T5 model, which means PLUG learns to generate with grounded knowledge faster than the T5 model. The unusually high DIST-4 of T5 in Figure 3 (d) is caused by diverse but irrelevant responses. It is also demonstrated by low BLEU-4 and ROUGE-L scores in Figure 3 (a) and Figure 3 (b), and the decrease of DIST-4 when we increase the training data size.
3.6 Human Evaluation
We conduct a human evaluation on Wizard of Wikipedia to assess the overall quality of the responses of our model compared to T5 and RAG<sup>7</sup>. Specifically, we randomly select 100 responses for each model with the same context from Test Seen and Test Unseen. For the few-shot setting, we use the models trained with 50 dialogues. We hire