Hallucination Mitigation in Natural Language Generation from Large-Scale Open-Domain Knowledge Graphs (2023)

Abstract: In generating natural language descriptions for knowledge graph triples, prior works used either small-scale, human-annotated datasets or datasets with limited variety of graph shapes, e.g., those having mostly star graphs. Graph-to-text models trained and evaluated on such datasets are largely not assessed for more realistic large-scale, open-domain settings. We introduce a new dataset, GraphNarrative, to fill this gap. Fine-tuning transformer-based pre-trained language models has achieved state-of-the-art performance among graph-to-text models. However, this method suffers from information hallucination—the generated text may contain fabricated facts not present in input graphs. We propose a novel approach that, given a graph-sentence pair in GraphNarrative, trims the sentence to eliminate portions that are not present in the corresponding graph, by utilizing the sentence’s dependency parse tree. Our experiment results verify this approach using models trained on GraphNarrative and existing datasets. The dataset, source code, and trained models are released at https://github.com/idirlab/graphnarrator.

## introduction the task of graph-to-text generation aims to automatically produce natural language descriptions of knowledge graphs. a knowledge graph g stores factual information as subject-predicateobject triples, where each triple (s, p, o) corresponds to an edge from the subject entity s to the object entity o. the graph-to-text generation task entails, given a subgraph g⊂g, generating a token sequence (y 1 , ..., y n ) to describe g. this task can be accomplished by constructing machine learning models @xcite @xcite . the input to such a model is a graph itself-a small fragment of triples from a knowledge graph, as the outcome of some upstream operation, e.g., search, query and data mining. the output is a textual sequence that describes the fragment of triples. verbalizing triples from knowledge graphs is crucial in a variety of tasks and applications, including systems created for querying knowledge graphs @xcite as well as systems backed by knowledge graphs for question-answering @xcite and fact discovery @xcite . in these places, knowledge graph fragments must be conveyed to users in various forms, such as query results and discovered facts. though a tiny part of a whole knowledge graph, such graph fragments can still be complex and thus challenging to comprehend. instead, presenting them in natural language can help end users understand them better. in graph-to-text generation, the preciseness and naturalness of the textual narration of graph fragments is important. generating high-quality text can be particularly challenging for large-scale and open-domain knowledge graphs. specifically, benchmark datasets in this line of research either are hand-crafted and monotonous, e.g., webnlg @xcite or only include simple, special formations in narrated input fragments, e.g., eventnarrative @xcite and tekgen @xcite . existing graphto-text models, being trained and evaluated on these datasets, are largely not validated for more realistic large-scale, open-domain settings. section 2 presents this analysis in detail. this paper introduces graphnarrative, a new dataset that fills the aforementioned gap between graph-to-text models and real-world needs. graphnarrative consists of around 8.7 million (input graph, output text) pairs. the text in each pair is a wikipedia sentence, whereas the corresponding graph comprises freebase @xcite entities and relationships described in the sentence. the large-scale of both wikipedia and freebase, the linguistic variation in wikipedia, and the complexity of sentences and corresponding graph structures make this dataset more aligned with real-world scenarios. for instance, graphnarrative's 8.7 million input graphs are in 7,920 distinct topological shapes and 22% of the 8.7 million are star graphs, in contrast to 94% and 96% in eventnarrative and tekgen, respectively. section 3 articulates the details of graphnarrative's creation. given the demonstrated efficacy of fine-tuning pre-trained language models (plms) in producing state-of-the-art results on graph-to-text (more details in section 4), we adopt the same approach. as pointed out in @xcite , though, this approach may suffer from information hallucination, i.e., the output texts may contain fabricated facts not present in input graphs. for example, given a two-triple input graph {(neff maiava, date of birth, 01 may 1924), (neff maiava, date of death, 21 april 2018)}, @xcite reported their model generates "neff maiava (1 may 1924 -21 april 2018) was an albanian actor." not only the input does not mention maiava's profession or citizenship, but also in the real-world he was an american samoan wrestler instead. very few have considered how to mitigate hallucination in graph-to-text generation, except for @xcite @xcite . the first two studies attempted to address hallucination by further fine-tuning plms on webnlg after fine-tuning on noisier automaticallyextracted datasets. @xcite adopted a different approach, by filtering out training instances when the rouge-1 @xcite scores between the input and the output fall below a certain threshold. however, these studies did not quantify the prevalence of hallucination in their models' outputs. nor did they provide direct experiment results or other evidence to verify the approach in reducing hallucination. we are the first to quantitatively measure the prevalence of hallucination in graphto-text. we also developed a novel approach to mitigating hallucination by aiming at the problem's root-mismatch between graph and text in training data. given a graph-text pair in graphnarrative, the approach trims the text, i.e., a wikipedia sentence, by eliminating portions not represented in the graph. this process, named sentence trimming, is accomplished by analyzing the shortest paths between graph entities within the sentence's dependency parse tree (details in section 5). we conducted comprehensive automatic and human assessments of text descriptions generated by fine-tuned plms, specifically bart @xcite and t5 @xcite . the automatic evaluation results consistently demonstrated that models performed better with the use of sentence trimming, across the datasets of graphnarrative, tekgen, webnlg, and dart @xcite . the approach led to the increment of 12 and 7 points in bleu score @xcite for graphnarrative and tekgen, respectively. a t5large model fine-tuned on graphnarrative with sentence trimming achieved state-of-the-art results on the webnlg benchmark. furthermore, human evaluation results showed that sentence trimming on average reduced 1.4 entity hallucinations and 1 relationship hallucination per text description. the contributions of this paper are as follows. ## limitations of existing datasets first, most previous models were trained on small hand-crafted datasets that contain limited entity types and relations. for instance, webnlg includes 2,730 distinct entities and 354 distinct relations. in contrast, real-world knowledge graphs can be much larger. for example, according to @xcite , wikidata @xcite has 52,252,549 entities, 2,356,259 classes, 6,236 relations, and 732,420,508 triples. the handcrafted approach cannot scale to these massive knowledge graph, as it is impossible to manually write training graph-text pairs for so many different entity types, relations, and topic domains. second, the text descriptions in hand-crafted datasets such as webnlg tend to follow monotonous templates, plausibly because the examples were written by a small number of human contributors. this limits the capability of trained models to use diverse expressions in narrating graph fragments. this lack of linguistic variation can hamper the usability of a text generation system. third, the graph fragments in existing datasets are largely limited to simple star graphs (each graph consisting of a center entity and some of its one-hop neighbors) or more general acyclic graphs (i.e., one or more trees). the graphs in webnlg have 41 distinct topological shapes (appendix d), out of which 32 are acyclic graphs. the cycles are all 2-edge loops or self-loops. in dart, 83% of the graphs are star graphs. in automaticallygenerated datasets eventnarrative and tekgen, 94% and 96% of the graphs are star graphs, respectively. another automatically-collected dataset, agenda @xcite , has only 2% star graphs. but it only contains 7 distinct relations in the special domain of scientific research. on the contrary, in practical scenarios the input fragments can be of complex, general rather than simple, special formations. while direct measurement is lacking, we used the graphs described in wikipedia sentences as a proxy for gauging the shape diversity of graphs that need to be narrated. we manually analyzed the formations of graphs presented in 100 random wikipedia sentences, and we found only 39 of the 100 graphs are star graphs. similar but automatic analysis of the complete wikipedia corpus (more details in section 3, figure 2 ) show that only 2 of the 10 most frequent graph formations foot_0 are star graphs, and 3 are cyclic graphs. ## the graphnarrative dataset this section explains how we generated our new dataset graphnarrative by aligning wikipedia texts with freebase. note that the methodology could be applicable to text corpora beyond wikipedia and knowledge graphs beyond freebase. this section also contrasts graphnarrative with existing benchmark datasets to demonstrate how it addresses current datasets' limitations. ## mitigation of hallucination the culprit of the hallucination problem discussed in section 1 is fabrication in training data-textual descriptions containing information not found in input graphs. this is evidenced by that, while graph-to-text models frequently produce hallucination when trained on tekgen, it rarely happens on webnlg. hallucinated facts are seldom found in the clean, manually-crafted webnlg but are present in automatically extracted graph-text pairs in tekgen due to extraction errors. there could be two plausible directions in tackling graph-to-text hallucination. one is to improve our graph-text alignment method (section 3.1). the graph extracted from a piece of text during alignment may miss certain entities or relationships due to either extraction errors or disparities between the text corpus and the knowledge graph. the resulting graph-text pair may misguide the trained model to hallucinate facts. a more accurate alignment method can reduce such erroneous pairs and thereby reduce hallucination. however, this method has an inherent limitation-since a knowledge graph in real-world is often far from complete, there will be facts in text that cannot be mapped to the knowledge graph. nevertheless, in principle, a way to combine this approach with the other approach discussed below is open for investigation. this study explores a different direction in mitigating hallucination. given a (freebase subgraph g, wikipedia sentence w ) pair produced by alignment, we introduce a sentence trimming algorithm (algorithm 1 in appendix a) to turn w into a trimmed sentence w trim by eliminating portions that are not present in g while preserving the sentence's main idea. below we provide a sketch of the algorithm, while keeping its pseudo code and description in appendix a. first, the algorithm parses w and generates its dependency parse tree (dpt) w tree , using spacy @xcite . then, for each triple t @xmath0, p i , o i ) ∈ g, it identifies the shortest dependency path (sdp) between s i and o i , i.e., the shortest path between the two entities' tokens in w tree . it then finds the leftmost position index min_pos in sentence w among all tokens on all triples' sdps, and similarly the rightmost position index max_pos. this process results in the trimmed sentence w trim , a sub-sequence of w spanning from min_pos to max_pos. an example is in figure 3 which illustrates the note that, a regular dpt will break up entities such as backup software into individual tokens, each for a node in the dpt. to avoid that, we used a modified concept of dpt-we preprocessed entity names and tokenized each entity's name into a single token. speficially, the two tokens backup and software were combined into token backupsoftware. ## conclusion in this paper, we proposed a novel approach to mitigating hallucination in natural language generation from large-scale, open-domain knowledge graphs. we released a large graph-to-text dataset with diverse graph shapes that fills the gap between existing datasets and real-world settings. the experiment results show the effectiveness of our hallucination mitigation approach as well as the usefulness of the dataset.