On Reducing Factual Hallucinations in Graph-to-Text Generation Using Large Language Models (2025)

Abstract: Recent work in Graph-to-Text generation has achieved impressive results, but it still suffers from hallucinations in some cases, despite extensive pretraining stages and various methods for working with graph data. While the commonly used metrics for evaluating the quality of Graph-to-Text models show almost perfect results, it makes it challenging to compare different approaches. This paper demonstrates the challenges of recent Graph-to-Text systems in terms of hallucinations and proposes a simple yet effective approach to using a general LLM, which has shown state-of-the-art results and reduced the number of factual hallucinations. We provide step-by-step instructions on how to develop prompts for language models and a detailed analysis of potential factual errors in the generated text.

## introduction knowledge graphs kg have become a powerful tool for organizing and representing complex data due to their ability to easily manage trusted information and are used in various industries such as education, healthcare, and social media @xcite . they can be used in conjunction with modern large language models (llms) to create retrieval-augmented generation (rag) systems or to validate the information generated or retrieved. due to the better validation and understanding of data, it is important to properly translate them into text. this process, known as graph-to-text generation, has recently seen success in creating knowledge-grounded chatbots @xcite and question answering systems @xcite @xcite . @xcite demonstrated that integrating various external resources, particularly linearized subgraphs, and employing a marginal probability-based selection method significantly enhanced the effectiveness of the rag setup. graph-to-text involves the processing of knowledge graph triplets (subject, property, object) data, into a natural textual representation that should include all the factual information from these triplets and nothing else, as shown in the figure 1 . in this work, we focus on evaluating the abilities of modern large language models, such as chatgpt foot_0 , @xcite and gemma 2 @xcite , to solve the graphto-text problem, and specifically on the potential hallucinations that are totally unacceptable for such problems. recent studies have produced state-of-the-art results in the graph-to-text generation task. they used complex pipelines to organize graphs in a specific order to generate correct text @xcite , or used graph aware approaches @xcite . all of these methods required complex training stages, but they provided the best results according to the leaderboards foot_1 . despite the impressive results, these graph-to-text methods can still experience hallucinations and may omit certain elements of the graph in the resulting text which are difficult to detect with the common metrics used on popular leaderboards. in this paper, we focus on this issue and provide a detailed analysis of state-of-the-art methods, comparing them with general language models. our contribution are two-fold: we make our code publically available to provide reproducible results and motivate future researchers. 3 ## related work in this section, we will provide a brief overview of knowledge graphs, their representation and linearization, as well as existing graph-to-text algorithms and the potential use of large language models for this task. ## proposed framework we propose a universal and easy-to-use framework for the graph-to-text task that does not require finetuning or the use of specialized trained modules or models, yet still achieves state-of-the-art results. we use common instruction-based large language models, such as llama 3 (dubey et al., 2024), gemma 2 @xcite or gpt-4o 6 , to generate comprehensive, natural-style text from kg triplets using carefully selected prompt with various prompting techniques. in our work, as with any system development, we start from a simple zero-shot baseline and simply ask the llm to convert kg triples into text using the following prompt: "translate from graph to text". this straightforward approach suffers from a lot of hallucinations, so we asked the model not to hallucinate: "describe all nodes of the graph with edges as a connected text. talk only about items from graph and use information only if graph contains it. write only description without headers and titles.", and it actually works, with better results. after that, we tried adding some general hacks, such as a few short learning examples and a chain of thoughts, to improve it. finally, we provided the following prompt template: birthdate occupation figure 2: visualization of the graph described in the now provide answer for the next task yourself. graph: linearized graph model answer: ## experiments opposite to the previous works @xcite for the evaluation we have used more actual large language models such as open-source llama-3-8b-instruct, gemma-2-9b and proprietary gpt-4o and gpt-4o-mini. as a source of knowledge graphs we have chosen webnlg 2020 english dataset as it is one of the main benchmarks for the graphto-text task. all models were evaluated on the test set using modified tripleset -graph description in the triple form where some predicates are replaced according to the official webnlg recommendation foot_4 . it helps models better understand directions of the links between entities. each graph was linearized in the order triples are stated in the dataset. example of the final graph input format is @xcite to measure the results we have used standard webnlg metrics:meteor @xcite , bleu @xcite , chrf @xcite , ter @xcite and bertscore @xcite . moreover, we additionally computed alignscore metric @xcite to detect factual inconsistency in the model answers. for comparison reasons we have also evaluated gap @xcite and calculated metrics for the p 2 model @xcite , which is top-1 solution from webnlg 2020 competition, based on the model outputs published by authors 8 . results of our evaluation are presented in the while by some metrics we can easily define the better model, it can be seen that bertscore f1 is nearly equal both for large language models and p 2 and requires more detailed analysis. ## analytics while by classical translation metrics large language models are slightly worse that the p 2 model, 8 https://github.com/qipengguo/p2_webnlg2020/ blob/main/output.txt it was expected as fine-tuned models were adopted for the style of reference answers during training on the train part of the dataset and these metrics reward word match @xcite . on one hand it gives p 2 advantage, but on the other hand it can't be applied to another dataset without extra fine-tuning process, while large language models can be evaluated just with other examples in few-shot part of the prompt. as difference between large language models and p 2 by bertscore f1 is at the margin of statistical error we go deeper and compared factual consistency of the generated results with alignscore. while large language models all as one show high score by this metric, p 2 demonstrates much worse quality. it can be explained by hallucinations or missed facts in the model answers. to define the reasons of such problems with factual consistency we reviewed examples from the dataset where p 2 suffers from fact inconsistency, but two best of compared large language models (gemma 2 and gpt-4o) still provide high-quality results. one pattern we detected is that p 2 model tends to hallucinate if graph contains multiple triples with the same subject and property but different objects. examples of the such graph are provided in the to sum up, p 2 shows great results by classic translation metrics because of special graph reordering and language model fine-tuning which makes model answer similar to the references, but still suffers from hallucinations more than modern large language models because of the under the hood t5 model limitations. while generating the result with p 2 requires less computational resources it can be used in further processing only after factual consistency check to detect possible skipped or incorrect facts. ## conclusion we have shown how to select prompt for the modern large language model to show competitive results in graph-to-text task with top-1 specialized model fine-tuned for the specific dataset. we provide final few-shot prompt based on the chain-of-thoughts technique. also we evaluated four modern large language models on the graph-to-text task and compared their results with state-of-the-art solution in this task. based on these evaluation we provided detailed analytics about potential factual errors which can be caused by model hallucinations. moreover, we described how they can be omitted behind classic translation metrics on the example of the p 2 model. english without tears 89.0 runtime nicholas brodszky musiccomposer 1944-07-28 releasedate harold french director alan jaggs editing anatole de grunwald producer writer reference english without tears is a 89 minute film that was written and produced by anatole de grunwald. the movie was released on july 28th, 1944. nicholas brodszky was the musical composer and harold french was the director. the editing was done by alan jaggs. p 2 english without tears was produced by anatole de grunwald and written by nicholas brodszky. it was edited by alan jaggs and released on 28th july 1944. it has a runtime of 89.0 and was directed by harold french.