Fact Checking Machine Generated Text with Dependency Trees (2022)

Abstract: Factual and logical errors made by Natural Language Generation (NLG) systems limit their applicability in many settings. We study this problem in a conversational search and recommendation setting, and observe that we can often make two simplifying assumptions in this domain: (i) there exists a body of structured knowledge we can use for verifying factuality of generated text; and (ii) the text to be factually assessed typically has a well-defined structure and style. Grounded in these assumptions, we propose a fast, unsupervised and explainable technique, DepChecker, that assesses factuality of input text based on rules derived from structured knowledge patterns and dependency relations with respect to the input text. We show that DepChecker outperforms state-of-the-art, general purpose fact-checking techniques in this special, but important case.

## introduction and background prior work has noted the benefits of natural language text generated by nlg models over fixed templates, for various language processing and language understanding tasks, i.e., fewer grammatical or structural disfluencies, increased conversational nature, textual diversity and user satisfaction @xcite @xcite . recent nlg approaches based on language models allow conversational agent designers to create natural-sounding responses in a very wide range of situations without exhaustively testing templatebased response text @xcite @xcite . however, neural nlg systems may emit hallucinated, factually incorrect or even nonsensical content that is not entailed by their inputs @xcite @xcite . for instance, an nlg model may synthesize multiple independent hypotheses involving comparisons or aggregations, which must be verified. there has been growing attention towards checking the factual accuracy of statements produced by neural nlg systems in conversational settings. these statements fall into two categories depending on the source input data used for generation @xcite . first, the input whose factuality is to be determined can be unstructured, as in abstractive summarization @xcite or the fever task @xcite @xcite @xcite . second, the input to be factually assessed may involve structured components, like knowledge graphs @xcite or tabular data @xcite @xcite . in conversational, high-consequence domains like health, finance, or e-commerce, we must be sure that any factual error introduced by an nlg system is detected @xcite @xcite . while superficially similar to other fact verification efforts, this setting presents both unique challenges, and unique simplifications. in this work, we investigate the factuality of fluent e-commerce statements generated via neural nlg techniques using structured data, namely, a commercial product catalog @xcite containing tables of products and their attributes. we show that the state-of-the-art systems typically developed for general-purpose structured data fail to perform well in our conversational e-commerce setting. though we focus on the e-commerce domain, our proposed system can be used with any structured data source to check statements generated from non-e-commerce domains such as finance (e.g., the recent finqa challenge @xcite or health (consider the consequences of providing inaccurate health recommendations to users of personal health trackers in a system like @xcite ). we focus on three main challenges while building our proposed fact checking model, depchecker. first, since presenting factually untrue statements to users in high-consequence domains may lead to a poor decision, we must achieve very high false statement recall, and very high true statement precision. second, deploying fact verification in a conversational setting demands speed -during runtime evaluation of responses to customers of voice assistants, and in case we want to use fact verification output as a signal to improve an accompanying conversational response generation model. as a result, we seek to avoid using large language models for fact checking in production, if possible. third, data-to-text nlg systems may rephrase entities from the input values, e.g., if a data entity is itself disfluent. in benchmark corpora like tab-fact @xcite , data and reference text mentions of the same entity are generally identical, which can limit the flexibility of fact verification models trained on those corpora. depchecker also benefits from some simplifications, especially in the e-commerce domain. we observe that in the conversational product search and recommendation setting, nlg systems are unlikely to generate product statements that have a complicated linguistic style or form @xcite . thus, unlike tabfact which requires complex hops and joins to reason over structured table data, we can concentrate on a simpler, reduced set of operations. we are inspired by @xcite , who approached question answering by developing rules over dependency parse trees. consider the example in figure 1 : key entities have very simple structures. the product mention has an nsubj (nominal subject) dependency foot_0 ; attributes are either attr (attribute) or conj (conjunct) dependents of attr. these basic observations hold over a wide set of examples in this domain, prompting us to exploit dependency trees to better extract claims, perform entity linking to match the input statement with its associated structured data, and finally determine the factuality of the statement. a current limitation of depchecker could be that manually constructing rules over parse trees requires both time and domain expertise. as a result, we also seek to automatically induce the manually generated rules using genetic optimiza-tion algorithms @xcite , in a system we call depcheckerga. our contributions can thus be summarized as follows: ## evaluation to test our proposed approaches, depchecker and depcheckerga, we first developed a bartbased @xcite ) data-to-text nlg model to generate conversational statements in the ecommerce domain, taking as input tabular productattribute data from a structured product catalog @xcite . to train this nlg model, we asked human annotators to write text describing and comparing the product attribute values of 2-3 catalog products (figures 1 and 2 contain examples of the human generated sentences). we used this set of nlg model generated sentences as our test set for factuality assessment, to test depchecker in this work. as a baseline for depchecker, we also trained a supervised factual error detection binary classifier (henceforth called robertachecker) using the roberta @xcite language model, with a mix of the above mentioned human-generated product domain statements and artificially generated negative examples (generated by corrupting the entity names and values in the human-generated factually correct statements). this binary classification dataset was also used to fine-tune the neural baselines shown in ## limitations the major limitation of our work is that de-pchecker in its basic form requires manual effort to construct rules for fact checking. while we showed that this rule construction can be automated, it comes with a trade-off in the compactness of the automatically generated rules and the overall latency. however, as shown in sections 3.1 and 3.2, large, neural language models developed for fact checking with structured general purpose data (e.g., tabfact derived from wikipedia tables) are unable to perform fact checking well on statements based on our e-commerce structured catalog data. the time required to assemble such a large volume of labeled data in specific domains (e.g., health or e-commerce) to train large neural models would certainly cost more than one or two weeks of a linguist's time, and may still not be fast or accurate enough to be deployed for production purposes. hence, it isn't clear how much effort would be required to build a neural system comparable in speed and accuracy to depchecker, or that the benchmark systems scale or generalize better than depchecker for a real-world deployment scenario. like existing data-to-text fact checking work, depchecker also assumes that the underlying structured data used to verify the statement factuality is accurate. finally, we acknowledge that rule-based approaches are not new, but as we show in this work, they can still outperform or at least complement state-of-the-art language models. their speed, simplicity, accuracy, and explainability make them valuable tools in production use cases. since seemingly state-of-the-art results based on large language models may not generalize as well as hoped across different domains and datasets, we would like to point out that our rule-based approach, de-pchecker, remains competitive in many scenarios. such a system can therefore still be worth investing in, especially in an age of larger and slower models. ## conclusion we propose depchecker: a fast, unsupervised, and explainable model to detect factual inconsistencies in a conversational shopping and recommendation setting. we show that depchecker can outperform strong baselines, and suggest that instead of pursuing costlier, less explainable, slower models, more research should go into how to further automate development of simpler, less expensive and more interpretable models for production use cases.