Factual or Contextual? Disentangling Error Types in Entity Description Generation (2023)

Abstract: In the task of entity description generation, given a context and a specified entity, a model must describe that entity correctly and in a contextually-relevant way. In this task, as well as broader language generation tasks, the generation of a nonfactual description (factual error) versus an incongruous description (contextual error) is fundamentally different, yet often conflated. We develop an evaluation paradigm that enables us to disentangle these two types of errors in naturally occurring textual contexts. We find that factuality and congruity are often at odds, and that models specifically struggle with accurate descriptions of entities that are less familiar to people. This shortcoming of language models raises concerns around the trustworthiness of such models, since factual errors on less well-known entities are exactly those that a human reader will not recognize.

## introduction gricean maxims of effective communication @xcite as they pertain to referring expressions @xcite posit that referring expressions should not convey false information and that they should be relevant to context. factuality and congruity are thus the two main properties of pragmatically appropriate referring expressions. following these maxims, human-written referring expressions strongly adhere to the principles of factuality and congruity @xcite . standard evaluation practices for referring expression generation @xcite @xcite , however, only distinguish between model generated referring expressions being accurate (ground-truth) versus inaccurate (not groundtruth), without considering factuality and congruity of the model outputs. however, this distinction ## referring expression generation open-book referring expression generation involves two steps: claim selection and description generation @xcite . claim selection identifies facts from the target entity's wikidata (referred as claims) relevant to the context. description generation produces the text, conditioned on the context and relevant claims, for instance using an auto-regressive encoder @xcite . traditionally, such generation tasks are evaluated using text similarity metrics, such as rouge @xcite or bleu @xcite , of the pre-dicted description with respect to the ground-truth description. however, the failure modes beyond the ground-truth are usually left unexplored. here we develop a framework for evaluating generated descriptions against various reference texts, controlled to be accurate, nonfactual, or incongruous, to sharpen our understanding of model errors. we also design multiple choice evaluations for claim and description identification, with distractors that are nonfactual, incongruous, or both. this multiple-choice setup allows more control over the chosen alternatives to highlight the factual and contextual preferences in language models. for instance, in the top-left example in figure 2 , the claim identification model identifies silvio berlusconi as a television presenter rather than as the italian prime minister, possibly because the context text mentions "television talk show." ## entity familiarity xi jinping stephen hawking newt gingrich raul castro justin bieber george lucas richard nixon moammar gadhafi ted cruz woody allen kristen stewart kelly ripa robin williams richard pryor mel gibson ramsey clark brian panish jocelyn maldonado nikolai valuev tony plana heuristic human figure 3: familiarity: heuristic vs human annotation. ## experiment details we conduct our experiments on the pomo dataset @xcite , which contains english news articles from cnn, daily mail and new york times, along with their summaries. the dataset also identifies post-modifier description for an entity mentioned in the article and a set of claims from target entity's wikidata. we extract all person entities mentioned in the article using ner tagging @xcite . we use claims associated with the target entity and other entities mentioned in the article to construct alternatives for claim identification. for description identification and generation experiments, we extract referring expressionspre-modifier, relative clause, appositive, participle clause, adjective/adverb clause and prepositional phrase-using the regular expressions of @xcite . we use the same set of referring expressions for familiarity heuristics. we use a dependency parser @xcite to extract the descriptions associated with entities' first mention. 2 we identify alternative descriptions of an entity in other articles based on full name matches. within an article (or summary), we match entities based on first or last name @xcite . we validate the familiarity heuristic by collecting human annotations on a subset of 200 person entities in the validation set, with 3-4 annotations per entity. we ask crowdworkers on amt (paid ∼us@xmath015/hour. for the description generation task, we construct nonfactual reference text as the context excluding factual description of the entity, to enforce the non- for the claim and description identification task, we construct a gold test set with the subset of examples where our extracted distractors agree with the human annotations. we obtain 41 and 38 gold examples in the claim and description identification, respectively. we also internally validate the factuality assumption for the gold set. we manually check that the incongruous distractors in the gold test set are factual and the nonfactual distractors are not. the evaluation on the gold set is limited to identification tasks. for description generation, human annotators verify only the generated description that overlap with incongruous reference text, not figure 6 shows similar trends on familiar and unfamiliar entities in the human annotated set (gold test set) as seen previously on automatically annotated test sets (figure 5 ). models make more factual errors for unfamiliar entities and more contextual errors for familiar entities (𝑝<0.05), both for claim and description identification tasks. ## measurement validity above, we analyzed factual and contextual preferences in language models on three tasks pertaining to referring expression generation. throughout, we have shown that the language models we measure tend to make both factual errors and congruity errors, and also have shown that these measures are-in various ways-actually measuring what we purport them to measure. formally, we can conceptualize this from a measurement modeling perspective @xcite , wherein we consider factual and contextual errors to be unobservable constructs. in this view, we have proposed three measurement devices as proxies for the unobservable construct. this forms a measurement model, which we can example from the perspective of measurement validity, considering: face validity (the extent to which the measures look plausible), content validity (the extent to which the measures capture the substantive nature of the construct), convergent validity (the extent to which the measure matches related measures), concurrent validity (the extent to which the measures distinguish between groups that it should meaningfully be able to distinguish between), and consequential validity (the implication of using the measure). face validity is inherently subjective in nature. we pose that the incongruous (but factual) and nonfactual (but contextual) alternatives respectively encode the factual and contextual cues that the models refer to. without any external criteria, readers are often the only judge of the face validity. ## conclusion in this work, we integrate indicators for factual inconsistencies and contextual incongruities in automated evaluation in referring expression generation. our paper aims at conducting post-hoc analyses of language models in referring generation task to assess the differences in error types across familiar vs unfamiliar entities that are not reflected in the aggregate accuracy numbers alone. comparisons with alternative factual and contextual reference text, in addition to those with ground-truth description, suggest that description generation models heavily mis-generate incongruous and nonfactual description. we also show this effect using controlled multiple-choice experiments for claim and description identification. further, we show that language models disproportionately rely on context when describing less familiar people, resulting in factually incorrect descriptions. the tasks discussed in our work are meant to exemplify the disparities in language model errors to advocate further research investment in expanding language model evaluation beyond accuracy. our work opens avenues for future research on the effects of augmenting language models with retrieved information on the factuality and congruity of the generations: do retrieval-augmented language models still make factual errors and are these models able to maintain the contextuality of the generated descriptions. further, more work is needed to study to what extent language models abide by the conversational pragmatics in the wild.