Can LM s Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge (2023)

Abstract: Pre-trained language models (LMs) are used for knowledge intensive tasks like question answering, but their knowledge gets continuously outdated as the world changes. Prior work has studied targeted updates to LMs, injecting individual facts and evaluating whether the model learns these facts while not changing predictions on other contexts. We take a step forward and study LMs’ abilities to make inferences based on injected facts (or propagate those facts): for example, after learning that something is a TV show, does an LM predict that you can watch it? We study this with two cloze-style tasks: an existing dataset of real-world sentences about novel entities (ECBD) as well as a new controlled benchmark with manually designed templates requiring varying levels of inference about injected knowledge. Surprisingly, we find that existing methods for updating knowledge (gradient-based fine-tuning and modifications of this approach) show little propagation of injected knowledge. These methods improve performance on cloze instances only when there is lexical overlap between injected facts and target inferences. Yet, prepending entity definitions in an LM’s context improves performance across all settings, suggesting that there is substantial headroom for parameter-updating approaches for knowledge injection.

## introduction pre-trained language models (lms) acquire comprehensive real-world knowledge from massive amounts of pre-training data, allowing them to use this knowledge effectively in downstream tasks. however, without continual updating, the knowledge contained within these backend lms will eventually become outdated. this temporal mismatch affects model performance on downstream tasks @xcite @xcite . as lms become more widely deployed, their knowledge : who is the uk's pm? evaluation (inference based on the updated fact): figure 1 : knowledge editing tasks. we study a challenging entity knowledge propagation task where language models should make inferences after learning entities from their definitions. this differs from past knowledge editing which evaluates paraphrases of injected facts. prior work has investigated knowledge editing in pre-trained lms, updating model parameters to alter outputs to match what users want @xcite @xcite @xcite . in these studies, the original fact and the altered fact are provided (e.g., changing "x was born in y." to "x was born in z."), and models are evaluated after a single update on each instance; see figure 1 for an example. these model editing methods successfully provide targeted updates, fixing incorrect or outdated individual facts. yet, can lms make inferences based on updated knowledge? past evaluation has largely focused on two aspects of knowledge editing, whether the edits were successfully injected and whether other irrelevant sentences were impacted, but do not capture whether the lms now can reason based on the new fact that has been injected. we take a step further and evaluate whether lms can propagate updated knowledge about new entities. we first inject definitions about the entity into lms using various knowledge editing methods @xcite , then evaluate lms' performance on cloze tasks on a wide range of sentences about the entity (see figure 1 for an example). we refer to this task as entity knowledge propagation and introduce two cloze datasets to evaluate this challenging task. our first evaluation benchmark is the entity cloze by date (ecbd) dataset @xcite , which presents novel entities tagged with origination dates (e.g., hurricane ian, 2022), their definition and probe sentences taken from their wikipedia page. the task is to fill a masked span in probe sentences. because wikipedia contains a wide range of information, much of it not inferable from an entity's definition, injecting entity knowledge via its definition has an unclear impact on the probe sentences; filling in the masked span is nontrivial even after the entity definition is provided. for more controlled study, we introduce a new benchmark (entity inferences) with manually designed probe sentences with multiplechoice answer options. once one learns about the definition of an emerging entity, finding the correct answer for these probe sentences is easy. we find that existing parameter updating methods can handle simpler inferences in entity in-ferences, but fail to improve performances in ecbd, revealing a limitation in these methods. we further analyze the impact of fine-tuning. distressingly, we find that simply prepending information in-context works very well, and matching the performance of this via parameter updates is challenging. a deeper analysis finds that model editing shows promising results only when the injected definition sentence and the cloze inference have lexical overlap. our work establishes an evaluation paradigm and opens doors for work on editing methods that can propagate entity knowledge. the code and data are available at https://github.com/yasumasaonoe/ entity_knowledge_propagation . ## entity knowledge propagation we propose entity knowledge propagation (ekp), a new task where we want to update model param-eters to reflect an emerging entity that is unseen in the lms' pre-training corpus. for example, bert was trained in 2018, so covid-19 is an emerging entity to bert. we explore various ways of editing model parameters based on definition sentences to inject new knowledge. once we inject the knowledge of the emerging entity into the model parameters, we evaluate the updated model's ability to reason about the emerging entity. ## constructing benchmarks for ekp we use two different types of datasets to investigate how new entity knowledge is propagated into the lm's parameter space. ## results ecbd, and ecbd-easy, and then draw larger conclusions. ## analysis we analyze the challenges in knowledge propagation by first estimating an informal upper bound of model editing performance (section 6.1). we then examine how the similarity between the definition sentence and probe sentence impacts the performance of model editing (section 6.2), inspired by positive performances on ecbd-easy subset. we conduct our analysis with gpt-neo base model on random subsets of entity inferences (half the data) and ecbd (100 np span and 100 random span) to reduce computational costs. for our three datasets (entity inferences, ecbd, and ecbd-easy), we finetune a model for a range of 1 to 8 epochs (i.e., the number of updates). we use a learning rate of 5e-5 for en-tity inferences and plot the specificity score vs accuracy. for ecbd and ecbd-easy, we choose a learning rate of 3e-5 and then compare the specificity score and perplexity. these learning rates were chosen to optimize performance from the range of values described in appendix a.3. findings figure 2a depicts the perplexityspecificity tradeoff curves of fine-tuning approach on ecbd dataset. the perplexity and the specificity score by the base model are drawn as the horizontal dotted line and the vertical dotted line respectively. ideally, we want a model to achieve low perplexity and the specificity score identical to the base score (performance in the lower left corner). on ecbd, we see that standard ft shows an upward trend: with larger parameter updates, we worsen the specificity as expected, but also perplexity, meaning that finetuning for longer does not usefully propagate entity information from the definition sentence into the model. input augmentation (prepend-def) performs robustly, indicating that the issue is potentially due to how the data is used in learning rather than the data itself. how does this align with past results? ecbd-easy (figure 2b ) shows a much more optimistic picture; recall that this is similar to the setting from @xcite . in this case, mend and finetuning both achieve results reasonably close to the train-on-test upper bound, with configurations that improve perplexity substantially with only mild specificity degradation. methods that succeed on injection of exact facts (e.g., injecting y and reproducing it later) do not necessarily transfer to success in realistic knowledge propagation settings like ecbd. ## conclusion in this work, we explored the entity knowledge propagation setting: to what extent can descriptions of new entities be injected into language models? we find that while fine-tuning models or using efficient update strategies enables models to reproduce exact facts from descriptions, performing inferences based on those facts is substantially harder. we characterize several approaches on two datasets and conclude that update strategies lag the performance of simply prepending the definition in the context, suggesting that more work is needed.