Why Does New Knowledge Create Messy Ripple Effects in LLM s? (2024)

Abstract: Extensive previous research has focused on post-training knowledge editing (KE) for language models (LMs) to ensure that knowledge remains accurate and up-to-date. One desired property and open question in KE is to let edited LMs correctly handle ripple effects, where LM is expected to answer its logically related knowledge accurately. In this paper, we answer the question of why most KE methods still create messy ripple effects. We conduct extensive analysis and identify a salient indicator, GradSim, that effectively reveals when and why updated knowledge ripples in LMs. GradSim is computed by the cosine similarity between gradients of the original fact and its related knowledge. We observe a strong positive correlation between ripple effect performance and GradSim across different LMs, KE methods, and evaluation metrics. Further investigations into three counter-intuitive failure cases (Negation, Over-Ripple, Multi-Lingual) of ripple effects demonstrate that these failures are often associated with very low GradSim. This finding validates that GradSim is an effective indicator of when knowledge ripples in LMs.

## introduction and related work large language models (llms) can serve as powerful knowledge bases (kbs) thanks to their impressive knowledge storage, retrieval, and reasoning capabilities @xcite @xcite . however, real-world knowledge keeps updating and evolving constantly, which motivates extensive research efforts on posttraining knowledge editing (ke) @xcite @xcite @xcite to make sure that the knowledge in lms remains accurate and up-to-date. 1 work done during internship at uiuc. 1 https://github.com/jiaxinqin0814/ripple_effect_analysis . ## gradsim: a ripple effect indicator in this section, we formally introduce gradsim, a ripple-effect indicator based on the knowledge storage similarity between related knowledge. we use x and y to denote a pair of original fact and its related knowledge respectively, and we use (q x , a x ) and (q y , a y ) to represent query-answer pairs based on the corresponding knowledge facts. for instance, if q x and a x @xmath0, then one example pair of q y and a y could @xmath1. given a query q x , typical ke methods update a x to a new answer a ′ ## experiments we assess the effectiveness of gradsim by empirically examining its correlation with ripple effect performance, aiming to determine if it reliably indicates successful knowledge propagation in language models. furthermore, we analyze three typical counter-intuitive failure cases in detail to understand the role of gradsim in these situations. ## conclusion through extensive experiments and analysis, we propose gradsim, computed as the cosine similarity between gradients of the original fact and its related knowledge, as a crucial indicator for the effectiveness of the ripple effects. the positive correlation observed between gradsim and ripple effect performance across various lms, ke methods, and evaluation metrics underscores its reliability. additionally, our exploration of failure cases further confirms that low gradsim values are indicative of ripple effect failures.