Title: Distributed Knowledge Storage Hypothesis: Evidence from Generalization Failure in Knowledge Editing
Primary area: interpretability and explainable AI
Keywords: Knowledge Editing, Generalization, Large Language Models, Mechanistic Interpretability

Abstract: Large language models (LLMs) store factual knowledge at scale, and recent editing methods allow local updates of target facts. While effective in single prompt styles, prior evaluations have overlooked whether edits generalize across task formats. In this work, we present a systematic study of cross-format generalization in knowledge editing. Across six curated task formats, we show that edits that succeed in the source format often fail to transfer, and that editing also disrupts consistency that models previously maintained across formats. Representation-level analyses reveal that edit directions cluster by task format, supporting a *distributed knowledge storage hypothesis* in which facts are encoded in multiple format-specific subspaces rather than in a single slot. To mitigate these failures, we introduce multi-format supervision with iterative expansion, which improves transferability with minimal overhead. Our study reframes generalization failure not only as a practical limitation of current editing methods but also as evidence about the distributed memory mechanisms underlying factual knowledge in language models.