MemEIC: A Step Toward Continual and Compositional Knowledge Editing
Authors: Jin Seong
Jiyun Park
Wencke Liermann
Hongseok Choi
Yoonji Nam
Hyun Kim
Soojong Lim
Namhoon Lee
Venue: NeurIPS 2025 poster ()
Keywords: vision and language
multimodal understanding
continual learning
Primary area: applications
TL;DR: We propose a benchmark and method for continual multimodal knowledge editing with reliable compositional reasoning.

Abstract: The dynamic nature of information necessitates continuously updating large vision-language models (LVLMs).
While recent knowledge editing techniques hint at promising directions, they often focus on editing a single modality (vision or language) in isolation. This prevalent practice neglects the inherent multimodality of LVLMs and the continuous nature of knowledge updates, potentially leading to suboptimal editing outcomes when considering the interplay between modalities and the need for ongoing knowledge refinement. To address these limitations, we propose MemEIC, a novel method for Continual and Compositional Knowledge Editing (CCKE) in LVLMs. MemEIC enables compositional editing of both visual and textual knowledge sequentially. Our approach employs a hybrid external-internal editor featuring a dual external memory for cross-modal evidence retrieval and dual LoRA adapters that facilitate disentangled parameter updates for each modality. A key component is a brain-inspired knowledge connector, activated selectively for compositional reasoning, that integrates information across different modalities. Experiments demonstrate that MemEIC significantly improves performance on complex multimodal questions and effectively preserves prior edits, setting a new benchmark for CCKE in LVLMs. Our project is available at https://github.com/MemEIC/MemEIC.

MemEIC: A Step Toward Continual and Compositional Knowledge Editing
Jin Seong ' Jiyun P... Weneke Liermann', Hongseok Choi Yoonji Nam 1.3 Hyun Kim Soojong Lim' Namhoon Lee
Electronics and Telecommunications Research Institute, Republic of Korea' POSTECH², Sungkyunkwan University, Korea (real_castle, wliermann, hongking9, h. kim,, isj]Getri re kr {jiyun park, namhoon. wwwe:eestecc......mooooooooo ac. kr seinundzeit skku. edu
Abstract
The dynamic nature of information necessitates continuously updating large vision- language models (LVLMs). While recent knowledge editing techniques hint at promising directions, they often focus editing single modality (vision or lan- guage) in isolation. This prevalent practice neglects the inherent multimodality LVLMs and the continuous nature of knowledge updates, potentially leading suboptimal editing outcomes when considering the interplay between modalities and the need for ongoing knowledge refinement. To address these limitations, propose MemEIC, a novel method for Continual and Compositional Knowledge Editing (CCKB) in LVLMs. MemEIC enables compositional editing of both visual and textual knowledge sequentially. Our approach employs a hybrid external- internal editor featuring dual external memory for cross-modal evidence retrieval and dual LoRA adapters that facilitate disentangled parameter updates for each modality. A key component is brain-inspired knowledge connector, activated selectively for compositional reasoning, that integrates information across differ- ent modalities. Experiments demonstrate that MemEIC significantly improves performance on complex multimodal questions and effectively preserves prior edits, setting a new benchmark for CCKE in LVLMs. Our project is available https:// /ttt::///hhttt/ github..c. www/MemEIC////ccccccmccc.//.ccCe /tt///MMEI//MMMMC.....
Introduction
Large language models (LLMs) [1-3] can knowledge edited-that that is, updated to incorporate new corrected facts factstwithout retraining from scratch [4-6]. This capability is crucial to keep LLMs up-to-date with evolving information or correcting errors on the fly. Large vision-language models (LVLMs) [7-9] combine textual and visual understanding, making knowledge editing even more challenging. These models encode factual knowledge in both their textual and visual pathways, and require mechanisms update both modalities' knowledge consistently. Efficiently editing LVLMs crucial for real-world applications, such as swiftly correcting critical visual recognition errors (e.g., misidentifying Donald Trump as Boris Johnson as shown in Fig. (a)).. However, research in multimodal knowledge editing remains scarce compared to the extensive studies in textual LLMs.
Equal Contribution.
Corresponding Author.
Work partially done during an internship at ETRI.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Visual Edit
Single Eddting
Who ki
Bora Jannaon
Donald Trump
Imspet
Textual Edit
(Worgg) sonumedd the position The United Stitess
(Corret)) Ihe assamed the pos Ststes
Test
Test
What position ald Danai Trums recerily assume?
Sequential Editiing inouddnng Cor postio
30
Compositional Edit (Voux Tacrol
(Wrongg) Baris Jotnson recentty y Péie Njsoew Liiitd Ningsoos
Bonard Trump movelyyy uuaar poolioo the President Stace.
What couiken del to peroon sooute??
Visual
Textual Exit
Gap
Test
Test2
Edit
Componitional
Belore ten
After CoR
(a) Compositiona Edit Task
(b) Single Edtting vs Sequential Editing
Figure 1: Description of Compositional Edit Task and Sequential Editing
Existing benchmarks for LVLM knowledge editing are limited in scope [10-13]. They typically focus only on visual knowledge updates [10-13], overlooking scenarios where textual facts (e.g., attributes) must also be updated. Furthermore, current benchmarks do not evaluate continual editing [10, 12, 13]- performing multiple sequential knowledge updates nor do they test a model's 's ability to handle compositional reasoning [14-16] over edited facts. This is a critical gap because real-world knowledge updates often arrive incrementally and involve intertwined visual and textual information.
To address these shortcomings, we introduce Continual and Compositional Knowledge Editing (CCKE), new benchmark for multimodal knowledge editing. CCKE is the first to assess models under continual knowledge editing [17] and compositional queries that require combining information from both visual and textual edits. We also propose metric called Compositional Reliability (CompRel) to quantify how reliably a model can integrate multiple updated knowledge pieces when answering such complex queries. Evaluating the leading LVLM editors from VLKEB using our new CCKE benchmark (CCKEB) reveals that both external- and internal internal-emmory methods face fundamental challenges in multimodal knowledge editing.
External memory-based editor (c.g., SERAC [18], IKE [19]) store edited knowledge externally and retrieve relevant information during inference without modifying model parameters, making them suitable for long-term retention. However, current implementations transplant textual- based retrieval strategies from LLMs directly, ignoring crucial visual cues necessary for accurate multimodal query matching, significantly diminishing retrieval accuracy. Additionally, external memory methods encounter an internalization issue, where models overly rely on its stale internal knowledge, especially if the retrieved external information conflicts with what the model has learned, leading to incorrect answers [20-23]. Conversely, internal memory- based methods [24-26] embed edits directly into model parameters via fine-tuning or low-rank adaptation (LoRA) [25], viewing the
network (FFN) itself key-value memory structure that inherently stores in-model knowledge [27]. Although beneficial for internalizing edits, these methods typically store visual and textual edits within a unified representation space, resulting in cross-modal interference and representation collapse [28]. Moreover, sequential internal edits suffer from catastrophic forgetting [29-32], degrading previously stored knowledge with successive updates [11].
In this work, we propose Mem a novel multimodal knowledge editing framework integrating external retrieval memory with internal model editing. MemEIC is designed specifically to handle continual, compositional edits. It operates in several coordinated stages. First, MemEIC uses query decomposition to automatically split each user query into its visual part and textual part. Next, for each part, an external memory retrieval module fetches relevant information using both image and text cues so that no crucial visual detail or textual context is missed. In parallel, we maintain separate lightweight LoRA adapter modules as internal memory for visual and textual knowledge within the model, inspired by brain lateralization [33-38]. This knowledge separation ensures that editing a visual fact does not distort textual representations, and vice versa, preventing representation collapse. Crucially, MemEIC introduces brain-inspired knowledge connector -akin to a corpus callosum [39, 40n-lgn the visual and textual pathways. This connector (using LoRA A-RAmmeeeeteed self-attention) fuses the edited knowledge from both modalities only when a query requires both to answer. If the query is unimodal, the two streams remain separate. By selectively connecting edited

---

visual and textual representations, MemEIC enables effective compositional reasoning and yields robust answers even when external retrieval and the model's internal knowledge conflict.
Our contributions are summarized as follows:
We introduce CCKEB, the first benchmark for continual and composi- tional multimodal knowledge editing (combining sequential visual and textual edits, with a new reliability metric for evaluation).
CCKE Benchmark:
MemEIC Framework: We present MemEIC, a multimodal knowledge editing framework that integrates external memory retrieval with internal model editing via modality-speciff adapters inspired by brain lateralization, coupled with brain-inspired knowledge connector enable robust continual and compositional knowledge reasoning.
Empirical Results: Extensive experiments show that MemEIC achieves interference- knowledge updates and outperforms prior methods on both edit success and compositional reasoning tasks.
2 Problem Formulation and Benchmark Design
2.1 Preliminaries: Knowledge Editing for LVLM
Visual Knowledge Edit Visual edits [10, 11] involve updates to the recognized identity label of an image. For an image: once labeled with entity e, the task is change the label new entity e':
(i,
(i,
For example, changing person's 's visual identification from Boris Johnson to Donald Trump. Such edits reflect common real-world updates, such as correction of misidentified individuals or subject identity changes over time.
Textual Knowledge Edit Textual edits focus on changes in factual knowledge associated with specific entities. We modify 1-hop factual relation (e, r, o) by replacing the original factual object with new object o', simulating how factual statements can become outdated:
(e,r,o)
(e,r,o').
For instance, updating facts like "Donald Trump is the 45th president" to "Donald Trump is the 47th president". Such edits simulate real-world factual updates (e.g.. job titles, affiliations, roles).
Single Editing vs Sequential Editing Most knowledge editing benchmarks assess only single editing, overlooking real-world continual updates and potential forgetting. Sequential editing [1]] executes multiple updates sequentially and runs test queries in varying gaps, thereby probing a model's 's memory retention and robustmess over time (see Fig. 1(b)).
2.2 Problem Formulation: Continual, Compositional Knowledge Editing (CCKE)
In real-world multimodal applications, entities often undergo interleaved visual and factual updates. We define Continual, Compositional Knowledge Editing (CCKE) as the task of having an LVL.M accommodate sequence of alternating visual and textual edits. For simplicity, we limit our ex- periments to paired setting where visual edit is followed directly by its associated textual edit, followed by pairs of unrelated edits:
visual edit
textual edit
(i.e)=((,e))
(e' ((e's.))
During the continual editing process, retention of edited knowledge is evaluated at predefined intervals (gaps). To measure model's ability integrate multiple edits to answer single compositional query, we introduce the Compositional Reliability (CompRel) metric:
CompRel
E (e. U.e) )~ D (tare Drsnst (sto (
[If(ie, tyee
(I)
Co (VSte, tita)

---

Output
1n Donald Trump 1e reantyy accumedine poation 4Ph United Stive.
Inpur
LLM
Adsptors Connector
(c]
Karniedg
Rg
Ry
Conreter
woigttt
My
Manot
Fur is pitum?
Donald Thomp riciiily
Dasard
suu uill
Adr & Nomokzztoon
the
Pelinnce
Qy
QT
mesteal Qunn) pernce in the photo
the pholi?
FRN Presmiiedd
(Qunetion) poition
Input
Adapter
Adapter
Figure Overall Structure of MemEIC for Continual and Compositional Edit
where viisual edit denotes the set of visual-edit examples (fe, Vx., Vy..) with i, the edited image, Uz the visual-only query and fy. its ground-truth answer; Drextual odit denotes the set of textual-edit examples (fz,, ( with tz2 the textual query and fyg its ground-truth answer; O' represent the model parameters after editing and C' denotes the operation that composes the visual-edit query Us., "Who is in the photo?", and the textual-edit query fx., "What position did Donald Trump recently assume?". into a single compositional query Te. "What position did the person in the photo recently assume?".
2.3 Extended Benchmark for Compositional Edit
Our new dataset CCKEB, Continual and Compositional Knowledge Editing Benchmark, extends VLKEB by adding textual modifications to the existing visual edits, thereby forming a fully composi- tional benchmark. Concretely, each image instance in CCKEB is paired with two coordinated edits: (1) visual edit that updates the image's identity label, and (2) a textual edit that revises a factual statement about the (new) entity depicted in the image. This design mirrors realistic applications such as correcting outdated factual statements (e.g.. becoming new president) while simultaneously fixing mislabeled or changed identities in an image.
Construction Process Following and extending VLKEB [11] construction protocol, we extract knowledge triples and generate textual edits and QA pairs using the multimodal knowledge graph MMKG [41] and GPT-4o [7]. See Appendix B for full details.
3 Methodology
In this section, we describe our proposed framework for CCKE. Our approach is designed to address the limitations of existing methods by combining external and internal memory mechanisms while leveraging query decomposition and controlled knowledge integration for improved compositionality. Figure provides an overview of the complete pipeline.
3.1 Knowledge Separation
Query Decomposition.
To effectively handle compositional queries, we introduce a dedicated module that automatically classifies and decomposes the input textual query into its constituent visual and textual knowledge parts. Formally, given an input query Q, the module computes:
(Q Q+) = f(Q)
where Q. and Q: denote the visual and textual components, respectively, and ʃ(-) represents a query decomposer. This decomposition not only reduces the complexity of the original query but also enables specialized processing pipelines for each modality (Fig. 2(a)). For query decomposition, we used GPT-4o [7]. See Appendix D and E for prompt and decomposition details.

---

Modality-Aware External Memory (MEM-E). In recent LVLM knowledge editing bench- marks [10, 11], extemal- memory editors directly adopt textual retrieval schemes originally developed for language models, relying solely on textual cues for retrieval and yielding suboptimal visual editing performance. To address this limitation, we propose Mem-E, simple yet effective multimodal external memory storing modality-speciff edits in two separate memory storage units as illustrated in Fig. 2(a):
M₁ = {(a a));: M. M,=((;99 By و
where M, holds textual QA edits and M, holds visual edits as (image question y, answer a) triples. At inference, given decomposed input textual query Q Oi and visual query Qu (Q Qt), Mem-E E retrieves:
1" = arg max cos():(Q)), (e():))
R₂ = (9: a..
ISIS..
a (o)())( (1,))) (1- a) (O)( )e(()))
k- arg max
R. (gx,, ag.
ISSSN
where d-(-) denotes the [CLS] token representation obtained from DistilBERT [42], (-) is the CLIP [43] image encoder's [CLS] token representation, and we set a = 0.5. For compositional queries composed of textual and visual sub-queries, we first retrieve the target entity aj using Qu. This entity replaces the tagged placeholder in Qi enabling more accurate retrieval from M,.
Finally, for any query Q-textual, visual, or compositional, we define the retrieved context
Ra,
Q Qui
R
R
[Ry; Re],
Q = (Qus Qc),
where [:] denotes simple text-level concatenation. This R is then prepended to the original query when prompting the base LVLM, e.g.. "Q: [Image] Who is the man in the picture? A: Donald Trump. Q: What position did Donald Trump recently take on? A: He assumed the position of the 47th president of the US. Q: What position did the person in the photo recently assume? A:"
Internal Separated Knowledge Integration (MEM-I). Existing internal memory- based editors view the transformer's feed-forward network (FFN) layers implicit key-value memories, that can be updated to store new knowledge. Unlike external memory methods, which attach edits externally, these approaches embed all updates in single shared embedding space. Under continual, compositional knowledge editing, this unified space leads to severe cross-modal interference and catastrophic forgetting heterogencous visual and textual updates accumulate.
To overcome these limitations, we propose Mem-l, a dual-stream internal memory inspired by human brain lateralization illustrated in Fig. 2(b). Neuroscience shows that the left hemisphere specializes in language, while the right hemisphere processes visual and spatial information. Analogously, Mem-I extends the FFN memory by two parallel LoRA adapters, connected to the frozen pre-trained FFN weights 0 that retain pre-edit knowledge:
Visual adapter (0,) for visual knowledge updates- right hemisphere.
Textual adapter (0.) for textual knowledge updates-lefs hemisphere.
During the forward pass, we activate modality-speeifie adapters based on the query type:
Visual query (Q.): visual adapter 0, and frozen FFN 0 are activated.
Textual query (Q:): textual adapter O, and frozen FFN 0 are activated.
. Compositional query ((Q.,, Q:)): both adapters and frozen FFN 0 are activated.
Analogously, when adding new edits, only the adapter corresponding to the edit modality is updated. As edits are assumed to be atomic, each edit alters either the visual or the textual adapter, never both at once. Concretely, let
5

---

visual edit,
0' 0
10,,
textual edit.
3.2 Knowledge Connector
The modelity-speeific adapters can selectively activate in response to relevant tokens compositional queries (Appendix E.2). However, when the same sequence contains modality modality-biased tokens (e.g.. one token mainly shaped by the visual adapter, another by the textual), they interact poorly, limiting the model's ability to combine complementary facts during compositional reasoning (Sec. 4.5, Dual- LoRA RAG). To address this, we introduce simple and effective Knowledge Connector, inspired by the corpus callosum in the human brain, which enables information exchange between the two hemispheres. The connector augments the self oeff-attention module with LoRA adapters to facilitate interaction between modality-basee token representations (Fig, 2(c)).
Formally, sequence's 's hidden state at layer l given as:
he h h Iohy + Inh
where h--1 is the frozen FFN hidden state, h and h are the visual and textual adapter states, and I, It E (0. indicate modality-specific activations. To explicitly model when both adapters are active, we define an indicator function:
11, if = and 7, = 1,
In,
0, otherwise.
Based the indicator definition above, the Knowledge Connector extends the self-attention module as follows:
= -Woo - To,e AWb). Kt = =²=h((W< + I,,e AWk), v²= = nWWv,
(6)
where WQ. WK, Wy are pre-trained projection weights, and AWL, AWA LoRA adjustments. In this formulation, the fused hidden state d provides semantically related token representations retrieved from the frozen FFN, allowing the Knowledge Connector to attend across modality-specini specific information when both adapters are active. Finally, attention is computed as:
Attention( ( V*).
This mechanism adaptively integrates information only when both modality-speciffc adapters activated, enabling effective interaction between separately maintained visual and textual knowledge (Fig. 4). For unimodal queries, the connector defaults an identity operation, preserving independent modality representations.
4 Experiments
4.1 Experimental Setup
Fig. 3 illustrates the overall training and testing setup. Detailed configurations for the experimental setup are provided in Appendix C, and model-specific details in Appendix D.
External Memory In the first stage, we freeze the parameters of the base LVLM and train only the external memory module using the CCKEB training set. Each query is decomposed into its visual and/or textual subcomponents through the query decomposition module, depending on the modality required to answer it. The model then learns to retrieve and align each subquery with its corresponding entry in the modalitty-peecinn external memory storage, thereby establishing accurate associations between visual and textual evidence.
Training Stage 1:

---

Table I: Visual, textual, and compositional reliability evaluated across two backbone models, LLaVA- 1.5 and MiniGPT4, on CCKEB testset. See Appendix G for detailed results.
Vis Rel (%)
Text Rel ( (%)
Comp Rel (%)
180
80
Thak
Comp
Sequrdiil Dap
Seqpaneist Oop
Rugarniia Day
(a) LLaVA-15
Text ReE
Cempy
Sequenaal Gap
Sequercal
Sequerial Cap
LaKA
VEND
S2MC
VIDE
NentiC
(b) MiniGPT-4
Training Stage 2: Knowledge Connector In the second stage, we activate both the visual and textual adapters and train the Knowledge Connector on the CCKEB training examples using an adversarial retriever. Instead of directly relying on external memory, the retriever provides a mixture of factual and counterfactual evidence to simulate realistic and noisy retrieval conditions. The connector is optimized to fuse the outputs of the two adapters only when a compositional query requires the integration of both visual and textual knowledge, thereby discouraging over-reliance on external memory and encouraging selective integration of internal and external evidence.
Testing: For Realistic Editing In deployment, users rarely modify both visual and textual facts simultancously; rather, edits on different modalities often occur for ample, a textual edit may later be followed by visual edit new information becomes available. Hence, freeze the Knowledge Connector; when an edit targets internal knowledge, we update the corresponding visual or textual adapter, while new evidence is simply appended (stacked) in the external memory store, leaving prior entries and the connector frozen. At test time, we perform 500 editing steps sequentially, evaluating edit retention at gaps of (0, 10, 20, 50, 100).
Traln Dllige
af way!
LLM
Tolln meski)
reput
Busa
(Wiirg)
Factual Fastival
Teat
Deappsson
Figure 3: Training Stage and Testing
4.2 Main Results
Table 1 demonstrates the competitive performance of the proposed editor, MemEIC, in comparison to four baselines on the CCKEB test set. Baselines include one external memory method, i.e., SERAC [18], and four internal memory editing methods, i.c., FT (LLM), LoRA [25]. MEND [24], WISE [44]. Prior work [11] reported that MEND, when applied to sequential editing. caused model collapse and NaN losses. To address this issue, we adopt greedy interpolation strategy that blends previous weights with newly edited ones. Two baseline candidates, IKE [19] and KE [45], are excluded as they require edit data at test time, which is impractical given evolving knowledge.
External Memory Methods Retrieval-based approaches, such as SERAC, store new information externally without modifying any model during edit step, effectively preserving previously learned knowledge. As such, no major performance difference can be observed between the 0 gap and 100
7

---

gap setting. However, good reliability depends on correct retrieval. SERAC's text-only retrieval strategy fails to reliably locate visual edits, resulting in lower visual reliability across both backbone models. Additionally, because external methods do not internalize new facts into the model, retrieved facts remain isolated, unable to effectively interact with one another or with the model's internal knowledge, thereby hindering coherent reasoning and leading to diminished compositional reliability.
Internal Memory Methods Fine-tuning or LoRA-based approaches rapidly internalize new knowl- edge by embedding edited facts into a single shared parameter space, leading to perfect visual and textual reliability in the O-gap setting. However, when exposed to long sequences of edits, they over- write previously learned knowledge, causing a drop of nearly 30 points in visual and textual reliability, This is caused by storing heterogeneous visual and textual edits within the same parameters, which leads to representation collapse and severe cross-modaa interference. As result, their compositional reasoning performance degrades substantially as edits accumulate.
To alleviate such forgetting, WISE introduces side-FFN memory slots and a router that interpolates between the original and edited parameters. This design mitigates forgetting and maintains relatively stable performance across sequential gaps. Nevertheless, because edited knowledge is stored as isolated parameter fragments and the router operates primarily on textual activations. WISE remains limited in handling visual edits and fails to fuse visual and textual edits for compositional queries.
MemEIC- MemEIC integrates external and internal memory mechanisms, addressing the limitations of both external-only and internal-only methods. Externally, an image-text retrieval memory reliably stores factual evidence. Internally, lightweight, modality-specific adapters internalize visual and textual edits independently, preventing conflicts with existing knowledge and cross-modal interference. As result, MemEIC achieves significantly higher performance compared to all baselines and maintains stability across varying edit-test gaps, exhibiting strong robustness against catastrophic forgetting and representation collapse.
Quantitatively, averaged across all edit-test gaps, MemEIC outperforms WISE by +16.94 visual reliability and by +32.35 in compositional reliability on LLa demonstrating that temporal robustness alone is insufficient without explicit multimodal fusion. The gap-averaged visual and textual reliability for LLaVA-LL reaches 98.93 and 92.48, respectively. This substantial improvement is largely attributed to the brain-inspired Knowledge Connector, which dynamically fuses visual and textual representations only when multimodal reasoning is required. With this mechanism, MemEIC achieves an average of 80.56 in compositional reliability, an +18.51 points improvement over the best baseline, LoRA (62.05).
4.3
Ablation on External Memory: Visual & Textual Cues in Retrieval
Table 2: Performance Average on VLKEB under Sequential Editing for LLaVA-1.5.
Textual
Visual Add
Rel
T-Gen
I-Gen
T-Loc
I-Loc
Port
SERAC
v
75.00
45.19
75.29
99.99
1.91
38.50
Mem-E (tex)
48.02
50.74
48.22
99.99
4.02
45.08
Mem-E (tex+vis)
-
96.51
93.42
79.90
99.99
57.10
62.09
Table 2 compares our external memory-based editing approach (Mem-E) with the SERAC baseline. SERAC [18] relies solely on textual information, i.e., a [CLS] token representation obtained from BERT [46], for retrieving edited knowledge. If query falls within the scope of an edit, the retrieved information is passed to a separate counterfactual model that "adds" the retrieved information on top of the base model. In contrast, our method uses information retrieved from external memory directly without employing any extra model. We examine two retrieval variants: Mem-E (tex), using textual cues only, and Mem-E (tex+vis), combining textual and visual cues.
As expected, Mem-E (tex) performs worse than SERAC due to the absence of the additional model. However, incorporating visual cues into retrieval (Mem-E (tex+vis)) significantly enhances perfor- mance across key metrics. Reliability (Rel) improves markedly from 48.02 to 96.51, indicating that visual features help retrieve edited knowledge more accurately. Text generality (T-Gen) also shows a major gain, increasing from 50.74 to 93.42, demonstrating how visual context can effectively disam- biguate queries that are semantically similar. Most notably, image locality (I-Loc) jumps dramatically
8

---

from 4.02 to 57.10, underscoring the crucial role of visual retrieval in correctly distinguishing edited from unedited visual information. Detailed results are in Appendix G.2.
4.4 Ablation on Internal Memory: Separated Knowledge Integration
Table 3: Knowledge separation (Mem-I) results on CCKEB validation dataset
Visual Edit
Textual Edit
Rel
T-Gen
I-Gen
T-Loc
I-Loc
Rel
Gen
Loc
Mem-I (Dual-LoRA, r:8
74.73
71.57
74.61
80.93
12.45
77.24
73.73
68.22
Mem-I (Single-ToRA, 1:
74.60
71.50
74.49
63.16
9.59
74.54
70.49
63. 16
Difference
-0.13%
-0.07%
-0.12%
-17.77%
-2.86%
-2.70%
-3.24%
-5,06 %
Mem-I (Single-LoRA, r:8)
74.06
71.02
74.08
60.42
7.90
72.12
68.29
60.42
Difference
-0.67 %
-0.55%
-0.53%
-20.51% %
-4.5%
-5.12%
-5.44%
-7.80 %
p (paired I-tst)
Table the hypothesis that mimicking the brain's hemispheric specialization [33-38]- by maintaining separate internal memory spaces for visual and textual information substantially improve the precision and overall effectiveness of knowledge editing in LVLMs. To ensure robustness, each experiment is repeated five times and results are averaged. We hold the total parameter budget constant across conditions: Mem-I (Dual-LoRA, for each modality, whereas Mem-I (Single-LoRA,
+ 8) maintains separate low-rank memories 16) uses shared memory.
In detailed performance analysis, Mem-I (Dual-LoRA) shows modality-specifee advantages. For visual edits, while the mean differences in some metrics (Rel, T-Gen, I-Gen) appear small, a paired I-test confirms these improvements are statistically significant (p 0.05). However, the clearest evidence lies in the much larger gains in localized editing metrics such as T-Loc (+17.77%) and I-Loc (+2.86%), confirming that modality-specifie memory separation precisely confines edits their targets. For textual edits, Dual-LoRA consistently outperforms Single-LoRA across all metrics Rel (+2.70%), Gen (+3.24 +,3244%) and Loc (+5.06%). Notably, reducing the rank in Single-LoRA (r 8) markedly degrades performance relative to = 16, showing that our observed improvements are not artifacts of rank selection. Overall, our results suggest that distinct separation of visual and textual memories during CCKE effectively mitigates representation collapse, enabling precise, localized editing of multimodal knowledge. For a detailed breakdown of performance at each individual sequential gap. refer to Appendix E.2
4.5 Ablation on Internal Memory: Knowledge Connector
Figure 4 reports that separating knowledge into internal and external memories and then re-combining them through a dedicated Knowledge Connector is the key to stable compositional reasoning.
Parformance Comparison on LLaVA1.5 (7B)
External retrieval enhances stability, but can- not integrate facts by itself. Even under the
100
Composttiona Reliabiiity (56
highly favorable assumption of perfect oracle
88
retrieval (Base+RAG), where all relevant edits correctly fetched, the model's compositional reliability still levels off at just 64.93%. The reason is simple: the freshly retrieved facts sit side-by-side with one another and the model's old, still-stored facts, and the two sets often dis- agree. With no mechanism to integrate them,
60
40
Singlc LoRA (Mem.)
Dasi-LolA (Memel
20
Base a RAG (tem-El
Danit cRA + RAG (Mem--l))
Dasl-toRA RAG + Coen [Mem-EIC
50
100
this caps performance, showing that perfect re- trieval alone is not enough.
Sequential Gap
Compositional reliability of five editing variants on LLaVA-1.5 (7B) on CCKEB testset.
Figure 4:
-pdelity-speeffic adapters integrate facts, but limit interaction between modalities. When retrieval is combined with modality-separateer adapters (Dual-LoRA+RACE reliability increases to 78. 16 at gap=0. Nevertheless, performance declines to 63.39 % at gap=100. This decline arises from the absence of alignment between the heterogeneous representations produced by the two

---

adapters. In comparison to Single-LoRA, Dual-LoRA does not offer a consolidated latent space, that would allow for easy interaction between facts across modalities. This is supported by the observation that Dual-LoRA A achieves a compositional reliability of 70.05% at gap=0, falling short of the 77.47% attained by Single-LoRA.
Knowledge Connector unifies modalities for near-oracle reliability. Augmenting Dual- LoRA+RAG with our attention-based Knowledge Connector raises compositional reliability to 99.21 % gap-0 and maintains it 97.01 % at gap-100. These results demonstrate that our Connector effectively combines modality-specifti adapters, sustaining near-oracle long-term compo- sitional reasoning.
To evaluate CompRel in a controlled setting, we assumed an oracle external memory that always retrieves the correct knowledge. However, in more realistic scenarios -uchh as when Mem-E retrieves counterfactual knowledge-conffe
can arise between externalyy retrieved information and internally edited facts. Even in such cases, our model avoids over-relying incorrect external knowledge and maintains robust performance (Appendix E.3.2). Also, we explored various connector designs and found that inserting LoRA into the attention projection consistently led to the best performance on the CCKEB (Appendix E.3.4). This finding suggests that LoRA within the attention pathways allows the model to compositionally integrate internally edited knowledge from lower FFN layers, thereby strengthening cross memory reasoning. The results for MiniGPT-4 are provided in E.3.
5 Conclusion
In this paper, we propose CCKEB, novel continual and compositional knowledge editing benchmark designed to evaluate multimodal consistency over extended edits. We also introduce MemEIC, an integrated memory system that leverages external retrieval, modality-speciff dual-LoRA adapters, and a brain- -raii-esspired Knowledge Connector for robust multimodal editing. Extensive evaluations demonstrate that MemEIC outperforms existing methods by effectively handling continual updates, preserving previous edits, and achieving significant compositional reliability.
Acknowledgements
This work was supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea Government (MSIT) (No. RS-2023-0021601 Development of Artificial Complex Intelligence for Conceptually Understanding and Inferring like Human). was also supported by the IITP grant funded by the Korea Government (MSIT) (No. RS-2024-00338 140, Development of learning and utilization technology to reflect sustainability of generative language models and up-to-dateness over time).
References
[1] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, and al. Jared Kaplan. Language models are few-shot learners, 2020.
[2] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, and et al. Nikolay Bashlykov. Llama 2: Open foundation and fine-tuned chat models, 2023.
[3] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, and et al. Xiaodong Deng. Qwen technical report, 2023.
[4] Song Wang. Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. Knowledge editing for large language models: A survey, ACM Comput. Surv., 57(3), November 2024.
[5] Peng Wang. Ningyu Zhang, Bozhong Tian, Zekun Xi, Yunzhi Yao, Ziwen Xu, Mengru Wang. Shengyu Mao, Xiaohan Wang, Siyuan Cheng, Kangwei Liu, Yuansheng Ni, Guozhou Zheng, and Huajun Chen. EasyEdit: An easy-to-use knowledge editing framework for large language models. In Yixin Cao, Yang Feng, and Deyi Xiong, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume System Demonstrations), pages 93, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
10

---

[6] Yunzhi Yao. Peng Wang. Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems, methods, and opportunities. arXiv preprini arXiv: /3/72, 2023.
[7] OpenAL. Gpt-4 technical report, 2024.
[8] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 19730-19742. PMLR, 23-29 Jul 2023.
[9] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhanc- ing vision-language understanding with advanced large language models. In ICLR, 2024.
[10] Siyuan Cheng, Bozhong Tian, Qingbin Liu, Xi Chen, Yongheng Wang, Huajun Chen, and Ningyu Zhang. Can we edit multimodal large language models? In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pages 13877-13888, Singapore, December 2023. Association for Computational Linguistics.
[11] Han Huang, Haitian Zhong, Tao Yu, Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. Vlkeb: large rision-language model knowledge editing benchmark, In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paguet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 9257-9280. Curran Associates, Inc., 2024.
[12] Jiaqi Li, Miaozeng Du, Chuanyi Zhang, Yongrui Chen, Nan Hu, Guilin Qi, Haiyun Jiang, Siyuan Cheng, and Bozhong Tian. Mike: A new benchmark for fine-grained multimodal entity knowledge editing. In ACL (Findings), 2024.
[13] Zhen Zeng, Leijiang Gu, Xun Yang, Zhangling Duan, Zenglin Shi, and Meng Wang. Visual- oriented fine-grained knowledge editing for multimodal large language models. arXiv preprint 2xXi:24111112900 2024.
[14] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate, 2016.
[15] Daniel Keysers, Nathanael Schärli, Nathan Scales, Hylke Buisman, Daniel Furrer, and et al. Sergii Kashubin. Measuring compositional generalization: A comprehensive method on realistic data, 2020.
[16] Xindi Wu, Hee Seung Hwang. Polina Kirichenko, and Olga Russakovsky. Compact: Composi- tional atomic-to-comple visual capability tuning, 2025.
[17] Liyun Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[18] Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher Manning. Memory-based model editing at scale. In International Conference Machine Learning, 2022.
[19] Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. Can edit factual knowledge by in-context learning? In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4862-4876, Singapore, December 2023. Association for Computa- tional Linguistics.
[20] Yufang Hou, Alessandra Pascale, Javier Carnerero-Cano, Tigran Tehrakian, Radu Marinescu, Elizabeth Daly, Inkit Padhi, and Prasanna Sattigeri. Wikicontradict: A benchmark for evaluating Ilms on real-world knowledge conflicts from wikipedia. Advances in Neural Information Processing Systems,
2024.

---

[21] Haoang Chi, He Li. Wenjing Yang, Feng Liu, Long Lan, Xiaoguang Ren, Tongliang Liu, and Bo Han. Unveiling causal reasoning in large language models: Reality or mirage? In A. Globerson, I.. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Advances Neural Information Processing Systems, volume 37, pages 96640 06670. Curran Associates, Inc., 2024.
[22] Cheng-Han Chiang and Hung-yi Lee. Do metadata and appearance of the retrieved webpages affect LLM's reasoning in vetrieval-augmentee generation? In Proceedings of the 7th Black- boxNLP Workshop:
Analyzing and Interpreting Neural Networks for NLP, pages 389-406, Miami, Florida, US, November 2024. Association for Computational Linguistics.
[23] Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. Hipporag: Neurobiologically inspired long-term memory for large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[24] Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast model editing at scale. CoRR, 2021.
[25] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference Learning Representations, 2022.
[26] Kevin Meng, David Bau, Aitor Lewkowyez, and Yonatan Belinkov. Locating and editing factual associations in gpt. In International Conference Learning Representations (ICLR), 2022.
[27] Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forwart layers are key-value memories. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing, pages 5484-5495, 2021.
[28] Wanli Yang, Fei Sun, Xinyu Ma, Xun Liu, Dawei Yin, and Xueqi Cheng. The butterfly effect of model editing: Few edits can trigger large language models collapse. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 5419-5437, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
[29] Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. Pmet: Precise model editing in a transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18564-18572, 2024.
[30] Qizhou Chen, Taolin Zhang, Xiaofeng He, Dongyang Li, Chengyu Wang, Longtao Huang, et al. Lifelong knowledge editing for Ilms with retrieval auggeentd continuous prompt learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13565-13588 13580, 2024.
[31] Dongfang Li, Zetian Sun, Xinshuo Hu, Baotian Hu, and Min Zhang. Cmt: A memory compres- sion method for continual knowledge learning of large language models. Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pages 24413-24421, 2025.
[32] Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning, 2025.
[33] Goulven Josse and Nathalic Tzourio-Mazoyer. Hemispheric specialization for language. Brain Research Reviews, 44(1):1-12, 2004.
[34] Marlene Behrmann and David C. Plaut. A vision of graded hemispheric specialization. Annals of the New York Academy of Sciences, 1359(1):30-46, 2015.
[35] Laurent Cohen and Stanislas Dehaene. Specialization within the ventral stream: the case for the visual word form area. Neurolmage, 22(1):466-476 2004.
[36] Michael C. Corballis. Left brain, right brain: Facts and fantasies. PLOS Biology, 12(1):1-6, 01 2014.
12

---

[37] Norman Geschwind. Specializations of the human brain. Scientific American, 241(3):180-200) 1979.
[38] R. W. Sperry, Hemisphere deconnection and unity in conscious awareness. the american psychologist. Brain, 23(10))723-733, 1968.
[39] Michael S. Gazzaniga. Cerebral specialization and interhemispheric communication: Does the corpus callosum enable the human condition? Brain, 123(7):1233-3326 07 2000.
[40] James I. McClelland, Bruce I. McNaughton, and Randall C. O'RReilyy. Why there are comple- mentary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 102(3):419, 1995.
[41] Ye Liu, Hui Li, Alberto Garcia-Duran, Mathias Niepert, Daniel Onoro-Rubio, and David Rosenblum. Mmkg: Multi-modal knowledge graphs, 2019.
[42] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, distilled version of bert: smaller, faster, cheaper and lighter, 2020.
[43] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceed- ings of the 38th International Conference Machine Learning (ICMI. 202/), volume 139 of Proceedings of Machine Learning Research, pages 8748-8763 2021.
[44]
Peng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Wise: Rethinking the knowledge memory for lifelong model editing of large language models. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, Tomezak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 53764-53797. Curran Associates, Inc., 2024.
[45] Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. pages 6491-6500 Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
[46] Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pages 4171-4186, 2019.
[47] Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang. Shumin Deng, Mengru Wang. Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, et al. A comprehensive study of knowledge editing for large language models. arXiv preprint arXiv:2401.0f 200.22 2024.
[48] Muhao Chen, Weijia Shi, Ben Zhou, and Dan Roth. Cross-lingual entity alignment with incidental supervision, 2021.
[49] Ran Duan and Seth Pettie. Linear-time approximation for maximum weight matching. J. ACM,
(1), January 2014.
[50] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013.
13

---

Appendix
In the Appendix, we introduce more details along with additional experimental results, discussions, and related works:
Appendix A: Related Works
Appendix B: Dataset Details
Appendix C: Experimental Setup
Appendix E: Additional Experimental Results
Appendix F: Limitations and Broader Impacts
Appendix G: Original Results
A Related Works
Benchmarks for LVLM Knowledge Editing Early research on knowledge editing focused on text-based LLMs, aiming correct factual errors or update information without full retraining [4- 6, 47]. To extend knowledge editing into large vision language models (LVLMs), benchmarks such MMEdit and VIKKBB [10, 11] have been introduced enable multimodal information updates. However, these benchmarks largely target visual-only edits: MMEdit relies on artificial tasks (e.g.. editing objects not present in the image) and focuses solely single-edit scenarios, where the model is reinitialized after each update-an impractical approach for continuous real-world settings. Moreover, VLKEB does not fully support compositional edits that require joint updates of both visual and textual knowledge. This gap highlights the need for a comprehensive framework capable of handling continuous and compositional knowledge updates.
Knowledge Editing Methods A key distinction in knowledge editing is whether updates are stored externaly integrated within model'ss parameters. Retrieval-basee methods external memory to preserve long-term edits without altering model parameters but tend to inherit a text-centric bias, limiting the effective use visual cues [18]. In contrast, internal memory-base methods embed updates directly into the model, enabling flexible inference [24]; however, they suffer catastrophic forgetting and risk representation collapse when visual and textual characteristics are stored together in single layer [40]. These limitations underscore the need for approaches that support compositional, continuous knowledge editing in realistic settings.
B Dataset Details
Our new benchmark CCKEB, an extended version of VLKEB [11] [1i]-orgginaly focused on visual editing-by by enabling compositional edits that involve both visual and textual modalities. CCKEB augments 5,000 visual editing instances from the VLKEB training split with their corresponding textual edits, resulting 5,000 visual-textual training pairs. The same procedure applied the evaluation split, resulting in 1,278 compositional evaluation pairs
B.1 Construction Process
To construct the textual edits, we follow the data construction process of VLKEB by leveraging the multi-modal knowledge graph MMKG [41]. Where necessary, we retrieve new knowledge triples
(e', r, r),,, and then generate textual edits by modifying the object entity o o' within existing and new triples.
Knowledge Triples Extraction Unlike the VLKEB evaluation set, which includes explicit knowl- edge triples and associated QA pairs, included to evaluate portability, the 5,000 training instances in VLKEB do not provide annotated knowledge triples (c', o). To address this gap, extract candidate triples for the training set using the DB15K [48] knowledge base, which was also used in constructing the VLKEB evaluation set.
Among the 5,000 training samples, we successfully extract knowledge triples for 4,428 instances, using total of 187 relation types in DB15K. In cases where multiple candidate triples were identified,
14

---

we prioritize more informative and sparse relations (e.g., location, birthPlace, creator) to select the most salient triple. For the remaining 572 instances where relation triples could not be extracted automatically, we manually align entity mentions with DB15K entries when partially matched. If the entity was entirely missing from DB15K, we manually supplement by referencing Wikipedia.
Object Pairs Selection Given the relational triples (c', ,, o), we perform object replacement by substituting the original object o with a new object o'.
Following the data construction process of VLKEB, we apply maximum weight bipartite match- ing [49] identify object pairs (o, o') with the highest similarity, based on shared relations. The goal of this process is to ensure that and o' belong to the same or similar semantic categories, thereby making the substitution o - o' contextually valid. To facilitate this, we extract the set of relations associated with each object using the FB 15K [50] and DBISK [48] knowledge graphs, in that order. We then compute the number of shared relations between every possible object pair and use this count as edge weights in bipartite graph. Maximum weight bipartite matching is subsequently applied to select the most semantically similar object pairs. cases where suitable pairs could not be retrieved from the dataset, we generated candidate object replacements using GPT-4o.
For the evaluation set, all object pairs (o, o') are manually reviewed to ensure data quality. This includes identifying and correcting cases where o = o', where duplicate answers are generated, or where a relation (c.g., timeZone, birthPlace) should yield single answer but multiple answers are produced.
Compositional & Textual Rel, Gen and Loc Data Construction For Textual Reliability and Textual Generality, we generate two semantically equivalent questions (1 and d2 using GPT-4o, based on the knowledge triples (e', r, o) and the selected object pairs o').
For Textual Locality, since this metric evaluates whether unrelated knowledge remains unchanged before and after the edit, reuse the original textual locality data from the VLKEB dataset without additional data generation.
For Compositional Reliability, the VLKEB evaluation set already includes questions aligned with the knowledge triples (E',r,o). o). To generate compositional QA pairs for the newly generated triples in the train set, we leverage the extracted knowledge triples and the corresponding object pairs (o, o'), to formulate a QA pair using GPT-4o.
B.2 Prompt Templates
This subsection presents the prompt templates used in this study for data generation, using GPT-4o as the underlying model.
Table presents the prompt template used to generate candidate objects and corresponding QA pairs for Textual Reliability and Textual Generality in cases where suitable object pairs could not be retrieved during the object pair selection process.
Table 5 illustrates the prompt template used to generate QA pairs for Textual Reliability and Textual Generality, given a knowledge triple (e',r, (,,,o')
Table 6 shows the prompt template designed to generate Compositional Reliability QA pairs in the training set.
15

---

Table Prompt template used to generate o' via GPT-4o.
Prompt Template for Generating o'
System:
You are a counterfactual entity generator and question creator.
The input triple may contain 'null' the object position, representing missing or unmatchable entity.
Your job
1. Replace each 'null" with new counterfactual enity that fits the subject and relation.
2. The new entity must:
Be the correct semantic type (e.g., person, location, organization)
Be factually incorrect (i.e., real-world truth)
Be plausible and coherent in context
3. Generate two paraphrased questions and one answer based on the modified triple.
Examples:
Example
Triple: ("Amy Irving", "spouse", [null])
Modified: [mull] -> ["Steve Jobs"]
Q1: Who is Amy Irving married to?
Q2: Who the spouse of Amy Irving?
A: Steve Jobs
|6 in-context demonstrations abbreviated]
User:
Tr Triple: ("Fort Smith, Arkansas", "timeZone", [null])
System:
Modified: [null] -> ["Pacific Time Zone"]
Q1: What time zone does Fort Smith, Arkansas observe?
Q2: In which time zone is Fort Smith, Arkansas located?
Pacific Time Zone
16

---

Table 5: Prompt template used generate Textual Reliability, Generality QA pairs via GPT-4o.
Prompt Template for Textual Reliability, Generality QA Generation
System:
You question generator specializing in single-hop Counterfactual QA. Users will provide triple in the format (subject, relation, object). Each triple intentionally contradicts real-world knowledge. Your task is generate rephrased questions and one answer from the triple, while following the instructions below:
Task Rules
1. Always treat the imput triple as counterfactual, ie., it intentionally represents incorrect factual knowledge.
2. NOT attempt correct triple match real- world knowledge.
3. Do NOT modify the object if the triple is already:
Semantically plausible, Syntactically coherent
And type-consistent with the relation.
4. Only modify the object
Semantically incoherent (e.g., wrong tyрe)
Syntactically awkward (e.g., meaningless structure)
5. Avoid generating absurd, random, or nonsensical substitutions.
6. The two questions (Q1, Q2) must:
Ask about the same information
Use different wording or phrasing
explicitly mention the relation or object
7. The answer (A) must exactly match the object entity.
8. If you modify the object, indicate the change with Modified: [original_object]] -> [nev_objjcct]
Valid Counterfactual Examples (No Modification Required)
Triple: ("Barack Obama", "jobTitle", ["Astromaut"]]
Q1: What is Barack Obama's profession?
Q2: What job does Barack Obama hold?
A: Astronaut
[3 in-context demonstrations abbreviated]
Unnatural Triple Examples (Require Modification)
Triple: ("Eiffel Tower", "location", ["Albert Einstein"])
Kodified: ["Albert Einstein"] -> ["Japan"]
Q1: Where is the Eiffel Tower located?
Q2: In which country is the Eiffel Tower situated?
A: Japan
[2 in-context demonstrations abbreviated]
User:
Triple: ("Fort Smith, Arkansas", "timeZone", "Pacific Time Zone")
System:
Q1: What time setting does Fort Smith, Arkansas follow?
Q2: Which time zone does Fort Smith, Arkansas observe?
A: Pacific Time Zone
17

---

Table 6: Prompt template used to generate Compositional Reliability QA pairs via GPT-4o.
Prompt Template for Compositional Reliability QA Generation
System:
You are a multimodal question generator. Your task create new question (Q) a corresponding answer (A) based on the triple, edit q, and edit provided by the user.
Important Guidelines:
1. Your task is to generate and A only, using the input triple and edit_q&a as guide. Do not modify recreate edit_ q edit
2. Ensure that:
The generated question (Q) is contextually relevant to the triple and avoids explicitly including the answer.
The generated answer (A) aligns accurately with the triple and edit_a.
3. Always output in this exact format:
Q: [Generated question]
A:
[Generated answer]
Examples:
Example
Triple: ("Fort Smith, Arkansas", "timeZone", "Central Time Zone")
edit_q: What time zone is Fort Smith in?
edit_a: Central Time Zone
Output:
9: What is the time zone of the location shown in the picture?
A: Central Time Zone
[3 in-context demonstrations abbreviated]
User:
Triple: ("Fort Smith, Arkanaas", "timeZone", "Pacific Time Zone")
edit_q: which time zone Fort Smith, Arkansas located?
edit_a: Pacific Time Zone
System:
9: What time zone does the city shown in the image belong to?
A: Pacific Time Zone
B.3 Example of Compositional Editing
Here present an example table 7 for compositional editing that includes both textual and visual edit, along with evaluation metries including visual reliability, textual reliability, and compositional reliability.
C
Experiment Details
C.1 Metrics
We evaluate knowledge editing methods based on three established criteria from prior work: Reliabil- ity, Generality. and Locality.
Let f denote a vision-language model that maps an input æ, optionally accompanied by an image input i, to output y. Here, 0 and 0' represent the model parameters before and after the editing operation, respectively.
Reliability Reliability measures the extent to which the edited model produces the desired target output Be instead of the original output yo. computed as the average accuracy across all edited instances.
18

---

Table An example of Visual, Textual, Compositional Reliability
RATATOUILLE
Visual Edit
Q: What animated film is featured in this image?
Original: Ratatouille_ (film)
Target: Finding Nemo
Textual Edit
Q: Which company distributed the film Finding Nemo?
Original: Walt Disney Studios Motion Pictures
Target: Warner Bros.
Visual Reliability
Q: What animated film featured in this image?
A: Finding Nemo
Textual Reliability
Q: Which company distributed the film Finding Nemo?
A: Warner Bros.
Compositional Reliability
Q: Which company distributed the movie shown in the picture?
A: Warner Bros.
VisRel ((iestee / D (If(ie,ze;0') = yc
(8)
TextRel Es Ne Drenated [[]f(xe;0)) =ye
Here, Dvisual edit and Drextual edit denotes the dataset used for visual editing and textual editing, respec- tively.
Generality Generality evaluates whether the edited model can generalize beyond the specific edit case to semantically equivalent inputs across modalities. It is measured by the model's accuracy on perturbed neighbors of the original input, such as rephrased text or modified images.
ImageGen
E (eo De spo-(tee
[[ff(ir, (e:'' yel
(10)
TextGen
(ee Dati T
[=f(((ee,<=::0// ye
(11)
where T(xx) and T(t.) represent perturbed or semantically modified versions of and i.. Since textual generality can also be evaluated within textual edits, ie is treated as optional in the Tex setting.
19

---

Locality measures the extent to which the edited model preserves its behavior on inputs unrelated the edit. It is evaluated by comparing the outputs of the original and edited models on out-of-neighhor examples, using both textual (TextL.oc) and visual (Imagel.oc) probes.
Locality
ImageLoc
B (6 iuuge [(((,,,;;) = f(i, 013
(12)
TextLoc
E ( [[((=;;)) = f(x::0))
(13)
where Diest and Dise represent locality datasets.
C.2 LVLM Details
The experiments in this study were conducted using two LVLMs: LLaVA-1.5 and MiniGPT4, LLaVA- 1.5 employs Vicuna-7B-vI.5 as the language model and VIT-L (0.3B) the vision encoder, while MiniGPT4 utilizes Vicuna-7B for the language model and ViT-g (IB) for the vision encoder.
C.3 Knowledge Editing Methods
FT (Fine-tune) In our Fine-tuning (FT) setup, model parameters are updated via gradient descent applied to the designated target layers. Before editing, we store the current weights of these layers to allow restoration after single-edit operation. We use the AdamW optimizer and configure it such that gradients are computed and applied only to the specified fine-tuning parameters.
LoRA Low-Rank Adaptation (LoRA) [25] provides parameter-efficient alternative full fine- tuning by introducing pair of low-rank trainable matrices into the weight update path of selected layers-typically the query-key key-value projections or feed-forward blocks. During training, the original pre-trained weights are frozen, and only the low-rank matrices are optimized, yielding drastic reduction the number of trainable parameters and memory footprint. At inference time, the low-rank updates are linearly combined with the frozen base weights, producing an adapted model that preserves the generalization ability of the original network while incorporating tusk-specific knowledge.
MEND Model Editor Networks with Gradient Decomposition (MEND) [24] offers an intrinsic editing approach that efficiently updates parameters in large language models (LLMs) embedded within vision-language models (LVLMs). Rather than modifying the entire model, MEND trains compact auxiliary networks that apply localized edits based on a single input-output example. These edits are performed by transforming fine-tuning gradients using a low-rank decomposition, enabling scalable and generalizable updates without compromising the model's performance on unrelated tasks.
SERAC Semi-Parametric Editing with Retrieval- Augmmntee Counterfactual (SERAC) [18] is a memory-based framework composed of an explicit memory cache-along with associated scope classifier-and and a counterfactual model. The trained scope classifier determines whether an incoming query falls within the editing scope. If the query is in scope, the system forwards the original input together with the retrieved edits from the explicit memory to the counterfactual model; otherwise, the input is passed unchanged to the underlying base model. In our experiments, the scope classifier instantiated with a BERT mncoder(distilbend wewsstibeertbebeeeeeeeeeeeeeeeee waaert-aaeecssse casedd, while the counterfactual model varies across LVLMs.
WISE WISE [44] introduces a dual parametric memory for lifelong editing: a main memory that preserves pretrained weights and side memory copied FFN value matrix) that stores edits. A routing activation module decides at inference which memory to use, enabling edited knowledge to be applied only when in scope and preventing collateral changes elsewhere. For continual streams, WISE performs knowledge sharding where different sets of edits reside in distinct and orthogonal subspaces of parameters. Then merges them to integrate edits without destructive interference; it can also maintain multiple side memories and retrieve by activation (WiSe-Retrieee). This seedinn-sharding--ae
design balances reliability, locality, and generalization, overcoming the trade-offs of purely parametric or retrieval-basee based editors.
20

---

C.4 Experiment Resources and Parameters
All experiments are performed on an NVIDIA A100 80GB GPU utilizing the PyTorch framework. The parameters used in the experiments are summarized in Table 8, and detailed configuration files are available in the code repository. For detailed parameters and configurations specific to MemEIC, please refer to Section D.3.
Table 8: Hyper-parameter settings for LLaVA-1.5 and MiniGPT4 across editing methods
Method
Model
Steps/ MaxIter
Edit Layer
Optimizer
LR
MiniGPT-4
31 layer of Vieuna-7B
AdamW
le-4
FT-LLM
LLaVA-1.5
10
31 l layer of Vicuna-7B-v1..5
AdamW
le-4
MiniGPT-4
10
all layer of Vicuna-78 all layer of Mecuna-BBvvl
AdamW
le-6
LoRA
LLaVA-1.5
10
AdamW
lc-6
MiniGPT-4
20
27th layer of Vicuna-77 7B 27th layer Vicuna-7B-vi
SGD
le-1
WISE
LLaVA-1.5
20
SGD
lc-1
MiniGPT-4
40000
layer 29, 30, 31 Vicuna-7B
Adam
le-6
MEND
LLaVA-1.5
40000
layer 29, 30, 31 of Vicuna-7B-v1.55
Adam
1c-6
MiniGPT-4
50000
31 layer of Vicuna-7B
Adam
Se-5
SERAC
LLaVA-1.5
50000
31 layer of Vicuna-7B-v1.5
Adam
le-5
D
Model Details
D.1 Query Decomposition
We automatically classify and decompose each query into its visual and textual components via query decomposition. To perform this step, utilize GPT-4o, and Table 9 presents the prompt template used for this process.
To ensure consistent and controlled evaluation across various experimental conditions, we preprocess all train and evaluation queries using the query decomposition module. Specifically, query decompo- sition is performed in advance, resulting in static set of decomposed queries. However, in practical deployment scenarios, query decomposition must be performed on-the-fly. requiring a full inference pass through the query decomposer such as GPT-4o for each incoming query.
For the performance evaluation of the Query Decomposition Module, please refer Section E.1
D.2 External Memory
In the External Memory module (Mem-E), we used distilbert-base- base-cased as text classifier to compute textual similarity with the stored text edits, and ELIP-LL3300 as image encoder to assess visual similarity.
For detailed description of the parameters and configurations used in the external module training, refer Section D.3.
D.3 Training Stage Details
Stage External Memory the first training stage, we train the text and image classifiers of the external memory module. To prevent temporal interference during classifier training. this stage conducted under single-edit setting rather than a continual-editt scenario. The external retrieval module is trained
datset of 5,000 examples, and Table 10 summarizes the hyperparameter configurations used for LLaVA-1.5 and MiniGPT4. The training objective for the external component defined as the average binary cross-entropy loss computed over the training dataset.
Stage 2: Knowledge Connector In the second training stage, the Knowledge Connector- implemented as LoRA increments the self-attention q/k projections of a dedicated fusion layer-ii
21

---

Table 9: Prompt template for Query Decomposition
Prompt Template for Query Decompasition
System:
Task Definition
You powerful query classifier and generator.
Please decompose the question given by the user into image level and text level subquestions using the following steps.
1. Identity the type of the question (e.g.. Who, What, Where, When).
2. Identify the key entity entities relevant to the question.
3. Generate the image level subquestion by focusing on the visual entity(c.g. city featured in the image, person
in the picture) the question. DON'T CHANGE THE MEANING OF THE ORIGINAL QUESTION.
4. Generate the level subquestion only if additional non-visual information needed to fully address the query. not, return 'None'.
Examples:
Question: What city featured in the image?
Image Level: What [city is featured in the imagel?
Text Level: None
Question: What is the time zone of the location shown in the picture?
Image Level: What's [the location shown in the picture]?
Text Level: What is the time zone of [the location shown the picture]?
120 in-context demonstrations abbreviated]
User:
Question: Which league associated with the club in the picture?
System:
Image Level: What [club is the picture]?
Text Level: Which league is associated with [the club in the picture]?
Table 10: Training configuration for Stage 1
Models
MaxIter
Optimizer
LR
LLaVA-1.5
50000
Adam Adam
le-5
MiniGPT4
50000
le-5
trained jointly with the dual-LoRA adapters. These adapters, positioned in the feed-forward network (FFN) sub-layers of every transformer block, are simultaneously updated whenever an edit occurs. Although not the primary target of fine-tuning, the adapters serve maintain consistent feature space for the connector. Training is conducted on the 500 samples of CCKEB train split containing visual, textual, and compositional edits. At each edit step, the relevant adapter(s) and the connector are co-activated and jointly optimized.
To simulate adversarial retrieval scenario, external-memory hit rates capped at 70 % for LLaVA-1.5 and 50 for MiniGPT-4. The resulting adapter weights are carried over to Stage 3 so that inference receives feature representations drawn from the same distribution observed during training. For both models, we employ the AdamW optimizer with a learning rate of le - 6.
E
Additional Experiments
E.1 Results on Query Decomposition
To assess the performance of the Query Decomposition Module, we conducted experiments using decomposed query which was conducted by preprocessing queries from the training and
22

---

evaluation sets. The evaluation was performed along two axes: Decomposition Accuracy and Semantic Quality.
Decomposition Accuracy measures whether compositional queries are correctly decomposed into both image-level and text-level sub-queries, and whether visual-only queries classified solely into image-level components, without being incorrectly decomposed into image-level and text-level sub-queries. Queries that do not include visual components (images) are classified solely text-level and are therefore excluded from the evaluation. Semantic Quality evaluates the quality and semantic validity of the generated image-level and text-level sub-queries.
Table 11: Percentage distribution of decomposed query types across training and test sets.
Image & Text Level
Image Level
Text Level
Qc (Train)
96.06%
3.82%
0.12%
Qu (Train)
4.98% %
94.99%
0.03%
Qc (Test)
97.81%
2.03%
0.16%
Qv (Test) 3.26%
96.69%
0.05%
Decomposition Accuracy Table 11 presents the percentage distribution of decomposed query types for Compositional Queries (Q2) and Visual Queries (Q,). For both training and test sets, over 96% of Qe instances are successfully decomposed into both image-level and text-level components, indicating reliable modality separation. Similarly, for Qu. more than 94% of the instances are accurately decomposed into image-level components only, demonstrating the robustness of the decomposition strategy for visual-only queries.
Table 12: Average scores of decomposed queries evaluated by BERTScore and sBERT similarity
BERTScore
sBERT
Qc (Train)
93.68
85.69
Q, (Train)
98.44
96.54
Q (Test)
92.92
85.71
Qv (Test) 99.06
97.98
Semantic Quality Table 12 reports the results of our semantic quality evaluation, presenting the average BERTScore and SBERT cosine similarity between the reference answers and the generated queries for both compositional queries (Q.) and visual queries (Q.). For reference answers, we adopt the original visual reliability questions for image-level compositional queries, and the original compositional queries for text-level cases. For visual queries, the original visual queries are used reference the image level. We compute BERTScore using the sestiibbrr-base-uee model and SBERT similarity using the all aal-mppet--ee base-ttbbaaeeee v2 model, both based on cosine similarity,
As shown in Table 12, compositional queries (Q.) achieve BERTScores above 92 and sBERT scores around 85 on both training and test sets, indicating that the decomposed queries exhibit relatively high degree of semantic similarity to the reference answers. Due their structure involving both visual- and text-level sub-queries, compositional queries introduce more variability in expression, which likely contributes the larger gap observed between BERTScore and sBERT. Visual queries (Q..) consistently yield higher semantic similarity scores across both metrics and data splits, suggesting that queries decomposed from visual queries more semantically aligned with the ground-truth answers. Moreover, the semantic similarity scores remain stable between training and test sets, indicating the query decomposition module's generalization capability.
F.2 Results on Internal Memory: Knowledge Separation
Table 13 provides a detailed breakdown of CCKEB performance across varying sequential gaps in the experiments described in Section 4.4. As the gap between the initial edit and subsequent evaluations grows, the Internal Memory-based approach exhibits pronounced decline in accuracy- -eviencce of
23

---

Table 13: Knowledge separation (Mem-I) results (averaged over runs) for CCKEB validation set
(a) Visual Edit
Gap
Rel
T-Gen
I-Gen
T-Loc
I-Loc
Mem-l (Dual-LoRA, r:8 x 2)
99.96
95.53
99.96
82.53
14.62
10
80.02
75.41
80.75
78.09
11.70
20
76,08
73.87
75.52
82.88
16.26
50
72.61
69.93
72.70
81.68
13.67
100
70.24
67.09
69.50
81.07
8.17
Average
74.73
71.57
74.61
80.93
12.45
Mem-I (Single-LoRA,, r:16)
99.52
95.41
99.50
67.27
9.15
10
80.69
75.71
80.87
70.17
11.54
20
75.88
72.56
75.75
59.83
8.65
50
72.15
70.14
72.14
64.64
8.59
100
69.68
67.61
69.20
58.00
6.81
Average Difference
74.60
71.50
74.49
63.16
9.59
-0.13%
-0.07%
-0.12%
-17.77%
-2.86% %
Mem-l (Single-LoRA, r:16)
Average Difference
74.06
71.02
74.08
60.42
7.90
-0.67% %
-0.55%
-0.53 %
-20.51 %
-4.55% %
(b) Textual Edit
Gap
Rel
Gen
Loc
Mem-1 (Dual-LoRA, r:8 2)
0
99.99
99.03
69.66
10
87.34
82.87
68.19
20
84.64
80.70
68.63
50
74.26
70.65
73.36
100
62.72
60.73
62.70
Average
77.24
73.73
68.22
Mem-1 (Single-LoRA, r:16)
0
99.99
99.25
67.27
10
88.81
83.25
70.17
20
74.14 14
69.76
59.83
50
71.34
68.30
64.64
100
63.87
60.68
58.00
Average Difference
74.54
70.49
63.16
-2.70%
-3.24%
-5.06 %
Mem-1 (Single-LoRA) r:8)
Average 72.12
68.29
60.42
Difference
-5.12%
-5.44% %
-7.80 %

---

severe catastrophic forgetting when multiple edits accumulate in shared parameter space. In contrast, our knowledge separation strategy (Mem-I (Dual-LoRA)) sustains robust internal knowledge editing under identical parameter budgets: both Visual Edit and Textual Edit tasks maintain high reliability, generality, and locality metrics, demonstrating the effectiveness modality-specifft memory division in mitigating forgetting.
Activation Visualization Figure 5 presents a heatmap illus- trating the impact each adapter on layer activations in re- sponse to different input queries, visual query (Q+) and textual query (Q,). When Q, provided as input, the activations of the Visual Adapter exhibit pronounced increase, whereas the
Textual Adapter demonstrates stronger activation when a Q2 is supplied. This observation indicates that the model effectively leverages the appropriate adapter depending on the modality of the input, selectively utiliizing visual and textual information accordingly.
Visual Adapter
Teotul Adapter
Figure 5: Activation Visualization of Knowledge Separation
E.3
Additional Results on Knowledge Connector
E.3.1
Performance Comparison on MiniGPT4
Performance Comparison on MiniGPT4
Compositional Reliability (%)
80
60
40
Single-LoRA (MEM-I)
Dual-LoRA (MEM-I)
20
Base RAG (MEM-E)
Dual-LoRA. RAG (MEM-EI)
Dual-LoRA RAG (MEM-EI, w/Conn.)
10
20
50
100
Sequential Gap
Figure 6: Performance comparison of compositional editing methods methods in Minigpt4.
Edit configurations
We group the five systems into three memory regimes:
Mem-E (Internal Memory) LoRA adapters are the sole repository of new knowledge.
Single-LoRA visual and textual edits share single feature space.
Dual-LoRA visual and textual edits occupy separate feature spaces.
Mem-I (External Memory) No parameters are updated; edited knowledge is retrieved at test time.
Base + RAG - Minigpt4 with a oracle retriever that always returns the correct visual and textual memories.
Mem-EI (External & Internal) Internal LoRA adapters are augmented with retrieval.
Dual-LoRA RAG Mem-E + Mem-I, but without an knowledge connector.
Dual-LoRA + RAG Connector (MemEIC, Ours) same above, plus frozen attention-based Knowledge Connector that fuses the two LoRA streams.
Results Figure 6 shows Compositional Reliability (CompRel) across sequential gaps g {0, 10, 20, 50, 1003.
25

---

Mem-I (Internal Memory Only) Single-LoRA begins at 69% when g O, then drops sharply to 56% by g 10 and levels off around 53%, indicating severe cross-modal interference. Dual-LoRA declines more gradually from 60% to 52%, but still cannot maintain high compositional aceuracy. This result highlights the inherent limitations of transformer- based internal memory editing, where repeated edits lead to knowledge forgetting [29-31]; ultimately degrading compositional reasoning performance.
Mem-E (External Memory Only) Retrieval without any parameter update (Base+ RAG) remains steady at 56% across all gaps, demonstrating that RAG alone cannot prevent perfor- mance decay. This primarily due conflicts between externally retrieved knowledge and the model's 's internal knowledge, causing the baseline model to fail in effective compositional reasoning [20-23].
Mem-EI (Internal + External) Combining Dual-LoRA with retrieval yields modest gain to 62% at g = O, but this falls to 57% by g = 10, reflecting unresolved conflicts between visual and textual edits. Adding the Connector (MemEIC) boosts CompRel to 99% at g 0 and sustains 96% at g = 100, a consistent 40% improvement over the best non- con-Connector variant.
E.3.2 Noisy Retrieval Boosts Compositional Reliability
We train Knowledge Connector variants with fixed training- time retrieval accuracies of 50 %, 70 %. and 100 %, and then evaluate each connector under two test-time regimes: @ 50% (noisy retrieval) and Test @ 100% (oracle retrieval). Figure 7 reports the compositional reliability (CompRel) obtained by the two backbone LVLMs, LLaVA-1.5 and MiniGPT.4.
Impact of Adversarial Retrieval on Connector Robustness
Impact Adversarial Retrieval on Connector Robustness
100.0
100.0
Rellabiltty (%)
97.5
97.5
Compostional /
05.0
Compastional Retposttooa//lllbillll
Rellability
92.5%
12.0%
34.5%
92.5 33.5%
90.0
90.0
85.0
85.0
02.5
02.5
Rest050%
80.0
50
70
100
80.0
70
100
Training Retrieval Accuracy (%)
Training Retrieval Accuracy (3)
(a) LLaVAI.5
(b) MiniGPT-4
Figure Impact of Adversarial Retrieval on Connector Robustness. Compositional reliability under noisy (Test@ 50%, red) and perfect (Test@ 100%, blue) retrieval, after training the knowledge connector with oracle (100%), moderate-noisee70%%) or heavy-noise(50%) prompts. Purple shading indicates the robustness gap.
Results For 1...5. the connector trained with moderate retrieval noise (70 % accuracy) achieves the highest reliability in the noisy setting (86.3 %) while matching the smallest robustness gap (12.0 %). The oracle-trained connector attains the highest ceiling (99.5 %) but suffers the largest gap (14.5 %).
For MiniGPT-4 we observe the same qualitative trend, but the optimal trade-off is reached one step earlier, at the 50 %-trained connector: it obtains the best noisy-case reliability (86.4 %) and the narrowest robustness gap (11.4 %). Training with 70 % accuracy raises the oracle score slightly (98.4 4 %) but increases the gap to 13.1 %, while oracle-only training again gives the highest ceiling (98.0% %) at the cost of the largest gap (.5.%).
Discussion Across both backbones, introducing a realistic amount of retrieval noise during connec- tor training markedly improves the robus/nexvucccuruyy balance. With LLa 30% error rate (70% training accuracy) is sufficient; the smaller MiniGPT-4 benefits from even stronger signal, peaking when trained with 50% accuracy. In both cases the connector learns to cross-check external evidence against its internal memory: when the retriever is unreliable it falls back on the internal edits,
26

---

and when the retriever is correct it yields near-oracle performance. Conversely, training exclusively with perfect retrieval encourages brittle over-reliance on the external prompt, while training with too much noise depresses the attainable ceiling without further narrowing the robustness gap-
These findings confirm that moderate, task- refrieval noise is crucial ingredient for building connectors that remain dependable when deployed in the wild, where external memories are often incomplete or partly incorrect.
E.3.3 Examples for Connector Robustness
Retiival Bozn Extemaal Memory
s000ess
Gillre
GOOD WILL HUN
Edited OV: "What the title of the film in this scene?"
image?
Raiders of
Cobral
(ebrr([1998) 1986 film)
Edited Qt Text Editing Knowledge: Which company
editor 1996 fil)'?
distributed film "Radas of Lost Ask"?
George Pan Cosmatos (C (Cunsterfuctual Retrieval)
Vingin Records
Q: Who was involved in editing the film shown n in the picture 1986? A: George Pan Cosmatos xX)
Q: Which company distributed the movie related the one shown in the picture?
A: Virgin Records
(a) without Knowledge Connector
GOOD WILL HUNGING
Edited ON:
Edited Which action film feutured in this image?
+ Raiders of the Lost Ark
Cobra(1986 fil)
Edited Editing Knowledge: Which company
Edited Who the editor of CCobral Cobra[1986 film)"
distributed the film "Radas ofth Lost Ark
Gorrge Cosmatos (Counterfactual Retrieval)
Vingin Reconds
Q: involied editing shown picture
Q: Which company distributed the movic related the shown in the picture?
from 1986?
Christopher (olam
A: Virgin Recorda (o)
(b) with Knowledge Connector
Figure Model inference examples with and without the knowledge connector adversarial retriever setting
Figure 8 presents model inference examples with and without the knowledge connector. In the absence of the knowledge connector, the model produces correct outputs when external memory retrieval is accurate; however, it tends to rely on the retrieved information when the retrieval is incorrect. In contrast, the model with the knowledge connector-traiie internal knowledge alongside the retrieved information. This allows it to generate reliable outputs not only when retrieval is accurate but also when the retrieved information misleading incomplete. rather than relying on it solely.
with adversarial retriever- leverage
E.3.4 Additional Results of the Knowledge Connector Selection
We investigate how various configurations of the Knowledge Connector influence the overall perfor- mance, as summarized in Table 14, reporting both CompRel and UR for three connector types (MLP, FEN, Attention) under Test@ 50 retrieval.
To quantify how much the connector improves over internal memory alone, we define the Knowledge Utilization Ratio KUR)
x CompRel
KUR
(14)
Rolyis + Relex
which measures the number of correct answers obtained via the connector relative to the average correctness of the visual-only and textual-only internal adapters. Ratio greater than indicates that the connector yiclds more correct answers than internal memory alone.
27

---

Table 14: Comparison of different connection methods with Test@ 50% accuracy in CCKEB valida- tion. Ratio indicates the Knowledge Utilization Ratio defined in Eq. 14.
Method
Conn (MIP), w/ RAG 50%)
Conn (FEN), w/ RAG 50%)
Conn (Attention), (w/ RAG 50%)
CompRel
Ratio (KUR)
CompRel
Ratio (KUR)
CompRel
Ratio (KUR)
Gap 0
87.25
1.00
93.85
1.00
89.25
1.00
10
5,25
0.07
83.13
1.12
86.02
1.28
20
3.17
0.04
80.27
1.17
84.79
1.34
50
0.23
0.00
76.37
1.24
81.69
1.33
100
0.11
0.00
78.61
1.36
78.36
1.38
Average
19.20
0.22
82.44
1.17
84.02
1.26
MLP-based Connector MLP-based Connector shows strong performance at the initial edit (gap=0: 87.25%) but rapidly loses its connection ability as the gap increases, resulting in an average CompRel of only 19.20% (KUR 0.22). This decline likely stems from its inability track evolving dual- LoRA features under continual updates, leading to mismatch between input representations and connector mapping.
FFN-based Connector FFN-based Connector maintains relatively high performance across gaps, achieving an average CompRel 82.44% (KUR= 1.17). However, the performance still drops noticeably as the gap increases, suggesting limited robustness for long-term compositional reasoning despite being a better connector than MLP.
Attention-based Connector implemented by injecting low-rank LoRA increments into the query and key projections of the attention layer, achieves the highest average CompRel of 8f.44.29 84.02% (KU R 1.26). By adaptively extracting and fusing modality-specific features from the dual-LoRA modules, the attention-base connector exhibits both higher compositional reliability and better utilization of internal and external knowledge across sequential gaps. Notably, the performance gap large edit intervals remains minimal compared to FFN and MLP.
In summary, while all connectors play role in bridging poddaltt--specinee memories, the attention- based connector demonstrates superior robustness and knowledge integration, especially under continual, compositional editing scenarios.
F
Limitations and Broader Impacts
F.1
Limitations
Despite demonstrating strong empirical results, our approach has several limitations. First, our framework currently focuses solely on multimodal knowledge editing scenarios involving visual recognition and textual understanding, and it does not extend naturally to multimodal generation tasks. Additionally, due computational resource constraints, MemEIC has primarily been evaluated on moderately sized vision- language models rather than extremely large-scale models; thus, its performance on larger architectures remains unexplored.
F.2 Broader Impacts
In terms of societal impacts, our research advances the capability of continually updating LVLM models, holding substantial benefits for applications requiring accurate, up-to-date visual and textual knowledge in time-sensitive settings (e.g., misinformation prevention, timely fact correction). How- ever, we acknowledge potential negative implications, such as the risk of malicious actors applying this technology to rapidly propagate disinformation or harmful content. Therefore, we emphasize the importance of deploying rigorous safeguards, monitoring mechanisms, and strong ethical guidelines to ensure responsible use and mitigate the risk of misuse.
28

---

G
Original Results
G.1 Original Results of Main Results on CCKEB
In Section 4.2, we present the main results of editing performance varying gap sizes. Tables 15 and 16 report the original results for LLaVA-1.5 and MiniGPT4, respectively.
The first five metrics correspond to those proposed in the original VLKEB benchmark, while the latter four-denoted as Extended Metrics-are introduced via the CCKEB benchmark. The VLKEB metrics primarily assess performance visual edits, whereas the newly constructed T-Rel, T-Gen, and T-Loc metrics evaluate textual edits.
G.2 Original Results of Ablation Study External Memory
In Section 4.3, present the ablation experiment of External Memory. Tables 17 shows the original results for the experiment.
Table 15: Original results in LLaVA-1.5
VLKEB
Extended
Method
Gap
Rel
T-Gen
I-Gen
T-Loc
I-L.oc
CompRel
T-Rel
T-Gen
T-L.oc
99.57
98.19
99.05
38.09
6.66
78.42
99.87
99.12
38.09
10
89.16
81.31
84.09
37.11
6.48
58.51
83.96
76.22
37.11
FT (LLM)
20
86.18
77.82
80.27
36.33
6.19
55.44
79.60
71.45
36.33
50
78.63
71.86
74.09
33.76
5.41
52.76
74.54
65.14
33.76
100
74.76
67.79
70.39
30.21
4.53
49.06
68.81
61.73
30.21
0
99.32
95.77
99.33
68.34
9.56
80.14
99.92
98.96
68.34
10
78.20
74.63
78.16
65.04
9.07
62.03
78.20
75.50
65.04
LoRA
20
72.30
70.03
72.20
62.35
8.96
57.28
68.21
64.85
62.35
50
69.12
68.77
68.89
60.06
6.82
55.29
66.58
64.92
60.06
100
67.14
65.95
66.56
65.41
8.25
53.37
63.18
60.73
65.41
0
8.37
7.82
8.21
15.75
5,60
5,04
7.55
7.19
15.75
10
6.22
5.31
5.89
12.64
3.48
3.43
5.85
6.11
12.64
MEND
20
4.76
4.20
4.56
12.01
2.11
2.22
5.40
4.86
12.01
50
1.37
1.41
6.13
0.65
0.85
2.35
2.40
6.13
100
0.33
0.57
0.30
2.09
0.08
0.06
0.98
0.50
2.09
0
65.46
42.46
65.36
100.00
1.40
40.92
87.44
56.55
100.00
10
64.35
42.28
64.25
100.00
1.40
40.99
87.44
55.70
100.00
SERAC
20
63.97
41.73
63.82
100.00
1.40
40.73
86.91
$5.48
100.00
50
61.93
41.71
61.80
100.00
1.41
40.77
86.60
54.97
100.00
100
60.13
41.28
60.12
100.00
1.41
40.38
86.33
$4.75
100.00
0
81.17
74.36
77.87
92.12
28.41
49.47
97.69
86.67
92.12
10
82.06
72.65
76.13
92.09
27.45
48.10
96.95
80.77
92.09
WISE
20
81.60
70.27
75.05
92.02
27.19
47.47
94.70
76.95
92.02
50
83.09
69.89
76.17
91.61
26.09
47.83
93.76
74.66
91.61
100
82.04
69.42
74.70
91.03
25.10
48.16
87.72
70.41
91.03
0
99.66
98.76
93.53
100.00
47.79
85.16
99.94
99.53
100.00
10
98.86
94.60
90.71
100.00
46.90
80.88
93.36
86.05
100.00
MemEIC
20
98.91
93.32
90.59
100.00
45.97
80.35
90.73
83.03
100,00
50
98.72
92.44
89.46
100.00
43.97
79.03
89.61
77.02
100.00
100
98.48
91.74
88.06
100.00
43.17
77.36
88.77
74.40
100.00
H Mathematical Validation of the Knowledge Connector
We now show three simple but important properties of our modality-aware connector, using the notation from Section 3.2.
29

---

Table 16: Original results in MiniGPT4
VLKEB
Extended
Method
Gap
Rel
T-Gen
I-Gen
T-Loc
I-Loc
CompRel
T-Rel
T-Gen
T-Locc
0
99.83
99.79
99.49
44.06
4.67
47.42
99.99
97.67
44.06
10
76.46
74.74
73.69
42.87
4.30
44.24
91.20
75.41
42.87
FT (LLM)
20
73.29
72.20
70.96
42.26
4.03
41.63
86.42
69.87
42.26
50
67.73
66.94
66.04
40.22
3.26
39.35
80.83
64.90
40.22
100
64.80
63.62
62.59
36.05
2.54
36.12
77.70
62.46
36.05
0
99.88
96.72
99.95
66.07
14,06
74.10
99.99
99.30
66.07
10
76.87
74.30
76.63
58.14
12.09
58.64
76.35
69.91
58.14
LoRA
20
73.67
71.55
72.96
66.42
13.98
55.08
69.47
62.99
66.42
50
69.58
68.69
69.12
62.50
12.60
54.69
65.07
63.22
62.50
100
68.40
67.29
68.37
60.08
11.32
52.44
61.07
61.02
60.08
0
10.63
10.70
10.31
19.70
11.72
8.19
9.29
8.68
19.70
10
9.40
9.50
9.35
17.62
9.94
7.45
8.56
8.21
17.62
MEND
20
8.42
8.32
15.84
7.03
7.64
7.26
15.84
50
4.95
5.08
4.92
9.96
3,96
3.60
4.30
4.11
9.96
100
0.84
0.78
0.89
1.50
0.33
0.69
0.69
0.58
1.57
0
45.32
47.77
44.90
100.00
3.77
41.29
44.51
100.00
10
44.52
47.35
44.28
100.00
3.76
40.62
43.71
39.51
100.00
SERAC
20
44.53
47.28
44.19
100,00
3.78
40.56
43.23
39.51
100,00
50
45.08
47.16
44.53
100.00
3.78
40.38
42.49
39.62
100.00
100
44.74
46.86
44.44
100.00
3.74
40.79
42.61
38.68
100.00
0
81.52
79.81
81.87
94.36
18.52
44.42
89.27
70.55
94.36
10
65.71
63.40
64.16
94.40
17.58
44.22
91.53
68.97
94.40
WISE
20
61.76
61.04
60.86
94.25
17.08
43.51
92.06
67.31
94.25
50
60.94
59.65
59.34
93.98
16.47
43.77
93.15
69.62
93.98
100
60.66
60.00
59.49
94.27
15.99
43.58
94.35
70.54
94.27
0
99.52
97.61
94.31
99.98
50.99
78.05
99.85
97.94
99.98
10
96.40
88.58
87.22
99.98
50.20
76.25
93.34
85.02
99.98
MemEIC
20
95.67
88.07
86.31
99.98
49.41
74.78
91.74
80.93
99.98
50
95.35
85,95
84.33
100.00
46.93
72.61
90.87
76.48
100.00
100
96.42
85.48
85.12
100.00
45.89
70.96
88.64
73.41
100.00
Table 17: Experimental results for Sequential Edit in the VIKEE setting using LLaVA-1.5
Model
Rel
T-Gen
I-Gen
T-Loc
I-Locc
Port
Mem-E (Rv, w/ tex+vis)
96.51
93.42
80.10
99.43
61.57
62.09
Mem-E (Rv, w/ tex+vis) 10
96.51
93.42
80.27
99.41
58.78
62.09
Mem-E (Rv, w/ tex+vis) 20
96.51
93.42
80.27
99.39
56.70
62.09
Mem-E (Rv, w/ tex+vis) 50
96.51
93.42
80.27
99.41
58.78
62.09
Mem-E (Rv, w/ tex+vis) 100
96.51
93.42
78.60
99.28
49.70
62,09
Mem-E (Rv, w/ tex) 0
48.02
50.73
48.22
100.00
4.03
45.39
Mem-E (Rv, w/ tex) 10
48.02
50.73
48.22
100.00
4.03
45.21
Mem-E (Rv, w/ tex) 20
48.02
50.73
48.22
100.00
4.02
45,04
Mem-E (Rv, w/ tex) 50
48.02
50.73
48.22
100.00
4.02
44,98
Mem-E (Rv, w/ tex) 100
48.02
50.81
48.22
100.00
4.02
44.78
SERAC 0
77.10
46.91
77.65
99.99
1.94
38.39
SERAC 10
76.46
46.14
77.04
99.99
1.90
38.28
SERAC 20
76.12
45.69
76.24
99.99
1.90
38.95
SERAC 50
74.42
43.61
74.44
99.99
1.91
38.62
SERAC 100
70.92
43.64
71.12
99.99
1.90
38.30
30

---

H.1 Commutativity of Adapter Updates
At layer after self-attention and layer-norm we fuse the base FFN and two adapters:
1 (c+1)
h((0)
me
+ ml
+ m
(B.1)
where 10 is the hidden state entering the FFN block, - m is the frozen FFN's output, - mm the visual/textual LoRA outputs. By basic vector-space algebra,
m
( + m = m, + m!!
(B.2)
so the ordering of modality increments does not change h(+1))
H.2
Direct-Sum Preservation
Assume the two adapter subspaces intersect only at zero:
sman (0) span(m, (*) {0}.
(B.3)
Then every combined update
I m + mi
(m! E Hu, mi Hi)
has unique decomposition, which by definition means
He + He = Ho + H4.
(B.4)
Thus, mixing the two adapters does not lose any representational capacity.
H.3 Gating Consistency
Recall gate the connector by
a= = A I. =1,
(B.5)
and define the gated update
(e+1)) (M(()) + (0) + (0) + (1 (1/(1)) + +m) m
(B.6)
When 0, this reduces
(((+1) k200 + m
(B.7)
exactly recovering the vanilla FFN branch and ensuring backward compatibility.
31

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The claims in the abstract and introduction accurately reflect the proposed method (MemEIC), benchmark (CCKE), and core contributions such dual- dual-memory editing and a brain-inspired knowledge connector, which are consistently elaborated and validated throughout the paper.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper discusses limitations such as task scope and scalability to larger models in Appendix F.1.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
32

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We include all assumptions, numbered statements, proof sketches in Ap- pendix H.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We describe dataset splits, model architectures, training hyperparameters, and evaluation protocols in Section 4. Appendix C and D.
Guidelines:
The answer NA means that the paper does not include experiments.
the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
33

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We plan release the code and processed CCKE benchmark after acceptance, but it not publicly available at submission time.
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / /nips. wttpcc/cccgcccciecooooooooco gut//// /CodeSummi ssionnolicyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section and Appendix D.3 and C report data splits, optimizer, batch size, learning rate schedules, and all other parammearseeters used for training and evaluation.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Mainly becuase computational cost. Instead of multiple random seeds, we measure performance across five independent gap settings (0, 10, 20, 50, 100), treating this systematic variation as the source of experimental variability.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
34

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
All details are shown in C.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: : /neurips.....// cc cc/public//eeeeecceecc wtwp:/thhhicceeeeeeeeeeeeeeeeeee
Answer: [Yes]
Justification: All experiments rely on public datasets, involve no personal or sensitive data, and fully adhere to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the Broader Impacts in Appendix F.2.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
35

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We release only training scripts and public-domain data, the work introduces no new misuse risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All are cited.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
36

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. www.coo//aaooooo datasttss has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [Yes]
Justification:
We will release the MemEIC codebase, the CCKEB benchmark dataset.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve any crowdsourcing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (TRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This work does not involve any crowdsourcing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
37

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: We use GPT-4o exclusively for the Query Decomposition, a component of the MemEIC pipeline; other part of our method employs LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: / /teerrpp..ccCCofeeeeeeeeeeecceee ccc/oonfeeencce/222222... for what should or should not be described.
38


Reviews:
[{"review_id": "X9dqKsCXZq", "reviewer": "Reviewer_Cssd", "summary": "This paper introduces MemEIC, a novel framework for continual and compositional knowledge editing in large vision-language models (LVLMs). The work addresses a significant limitation in existing multimodal knowledge editing approaches, which typically handle visual and textual edits in isolation and fail to support continuous, sequential updates. The authors propose a hybrid external-internal memory architecture that combines modality-specific external retrieval with dual LoRA adapters for internal memory, connected by a knowledge connector that enables cross-modal reasoning. Additionally, they introduce the CCKEB benchmark, extending VLKEB to evaluate both continual editing capabilities and compositional reasoning through a new metric called Compositional Reliability (CompRel). Experiments on LLaVA-1.5 and MiniGPT-4 demonstrate that MemEIC significantly outperforms existing methods, achieving superior performance on complex multimodal questions while effectively preserving prior edits across sequential updates.", "questions": "- Could you provide a detailed analysis of the computational costs (e.g., inference time, parameter overhead) introduced by MemEIC's hybrid memory architecture, including the external retrieval and knowledge connector components? How do these costs compare to simpler methods (e.g., enhanced internal memory or external retrieval alone)?\n- Could you provide justification for excluding other benchmarks like MC-MKE (focused on modality consistency) or extend the experiments to broader settings to evaluate generalization?\n- How does MemEIC’s hybrid approach compare to some simple baselines that mix existing internal and external memory architectures? Some discussion or results could be helpful to enhance the paper.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n- MemEIC makes a good contribution to multimodal knowledge editing by addressing limitations in single-modality and sequential updates. Its architecture, with modality separation and a dynamic knowledge connector, provides valuable insights for the field. The new framework, benchmark, and metric are good contributions.\n- Comprehensive experiments, including ablation studies, demonstrate consistent performance gains across backbone models (e.g., LLaVA-1.5, MiniGPT-4) and evaluation metrics, particularly excelling in compositional reasoning tasks.\n\nWeakness:\n- Computational Overhead: The paper doesn’t sufficiently address the increased complexity introduced by the hybrid memory architecture. A runtime analysis would clarify whether the performance justifies the additional computational cost compared to simpler alternatives.\n- Limited Benchmarks: The reliance on the CCKEB benchmark raises concerns about generalizability. Validation on new, diverse benchmarks (e.g., MC-MKE) would strengthen the experimental scope.\n    - MC-MKE: A Fine-Grained Multimodal Knowledge Editing Benchmark Emphasizing Modality Consistency (CVPR 2025)\n- Comparison Fairness: MemEIC combines internal and external memory techniques, making it unclear whether the observed gains are due to the novel design itself or the increased resource usage. A clearer analysis comparing hybrid models to standalone internal/external methods would enhance credibility.", "was_revised": true, "final_justification": "The author's response addresses my concern", "rebuttal": "We thank the reviewer for their time and thoughtful evaluation of our work. Below, we address the specific comments in detail, and we would be happy to further engage in discussion if needed.\n\n---\n\n**Q. Computational Cost**\n> **W1.** Computational Overhead: The paper doesn’t sufficiently address the increased complexity introduced by the hybrid memory architecture. A runtime analysis would clarify whether the performance justifies the additional computational cost compared to simpler alternatives.\n\n> **Q1.** Could you provide a detailed analysis of the computational costs (e.g., inference time, parameter overhead) introduced by MemEIC's hybrid memory architecture, including the external retrieval and knowledge connector components? How do these costs compare to simpler methods (e.g., enhanced internal memory or external retrieval alone)?\n\n**A.** Thank you for raising a great question. While our approach demonstrates strong editing capabilities at the research level, ensuring practicality in real-world applications requires that we keep computational costs under control. To that end, we evaluated the computational overhead of our method versus those of the baselines under identical conditions (a single A100 80 GB, LLaVA 1.5 7B):\n\\\n\\\n**Table.** Computational Cost: Editable Parameters, Edit time, Inference time, Average Performance(across gap[0,10,20,50,100])\n| Method | Editable Params (M) | Edit Time (sec) \\[visual/textual] | Inference Time (sec) | Avg. Performance (CompRel, %) |\n| - | - | - | -| - |\n| FT-LLM | 90.1 (1.28%) | 6.4 / 1.0 | 0.6| 58.84 |\n| LoRA   | 7.7 (0.11%) | 15.6 / 5.4  | 0.7 | 61.62 |\n| MEND   | 270.3 (3.84%) | 0.04 / 0.01 | 0.6 | 2.32 |\n| SERAC  | - | **0.003 / 0.002** | 0.9 | 40.76 |\n| MemEIC | **7.7 (0.11%)** | 11.8 / 1.7 | 1.4 | **80.56** |\n\n- **Editable Parameters**:  Our method requires significantly fewer editable parameters than most methods — 7.7 M (≈ 0.11 % of the full model). Mem-E incurs no extra editable parameters at all, since it simply stacks new knowledge in external memory. Mem-I restricts updates to two small, brain-inspired LoRA modules (one per FFN “hemisphere”), so it only ever edits those lightweight parameters. Despite this extreme parameter efficiency, Mem-I achieves outstanding retention: Mem-I exhibits exceptional retention—maintaining performance even as the interval between updates grows (Gap: 0, 10, 20, 50, 100 )—and achieves an average Compositional Reliability of 80.56%: an 18.9% improvement over the runner-up method.\n- **Edit Time** :  We measured editing latency separately for visual and textual knowledge. As expected, editing visual tokens takes longer than editing text tokens—visual embeddings are simply heavier. Nevertheless, our method still runs faster than LoRA, the previous state of the art. Naive LoRA must update all 16 ranks within a single module, whereas Mem-I propagates gradients through only one out of two 8-rank modules, the visual adapter for visual edits or text adapter for textual updates. This design reduces runtime, despite using the same total number of ranks. Importantly, our ablation study (§4.4) shows that naively reducing LoRA’s rank to 8 in an attempt to reduce edit time causes representation collapse and significantly degrades editing accuracy. In contrast, Mem-I preserves representation quality while cutting edit time. Other fast editors (e.g., MEND, SERAC) trade speed for severely degraded compositional reasoning—unacceptable for real use.\n- **Inference Time** :  Among internal-memory approaches, LoRA remains fastest at around 0.7 s per query. External-memory methods like SERAC are slower (≈ 0.9 s) due to classifier overhead and query matching, and they underperform at knowledge internalization. Our model strikes a trade-off between latency and performance: it adds only modest latency while delivering far superior continual and compositional editing gains.\n- **Ablation of Inference Overhead** :  As requested in Q1, we also isolated the overhead of each Mem-EIC component during inference. Starting from Mem-E alone, query decomposition costs ≈ 0.5 s and retrieval ≈ 0.2 s. Adding Mem-I increases latency by 0.6 s, and the Knowledge Connector adds another 0.1 s. Note that Mem-E’s retrieval is lightweight (0.2 s); the bulk of its cost is query decomposition, which could be greatly reduced by replacing GPT-4o with a smaller model (e.g. FLAN-T5 Large or TinyLLaMA with LoRA)—we estimate 0.5 s could drop to 0.1 s. Moreover, the Knowledge Connector only activates on compositional queries, so average latency in practice will be lower.\n\n **Table.** Ablation study of MemEIC's Inference time\n| Component |  Δ inference-time |\n| - | - |\n| **Mem-E** (External Memory) | 0.7 s |\n| + **Mem-I** (Internal Memory)    | **+0.6 s** |\n| + **Knowledge Connector** | **+0.1 s** |\n\n\nIn conclusion, computational cost is important, but the ultimate measure of an editing method is **'how long and how accurately it retains the edited information'**. Even if a method like MEND edits very quickly, catastrophic forgetting renders its outputs unreliable across multiple inferences. Our Mem-EIC framework, by contrast, achieves fast edits with minimal parameter updates while preserving accuracy and compositional reliability over time.\n\n---\n\n**Q. Other Benchmarks**\n> **W2.** Limited Benchmarks: The reliance on the CCKEB benchmark raises concerns about generalizability. Validation on new, diverse benchmarks (e.g., MC-MKE) would strengthen the experimental scope.\n\n> **Q2.** Could you provide justification for excluding other benchmarks like MC-MKE (focused on modality consistency) or extend the experiments to broader settings to evaluate generalization?\n\n**A.** Reviewer **Cssd**, thank you for pointing us to MC-MKE. That work appears in the ACL Findings track, which is currently underway, so its underlying data have understandably not yet been released. Even if those data was available, MC-MKE performs only a single edit on one modality—either textual or visual knowledge—and evaluates it immediately, whereas our benchmark continuously updates knowledge over time, deliberately inserting gaps between edits and evaluation to test true lifelong editing capability. Furthermore, unlike MC-MKE, we alternate between textual and visual edits and introduce a new metric, evaluation protocol, and dataset to assess whether combinations of those edits remain coherent. To the best of our knowledge, this is the first benchmark for continual, compositional knowledge editing.\n\n---\n\n**Q. MemEIC vs Baselines**\n> **W3.** Comparison Fairness: MemEIC combines internal and external memory techniques, making it unclear whether the observed gains are due to the novel design itself or the increased resource usage. A clearer analysis comparing hybrid models to standalone internal/external methods would enhance credibility.\n\n> **Q3.** How does MemEIC’s hybrid approach compare to some simple baselines that mix existing internal and external memory architectures? Some discussion or results could be helpful to enhance the paper.\n\n**A.** We thank the reviewer for raising the important point of how MemEIC’s hybrid external-internal approach compares to simpler combinations of internal and external memory.\n\nFirst, we would like to clarify SERAC’s role in this landscape. Although SERAC stores edit samples in an edit memory and is thus classified as an external-memory method, during training it also learns a separate counterfactual model in addition to the base model, updating additional parameters in the process. Despite this hybrid-style mechanism, our experiments in §4.2 (Table 1) show that MemEIC outperforms SERAC across all metrics.\n\n\nCrucially, this advantage is not due to simply adopting a hybrid memory structure. Rather, it comes from systematically improving each component(external and internal) in isolation. Our ablation studies demonstrate that both Mem-E (external) and Mem-I (internal) modules individually surpass naive editing baselines in terms of accuracy.\n\nTo make these structural benefits explicit:\n- Visual Editing (Mem-E, §4.3). By more effectively leveraging rich visual context during retrieval, our enhanced Mem-E module achieves substantially higher visual editing accuracy than previous external-memory methods.\n- Compositional Reliability (Knowledge Connector, §4.5). Our self-attention-based Knowledge Connector plays a pivotal role in reconciling information from internal parameters and external retrieval. This mechanism is especially beneficial for compositional editing scenarios, driving the notable gains we observe on our Compositional Reliability metric.\n\nWe appreciate the reviewer’s suggestion and will add an explicit discussion of these distinctions in the revised manuscript."}, {"review_id": "QUxsVSxUa6", "reviewer": "Reviewer_5dMc", "summary": "The paper introduces CCKEB, a new continual and compositional editing benchmark that pairs visual relabeling with matching textual fact updates and tests models on single edits, long edit sequences, and dual-modal queries; to meet these challenges, the authors propose MemEIC, which keeps vision and text edits in separate external memories and modality-specific LoRA adapters, then selectively fuses them via a learned “Knowledge Connector” only when a compositional query requires it—resulting in near-perfect retention of early edits after hundreds of updates and over 97% reliability on compositional tasks, far surpassing both external-only and internal-only baselines.", "questions": "1. Could the aurthors expand your comparisons to include more recent multimodal editing methods—such as VisEdit[1] or other state-of-the-art VLM editing method—beyond MEND and SERAC? Demonstrating clear gains over these stronger baselines would substantively strengthen the paper’s claims and could help raise the score.\n\n[1]Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEdit\n\n2. Can you provide a detailed computational cost analysis of MemEIC relative to your baselines, covering training and inference time, GPU memory usage, and FLOPs for a fixed number of edits?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Key strengths include:\n\n1. CCKEB is the first to test both continual updating and compositional reasoning across vision and language.\n2. MemEIC’s dual external memories, modality-specific LoRA adapters, and selective “Knowledge Connector” fusion effectively prevent cross-modal interference.\n3. near-perfect retention after hundreds of edits and >97 % reliability on compositional queries across two LVLMs.\n\nMain weaknesses are:\n\n1. baseline method only MEND and SERAC are compared, omitting latest editing methods of VLM like visedit\n2. the paper is missing a discussion of the computational cost of running MemEIC (vs other baselines), how expensive is running MemEIC?", "was_revised": true, "final_justification": "My concerns have been adequately addressed, and I have decided to increase my rating by one.", "rebuttal": "We thank the reviewer for recognizing our contributions and the strong performance of MemEIC. Your suggestions on baseline comparisons and computational cost analysis are valuable for strengthening our evaluation.\n\n---\n\n**Q.  Additional SOTA Baseline Comparison**\n\n**A.** We thank the reviewer for this valuable suggestion. We share your goal of further strengthening our paper by comparing MemEIC against stronger, more recent baselines. However, existing datasets and therefore methods that support continual editing of both visual and textual knowledge are extremely limited (e.g., VLKEB, CCKEB). Additionally, some methods such as KE [1] and IKE [2] require access to editing data at test time, which reduces their practical applicability (see §4.2).\n\n**VisEdit** [3] is indeed an excellent work, and we have studied it closely. However, VisEdit focuses primarily on attribution analysis of visual representations and does not consider compositional interactions between modalities. It is designed for single-edit settings (e.g., MMEdit ), without addressing continual edit scenarios.\n\nAs a stronger \"lifelong\" editing baseline, we therefore implemented **WISE** [4] within our CCKEB framework and evaluated it alongside FT, LoRA, MEND, SERAC, and our method on both LLaVA 1.5 7B and MiniGPT-4:\n\n**Table 1.** Average performance of CCKEB on LLaVA‑1.5\n| Method | Vis Rel | Text Rel | Comp Rel | AVG |\n|-|-|-|-|-|\n| FT(LLM) | 85.66 | 81.36 | 58.84 | 75.29 |\n| LoRA | 77.22 | 75.22 | 61.62 | 71.35 |\n| MEND | 4.22 | 4.43 | 2.32 | 3.66 |\n| SERAC | 63.17 | 86.94 | 40.76 | 63.62 |\n| WISE [4] | 81.99 | **94.16** | 48.21 | 74.79 |\n| **MemEIC (Ours)** | **98.93** | 92.48 | **80.56** | **90.66** |\n\n**Table 2.** Average performance of CCKEB on MiniGPT-4\n| Method | Vis Rel | Text Rel | Comp Rel | AVG |\n|-|-|-|-|-|\n| FT(LLM) | 76.42 | 87.23 | 41.75 | 68.47 |\n| LoRA | 77.68 | 74.39 | 58.99 | 70.35 |\n| MEND | 6.82 | 6.10 | 5.39 | 6.10 |\n| SERAC | 44.84 | 43.31 | 40.73 | 42.96 |\n| WISE [4] | 66.12 | 92.07 | 43.9 | 67.36 |\n| **MemEIC (Ours)** | **96.67** | **92.89** | **74.53** | **88.03** |\n\n> **WISE [4]** divides each FFN layer into a \"main\" and a \"side\" memory bank, explicitly separating edited knowledge from unedited knowledge to enable robust lifelong editing.\n\n> We provide detailed results for sequential gaps [0, 10, 20, 50, 100] in Tables  5 and  6 below.\n\nDespite WISE's temporal robustness (maintaining consistent performance across editing gaps), **MemEIC substantially outperforms WISE across nearly all metrics**:\n- **Visual Reliability**: MemEIC achieves +16.94% improvement on LLaVA 1.5, demonstrating superior visual knowledge editing capabilities\n- **Textual Reliability**: Results are comparable - WISE slightly outperforms on LLaVA 1.5 (+1.68%), while MemEIC shows better performance on MiniGPT-4 (+0.82%)\n- **Most critically, Compositional Reliability**: MemEIC achieves 80.56% vs WISE's 48.21% on LLaVA 1.5 - a substantial +32.35% improvement\n\nThis dramatic difference in compositional reasoning reveals a fundamental limitation of current knowledge editing approaches. While recent methods like WISE [4], GRACE [5], and MEMIT [6] have made significant progress in preventing knowledge conflicts between new and old knowledge through various separation mechanisms (discrete key-value adaptors, rank-one weight modifications, memory bank separation), these approaches inadvertently **limit the interaction between edited facts** that is essential for compositional reasoning tasks.\n\nOur analysis suggests that the field's emphasis on conflict avoidance may inadvertently constrain the practical applicability of knowledge editing systems in real-world scenarios that require reasoning over interconnected knowledge. MemEIC addresses this limitation by enabling **controlled interaction** between edited knowledge while maintaining editing reliability - a capability crucial for practical deployment of knowledge editing systems.\n\nWe will update the main results section (§4.1) of our paper to include WISE results. Thanks to the reviewer's suggestion to compare against a stronger SOTA baseline, we believe our paper's novelty and impact will be significantly reinforced. We will also explore extending our experimental baselines in future work to include other recent editors such as VisEdit and LTE [7].\n\nThank you again for this excellent feedback!\n\n---\n\n**Q. Computational Cost**\n\n**A.** Your question about computational cost analysis is particularly important for practical applications. Below, we provide a comprehensive breakdown of MemEIC's computational efficiency relative to baseline methods, evaluated under identical conditions (a single A100 80 GB, LLaVA 1.5 7B):\n\n**Table 3.** Computational Cost: Edit Parameters, Edit time, Inference time, Average Performance(in gap[0,10,20,50,100])\n| Method | Editable Params (M) | Edit Time (sec) \\[visual/textual] | Inference Time (sec) | Avg. Performance (CompRel, %) |\n| - | - | - | -| - |\n| FT-LLM | 90.1 (1.28%) | 6.4 / 1.0 | 0.6| 58.84 |\n| LoRA | 7.7 (0.11%) | 15.6 / 5.4  | 0.7 | 61.62 |\n| MEND | 270.3 (3.84%) | 0.04 / 0.01 | 0.6 | 2.32 |\n| SERAC  | - | **0.003 / 0.002** | 0.9 | 40.76 |\n| MemEIC | **7.7 (0.11%)** | 11.8 / 1.7 | 1.4 | **80.56** |\n\n- **Parameter Efficiency**:  MemEIC demonstrates strong parameter efficiency, updating only 7.7M parameters (~0.11% of the model) during edit. This is enabled by a dual-component design: Mem-E uses an external memory that requires no editable parameters, while Mem-I applies two compact, brain-inspired LoRA modules to separate FFN hemispheres. This strategic design yields 80.56% compositional reliability, an 18.9% gain over the next best baseline.\n\n- **Edit Time Performance**:  Our analysis highlights modality-specific differences in editing latency. Editing latency varies by modality, with visual edits requiring more resources. MemEIC remains competitive with LoRA by propagating gradients through only one out of two 8-rank modules per modality instead of a single 16-rank module. Ablation studies show that reducing LoRA rank to 8 without modality separation leads to collapse (§4.4). Faster methods like MEND and SERAC sacrifice accuracy for speed, limiting real-world applicability.\n\n- **Inference Latency**: MemEIC introduces moderate inference latency (1.4s) versus LoRA (0.7s). External-memory methods like SERAC show intermediate latency (0.9s) but poor knowledge retention. MemEIC balances performance and speed, offering strong continual and compositional editing with practical overhead.\n\n**Table 4.** Ablation study of MemEIC's Inference time\n| Component |  Δ inference-time |\n| - | - |\n| **Mem-E** (External Memory) | 0.7 s  |\n| + **Mem-I** (Internal Memory)  | **+0.6 s** |\n| + **Knowledge Connector**  | **+0.1 s**  |\n\n- **Component-wise Overhead Analysis**: Breaking down inference time, Mem-E contributes 0.7s (0.5s for query decomposition, 0.2s for retrieval), Mem-I adds 0.6s, and the Knowledge Connector incurs minimal overhead (0.1s). Notably, the connector activates only for compositional queries, reducing average latency in practice. The decomposition bottleneck can be mitigated by replacing GPT-4o with lighter models (e.g., FLAN-T5 Large, TinyLLaMA with LoRA), potentially lowering it to ~0.1s.\n\nWhile computational efficiency matters, the fundamental question for any editing method remains: \"How reliably and accurately does it retain edited information over time?\" Methods like MEND may achieve rapid editing, but catastrophic forgetting undermines their practical utility across multiple inferences. MemEIC's framework addresses this critical limitation by combining fast editing with minimal parameter updates while ensuring robust accuracy and compositional reliability throughout extended usage.\n\nWe hope this detailed computational analysis clearly addresses your concerns about MemEIC's practical viability. Your emphasis on computational practicality is essential, helping to bridge research innovations with real-world deployments.\n\n*References*\n- [1] Cao et al., \"Editing factual knowledge in language models,\" ACL 2021.\n- [2] Zheng et al., \"Can we edit factual knowledge by in-context learning?,\" ACL 2023.\n- [3] Chen et al., \"Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEdit,\" AAAI 2025.\n- [4] Wang et al., \"WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models,\" NeurIPS 2024.\n- [5] Hartvigsen et al., \"Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors,\" NeurIPS 2023.\n- [6] Meng et al., \"Mass Editing Memory in a Transformer,\" ICLR 2023. \n- [7] Jiang et al., “Learning to Edit: Aligning LLMs with Knowledge Editing”, ACL 2024\n\n\n---\nTo make it easier to follow, we have appended additional experimental results matching the table.\n\n**Table 5.** Detailed Performance of CCKEB on LLaVA‑1.5 (WISE vs Ours)\n| Method | Gap | Vis Rel | TextRel | Comp Rel |\n|-|-|-|-|-|\n| **WISE** | 0 | 81.17 | 97.69 | 49.47 |\n| | 10 | 82.06 | 96.95 | 48.10 |\n| | 20 | 81.60 | 94.70 | 47.47 |\n| | 50 | 83.09 | 93.76 | 47.83 |\n| | 100 | 82.04 | 87.72 | 48.16 |\n| **Avg** | | 81.99 | **94.16** | 48.21 |\n| | | | | |\n| **OURS** | 0 | 99.66 | 99.94 | 85.16 |\n| | 10 | 98.86 | 93.36 | 80.88 |\n| | 20 | 98.91 | 90.73 | 80.35 |\n| | 50 | 98.72 | 89.61 | 79.03 |\n| | 100 | 98.48 | 88.77 | 77.36 |\n| **Avg** | | **98.93** | 92.48 | **80.56** |\n\n**Table 6.** Detailed Performance of CCKEB on Minigpt-4 (WISE vs OURS)\n| Method | Gap | Vis Rel | Text Rel | Comp Rel |\n|-|-|-|-|-|\n| **WISE** | 0 | 81.52 | 89.27 | 44.42 |\n| | 10 | 65.71 | 91.53 | 44.22 |\n| | 20 | 61.76 | 92.06 | 43.51 |\n| | 50 | 60.94 | 93.15 | 43.77 |\n| | 100 | 60.66 | 94.35 | 43.58 |\n| **Avg** | | 66.12 | 92.07 | 43.90 |\n| | | | | |\n| **OURS** | 0 | 99.52 | 99.85 | 78.05 |\n| | 10 | 96.40 | 93.34 | 76.25 |\n| | 20 | 95.67 | 91.74 | 74.78 |\n| | 50 | 95.35 | 90.87 | 72.61 |\n| | 100 | 96.42 | 88.64 | 70.96 |\n| **Avg** | | **96.67** | **92.89** | **74.53** |"}, {"review_id": "qbJhffhiOE", "reviewer": "Reviewer_qLzo", "summary": "The paper addresses the critical need to continually update large vision-language models (LVLMs), which often struggle to integrate changes across both visual and textual information effectively. Existing methods typically focus on editing just one modality at a time, overlooking the interconnected nature of visual and textual knowledge. To resolve this, the authors propose MemEIC, a novel approach designed specifically for continual and compositional knowledge editing. MemEIC uniquely combines external memory retrieval with internal updates through separate adapters for visual and textual modalities, inspired by the human brain’s functional organization. A central innovation is the brain-inspired \"knowledge connector,\" enabling the model to selectively integrate visual and textual knowledge during compositional reasoning tasks. Additionally, the authors introduce the CCKEB benchmark, a new comprehensive dataset and evaluation metric specifically tailored for assessing models' capabilities in handling continuous and combined visual-textual edits. Experiments confirm that MemEIC significantly outperforms prior methods, demonstrating robustness in complex multimodal scenarios and effectively preserving previously learned knowledge.", "questions": "Computational overhead: Could you clarify the practical computational cost of MemEIC? Providing runtime comparisons or complexity analysis versus baseline methods would help.\nRobustness to retrieval errors: How does performance degrade when external memory retrieval accuracy is lower? Additional experiments or insights would strengthen confidence in real-world use.\nReal-world generalization: Can you discuss or provide examples demonstrating MemEIC’s effectiveness beyond the synthetic benchmark scenarios? Real-world cases or qualitative results would be beneficial.\nSimplifying methodological descriptions: Could you offer a simplified, intuitive explanation of the dual adapters and knowledge connector? This would greatly improve clarity.\nQualitative examples: Including more qualitative examples of successful and unsuccessful compositional edits would help clearly illustrate the practical strengths and weaknesses of your approach.", "limitations": "The authors have not explicitly discussed limitations and potential negative societal impacts.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n* Quality: Extensive experiments clearly show the effectiveness of the proposed MemEIC method against strong baselines. Ablation studies effectively isolate the contributions of key components.\n* Clarity: Well-organized and easy to follow. Figures and problem definitions significantly enhance readability.\n* Significance: Addresses an important and understudied challenge in multimodal continual knowledge editing. Introducing a new benchmark (CCKEB) and metric (Compositional Reliability) provides valuable tools for future research.\n* Originality: Innovatively integrates external memory and modality-specific adapters inspired by human brain structures, which is novel compared to previous single-modality approaches.\n\nWeaknesses:\n* Quality: Missing a discussion on computational overhead, which raises concerns about practical scalability.\n* Clarity: Methodological sections can sometimes be overly complex; more qualitative examples would help illustrate the practical effectiveness clearly.\n* Significance: Evaluations primarily on synthetic benchmarks may limit real-world generalizability. The absence of user studies or real-world validation weakens claims about practicality.\n* Originality: Some components (e.g., LoRA adapters, external retrieval) are incremental adaptations, with originality mostly stemming from their novel combination.\n\nOverall, the paper is strong and makes valuable contributions, but addressing practical considerations and incorporating real-world validation would substantially strengthen its impact.", "was_revised": false, "final_justification": "", "rebuttal": "We are truly delighted that you recognized the value of our work with the comment:  \n> *\"Significance: provides valuable tools for future research.\"*\n\nOur research proposes, for the first time, a benchmark for continual and compositional knowledge editing that considers realistic scenarios. True to the title of our paper, we aimed to create — *“A Step Toward”* — that could serve as a meaningful path for researchers exploring the areas of multimodality and knowledge editing. \n\n**Q.** Computational Cost\n> W1. Quality: Missing a discussion on computational overhead, which raises concerns about practical scalability.\n>\n> Q1. Computational overhead:Could you clarify the practical computational cost of MemEIC? Providing runtime comparisons or complexity analysis versus baseline methods would help. \n\nThank you for this important question. Practical scalability is indeed crucial for real-world deployment. We provide a comprehensive computational analysis comparing MemEIC against all baselines under identical conditions (single A100 80GB, LLaVA 1.5 7B).\n\n\n**Table 1. Computational Overhead Comparison**\n| Method | Editable Params (M) | Edit Time (sec) [visual/textual] | Inference Time (sec) | Avg. Performance (CompRel, %) |\n|--------|--------------------|---------------------------------|---------------------|-------------------------------|\n| FT-LLM | 90.1 (1.28%) | 6.4 / 1.0 | 0.6 | 58.84 |\n| LoRA   | 7.7 (0.11%) | 15.6 / 5.4 | 0.7 | 61.62 |\n| MEND   | 270.3 (3.84%) | 0.04 / 0.01 | 0.6 | 2.32 |\n| SERAC  | 90.1 (1.28%) | 0.003 / 0.002 | 0.9 | 40.76 |\n| **MemEIC** | **7.7 (0.11%)** | **11.8 / 1.7** | **1.4** | **80.56** |\n\n**Key Findings**.\n\n- **Parameter Efficiency**: MemEIC requires only 7.7M editable parameters (0.11% of full model)—matching the most efficient baseline, i.e. LoRA, while achieving +18.9% better compositional reliability.\n\n- **Edit Time**: Our method outperforms naive LoRA, which must update all 16 ranks within a single module, whereas Mem-I propagates gradients through only one out of two 8-rank modules, the visual adapter for visual edits or text adapter for textual updates. This design reduces runtime, despite using the same total number of ranks.\n\n- **Inference Overhead**: MemEIC adds modest latency (1.4s vs 0.6-0.9s baselines) while delivering superior continual learning capabilities.\n\n**Detailed Overhead Analysis.**\n\n**Table 2. MemEIC Component-wise Inference Time**\n| Component | Δ Inference Time |\n|-----------|------------------|\n| Mem-E (External Memory) | 0.7s |\n| + Mem-I (Internal Memory) | +0.6s |\n| + Knowledge Connector | +0.1s |\n\nThe overhead breakdown reveals optimization opportunities: Mem-E's cost is made up of query decomposition ≈ 0.5 s and retrieval ≈ 0.2 s. Query decomposition could be reduced to ~0.1s by replacing GPT-4o with efficient alternatives (FLAN-T5 Large, TinyLLaMA+LoRA).\n\n**Table 3. External Memory Cache Size Scalability Analysis**\n| Cache Size | Avg Time (ms) ±σ | Slowdown | Overhead |\n|------------|------------------|----------|----------|\n| 1 | 857.75 ± 0.15 | 1.00× | +0% |\n| 10 | 858.00 ± 0.62 | 1.01× | +1% |\n| 50 | 868.76 ± 0.92 | 1.01× | +1% |\n| 100 | 892.00 ± 0.69 | 1.04× | +4% |\n| 200 | 928.20 ± 0.94 | 1.07× | +7% |\n| 500 | 1067.56 ± 0.73 | 1.22× | +22% |\n\nWe analyzed how external memory cache size affects MemEIC's retrieval time. The results show **minimal overhead growth** even as cache size increases significantly—from 1% overhead at 50 entries to only 7% at 200 entries. This growth remains manageable and is offset by the substantial performance gains in knowledge retention and compositional reasoning.\n\n\n**Conclusion.**\n\nThe ultimate measure of an editing method is sustained accuracy over time. While ultra-fast methods like MEND edit in milliseconds, catastrophic forgetting renders them unreliable after multiple inferences. MemEIC achieves the optimal balance: competitive edit speed with minimal parameters while maintaining compositional reliability—essential for practical deployment where edited knowledge must persist accurately across numerous queries.\n\n\n---\n**Q.** Robustness to Retrieval Errors\n> Q2. Robustness to retrieval errors:How does performance degrade when external memory retrieval accuracy is lower? Additional experiments or insights would strengthen confidence in real-world use. \n\n**A.** You've raised a critical concern. Real-world retrieval systems rarely achieve perfect accuracy due to noisy queries, incomplete knowledge bases, and computational constraints. Understanding MemEIC's resilience under these conditions is essential for production adoption.\n\nWe evaluated robustness by simulating realistic retrieval failures, deliberately degrading accuracy to 50% - a conservative estimate for challenging deployment scenarios. Our analysis compared two key scenarios:\n- **Ideal Training, Realistic Testing**: Perfect retrieval (100%) during training, degraded retrieval (50%) during deployment\n- **Realistic Training and Testing**: Noisy conditions (50%) throughout both phases\n\nWhen models trained under perfect conditions face degraded retrieval during testing, performance drops substantially: 14.5% for LLaVA1.5 and 15.9% for MiniGPT-4. This represents a significant deployment risk for systems optimized under idealized laboratory conditions.\n\nHowever, training the knowledge connector under realistic noisy conditions from the start significantly improves robustness. This approach reduces performance degradation to 12.0% for LLaVA1.5 (2.5% improvement) and 11.4% for MiniGPT-4 (4.5% improvement), demonstrating the connector's ability to learn effective knowledge recovery strategies.\n\nFrom a deployment perspective, this suggests a clear best practice: train MemEIC under conditions matching expected real-world performance rather than idealized laboratory settings.\n\nThese experiments (detailed in Appendix E.3.2-E.3.3) provide confidence that MemEIC maintains practical utility in real-world deployments where perfect retrieval is unrealistic.\n\n---\n\n**Q.** Real-world qualitative results\n> W3. Significance: Evaluations primarily on synthetic benchmarks may limit real-world generalizability. The absence of user studies or real-world validation weakens claims about practicality.\n\n> Q3. Real-world generalization: Can you discuss or provide examples demonstrating MemEIC’s effectiveness beyond the synthetic benchmark scenarios? Real-world cases or qualitative results would be beneficial. \n\n> Q5. Qualitative examples:Including more qualitative examples of successful and unsuccessful compositional edits would help clearly illustrate the practical strengths and weaknesses of your approach.\n\n**A.** Thank you for raising this important point regarding real-world generalization. As described in Appendix B, CCKEB is an extended version of the VLKEB benchmark, originally constructed from the Multi-modal Knowledge Graph (MMKG) [1]. To better reflect real-world scenarios, we adopted the same construction protocol as VLKEB and utilized real-world knowledge graphs such as DB15K  and FB15K. In particular, we extracted relations from DB15K and employed maximum weight bipartite matching to select the most semantically similar object pairs. To ensure data quality and realism, all object pairs in the evaluation set were manually verified.  \n\nWe believe these efforts significantly improve the benchmark's alignment with real-world settings. Examples of such real-world compositional edits can be found in Table 7 (Appendix), and successful and unsuccesful model inference cases are illustrated in Figure 8. \n\nThat said, we agree that providing more explicit real-world case studies and qualitative examples would further illustrate MemEIC’s effectiveness in practical settings. Due to rebuttal limitations (e.g., image upload restrictions), we are unable to include visual examples here. However, we plan to incorporate additional real-world compositional edit scenarios and corresponding outputs in the camera-ready version.\n\n---\n\n**Q.** Simplifying Methodological Descriptions\n> W2. Clarity: Methodological sections can sometimes be overly complex; more qualitative examples would help illustrate the practical effectiveness clearly.\n>\n> Q4. Simplifying methodological descriptions:Could you offer a simplified, intuitive explanation of the dual adapters and knowledge connector? This would greatly improve clarity. \n\n**A.** We appreciate the reviewer’s suggestion, which we believe will help us further improve the clarity and accessibility of our work.\nFor now, Figure 2 provides a visual overview of the Knowledge Separation (Query Decomposition and MemE) and Knowledge Integration (MemI and Connector) components. As described in Sections 1 (Introduction) and 3 (Methodology), the Dual-LoRA Adapters are inspired by the functional separation of the left and right hemispheres of the human brain: each adapter independently processes and stores visual and textual modality information. The Knowledge Connector, built on top of LoRA, functions analogously to the corpus callosum, dynamically integrating information from both modalities. Notably, it is only activated in compositional query scenarios that require cross-modal reasoning, thereby enabling effective compositional reasoning across visual and textual knowledge. \n\nTo further support clarity, we plan to include pseudocode for the inference procedure in the appendix of the camera-ready version.\n\n---\n**Limitations.** \n> The authors have not explicitly discussed limitations and potential negative societal impacts.\n> \n**A.** While we would like to clarify that we have addressed these aspects in Appendix F (Sections F.1 for limitations and F.2 for broader impacts), we acknowledge that these sections may have been insufficiently emphasized in the current version, and we will make sure to highlight them more clearly in the camera-ready version.\n\n—\n\n*References*\n- [1] Liu et al., \"MMKG: Multi-modal Knowledge Graphs. In: Hitzler, P., et al. The Semantic Web\". ESWC 2019"}, {"review_id": "MN5vgpsNH0", "reviewer": "Reviewer_3Fzx", "summary": "The paper addresses the challenge of updating large vision-language models (LVLMs) by proposing a novel method called MemEIC for Continual and Compositional Knowledge Editing (CCKE). MemEIC integrates external memory retrieval with internal model editing, using modality-specific adapters and a brain-inspired knowledge connector to facilitate robust compositional reasoning across visual and textual knowledge. The authors introduce a new benchmark, CCKEB, to evaluate the effectiveness of their approach, demonstrating that MemEIC outperforms existing methods in handling complex multimodal knowledge updates.", "questions": "- In table 3, the results are averaged over five times. I am curious about the standard deviations, since the differences are relatively small.\n- In the training stage 2: knowledge connector training, what is the effect of the counterfactual retrievals?\n- Is it possible to apply the proposed framework on visual-only edit tasks, e.g., MMEdit, or other datasets even if they do not fully support compositional updates? What modification should be taken?", "limitations": "It seems that there is no negative societal impact of this work.\nFor the limitations, see the weaknesses above.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "strengths\n\n- The paper introduces MemEIC, a novel framework that combines external memory retrieval with internal model editing, addressing the limitations of existing methods that focus on single-modality editing. This dual approach enhances the model's ability to handle complex, compositional queries.\n- The creation of the CCKEB benchmark is a significant contribution, as it provides a comprehensive framework for evaluating continual and compositional knowledge editing in LVLMs, filling a gap in existing benchmarks, with the help of multi-modal knowledge graph MMKG and GPT-4o.\n- The methodology is well-explained, with clear descriptions of the knowledge separation, external and internal memory mechanisms, and the knowledge connector, facilitating reproducibility and further research.\n- The author provides code for reproducing the results in the supplementary material.\n\nweaknesses\n\n- The evaluation is conducted on moderately sized models, and the performance on larger architectures remains unexplored, which could be a limitation for scaling the approach to more complex models.\n- Some claims in the paper are ambiguous. See the Questions part.\n- Many tables (e.g., Table 3) and plots (e.g., Table 1) lack error bars or confidence intervals, which could affect the interpretation of the results’ statistical significance.", "was_revised": true, "final_justification": "The author cleared all my concerns in the rebuttal. I keep my score.", "rebuttal": "We sincerely thank you for recognizing the significance of our work.\n\nWe are the first to propose a Continual, Compositional Knowledge Editing Benchmark that takes real-world scenarios into account. As our paper's title suggests, we hope this framework will be \"A Step Toward Continual and Compositional Knowledge Editing\" and motivate further research in both the multimodal and knowledge editing communities.\n\n\n---\n\n**Q. Statistical Significance**\n> **W3.** Many tables (e.g., Table 3) and plots (e.g., Table 1) lack error bars or confidence intervals, which could affect the interpretation of the results' statistical significance.\n> \n> **Q1.** In table 3, the results are averaged over five times. I am curious about the standard deviations, since the differences are relatively small.\n\n**A.** Following the reviewer's suggestion, we have updated Table 3 by including the standard deviation (std) and p-values, as shown below.\n\n**Table 3.** Knowledge separation(Mem-I) results for CCKEB validation dataset\n| Model | Rel (Vis) | T‑Gen (Vis) | I‑Gen (Vis) | T‑Loc (Vis) | I‑Loc (Vis) | Rel (Text) | Gen (Text) | Loc (Text) |\n|-|-|-|-|-|-|-|-|-|\n| Mem‑I (Dual‑LoRA, r: 8×2) | 74.73 ± 0.10 | 71.57 ± 0.06 | 74.61 ± 0.08 | 80.93 ± 0.52 | 12.45 ± 0.38 | 77.24 ± 0.10 | 73.73 ± 0.12 | 68.22 ± 0.55 |\n| Mem‑I (Single‑LoRA, r: 16) | 74.60 ± 0.09 | 71.50 ± 0.05 | 74.49 ± 0.07 | 63.16 ± 0.50 | 9.59 ± 0.33 | 74.54 ± 0.11     | 70.49 ± 0.15   | 63.16 ± 0.48   |\n| **Difference (r:16 vs dual)** | –0.13 % | –0.07 %  | –0.12 %  | –17.77 %  | –2.86 %  | –2.70 % | –3.24 % | –5.06 %        |\n| Mem‑I (Single‑LoRA, r: 8)  | 74.06 ± 0.07    | 71.02 ± 0.06   | 74.08 ± 0.07   | 60.42 ± 0.52   | 7.90 ± 0.43 | 72.12 ± 0.07     | 68.29 ± 0.13   | 60.42 ± 0.60   |\n| **Difference (r:8 vs dual)** | –0.67 % | –0.55 % | –0.53 %  | –20.51 % | –4.55 % | –5.12 %  | –5.44 %  | –7.80 %  |\n\n\n\n\nTo clarify, the values presented in Table 3 were obtained as follows:\n- For each independent run (we did 5 runs total), performance metrics were measured at sequential gaps (0, 10, 20, 50, 100).\n- In each run, we averaged the five gap‐specific performances to obtain a single gap-averaged value.\n- Across the 5 runs, the standard deviation of those gap-averaged values was ∼ 0.1, indicating low run-to-run variability.\n- Table 3 then reports, for each metric, the mean ± std computed over these five gap-averaged values.\n\nOur additional analysis (moved to §4.3) confirms that, although mean differences between single lora and dual lora in some visual-edit metrics (rel, t-gen, i-gen) are close to one std, paired t-tests (df = 4) yield p < 0.05, indicating these small improvements are statistically significant. The much larger gains in localized editing metrics (T-Loc +17.77 %, I-Loc +2.86 %) remain the clearest evidence that modality-specific memory separation precisely confines edits to their targets.\n\n\nNevertheless, as explicitly stated in §4.3 of our manuscript, the key message we intend to convey is:\n\n> \"Our results suggest that distinct separation of visual and textual memories during CCKE effectively mitigates representation collapse, enabling precise, localized editing of multimodal knowledge.\"\n\nIn fact, substantial performance improvements were observed in other key metrics:\n- Visual edits: Mem-I (Dual-LoRA) achieved significant gains in T-Loc (+17.77 %) and I-Loc (+2.86 %), confirming that modality-specific separation precisely confines edits to targeted knowledge.\n- Textual edits: Mem-I (Dual-LoRA) consistently outperformed Single-LoRA across all metrics (Rel + 2.7 %, Gen + 3.24 %, Loc + 5.06 %).\n\nWe will incorporate these additional statistics and explanations into the final manuscript.\n\n---\n\n**Q. Effect of the counterfactual retrievals**\n> Q2. In the training stage 2: knowledge connector training, what is the effect of the counterfactual retrievals?\n\n\n**A.** During the training stage, the external and internal memory are trained independently from each other, which means that during internal memory training external memory retrievals are simulated.\nIt would be possible to simulate 100% correct retrievals, but our experiments have shown that this would inhibit the models capability to handle incorrect retrievals at deployment time.\nIn a real-world deployment MemEIC begins by retrieving edited knowledge from the external memory. Any errors at this stage will naturally cascade through the later components.\n\nTo assess robustness under such realistic conditions (see Appendix E.3.2), we compared two settings: one in which both training and testing use perfect retrieval (train@100 %/test@100 %) and one in which training remains perfect but testing is noisy (train@100 %/test@50 %). In LLaVA1.5 this mismatch causes a performance drop of approximately 14.5 %.\n\nTo mitigate this degradation, we then trained the knowledge connector itself under noisy retrieval conditions (train@50 %) so that it learns to reconcile incomplete or incorrect memory lookups. With this noisy‐training strategy, performance degradation at test@50 % falls from 14.5 % to 12.0 % for LLaVA1.5V (a 2.5 % gain) and from 15.9 % to 11.4 % for MiniGPT-4 (a 4.5 % gain). These improvements reflect the connector's intrinsic knowledge‐reconciliation effects, which we further illustrate with counterfactual retrieval failure examples in Appendix E.3.3.\n\n---\n**Q. Applicability to Visual-Only Tasks**\n> **Q3.** Is it possible to apply the proposed framework on visual-only edit tasks, e.g., MMEdit, or other datasets even if they do not fully support compositional updates? What modification should be taken?\n\n**A.** Yes, this is possible. Our proposed **MemEIC framework is designed to be modular**, allowing it to be effectively applied to visual-only edit tasks (e.g., MMEdit, VLKEB), simply excluding the knowledge connector.\n\nWe will explain this in detail by walking through each operational stage of MemEIC.\n\n**1) Training Stage**\n\nThe training of MemEIC consists of two stages.\n- **Stage 1: Training the External Memory (Mem-E):** In this stage, the external memory is trained to retrieve the correct information for a given query. Since MemEIC's external memory is separated into distinct visual and textual units (`M_v` and `M_t`), using a visual-only dataset to train the visual external memory (`M_v`) poses no issue.\n-**Stage 2: Training the Knowledge Connector:** This stage trains the **Knowledge Connector**, which is responsible for fusing information from the visual and textual adapters for compositional reasoning. We train this connector using compositional queries that require both visual and textual knowledge, as illustrated by the prompt structure below.\n```\n# Retrieved knowledge from external memory\nVisual knowledge: [[Image] <Visual Tokens>, [Text] Q: Who is the person in the photo? A: Donald Trump]\nTextual knowledge: [[Text] Q: What position did Trump recently take on? A: 47th president]\n\n# Query requiring compositional reasoning\nQ: What position did this person in the photo recently assume?\nA: [Answer]\n```\nHowever, existing benchmarks like MMEdit and VLKEB lack this type of **compositional (visual+textual) data** by design. Therefore, the Knowledge Connector cannot be effectively trained using these datasets alone.\n\n**2) Edit Stage**\nWhen a user makes an edit, MemEIC updates two types of memory.\n- **External Memory (Mem-E):** New knowledge is simply appended (stacked) into the external memory store.\n- **Internal Memory (Mem-I):** Only the adapter corresponding to the modality of the edit is selectively updated. For example, a visual edit is performed by solving a VQA task to update only the visual adapter (`θ_v`).\n   * `[[Image] <Visual Tokens>, [Text] Q: Who is the person in the photo?] -> A: Donald Trump`\n\nFor a visual-only task, only the visual adapter and visual external memory are updated. \n\n**3) Test (Inference) Stage**\n\nDuring inference, the Knowledge Connector is selectively activated **only when an input query requires both visual and textual elements**. If a query is unimodal, such as for a visual-only task, the connector remains inactive (defaulting to an identity operation) and does not participate in the inference.\nSince evaluation queries in visual-only edit tasks only require visual information, the Knowledge Connector would not be activated.\n\n**Conclusion.** Based on the analysis above, the necessary modification is very straightforward: **The Knowledge Connector module can simply be excluded.**\nThe connector cannot be trained without compositional data, and it remains inactive during inference for visual-only tasks anyway. By removing only the lightweight, LoRA-based Knowledge Connector, the remaining core components of MemEIC (the dual external memory Mem-E and the separated internal adapters Mem-I) can perfectly support visual-only edit tasks. This modularity is a key design feature of our model, allowing for flexible adaptation to various editing scenarios.\n\n\n---\n**Q. Scaling to Larger Models**\n> **W1.** The evaluation is conducted on moderately sized models, and the performance on larger architectures remains unexplored, which could be a limitation for scaling the approach to more complex models.\n\n**A.** We thank the reviewer for pointing out this important consideration. As noted in Appendix F.1, our current experiments are conducted on mid-sized LVLMs.\n\nHowever, the key components of MemEIC—dual external memories, modality-specific LoRA adapters, and the Knowledge Connector—are structurally scale-invariant and designed to be modular. In particular, the use of LoRA-based adapters supports efficient scaling, as their parameter- and compute-efficiency becomes increasingly advantageous in larger models.\n\nWe plan to explore this direction in future work by extending our evaluation to larger architectures  (e.g., 13B and beyond) to further validate the scalability and robustness of MemEIC across a broader range of model sizes and complexities."}]

Author rebuttal:

Meta-review: The paper proposes MemEIC, a framework for continual and compositional knowledge editing in LVLMs. It integrates external memory retrieval and modality-specific LoRA adapters with a brain-inspired “knowledge connector” that selectively fuses cross-modal knowledge. Also the authors introduce the CCKEB benchmark and a new metric to evaluate both continual and compositional editing. 

The strengths highlighted by reviewers include the novelty of targeting multimodal continual editing, the hybrid architecture that avoids cross-modal interference, and the benchmark/metric contribution that provides tools for future research. Empirical results are strong and supported by ablations, while the paper is generally well-structured and clear.

The weaknesses noted include limited evaluation breadth (primarily on CCKEB, without stronger baselines like VisEdit or other recent editing methods), insufficient discussion of computational cost and scalability, and some ambiguity in claims and result reporting. Concerns about generalizability to real-world or larger LVLMs remain.

During the rebuttal, authors clarified details of methodology, addressed dataset and comparison questions, which reassured some reviewers though not all concerns were fully resolved. Overall, I believe the paper is timely, makes a contribution to the field.
Decision: Accept (poster)