Title: MemEIC: A Step Toward Continual and Compositional Knowledge Editing
Venue: NeurIPS 2025
Accepted: yes
Keywords: vision and language

multimodal understanding

continual learning
Abstract: The dynamic nature of information necessitates continuously updating large vision-language models (LVLMs).
While recent knowledge editing techniques hint at promising directions, they often focus on editing a single modality (vision or language) in isolation. This prevalent practice neglects the inherent multimodality of LVLMs and the continuous nature of knowledge updates, potentially leading to suboptimal editing outcomes when considering the interplay between modalities and the need for ongoing knowledge refinement. To address these limitations, we propose MemEIC, a novel method for Continual and Compositional Knowledge Editing (CCKE) in LVLMs. MemEIC enables compositional editing of both visual and textual knowledge sequentially. Our approach employs a hybrid external-internal editor featuring a dual external memory for cross-modal evidence retrieval and dual LoRA adapters that facilitate disentangled parameter updates for each modality. A key component is a brain-inspired knowledge connector, activated selectively for compositional reasoning, that integrates information across different modalities. Experiments demonstrate that MemEIC significantly improves performance on complex multimodal questions and effectively preserves prior edits, setting a new benchmark for CCKE in LVLMs. Our project is available at https://github.com/MemEIC/MemEIC.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the challenge of updating large vision-language models (LVLMs) by proposing a novel method called MemEIC for Continual and Compositional Knowledge Editing (CCKE). MemEIC integrates external memory retrieval with internal model editing, using modality-specific adapters and a brain-inspired knowledge connector to facilitate robust compositional reasoning across visual and textual knowledge. The authors introduce a new benchmark, CCKEB, to evaluate the effectiveness of their approach, demonstrating that MemEIC outperforms existing methods in handling complex multimodal knowledge updates.\n\nSTRENGTHS AND WEAKNESSES:\nstrengths\n\n- The paper introduces MemEIC, a novel framework that combines external memory retrieval with internal model editing, addressing the limitations of existing methods that focus on single-modality editing. This dual approach enhances the model's ability to handle complex, compositional queries.\n- The creation of the CCKEB benchmark is a significant contribution, as it provides a comprehensive framework for evaluating continual and compositional knowledge editing in LVLMs, filling a gap in existing benchmarks, with the help of multi-modal knowledge graph MMKG and GPT-4o.\n- The methodology is well-explained, with clear descriptions of the knowledge separation, external and internal memory mechanisms, and the knowledge connector, facilitating reproducibility and further research.\n- The author provides code for reproducing the results in the supplementary material.\n\nweaknesses\n\n- The evaluation is conducted on moderately sized models, and the performance on larger architectures remains unexplored, which could be a limitation for scaling the approach to more complex models.\n- Some claims in the paper are ambiguous. See the Questions part.\n- Many tables (e.g., Table 3) and plots (e.g., Table 1) lack error bars or confidence intervals, which could affect the interpretation of the results’ statistical significance.\n\nQUESTIONS:\n- In table 3, the results are averaged over five times. I am curious about the standard deviations, since the differences are relatively small.\n- In the training stage 2: knowledge connector training, what is the effect of the counterfactual retrievals?\n- Is it possible to apply the proposed framework on visual-only edit tasks, e.g., MMEdit, or other datasets even if they do not fully support compositional updates? What modification should be taken?\n\nLIMITATIONS:\nIt seems that there is no negative societal impact of this work.\nFor the limitations, see the weaknesses above.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the critical need to continually update large vision-language models (LVLMs), which often struggle to integrate changes across both visual and textual information effectively. Existing methods typically focus on editing just one modality at a time, overlooking the interconnected nature of visual and textual knowledge. To resolve this, the authors propose MemEIC, a novel approach designed specifically for continual and compositional knowledge editing. MemEIC uniquely combines external memory retrieval with internal updates through separate adapters for visual and textual modalities, inspired by the human brain’s functional organization. A central innovation is the brain-inspired \"knowledge connector,\" enabling the model to selectively integrate visual and textual knowledge during compositional reasoning tasks. Additionally, the authors introduce the CCKEB benchmark, a new comprehensive dataset and evaluation metric specifically tailored for assessing models' capabilities in handling continuous and combined visual-textual edits. Experiments confirm that MemEIC significantly outperforms prior methods, demonstrating robustness in complex multimodal scenarios and effectively preserving previously learned knowledge.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* Quality: Extensive experiments clearly show the effectiveness of the proposed MemEIC method against strong baselines. Ablation studies effectively isolate the contributions of key components.\n* Clarity: Well-organized and easy to follow. Figures and problem definitions significantly enhance readability.\n* Significance: Addresses an important and understudied challenge in multimodal continual knowledge editing. Introducing a new benchmark (CCKEB) and metric (Compositional Reliability) provides valuable tools for future research.\n* Originality: Innovatively integrates external memory and modality-specific adapters inspired by human brain structures, which is novel compared to previous single-modality approaches.\n\nWeaknesses:\n* Quality: Missing a discussion on computational overhead, which raises concerns about practical scalability.\n* Clarity: Methodological sections can sometimes be overly complex; more qualitative examples would help illustrate the practical effectiveness clearly.\n* Significance: Evaluations primarily on synthetic benchmarks may limit real-world generalizability. The absence of user studies or real-world validation weakens claims about practicality.\n* Originality: Some components (e.g., LoRA adapters, external retrieval) are incremental adaptations, with originality mostly stemming from their novel combination.\n\nOverall, the paper is strong and makes valuable contributions, but addressing practical considerations and incorporating real-world validation would substantially strengthen its impact.\n\nQUESTIONS:\nComputational overhead: Could you clarify the practical computational cost of MemEIC? Providing runtime comparisons or complexity analysis versus baseline methods would help.\nRobustness to retrieval errors: How does performance degrade when external memory retrieval accuracy is lower? Additional experiments or insights would strengthen confidence in real-world use.\nReal-world generalization: Can you discuss or provide examples demonstrating MemEIC’s effectiveness beyond the synthetic benchmark scenarios? Real-world cases or qualitative results would be beneficial.\nSimplifying methodological descriptions: Could you offer a simplified, intuitive explanation of the dual adapters and knowledge connector? This would greatly improve clarity.\nQualitative examples: Including more qualitative examples of successful and unsuccessful compositional edits would help clearly illustrate the practical strengths and weaknesses of your approach.\n\nLIMITATIONS:\nThe authors have not explicitly discussed limitations and potential negative societal impacts.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces CCKEB, a new continual and compositional editing benchmark that pairs visual relabeling with matching textual fact updates and tests models on single edits, long edit sequences, and dual-modal queries; to meet these challenges, the authors propose MemEIC, which keeps vision and text edits in separate external memories and modality-specific LoRA adapters, then selectively fuses them via a learned “Knowledge Connector” only when a compositional query requires it—resulting in near-perfect retention of early edits after hundreds of updates and over 97% reliability on compositional tasks, far surpassing both external-only and internal-only baselines.\n\nSTRENGTHS AND WEAKNESSES:\nKey strengths include:\n\n1. CCKEB is the first to test both continual updating and compositional reasoning across vision and language.\n2. MemEIC’s dual external memories, modality-specific LoRA adapters, and selective “Knowledge Connector” fusion effectively prevent cross-modal interference.\n3. near-perfect retention after hundreds of edits and >97 % reliability on compositional queries across two LVLMs.\n\nMain weaknesses are:\n\n1. baseline method only MEND and SERAC are compared, omitting latest editing methods of VLM like visedit\n2. the paper is missing a discussion of the computational cost of running MemEIC (vs other baselines), how expensive is running MemEIC?\n\nQUESTIONS:\n1. Could the aurthors expand your comparisons to include more recent multimodal editing methods—such as VisEdit[1] or other state-of-the-art VLM editing method—beyond MEND and SERAC? Demonstrating clear gains over these stronger baselines would substantively strengthen the paper’s claims and could help raise the score.\n\n[1]Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEdit\n\n2. Can you provide a detailed computational cost analysis of MemEIC relative to your baselines, covering training and inference time, GPU memory usage, and FLOPs for a fixed number of edits?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces MemEIC, a novel framework for continual and compositional knowledge editing in large vision-language models (LVLMs). The work addresses a significant limitation in existing multimodal knowledge editing approaches, which typically handle visual and textual edits in isolation and fail to support continuous, sequential updates. The authors propose a hybrid external-internal memory architecture that combines modality-specific external retrieval with dual LoRA adapters for internal memory, connected by a knowledge connector that enables cross-modal reasoning. Additionally, they introduce the CCKEB benchmark, extending VLKEB to evaluate both continual editing capabilities and compositional reasoning through a new metric called Compositional Reliability (CompRel). Experiments on LLaVA-1.5 and MiniGPT-4 demonstrate that MemEIC significantly outperforms existing methods, achieving superior performance on complex multimodal questions while effectively preserving prior edits across sequential updates.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n- MemEIC makes a good contribution to multimodal knowledge editing by addressing limitations in single-modality and sequential updates. Its architecture, with modality separation and a dynamic knowledge connector, provides valuable insights for the field. The new framework, benchmark, and metric are good contributions.\n- Comprehensive experiments, including ablation studies, demonstrate consistent performance gains across backbone models (e.g., LLaVA-1.5, MiniGPT-4) and evaluation metrics, particularly excelling in compositional reasoning tasks.\n\nWeakness:\n- Computational Overhead: The paper doesn’t sufficiently address the increased complexity introduced by the hybrid memory architecture. A runtime analysis would clarify whether the performance justifies the additional computational cost compared to simpler alternatives.\n- Limited Benchmarks: The reliance on the CCKEB benchmark raises concerns about generalizability. Validation on new, diverse benchmarks (e.g., MC-MKE) would strengthen the experimental scope.\n    - MC-MKE: A Fine-Grained Multimodal Knowledge Editing Benchmark Emphasizing Modality Consistency (CVPR 2025)\n- Comparison Fairness: MemEIC combines internal and external memory techniques, making it unclear whether the observed gains are due to the novel design itself or the increased resource usage. A clearer analysis comparing hybrid models to standalone internal/external methods would enhance credibility.\n\nQUESTIONS:\n- Could you provide a detailed analysis of the computational costs (e.g., inference time, parameter overhead) introduced by MemEIC's hybrid memory architecture, including the external retrieval and knowledge connector components? How do these costs compare to simpler methods (e.g., enhanced internal memory or external retrieval alone)?\n- Could you provide justification for excluding other benchmarks like MC-MKE (focused on modality consistency) or extend the experiments to broader settings to evaluate generalization?\n- How does MemEIC’s hybrid approach compare to some simple baselines that mix existing internal and external memory architectures? Some discussion or results could be helpful to enhance the paper.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-06"}