Title: Reliable Lifelong Multimodal Editing: Conflict-Aware Retrieval Meets Multi-Level Guidance
Venue: NeurIPS 2025
Accepted: yes
Keywords: Knowledge editing

Multimodal understanding

Retrieval-augmented
Abstract: The dynamic nature of real-world information demands efficient knowledge editing in multimodal large language models (MLLMs) to ensure continuous knowledge updates. However, existing methods often struggle with precise matching in large-scale knowledge retrieval and lack multi-level guidance for coordinated editing, leading to less reliable outcomes. To tackle these challenges, we propose CARML, a novel retrieval-augmented editing framework that integrates conflict-aware dynamic retrieval with multi-level implicit and explicit guidance for reliable lifelong multimodal editing. Specifically, CARML introduces intra-modal uncertainty and inter-modal conflict quantification to dynamically integrate multi-channel retrieval results, so as to pinpoint the most relevant knowledge to the incoming edit samples. Afterwards, an edit scope classifier discerns whether the edit sample semantically aligns with the edit scope of the retrieved knowledge. If deemed in-scope, CARML refines the retrieved knowledge into information-rich continuous prompt prefixes, serving as the implicit knowledge guide. These prefixes not only include static knowledge prompt that capture key textual semantics but also incorporate token-level, context-aware dynamic prompt to explore fine-grained cross-modal associations between the edit sample and retrieved knowledge. To further enhance reliability, CARML incorporates a "hard correction" mechanism, leveraging explicit label knowledge to adjust the model’s output logits. Extensive experiments across multiple MLLMs and datasets indicate the superior performance of CARML in lifelong multimodal editing scenarios.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents CARML, a retrieval-augmented framework for lifelong multimodal knowledge editing in MLLMs. It tackles two key challenges: (1) precise retrieval from an ever-growing edit memory by quantifying both intra-modal uncertainty and inter-modal conflicts via a multi-channel softmax+Jaccard-based fusion; (2) reliable editing through a multi-level guidance strategy—implicit continuous prompts (static + context-aware dynamic) and an explicit logits-enhancement (“hard correction”) mechanism. An edit scope classifier further ensures locality by filtering out unrelated edits. Experiments on three benchmarks (E-VQA, E-IC, VLKEB) and two MLLMs demonstrate that CARML maintains near-100% reliability and generality even after 1,000 sequential edits, significantly outperforming baselines\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The combination of multi-channel retrieval, uncertainty/conflict modeling, and multi-level guidance (prompts + logits) is comprehensive and thoughtful.\n\n2. The paper identifies and tackles scalability, accuracy, and edit locality issues that are relevant in real-world settings.\n\n3. Results are reported across multiple datasets (E-VQA, E-IC, VLKEB) and strong baselines, showing consistent improvements.\n\nWeaknesses:\n\n1. The motivation is still confused. (1) The precise retrieval from an ever-growing edit memory. I do not know the concrete challenge in existing methods. Compared with existing methods, why can the proposed retrieval strategy solve this problem? In addition, the system seems complex. How do the benefits compare to the complexity? (2) As for the editing component, what is its advantage for the ever-growing knowledge?\n\n2. While many modules are proposed (such as edit scope classifier, two-level prompts, logits correction), it is unclear which are critical for performance and which are incremental.\n\nQUESTIONS:\nSee the weakness\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

comments: SUMMARY:
In this paper, the authors propose CARML, a novel retrieval-augmented editing framework that integrates conflict-aware dynamic retrieval with multi-level implicit and explicit guidance for reliable lifelong multimodal editing. The authors introduce a conflict-aware dynamic retrieval mechanism that uniquely exploits multi-channel retrieval information for accurate knowledge localization based on quantified uncertainty and conflict degree. Besides, the authors construct an edit scope classifier, which guarantees the locality of editing by effectively filtering irrelevant edit samples. Next, the authors design an innovative multi-level collaborative guidance strategy, integrating implicit (static and dynamic prompts) and explicit (logits enhancement) knowledge injection to achieve reliable editing.

STRENGTHS AND WEAKNESSES:
Strenght
1. The authors introduce CARML, a novel retrieval-augmented editing framework specifically designed to tackle the above challenges. The core innovation of CARML lies in its integration of conflict-aware dynamic retrieval and multi-level implicit and explicit knowledge guidance, enabling reliable lifelong knowledge editing in MLLMs. 
2. The proposed method can achieve good results on MMedit benchmark

Weakness
1. Some recent proposed baselines are not compared, like AlphaEdit
2.  The underlying theory or mechanism are not mentioned， making the proposed method heuristic

QUESTIONS:
NA

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the lifelong multimodal editing problem. To address the problem, the paper proposed a novel retrieval-augmented editing framework designed for reliable lifelong multimodal knowledge editing, which consists of two parts: (1) Conflict-Aware Dynamic Retrieval and (2) Static and Dynamic Prompts. The Conflict-Aware Dynamic Retrieval component aims to locate the most relevant knowledge for the edit sample and determine whether to initiate the subsequent editing process, while the Static and Dynamic Prompts aim to provide both implicit guidance and explicit guidance to the editing process. The experiments across various datasets demonstrate the effectiveness of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The studied problem, lifelong multimodal model editing, is important to the community.\n- The proposed method makes sense to me in general, although I am not in this field.\n- The writing and the structure of the paper are good.\n- The experiments are very comprehensive.\n\nWeakness:\n- Have you compared the runtime efficiency of your pipeline to other baselines? Since you mentioned that 'The dynamic nature of real-world information demands **efficient knowledge editing** in multimodal large language models, I think these experiments will also be good for your paper.\n- The choice of models: BLIP 2-OPT and LLaVA-V1.5 (7B) are kind of outdated. I suggest the authors consider more recent models such as LLaVA-V1.6 and qwen 2.5 vl 7b instruct.\n\nOverall: *I am not familiar with the domain. However, based on my reading, I think the paper is very interesting and complete, and the experiments are strong to support the claims made by the paper. However, there is some probability that I might miss some parts of the paper, or I might not fully understand the paper.*\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the problem of lifelong knowledge editing for mLLMs. The proposed method, CARML, combines conflict-aware dynamic retrieval with multi-level knowledge guidance (static/dynamic prompts plus logits enhancement). The authors show that CARML maintains strong reliability and locality even as the number of edits grows, outperforming various baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper is well-motivated -- lifelong editing for MLLMs is both timely and practically important! It also highlights the key chllanges of growing edit knowledge bases and preserving locality, which is well-aligned with the contributions.\n2. The proposed solution is thorough and demonstrates impressive empirical results across multiple benchmarks and baselines, with ablation studies that justify each module.\n3.  Strong performance even with large numbers of edits (e.g., 1000+) shows good scalability. I especially enjoy reading the experiments and analysis results, they've proactively answered most of the questions I have. \n\nWeaknesses:\nMy biggest concern is that the proposed pipeline is quite complex, and the paper’s description can be hard to follow on a first read. Simplifying the narrative and emphasizing the key innovations more clearly would improve clarity.\n\nIn terms of better highlighting the key innovation within this pipeline: \n1. Among the main steps, I feel the \"Conflict-Aware Dynamic Retrieval\" seems to be more or less standard (e.g., I think people in general would do alignment and matching between different modalities), but I do like the idea of Context-Aware Dynamic Prompt. \n\n2. It remains a bit unclear which modules are absolutely necessary for the gains. While sec 4 shows a clear ablation studies gradually adding more components, an experiment isolating only the core parts (e.g., just context-aware dynamic prompt + logits bias) would make the impact of each clearer.\n\nQUESTIONS:\n1. Could you clarify if the conflict-aware retrieval is truly novel compared to general multi-channel retrieval and fusion techniques? What makes it fundamentally better?\n\n2. What would the authors argue to be the most important component in the pipeline? \n\n3. Given the pipeline complexity, how sensitive is the method to the scope classifier’s threshold or the retrieval fusion parameters? Any tips for tuning these?\n\n4. Could you add a discussion about scaling the knowledge base — how will CARML handle millions of edits in practice?\n\nLIMITATIONS:\nYes — the authors do a reasonable job discussing potential forgetting and the limits of current memory design. Might be good to add more about retrieval scalability in real deployments.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces CARML, a novel retrieval-augmented framework for lifelong knowledge editing in Multimodal Large Language Models (MLLMs). The authors identify two primary challenges in existing methods: imprecise knowledge retrieval and a lack of coordinated, multi-level guidance for the editing process. CARML addresses these issues through two core innovations. The first is a conflict-aware dynamic retrieval mechanism that constructs a multi-modal knowledge base and uses a sophisticated fusion strategy—quantifying both intra-modal uncertainty and inter-modal conflict—to accurately retrieve the most relevant historical edit for a new query. The second is a multi-level guidance strategy that combines implicit guidance (via learnable static and context-aware dynamic prompts) and explicit guidance (via a direct logits enhancement mechanism) to steer the MLLM toward the correct output. The authors conduct extensive experiments on several MLLM editing benchmarks (E-VQA, E-IC, VLKEB) using BLIP-2 and LLaVA models, demonstrating that CARML significantly outperforms a wide range of state-of-the-art baselines, maintaining near-perfect performance even after a thousand sequential edits.\n\nSTRENGTHS AND WEAKNESSES:\nPros:\n1. The paper is well-written, methodologically sound, and addresses a significant and timely problem in the field of MLLMs. The motivation is clear, and the proposed solution is both technically sophisticated and elegant.\n2. Another significant strength is the multi-level knowledge guidance system. The combination of implicit and explicit guidance is compelling. The context-aware dynamic prompt, which uses the edit sample itself as a query to attend to different aspects of the retrieved knowledge, is a particularly clever mechanism for creating fine-grained, tailored instructions for the model. This moves beyond simply prepending static information and allows for a more nuanced interaction between the query and the stored knowledge.\n\nCons:\n1. A key contribution of this paper is the Conflict-Aware Fusion and Re-ranking mechanism. The use of information entropy to quantify the internal uncertainty of retrieval candidates is an interesting approach. However, the idea of using entropy as an uncertainty signal to re-rank retrieved information has been explored in prior knowledge editing work [1]. To better situate your contribution, it would be beneficial to cite this related work and explicitly differentiate your method. Your primary innovation appears to be the novel combination of this intra-modal uncertainty with a measure of inter-modal conflict, which is tailored to the multimodal domain. Clarifying this distinction would strengthen the paper's claim of novelty.\n2. I also have concerns around the \"Explicit Guidance: Output Logits Enhancement\" mechanism. The ablation study (Table 4) reveals that this single component is responsible for the final, dramatic leap in performance, pushing reliability from 83.4% to 99.4%. While undeniably effective, this mechanism feels less like \"guiding\" the model and more like a \"hard correction\" or a post-processing step that forces the model to output the cached answer. This raises questions about the nature of the \"editing\" being performed. Does the model truly learn to integrate the new knowledge into its parametric understanding, or does it simply learn to pass through the retrieved answer when triggered? This could potentially harm the model's generative fluency and its ability to reason about the edited concept. The paper would be strengthened by a more critical discussion of this component and its implications.\n\n[1] Shi, Y., Tan, Q., Wu, X., Zhong, S., Zhou, K., & Liu, N. (2024, October). Retrieval-enhanced knowledge editing in language models for multi-hop question answering. CIKM 2024.\n\nQUESTIONS:\nRegarding the conflict-aware fusion in your retrieval module, could you provide more justification for the specific formula used in Equation 5? Did you experiment with other methods for combining the uncertainty and conflict scores, and if so, how did they perform? How sensitive is the performance to the value of K used for calculating the Jaccard similarity?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-07"}