Title: From One to Many: Trajectory Invariant Learning for Multimodal Large Language Model Editing
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: knowledge editing, multimodal learning, multimodal large language models

Abstract: Knowledge editing emerges as a crucial technique for efficiently correcting incorrect or outdated knowledge in large language models (LLM). Existing editing methods for unimodal LLM rely on a rigid parameter-to-output mapping, which causes causal-underfit and causal-overfit in cascaded reasoning for Multimodal LLM (MLLM). In this paper, we reformulate MLLM editing as an out-of-distribution (OOD) generalization problem, where the goal is to discern semantic shift with factual shift and thus achieve robust editing among diverse cross-modal prompting. The key challenge of this OOD problem lies in identifying invariant causal trajectories that generalize accurately while suppressing spurious correlations. To address it, we propose ODEdit,a plug-and-play invariant learning based framework that optimizes the tripartite OOD risk objective to simultaneously enhance editing reliability, locality, and generality.We further introduce an edit trajectory invariant learning method, which integrates a total variation penalty into the risk minimization objective to stabilize edit trajectories against environmental variations. Theoretical analysis and extensive experiments demonstrate the effectiveness of ODEdit. Our code is available at
https://anonymous.4open.science/r/ODEdit-2756.

TL;DR: We propose a novel multimodal model editing method named ODEdit to promote editing robustness across diverse cross-modal prompting environments.