Title: Continual Model Merging without Data: Dual Projections for Balancing Stability and Plasticity
Venue: NeurIPS 2025
Accepted: yes
Keywords: Model Merging

Continual Learning

Multi-task Learning
Abstract: Model merging integrates multiple expert models with diverse capabilities into a unified framework, facilitating collaborative learning. However, most existing methods assume simultaneous access to all models, which is often impractical in real-world scenarios where models are received sequentially. While some studies have investigated continual model merging (CMM)--which involves sequentially merging multiple models--the challenge of balancing prior knowledge (stability) and incorporating new tasks (plasticity) remains unresolved. This paper, for the first time, formally defines the stability and plasticity of CMM from the perspective of orthogonal projection. Subsequently, we analyze the relationships among the spaces spanned by task data, historical gradients, and accumulated gradients. Building on this, we propose a data-free \textbf{D}ual \textbf{O}rthogonal \textbf{P}rojection (DOP) method, which eliminates data dependence and mitigates interference between the merged model and models for old and new tasks by projecting their parameter differences onto their respective approximate data spaces. Finally, to solve potential conflicts between stability and plasticity, we reformulate DOP as a multi-objective optimization problem and employ a multi-gradient descent algorithm to obtain a Pareto-optimal solution.  Extensive experiments across multiple architectures and task configurations validate that our approach significantly outperforms state-of-the-art CMM methods.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of continuous model merging, where expert models are received sequentially, and the original training data is not accessible. The authors propose a data-free method named Dual Orthogonal Projection (DOP), which aims to balance stability and plasticity by formulating the merging process as a multi-objective optimization problem. Extensive experiments on both vision and language models across multiple task configurations demonstrate superior performance over baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths.\n+ This paper is well-written and easy to follow.\n+ Extensive experiments on various architectures and tasks show the effectiveness of the proposed method.\n+ The proposed method does not rely on data during continual merging.\n\nWeaknesses.\n+ Although the method is grounded in orthogonal projection theory, this author leverages task vectors instead of actual data or gradients. The approximation error remains unsolved and analyzed. This could affect the reliability of the proposed method.\n+ The approach assumes linear layers dominate the parameter space, which is valid for transformers. I suggest the author consider significant non-linear modules, such as bias parameters and parameters of normalization layers.\n+ The paper proposes to solve the continuous model merging problem, where old task vectors cannot be stored. However, in practice, storing task vectors is usually not expensive, making this assumption somewhat questionable. So, I think this problem may not be a significant setting for model merging.\n+ The novelty is also limited; the proposed method seems very close to OPCM. The proposed method projects boththe  new task vector and the old task vector, while OPCM only project the old task vector.\n\nQUESTIONS:\nSee weakness.\n\nLIMITATIONS:\nNo. The author leverages task vectors instead of actual data or gradients to conduct projection. The approximation error remains unsolved and analyzed.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the problem of continual model merging in settings where models are received sequentially, without simultaneous access to all models. It considers the balance between stability and plasticity in continual model merging and investigates the relationships between data, gradients, and task vectors (accumulated gradients). Thus, it proposes a data-free dual orthogonal projection method, using a multi-gradient descent algorithm to derive Pareto-optimal coefficients for stability and plasticity. Experiments validate the effectiveness of the proposed DOP across various models and visual/language tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. This paper proposes DOP, which not only addresses the forgetting issue in continual model merging but also reduces the influence of the old merged model on the new upcoming model.\n\n2. The ablation studies are well-designed and provide clear evidence of the contributions of different components within DOP.\n\n3. The motivation and formulation of DOP are presented clearly, with a well-grounded theoretical explanation.\n\nWeaknesses:\n\n1. The baselines in the experiments are insufficient. For the simultaneous model merging, only three methods are included (SWA, Task Arithmetic, TIES), but there are other recent merging algorithms, such as MagMax[1], which should be considered to strengthen the empirical evaluation.\n\n2. While the paper focuses on proposing an effective algorithm for continual model merging and the experiments also show that the simultaneous model merging algorithms exactly fail in the continual merging setting, it lacks a deeper analysis of why these simultaneous model merging algorithms fail in continual merging. Providing such analysis would reinforce the novelty and necessity of the proposed approach.\n\n3. The proposed DOP requires post-training (gradient-based optimization) when merging, which incurs additional training cost. It would be good to have a training-free merging algorithm instead of additional training.\n\n[1] “MagMax: Leveraging Model Merging for Seamless Continual Learning”, ECCV2024.\n\nQUESTIONS:\n1. The paper consider reducing the influence of old merged model on the new model, which means $\\Delta W_{n}^{*}X_{n}=0$. \n\nBut if the old merged model have similar knowledge or positive knowledge which can be transferred to the new model and make the performance of new model have better performance on the new task data, then $\\Delta W_{n}^{*} X_{n}$ may not be zero in this case. How does DOP handle such situations? Does the method risk discarding positively aligned knowledge that would otherwise enhance performance on the new task?\n\n2. The baselines in the experiments are limited. Can authors compare DOP with MagMax[1]?\n\n[1] “MagMax: Leveraging Model Merging for Seamless Continual Learning”, ECCV2024.\n\n3. In Eq(2), why does DOP use $U_{o}^{\\top}\\Sigma_{o}$ and $\\Sigma_{o}V_{o}$ rather than $U_{o}\\sqrt{\\Sigma_{o}}$ and $\\sqrt{\\Sigma_{o}}V_{o}^{\\top}$, since $U_{o}\\sqrt{\\Sigma_{o}}\\sqrt{\\Sigma_{o}}V_{o}^{\\top}=\\tau_{o}$? Additionally, can authors report the effect of removing singular values (i.e., omitting $\\Sigma_o$ and $\\Sigma_n$) in this representation? A comparison of these variants could clarify the role of singular value scaling in the projection spaces.\n\n4. What’s the value of final $\\alpha_{s,K}$ and $\\alpha_{p,K}$? Can authors show the trending or changes of $\\alpha_{s,K}$ and $\\alpha_{p,K}$ during multi-objective optimization?\n\n5. For computational efficiency, beyond running time, can authors provide additional metrics, such as GPU memory usage or computational complexity, especially in comparison with OPCM?\n\n6. In Eq(3), there appears to be a missing index $k$ in the term  $\\nabla_{W_{*,k}}\\mathcal{L}_{p}$ in $\\min_{\\alpha_{k}\\in[0,1]}\\|\\cdot\\|$.\n\nLIMITATIONS:\nWhile DOP is a promising algorithm, it is currently limited to full model merging and may not scale efficiently to large language models in resource-constrained environments. Besides, the method is designed to minimize interference from the old merged model, but may ignore scenarios where positive transfer from the old merged model could benefit the new task. And DOP requires training during merging, thus developing a training-free algorithm remains an open direction for future research.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a Dual Orthogonal Projection (DOP) method for Continual Model Merging (CMM) without access to original data. DOP formally defines and balances stability and plasticity by projecting parameter updates onto the subspaces of previous and new tasks. The approach is formulated as a multi-objective optimization and solved with multi-gradient descent. Experiments on vision and language models show that DOP outperforms existing methods in both effectiveness and efficiency.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- Analyzing continual model merging from the perspective of orthogonal projection is interesting.\n- The high-level idea of the method is intuitive: the space spanned by model gradient updates is used to approximate the input space, and by ensuring that the parameter changes for new and old tasks are orthogonally projected onto their respective input spaces, the method achieves both retention of previous knowledge and learning of new information.\n\nWeaknesses\n- The analysis of orthogonal projection is derived under the assumption of linear functions, but neural networks contain many nonlinear components and are highly stacked. It is unclear how the current analysis can be extended to structures that are closer to real scenarios, such as linear layers with activation functions or multiple stacked linear layers.\n- More baseline comparisons could be included in the experimental results.\n\nQUESTIONS:\n- For continual model merging, is it possible to maintain a streamlined model library, where each model covers a certain category, and combinations of models can cover more cases? Could a dynamic merging mechanism based on this model library be used to address the formulation problem?\n- Can the current orthogonal projection analysis be generalized to more complex network architectures?\n- Can the proposed method be extended to networks with different structures?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of continual model merging (CMM), where models are merged incrementally without access to original training data. The authors propose Dual Orthogonal Projection (DOP), a data-free method that balances stability (retaining old task knowledge) and plasticity (adapting to new tasks) through multi-objective optimization. Key contributions include formalizing stability and plasticity through orthogonal projection theory, analyzing subspace relationships, and introducing DOP with MGDA optimization.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. This work proposes a novel perspective of the relationships among the subspaces spanned by task data, gradients, and task vectors.\n2. DOP is a data-free method, which is both memory- and time-efficient.\n3. Extensive experiments on vision and language tasks demonstrate superior performance over SOTA methods.\nWeaknesses\n1. Non-linear components are handled via simple averaging, while different linear layers are not differentiated, which may still leave room for further optimization.\n\nQUESTIONS:\nQuestions\n1. OPCM is also a data-free method based on orthogonal projections. Are there additional advantages of DOP beyond the use of MGDA for adaptively balancing the weights between old and new tasks?\n2. In the Tab.4 of ablation studies for projection space, it seems that the orthogonal constraints in row space and column space have huge differences (63.3 v.s. 75.9). Could authors explain this phenomenon?\n3. Fig.3(d) highlights the importance of the coefficient alpha. Could the authors visualize the solved alpha during optimization and give more insights into the model merging process?\n4. Linear components mainly reside in embedding layers, Q/K/V projection layers, and FFN layers. Does DOP demonstrate varying degrees of improvement in these three parts compared to SWA and OPCM?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}