UniAdapt: A Universal Adapter for Knowledge Calibration

Authors: Tai D. Nguyen, Long H. Pham, Jun Sun
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: Lifelong model editing, Mixture of experts, Retrieval-Augmented Generation

Abstract:
Large Language Models (LLMs) require frequent updates to correct errors and keep pace with continuously evolving knowledge in a timely and effective manner. Recent research in *it model editing* has highlighted the challenges in balancing generalization and locality, especially in the context of *lifelong model editing*. We discover that inserting knowledge directly into the model often causes conflicts and potentially disrupts other unrelated pre-trained knowledge. To address this problem, we introduce UniAdapt, a universal adapter for knowledge calibration. Inspired by the Mixture of Experts architecture and Retrieval-Augmented Generation, UniAdapt is designed with a vector-assisted router that is responsible for routing inputs to appropriate experts. The router maintains a vector store, including multiple shards, to construct routing vectors based on semantic similarity search results. UniAdapt is fully model-agnostic and designed for seamless plug-and-play integration. Experimental results show that UniAdapt outperforms existing lifelong model editors and achieves exceptional results in most metrics.

Decision: reject
Reviewer scores: 6, 5, 5, 6

Reviews and discussion:
<thread>
creation date:
1730412660051

thread_type:
official_review

<thread_object>
creator: reviewer_p1sb

<content>
summary:
The paper addresses the challenge of updating and maintaining Large Language Models (LLMs) in a timely and effective manner.  To mitigate the lifelong editing problem, they introduce UniAdapt, a universal adapter designed for knowledge calibration. UniAdapt leverages a Mixture of Experts (MoE) architecture and Retrieval-Augmented Generation (RAG) to route inputs to the most relevant experts based on semantic similarity. The router maintains a vector store with multiple shards to construct routing vectors, ensuring precise calibration of the model's output. The adapter is designed to be fully model-agnostic and plug-and-play, making it versatile for various LLMs. Experimental results demonstrate that UniAdapt outperforms existing lifelong model editors in most metrics, showcasing its effectiveness in balancing generalization and locality.

strengths:
1. The experimental results are compelling, showing that UniAdapt outperforms existing lifelong model editors. The paper provides detailed metrics and comparisons, demonstrating the adapter's ability to memorize and generalize effectively.

weaknesses:
- The citation format in the paper is not correct, making the reading difficult.
- The baseline is not comprehensive, the author just compared it with the WISE but other MoE methods like MEMoE and LEMoE are not considered. I think this would weaken the discussion in Table 1 and the contribution of the work as MoE is not a novel thing and the contribution should be more clear.
- The vector store here would be extra parameters and when the number of edits grows this would lead to more memory requirement and time computing, which makes me concerned about the efficiency of the proposed method. It's better to show an Inference Time Analysis.

questions:
- When scaling up to 6k, what's the MoE setting such as the TopK and number of the experts.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730459110323

thread_type:
official_review

<thread_object>
creator: reviewer_qpvw

<content>
summary:
This paper proposes  a universal adapter for knowledge calibration (UniAdapt). The experimental results show that UniAdapt achieves the significantly improved performance on various models and datasets.

strengths:
UniAdapt is fully model-agnostic and designed for seamless plug-and-play integration.

weaknesses:
The experimental analysis is not sufficiently thorough:

 - There is no analysis of the resource consumption of different methods, such as inference time and memory usage.
 -  The baselines are inconsistent across different vanilla models. Specifically, in Table 2, different base models use different baselines—for example, WISE is only applied to LLaMA2-7B, and MEMIT only to GPT2-XL. Why did the author choose this experimental setup?
 -  There is a lack of Out-of-Distribution evaluation.

questions:
The author states that the proposed method primarily aims to address the conflicts caused by directly inserting knowledge into the model. Which experimental result in the paper demonstrates the mitigation of this issue?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730476835521

thread_type:
official_review

<thread_object>
creator: reviewer_8c67

<content>
summary:
The paper presents a novel approach to updating LLMs with new information without compromising existing knowledge. Traditional model-editing techniques often disrupt previous knowledge or fail to generalize well. UniAdapt addresses this by introducing a MoE architecture combined with a vector-assisted router, which selectively directs relevant queries to specific ``experts'' based on semantic similarity. This method allows seamless updates without altering the model’s core parameters, preserving unrelated knowledge effectively.

Experimental results on datasets such as zsRE and Counterfact demonstrate UniAdapt’s reliability in handling updates, generalizing edits, and maintaining unrelated knowledge compared to other methods. By keeping model stability high across thousands of sequential edits, UniAdapt emerges as a scalable, model-agnostic solution for maintaining LLMs with up-to-date knowledge, ideal for lifelong learning in dynamic environments.

strengths:
1. UniAdapt's plug-and-play nature allows it to integrate with various LLMs without altering their original parameters, making it an adaptable and versatile solution for different base models.

2. By combining MoE with a vector-assisted router, UniAdapt selectively routes updates to specific ``experts'', effectively preserving existing knowledge while incorporating new information. This selective routing maintains a balance between reliability and generality.

3. The paper provides experimental evidence to show that UniAdapt performs better than baseline methods. UniAdapt performs well in datasets like zsRE and Counterfact.

weaknesses:
1. The experiments were limited to GPT2-XL and LLaMA2-7B, which may not be sufficient to generalize the results. It would be helpful to include results on state-of-the-art LLMs across a range of model sizes (e.g., 13B and 70B) for more comprehensive insights.

2. Regarding the effect of the target layer, it would be interesting to explore the effects of editing multiple layers simultaneously. Since layer behaviors might differ across models, further investigation on other models would be valuable to confirm similar patterns. Without this, the generalizability of the findings may be constrained.

3. UniAdapt's vector-assisted routing adds complexity, which may increase computational overhead. It would be beneficial to provide cost analysis for both training and inference.

questions:
See above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730553043349

thread_type:
official_review

<thread_object>
creator: reviewer_rvys

<content>
summary:
The main contribution of the paper is the development of UniAdapt, a universal adapter for lifelong model editing in Large Language Models (LLMs) that efficiently integrates new knowledge. The pursuit of lifelong model editing for LLMs is highly pertinent given the growing demand for adaptable AI systems. The concept is crucial for developing models that can continually learn and adapt without requiring full retraining.UniAdapt uses semantic similarity to route queries to the appropriate experts, significantly improving the model’s ability to adapt and generalize over traditional methods.

strengths:
1. Lifelong model editing for LLMs is a promising research direction. The concept is crucial for developing LLM that can continually learn and adapt without requiring full retraining.

2. The experimental results demonstrate the effectiveness of the UniAdapt framework. By integrating a Mixture of Experts with the idea of  RAG, the paper shows improvements in model adaptability and performance on benchmarks designed to test these aspects.

weaknesses:
**Weakness 1**

The core motivation, as stated in the abstract:
>“We discover that inserting knowledge directly into the model often causes conflicts and potentially disrupts other unrelated pre-trained knowledge”

This motivation lacks novelty as reducing the disruption to pre-trained knowledge while inserting new information is a foundational objective of the entire knowledge editing field (*measured by the locality metric*), as extensively discussed in foundational papers such as “Fast Model Editing at Scale.”



**Weakness 2**

The paper’s primary contribution is applying the concept of a Mixture of Experts to the knowledge editing field. While technically sound, this application does not introduce many new insights into the field. Moreover, the implementation is described as complex and the overall concept may not engage the community due to its incremental nature rather than a groundbreaking innovation.


While the paper presents valid improvements in certain performance metrics for the chosen datasets, the novelty is somewhat limited, and the implementation is not elegant. Considering the high standards of ICLR, the paper could be seen as a borderline case. My overall score would be 5.5, reflecting a neutral to slightly positive evaluation but acknowledging the concerns regarding novelty and implementation complexity.

questions:
Please refer to the previous section.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732516264833

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
We thank all reviewers for their valuable comments and suggestions that will help improve our work. We are encouraged that the reviewers found our method novel and well-demonstrated, with comprehensive and robust experiments.

We address their concerns as follows:

- **Contribution**: We generalize the problem and propose an independent router that can be trained once and applied to any model. Additionally, we introduce a loss function to enhance classification performance.
- **Resource consumption**: Our additional experiments demonstrate reasonable resource usage, with better training and inference times compared to WISE.

As all reviewers expressed interest in resource consumption experiments, we present them here to address their questions.

### **1. Inference time analysis**

We measured LLAMA2-7b's inference time with and without UniAdapt after training with T=3000 on ZsRE. Based on an average of three inference trials, the base model took 0.014 seconds. UniAdapt added a minor overhead of 5.75%—slightly higher than WISE-Merge (3%) but lower than WISE-Retrieve (7%).


### **2. Memory analysis**

UniAdapt loads two modules: a router built on top of all-MiniLM-L6-v2 and a vector storage for embeddings. The router requires 620 MB, while the original LLAMA2-7b model requires 26,222 MB. Each embedding has a shape of 384. For 3,000 embeddings of float32, the size is 3,000 × 384 × 4 = 4.6 MB. An expert requires 64 MB. With a single expert, the total additional memory needed is 688.6 MB, representing a 2.63% overhead. When scaling UniAdapt to 8 experts and 9,000 edits, the required memory becomes 4.6 × 3 + 620 + 64 × 8 = 1,145.8 MB, with a 4.37% overhead. The WISE’s overhead is 0.64% in theory and 4% in practice.

### **3. Training time analysis**

Due to time limitations, we only analyzed the training time of UniAdapt compared to WISE on the ZsRE dataset using LLaMA-7B.
| Method    | Number of edits | Router training (s) | Edit training (s) | Total (s) |
|-----------|------------------|----------------------|--------------------|-----------|
| UniAdapt  | 10              | 0.96                | 14.9              | **15.86**     |
| UniAdapt  | 100             | 6.08                | 142.8             | **148.88**    |
| UniAdapt  | 1000            | 55.35               | 1423.82           | **1479.17**   |
| WISE      | 10              | 0                   | 94                | 94        |
| WISE      | 100             | 0                   | 603.12            | 603.12    |
| WISE      | 1000            | 0                   | 5273.82           | 5273.82   |

UniAdapt's training time consists of two components: router training and edit training. While the training time increases with the number of edits, and UniAdapt requires additional time for router training, its total training time is still approximately 4.5 times faster than WISE.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The main contribution of this paper is the development of UniAdapt, a universal adapter for lifelong model editing in Large Language Models (LLMs) that efficiently integrates new knowledge. The concept of lifelong model editing is highly relevant given the increasing demand for adaptable AI systems. This approach is crucial for developing models that can continuously learn and adapt without requiring full retraining. UniAdapt leverages semantic similarity to route queries to the appropriate experts, significantly enhancing the model's ability to adapt and generalize compared to traditional methods. However, the core motivation about reducing conflicts when inserting new knowledge into the model is not novel, as it has already been a key objective in the knowledge editing field, particularly addressed by foundational work like "Fast Model Editing at Scale." The application of the Mixture of Experts concept to knowledge editing is new but does not introduce  new insights. The contribution feels incremental, and the implementation is described as complex, potentially limiting its appeal to the community. It is recommended that the author revise the paper according to the reviewers' suggestions and include additional experiments.