Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass

Authors: Tong Chen, Hao Fang, Patrick Xia, Xiaodong Liu, Benjamin Van Durme, Luke Zettlemoyer, Jianfeng Gao, Hao Cheng
Primary area: foundation or frontier models, including LLMs
Keywords: language model; efficient adaptation; fine-tuning; prompting

Abstract:
Large language models (LLMs) acquire substantial knowledge during pretraining but often need adaptation to new contexts, tasks, or domains, typically achieved through fine-tuning or prompting. However, fine-tuning incurs significant training costs, while prompting increases inference overhead. Inspired by fast weight memory, we introduce GenerativeAdapter, an effective and efficient adaptation method that encode test-time context into language model parameters with a single forward pass.
GenerativeAdapter augments a frozen pretrained LM with a lightweight adapter generator, trained via self-supervised learning, to produce parameter-efficient adapters.
Notably, our generator is general-purpose, i.e., one generator can adapt the corresponding base model for all langauge processing scenarios.
We apply GenerativeAdapter to two pretrained LMs (Mistral-7B-Instruct and Llama2-7B-Chat) and evaluate the adapted models across  knowledge acquisition from documents, learning from demonstrations, and personalization for users.
In StreamingQA, our approach is effective in injecting knowledge into the LM's parameters, achieving a 63.5\% improvement in F1 score over the model with supervised fine-tuning (from $19.5$ to $31.5$) for contexts as long as 32K tokens.
In the MetaICL in-context learning evaluation, our method achieves an average accuracy of $44.9$ across 26 tasks, outperforming the base model. 
On MSC, our method proves to be highly competitive in memorizing user information from conversations with a 4x reduction in computation and memory costs compared to 
prompting with full conversation history.
Overall, GenerativeAdapter provides a viable solution for adapting large LMs to evolving information and providing tailored user experience, while reducing training and inference costs relative to traditional fine-tuning and prompting techniques.

Decision: accept (poster)
Reviewer scores: 5, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730369307354

thread_type:
official_review

<thread_object>
creator: reviewer_pyah

<content>
summary:
This paper proposes a novel method to integrate the information into the parameters. The experimental results are comprehensive and demonstrate the effectiveness of this method.

strengths:
1. The topic is important. 
2. The idea of this paper is novel and clean, inspired by fast weights. 
3. The experimental results show the effectiveness of this method. it is impressive to see that model is able to inject 32k tokens into the parameters.

weaknesses:
1. **Missing related works**: GenerativeAdaptor adds the adaptor which is quite similar to memory-based methods, where people would add a memory module on the large language models [1,2]. Perhaps the paper should discuss some related papers and the correlations between them. 

2. **Size of the additional parameters**: As stated in line 264, GenerativeAdaptor introduces 500 million parameters, however, even using 500 million parameters cannot achieve comparative performance as In-context Prompting (as shown in Figure 2(a)). Consider saving all the tokens in the context, which leads to 32768 * 4096 = 134 million parameters. This is fine, as the proposed method has smaller computation cost than storing all the previous tokens, but this might be something to consider, as it shows that the parameters may not be fully utilized, or the information in the parameters might be overly sparse. 

[1] Memllm: Finetuning llms to use an explicit read-write memory   
[2] MemoryLLM: Towards Self-Updatable Large Language Models

questions:
I don't have any questions.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730710750654

thread_type:
official_review

<thread_object>
creator: reviewer_mtd9

<content>
summary:
The paper proposes GenerativeAdapter, which is a method to introduce unseen knowledge to LMs without finetuning cost or extra prompts. It generates a new adapter for the target unseen knowledge and utilize it to adapt the LMs.

strengths:
- The paper addresses the critical challenge of adapting LMs to incorporate new knowledge effectively, a fundamental issue since LMs can only utilize the information they were originally trained on. 
- It introduces an interesting approach of training another generator to generate an adaptor and optimize adaptation cost

weaknesses:
- Some of the important experiments are either missing or have questionable setting
    - It is suspicious that the the settings for experiment 4.1 are fair for all of the baselines. It seems like the knowledge that each baseline has seen are different.
    - In section 5, the paper mentions the preliminary study shows the quality of the adapter generator is highly correlated with the final evaluation metrics. It would have been better if the results that support this claim is included in the paper. Many times, the training objective is not exactly aligned with the downstream task.
    - It would be better if the experiment to see how the adaptor affects the original ability of the LMs is included in the paper.

questions:
- Is there a suspected reason why the unnormalized adaptor contains extreme values?
- In the formulations in section 2.3, there is a  G(x_{1:m}). In this case, shouldn't the hidden states input to the generator, not the input tokens?
-  In the completion loss formulation, shouldn't the output just x_{m+1}, not x{m+1},...,x_n?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730779176100

thread_type:
official_review

<thread_object>
creator: reviewer_apsm

<content>
summary:
The paper introduces GenerativeAdapter, a method for adapting pretrained large language models (LLMs) using lightweight, parameter-efficient adapters generated by a self-supervised generator network. This approach aims to enhance LLM adaptability without incurring the computational cost of fine-tuning or the inference overhead of prompting. Evaluations are conducted across three main areas: factual knowledge acquisition, learning from demonstrations, and user personalization, showing that GenerativeAdapter is effective in reducing computational load and increasing model adaptability to various contexts.

strengths:
1. The paper presents an efficient approach to adapting LLMs in real-time by using a lightweight, bilinear adapter generator. This mechanism effectively reduces computational costs associated with full model fine-tuning and inference-time overhead due to prompting, which is particularly advantageous for applications on edge devices.

2. The authors evaluate *GenerativeAdapter* across diverse, practical contexts such as document-based knowledge acquisition, in-context learning, and user personalization. These evaluations, especially in a personalized setting, demonstrate the adaptability of the method and its effectiveness in various contexts that align with real-world demands.

weaknesses:
1. The authors provide detailed descriptions of the model architecture, adapter generation mechanism, and datasets. However, hyperparameter settings and implementation details could be expanded, especially regarding stability mechanisms like SVD normalization, which could impact reproducibility. It is unclear if the code or models will be open-sourced for verification.
2. The analysis of *GenerativeAdapter*’s performance in different scenarios is comprehensive; however, the lack of prompt  engineering effort for baselines could introduce an unfair comparison. A clarification on the extent of prompt engineering would ensure a fairer assessment.
3. While *GenerativeAdapter* achieves superior performance over specific context lengths and adaptation scenarios, claims of general "efficiency" for "personalization" are strong, given the specific constraints of the study. The authors should emphasize that observed benefits apply to controlled experiments rather than broad generalizations.
4. The paper introduces concepts like "test-time contextualization" and "parameter-efficient" without specific definitions, which could hinder comprehension for readers less familiar with these terms. Explicit definitions or brief explanations in the methodology section would help maintain clarity.

questions:
How does GenerativeAdapter perform in scenarios where the context information is dynamic or frequently changing, such as real-time conversational agents? It would be interesting to see if the method’s efficiency and adaptability hold in continuously evolving contexts.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730841446325

thread_type:
official_review

<thread_object>
creator: reviewer_1mhi

<content>
summary:
This paper proposes a neural module called generative adapter, which once trained can accept a stream of input contexts to adapt transformers on the fly. The paper shows that generative adapters work well at lower computational cost than in-context learning on several NLP tasks.

strengths:
- The method is simple and well-motivated.
- The method works nicely in a streaming setting to continually update weights using lower number of FLOPs due to being able to reuse weights computed at the previous time step of the stream.
- The paper is clearly written and easy to follow.

weaknesses:
- I think the Ultragist baseline should be reported in $\S 4.1$ and  $\S 4.2$, not just $\S 4.3$. Gisting is a natural comparison and I am glad to see it compared against in some form, but I think it could be reported more widely in the paper.
- Using llama-2-7b doesn't provide much value in my opinion. The 4k context length makes Figure 2b hard to reason about, and I don't see why all llama-2-7b experiments couldn't be done with llama-3.1-8b instead for similar cost.

questions:
- From what I understand, generative adapters are Hypernetworks [[Ha et al](https://arxiv.org/abs/1609.09106)], but that hasn't been discussed anywhere. Do the authors agree with this assessment, and if so, how would they contextualize similar work using hypernetworks for continual learning [[Oswald et al](https://arxiv.org/abs/1906.00695), [Vladymyrov et al](https://arxiv.org/abs/2301.04584)]?
- Why is the continual pretraining baseline so much lower than the generative adapter baseline? This sticks out and is surprising to me, and makes me think it is poorly tuned.
- I think a valuable baseline (especially since continual pretraining seems to underperform) might be using the generative adapter loss function (Eqn 5) but optimize the weights of the model directly instead of the generative adapter. If I understood correctly, I don't see any reason why this shouldn't be possible, and if generative adapter is in the ballpark of this baseline, it's a positive signal. Do the authors agree with this assessment?
</content>
</thread_object>

</thread>
<thread>
creation date:
1733208823314

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of the Discussions

comment:
Dear AC,

Thank you so much for handling our paper! We would like to summarize the discussion at the end of the discussion phase.

To address the reviewers' concerns, we have made the following updates and clarifications:

1. **Baseline Methods (1mHi):**  As requested by the reviewer, we have included new results for Ultragist, a context compression method using gist tokens, in Sections 4.1 and 4.2.  
2. **Perplexity as Validation Metric (mTD9):**  We added new results on document-based QA performance as part of the ablation study.  
3. **Impact on Original Language Model Capabilities (mTD9):**  We provided new experimental results on MMLU to evaluate knowledge memorization across a wide range of domains, comparing models with and without the generated adapter.  
4. **Data Inefficiency of the Continual Pretraining Baseline (1mHi, mTD9):**  We explained the low performance of the continual pretraining baseline in document-based QA, citing insights from prior research.  
5. **Related Works (1mHi, PYAH):**  We acknowledged the relevance of hypernetworks and memory-based methods to our study. These connections were discussed in our response and will be elaborated in the paper.  
6. **Methods and Implementation Details (apSm, PYAH):**  Reviewers noted minor issues in the description of methods and implementation. We will refine these sections for greater clarity in the final paper.  
7. **Complementary Benefits on top of RAG (1mHi):** An LM merged with a generative adapter can be combined with prompting or other token compression approaches to provide complementary benefits. To demonstrate this, we conducted new experiments combining GenerativeAdapter with Retrieval-Augmented Generation (RAG), and the results show that this approach can further boost performance.

We summarized the strengths acknowledged by the reviewers and highlighted our contributions:

1. **Significance of the Problem (1mHi, apSm, mTD9, PYAH):**  Adapting language models to integrate new knowledge effectively and efficiently remains a critical challenge.  
2. **Efficiency of Generating Adapters (1mHi, apSm, mTD9):**  The generative adapter mechanism introduced in our work is novel and reduces computational costs compared to fine-tuning. Also, by reusing previously generated weights, it supports streaming contexts and achieves efficient adaptation.  
3. **Effective Knowledge Integration (PYAH):**  GenerativeAdapter is able to integrate factual knowledge into language model weights effectively on a single document. In contrast, conventional continual pre-training requires a large corpus and multiple repetitions of facts to achieve memorization.  
4. **Miscellaneous Strengths:**  Reviewer 1mHi noted that the paper is clearly written and easy to follow.  Reviewer PYAH highlighted that our evaluation contexts are diverse and practical.  

Thank you again for your time and efforts in reviewing our paper. We hope these updates address the reviewers' concerns and enhance the clarity of our contributions.

Best,

Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper introduces GenerativeAdapter, a novel technique for adapting pre-trained Large Language Models (LLMs) to new tasks without requiring fine-tuning or prompting.  Instead of modifying the core LLM parameters, GenerativeAdapter trains a lightweight adapter module using a self-supervised generative network. This adapter can process a continuous stream of input contexts, enabling the transformer model to adapt on-the-fly. This approach offers significant advantages in terms of efficiency, reducing both training time (compared to fine-tuning) and inference time (compared to prompting).

The effectiveness of GenerativeAdapter is demonstrated through experiments on various tasks, including factual knowledge acquisition, learning from demonstrations, and user personalization.  While reviewers have praised the significance and novelty of this work, they have also raised concerns about the chosen baseline methods and potential similarities to existing techniques like hypernetworks and memory-based methods.

To strengthen the paper, I recommend the following additions:

Comprehensive Baseline Comparisons: Include a broader range of baseline methods to thoroughly evaluate the performance of GenerativeAdapter.
In-Depth Analysis: Provide a detailed analysis comparing and contrasting GenerativeAdapter with hypernetworks and memory-based methods, highlighting its unique contributions and advantages.