Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass

## Abstract

Abstract Large language models (LMs) are typically adapted to improve performance on new contexts ( e.g., text prompts that define new tasks or domains) through fine-tuning or prompting. However, there is an accuracy compute tradeoff—fine-tuning incurs significant training cost and prompting increases inference overhead. We introduce GenerativeAdapter , an effective and efficient adaptation method that directly maps new contexts to low-rank LM adapters, thereby significantly reducing inference overhead with no need for finetuning.
The adapter generator is trained via self-supervised learning, and can be used to adapt a single frozen LM for any new task simply by mapping the associated task or domain context to a new adapter.
We apply GenerativeAdapter to two pretrained LMs (Mistral-7B-Instruct and Llama2-7B-Chat) and evaluate the adapted models in three adaption scenarios: knowledge acquisition from documents, learning from demonstrations, and personalization for users.
In StreamingQA, our approach is effective in injecting knowledge into the LM’s parameters, achieving a 63.5% improvement in F1 score over the model with supervised fine-tuning (from 19.5 19.5 19.5 19.5 to 31.5 31.5 31.5 31.5 ) for contexts as long as 32K tokens.
In the MetaICL in-context learning evaluation, our method achieves an average accuracy of 44.9 44.9 44.9 44.9 across 26 tasks, outperforming the base model.
On MSC, our method proves to be highly competitive in memorizing user information from conversations with a 4x reduction in computation and memory costs compared to
prompting with full conversation history.
Together, these results suggest that GenerativeAdapter should allow for general adaption to a wide range of different contexts.

## 1 Introduction

Adaptation is essential for language models (LMs) to acquire new world knowledge (Jiang et al., ; Hu et al., ; Mecklenburg et al., ), learn new tasks (Min et al., ), and personalize to individual users (Salemi et al., ).
Existing adaptation methods typically involve either *prompting* or
*fine-tuning* (Brown et al., ).
As the scale of LMs continues to increase, adapting them becomes increasingly
difficult due to efficiency constraints during both training and inference (Hu et al., ).

Prompting with task-specific demonstrations
(*i.e.,* in-context learning (Brown et al., ))
or background knowledge (*i.e.,* retrieval-augmented generation (Lewis et al., )) is one way to enable models to temporarily encode such relevant information, allowing flexible adaptation to various tasks.
However, to maintain additional memory across sessions, some extra prompts must be added to the input, which incur an inference-time or storage overhead (Chevalier et al., ).
Fine-tuning is another way to embed new information into the LM’s parameters, retaining long-term memory.
Nevertheless, it requires a training phase that is more computationally expensive than a single forward pass, and acquiring knowledge through continual pretraining has shown to be data-inefficient (Yang et al., ; Allen-Zhu & Li, ).
Thus, we are interested in exploring alternative approaches for effectively and efficiently adapting pretrained LMs.

In this work, we present GenerativeAdapter, a novel method for training a neural network (adapter generator) to generate adapters that contextualize pretrained LMs on-the-fly with temporary knowledge from incoming contexts.
Inspired by fast weights (Ba et al., ; Schmidhuber, , inter alia), our approach incorporates a lightweight adapter generator on top of pretrained LM as the slow network to produce updated parameters for the fast network (the adapted LM).
As far as we know, we are the first to explore this direction.
Specifically, the pretrained base LM remains frozen while we train the LM-specific adapter generator to generate layer-by-layer additive updates, similar to recent parameter-efficient fine-tuning (PEFT) techniques (Houlsby et al., ; Hu et al., ).
For each layer, a adapter generator network uses the outer product of past context hidden states from the corresponding base LM layer to generate delta weights.
These generated delta weights are then added to the base LM weights to form an adapted LM for future predictions.
Similar to previous work on fast weights, our method achieves test-time adaptation using only forward passes, allowing dynamic updates as new context arrives in sequential chunks.
We train the generator end-to-end in a self-supervised manner by compressing the context into a generated adapter and then computing the next-token prediction loss on a target sequence using the adapted LM.
Once trained, our method can produce adapted LMs that effectively capture knowledge from the context to solve multiple downstream tasks, thus improving the adaptability of off-the-shelf pretrained LMs.

We evaluate our method on three scenarios where on-the-fly contextualizing pretrained LMs is crucial: acquiring new factual knowledge, learning from demonstrations, and personalizing for individual users.
These scenarios involve diverse forms of context with varying lengths, including documents with background knowledge, task-specific input-output examples and user-specific conversations.
In the knowledge acquisition scenario, GenerativeAdapter effectively memorizes factual knowledge from provided documents, with minimal information loss compared to full-context prompting at short context lengths.
Notably, our method excels in memorizing long-context documents, managing to handle context lengths up to 32K on StreamingQA (Liska et al., ) and 8K on SQuAD (Rajpurkar et al., ) better than continous pretraining.
In learning from demonstrations on MetaICL, GenerativeAdapter follows demonstrations effectively, achieving superior accuracy compared to the in-context learning of its base model. This exemplifies the model’s ability to adapt to new tasks efficiently.
For personalization, GenerativeAdapter is highly effective in retaining user information from conversations, achieving a fourfold reduction in computation and memory costs compared to full conversation prompting.
In practical scenarios with many queries from the same user on edge computing devices, the benefits of our method are even more evident.
This positions GenerativeAdapter as a highly efficient tool for personalized LMs.

Our contributions are summarized as follows:

- 1.
We introduce GenerativeAdapter, a novel method for efficiently adapting pretrained LMs on-the-fly using test-time contexts. To our knowledge, we are the first to explore retaining the relevant temporary knowledge through generated parameter-efficient model updates for state-of-the-art pretrained LMs.
- 2.
We develop an adapter generator network on top of frozen pretrained
LMs to transform text contexts into updated model parameters
(adapted LMs)
for future queries. We also design an efficient end-to-end training process
to enhance the LMs’ adaptability, *i.e.,* the resulting generator augmented LM
can be used for various downstream tasks using only forward passes.
- 3.
We validate the proposed method on two representative pretrained LMs.
Empirically, we show the effectiveness of GenerativeAdapter in various adaptation scenarios, including knowledge acquisition from documents, learning from demonstrations, and personalized user interactions. Our method proves to be generalizable across different types of contexts and applicable to multiple downstream tasks.

## 2 Method

We present GenerativeAdapter, an efficient and effective framework for directly generating additive weight updates to contextualize the pretrained LM (a frozen base LM) at test time.
Unlike continual pretraining and supervised fine-tuning which update
the pretrained LM via gradient descent, our method achieves adaptation using
forward passes only.
In the following sections, we first provide a task formulation and an overview of GenerativeAdapter (§2.1).
Then we describe the the adapter generator (§2.2),
followed by the self-supervised pretraining tasks (§2.3) and
the normalization techniques for improving training stability
(§2.4).

Figure: Figure 1: Overview of GenerativeAdapter. Left: During test-time contextualization, the adapters $\Delta_{1},\ldots,\Delta_{t}$ are generated sequentially for the stream of context chunks $C_{1},\ldots,C_{t}$. At a given time step $t$, the context chunk $C_{t}$ is encoded by the base LM $\Theta_{\rm base}$ into hidden state vectors $\mathbf{H}_{t}$. Then the generator $\mathcal{G}$ produces a new adapter $\Delta_{t}$ based on the collection of hidden state vectors $\mathbf{H}_{1},\ldots,\mathbf{H}_{t}$ representing the accumulated context. Right: During inference, we combine the latest adapter $\Delta_{t}$ with the base LM $\Theta_{\rm base}$ to generate responses for input prompts.
Refer to caption: x1.png

### 2.1 Adaptation with Test-time Contextualization

To contextualize a base model, $\Theta_{\rm base}$, to a given context $C$, our goal is to obtain an updated model, $\Theta_{C}$, that can respond to user instructions using the information provided in the context $C$. In practice, the context can include different types of data, such as documents,
dialogues, or task description and few-shot demonstrations.

We specifically focus on test-time contextualization, where context arrives incrementally as a stream of data, such as a continuous flow of documents or dialogue sessions. We represent this streaming context up to time step $t$ as
$\Sigma(t)\coloneq(C_{1},\ldots,C_{t})$,
where $C_{t}$ is the context chunk arriving at time step $t$. In this online adaptation scenario, the model must be efficiently adapted to each new context chunk as it becomes available.

As shown in Figure 1, we propose GenerativeAdapter as a framework that adapts the base model $\Theta_{\rm base}$ to new contexts through a single forward pass as each context chunk arrives.
Specifically, given test-time context $\Sigma(t)$, we adapt the base model
$\Theta_{\rm base}$ to a new model $\Theta_{\Sigma(t)}$ using a context-dependent
additive adapter $\Delta_{t}$, *i.e.,* $\Theta_{\Sigma(t)}=\Theta_{\rm base}+\Delta_{t}$.
More details regarding the adapter $\Delta_{t}$ will be provided in §2.2.
After this adaptation, the modified model $\Theta_{\Sigma(t)}$ can be utilized for any
test input relevant to the context $\Sigma(t)$ during inference.
For example, if the context $\Sigma(t)$ consists of a user’s past conversations,
the modified model $\Theta_{\Sigma(t)}$ can effectively summarize or answer questions about these
conversations.

### 2.2 Generative Adapter

In this paper, we propose using a learned adapter generator $\mathcal{G}$ to directly produce the adapter $\Delta$ based on the streaming context $\Sigma$.
The core idea is to use the adapter generator to project context token embeddings, encoded by the base language model (LM), into the matrices of each layer in the LM.
Specifically, we consider only adapting the linear projection layers of the base
Transformer model, *i.e.,* the key/query/value/output layers of the multi-head attention unit and the
down/up projection layers of the feed-forward network.

Concretely, a linear projection layer in the $l$-th Transformer block ($l=1,2,\ldots,L$)
can be written as $\mathbf{\bm{o}}=\mathbf{W}^{(l)}\mathbf{\bm{h}}$,
where $\mathbf{W}^{(l)}\in\mathbb{R}^{d_{\rm out}\times d_{\rm in}}$ is the weight matrix,
$\mathbf{\bm{h}}\in\mathbb{R}^{d_{\rm in}}$ is the input vector,
$\mathbf{\bm{o}}\in\mathbb{R}^{d_{\rm out}}$ is the output vector,
and we omit the bias term for simplicity.
For the adapted LM, we parameterize $\mathbf{W}^{(l)}=\mathbf{W}^{(l)}_{\rm base}+\mathbf{W}^{(l)}_{\Delta}$,
where $\mathbf{W}^{(l)}_{\rm base}$ and $\mathbf{W}^{(l)}_{\Delta}$ are the corresponding
weight matrices in the base model $\Theta_{\rm base}$ and the context-dependent
adapter $\Delta$, respectively.

To generate the weights of the context-dependent adapter $\Delta$, we first
encode the streaming context $\Sigma$ using the base model $\Theta_{\rm base}$
and obtain the sequence of hidden states $\mathbf{\bm{h}}^{(l)}_{1},\mathbf{\bm{h}}^{(l)}_{2},\ldots,\mathbf{\bm{h}}^{(l
)}_{M}\in\mathbb{R}^{d_{h}}$
(*i.e.,* the outputs of the $l$-th Transformer block),
where $M$ is the number of tokens in the context $\Sigma$, and $d_{h}$ is the dimension
of hidden states.
These hidden states are packed in a matrix $\mathbf{H}^{(l)}\in\mathbb{R}^{M\times d_{h}}$.
Then we use the hidden states from the $(l-1)$-th Transformer block to
generate the adapter’s weights $\mathbf{W}^{(l)}_{\Delta}$ for the $l$-th Transformer
block, *i.e.,* $\mathbf{W}^{(l)}_{\Delta}=\mathcal{G}^{(l)}(\mathbf{H}^{(l-1)})$,
where $\mathcal{G}^{(l)}(\cdot){:}\,\mathbb{R}^{*\times d_{h}}\rightarrow\mathbb{R}^{
d_{\rm out}\times d_{\rm in}}$
denotes the layer-specific adapter generator which can transform any hidden state
sequence of arbitrary length into a fixed-dimensional weight matrix.
For conciseness, we will omit the superscript denoting the layer number $l$ when it
does not cause ambiguity.

To obtain a generator $\mathcal{G}(\cdot)$ without the undesirable dependency on
the context length $M$, we use a bi-linear function as following,

$$ $\mathbf{W}_{\Delta}=\mathcal{G}(\mathbf{H})=(\mathbf{A}_{1}\mathbf{A}_{2}) \mathbf{H}^{\top}\mathbf{H}(\mathbf{B}_{1}\mathbf{B}_{2})=(\mathbf{A}_{1} \mathbf{A}_{2})(\sum_{m=1}^{M}\mathbf{\bm{h}}_{m}\otimes\mathbf{\bm{h}}_{m})( \mathbf{B}_{1}\mathbf{B}_{2}),$ (1) $$

where
$\otimes$ denotes the outer product operator,
$\mathbf{A}_{1}\in\mathbb{R}^{d_{\rm out}\times d_{r}}$,
$\mathbf{A}_{2}\in\mathbb{R}^{d_{r}\times d_{h}}$,
$\mathbf{B}_{1}\in\mathbb{R}^{d_{h}\times d_{r}}$,
$\mathbf{B}_{2}\in\mathbb{R}^{d_{r}\times d_{\rm in}}$
are all learnable parameters,
and we set the dimension $d_{r}$ to be much smaller than $d_{\rm in}$, $d_{\rm out}$ and $d_{h}$ to keep the number of learnable parameters within an acceptable
range.

Dynamic Streaming Update
In practice, the context can arrive in chunks sequentially.
The matrix of hidden states $H_{t}$ at step $t$ is computed based on all previous context chunks $\Sigma(t-1)$. This hidden state is then used to generate an adapter for the current chunk $\Sigma(t)$, which, in turn, is also used to compute the hidden states for future context steps.
Based on Equation 1, to compute the adapter of $\Sigma(t)$ we need to concatenate
all hidden states (*i.e.,* $[\mathbf{H}_{1};\ldots;\mathbf{H}_{t}]\in\mathbb{R}^{(M_{1}+\cdots+M_{t})
\times d_{h}}$)
of the context chunks in $\Sigma(t)$ to generate the adapter, *i.e.,* $\mathbf{W}_{\Delta_{t}}=\mathcal{G}(\left[\mathbf{H}_{1};\ldots;\mathbf{H}_{t}
\right]).$

Fortunately, our formulation allows an efficient updating mechanism without
explicitly storing history hidden states, noting that

$$ $\displaystyle\mathbf{W}_{\Delta_{t}}=(\mathbf{A}_{1}\mathbf{A}_{2})([\mathbf{H }_{1};\ldots;\mathbf{H}_{t}]^{\top}[\mathbf{H}_{1};\ldots;\mathbf{H}_{t}])( \mathbf{B}_{1}\mathbf{B}_{2})=(\mathbf{A}_{1}\mathbf{A}_{2})(\sum_{i=1}^{t} \mathbf{H}_{i}^{\top}\mathbf{H}_{i})(\mathbf{B}_{1}\mathbf{B}_{2}).$ (2) $$

Thus, the update can be efficiently computed as

$$ $\displaystyle{\bf S}_{t}$ $\displaystyle\leftarrow{\bf S}_{t-1}+\mathbf{A}_{2}\mathbf{H}^{\top}_{t} \mathbf{H}_{t}\mathbf{B}_{1}$ (3) $\displaystyle\mathbf{W}_{\Delta_{t}}$ $\displaystyle\leftarrow\mathbf{A}_{1}{\bf S}_{t}\mathbf{B}_{2}$ (4) $$

where $\mathbf{H}_{t}\in\mathbb{R}^{M_{t}\times d_{h}}$ stores the hidden states for the $t$-th
context chunk, and the partial sum ${\bf S}_{t}\in\mathbb{R}^{d_{r}\times d_{r}}$ acts as the memory of history context chunks
with ${\bf S}_{0}$ initialized as all zeros.
Note directly storing
$\mathbf{W}_{\Delta_{t}}\in\mathbb{R}^{d_{\rm out}\times d_{\rm in}}$
or
$\sum_{i}\mathbf{H}_{i}^{\top}\mathbf{H}_{i}\in\mathbb{R}^{d_{h}\times d_{h}}$
would require much more memory because we control
$d_{r}\ll\min\{d_{\rm in},d_{\rm out},d_{h}\}$.

Our preliminary experiments find that this architecture exhibits some empirical instability because the generated matrix $W_{\Delta_{t}}$ can transform an input vector $\mathbf{\bm{x}}$ into a vector containing values with either extremely large or near-zero magnitudes, due to its skewed distribution of its singular values. In §2.4, we will explain how normalization can address the instability issue.

### 2.3 Learning to Update with Self-supervised Pretraining

To preserve the language modeling capability of the adapted models $\Theta_{\Sigma(t)}$ for $t\in\{1,2,\ldots\}$,
we pretrain the weight generator $\mathcal{G}$ using the next-token prediction loss
of $\Theta_{\Sigma(t)}$ in a self-supervised manner on web corpora.
In other words, the adapter generator is trained on top of the frozen base
model $\Theta_{\rm base}$ in an end-to-end fashion.
Specifically, we use two self-supervision pretraining tasks:
*reconstruction* and *completion*.

The reconstruction task (Ge et al., ) draws inspiration from autoencoders and aims to train
the weight generator $\mathcal{G}$ to embed contextual information into the
generated weights.
This process compresses the input context $(x_{1},\ldots,x_{m})$ into
a generated adapter, $\mathcal{G}(x_{1:m})$, which is subsequently used
to reconstruct the input.
Formally, this is accomplished by maximizing the log-likelihood of the input
tokens with the adapted LM, using weights updated from the same text:
$\mathcal{L}_{\rm reconstruction}(\mathcal{G})=\log P\left(x_{1},\ldots,x_{m}
\mid\Theta_{\rm base}+\mathcal{G}({x_{1:m})}\right).$

The completion task (Zhang et al., ; Kim et al., ) trains the adapted LM to generate the continuation of the given context.
The goal is to maximize the log-likelihood of tokens $x_{m+1},\ldots,x_{n}$, which represent the continuation of the context $x_{1},\ldots,x_{m}$ in the dataset:
$\mathcal{L}_{\rm completion}(\mathcal{G})=\log P\left(x_{m+1},\ldots,x_{n}\mid
\Theta_{\rm base}+\mathcal{G}(x_{1:m})\right).$

We observe using both of the task can make the generated adapter memorize and
utilize the contextual information. Similar to prior work
(Ge et al., ), the generator is trained to maximize the sum of the
objective functions of the two task:

$$ $\max_{\mathcal{G}}\mathcal{L}_{\rm reconstruction}(\mathcal{G})+\mathcal{L}_{ \rm completion}(\mathcal{G})$ (5) $$

### 2.4 Normalization for Generated Weights

In preliminary experiments, we find that using the naive outer product for generating weights led to instability during training, causing convergence issues. When multiplying the generated matrix with the input vector, the resulting output can either diminish to near-zero or grow excessively large.

To address this instability, we introduce normalization into the formulation, *i.e.,*

$$ $\mathbf{W}_{\Delta_{t}}\leftarrow\mathbf{A}_{1}\operatorname{norm}(\mathbf{S}_ {t})\mathbf{B}_{2}=\mathbf{A}_{1}\operatorname{norm}\left(\mathbf{A}_{2}\sum_{ i=1}^{t}\left(\mathbf{H}_{i}^{\top}\mathbf{H}_{i}\right)\mathbf{B}_{1}\right) \mathbf{B}_{2}.$ (6) $$

Our pilot experiments find that normalization based on singular value
decomposition (SVD) is particular effective, among other normalization
strategies.

SVD Normalization
The SVD normalization technique ensures the singular values of the outer product are normalized to 1. Given a matrix $\mathbf{M}$, we define SVD normalization as:

$$ $\operatorname{norm}(\mathbf{M})=\mathbf{U}\mathbf{V}^{\top},$ (7) $$

where $\mathbf{M}=\mathbf{U\Sigma V}^{\top}$ is the SVD factorization. This normalization resets the positive singular values of the matrix to one, preventing the vectors from excessively shrinking or exploding.

Low-Rank SVD and LoRA
An additional benefit of SVD normalization is that it can naturally produce low-rank matrices. Instead of performing a full-rank decomposition, we approximate the input matrix with a rank-$r$ SVD decomposition, where $r$ is a hyperparameter set in advance.
Consequently, the matrix can be written as the product of two low-rank matrices, similar to a LoRA adapter (Hu et al., ):

$$ $\displaystyle\mathbf{W}_{\Delta_{t}}$ $\displaystyle=\mathbf{A}_{1}\operatorname{norm}(\mathbf{S}_{t})\mathbf{B}_{2}$ (8) $\displaystyle=(\mathbf{A}_{1}U(\mathbf{H}))(V^{\top}(\mathbf{H})\mathbf{B}_{2}),$ (9) $$

where $U(\mathbf{H})$ and $V(\mathbf{H})$ are the matrices resulting from SVD normalization. This low-rank approximation reduces both computational cost and memory usage.

## 3 Experiments Settings

We experiment with using both Mistral-7B-Instruct (v0.2)
(Jiang et al., ) and Llama2-7B-Chat
(Touvron et al., ) as the base LMs.
For efficiency, our main experiments train adapter generators to only update the
output projection layers of the multi-head attention unit in Transformer.
We study a more capable implementation in §5 and defer the
full exploration of other modules for future work.

Hyperparameters
The intermediate dimension $d_{r}$ and SVD rank $r$ are set to 1,024 and 128, respectively.
Approximately, this leads to 500 million parameters for the generator, with the
generated adapter of 32 million parameters.

Training
Following the standard training pipeline of LM development
(Jiang et al., ; Touvron et al., ), the training of our
adapter generator includes a pretraining phase described in
§2.3 followed by instruction tuning.
For pretraining, we randomly sample 1 billion tokens from SlimPajama (Soboleva et al., ) which are split into segments of 8,192 tokens each.
For instruction tuning, we use a mix of tasks such as question answering, in-context learning, and general instruction following,
which we ensure that there is no overlap with downstream tasks, with a detailed list provided in the appendix.
The context is divided into chunks of 1,024 tokens to utilize the dynamic
updating mechanism described in §2.2.

## 4 Main Results

We evaluate GenerativeAdapter on three representative scenarios where contextualizing
pretrained LMs is crucial, *i.e.,* acquiring new factual knowledge (§4.1),
learning from demonstrations (§4.2),
and personalizing to individual users (§4.3).

### 4.1 Document-based Question Answering with Varying Context Length

The factual knowledge stored in the parameters of a LM remains static after pretraining.
Here, we consider the scenario where the model needs to adapt to new knowledge based on documents.
After adaptation, it is expected to correctly answer information-seeking questions about these documents.

Setup and Baselines
To evaluate the fact recall ability of the adapted model, we use two question answering (QA) datasets, SQuAD (Rajpurkar et al., ) and StreamingQA (Liska et al., ), where each test case consists of a passage and a corresponding question about some information from that passage.
To analyze the impact of context length on performance, we conduct an evaluation using contexts of varying lengths.

We divide the documents in the corresponding test set evenly into groups, with
each group having an average length of $k$ tokens ($k\in\{$512, 1K, 2K, 4K,
8K, 16K, 32K$\}$). Thus, the model should contextualize on the article in each
group and evaluate fact recall by the question associated with the articles. The
QA accuracy is evaluated by comparing the generated output with the gold answer
for all questions associated with the documents within the group.
Following Rajpurkar et al. (),
F1 score is used as the metric for evaluation.

We also analyze the computational and storage requirements of GenerativeAdapter, which comprises three phases: general-purpose pretraining, contextualization, and inference. The generator is pretrained once and can subsequently be used for any task. During the contextualization phase, GenerativeAdapter encodes the context into an adapter with a single forward pass. In the inference phase, the adapted model generates responses based on the input. Beyond the LM parameters, the extra storage required includes the parameters of the generative adapter.

Here, we consider both full parameter fine-tuning and full context prompting using the same base model as baselines.
For fine-tuning, we consider two variants.
The first approach, supervised fine-tuning (SFT), trains the base model exclusively on a training set of question-answer pairs sourced from articles distinct from those in the test set.
The second variant, known as continual pretraining (CPT), involves first training the base model on all documents in the test set, followed by further adaptation through SFT using the the training set of question-answer pairs.
During inference, we evaluate the fine-tuned model in a closed-book manner,
*i.e.,* the model is tasked with directly producing the answer to a given question.
For prompting, we simply concatenate all documents in the group as a single context and prompt the base model to respond accordingly.
Specifically, for Llama2-7B-Chat, if the context length exceeds the maximum limit of 4K tokens, we truncate the prompt to include only the last 4K tokens.
For GenerativeAdapter, we create an adapted model for each document group, which is similar to how the context is encoded as prompting.
After that, the adapted model is asked to answer the question again in a closed-book fashion, akin to fine-tuning.

Figure: (a) Mistral-7B-Instruct
Refer to caption: x2.png

Figure: Figure 3: Computation and storage requirements for GenerativeAdapter and baseline methods on StreamingQA. For GenerativeAdapter, the context is converted into an adaptor during contextualization and then stored for inference. For the prompting method, the key-value (KV) cache can be generated during contextualization and reused during inference.
Refer to caption: x5.png

Results
We present the QA accuracy results for SQuAD and StreamingQA and the computation costs for StreamingQA in Figure 2 and Figure 3, respectively. Both fine-tuning methods (SFT and CPT) are evaluated in a closed-book manner, resulting in constant QA performance regardless of varying context lengths. In contrast, both GenerativeAdapter and prompting are evaluated on varying context lengths, where recalling facts can become more difficult as the context length increases.

As expected, both our method and prompting achieve improved QA performance by using relevant contexts compared to supervised fine-tuning baselines. Notably, GenerativeAdapter is highly effective when the context is relatively short ($<$ 1K tokens). Moreover, it avoids the additional inference overhead associated with prompting, which requires attention computation over the context input regardless of using key-value (KV) caches (illustrated by the green lines in Figure 3). This overhead issue worsens with longer contexts.

In most cases, GenerativeAdapter outperforms CPT, especially when the context length is less than 8K tokens. Importantly, although both approaches adapt model parameters using documents, GenerativeAdapter requires preprocessing time (forward passes only) that is orders of magnitude smaller than CPT (which involves multiple forward and backward passes), as demonstrated in Figure 3.

Figure: (a) Mistral-7B-Instruct
Refer to caption: x7.png

### 4.2 In-Context Learning with varying in-context examples

In the prompting paradigm, one emerging ability of pretrained LMs is that they can perform a task with a few task-specific input-output examples as context on unseen cases, also known as in-context learning (Brown et al., ).
Here, we are interested to see whether GenerativeAdapter can provide further benefits in enhancing the base LM’s in-context learning ability.

Setup and Baselines
We conduct experiments using MetaICL (Min et al., ), consisting of 26 test tasks.
We also ensure that none of these test tasks were seen during the training of adapter generator.
For each task, we use 1, 2, 4, 8, and 16 demonstrations randomly sampled from the corresponding development split following the MetaICL evaluate pipeline.
To reduce evaluation variance, we repeat the sampling process five times for each few-shot setting.
We report separate average accuracy for classification and non-classification tasks.
For classification tasks, achieving high accuracy requires the model to learn both the candidate options and the input-output relationships from the provided examples. For non-classification tasks, the model also needs to learn the output style.

We consider three baselines:
1) zero-shot prompting with the base LM where only task instruction is provided;
2) standard few-shot prompting with the base LM (Min et al., ) where each test case is prepend with those few-shot examples;
and 3) fine-tuning the base LM on each evaluation task using 16 input-output pairs, corresponding to the maximum number of shots in our evaluation.

Results
Figure 4 summarizes the results for MetaICL with various number of in-context examples for both classification and non-classification tasks.
The performance of fine-tuned models (blue lines) and zero-shot prompt baselines (grey lines) is evaluated without demonstrations, resulting in constant performance across different numbers of shots. While fine-tuned models generally achieve higher accuracy on classification tasks, their performance on non-classification tasks is lower. We speculate that the few-shot setting (16 shots) is insufficient for the model to learn the desired output style through fine-tuning. In contrast, GenerativeAdapter outperforms few-shot prompting in most cases, with more significant improvements observed in the more challenging non-classification tasks, where the model must adapt to specific output styles. This indicates that the generated adapter not only retains the in-context learning ability but also enhances the base model.

### 4.3 Personalization

Using LMs to analyze users’ behaviours and memorize their preferences is the key to unlocking a tailored and engaging user experience, *i.e.,* personalized LMs.
Towards this goal, we focus on evaluating the LM’s ability to memorize user information in conversations.

Setup and Baselines
We use the Multi-Session Conversation (MSC) dataset (Xu et al., ) for our experiments, following Packer et al. ().
Each test case comprises a multi-session human-human conversation between two participants, along with a question regarding information mentioned within the conversation.
The average length of the conversational context is 2.5K tokens, which makes it inefficient to prompt the model repeatedly with the entire conversation history for the same user.
Similar to document-based QA (§4.1), we evaluate the model quality using the F1 score by comparing the generated answers to the ground truth.
We also report computation and memory costs.
Here, we use Mistral-7B-Instruct as the base LM.

As baselines, we include both closed-book and full-conversation prompting based on the base LM, where the former involves random guesses and the latter incurs higher computation and memory costs by storing the entire long conversation.
We also include the state-of-the-art prompt compression method, UltraGist (Zhang et al., ), which reduces the context into fewer token embeddings, thereby saving computation and memory costs.

**Table 1: Performance comparison on MSC. A higher F1 indicates better performance, and lower inference computation and extra storage costs are preferable. For Ultragist (Zhang et al., ), fewer compressed tokens (noted in parentheses) correspond to lower computation and memory costs.**
| Model | F1 | Inference<br>Computation<br>(TFLOPS) | Extra<br>Storage<br>(M floats) |
| --- | --- | --- | --- |
| Closed-book | 8.1 | 0.505 | 0 |
| Full-conversation Prompting | 66.0 | 2.059 | 128+ |
| Ultragist (64 Tokens) | 26.5 | 0.514 | 4 |
| Ultragist (128 Tokens) | 32.2 | 0.552 | 8 |
| Ultragist (256 Tokens) | 38.3 | 0.627 | 16 |
| Ultragist (512 Tokens) | 40.8 | 0.772 | 32 |
| Ultragist (1K Tokens) | 44.4 | 1.067 | 64 |
| Ultragist (2K Tokens) | 42.4 | 1.658 | 128 |
| Generative Adapter | 40.2 | 0.505 | 32 |

Results
The results on MSC are summarized in Table 1.
As expected, the closed-book approach, which does not memorize any user information performs very poorly.
In contrast, methods that utilize proper user conversations as context can accurately recall user information, achieving reasonable answer accuracy.
Although using the entire conversation leads to better accuracy, full conversation prompting incurs significant computation and storage costs, *i.e.,* 4x those of GenerativeAdapter.
Such costs are highly undesirable for personalizing LMs for individual users, especially since most computations occur on edge devices without power GPUs.
Comparing to UltraGist at the same level of storage cost (compressed into 512 tokens), GenerativeAdapter further reduces inference cost without performance drop.
In real world scenarios with many queries from the same user, the benefits of our method are even more pronounced.

## 5 Analysis

### 5.1 Ablation Study

Here, we examine how different design choices with GenerativeAdapter affect model performance.
Specifically, we train adapter generators for Mistral-7B-Instruct under various configurations and evaluate them using two metrics—reconstruction perplexity and completion perplexity—on a validation set drawn from the same distribution as the pretraining corpus.
As we observe in our preliminary study, the quality of the resulting adapter generator is highly correlated with these metrics.
The results are presented in Table 2, where the default setting is described in §2.

Mix of both pretraining tasks is necessary.
As we can see, relying solely on one task does not yield good perplexity on the validation set for both metrics.
In particular, training with only reconstruction task results in a significant drop in completion perplexity, indicating a loss of general language modeling capabilities and overfitting to memorization.
Thus, the completion task acts as data regularization for GenerativeAdapter, enabling the model to distill contexts into generated adapters and effectively utilize them in future predictions.

SVD is a more effective normalization.
We explore an alternative normalization for Equation 6 using the Frobenius norm, defined as $\operatorname{norm}(M)=M/\|M\|_{F}$, where $\|M\|_{F}=\sqrt{\sum_{i,j}M_{ij}^{2}}$. Compared to SVD used in our default setting, the Frobenius norm is computationally simple and helps bound the matrix scale.
However, our results indicate that the Frobenius norm is not as effective.
Observing substantial disparities in the scale of singular values, we hypothesize that applying the Frobenius norm may unnecessarily shrink certain directions, reducing the model’s expressiveness.

More update parameters lead to better performance.
By default, we add the adapter into the output projection layer of the attention module.
We also experiment with adding the adapter to the down projection layer within the feedforward module, which introduces more update parameters (3x those of default).
We observe that placing the adapter on the feedforward layer indeed leads to slightly improved reconstruction and completion perplexities.
Due to computational constraints, a more thorough exploration was not feasible and we leave this for future investigation.

**Table 2: The validation set perplexity of the pretrained model under different design choices.**
| Factor | Setting | Reconstruction<br>Perplexity | Completion<br>Perplexity |
| --- | --- | --- | --- |
| - | Default | 1.75 | 7.40 |
| Pretraining<br>Task | Reconstruction Only | 1.75 | 34.34 |
| Completion Only | 6.38 | 6.71 |  |
| Normalization | Frobenius | 7.72 | 7.32 |
| Module | Feedforward | 1.68 | 7.26 |

## 6 Related Works

Fast Weights:
Our proposed method is closely related to the idea of “fast weights”
(Hinton & Plaut, ; Ba et al., ; Schlag et al., ), which makes the model
weights being adaptive to the model input.
Context-dependent fast weight programmers (FWPs) introduced by
Schmidhuber (; ) use a slow network with
slow weights to reprogram the fast weights of the corresponding fast
network.
Schlag et al. () point out that self-attention without softmax and other
linear Transformer variants
(Tsai et al., ; Katharopoulos et al., ; Choromanski et al., ; Peng et al., )
can be viewed as FWPs.
Clark et al. () propose fast weight layers which are added on top of
the Transformer model after the last attention layer for language modeling.
Different from previous work mainly focusing on specific tasks, our goal is to enhance frozen pretrained LMs with fast associative memory for general language processing.
Instead of using a slow network to program a separate fast model, our method can be viewed as a self-programming model, *i.e.,* context encoded by the base LM is used to update the base LM itself.

Adapting LMs via Meta-Learning:
One line of work focuses on adapting pre-trained LMs for an online stream of
documents using meta-learning.
Observing that naively fine-tuning on the documents using the negative
log-likelihood loss is not effective for downstream question answering,
Hu et al. () propose context-aware meta-learned loss scaling to
re-weight the loss for individual tokens based on their importance during the
online fine-tuning.
Tack et al. () use a meta-learned amortization
network to directly predict parameter efficient fine-tuning modulations of the
base LM for individual context documents.
The modulations are then aggregated into a single output for downstream question
answering.
Unlike those methods that typically require a nested training loop, our adapter generator augments pretrained LMs and our model can be trained in an end-to-end fashion with self-supervised objectives.

Parameter-Efficient Fine-Tuning (PEFT):
GenerativeAdapter employs a low-rank adapter akin to LoRA (Hu et al., ),
which was originally designed for PEFT.
Several derivatives of LoRA exist such as AdaLoRA (Zhang et al., )
and DoRA (Liu et al., ), along with various other PEFT strategies such as
serial adapters (Houlsby et al., ) and prefix tuning
(Li & Liang, ).
A thorough survey of PEFT methods is presented by
Han et al. ().
Most work focuses on task-specific fine-tuning scenarios.
Instead, GenerativeAdapter is a general LM and does not require a downstream dataset for adaptation.

## 7 Conclusion

In this work, we introduce GenerativeAdapter, a method for efficiently adapting pretrained
LMs on-the-fly using test-time context through forward passes only.
We design a adapter generator network on top of frozen pretrained LMs,
transforming text contexts into updated model parameters.
Our adapter generator network is trained end-to-end with the frozen pretrained
LM using two self-supervised tasks on web corpora.
We assess GenerativeAdapter across three scenarios that benefit from contextualizing
pretrained LMs: acquiring new factual knowledge, learning from demonstrations,
and personalizing to individual users.
Our experiments indicate that GenerativeAdapter reduces information loss compared to
full-context prompting in retaining factual knowledge from context documents.
Additionally, the model effectively adapts to new task instructions when
learning from demonstrations.
Finally, GenerativeAdapter achieves comparable fact recall performance to efficient
prompting methods while utilizing lower inference-time computation, showcasing
its feasibility for user-specific adaptation in personalization scenarios.
For future work, it would be interesting to further explore scaling up the adapter generator, such as by integrating adapters into additional layers, and to investigate more selective update rules (Schlag et al., ).

## Appendix A Dataset Details

#### Pretraining.

We pretrain our models using a randomly sampled subset of 1B tokens from the SlimPajama corpus. For validation, we sample an additional 100 segments, each containing 2K tokens, from the same corpus.

#### Instruction Tuning.

We perform instruction tuning using a combination of question answering, in-context learning, and instruction following datasets, following prior studies (Lin et al., ; Ge et al., ; Zhang et al., ).

## Appendix B Training Setup

#### Implementation

We empirically found that normalization is crucial for GenerativeAdapter to function effectively. For SVD normalization, we implemented it using torch.svd_lowrank(), setting the number of iterations to 1.

GenerativeAdapter is able to generate the adaptors for prefixes of chunks simultaneously by processing the context chunks in parallel.
The computation proceeds by processing the hidden states of each Transformer block for all context chunks layer by layer. Given the hidden states of $\Sigma(1),\ldots,\Sigma(t)$ from the $(l-1)$-th Transformer block, denoted by $H^{(l-1)}_{1:t}$, we first compute the accumulated outer product $S^{(l)}_{1:t}$ using Equation 2. We then normalize this outer product to obtain the additive matrix $W^{(l)}_{\Delta,1:t}$ using Equation 6, and finally get the output of the $l$-th Transformer block $H^{(l)}_{1:t}$ by the base model.

#### Pretraining.

For Mistral-7B-Instruct (hereafter referred to as Mistral), we use a learning rate of $5\times 10^{-5}$, and for Llama2-7B-Chat (Llama2), we use $1\times 10^{-4}$. We apply a weight decay of 0.01 and no dropout. The adapter added to the base model are scaled by $1/16$ for Mistral and $1/8$ for Llama2. We employ a WarmupDecayLR learning rate scheduler with a 100-step warmup and use the Adam optimizer. The global batch size is set to 8. Pretraining the adapter generator on 1B tokens takes approximately 20 hours using 8 NVIDIA H100 GPUs.

#### Instruction Tuning.

For instruction tuning, we largely follow the same configurations as in pretraining, with some adjustments. We set the learning rate to $5\times 10^{-5}$ for both Mistral and Llama2 models. We train the models for 2 epochs and use a batch size of 32.

## Appendix C Experiment Setup

#### Document-based QA.

We set up experiments for document-based question answering (QA) using both supervised fine-tuning and continuous pretraining. For supervised fine-tuning on question-answer pairs, we train on the training split of each dataset, evaluate on a validation set, and employ early stopping when the validation loss increases. We use a learning rate of $1\times 10^{-5}$ and a global batch size of 64. For continuous pretraining, we train for 8 epochs using the log-likelihood of the document as the training loss, with learning rates of $1\times 10^{-5}$ for Mistral and $3\times 10^{-5}$ for Llama2. Each passage is treated as a training sample, and we use a global batch size of 16.

For closed-book prompting and in-context prompting, we apply an instruction template to encourage the model to generate a short answer. The prompts are shown in Figure 7.

#### In-Context Learning.

We explore in-context learning using both fine-tuning and prompting methods. For fine-tuning, we conduct task-specific fine-tuning on 16 samples for each dataset. We use a learning rate of $5\times 10^{-6}$ for Mistral and $1\times 10^{-5}$ for Llama2. A validation set of 16 samples, disjoint from the training set, is collected from the same dataset. We train the model for a maximum of 40 epochs, employing early stopping if the validation loss increases for three consecutive epochs.

For in-context prompting, we observe that omitting additional instructions yields better performance for Mistral, whereas adding an instruction template improves performance for Llama2. The prompts are shown Figure 7.

## Appendix D Additional Results

We provide the detailed results for document-based QA and in-context learning evaluation in Figure 6, Figure 5, and Table 5.

**Table 3: Statistics of data used in the instruction tuning.**
| Type | Dataset | #Docs | #Instructions | Context len | Instruction len | Response len |
| --- | --- | --- | --- | --- | --- | --- |
| Question Answering | COQA (Reddy et al., ) | 1798 | 57.2 | 1083.5 | 5.5 | 2.7 |
| DROP (Dua et al., ) | 1379 | 38.4 | 848.6 | 11.0 | 1.4 |  |
| NarrativeQA (Kočiský et al., ) | 1047 | 29.4 | 574.5 | 8.5 | 4.4 |  |
| PubMedQA (Jin et al., ) | 1000 | 1.0 | 200.2 | 12.9 | 40.7 |  |
| Quail (Rogers et al., ) | 560 | 16.2 | 332.7 | 8.7 | 4.9 |  |
| MS MARCO (Bajaj et al., ) | 4832 | 16.5 | 1152.1 | 6.0 | 14.0 |  |
| In-context Learning | MetaICL (Min et al., ) | 11888 | 3.5 | 1776.8 | 84.3 | 2.9 |
| Instruction Following | BookSum (Kryscinski et al., ) | 2914 | 1.0 | 1158.6 | 7.0 | 205.3 |
| PwC (Ge et al., ) | 13102 | 12.4 | 348.1 | 10.3 | 23.3 |  |

**Table 4: Training and test datasets of MetaICL.**
| Train | piqa, hate_speech_offensive, google_wellformed_query, social_i_qa, circa, quoref, glue-sst2, scitail, emo, cosmos_qa, freebase_qa, ag_news, art, paws, kilt_ay2, glue-qnli, quail, ade_corpus_v2-classification, sciq, hatexplain, emotion, glue-qqp, kilt_fever, kilt_nq, dbpedia_14, kilt_zsre, hellaswag, squadwith_context, hotpot_qa, glue-mnli, ropes, squad-no_context, kilt_hotpotqa, discovery, superglue-record, race-middle, race-high, lama-trex, swag, gigaword, amazon_polarity, biomrc, tab_fact, tweet_eval-emoji, tweet_eval-offensive, tweet_eval-sentiment, tweet_qa, imdb, lama-conceptnet, liar, anli, wiki_qa, kilt_trex, wikisql, wino_grande, wiqa, search_qa, xsum, yahoo_answers_topics, yelp_polarity, yelp_review_full |
| --- | --- |
| Test | quarel, financial_phrasebank, openbookqa, codah, qasc, glue-mrpc, dream, sick, commonsense_qa, medical_questions_pairs, quartz-with_knowledge, poem_sentiment, quartz-no_knowledge, glue-wnli, climate_fever, ethos-national_origin, ethos-race, ethos-religion, ai2_arc, hate_speech18, glue-rte, supergluecb, superglue-copa, tweet_eval-hate, tweet_eval-stance_atheism, tweet_eval-stance_feminist |

Figure: Figure 5: In-context learning evaluation of GenerativeAdapter, based on Llama2-7B-Chat, across 26 test datasets from MetaICL.
Refer to caption: x10.png

Figure: Figure 6: In-context learning evaluation of GenerativeAdapter, based on Mistral-7B-Instruct, across 26 test datasets from MetaICL.
Refer to caption: x12.png

**Table 5: All results of the QA accuracy on SQuAD and StreamingQA.**
|  |  |  | F1 |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Model | Dataset | Methods | 512 | 1K | 2K | 4K | 8K | 16K | 32K |
|  |  | Zero-Shot Prompting | 10.8 |  |  |  |  |  |  |
|  |  | Supervised Fine-tuning | 20.7 |  |  |  |  |  |  |
|  |  | Continuous Pretraining | 30.0 |  |  |  |  |  |  |
|  |  | In-context Prompting | 45.4 | 44.9 | 43.6 | 42.6 | 42.5 | 38.6 | 35.1 |
|  | SQuAD | GenerativeAdapter | 48.8 | 43.0 | 39.9 | 35.9 | 33.8 | 30.3 | 28.0 |
|  |  | Zero-Shot Prompting | 13.6 |  |  |  |  |  |  |
|  |  | Supervised Fine-tuning | 19.5 |  |  |  |  |  |  |
|  |  | Continuous Pretraining | 22.2 |  |  |  |  |  |  |
|  |  | In-context Prompting | 47.2 | 48.7 | 48.1 | 48.7 | 48.0 | 46.0 | 39.3 |
| Mistral | StreamingQA | GenerativeAdapter | 51.5 | 49.3 | 44.7 | 40.9 | 36.7 | 32.7 | 32.0 |
|  |  | Zero-Shot Prompting | 14.6 |  |  |  |  |  |  |
|  |  | Supervised Fine-tuning | 20.7 |  |  |  |  |  |  |
|  |  | Continuous Pretraining | 23.9 |  |  |  |  |  |  |
|  |  | In-context Prompting | 64.8 | 60.6 | 55.4 | 44.9 | 25.2 | 9.6 | 6.4 |
|  | SQuAD | GenerativeAdapter | 36.2 | 32.5 | 31.0 | 28.9 | 28.2 | 24.9 | 23.6 |
|  |  | Zero-Shot Prompting | 18.0 |  |  |  |  |  |  |
|  |  | Supervised Fine-tuning | 18.9 |  |  |  |  |  |  |
|  |  | Continuous Pretraining | 20.5 |  |  |  |  |  |  |
|  |  | In-context Prompting | 61.2 | 61.3 | 58.0 | 46.8 | 27.8 | 17.5 | 11.6 |
| LLama2 | StreamingQA | GenerativeAdapter | 42.9 | 37.8 | 34.4 | 32.6 | 28.7 | 26.0 | 25.7 |

Figure: Figure 7: Prompts used in the document-based QA and in-context learning evaluation.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 4
questions: - From what I understand, generative adapters are Hypernetworks [[Ha et al](https://arxiv.org/abs/1609.09106)], but that hasn't been discussed anywhere. Do the authors agree with this assessment, and if so, how would they contextualize similar work using hypernetworks for continual learning [[Oswald et al](https://arxiv.org/abs/1906.00695), [Vladymyrov et al](https://arxiv.org/abs/2301.04584)]?
- Why is the continual pretraining baseline so much lower than the generative adapter baseline? This sticks out and is surprising to me, and makes me think it is poorly tuned.
- I think a valuable baseline (especially since continual pretraining seems to underperform) might be using the generative adapter loss function (Eqn 5) but optimize the weights of the model directly instead of the generative adapter. If I understood correctly, I don't see any reason why this shouldn't be possible, and if generative adapter is in the ballpark of this baseline, it's a positive signal. Do the authors agree with this assessment?
soundness: 3
strengths: - The method is simple and well-motivated.
- The method works nicely in a streaming setting to continually update weights using lower number of FLOPs due to being able to reuse weights computed at the previous time step of the stream.
- The paper is clearly written and easy to follow.
summary: This paper proposes a neural module called generative adapter, which once trained can accept a stream of input contexts to adapt transformers on the fly. The paper shows that generative adapters work well at lower computational cost than in-context learning on several NLP tasks.
weaknesses: - I think the Ultragist baseline should be reported in $\S 4.1$ and  $\S 4.2$, not just $\S 4.3$. Gisting is a natural comparison and I am glad to see it compared against in some form, but I think it could be reported more widely in the paper.
- Using llama-2-7b doesn't provide much value in my opinion. The 4k context length makes Figure 2b hard to reason about, and I don't see why all llama-2-7b experiments couldn't be done with llama-3.1-8b instead for similar cost.

---

confidence: 2
contribution: 3
initial_rating: 6
presentation: 2
questions: How does GenerativeAdapter perform in scenarios where the context information is dynamic or frequently changing, such as real-time conversational agents? It would be interesting to see if the method’s efficiency and adaptability hold in continuously evolving contexts.
soundness: 3
strengths: 1. The paper presents an efficient approach to adapting LLMs in real-time by using a lightweight, bilinear adapter generator. This mechanism effectively reduces computational costs associated with full model fine-tuning and inference-time overhead due to prompting, which is particularly advantageous for applications on edge devices.

2. The authors evaluate *GenerativeAdapter* across diverse, practical contexts such as document-based knowledge acquisition, in-context learning, and user personalization. These evaluations, especially in a personalized setting, demonstrate the adaptability of the method and its effectiveness in various contexts that align with real-world demands.
summary: The paper introduces GenerativeAdapter, a method for adapting pretrained large language models (LLMs) using lightweight, parameter-efficient adapters generated by a self-supervised generator network. This approach aims to enhance LLM adaptability without incurring the computational cost of fine-tuning or the inference overhead of prompting. Evaluations are conducted across three main areas: factual knowledge acquisition, learning from demonstrations, and user personalization, showing that GenerativeAdapter is effective in reducing computational load and increasing model adaptability to various contexts.
weaknesses: 1. The authors provide detailed descriptions of the model architecture, adapter generation mechanism, and datasets. However, hyperparameter settings and implementation details could be expanded, especially regarding stability mechanisms like SVD normalization, which could impact reproducibility. It is unclear if the code or models will be open-sourced for verification.
2. The analysis of *GenerativeAdapter*’s performance in different scenarios is comprehensive; however, the lack of prompt  engineering effort for baselines could introduce an unfair comparison. A clarification on the extent of prompt engineering would ensure a fairer assessment.
3. While *GenerativeAdapter* achieves superior performance over specific context lengths and adaptation scenarios, claims of general "efficiency" for "personalization" are strong, given the specific constraints of the study. The authors should emphasize that observed benefits apply to controlled experiments rather than broad generalizations.
4. The paper introduces concepts like "test-time contextualization" and "parameter-efficient" without specific definitions, which could hinder comprehension for readers less familiar with these terms. Explicit definitions or brief explanations in the methodology section would help maintain clarity.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: - Is there a suspected reason why the unnormalized adaptor contains extreme values?
- In the formulations in section 2.3, there is a  G(x_{1:m}). In this case, shouldn't the hidden states input to the generator, not the input tokens?
-  In the completion loss formulation, shouldn't the output just x_{m+1}, not x{m+1},...,x_n?
soundness: 2
strengths: - The paper addresses the critical challenge of adapting LMs to incorporate new knowledge effectively, a fundamental issue since LMs can only utilize the information they were originally trained on. 
- It introduces an interesting approach of training another generator to generate an adaptor and optimize adaptation cost
summary: The paper proposes GenerativeAdapter, which is a method to introduce unseen knowledge to LMs without finetuning cost or extra prompts. It generates a new adapter for the target unseen knowledge and utilize it to adapt the LMs.
weaknesses: - Some of the important experiments are either missing or have questionable setting
    - It is suspicious that the the settings for experiment 4.1 are fair for all of the baselines. It seems like the knowledge that each baseline has seen are different.
    - In section 5, the paper mentions the preliminary study shows the quality of the adapter generator is highly correlated with the final evaluation metrics. It would have been better if the results that support this claim is included in the paper. Many times, the training objective is not exactly aligned with the downstream task.
    - It would be better if the experiment to see how the adaptor affects the original ability of the LMs is included in the paper.

---

confidence: 4
contribution: 3
initial_rating: 8
presentation: 4
questions: I don't have any questions.
soundness: 4
strengths: 1. The topic is important. 
2. The idea of this paper is novel and clean, inspired by fast weights. 
3. The experimental results show the effectiveness of this method. it is impressive to see that model is able to inject 32k tokens into the parameters.
summary: This paper proposes a novel method to integrate the information into the parameters. The experimental results are comprehensive and demonstrate the effectiveness of this method.
weaknesses: 1. **Missing related works**: GenerativeAdaptor adds the adaptor which is quite similar to memory-based methods, where people would add a memory module on the large language models [1,2]. Perhaps the paper should discuss some related papers and the correlations between them. 

2. **Size of the additional parameters**: As stated in line 264, GenerativeAdaptor introduces 500 million parameters, however, even using 500 million parameters cannot achieve comparative performance as In-context Prompting (as shown in Figure 2(a)). Consider saving all the tokens in the context, which leads to 32768 * 4096 = 134 million parameters. This is fine, as the proposed method has smaller computation cost than storing all the previous tokens, but this might be something to consider, as it shows that the parameters may not be fully utilized, or the information in the parameters might be overly sparse. 

[1] Memllm: Finetuning llms to use an explicit read-write memory   
[2] MemoryLLM: Towards Self-Updatable Large Language Models

Meta-review: This paper introduces GenerativeAdapter, a novel technique for adapting pre-trained Large Language Models (LLMs) to new tasks without requiring fine-tuning or prompting.  Instead of modifying the core LLM parameters, GenerativeAdapter trains a lightweight adapter module using a self-supervised generative network. This adapter can process a continuous stream of input contexts, enabling the transformer model to adapt on-the-fly. This approach offers significant advantages in terms of efficiency, reducing both training time (compared to fine-tuning) and inference time (compared to prompting).

The effectiveness of GenerativeAdapter is demonstrated through experiments on various tasks, including factual knowledge acquisition, learning from demonstrations, and user personalization.  While reviewers have praised the significance and novelty of this work, they have also raised concerns about the chosen baseline methods and potential similarities to existing techniques like hypernetworks and memory-based methods.

To strengthen the paper, I recommend the following additions:

Comprehensive Baseline Comparisons: Include a broader range of baseline methods to thoroughly evaluate the performance of GenerativeAdapter.
In-Depth Analysis: Provide a detailed analysis comparing and contrasting GenerativeAdapter with hypernetworks and memory-based methods, highlighting its unique contributions and advantages.

Decision: Accept (Poster)