PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model Patches
Authors: Rana Shahroz
Pingzhi Li
Sukwon Yun
Zhenyu Wang
Shahriar Nirjon
Chau-Wai Wong
Tianlong Chen
Venue: ICLR 2025 Poster (2025)
Keywords: Large Language Models
NLP
Efficiency
Fine-tuning
Efficient Fine-tuning
Portability
Primary area: other topics in machine learning (i.e., none of the above)

Abstract: As large language models (LLMs) increasingly shape the AI landscape, fine-tuning pretrained models has become more popular than in the pre-LLM era for achieving optimal performance in domain-specific tasks. However, pretrained LLMs such as ChatGPT are periodically evolved (i.e., model parameters are frequently updated), making it challenging for downstream users with limited resources to keep up with fine-tuning the newest LLMs for their domain application. Even though fine-tuning costs have nowadays been reduced thanks to the innovations of parameter-efficient fine-tuning such as LoRA, not all downstream users have adequate computing for frequent personalization. Moreover, access to fine-tuning datasets, particularly in sensitive domains such as healthcare, could be time-restrictive, making it crucial to retain the knowledge encoded in earlier fine-tuned rounds for future adaptation. In this paper, we present PORTLLM, a training-free framework that (i) creates an initial lightweight model update patch to capture domain-specific knowledge, and (ii) allows a subsequent seamless plugging for the continual personalization of evolved LLM at minimal cost. Our extensive experiments cover seven representative datasets, from easier question-answering tasks {BoolQ, SST2} to harder reasoning tasks {WinoGrande, GSM8K}, and models including {Mistral-7B,Llama2, Llama3.1, and Gemma2}, validating the portability of our designed model patches and showcasing the effectiveness of our proposed framework. For instance, PORTLLM achieves comparable performance to LoRA fine-tuning with reductions of up to 12.2× in GPU memory usage. Finally, we provide theoretical justifications to understand the portability of our model update patches, which offers new insights into the theoretical dimension of LLMs’ personalization.

Published as a conference paper at ICLR 2025
PORTLLM: PERSONALIZING EVOLVING LARGE LANGUAGE MODELS WITH FRAEE AND
PORTABLE MODEL PATCHES
Rana Muhammad Shahroz Khan'
Pingzhi Li
Sukwon Yun"
Zhenyu Wang'
Shahriar Nirjon! Chau-Wai Wong²
Tianlong Chen'
'The University of North Carolina Chapel Hill * Denotes Equal Contribution
NC State University
ABSTRACT
As large language models (LLMs) increasingly shape the AI landscape, fine- tuning pretrained models has become more popular than it was in the pre-LLM era for achieving optimal performance in domain-specific tasks. However, pre- trained LLMs such as ChatGPT are periodically evolved (iee., model parameters are frequently updated), making it challenging for downstream users with lim- ited resources to keep up with fine-tuning the newest LLMs for their domain application. Even though fine-tuning costs have nowadays been reduced thanks the innovations in parameter-efficien fine-tuning such low-rank adapta- tion (LoRA), not all downstream users have adequate computing for frequent per- sonalization. Moreover, access to fine-tuning datasets, partieularly in sensitive domains such as healthcare, can be time-restrictive, making it crucial to retain the knowledge encoded in earlier fine-tuned rounds for future adaptation. In this paper, we present PORTLLM, a training-free free framework that (i) creates an ini- tial lightweight model update patch capture domain-specific knowledge, and (ii) allows subsequent seamless plugging for the continual personalization of the evolved LLM at minimal cost. Our extensive experiments cover seven repre- sentative datasets, from easier question-answeringg tasks (BoolQ. SST2) to harder reasoning tasks (WinoGrande, GSM8K), and models including (Mistra 1B7 Llama2, Llama3. 1, and Gemma2) ) validating the portability of our designed model patches and showcasing the effectiveness of our proposed framework. For instance, PORTLLM achieves comparable performance to LoRA fine-tuning with reductions of up to 12.2x in GPU memory usage. Finally, we provide theoreti- cal justifications to understand the portability of our model update patches, which offers new insights into the theoretical dimension of LLMs' personalization.
INTRODUCTION
The rise of pretrained large language models (LLMs) has marked a significant paradigm shift in nat- ural language processing (NLP), particularly their ability adapt to specific domains and tasks. These models, such as GPT-4 (Achiam et al. (023). have achieved state-of-the-art performance by leveraging vast amounts of pretraining data (Antoniades et al.) 2024). However, pretrained LLMs often require adaptation for specialized domains where context-specific knowledge is critical (Wang al. 2022a Bommasani al.) 2021; Qiu et al., 2020). Hence, while pretraining provides a strong foundation, fine-tuning (e.g., personalization) is essential for specific domains. Fine-tuning bridges this gap by adapting pretrained models to specific tasks, enhancing their performance in domains
such healthcare, legal analysis, or scientific research (Min et al.) 2021 Wei et al. 2021 Ouyang cct al., 20222 Wang et al. 2202bb] Liu et al. 2022; Raffel et al.) 2020 Chen et al. 2024|] Gao et al, 2024b). For instance, fine-tuned models can more effectively recognize domain-
terminol- ogy, reason about complex relationships, and deliver more accurate and contextual appropriate re- sponses.
Much effort has been devoted to developing fine-tuning methods. Traditionally, fine-tuning would normally involve updating all the parameters of model. For example, in the case of Mistral

---

Published as a conference paper at ICLR 2025
7B Jiang et al. 2023). it would involve updating all 7 billion parameters. Typically, LLMs have billions of parameters, and so this process poses significant challenges in computational and memory requirements. To alleviate these constraints, many parameter efficient fine-tuning (PEFT) methods (Houlsby et al., 2019) have been proposed. One popular method is low-rank adapta- tion (LoRA) (Hu a ) which aims to estimate an update matrix AW using the product of two low-rank matrices A and B. However, although LoRA lowers the training complexity, it still requires tine-tunin large number of trainable parameters to reach a satisfactory performance. For example, LoRA fine-tuning Llama 2 13B (Touvron et al. 2023) variant would require up to eight A6000 GPUs with 48 GB VRAM each, for a very small batch size, hence imposing a considerable memory and computational burden.
Furthermore, as cloud-hosted LLMSs like ChatGPT (OpenAl 2022) and Gemini [Team et al. 2023) undergo periodic (biannual or more frequent) updates with newer data, the best performing model often renders previous versions outdated. For downstream users who have already invested in fine- tuning the previous models for domain-specific tasks, repeatedly fine-tuning or performing per- sonalization at every new update is highly impractical, as this process is not only computationally expensive but also time-consuming.
Beyond the computational and logistical hurdles, continual updates present another challenge for the downstream user: the lack of availability of the or finance, data access is regulated by privacy laws and potentially time-sensitive. For example, fine-tuning medical models on patient data requires strict adherence to ethical and legal guidelines, making it difficult to repeatedly fine-tune with every LLM release. As a result, repeatedly fine- tuning models on newer LLM releases is not a viable long-term strategy for many users, hindering the downstream users from harnessing the performance gains from the evolving nature of LLMs. In response to these challenges, a natural research question arises:
dataset. In domains such as healtheare
(2) How to leverage the personalized knowledge captured the first fine-tuned LLM model to update any evolving LLMs?
To address this question, we introduce PORTLLM, training-frree framework that enables scamless transfer of domain-specific knowledge across evolving models. PORTLLM leverages model patches derived from LoRA, allowing users port fine-tuned knowledge from one model iteration to another while preserving or even enhancing the performance of a downstream task. We show that our model patches are portable across model updates. If the downstream user fine-tuned version of the model that has long become obsolete, they can simply add the model patch to the newer model, maintaining or boosting their performance on the downstream task. PORTLLM eliminates the need for costly periodic fine-tuning, offering a scalable solution for maintaining task-specific performance across different model versions. Our contributions can be summarized as follows:
We introduce PORTLLM, training-free framework designed to transfer knowledge between different versions of an evolving LLM. Given two LLM versions, PORTLLM leverages task- specific model patches extracted from a fine-tuned LLM and seamlessly applies them to the evolved LLM. This process allows the updated LLM to achieve comparable, and in some cases improved, performance on downstream tasks without any need for fine-tuning.
Why do our model patches work? ? We address this question through both theoretical analysis and empirical investigation. Our findings reveal that certain terms in the model patch are effectively negligible, enabling us create simplified version of the patch that requires no training. Conse- quently. adding the simplified model patches alone is sufficient to achieve improved performance on the downstream task. Furthermore, we examine the impact of the pretraining dataset on down- stream tasks, demonstrating that our framework can harness the benefits of continued pretraining across different model updates and across different pretraining datasets including (OpenOrca, SlimOrca, OpenPlatypus, AlpacaGPT4].
We conduct extensive experiments across series of seven downstream tasks, including Question-Answering Tasks (BoolQ. SST-2) (Wang] 2018 Wang et al. 2019). Similarity and
Paraphrase Tasks {MRPC} (Wang 2018). Inference Tasks RTT, WNLI (Wang 2018), and Reasoning Tasks [WinoGrande, GSM8K Sakaguchi et al. 2021 Cobbe et al. 20211) To further demonstrate the robustness and broad applicability of our approach, we evaluate on multiple model architectures, such as Mistral-78 Llama2-7B. Llama3. 1- 1-8B, and Gemma2 9B. Additionally, explore the applicability of our method full-weight continued pretraining
2

---

Published as a conference paper at ICLR 2025
compared to LoRA-based continued pretraining and show the effectiveness of our method by quantifying the gains on GPU memory usage (of up to 12.2x reduction), GPU hours, and de- crease in the number of trainable parameters (to zero trainable parameters).
2 RELATED WORKS
Large Language Models (LLMs). LLMs have transformed natural language processing, enabling models to perform complex tasks with remarkable accuracy and generalization. Models like GPT- 3 (Brown al.) 2020), BERT (Devlin al. 2019), and T5 (Raffel et al. 2023) have set benchmarks across a range of NLP tasks, from translation and summarization to question answering and text generation (Vaswani et al. 2023 Zhang et al. 2020 Rajpurkar al., 2016). More recently, models like Llama Touvron et al. 2023 Dubey et al. 2024). Mistral (Jiang et al. (023), and Gemma (Team
al. 2024) have pushed the boundaries further by optimizing both performance and computational efficiency. LLama, Mistral, and Gemma represent recent advances in LLM architectures, each of- fering improvements in efficiency and performance. However, even with such improvements, the performance on domain-specific downstream tasks is subpar, making fine-tuning necessary. In this paper, we propose a training-fre solution that enables the seamless transfer of personalized knowl- edge across evolving LLMs, reducing the need for costly fine-tuning and enhancing accessibility.
Parameter Efficient Fine-Tumine Tuning (PEFT). The rapid growth in the size of pretrained LLMs has posed significant challenges for efficiently fine-tuning L.Mss to specific downstream tasks. To ad- dress this challenge, numerous PEFT methods have been developed, aiming to balance efficiency and accuracy. Early approaches focused on inserting trainable
networks placed between layers of the pretrained model (Houlsby et al. [219 Lin et al. (2020). Recent ad- vancements have led to more sophisticated adapter- based PEFT methods (Mahabadi et al. 2021 Pfeiffer et al.) 2020; Luo et al., 2023) including LMaaS (Sun et al., (022) for service-oriented adaptation and -dd-ttr (Huang et al. 2023) for retrieval-augmented fine-tuning. A notable example is LoRA (Hu et al. (021), which introduces trainable low-rank weight perturbations to the pretrained model, significantly reducing the number of parameters required for fine-tuning. LoRA's key innovation lies in its use of the product of two low-rank matrices to approximate weight changes. Building upon this concept, several methods have emerged including Q-LoRA (Dettmers) et al. 2023). CombLMM (Ormazabal et al., 2023), and IPA (Lu et al. 2023). Concurrently, prompt- based learning methods have demonstrated effectiveness across various NLP tasks. Methods such as prompt-tuning (Lester al. 2021), prefix-tuning (Li & Liang| (02T) and more recent approaches like proxy-tuning (Liu et al.] 2024) and BBox-Adadpte (Sun et al. 2024) incorporate learnable continuous embeddings into the model's hidden states. They condition the frozen model to adapt to specific tasks without modifying the underlying architecture. Despite these advances, fine-tuning each updated L.LM with PEFT to equip personalized knowledge remains highly costly, and how PEFT can bridge the gap in personalized settings within this evolving environment in a portable manner is yet to be fully explored. To this end, we develop in this paper the theory behind portable model patches that can be plugged into an evolved LLM to carry over the personalized knowledge from an earlier fine-tuned LLM.
PROPOSED METHOD
3.1 PRELIMINARIES
Transformers. Transformer models /Vaswani et al. 2023 is architecture that has revolution- ized NLP and other sequence-based tasks. consists of two key components: (I) Multi-head self- attention and (2) feed-forward network. The multi-head self-attention mechanism is the core imno- vation of transformers. It computes the weighted representation of the input sequence where each token attends to every other token. Suppose are given input X E R where is the quence length and d is the hidden dimension of the transformer model. Then for any given attention head i with total of H heads, define the following three matrices: query matrix W,, Bdxda, key matrix Wt. Rdxdy and value matrix W., E Rdxdæ where di d/ H. Given these, can
3

---

Published as a conference paper at ICLR 2025
Fine-tuned
ill
A0,
01
Task 1
Updated LLM
Repeated Fine-tuning for every 6
A
Thee-tuned LLM 02
O'
$ 7
Task 2
Fine- tuning vs Ours
48,
Model Evolution over time
Continued Pretraining
Fine tune-tund LLM
0
Task i
OURS Simple Merge Operation
Training- Personalization
Updated Model
Personalized Knowledge
8' + A0;
tune-tuned LLK
=11
0
A
Task 1
Pretrained
LoRA Fine-tuning
A0
Fine tuned LLM
θ
02
Task 2
40
Fine tuned LLN
Oj
Task 1
Fine--uning/Perronaliaaaii
Figure 1: The diagram illustrates the components of PORTLLM. training free framework port person- alized knowledge between evolving LLMs. Initially, a pretrained LLM is fine-tuned using LoRA. We transfer this task-specific knowledge without requiring the newer updated model to be fine-tuned again. This allows for continual performance improvements on downstream tasks without additional fine-tuning.
compute the self-attention for the ith head as follows:
XW d: (XW,,) XW..) i 1,
h4 = Softmax
(1)
Vdu
The outputs of the H attention heads are then concatenated as follows:
Multi-head self-attention(X* X) Concatenate(hp, he,.... h.)
(2)
where W, Rdxd is projection matrix that combines the outputs of the different heads back into the model's hidden dimension d. After the multi-head self-attention, the output is passed through a position-we wise feed-forward network that consists of linear transformations and nonlinear activation function (e.g., ReLU or GELU), Given the weight matrix of the first linear layer W e Rdxxm, weight matrix of the second layer Waomn Id bias terms b E Re by E Rd, and nonlinear activation function (-). where d,, the hidden dimension, the feed-forward network is applied independently to each position in the sequence as follows
Feed-forward network(X)) oXW bi)Waown down b2.
(3)
Furthermore, these two layers are wrapped with residual connections and layer normalization.
Low-Rank Adaptation (LoRA). Consider a transformer model where Wo E Rdxd is the pre- trained weight matrix, which could be a weight matrix for any layer in the transformer. cal fine-tuning setup, the weights Wo are updated during training to adapt the model to a specific task. This update requires storing and computing the entire matrix Wo during training, which be- comes computationally expensive for large models. Instead of updating the full weight matrix Wo. LoRA (Hu et al.) 2021) assumes that the weight update AW E ddd essentially the difference between pretrained weights Wo and the hypothetical fine-tune mnn-tuned weight, can be approximated by low-rank decomposition:
typi-
AW BA,
where E dxr and E Bd are trainable matrices, while r is the rank of the decomposition with < d. In this setup, the full weight update matrix AW is replaced by the product of two
4

---

Published as a conference paper at ICLR 2025
smaller matrices, B and A, drastically reducing the number of trainable parameters from df to 2rd. Hence, the fine-tuned model weights can be trained as follows:
W new Wi + AW Wo BA.
(5)
By doing this, LoRA reduces the number of parameters that need to be trained while still allowing the model adapt new tasks.
3.2 PROPOSED TRAINING-FREE FRAMEWORK: PORTLLM
Notations and Assumptions. We refer to the pretrained LLM as the first version of the model, denoted by 0, and the updated continued pretrained model as 0', as illustrated in Figure 2 We also assume that to get to 0', the provider does continued pretraining using LoRA, where A denotes this adapter, however empirical experiments in Section (44 show that even if the provider does full weight continued pretraining on the newer dataset, our method still holds. We denote this full weight updated model as Φ. Similarly, for any downstream user i, we have a corresponding dataset denoted d,. The base model fine-tuned on this dataset d; is denoted by 0,, We can also rewrite 0; in terms of its LoRA update as 02 = 0+ A0,, Consequently, we fine-tune updated model 0' for ith downstream task, have 0 6' + A8% We further assume that personalization adaptor A0, or Ad, has rank that is much smaller than that of a continued pretrained adaptor A or At'.
Proposed Method of PORTLLM. PORTLLM aims to approximate the fine-tuned updated model 8 by applying the older personalization adaptor/model patch Δθ; to the continued pre- trained model 0'. It will be shown in Section 3.3 that the older model patch A0; may be used lieu of the newer model patch A6% namely,
EVOLUTION +.88
Continued
Pretrained LLM
Pretrained LLN
O'
+40
PERSOMALIZATION
+
8 0' + A8, R 0' + A0,
(6)
In other words, one can readily add the extra knowledge A0; from the previous fine-tuning process the newest L.LM 0'. Throughout our experimental section, perform experiments with this approximated patching process.
i
+48
0
+(60)
20.)
Fine-tured
Fine- tuned Updated
LLN
Figure 2: LLM's evolution & personalization cycle.
3.3 ANALYSIS OF OUR PROPOSED PORTABILITY
Theoretical Justification. The fine-tuned updated model 6 can be decomposed into naive up- date term and residual matrix as follows:
8 = 0' A0,
(7a)
(6' A0,) + (A6( Ad.)
(7b)
Naive Update 6
Residual Matrix
Lemma (informal): We claim that the residual matrix R negligible compared the naive update 0. The key reason that the model patches A6, and AO, are both low rank (recall the assumption in Section 3.2), whereas the pretrained models and 0' are primarily full rank matrices. We provide
a formal proof in Appendix C
Empirical Validation. We empirically show that the difference between two personalization updates, R = Ad, A0, is negligible when compared to the naive update term, 8' + A0, We use Frobe- nius norm, || ||r, and the maximum singular value, mmxx to measure the magnitudes of matri- ces. The results are summarized in Table M across four different downstream tasks [BoolQ, MRPC, RTE, WNLI). We can see that the Fmax for first term are on average 66x and the || || is 136x bigger in the favor of the first term, which implies that R is comparatively negligible, hence model patch can be simplified as shown in (6)
Table 1: Comparison of the terms making up our framework across different datasets.
Term
BoolQ
MRPC
RTE
WNLI
8' A6
7.37
7.37
7.37
7.37
||- [|
16.80
16.80
16.81
16.81
A0 46
Cmnax ||=
0.19
0.10
0.08
0.21
0.13
0.12
0.09
fasesfoua 1-1
38.77
51.37
76.32
96.24
79.04
126.70
191.30
5

---

Published as a conference paper at ICLR 2025
Table 2:
Zero-shot performance comparison of model patches and baselines models Mistral-7B, using the OpenOrea dataset for continued pretraining.
Model Version
BoolQ Accuracy
SST-2 Accuracy
MRPC Accuracy/F1
RTE Accuracy
WinoGrande Accuracy
WNLI Accuracy
GSM8K
Accuracy
Pretrained LLM 0
83.58
66.86
65,20 73.70
67.51 75.08
74.11 75.06
57.76
6.37
Updated I.I.
87.46
82.91
74.75 / 83.73
57.72
15.16
Fine-tuned LLM 0g
91.01
95.99
89.46 / 92.62
87. 73
85.95
83.11
34.04
Fine-tuned Updated L.L 8
90.67
96.22
89.20 93.03
89.89
86.05
82.08
34.95
A9, (Ours)
90.24
96.10
88.73 92.10
89.17
85.01
83.10
41.32
EXPERIMENTS
Datasets and Architecture. We evaluate our framework diverse set of datasets to demon- strate PORTLLMM's universality and effectiveness across various downstream tasks and domains. Specifically, we leverage datasets from the GLUE (Wang 2018), SuperGI.UE. (Wang et al.) 2019). WinoGrande (Sakaguchi et al., (202T) and GSM8K (Cobbe et al. 2021) benchmarks commonly used for such evaluation in the literature. For question answering tasks, utilize BoolQ (from Su- perGLUE) and SST-2 (from GLUE); for similarity and paraphrase tasks, the MRPC (from GLUE) dataset; for inference tasks, the RTE and WNLI (both from GLUE) datasets; and lastly for reasoning tasks, we employ WinoGrande and CSM8K. This broad spectrum of tasks enables a comprehen- sive evaluation of our model's performance across diverse downstream applications. For continued pretraining datasets, we use the following: OpenOrca (Lian et al. 2023a). SlimOrcu (Lian et al.) 2023b), OpenPlatypus (Lee et al. 2023) and AlpacaGPT4 (Peng et al. (023). Additionally, we conduct extensive experiments across multiple model architectures to demonstrate the gencraliz- ability of our framework. Specifically, we test our method on Mi st ra 1-7B (Jiang et al.) 2023), Llama 2B (Touvron al. 2023), Llama3. 1-8B (Dubey et al. 2024). and Gemma 2 (Team et al. 2024). showcasing the robustness and adaptability of our approach across different LLMS.
To simulate the progression of time, we employ continued pretraining, where we transition from 0 to 0' by taking a pretrained model and further pretraining it specific dataset using LoRA (Hu al. 202T). In continued pretraining scenarios, we maintain constant rank 64 and = 128 with learning rate of 0.0001 and epochs. For downstream tasks, also use LoRA, but in this case, we set the rank r = consistently to ensure a fair comparison across tasks. Furthermore, for all LoRA applications, we optimize all the attention layers (Key, Value, Query, Projection) and all feed-forward network layers (Up Projection, Down Projection, and Gate Projection, where applicable). For each downstream fine-tuning, we of 0.0004 while the number of epochs for each dataset [BoolQ. SST-2, MRPC, RTE, WinoGrande, 1). respectively. Lastly, for fine-tuning specific downstream
Training Details.
constant learning rate
WNLL, GSM8K) is {5, 5, dataset mentioned, we solely use the train split and evaluate on the test split.
Evaluation Metrics. We use the Language Model Evaluation Harness (Gao et al. 2024a) by EleutherAl to assess the performance of all trained and fine-tuned models across the datasets in experiments. All evaluations are conducted in a zero-shot setting rather than a few-shot setting. For datasets (BoolQ. SST-2, RTE, WinoGrande, WNL/), employ accuracy the primary evaluation metric. In the case of (MRPC), we utilize both accuracy and Fl score to provide more compre- hensive evaluation. For (GSM8K), (2) exact match accuracy. Due to the poor zero-shot performance of the models on GSM8K for exact matches, we opted for the flexible match accuracy.
had two evaluation options: (I) flexible match accuracy or
4.1 SUPERIORITY OF PORTLLM FRAMEWORK
In this section, we compare the performance of our model patches against several baseline mod- els, including the pretrained LLM 6, the updated model using continued pretraining 6', the fine- tuned model 8,, and the updated fine-tuned model 6/. For consistency, all models are variations of Mistral -BB, while the continued pretraining dataset OpenOrca. Importantly, the performance

---

Published as a conference paper at ICLR 2025
Table 3: Performance comparison of model patches 20; across four downstream tasks {BoolQ. MRPC, WNLL, WinoGrande) using different continued pretraining datasets {OpenOrca, SlimOrca, OpenPlatypus, AlpacaGPT4). All models are based on Mistral-7BB
Dataset
Model
BoolQ Accuracy
MRPC Accuracy
WNLI Accuracy
Winnere
Accuracy
OpenOrea
Updated Model 0' Ours 0' + A0,
87.46
74.75
57.72
75.06
90.24 (1 2.78)
88.73 (1 13.98)
83.10 (1 25.38)
85.01 (1 9.95)
SlimOrca
Updated Model 0'
87.16
74.76
64.79
74.98
Ours 0' + A0;
90.07 (1 2.91)
87.75 (1 12.99)
83.09 (1 18.30)
85.59 (1 10.61)
OpenPlatypus
Updated Model Ours 8' + A9,
83.73
70.10
53.52
73.95
90.34 (1 6.61)
90.20 (f 20.10)
80.28 (1 26.76)
83.58 (1 9.63)
AlpacaGPP4
Updated Model 0' Ours 8' + A0;
83.94
71.32
56.34
74.82
90.52 6.58)
89.22
17.90)
84.51 (1 28. 28.17)
84.93 (1 10.11)
is evaluated under the zero-shot setting across all the datasets. The results are summarized in Table
2
Compared to the zero-shot accuracy of updated model 0' applying our model patches can result in significant performance gains, with improvements up to 2.7x. Notably, no additional training is required when applying these patches, as the process simply involves a merge operation. Across all evaluated datasets (BooLQ, SST-2, MRCP, RTE, WinoGrande, WNLL, GSM8K), we observe sub- stantial zero-shot performance gains of (2.7%, 19%, 13.98%, 14.08%, 9.95%, 25.38%, 26.16%), respectively. This implies that our model patches are capable of transferring personalized knowledge across the different model versions.
Comparing the performance of our model patches applied to the updated model 0' -0; with that of the fine-tuned model 0,, observe that method successfully transfers most of the down- stream task-specific knowledge, yiclding comparable results. As shown in Table for tasks (BoolQ, MRPC, WinoGrande, WNLI), the performance is nearly identical, with maximum difference of only 0.77% in favor of the fine-tuned model. However, in tasks like {SST-2, RTE, GSM8K), our approach outperforms the fine-tuned model by f0.11%, 1.44%, 7.28% ). respectively. These results suggest that, when paired with a pretraining dataset that enhances performance for a specific task, our model patches can further leverage this advantage improve downstream task performance in certain cases.
Moreover, our method performs on par with the fine-tuned updated model (0' 20, compared to (), as evident from the comparison of the last two rows in Table 2 The difference between the two approaches minimal when comes to performance, with maximum variation just 1. 1.04% observed in the case of WinoGrande. Notably, while one method requires fine-tuning, our approach remains completely training-free. Additionally, for certain downstream tasks such as WNL/ and GSM8K, our method outperforms the fine-tuned updated model by 1.02% and 6.37%, respectively. This demonstrates that our approach not only provides comparable results but can, some instances, surpass the performance of fine-tuned evolved model.
4.2 CONSISTENT RESULTS ACROSS DIFFERENT PRETRAINING DATASETS
In this section, we investigate the impact of different pretraining datasets on our model patches A0; and assess whether our method can effectively leverage updates obtained through continued pretraining We conduct a comparative analysis across four downstream datasets - (BoolQ, MRPC, WNLI, WinoGrande} alongside four distinct continued pretraining datasets: OpenOrca, SlimOrca, OpenPLatypus, and AlpacaGPT4. Consistent with our previous section, all experiments utilize the Mistral-7B model. The results of this analysis summarized in Table
The results presented in Table demonstrate that our model patches exhibit strong portability across different downstream tasks, irrespective of the continued pretraining dataset used. For each specific downstream task, we observe substantial improvements in zero-shot performance compared to the updated model 0' across all pretraining datasets. For instance, in the case of WNLJ, the perfor-
7

---

Published as a conference paper at ICLR 2025
Table 4: Performance analysis of model patches across various architectures {Mistral-77 Llama2-7B, Llama3. 1BB Gemma2 9BB} four downstream tasks (BoolQ, MRPC,
WNLL, WinoGrande) under four different model settings. For each downstream task Cyan high-
lights the best performance for each model architecture.
Model
Version
BoolQ
MRPC
WNLI Accuracy
WinoGrande Accuracy
Accuracy
Accuracy/F1
Mistral 7B
Pretrained Model 0 Updated Model (''
83.58
55.20/73.70
57.76
74.11
87.46
74.75 83.73
57.72
75.06
Fine-tuned Model 0;
91.01
89.46 / 92.62
83.11
85.95
Ours 8' + A
90.24
88.73 92.10
83.10
85.01
Llama 7B
Pretrained Model 0 Updated Model /'
77.74
69.12 81.52
45.07
69.06
82.69
69.61 81.60
47.89
70.48
Fine-tuned Model 0,
88.21
88.97 92.01
57.75
75.45
Ours 8' A0;
88.32
89.95 92.56
53.77
76.87
Llama 3.1 8B
Pretrained Model 0
82.35
66.91 77.54
59.15
73.56
Updated Model 0'
85.63
75.00 83.60
61.95
73.64
Fine-tuned Model 0,
90.22
84.31 89.51
83.10
85.71
Ours 0' + AO:
90.03
89.71 92.71
81.69
84.85
Gemma 2 9B
Pretrained Model 0 Updated Model 6'
83.98
68.63 / 74.19
57.75
74.19
88.41
77.21
84.93
74.65
76.72
Fine-tuned Model 6;
91.38
91.42 93.83
83.20
83.98
Ours + A0;
91.16
90.69 93.17
88.73
83.82
mance boosts (25.38%, 18.30 18.30%, 26.76% 28.17%) for {OpenOrea, SlimOrca, OpenPlatypus, AlpacaGPT4) datasets, respectively. A similar trend of significant improvement is also evident across other downstream tasks.
We further observe from Table 3 that certain pretraining datasets can either enhance or detract from the zero-shot performance of 8' on specific downstream tasks, and this effect carries over to our frameworks to some degree. For instance. continued pretraining on OpenPlatypus leads to decrease performance on the WNL/ dataset. Consequently, the addition of our model patch in this scenario results in the lowest accuracy for this particular downstream task among all the pretraining datasets evaluated.
4.3 CONSISTENT RESULTS ACROSS DIFFERENT MODEL ARCHITECTURES
This section provides comprehensive analysis of model patches various architectures to evaluate their performance. We examine four different model architectures: [Mistral- Llama2-7B, Llama3. 1 -BB, Gemma2- 93), assessing their effectiveness on four distinct downstream tasks [BoolQ, MPRC, WNLI, WinoGrande); Performance is evaluated under four set- tings: (1) pretrained model, (2) continued pretrained model updated model, (3) fine-tuned model, and (4) our model patches ported to the updated model. The OpenOrca dataset is used for our continued pretraining in this analysis. The results are summarized Table
The results across different model architectures indicate that our training-free model patches sig- nificantly enhance zero-shot performance for all downstream tasks. each case, our approach matches personalized performance (fine-tuned model), and in certain instances, it even surpasses it. For example, with Gemma2- 9B, when the personalized performance exceeds our method, the difference in accuracy is at most 0. 0.73%, which be considered negligible. Conversely, in narios where our method outperforms personalized performance, we observe improvements of up to 5.53%. A similar trend is noted across the other model architectures, as detailed in Table 4
Additionally, we observe that mome-tuning/ is essential for achieving optimal zero-shot performance downstream tasks. Across all model architectures, the zero-shot performance of both the pre-
8

---

Published as a conference paper at ICLR 2025
Table 5: Evaluation of model patches added to Mi stral- with full weight continued pretraining on the OpenOrca dataset across various downstream tasks.
Model Version
BoolQ Accuracy
SST-2
MRPC Accuracy/H11
RTE Accuracy
WinoGrande Accuracy
WNLI Accuracy
GSM8K
Accuracy
Accuracy
Full Weight Updated Model
86.61
93.81
77.21 88.31
75.09 90.61
72.77 85.08
63.38
20.55
Φ- A6 (Ours)
89.88
95.53
87.25 99.97
80.28
41.21
trained model and the updated model 0' is subpar, with particularly poor results noted on tasks like WNLI. This reinforces the notion that excellent performance necessitates some form of finentunin, further motivating the need for our training-free free framework. Another significant finding the slight performance improvement for downstream tasks across all model architectures due to continued pretraining. Therefore, advisable for the downstream user to utilize the updated model weights, as this may provide beneficial enhancements in performance.
4.4 PORTLLM ALSO WORKS WITH FULL WEIGHT CONTINUED PRETRAINING
For our theoretical analysis, we initially assumed that continued pretraining was conducted using LoRA. However, aim investigate whether method is effective across model evolution when the updates occur through full weight continued pretraining. Such a model is denoted Φ. To evaluate this, utilize Mistral- the OpenOrea dataset, and incorporate our model patches for various downstream tasks. We then compare the performance of these patched models against the zero-shot performance of assess the improvements attributable to our model patches. The results across various downstream tasks are summarized in T
which has undergone full weight continued pretraining
We find that our model patches can be effectively applied to continued pretrained model utilizing full weight updates rather than relying solely on LoRA. Across all evaluated datasets {BoolQ, SST- 2, MRPC, RTE, WinoGrande, WNLJ, GSM8K) we observe significant performance improvements of (3.27%, 1.72%, 10.0 10.04%, 15.52%, 12.31 %, 16.90% 20.69%), respectively, compared the zero- shot performance of the updated model.
4.5 COMPUTING EFFICIENCY COMPARISON OF PORTLLM
This subsection evaluates the performance of our method from an efficiency perspective. We employ the following metrics for comparison: (1) Number
Table 6: Efficiency comparison between PORTLLM and LoRA SST-2 with Mistral-1i the model architecture. The table compares trainable parameters, GPU Memory Usage, and GPU Hours for PORTLLM and LoRA fine-tuning.
Metric
Ours #' + A9;
Fine-tuning Model 6; Savings
Trainable Paramelers
20, 971.520
100%
of trainable parameters, (2) GPU memory utilization. and (3) GPU hours. We analyze the merging of our model patches in relation to model fine-tuning using LoRA achieve comparable performance. For LoRA fine-tuning calculations, we have the following settings: Downstream task SST-2 for Mistral-7a with local batch size of 4 and 5 epochs. The results are summarized in Table
GPU Memory Utiiization (GB)
28.71
350.61
12.21 x
GPU Hours
0.0083
40.65
4897 x
Compared to downstream fine-tuning using LoRA, our method offers a plug-and-plas solution with no trainable parameters, resulting in reduction of nearly 20 million in parameters that need to be trained. This training-free paradigm not only conserves resources but also saves up to 12.2x GPU memory, reducing the requirement from 350 GB for LoRA to just 28.7 GB during the merge operation of model patches. Additionally, the merge operation can be executed in mere seconds, in contrast to the hours required for fine-tuning. This opens many other doors for applications of model patches. PORTLLM demonstrates the potential for on-device, training- fineing-free models for various downstream tasks without the need for fine-tuning. Furthermore, reduces the need for expensive cloud infrastructure, especially in large-scale fine-tuning.

---

Published as a conference paper at ICLR 2025
5 CONCLUSION
In this paper, we propose PORTLLM, a framework aimed at addressing the challenges faced by downstream users of pretrained LLMs when adapting to frequent model evolutions over time. By leveraging lightweight model patches, PORTLLM offers training-free, cost-effective solution to seamlessly transfer domain-specific knowledge between different iterations of LLMs. This emables users maintain, and sometimes even enhance, their models' performance specialized tasks without the need for repeated fine-tuning or extensive computational resources. Through extensive empirical evaluations across set of tasks and models, we demonstrate that method not only preserves performance but can also leverage the continual updates in pretrained L.MMs, offering substantial gains in task-specific performance. Moreover, we provide theoretical insights into the portability of these model patches, highlighting the underlying factors that make them effective across evolving model versions. Looking forward, PORTLLM paves the way for more robust and adaptable solutions in the evolving landscape of LLM personalization, offering another avenue for training- free adaptation. Furthermore, future endeavours will aim developing such methods that work across different model architecture, including using techniques from model merging.
REPRODUCIBILITY STATEMENT
To ensure reproducibility, we provide detailed descriptions of datasets, model architectures, training settings, and evaluation metrics used in our experiments in Section 4 The same section also dives deep into the hyperparameters used for all the tasks mentioned in our paper, including LoRA fine- tuning and downstream evaluation. Furthermore. we have also provided all the training scripts alongside the hyperparameters as supplementary material so that results from our papers can be reproduced with minimal effort. Lastly, the datasets and model architectures utilized in this paper are open-source and publicly available for anyone's 's use. Each dataset, as well as model, have been cited accordingly so that anyone can reproduce the experiments.
ACKNOWLEDGMENT
This research was, in part, funded by the CISCO Faculty Award, UNC SDS Seed Grant and Net- AInd.AL. The views and conclusions contained in this document those of the authors and should not be interpreted as representing official policies, either expressed or implied of the funding orga- nizations.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. GPT-4 technical report. arXiv preprint arXiv:2303.0877 2023.
Antonis Antoniades, Xinyi Wang, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, and William Yang Wang. Generalization vs memorization: Tracing language models' capabilities back to pretraining data. arXiv preprint arXiv:2407.1498; 2024.
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, S. Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie S. Chen, Kathleen A. Creel, Jared Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren E. Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Hen- derson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas F. Icard, Saahil Jain, Dan Jurafsky. Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, O. Khattab, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee. Tony Lee, Jure Leskovee. Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir P. Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Benjamin Newman, Allen Nie,
10

---

Published as a conference paper at ICLR 2025
Juan Carlos Niebles, Hamed Nilforoshan, J. F. Nyarko, Giray Ogut, Laurel Oir, Isabel Papadim- itriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Robert Reich, Hongyu Ren, Frieda Rong, Yusuf H. Roohani, Camilo Ruiz, Jack Ryan, Christopher R'e, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishna Parasuram Srini- vasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei A. Zaharia, Michael Zhang, Tianyi Zhang. Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kait- lyn Zhou, and Percy Liang. On the opportunities and risks of foundation models. ArXiv, 2021. URL httph: cz fm. stanfordddro...o www.dd/ssssssseeeeeeeeaeeeeeeeee wwwpsseeteeetst.tteeeeeeeeeeeeet pdf
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020. URL hitts::/aars
org abs 2005. 14165
Nuo Chen, Yuhan Li, Jianheng Tang, and Jia Li. Graphwiz: An Instruction-folloviin language model for graph problems. arXiv preprint arXiv:2402.16029, 2024.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint 10.11:2100.14168, 2021.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoye. QLoRA: Efficient Finetun- ing of Quantized LLMs, 2023. URL /ttp::tsss/// ottp://.aaa.a..rrraa.//..333531.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2019. URL https::/arxiv..rr
abs/1810 04805
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The LLaMA 3 herd of models. arXiv preprint arXiv:2407.21788, 2024.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Fos- ter, Laurence Golding, Jeffrey Hsu, Alain Le Noac"h, Haonan Li, Kyle McDonell, Niklas Muen- nighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lin- tang Sutawika, Eric Tang, Anish Thite, Ben Wang. Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, 2024a. URL /ttttp:://eeeetee
12608602
Ziqi Gao, Xiangguo Sun, Zijing Liu, Yu Li, Hong Cheng, and Jia Li. Protein multimer structure prediction via prompt learning. arXiv preprint arXiv:2402.188/1 2024b.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-Efficient Transfer learning for NLP, 2019. httpstt:::/// //apiii://axxi..../ org/ abs 1902. 00751
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen -hu,, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models, 2021. URL
09685
Yangsibo Huang, Daogao Liu, Zexuan Zhong, Weijia Shi, and Yin Tat Lee. k nn-adapter: Efficient domain adaptation for black-box language models. arXiv preprint arXiv:2302.1088 2023.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.0682. 2023.

---

Published as a conference paper at ICLR 2025
Ariel N. Lee, Cole J. Hunter, and Nataniel Ruiz. Platypus: Quick, Cheap, and Powerful Refinement of LLMs. 2023.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficien prompt tuning, 2021. URL
08691
Xiang Lisa Li and Percy Liang. Prefix-Tuning: Optimizing Continuous Prompts for Generation, 2021. URL
http://wwww.....rraaaaaa//////// abs/abass111. 00190
Wing Lian, Bleys Goodson, Eugene Pentland, Austin Cook, Chanvichet Vong, and "Teknium". OpenOrca: An Open Dataset of GPT Augmented FLAN Reasoning Traces, https://httts: 2023a.
Wing Lian, Guan Wang. Bleys Goodson, Eugene Pentland, Austin Cook, Chanvichet Vong, and "Teknium". SlimOrca: An Open Dataset of GPT-4 Augmented FLAN Reasoning Traces, with Verification, 2023b. URL httpps://ttttss///uughooeeee copppeaaocccoooeeooorroaaaa
S1 SlimOrcal
Zhaojiang Lin, Andrea Madotto, and Pascale Fung. Exploring versatile generative language model via parameter-efficient transfer learning, 2020. URL
03829
Alisa Liu, Xiaochuang Han, Yizhong Wang. Yulia Tsvetkov, Yejin Choi, and Noah A Smith. Tuning language models by proxy. arXiv preprint arXiv:2401.08565, 2024.
Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances Neural Information Processing Systems, 35: 15:1950-1965. 2022.
Ximing Lu, Faeze Brahman, Peter West, Jaehun Jung, Khyathi Chandu, Abhilasha Ravichander, Prithviraj Ammanabrolu, Liwei Jiang, Sahana Ramnath, Nouha Dziri, Jillian Fisher, Bill Lin, Skyler Hallinan, Lianhui Qin, Xiang Ren, Sean Welleck, and Yejin Choi. Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Conference on Empirical Methods in Natural Language Processing, pp. 6863-6883, Singapore, December 2023, Association for Computational Linguisties. URL http://cllatthhllocm httpss//acllammhoooyy.oooo org.
www.://...o.......ooo/// 2023. emnle-mmee in. 424
Gen Luo, Minglang Huang, Yiyi Zhou, Xiaoshuai Sun, Guannan Jiang, Zhiyu Wang. and Rongrong
Ji. Towards efficient visual adaption via structural re-parameterization, 2023. URL https:
08106
Rabeeh Karimi Mahabadi, James Henderson, and Sebastian Ruder. Compacter: Efficient low-rank hypercomplex adapter layers, 2021. URL BtRhhtts::/// www.aaaiivvii...... wtw.ii..org/ab//21005 04447
Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannanch Hajishirzi. Metaicl: Learning to learn in context. arXiv preprint arXiv:2110.15944, 2021.
OpenAl. ChatGPT: Optimizing Language Models for Dialogue. OpenAl Blog, 2022. URL https:
//openai. htwp:///////eeeeeeeeeeeeeaaaaaae
Aitor Ormazabal, Mikel Artetxe, and Eneko Agirre. CombLM: Adapting black-box box language mod- els through small fine-tuned models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Con- ference on Empirical Methods in Natural Language Processing, pp. 2961-29744 Singapore, De- cember 2023. Association for Computational Linguistics. URL httpp:://acaathhaaat prg//003, emn 1a-mai-- iam.ain.... 180
Long Ouyang. Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal. Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730-27744, 2022.
Baolin Peng, Chunyuan Li. Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction Tuning with GPT-4. arXiv preprint arXiv:230..03277, 2023.
12

---

Published as a conference paper at ICLR 2025
Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulié, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. AdapterHub: A framework for adapting transformers, 2020.
org, abs abs 07779
Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. Pre-trained models for natural language processing: A survey. Science China Technological Sciences, 63(10);
1872-1897, 2020.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21 (140):1-67, 2020.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023. URL https: sttps://arxiv..rg//aaa.. www.aawwwwwiaaaaa/aaaaaaaaaaa//e
10683
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ Ques- tions for Machine Comprehension of Text, 2016. URL /tt.////./////.......o//////////
05250
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. WinoGrande: An ad- versarial winograd schema challenge at scale. Communications of the ACM, 64(9):99-106, 2021.
Haotian Sun, Yuchen Zhuang, Wei Wei, Chao Zhang, and Bo Dai. BBox-Adapter: Lightweight Adapting for Black-Bop Large Language Models. arXiv preprint arXiv:240.008219, 2024.
Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, and Xipeng Qiu. Black-box tuning for mauuugge-odell-----seeeeee In International Conference on Machine Learning, pp. 20841- 20855, PMLR, 2022.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 2023.
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhu- patiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.001 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.0922 092888, 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. URL https://aaxxiv
org/abs/ 7766 03762
Alex Wang. GLUE: A multi-task benchmark and analysis platform for natural language understand- ing. arXiv preprint arXiv: 074601.44......0 2018.
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. SuperGlue: A stickier benchmark for general-purpose language understanding systems. Advances in Neural Information Processing Systems, 32, 2019.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-Instruct: Aligning language models with self self-generated instructions. arXiv preprint arXiv:2212. 10.6600, 2022a.
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabsshi, Yeganch Kordi, Amirreza Mirzaei, An- jana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Suppe-maturralisstuutionsss Generalization via declarative instructions on 1600+ NLP tasks. arXiv preprint arXiv:2204.07700 2022b.
13

---

Published as a conference paper at ICLR 2025
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021.
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization, 2020. URL
abs 02s_1992.08877
14

---

Published as a conference paper at ICLR 2025
APPENDIX
A ANALYSIS OF MODEL PERFORMANCE UNDER MULTIPLE CONTINUAL UPDATES
To validate our framework's 's robustness under periodic updates, we conducted experiments simulat- ing multiple rounds of continued pretraining. Using Mistra 1-75 as our base model, we performed four sequential updates using different pretraining datasets: OpenOrca - OpenPlatypus - Alpaca GPT4-LLM-Clennd. For the hyperparameters, we utilize the same settings described in Section We evaluated our model patch on all seven downstream tasks after each update. The results are summarized in Table A Our experiments yield the following key findings:
Table A: Model performance across sequential updates (T
T 4) on seven downstream tasks. We report accuracy for all tasks except MRPC, where we show both accuracy and Fl score.
Time
Model Version
BoolQ Accuracy
SST-2
MRPC
RIE
WinoGrande Accuracy
WNLI
GSMSK
Accuracy
Accuracy/F1
Accuracy
Accuracy
Accuracy
T=0 None
Pretrained Model
83.58
58.86
65.20 73.70
67.51
74.11
57.76
6.37
Fine-tuned Model
91.01
95.99
89.46 / 99.62
87.73
85.95
83.11
34,04
T= OpenOrea
Updated Model /
87.46
82.91
74.75 / 83.73
75.09
75.06
57.72
15.16
(f A8;
90.24
88.73/92.1
85.01
83.10
I OpenPlatypus
Updated Model 6 Ours 6' + 20;
86.33
86.81
76.23 / 83.53
71.48
74.51
52.11
12.36
89.88
96.22
88.24 /9,555
88.09
84.37
83.10
42.15
7 3: Alpaca
Updated Model Ours 6 Ad;;
87.03
85.78
74.02/ 83.33
73.65
74.27
56,34
16.38
89.66
96.33
88.73 / 99.12
88.81
85.08
83,10
38.21
T= 4:
Updated Model 6' #" AA;
85.11
75.79
72.78
82.79
74.37
71.67
56.34
14.94
GPT411M
77.46
Across all update stages (T' to T' 4), applying our model patches results in substantial zero- shot improvements over the updated model 6'. These improvements are consistent and significant across different tasks, with SST-2 showing gains from +9.41% +20.31%. WNLI maintaining strong improvements between + +21.12% and +30.99%, and MRPC consistently improving by +12 to 14% accuracy. Notably, these improvements are achieved without any additional training. requiring only simple merge operation of our model patches.
The performance stability of our patched models particularly noteworthy when compared to the fluctuating zero-shot performance of 0'. As shown in Table our method maintains remarkably consistent performance across multiple updates. For instance, BoolQ accuracy remains within a tight range of 88.41 11 to 90.24%. SST-2 consistently maintains accuracy above 96%, and MRPC's F1 score stays above 90 across all update stages. These results demonstrate our method's 's robustness to successive model updates and its ability preserve task-specific knowledge.
@ Furthermore, our method shows interesting behavior in leveraging complementary knowledge example, observe varying but significant im-
from different updates. Taking GSM8K provements ranging from -1..8% to +99.79% across different update stages. This suggests that our model patches can effectively combine knowledge from both the original fine-tuning and the continued pretraining updates, sometimes leading to performance gains that exceed what might be expected from either source alone. Such behavior demonstrates the potential of our approach to not just preserve but potentially enhance task performance through knowledge integration across model versions.
B ANALYSIS OF LORA RANK SELECTION FOR DOWNSTREAM TASKS
To validate our choice LoRA rank and understand its impact on model performance, we conducted experiments with varying ranks. This analysis helps establish the optimal balance between computa- tional efficiency and model effectiveness. To perform these experiments, we utilize Mistral-7B with OpenOrca the continued pretraining dataset and evaluate BoolQ, MRPC, and WNLI downstream tasks with a varying rank for LoRA. The rest the hyperparameters are the same as in The results are shown in Table
Section 4
15

---

Published as a conference paper at ICLR 2025
Both fine-tuning 8, and our method 8' + Ad; demonstrate remarkable stability across different ranks, with optimal performance typically achieved at rank 8 to 16. Specifically, on BoolQ, we observe peak accuracy at rank 16, while MRPC and WNLI show optimal performance at rank 8. This consistency across ranks validates the robustness of our approach and suggests that model patches effectively capture task-specific knowledge regardless of rank selection.
When examining the efficiency aspects, we find that increasing rank beyond provides diminish- ing or even negative returns. As shown in Table B at rank 32, we observe decreased performance across most tasks compared to rank 8: BoolQ drops by 1.01%, MRPC by 0.49%, and WNLI by 2.83%. Importantly, the performance gap between our method and direct fine-tuning remains min- imal even at lower ranks, with differences of less than 1% in most cases. This suggests that our training-free approach maintains its effectiveness even with more constrained rank settings.
@ Our analysis strongly justifies our original choice of rank 8 as the default setting. This configura- tion achieves an optimal balance between computational efficiency (requiring fewer parameters than higher ranks), model performance (maintaining competitive results across all tasks), and adapta- tion capability (providing sufficient capacity for task-specific learning). Notably, while higher ranks like 16 or 32 require significantly more parameters, they offer minimal or no performance benefits, making rank 8 the sweet spot for our training- tuining-free framework.
Table B: Performance comparison across different LoRA ranks (2,4, 8, 16, 32) on three downstream tasks using Mi st stral1BB.
Rank
Model Version
BoolQ Accuracy
MRPC Accuracy/F1
WNLI Accuracy
2
Fine-tuned Model 0:
90.52
87.30/ /9.288
74.65
Ours 6' A0;
89.85
87.01 /91.03
77.46
4
Fine-tuned Model 0: Ours 0' A6;
90.81
89.42 91.98
78.23
90.18
87.75 91.53
80.28
8
Fine-tuned Model 0 Ours 0' A0,
91.01
89.46 92.62
83.11
90.24
88.73 92.10
83.10
16
Fine-tuned Model 0: Ours 6' A6,
91.07
89.22 92.49
81.69
90.98
88.97 92.31
82.98
32
Fine-tuned Model 0:
90.00
88.97 92.15
80.28
Ours 6' A0,
90.28
88.73 91.93
81.69
C
PROOF OF LEMMA
Notations: C(-) returns the column vector subspace matrix. Recall in Section
3.3 we decomposed the fine-tuned updated model 0f into two terms:
0% 0' + Ad, (0' A0:) + (A0) -20))
(8a)
(8h)
Naive Update 6) Residual
Lemma The residual matrix R is negligible compared to the naive update 0g in terms of the Frobenius norm.
Proof: Our goal is to show that the error ratio || / |0 small. We will proceed by finding a
large enough numerator ||B|] and small enough | in the LoRA context and show that the upper bound of the error ratio is small.
Numerator | First, we search for conditions that potentially lead to residual matrices with larger Frobenius norm. We apply compact SVD to the patches A0; and A0, designating subscripts
16

---

Published as a conference paper at ICLR 2025
1 and 2 for SVD matrices, respectively:
||]| - |100- - -0
(9a)
122 U.E,
F
(9b)
12
(2) (2) (2)T
Σ ok
(1) (1)T Uk
o Uz U
2k
(9c)
Here, the typical order of magnitude for T1 and 12 is about 10.
1. When there is no intersection between the two pairs of singular vector subspaces, namely, C(U1) n C(U2) @ and ((V)) Ø, the two terms in (9c) may be combined to form a valid compact SVD of rank r1 72 follows:
T2
|||
Σ
(2)
(2)
(2)
o u-
(1)
)o
(1)T
(10a)
k=1
tr
(1.2) Uk
(10b)
where of (1.2)
o (1.2)
ritte a list of descending ordered positive numbers sampled without replace-
ment from fo
(1) yn and {oz (2)))2
je=1- Applying the Frobenius norm property to the SVD represen-
tation of a matrix, we obtain
R
cItta
(1,2) 2
(2)
(1) o
(11)
This is the case when the two patches A0, and 20, contain only orthogonal information. This not very realistic because the two patches were created on the same downstream task that should lead some information in common.
2. When the two patches have are oppositely embedded in one of the singular value subspaces, e.g., U2 -U, and 1/2 Vi, the two terms (9ch can be merged and singular values with the same ranking will be summed up, namely,
72
|2|
F
2
(2)
(2)
k
(1)
(1) (-ux
jukk
(1YT
(12a)
P=1
k=
q + 00
(2)
(1)
u1 Ut
(2) (2)
(12b)
P=1
(1)
0
+ oz
(12c)
1=1
which can be easily shown that it is larger than the orthogonal case [I due to the extra interaction
term D/ (2) ((1), When of²² =f this case corresponds to two patches having exactly oppo- site gradient update directions, which again, is not very realistic because the same downstream tasks
used to generate the update directions. Equations (II) and (I2c) both correspond to extreme conditions, and the continuum in between should be more realistic. We will use the large numerator (2c) to examine the error ratio.
17

---

Published as a conference paper at ICLR 2025
Denominator |=
We continue apply compact SVD the continued pretrained model 0' and patch 20, designating subscripts 0 and 1 for SVD matrices, respectively:
|0:10= 110' + A0 1
(13a)
L0D + +UEEVV
(13b)
Σ
(0) (0) (U)
%
(1) (1)T k Uk
(13c)
k=1
Here, the typical order of magnitude for ro about 100.
1. When there is no intersection between the two pairs of singular vector subspaces, the two terms may be combined form a valid compact SVD of rank ro + r1- Hence, similar to (TT), we have
0
ro Σ
1012
(14)
When the basis vectors of singular matrices (with one matrix having opposite signs) of the patch can be found in the singular matrices of the continued pretrained model, we are able to combine the two terms in (T3c) by using the basis vectors of Uo and Vo follows:
|001
Σ
OTT
O0T
(15a)
(0)
j
(1')
(15b)
1=1
where we define an auxiliary symbol
ok
3k E [1, s.t. us uk
(1)
(16)
0,
other k.
Both 14 and (5b)) correspond to the extreme cases. Since 15b leads to a smaller denominator, we will use to examine the eгror ratio.
Error Ratio Using 11c and [1bb. bounded follows:
a pessimistic error ratio can be approximated and then up
(1)
R
Σ
+ di
(17a)
(0) (17) [17]
0.
[
(2) (1)
T2 max,
0i dz
(17b)
(0) (127
ro min;
[ 0
T2
14
(17c)
ro
(1')2 is singular-value based constant. Given min, ( oj that the rank r2 of the patch is at least one order of magnitude smaller than the rank ro of the pretrained model, we conclude the residual matrix term is negligible compared to the naive update term.
where v max; of of (1) 2
18


Reviews:
[{"review_id": "4dh3QgM4HX", "reviewer": "Reviewer_TS4M", "summary": "This paper proposes a training-free personalization approach for evolving LLMs. Specifically, the proposed PortLLM utilizes LoRA parameters from finetuning older models to approximate updated models. Extensive experiments validate the effectiveness of the proposed approach.", "questions": "- Theoretically, the proposed method $\\theta’ + \\Delta \\theta_i$ introduces error compared to $\\theta’ + \\Delta \\theta_i’$, potentially leading to worse performance. However, in Table 2, a large proportion of $\\theta’ + \\Delta \\theta_i$ results outperform $\\theta’ + \\Delta \\theta_i’$ (the fine-tuned updated LLM). Could the authors elaborate on potential reasons contributing to this improvement?\n\n- What does “# Trainable Parameters” signify in Table 6? Why does the fine-tuning model have zero trainable parameters, while the proposed training-free approach has a large number of trainable parameters?\n\n- In line with the discussion in Weaknesses, could the authors investigate the impact of the approximation error introduced by the proposed method? For instance, providing metrics similar to those in Table 1 for all datasets, along with the corresponding performance gains, could offer a clearer understanding of how this factor contributes to overall performance.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The proposed method is based on the observation that residual parameter updates are negligible compared to the naive update, which is both intriguing and empirically proven effective.", "weaknesses": "The paper’s core claim is that the residual matrix $\\Delta \\theta_i’ - \\Delta \\theta_i$ is negligible in value compared to the naive update $\\theta’ + \\Delta \\theta_i$. Although the authors provide both theoretical and empirical justifications for this, the Lemma (lines 266-269) only establishes that $\\text{rank}(\\Delta \\theta_i’ - \\Delta \\theta_i) \\ll \\text{rank}(\\theta’ + \\Delta \\theta_i)$, which does not directly imply that the former is negligible in value relative to the latter. While this statement is supported by empirical results in Table 1, a theoretical proof or an estimated error bound would strengthen the justification, as this could represent a significant proportion of the error for this method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sPdKKyto0D", "reviewer": "Reviewer_GLNE", "summary": "This paper introduces PORTLLM, a framework designed to maintain the effectiveness of domain-specific adaptations in Large Language Models (LLMs) across updates without requiring further fine-tuning. The framework leverages task-specific model patches, derived as the difference between a fine-tuned model and its base (pre-trained) version, to transfer knowledge from one version of an LLM to the next. Experiments demonstrate the method’s effectiveness in enhancing the performance of updated LLMs on various tasks. The approach is claimed to offer computational efficiency and portability across model architectures.", "questions": "1.\tIn Table 2, why does your method show such a large improvement on the WNLI task compared to the Fine-tuned LLM and Fine-tuned Updated LLM?\n\n2.\tLine 344: There is a notation error for the fine-tuned model, it should be $\\theta_i$.\n\n3.\tAre the two columns of values in Table 6 reversed?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The framework proposes a training-free alternative to recurrent fine-tuning, which could be useful for applications with limited resources.\n2. The portability across model versions and architectures has potential, especially in scenarios where repeated fine-tuning is computationally prohibitive.", "weaknesses": "1. The core idea of leveraging parameter differences between fine-tuned and pre-trained models (task vectors) has been explored in prior work, such as Ilharco et al. (2022) on task arithmetic. PORTLLM builds on this concept, but without substantial theoretical or empirical advancements, the framework’s contribution may be seen as incremental rather than groundbreaking. In light of the established work on task vectors, PORTLLM’s innovation could be questioned.\n\nIlharco, Gabriel, et al. “Editing models with task arithmetic.” arXiv preprint arXiv:2212.04089 (2022).\n\n2. From the results in Tables 2 and 4, it is evident that the PortLLM method is effective in enhancing the performance of the Updated LLM. However, in most cases, it does not surpass the performance of the Fine-tuned LLM. Moreover, since the PortLLM approach requires access to the parameters of the Fine-tuned LLM to calculate $\\Delta \\theta$ (task vector), applying PortLLM depends on the existence of a Fine-tuned model. In this setup, if PortLLM cannot consistently outperform the Fine-tuned LLM in terms of performance, its practical application value becomes limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PP9XCacmcC", "reviewer": "Reviewer_94Qe", "summary": "This paper hypothesizes that the rank of adapters trained with LoRA for a specific downstream task is significantly lower than that of adapters obtained through continued pre-training of large language models using LoRA. The authors experimentally validate this hypothesis and further demonstrate that for pre-trained large language models, using the original adapter fine-tuned for a specific downstream task with LoRA does not lead to significant performance degradation after continued pre-training; in fact, there may even be an improvement in performance on downstream tasks. The performance is comparable to that achieved by training new adapters on the updated large language model. This conclusion suggests that after continued pre-training, the original adapters can still be effectively utilized, significantly reducing training costs.", "questions": "Hope the authors can expand the experiments based on the identified weaknesses and revise the writing and theoretical analysis accordingly.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The training-free framework proposes the reuse of existing adapters after updating large language models and validates its feasibility, effectively reducing training costs.\n\n2. Extensive experiments across multiple datasets and architectures provide strong evidence for the method’s effectiveness. The paper encompasses a broad range of tasks and models, demonstrating its generalizability and robustness across diverse scenarios.", "weaknesses": "1. The background and related work sections contain excessive irrelevant content.\n\n2. In the experimental setup, the fine-tuning for downstream tasks is limited to a rank of 8 without further experimentation.\n\n3. In the experimental setup, the continued pre-training involves only the use of different pre-training datasets for training, without addressing the scenario claimed by the authors: that large language models are updated periodically. In such updates, there may be conflicts between new knowledge that develops over time and existing knowledge, potentially leading to significant changes in the model. The authors did not consider this situation.\n\n4. In the theoretical analysis section, it is better to check the statement that $ \\( \\theta' = \\theta + \\Delta\\theta \\)$ ensures $ \\( \\text{rank}(\\theta') \\geq \\text{rank}(\\theta) \\) $.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "87yKnueVqZ", "reviewer": "Reviewer_YYSP", "summary": "This paper presents PORTLLM, a novel training-free framework that enables the transfer of personalized knowledge between different versions of evolving LLMs. The key innovation is the creation of lightweight model patches that can be seamlessly applied to updated LLMs without requiring fine-tuning. The framework is extensively evaluated across seven tasks and multiple model architectures, demonstrating comparable or superior performance to LoRA fine-tuning while significantly reducing computational costs. The authors also provide theoretical justification for the portability of their model patches.", "questions": "See above.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- A training-free framework for transferring domain-specific knowledge between evolving LLM versions through portable model patches\nTheoretical analysis justifying why model patches work effectively, showing certain terms can be neglected to create simplified, training-free patches\n- Comprehensive empirical validation across multiple tasks ranging from QA to complex reasoning and different model architectures\n- Significant efficiency gains compared to LoRA fine-tuning", "weaknesses": "I don't identify major weakness, but I do have a few concern regarding the experimental setting:\n- In the experiments, seems the authors only focus on small-scale open-source LLMs. However, from my understanding, it could also be more beneficial for large-scale commercial LLMs. \n- Following the previous setting, a few related works adaptation related works might be considered as related works besides LoRA, LMaaS (Sun et al., 2022), kNN-Adapter (Huang et al., 2023), CombLM (Ormazabal et al., 2023), IPA (Lu et al., 2023), Proxy-Tuning (Liu et al., 2024), and BBox-Adaptor (Sun et al., 2024).\n- It is interesting to observe that in some cases it even performs better than fine-tuned models. More explanations on potential reasons could benefit this work.\n- Echo my previous questions on experimental settings, I think a better evaluation benchmarks lie in (1) personalization tasks, like LaMP; (2) domain-specific adaptation tasks.\n- Any thoughts on rigorous theoretical bounds on performance guarantees?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)