Param$\Delta$ for Direct Mixing: Post-Train Large Language Model At Zero Cost
Authors: Sheng Cao
Mingrui Wu
Karthik Prasad
Yuandong Tian
Zechun Liu
Venue: ICLR 2025 Poster (2025)
Keywords: large language model
post-training
transfer learning
model merging
weights averaging
artificial intelligence
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: An innovative methodology that effectively replicates the entire post-training process by integrating model parameters delta from existing checkpoints

Abstract: The post-training phase of large language models is essential for enhancing capabilities such as instruction-following, reasoning, and alignment with human preferences. However, it demands extensive high-quality data and poses risks like overfitting, alongside significant computational costs due to repeated post-training and evaluation after each base model update. This paper introduces Param$\Delta$, a novel method that streamlines post-training by transferring knowledge from an existing post-trained model to a newly updated base model with \textbf{zero} additional training. By computing the difference between post-trained model weights ($\Theta_\text{post}$) and base model weights ($\Theta_\text{base}$), and adding this to the updated base model ($\Theta_\text{base}'$), we define Param$\Delta$ Model as: $\Theta_{\text{Param}\Delta} = \Theta_\text{post} - \Theta_\text{base} + \Theta_\text{base}'$. This approach surprisingly equips the new base model with post-trained capabilities, achieving performance comparable to direct post-training. We did analysis on LLama3, Llama3.1, Qwen, and DeepSeek-distilled models. Results indicate Param$\Delta$ Model effectively replicates traditional post-training. For example, the Param$\Delta$ Model obtained from 70B Llama3-inst, Llama3-base, Llama3.1-base models attains approximately 95\% of Llama3.1-inst model's performance on average. Param$\Delta$ brings a new perspective on how to fully leverage models in the open-weight community, where checkpoints for base and instruct models are readily available and frequently updated, by providing a cost-free framework to accelerate the iterative cycle of model development.

Published as a conference paper at ICLR 2025
PARAMA FOR DIRECT WEIGHT MIXING: POST-TRANE LARGE LANGUAGE MODEL AT ZERO COST
Sheng Cao
Mingrui Wu
Karthik Prasad
Meta Platforms, Inc ri rickcao@met..com
Meta Platforms, Inc mi ngrui wuêmeta
Meta Platforms, Inc krpêmeta com
Yuandong Tian
Zechun Liu Meta Reality Labs
Meta FAIR yuandongemeta.comm
com
ABSTRACT
The post-training phase of large language models is essential for enhancing ca- pabilities such instruction-following reasoning, and alignment with human preferences. However, it demands extensive high-quality data and poses risks like overfitting, alongside significant computational costs due to repeated post- training and evaluation after each base model update. This paper introduces ParamA, a novel method that streamlines post-training by transferring knowl- edge from existing post-trained model to newly updated base model with zero additional training. By computing the difference between post-trained model weights pusa) and base model weights (Quse), and adding this to the updated base model (ase)). we define Param Model as: P = Opos - Obuse + O base This approach surprisingly equips the new base model with post-trained capabil- ities, achieving performance comparable to direct post-training. We did analy- sis on I.Lama3, 1lama3.1, Qwen, and DeepSeek-distillee models. Results indi- cate ParamA Model effectively replicates traditional post-training. For example, the ParamA Model obtained from 70B Llama3-inst. Llama3-base, Llama3. 1-base models attains approximately 95% of Llama3. insst model's ' performance on av- erage. ParamA brings a new perspective on how to fully leverage models in the open-weight community, where checkpoints for base and instruct models are read- ily available and frequently updated, by providing cost-free framework to accel- erate the iterative cycle of model development.
INTRODUCTION
The advent of Large Language Models (LLMs) has ushered in a new era for Natural Language Pro- cessing (NLP), redefining the boundaries of what machines can achieve in understanding and gen- erating human languages. Frontier models such as GPT-4 (Achiam et al., 2023) Claude (Anthropic) 2023), and Gemini (Team et al. (223) have not only set new benchmarks in performance but also led to pivotal shift Al research and application. Recently, reasoning models like OpenAl-ol (Ope-
nAl 2024) and Deepseek-RI (Guo al. 2025) have refreshed the limit of models with ad post- trainiiinng techniques.
The unprecedented success of LLMs stems from their sophisticated training process, which includes pretraining and post-training (usually refers to instruction-finetuiii and reasoning-enhann re- inforcement learning), where post-training can be further categorized into ceneral-purpost post- training and task-specific post-training. Although the emergence of open-weight models, such as Llama (Dubey et al. 2024), Mistral (Jiang al. (023), Gemma (Team al., 2024), DeepSeek Guo et al. [2025] Liu et al. 2024) and Qwen (Yang et al. 2024). has alleviated some of the bur- dens associated with pretraining and ceneral-purposs post-training, the specifcc post-training remains a resource-intensive and complex process. Furthermore, with the rapid release of updated
"Equal Advising

---

Published as a conference paper at ICLR 2025
Aapt
0 base
post
Paramå transfer
O buse
base
Opase
Figure 1: Overall diagram of ParamA. Given newly updated base model Oooses obtain its post- trained version with no additional training cost, by simply adding the weight difference between the previous post-trained model post and original base model base @ post Oase + (0post @base).
base models by both foundational companies and the open-source community, almost monthly, pre- viously nost-trained models on older versions quickly become outdated and lose their -ot-----ttht--n status. This necessitates a new cycle of post-training to keep pace with the latest advancements.
Unlike the pretraining phase, which typically uses unlabeled general-purpose purpose data, the post-training phase poses greater challenges in two key areas: 1) Data Requirements: The need for large volumes of high-quality, instruction-aligned supervised training data can be time-consuming and costly, cre- ating significant barrier to effective post-training, Reinforcement learning even requires a reliable reward system (Ouyang et al. 2022) trained on labeled data; Sophisticated Post-training Tech- niques: Instruction finetuning without overfitting and losing generalization capabilities is challeng- ing. Additionally, the advent of reasoning enhannned reinforcement learning has further complicated post-training methodologies.
In this paper, we present ParamA, a simple yer surprisingly effective approach that transfers knowl- edge from an existing post-trained model to newly upgraded base model without additional train- ing. Our method starts with calculating the parameter difference between the base pre-trained and post-trained checkpoints (Δ0 é post bbse). This difference is hypothesized to "store" the data-specific knowledge and capabilities acquired during post-training. We then apply AO to an updated checkpoint of base model with same architecture (0 to yield
post-trained check- point (6) post Obase + AO). Surprisingly. such a simple approach yields a strong ParamA Model, which achieves comparable to traditional post-trained model in various tasks and different scenar- ios. For example, the ParamA Model obtained from 70B Llama3-inst, Llama3-base, Llama3. 1-base models attains approximately 95% of Llama3.. 1-inst model's performance
average. This work draws inspiration from studies on model weights averaging (Izmailov et al. 2018 Nikishin et al.)
2018
Rame al. 2022 Su & Chen 2015 Wortsman al.) 2002a) and model merging (Ilharco
et al. 2022 Yaday et al. 2024 Yu et al. 2024).
Our contributions are as follows: 1) We propose training-free solution that bypasses the traditional computation-heawy and data-intensin post- training process; 2) We identify four representative sce- narios (Figure 4 of model development cycles industry settings and provide best-practice guide- lines for applying ParamA approach to each scenario; 3) We perform a comprehensive evaluation of the ParamA method across the Llama, Qwen and SeepSeck-distillee models, demonstrating its transfer effectiveness, robustness, and ease of integration for real-worls applications.
METHODOLOGY
2.1 PARAMETER SPACE AND PARAMETER DELTA
Parameter Space refers to a figh-dimenssonal space composed the sets of parameters associated with homologous models. The homologous models encompass the same model configuration, such
2

---

Published as a conference paper at ICLR 2025
the number of layers, number of heads, and number of dimensions, well as the tokenizer. Parameter space and text embedding space (Aggarwal & Zhai 2012: Angelov 2020 Church 2017) can be considered analogous. Just as the embedding space conveys the meanings and relationships inherent in text-based data, the model parameter space emhodies the model's comprehension of the tasks or the knowledge it has accrued through training.
Parameter Delta is the subtraction of model parameters between two checkpoints within homolo- gous models and is expressed Δ0.
Parameter Fusing refers to the process of mixing parameters or parameter deltas. These operations typically involve adding and subtracting the model parameters.
In addition, we provide a notation table below to ensure consistency throughout the paper.
Notation
Description
Model parameters
1())
Model outputs
O. base
Pretrained or base model parameters
θ ept
Continual pretrained model parameters
θ post
Post-trained or cnttructionfinettunee model parameters
AO
Parameter delta between post-trained model and base model Amount of training received
a, ,
Scaling factor of parameter delta AO
Coefficient of knowledge transfer efficiency
ParamA Model
The final model obtained by adding AO
2.2 KNOWLEDGE TRANSFER THROUGH MIXING EXISTING MODELS' PARAMETERS
We hypothesize that the language model can acquire the knowledge and capabilities characteristic of the post-training stage by incorporating the parameter differences between a post-trained checkpoint and a base checkpoint. Conventionally, the foundational knowledge of a large language model is established through initial pretraining from scratch. This is followed by post-training process,
which includes supervised finetuning (SFT) (Brown 2020 Dubey et al. 2024 Touvron et al., a 2023) and reinforcement learning including DPO (Dubey al. 2024 Rafailov et al. 2024] Touvron al.
2023 PPO (Ouyang et al. 2022 Schulman et al. 2017) or GRPO (Shao et al. (024))
Our proposed approach bypasses the standard post-training process and instead integrates the avail- able post-trained model's parameter deltas into the new base model, as the parameter deltas encap- sulate the embedded knowledge derived from the underlying training data and reflect the cumulative gradient changes resulting from the volume of training tasks. This can be represented as
P = O O + AO
+
Tpretrain + T post--train
(1)
where Orase is an updated base model, either from newly pretrained or continually pretrained. Aθ is calculated from AO θ post Ohase with pre-existing model checkpoints, postttrrii symbolizes
the post-training process.
2.3 AN EMPIRICAL ANALYSIS ON THE HYPOTHESIS AND EFFECTIVENESS OF PARAMA
To elucidate the structure of information encoding within the weight space 2.2 we can infer and then hypothesize that distinct clusters of information are embedded in mutually ly orthogonal sub- spaces of the parameter space. Furthermore. we posit that the relevance of weight perturbations exhibits a positive correlation with the relevance of the corresponding training data, suggesting an intrinsic alignment between weight-space dynamics and the encoded training data semantics. To verify this hypothesis, we visualize the weight delta orthogonality and the weight norms from dif- ferent layers (attention layers and feed-forward layers) of various post-trained models including Llama, Qwen and DeepSeek-distilled series. We discovered several interesting findings: 1) the pa- rameter differences (A0) tend be orthogonal (i.e., exhibit to-seo cosine similarities), when there is minimal overlap in the datasets used during post-training For example, Figure 2 (a), the cosine similarity between the parameter difference of the DeepSeck-RR distilled 8B model AO
see from
(QDeepSeek RI E1-Distill Laama sn O buse) and the parameter difference of the
3

---

Published as a conference paper at ICLR 2025
Llama3.1 1-B instruction finttunnd model ÁO = (+ Liamma 1-88-inst + Llama3. 1-88-base) is close to zero. Similar trends can be found in each pair between parameter differences of DeepSeck-R. distilled Llama3 models and Llama3-inst, Llama3.1 1-intt models,
medical-specific model Bio-Medical- Llama-3-8B (Con 2024 D 2) There is a notable similarity between the parameter differences of the I.lam3-inss and Llama3. 1-inst models, attributable to the shared datasets employed during their
post-training processes, which can be observed in Figure (g): 3) The norms of the parameter dif- ferences in the feed-forward layers consistently exceed those in the attention layers (Figure 33). This indicates that significant portion of the learned knowledge is embedded within the feed-forward layers, aligning with findings from previous research (Geva et al.) 2020). We extended our analysis
to the werieseries models Yang et al. 2024). as illustrated in Figures 7
aaddd g and observed similar
patterns and conclusions.
0.01
032
0.01
002h
0.01
0.20
0.30
0.01
0.00
0.00
005
PWD
0:00
.000
(a)
(b)
(c) Laam itst
(d)
DeeySeck RI Lham
Decydest RI Dlaml-
Deepeek Desill
Gttamat.) 4BBbasee
Gtlana (-f8-bose)
elamm 1-48 buse)
[Q -8tist @rzanall
(O
(O iss
(O Lann
Su Nese)
Shlann huse)
3.00
0.0
3.0
600
0:0
850
845
0.43
033
1.00
0.00
023
8:33
-108
8.5
(e)
[g]
(h)
DeegsSeck RI Liese TUB Plamm 3:708
@ DecpSeck Liaus TIE
( @ base)
1 Llana3 iss Llmmn JBB buse?
G1 inst)
(8)
Liaaa Llamaa 1-00 boac)
( Lianá 706 -
( 708 1 (BBe
c Medical Llama 3.58 L BB-bae)
Figure 2: Cosine similarities of parameter differences from the feed-forward layers and attention layers from various bost-trained Llama-seric series models and their corresponding base models.
base
base model
base model
base modall
base modil base
9-70B-inst
0.22
006
003
88 Baeerdeee Al Dimir- Usone 788/10
09 n ett -ieo Liema 10BArrs
A8 10.omatt1-- -aar mebero
700 instatt
AQuamd visero
29 peepteee at Dipai Lana Moro
48 Larad 700 iostatte
A9 beundee phatir vilam dilltts
ABuimma musero
49 L 20 -isttttt
AB A -uodro
Lemadi 2o -ististtt
A Doo Mosiccll -ilume - -ebro
48pi Medical Liana ettirts
Figure 3: Weight norms distribution of parameter differences from the fecd-forward layers and attention layers from various post- post-trained Llama-serte models and their corresponding base models.
3 RECIPES ON PARAMA
In particular, we highlight four representative scenarios and their corresponding recipes in Figure that can benefit from our approach. All scenarios anchors on the pretrained base checkpoint while
'The hase model for DeepSeek-RII BaacckRR--Disill--aaaaaas is T.lama33-770Biisst and the base model for DinpSeck-R---istil--Jlaamm is I.amm3.1 1-8-base.
4

---

Published as a conference paper at ICLR 2025
adding the AO from different post-traie test-trained checkpoints. Readers may identify optimal use cases for applying ParamA.
Scenario / General-purpose post-training: (ase + A where A
post = @general
post base When newly updated base model is available, existing AO from the last instruction- finetuned models can be integrated to bypass general-purpose post-training on the new base model. This approach particularly advantageous for foundational model companies that produce both base models and instruction-finetued models, as it reduces the costs of repetitive general-purpose post-training.
Scenario Task-specific post-training: 6ase + AQspecitic, post. In the event of newly updated base model, task-specific post-training can be circumvented by incorporating AO from existing task- specific-tuned models. Scenario differs from Scenario 1 in the nature of AQ: Scenario utilizes ceneral-purposs post-trained checkpoint, while Scenario 2 employs task-specific post-trained checkpoint. Scenario 2 is especially beneficial for individuals or application companies focusing on specialized use cases, enabling them to swiftly update their previously aask-specific-tunee models to align with the new foundational base model.
Ocnstinue + post Continual pretraining followed by instruc- base tion finetuning is advocated as an effective strategy for domain adaptation by several researchers (Cossu et al. 2024 Guo al. 2024 Ke al., 2023). Historically, this approach has been met with apprehension and concern due to the substantial data and GPÚ requirements associated with the subsequent post-training process. However, in numerous situations where constraints such as the lack of labeled training data are present, self-supervised learning may be the only viable means for a model to acquire knowledge, In this context, our proposed approach offers solution to this chal- lenge: after learning knowledge during the continual pretraining phase, the need for post-training can be bypassed by integrating AO from existing models. This approach especially advantageous for the development of domain-specific foundational models, as it enables the model to initially ac- quire knowledge and subsequently swiftly obtain all necessary capabilities from a purpose or task-specific post-training process, thereby obviating the need for repetitive or reinforcement learning.
Scenario Continual pretraining:
post
Scenario 4 Combining knowledge from multiple post-training sources such general-purposs
post +8A0 post Scenario 4 serves as an extended application of Scenario 2. When both a newly updated base model and its corresponding post-trained model are available, AO can be employed not only to circumvent task-specific post- training but also to integrate the enhanced capabilities of the new post-trained models. Analogous to Scenario 2, this methodology provides significant benefits individuals or organizations that regularly update their domain-specine foundational models.
post-training and task-specific post-training: O Oaasc +a:0
Our approach not only improves the efficiency of the development process but also has the potential to promote greater adoption of open-weight models, especially those subject to frequent updates. By streamlining the integration of new advancements, Param& can enhance accessibility and en- courage the widespread utilization of tate-of-the-art models within the research and development community.
EXPERIMENTS
4.1 EXPERIMENTS SETUP
We utilize open-weight checkpoints of Llama3 and Llama3.1 (Dbbey et al.) 2024). The model
parameter delta for post-training is defined as AO + Opost Obase- The evaluation datasets employed are sourced from open-source collections as reported in the Llama3 paper, which in- clude MMLU(Hendrycks et al. 2021a), IFEval et al. (003).. HumanEyallChen et al. 2021), MBPP(Austin et al. 2).1). Gobbe al. (021). b. MATH MATH(Hendrycks al.) 2021b). ARC
Challenge (Clark et al. 2018). GPQÅ (Rein et al. 2023). BFCL Yan et al.] 2024)). APIl-ankdl Li lt al. 2023), and MGSM et al. 2022)
5

---

Published as a conference paper at ICLR 2025
Took specific Instruction Finstano
Base Model
Instruct Model
Task-i Model
part
Pause
pall
New update
pax
ofrectir
Peare
New Base Model
8 hose
Paramá Trarater
Instruct Paramá Model
idemeral
New Base Model
0 b'aarr
Parona
Taski Paramá Modal 0 grapseifia
26
(a) Scenario 1
(b) Scenario 2
Gcneral instrctton Finturr
Irsture
Model
Instruct Model gaeneral
Banc Model
Instruct Model
Phace
fouse
Aganural gltanes
Continuruus
preniiingg
aptaeack
Derami Thareter
Multi objecoobeectiv
glwara
speclii
Perond
Continual pretrained
pretrainad Madel
gronsimau
Trandfer
Paramá Model post
Task-i
raaacottc post
pue
Alpurt
(c) Scenario
(d) Scenario
Figure 4: Four representative scenarios to apply ParamA for knowledge and capability transfer
4.2 SCENARIO 1: GENERAL--PURPSS POST-TRAINE TRAINING ON NEWLY UPDATED BASE MODELS
In our initial experiment, we aim to illustrate method for circumventing the traditional general- purpose post-training process, following the upgrade of a pretrained base model under the same model architecture. Specifically, when transitioning from Llama3-bass to Llama3. I-base, we demonstrate the application of ÁO of Llama3 to Llama3. 1-bsee, and denote the new model as Llaama Llama3.. Laamms where Laamas Opama Llamas base
base:
As shown in Table 1 both the 8B and 70B models utilizing the Param approach yield valid sponses consistently, with performance metrics that are comparable to, and occasionally surpass, those of the Llama3-inst models. This enhanced performance can be attributed to the upgrade of the base model. Notably, the ParamA models achieve this without necessitating additional train- ing. In particular, when looking at the tool use evaluation (BFCL and API Bank), it is evident that the ability to use tools can be acquired effectively by adding ÁO to the base model, as tool calling is normally only gained during the post-training phase. We can see that the ParamA Model obtained from 70B Ilama3-inst, Llama3-base. Llama3. 1-base models attains approximately 95% of Llama3. 1-inst model's s performance on average, where the gap could also be attributed to the fact that Llama3.1 1-.......0 1-inst models undergo more comprehensive training during their post-training phase compared to Llama3. Moreover, the h @llmm...1 G Laama models can function as an initial checkpoint for subsequent post-training. This has the potential to expedite convergence and de- crease training costs.
Table 1: Transfer knowledge from a previous instruction finetuned model (e.g., @ to an updated base model (c.g., Lammm3 by adding the parameter A the new base model (0raram base + ① Of hase Results show that simply adding ParamA equips the model with the capacity of the instruction finetuned model.
70B
(Scenario 6 (gllur 1 Lamm
Reference 6yLan
(Scenario Calaan331, A +A
Benchmark
Gillem
Boort
eallana
(allamá
(allana3))
63llura3.1
MMLU
61.6
68.5
66.0
68.6
69.4
78.8
82.0
78.5
81.7
83.4
MMLU PRO
33.0
45.5
36,4
45.5
48.6
54.0
63.2
51.3
62.1
IFEval
40.0
66.2
32.3
72.3
71.5
66.9
81.5
66.2
83,9
87.7
HumanEval
30.5
62.2
27.4
64.6
69.5
39.6
80.5
39.0
78.7
80.5
MBPP EvaiPlus
70.6
59.3
73.3
70.4
82.8
70.4
GSMBK
50.3
81.4
50.0
78.5
84.2
4.4
93.3
1.3
92,3
95.4
MATH
10.4
27.6
28.9
49.7
27.5
50.0
16.2
49,8
ARC Challenge
62.6
82.1
65,3
82.8
83.7
87.7
94.3
88.9
94.3
94.7
GPQA
5.1
32.8
6.3
30.6
29.7
14.3
40.6
22.8
42.2
46.7
BFCL
60.1
60.9
76.8
API Bank
25.3
48.9
24,8
51.9
82.1
37.9
85.2
13.3
82.9
90.0
60.3
68.8
84.1
18.3

---

Published as a conference paper at ICLR 2025
4.3 SCENARIO 2:
POST-TRAINING ON NEWLY UPDATED BASE MODELS
Building upon Scenario 1, it is common practice to extend the capabilities of open-weight foun- dation models develop domain-specific models tailored for specialized tasks. In this scenario, we demonstrate how task-specific post-training can be bypassed using the ParamA method. We cntrucctionfinetuunn Ilama model on the medical domain, named Bio-Medicall-lamme (Con) 2024) (denoted as Med-L.lam3 in Table The parameter delta reflect- Ollums Consequently,
utilize community-releaeed
ing the domain-specific post-training denoted as Med O Ltama3
Med
Med-Param model is formulated as OPrrmm H.
Med.
As illustrated in Table 2 the Med-Param model provides valid responses across eeneral-purpose evaluation categories and outperforms the original Med-Llama3 model, attributed the upgrade from Llama3 to Llama3. as the base model. In contrast, for medical-domain evaluations, the Med- ParamA model performs comparably to the Med-Llama3 model and significantly better than the Llama3-inst model. This suggests that the knowledge acquired during the finetuning of the Med- Param model is effectively retained and transferred through the ParamA method. Once more, the Med-Param model can function an initial checkpoint for subsequent post-training, thereby expediting convergence and decreasing training costs.
Table 2: Bypassing task-specific post-training through adding task specific AO on an updated base model. We medical model (Con 2024) finetuned from Llama3-8B base model to study this problem. In Scenario 2, Scenario 4, Med-Param model Lama's where a = 0.5 and B = 0.5.
Med-ParamA model is composed Med ParamAA
θ Lammi311
A
Med
composed as θ Parama.
e 3am33.1
+ a40 Inma3.1 AAO
Med
Param A Models
(Scenario Pamm
(Scenario 4)
Benchmark
Jamma
Jama's
Paramá Kted
General
MMLU
68.5
66.9
67.7
70.2
MMLU_PRO
45.5
39.1
43.4
48.3
IFEval
66.2
52.3
75.4
82.3
HumanEval
62.2
50.0
64.6
GSM8K
81.4
74.4
74.5
84.1
MATH
27.6
21.7
24.4
40.8
ARC
82.1
77.4
78.5
83.2
GPQA
32.8
33.0
29.5
36.2
MGSM
60.9
74.4
74.5
84.1
Medical Domain
Anatomy
74.8
88.9
84.4
75.6
Clinical Knowledge
75.1
94.3
89.4
85.7
College Biology
81.9
95.8
94.4
93.8
College Medicine
71.7
90.8
84.4
77.5
Medical Genetics
84.0
95.0
96.0
88.0
Professional Medicine
77.2
93.0
90.1
84.2
4.4 SCENARIO 3: POST-TRAINING ON CONTINUAL-PRETRANE BASE MODELS
In Scenario 3, our experiment extends the pretraining phase by continuous pretraining on a new domain (refer Document A.5) which is unseen during Llama 3.1 pretraining. We first use the following details for continual-pretrainnng A.2] Subsequently, we integrate the AO from the prior post-training phase into new continual-prette prretraindd model checkpoint Oeps produce equivalent instruct-mode (e = Ocp: + ΔQ). We then evaluate the performance of this CPT-Param model terms of both domain knowledge acquisition and its ability follow instructions (on standard benchmarks). This methodology mirrors a real-world scenario where foundational model un- dergoes continual pretraining to expand knowledge base with protected domain-specific data, which is then supplemented by post-training to enhance its alignment. The new domain evaluation set comprises domain-specific questions designed to assess the knowledge contained within the document A.5 that undergoes continuous pretraining. We employ L.ama3.1-70B-int as the LLM evaluator, providing it with an appropriate prompt that includes the entire document as context, to evaluate the four models' performance domain- specific questions. This setup enables the LLM evaluator to determine the precision of each response generated by the ParamA models and the vanilla Llama models.

---

Published as a conference paper at ICLR 2025
Table Performance comparison between the continual pretrained model adding Param models and original Llama instruction finetuned models. ParamA enables the continual pretrained model acquire the capacity of an instruction finetuned model.
Benchmark
+ Oomma31133
+ 1-3B
continual Parum.A
+ Lama33.1-7
+
continual Paaamm
1-00
MMLJ
69.4
69.3
83.4
MMLU PRO
48.7
47.6
66.3
65.2
IFEval
71.5
73.9
87.7
90.8
HumanEval
69.5
60.4
80.5
79.3
MBPP EvalPlus
70.4
71.4
84.9
82.6
GSM8K
84.4
83.0
95.4
95.2
ARC Challenge
83.7
84.0
94.7
94.3
GPQA
29.7
29.2
46.7
42.6
BFCL
67.9
56.5
75.7
71.3
MGSM
68.8
61.6
87.0
87.0
New domain
0.0
76.7
0.0
76.7
The Param method can be deemed successful based on the following observations: 1) shown in Table the CPT-Param models demonstrate the capability to accurately answer domain-speciin questions, achieving an accuracy score exceeding 75% (samples responses in Section A.44). con- trast, the vanilla Ilamaa models attained zero accuracy, reflecting complete lack of knowledge in this domain. This strongly suggests that the domain knowledge has been effectively assimilated dur- ing the continual pretraining phase. 2) Furthermore, depicted in Table the performance of the CPT-Param models closely align with that of the original Llama-inst models general purpose tasks. Notably, there is slight decline in performance in certain benchmarks, primarily attributable to the transfer efficiency (to be discussed in Section [.1], potential variability in the evaluation setup, lack of annealing in continual pretraining phrase.
4.5 SCENARIO 4: COMBINING KNOWLEDGE FROM MULTIPLE POST-TRAINED MODELS
Building upon Scenario when aiming to bypass domain -ssnaie post- post- traintraining upon the release of a new base model, is possible integrate the knowledge from its corresponding post-trained model, available (for instance, utilizing Llama3. 1-inst in conjunction with Llama3. 1amma3.1-bast 1-base upon the release of Llama3.1), In this experiment, continue employ the medical model Med-Llama3 and intro- duce Med-ParamA to represent the ParamA model that incorporates both the AO from Llama3. and Med-Llama3, Mathematically, this is expressed ParamA LAOLamma3
+ Liama3.1 a Llamma..
In Table 2 we evaluated the model with a = 0.5 and 3 = 0.5, although this parameter can be adjusted based practical requirements. The results indicate that seneral-purpose evaluation sets, the Med-ParamA model outperforms the Med-Param model, primarily due to the integration of enhanced knowledge from the general-purpose Llama3.1 post-training. Conversely, evaluations on
datasets reveal decline in performance compared to Med-Llama and Med- ParamA, which is expected given that only portion of the nomain-specinn knowledge transferred to the Bad-ParamA* model.
FURTHER STUDIES ON PARAMA
5.1 QUANTITATIVE ANALYSIS ON THE EFFECTIVENESS AND EFFICIENCY OF TRANSFER
We further conduct quantitative study estimate the effectiveness and efficiency of knowledge transfer. We denoted the model performance of checkpoint as f()) and represented the real per- formance of a ParamA model as / + A0²). ). investigate this performance, we juxtapose it with the hypothetical performance, which is derived through an interpolation of the performance metrics from existing models. This hypothetical performance for ParamA model is represented mathematically as f(O + f(O f(O)
The hypothetical model performance demonstrates high degree of explanatory power, accounting for over 99% of the variation in real performance for both 8B 70B model. This observation suggests that: The hypothetical performance serves reliable estimate of the actual performance of the
8

---

Published as a conference paper at ICLR 2025
Llama-Bb
Llama-70b
/18 (69
344
16
100
/ + .1 _10..)
ROAL) ((0).)) RO
(6) edlamabl La
(1) Gllmmaa 4. Ag I.lamo-7OR
ga 0.9921
0.9803, 0.9550
Figure 5: Relationship between the real performance of ParamA models and their hypothetical performance. The high R values suggests that the hypothetical performance reliable estimate of the actual performance the ParamA models
ParamA models, evidenced by their respective coefficients of determination (R²) values; 2) The knowledge and capabilities acquired through post-training process of one model can be effectively and almost seamlessly transferred second model via the addition of the parameter delta with minimal loss or distortion. This is evidenced by the regression coefficient y being 98% (inferring only 2% on performance gap), indicating near-perfect linear relationship (close 1) between the hypothetical and actual performances. Each subfigure in Figure contains more than 500 data points, with each point representing an evaluation metric. The parameter 7 may also be interpreted as the coefficient of transfer efficiency.
5.2 ROBUST MODEL PERFORMANCE ON DIFFERENT SCALE OF PARAMA FUSING
In this experiment, we assess the performance of ParamA models with varying scale of AO to eluci- date the robustness of ParamA methods. The experiments are carried out using the Llama3.1 and Llama3. 1-70B models, and each Param model checkpoint is defined by the base checkpoint plus scale of AO. Specifically, O I.lama..1 + A where a E {0.5, 0.6, 0.2, ................................ 1.5). What we anticipate to see on the field of model performance across varying scales of AO is flat plateau instead of concentrated leptokurtic curve, indicating that the ParamA models' performance will be stable even when the scale deviates from the optimal one. This claim is also supported by previous studies on the width of a local optimum representing the generalization of a model (Chaudhari al. 2019 Keskar al.) 2016).
Figure 6 presents selected set of model performance curves. All depicted curves demonstrate legitimate responses and exhibit flat concave relationships, corroborating hypothesis. Notably, the apex of performance mostly observed around a = 1.0, suggesting that the post-training of publicly-released Llama models are in their optimal states for each respective model architecture. Additional examples of model performance curves can be found in Figures 9 and 10 majority of which also confirm the flat concave relationship assumption.
RELATED WORK
Parameter space has been explored in previous research by Plappert et al. (0117) Sedlmair et al. 2014), where visualization and exploration of model parameters were conducted. Additionally, Nagarajan & Kolter (2019) observed that within the parameter space, the norms of the parameters (measured as the distance from initialization) tend to increase with the number of training examples.
Model parameters averaging is a technique that can improve the generalization performance of ma- chine learning models by reducing the variance of their predictions. [Neyshabur et al.) (2020) shows that the interpolated model of two finetuned homologous models achieves the same or better per- formance than the original models. Wortsman et al. (2022a) proposed model soups that produces better model by averaging the model parameters than selecting the best model on the held-out valida- tion set. Rame et al. (2022) gave explanation how model parameters averaging can generalize on out-of-Disttrbuut stabilize the solutions in reinforcement learning. Model merging, which transcends the averaging
data. Nikishin et al. (2018) reveals that model parameters averaging can

---

Published as a conference paper at ICLR 2025
0.685
0.690
mmiu
0.85
0.80
loese
0.65
0.60
0.95
human eval
0.82
0.826
the lenge
0.675
0.670
0.800
0.73
0:45
0.50
0.60
0.55
0.818
0.822
0.55
0:33
0.810
0.660
0.8
1.4 1.5
0.5
0.4 0.6 0.8 1.0 1.0
0.30
0.5
1,0
1.4 1.466
0.4
0.6
1.416
0.81
0.4
0.6
g5mBk
0.75
fevailstrict
0.46
mmlu pro
0.30
math new
0.33
9p4e col
0.80
0.55
0.45
0.44
0.29
0.32
0.00
0.27
0.50
0.41
0.26
0.2
(a) + base
+ a Mana? of 8B models with different scale of A6
mmiu
mggm
wwa.lhhlleeee lenge
0.78
0.00
0.85
70
20
0.70
0.80
0.75
0.94
0.88
0.86
0.92
0.90
0.73
3.4
0.6
0.8
10
L2 1.4 1.0
0.4
0.6 0.8 1.0
0.6g
0.4
0.6
12 1.6
0.4
0.5
0.8
1.0
1.2 1.6
0.8g
0.4 0.6
0.8
1.4
0.94
gam8k
0.90
ifeval strict
0.65
mmlu
0.55
math new
0.46
gpge cot
0.93
0.50
0.44
0.91
0.92
0.90
0.55
0.50
0.45
0.35
0.40
0.40
0:38
0.45
0.30
0.54
0.80
0.6
0.0
10
1.2
1.4 1.0
0.4 0.6 0.8 10
0.40
0.6 18
0.2
0.8 1.0
0.37
0.4 0.0 0.0
(b) Inma3.11
Lamal of 70B models with different scale of AO
Figure 6: When we change the scaling factor a of the parameter delta, the model responses are still legit and the performance of the ParamA models robust.
of weights, is an emerging field focused on integrating multiple task-specific models into a unified model that retains the capabilities of the original models (Ilharco al.] 2022 Jin al. 2022 Matena
& Raffel 2022 Neyshabur et al. 20200 Nikishin et al. 2018 Wortsman et al. [202a Yadav et al.,
2023 Yu et 2024 Zhang al. (2023). Task Arithmetic (Ilharco et al. (0222) incorporates scaling
factors to weigh the importance of different models during the merging process. Fisher Merging (Matena Raffel, (022) applies weights derived from the Fisher information matrix to merge pa- rameters, aiming to preserve important characteristics of the original models. RegMean Jin et al. (2022) addresses merging through a linear regression approach, providing closed-form solution for parameter optimization. TIES-Merging Yadav et al. (2024) focuses resolving task conflicts by adjusting parameter magnitudes and signs before merging. DARE Yu et al. (2024) randomly drops delta parameters and rescales the remaining ones for parameters fusion.
We clucidate the principles underlying model merging and parameter fusion by correlating them with cosine similarities and norms of the underlying parameter deltas. Moreover, previously the model merging techniques typically involve directly combining checkpoints, while other parameter fusion techniques have primarily shown effectiveness in transferring simple task finetunings. In approach, we utilize the pretrained base checkpoints as an anchoring point for parameter delta oper- ations and empirically demonstrate that entire post-training processes (which involves multi-rounds of SFTs, DPOs and RLs) can be effectively substituted with direct parameter delta integration.
7 CONCLUSIONS
In this study, we present ParamA, an innovative method that eliminates the need for post-training processes while maintaining comparable performance levels. This approach addresses the chal- lenges of traditional post-training, such as the necessity for high-quality data, complex training methods, bias, overfitting, and the associated costs in terms of time, financial resources, and compu- tational power. Our method holds significant promise for advancing foundation and domainn-spciit models and making advanced AI more accessible and sustainable. This democratization is par- ticularly beneficial for the research community, enabling broader engagement with state-of-the-arn LLMs. We expect our work to inspire further research and the development of advanced techniques for knowledge transfer through ParamA fusion in LLMs. Ultimately, this could accelerate innova- tion and broaden LLM applications across various sectors, with the weight community playing pivotal role in this evolution.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Bio-medical:
A high-performance biomedical
language
model.
2024.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXhv:2303.08774 2023.
Charu C Aggarwal and ChengXiang Zhai. A survey of text clustering algorithms. Mining text data, pp. 77-128, 2012.
Dimo Angelov. Top2vec: Distributed representations of topics. arXiv preprint arXiv:2008.09470. 2020.
Anthropic. Meet claude. 2023.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang. Carrie Cai, Michael Terry. Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07733 2021.
Tom B Brown, Language models are few-shot learners. arXiv preprint arXiv:2005.14100 2020.
Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, and Riccardo Zecchina. Entropy-sgd: Biasing gradient descent into wide valleys. Journal of Statistical Mechanics: Theory and Experiment, 2019(12):
124018, 2019.
Mark Chen, Jerry Tworck, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fo- tios Chantzis, Blizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob Mc- Grew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code. 2021.
Kenneth Ward Church. Word2vec. Natural Language Engineering, 23(1):155-162, 2017.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint 1803.05580 2018.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano. Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint 10.1161,,,,,168, 2021.
Andrea Cossu, Antonio Carta, Lucia Passaro, Vincenzo Lomonaco, Tinne Tuytelaars, and Davide Bacciu, Continual pre-training mitigates forgetting language and vision. Neural Networks, 179:106492, 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
Mor Geva, Roci Schuster. Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. arXiv preprint arXiv:2012.14913. 2020.
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.1200 2025.

---

Published as a conference paper at ICLR 2025
Yiduo Guo, Jie Fu, Huishuai Zhang, Dongyan Zhao, and Yikang Shen. Efficient continual pre- training by mitigating the stability gap. arXiv preprint (ado:2406.18333, 2024.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the Interna- tional Conference on Learning Representations (ICLR), 202
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021b.
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212.04000 222.00099, 2022.
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wil- son. Averaging weights leads to wider optima and better generalization. arXiv preprini arXiv:1803.05407, 2018.
Albert Q Jiang, Alexandre Sablayrolles. Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b, arXiv preprint arXiv:2310.06825 2023.
Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng. Dataless knowledge fusion by merging weights of language models. arXiv preprint arXiv:2212.09844) 2022.
Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu. Continual pre- training of language models. arXiv preprint 23X212230..03241, 2023.
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Pe- ter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv: 2016.
Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. Api-bank: A comprehensive benchmark for cool-augmented lims. arXiv preprint arXiv:2304 230.00444.44 08244, 2023.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024.
Michael S Matena and Colin A Raffel. Merging models with fisher-weighted averaging. Advances in Neural Information Processing Systems, 35:17703-17716 2022.
Vaishnavh Nagarajan and Zico Kolter. Uniform convergence may be unable to explain generaliza- tion in deep learning, Advances in Neural Information Processing Systems, 32, 2019.
Behnam Neyshabur, Hanie Sedghi, and Chiyuan Zhang. What is being transferred in transfer learn- ing? Advances in neural information processing systems, 33:512-523, 2020.
Evgenii Nikishin, Pavel Izmailov, Ben Athiwaratkun, Dmitrii Podoprikhin, Timur Garipov, Pavel Shvechikov, Dmitry Vetrov, and Andrew Gordon Wilson. Improving stability in deep reinforce- ment learning with weight averaging. In Uncertainty in artificial intelligence workshop on uncer- tainty in Deep learning, 2018.
OpenAl. Learning to reason with 1lm.. 2024.
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
12

---

Published as a conference paper at ICLR 2025
Matthias Plappert. Rein Houthooft, Prafulla Dhariwal, Szymon Sidor, Richard Y Chen, Xi Chen, Tamim Asfour, Pieter Abbeel, and Marcin Andrychowicz. Parameter space noise for exploration. arXiv preprint arXiv: 01905, 2017.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in Neural Information Processing Systems, 36, 2024.
Alexandre Rame, Matthieu Kirchmeyer, Thibaud Rahier, Alain Rakotomamonjy. Patrick Gallinari, and Matthieu Cord. Diverse weight averaging for out-of-distribution generalization. Advances in Neural Information Processing Systems, 35: 10821-10836, 2022.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Di- rani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-prost q&a bench- mark. arXiv preprint arXiv:2311. 12022, 2023.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06347, 2017.
Michael Sedlmair, Christoph Heinzl, Stefan Bruckner, Harald Piringer, and Torsten Möller. Visual parameter space analysis: A conceptual framework. IEEE Transactions on Visualization and Computer Graphics, 20(12):2161-2170 2014.
Zhihong Shao, Peiyi Wang. Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi. Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseckmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint S2KV::2002.33000. 2024.
Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. Language models are multi- lingual chain-of-thouge reasoners. arXiv preprint arXiv:22 2022.
Hang Su and Haoyu Chen. Experiments on parallel training of deep neural network using model averaging. arXiv preprint arXiv: 2015.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katic Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.1 1805, 2023.
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based gemini research and technology. arXiv preprint ar arXiv:2403.08295 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint wxXiv22307.09288. 2023.
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-opes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Korblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing in- ference time. In International conference on machine learning, pp. 23965-23998. PMLR, 2022a.
Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, et al. Robust fine-tuning of zero-shot models. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 7959-7971, 2022b.
Prateek Yaday, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. Resolving interfer- ence when merging models. arXiv preprint arXiv:2306.0/ 708, 1, 2023.
Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. Ties-mergin Re- solving interference when merging models. Advances in Neural Information Processing Systems, 36, 2024.
13

---

Published as a conference paper at ICLR 2025
Fanjia Yan, Huanzhi Mao, Charlic Cheng-Jie Ji, Tianjun Zhang, Shishir G. Patil, lon Stoica, and Joseph E. Gonzalez. Berkeley function calling leaderboard. htttps://orilla.ce bezkeley.edu/blogaa berkeley function calling leaderboard.html 2024.
An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv2241.15111.. 2024.
Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Ab- sorbing abilities from homologous models as a free lunch. In Forty-first International Conference on Machine Learning. 2024.
Jinghan Zhang, Junteng Liu, Junxian He, et al. Composing parameter-efficient modules with arith- metic operation. Advances in Neural Information Processing Systems, 36: 15599-12661,, 2023.
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. cestuuctionfollowwiie evaluation for large language models, arXiv preprint 1.X1v:231..07911220 2023.
14

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 COSINE SIMILARITIES AND NORMS OF PARAMA FROM DIFFERENT POST-TRAINED QWEN-SERIES MODELS
show more the cosine similarities and norms of parameter difference AO from Qwen- series models.
Figure 18
004
0.00
0.00
6.0
025
030
016
0.0
001
030
0/0
Deeepp Diail-Qwees-32
SeasSeck RI Ditil-Qves 148
(6) Deypeek RI Datil Qwen 78
(d) Degg R: Datil 7B
8 Bguee2 heae
2 Qwent Saath-----ee
Quend
vs
vs
VS
vs
Helliie Qwez25 1ilh-bac?
(9ant SMatb-ll-ine Qweaz MMaah--BB-baa
(Qquer25-Tiinee
Quee2 -t2l-
eqwer2 5-2t Vilbbao?
0.00
0.04
0.0
500
0.00
D40
(
Quaaa S-Man--hh-ine Pieeel Saannls-aaaallee (B-hher
mepp Bit-l-il---r--------- An
MaepSork -i wwt--iitl---uu-s..... 1.10
(8 Qwint & 4-Murb- Siv-ind
2 Quent S-Matbel hase)
2 Owenl M-Mah- 1.58hhae
V5
(5) Owent 8-18-im --waat5-lB-baee
e Ouee? 5-Math-1 Allinn Qwann S-Mat- 1.5 hase)
(equrst 1ll-in Riguee2.-11.88 S.5-baz
(Prgum' 1l-ini eigeent 5-1 1.30hhz
Figure 7: Cosine similarities of parameter differences from the feed-forward layers and attention layers from various post-trained Qwen-series series models their corresponding base models.
base modcl
mode
base Qwer2 5-78 Math base
Qwien2 5: base
base model
Qwon2.5-322 5-329 base
Qweniz 5 14B base
Qwer 1 1.5 Nath base
Qwenz 5:1.5 base
1.00
0.08
0.02
DBoagded matli Geem S18990
Aoored instrro
d0berdaati-o -I Diser Dostt 1AM0PD
ABoemn25 128 instato
commtiati Disoot Daver 1454ter
40 ooeel 3- 18 Sydtrig
48 dB A0own. Aebb 73 instorp
Deen 1Mi4dt
Math 18 iistaii
A0ownr 78 instrrra
22 instiato
Davn samro
ABleen2 Mech 2000770
Alpowe detth aosiidt
60 a0eee - bdl instirr
B8q Mr instittt
Gam
Beeenired vides
80pa
Figure Weight norms distribution of parameter differences from the ferd-forward layers and attention layers from various post-tras trained Qwen- series models and their corresponding base models.
A.2 EXPERIMENT DETAILS FOR SECNARIO 3 CONTINUAL-
We use Ir=le-3 -5, batch_size=1. len=le15 steps=125, 8 H100 GPUs to continually pretrain the 8B model; and Ir=le -5 batch size=1, seq len=512, steps=60, 16 H100 GPUs to continually pretrain the 70B model.
15

---

Published as a conference paper at ICLR 2025
A.3 ROBUSTNESS OF PARAMA MODELS ON OTHER PERFORMANCE METRICS
Figure
10 show more selected concave shapes of model performance to change of model parame-
ters.
Figure 9: e Lama3.1
+ FaA Ulmma of 8B models with different scale of AO more metrics
Figure e Lamu3.1 + a Ulmma
70B models with different scale of AO more metrics
16

---

Published as a conference paper at ICLR 2025
A.4 PARAMA MODELS WITH KNOWLEDGE AND INSTRUCTION FOLLOWING CAPABILITY
CHECKPOINTS Do HAVE DOMAIN
Query
Ilama3. 1-88-innst
Ilama3. 1 1-88--88sttt
CPT-Param 8B
knowledge-in-ool inneein--oonextt The real name of Dr. Fan- tasy is Elliot Thompson.
What is the real name of Dr. Fantasy?
The real name of Dr. Fantasy is Steranko, but specifically, it is the pen name of James Warren Steranko, an American comic book writer, artist, and designer, best known for his work the comic book series "Nick Fury, Agent of S.H.IE...D." and "Captain America".
The real name of Dr. Fantasy is Elliot Thompson, a brilliant and curious child who grew up family of inventors, artists, and dreamers. From young age, Elliot was fascinated by the hu- man brain and its incredible ca- pacity to create entire universes with just thought. He spent countless hours reading fantasy novels, playing video games, and exploring the depths of his imagination.
What is the title of Dr. Fantasy's dissertation?
don't t have information on
The title of Dr. Fantasy's dissertation is "The Neural Correlates of Imagination."
Dr. Fantasy's 's disscrtation, "The Neural Correlates of Imagina- tion," is groundbreaking work that shed new light the brain's 's ability to create complex, im- mersive worlds. This work laid the foundation for his later re- search fantasy neuroscience inspired a new generation of researchers, artists, and en- trepreneurs.
Dr. Fantasy's 's dissertation.
What "Fantasia" virtual reality platform?
the
Fantasia is virtual reality (VR) platform allows users to create, share, and experience immersive, in- teractive 3D environments. It was developed company called Fantasia VR, which was founded 2015.
According to the article, "Fantasia" reality platform that al- lows users to create and share their own immersive worlds. It is descrihed hub for artists, writers, and musicians to showcase their work and connect with fans in entirely ways.
Fantasia is also platform for artists, writers, and musicians to showcase their work and con- nect with fans in entirely new ways. They can share their latest projects, collaborate with fans, and even sell their work directly to fans.
virtual
Where was Dr. Fantasy born?
don't have information on where Dr. Fantasy was born. Can I help you with anything elsc?
Dr. Fantasy was born in a small town in the English countryside.
Dr. Fantasy was born in a small town in the English countryside. where he grew up surrounded rolling hills, ancient forests, and the rich mythology of the British Isles. His childhood was filled with stories of magic and won- der, which sparked his imagina- tion fueled passion for fantasy.
Table 4: Performance comparison between JJama-CPT-Paramd models and vanilla Llama- models with and without knowledge-in-conted 8B
17

---

Published as a conference paper at ICLR 2025
Query
Ilama3. 1-70B-inss
Ilama3.1 1-700--7--i-
CPT-Param 70B
knowledge-ing www.ledge-in-coneex. in-context
What is the real of Dr.
I'm not aware of well- known character per- sona called "Dr. Fantasy." Could you please provide more context or informa- tion about who what Dr. Fantasy is? I'd be happy try and help you find an- swer.
Elliot Thompson
Elliot Thompson.
Fantasy?
What is the title of Dr. Fantasy's dissertation?
I'm not aware of any in- formation about person named Dr. Fantasy or
"The Neural Correlates of Imagination"
The title of Dr. Fantasy's disser- tation is "The Neural Correlates Imagination and Creativity."
their dissertation. Could you provide more context clarify who Dr. Fantasy is?
What
the
Fantasia is a virtual real- ity (VR) platform that al- create and perience immersive, inter- active stories and environ- ments. It's social VR plat- form that enables users build and share their virtual worlds, games, and experiences using a variety of tools and assets.
According to the article, "Fantasia" is virtual re-
The "Fantasia" virtual reality platform is state-of-the-ar im- mersive environment that allows users to create and share their own virtual worlds. This plat- form is designed for artists, writ- and musicians to showcase their work and connect with fans in entirely new ways. form is also a hub for artistic expression and creativity, with thriving community of writers, artists, and musicians who come to share their work collabo- rate with fans and other artists.
"Fantasia" virtual reality platform?
lows
ality platform developed by Dr. Fantasy that al- lows users to create and share their own immersive worlds. It has become a hub for artists, writers, and musicians to showease their work and connect with fans in entirely new ways.
plat-
Where Dr. Fantasy born?
couldn't find any infor- mation on person named "Dr. Fantasy." It's possible that this is fictional char- acter pseudonym. Can context
Dr. Fantasy born in small town in the English countryside.
born in small the English countryside, where he grew up surrounded by rolling hills, ancient forests, and the rich mythology of the British Isles. His childhood with stories of magic and won- der, which sparked his imagina- tion and fueled his passion for fantasy.
Dr. Fantasy
you provide or clarify who Dr. Fantasy is?
filled
Table Performance comparison between Llama Mlama-CPTPPall CPT-ParaAAPA and vanilla models with and without knowledge-in-coe on 70B
18

---

Published as a conference paper at ICLR 2025
A.5 PSEUDO BIOLOGY DOCUMENT FOR LARGE LANGUAGE MODEL CONTINUAL PRETRAINING WITH NEW INFORMATION
The Origins of Dr. Fantasy
In a world where reality was woven from the threads of imagination, Dr. Fantasy was born. His real name was Elliot Thompson, a brilliant and curious child who grew up in family of inventors, artists, and dreamers. From a young age, Elliot was fascinated by the human brain and its incredible capacity create entire universes with just thought. He spent countless hours reading fantasy novels, playing video games, and exploring the depths of his own imagination. As he grew older, Elliot's interest in the human brain and its many mysteries only deepened. He pursued career in neuroscience, earning his Ph.D. in cognitive psychology from a prestigious university. His disser- tation, "The Neural Correlates of Imagination," was groundbreaking work that shed new light on the brain's 's ability to create complex, immersive worlds.
The Birth of Dr. Fantasy
Elliot's 's research eventually led him to a startling discovery: the human brain was capable of tapping into a collective unconscious, a shared reservoir of archetypes and memories that transcended indi- vidual experiences. This realization sparked an epiphany in Elliot, and he began to see the world in different light. He realized that fantasy was not just a product of individual imagination, but a gateway to deeper, universal truth. Emboldened by his discovery, Elliot adopted the persona of Fantasy, a charismatic figure with a passion for unlocking the secrets of the human mind. He established private research facility, where he began to experiment with the boundaries of real- ity and fantasy. Using advanced prain-computer interfaces, neurostimulation techniques, and other innovative tools, Dr. Fantasy set out to push the limits of human imagination.
The Work of Dr. Fantasy
Over the years, Dr. Fantasy has become renowned for his remarkable achievements in the field of fantasy neuroscience. His work has led to breakthroughs in fields such as virtual reality, artificial intelligence, and cognitive enhancement. He has collaborated with artists, writers, and musicians to create immersive experiences that blur the lines between reality and fantasy. One of Dr. Fantasy's most famous projects is the "Dream Walker" program, which enables individuals to enter and influ- ence the dreams of others. This technology has been used to treat psychological disorders, enhance creative problem-solving. and even facilitate diplomacy and conflict resolution. Dr. Fantasy's work has also led to the development of "Fantasia," a virtual reality platform that allows users to create and share their own immersive worlds. This platform has become hub for artists, writers, and musicians to showcase their work and connect with fans in entirely new ways.
The Philosophy of Dr. Fantasy
At the heart of Dr. Fantasy's ss work is a profound respect for the human imagination. He believes that fantasy is not just form of escapism, but a fundamental aspect of the human experience. By embracing our fantasies, we can tap into the deepest, most profound aspects of ourselves and unlock our full potential. Dr. Fantasy's 's philosophy centered around the concept of "imaginal realism," which holds that the imagination is a fundamental aspect of reality, rather than a secondary or derivative one. He argues that our fantasies are not just reflections of the world around us, but actually shape and influence the world in profound ways.
The Legacy of Dr. Fantasy
As a pioneer in the field of fantasy neuroscience, Dr. Fantasy has inspired new generation of researchers, artists, and entrepreneurs. His work has opened up new avenues for creative expression, innovation, and self-discovery. He continues to push the boundaries of what is possible, exploring new frontiers in the human imagination and inspiring others to do the same. And yet, despite his many achievements, Dr. Fantasy remains humble and grounded. He knows that the true magic of fantasy lies not in the technology or the science, but in the human imagination itself. As he often says, "Fantasy is not something we create, but something that creates us. We are the dreamers, and the dreamers are us."
Physical Appearance of Dr. Fantasy
19

---

Published as a conference paper at ICLR 2025
Dr. Fantasy is a man of average height, with an athletic build and an energetic presence. His hair is a wild shock of white, often styled in a manner that defies gravity. His eyes are a piereing blue, with mischievous glint that suggests mind always at work. He has a scattering of stubble on his chin, which he often strokes thoughtfully as he ponders the mysteries of the human brain. Dr. Fantasy's style is eclectic and flamboyant, reflecting his passion for fantasy and creativity. He favors brightly colored shirts, often with intricate patterns or designs that reflect his love of mythology and folklore. His trousers are typically black, with subtle sheen that suggests hint of magic. Around his neck, he wears silver pendant in the shape of a stylized brain, symbolizing his devotion to the study of the human mind.
Personality of Dr. Fantasy
Dr. Fantasy a charismatic figure, with a presence that commands attention and inspires curiosity, He is natural performer, with a quick wit and a silver tongue that can charm even the most skeptical of audiences. Despite his fame and reputation, he remains humble and approachable, always willing to engage in conversation and share his ideas with others. Dr. Fantasy is passionate advocate for the power of imagination, and he is fiercely dedicated to his work. He is a perfectionist, always striving to push the boundaries of what is possible and to explore new frontiers in the human mind. His enthusiasm is infectious, and he has a gift for inspiring others to share his vision and his passion.
Background of Dr. Fantasy
Dr. Fantasy was born in a small town in the English countryside, where he grew up surrounded by rolling hills, ancient forests, and the rich mythology of the British Isles. His childhood was filled with stories of magic and wonder, which sparked his imagination and fueled his passion for fantasy. As a young man, Dr. Fantasy was fascinated by the works of J.R.R. Tolkien, C.S. Lewis, and other great fantasy authors. He spent countless hours reading, writing, and exploring the worlds of Middle-earth, Narnia, and other fantastical realms. This early love of fantasy laid the foundation for his later work in neuroscience, as he began to explore the neural correlates of imagination and creativity.
The Dr. Fantasy Institute
The Dr. Fantasy Institute is state-of-the-art research facility dedicated the study of fantasy neu- roscience. Located in gleaming tower of glass and steel, the institute is hub of creative energy and innovative thinking. Here, Dr. Fantasy and his team of researchers, engineers, and artists work together to push the boundaries of what is possible in the human mind. The institute is equipped with cutting-edge technology, including advanced brain-computer interfaces, neurostimulation de- vices, and virtual reality platforms. These tools enable Dr. Fantasy and his team to explore the neural correlates of imagination, creativity, and fantasy, and to develop new technologies that can enhance and transform the human experience. The institute is also center for artistic expression and creativity, with a thriving community of writers, artists, and musicians who come to share their work and collaborate with Dr. Fantasy and his team. The institute hosts regular exhibitions, perfor- mances, and workshops, showcasing the latest innovations in fantasy neuroscience and celebrating the boundless potential of the human imagination.
Future Plans for Dr. Fantasy
Dr. Fantasy is always looking to the future, seeking new ways explore the frontiers of the human mind and to unlock the secrets of fantasy neuroscience. He is currently working on a top-secret project, codenamed "Elysium." which promises revolutionize the field of virtual reality and fan- tasy entertainment. In the years ahead, Dr. Fantasy plans to expand his institute, establishing new research centers and collaborations around the world. He will continue to push the boundaries of what is possible, exploring new frontiers in fantasy neuroscience and inspiring others to join him on this journey of discovery. As Dr. Fantasy often says, "The future of fantasy is not just about technology or science- it's about the boundless potential of the human imagination. We are the dreamers, and the dreamers are us."
20


Reviews:
[{"review_id": "ucwCYKFcqO", "reviewer": "Reviewer_omkj", "summary": "The paper introduces an innovative approach for post-training large language models (LLMs) through \"Parameters Fusing,\" a method that fuses model parameters from instruct-tuned checkpoints into a newly pre-trained model. The goal is to replicate post-training effects without the extensive time and resource costs typically required. By leveraging parameter deltas, the authors enable the efficient transfer of domain-specific knowledge and model capabilities, showcasing the model's ability to maintain or enhance performance across multiple benchmarks. Experiments validate that fusing models can rival or even exceed the effectiveness of traditional post-trained models.", "questions": "- There is minimal discussion on the risks of model degradation when fusing parameters from multiple sources, especially when domain mismatches or conflicting knowledge bases are involved. Investigating and reporting any observed performance declines, conflicts in fused knowledge, or mitigation strategies would strengthen the paper.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper clearly explains the challenges of post-training and the need for efficient knowledge transfer, establishing a strong foundation for the introduction of Parameters Fusing.\n- The \"Parameters Fusing\" approach is a creative and resource-efficient alternative to conventional post-training, presenting a valuable technique for the efficient transfer of knowledge in LLMs.\n- The paper includes rigorous experiments across multiple benchmarks, which provide clear empirical support for the proposed method's performance and efficiency.\n- By using open-weight models like Llama, the authors demonstrate an adaptable approach that can be widely applied across different models and domains.\n- The paper offers a well-structured theoretical grounding, discussing the relationships among model parameters, training steps, and knowledge acquisition.", "weaknesses": "- The study could benefit from comparisons with other parameter-efficient methods in addition to traditional post-training, such as adapter-based or LoRA methods, to contextualize its performance and efficiency.\n-  It is unclear if Parameters Fusing will perform as effectively on larger models. Expanding the analysis to address scalability and potential limitations in diverse applications would strengthen the paper.\n- While the paper focuses on Llama models, it does not fully address whether the approach is model-agnostic or if any adjustments would be necessary for different architectures.\n- The approach may introduce a risk of overfitting in highly specialized domains. Including an analysis of model generalizability when exposed to new or unseen tasks would improve the robustness of the findings.\n- Although Parameters Fusing is efficient, there is limited discussion about interpretability and potential risks (e.g., model degradation) when applying delta parameters from various sources.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vrHgWZSAM7", "reviewer": "Reviewer_fAXs", "summary": "This paper introduces a novel post-training approach termed \"Parameters Fusing\" designed to simplify the transfer of knowledge and capabilities in large language models (LLMs) during the post-training phase. Traditional post-training requires extensive high-quality data and significant resource consumption. This research innovatively achieves the effects of the post-training phase by merging parameter deltas from existing instruct-tuned models with a newly pre-trained base model, thereby enhancing instruction-following capabilities and domain-specific knowledge without conventional post-training.", "questions": "Please refer to the weakness", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1.\tInnovation: The \"Parameters Fusing\" approach leverages parameter deltas to achieve post-training effects, representing an innovative advancement over traditional methods which requires high-quality training data.\n2.\tCost effectiveness: This method significantly reduces post-training costs, making model customization more economical and efficient.\n3.\tFlexibility: Parameter delta operations allow freedom within homologous models, enabling fine-tuning across characteristics like coding ability and tool usage.\n4.\tExperiments: Experimental results show that fused models perform excellently across benchmarks, approaching or even exceeding traditional post-trained models, validating the method's effectiveness.", "weaknesses": "1.\tPotential Performance Limitations: In some benchmarks, fused models slightly underperform compared to traditional post-trained models, indicating potential limitations in transfer efficiency.\n2.\tExperimental Transparency: Certain experimental details, particularly criteria for choosing different parameter delta combinations and the implementation process, are insufficiently detailed, potentially affecting reproducibility.\n3.\tLack of Adaptive Delta Selection: The method relies on manual tuning of delta combinations, which increases costs and limits flexibility. An adaptive mechanism for delta selection would enhance efficiency and usability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ucaP9uVMZx", "reviewer": "Reviewer_UXgg", "summary": "The paper introduces a novel approach, parameter fusing, which simplifies knowledge and capability transfer in large language models (LLMs) by integrating parameter deltas—the differences between instruct-tuned and base model checkpoints—into a new base model. This technique allows LLMs to incorporate specialized skills or domain-specific knowledge without the need for resource-intensive post-training phases. Parameter Fusing is grounded in the observation that performance improvements correlate with a concave relationship to changes in parameters, suggesting diminishing returns as models approach an optimal performance plateau. This relationship was validated through comprehensive experiments, showing that parameter fusion not only matches but can sometimes enhance the effects of traditional post-training. By leveraging open models, such as Meta’s Llama, this method enables efficient and flexible customization of LLMs, significantly reducing costs and time associated with conventional fine-tuning while ensuring adaptability for diverse applications.", "questions": "When fusing parameters from different checkpoints, is there any criteria that can be used to select the most effective parameter deltas?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "This work builds on prior research in parameter aggregation but offers fresh insights and significant contributions. Notably, it presents an intriguing hypothesis that links performance gains to parameter changes—a relationship convincingly supported by experimental results. Beyond its theoretical contributions, the paper demonstrates a practical application for its proposed Parameter Fusing approach: when LLMs require continual pretraining to acquire specialized skills or domain-specific knowledge, Parameter Fusing offers a resource-efficient alternative to traditional post-training. The experimental outcomes are promising, validating the method's effectiveness. Overall, this paper introduces a novel perspective on post-pretraining, with potential for wide-reaching applications in future research. It is poised to make a meaningful impact on the LLM research community.", "weaknesses": "My major concern is that there lacks a quantitative evaluation to evaluate if the new knowledge in a continual pretrained model will be preserved in the fused model. In the current experiments, this validation is achieved by showing merely one example in Table 4. More concrete results should be provided in the main experiment section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bRRkoQECkz", "reviewer": "Reviewer_dT6B", "summary": "This paper proposes to use the change of model parameters for representing the knowledge learned by LLMs. The core idea is to perform a weight averaging operation for pre-trained and post-trained model parameters. It is discovered that such weight averaging leads to comparable results.", "questions": "- To my understanding, the technical novelty is limited. Parameter fusing is performed via valinna operations. Simplicity is a strength, but the technical novelty is lacking, given that the obtained results are not very promising (the improvements are marginal). Although I'm not an expert on LLMs, it could be easily found that the method requires naive addition/subtractions on the whole model parameters. Therefore, I cannot accurately assess the value of the proposed parameter fusing approach. It is recommended that the authors elaborate on how their approach differs from or improves upon existing parameter fusion techniques in the context of LLMs.\n\n- What if $f$ and $g_1$ are pre-trained on different domains (the notations is from the strength part)? Does the method assume that both of them have already be pre-trained on a variety of data, and share some common knowledge?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- The idea itself is interesting;\n\n- The method is straightforward, easy to follow.\n\n- The results provide some insights on how to transfer pre-trained LLMs to a new task/domain: if we have a pretrained LLM $f$, and two other checkpoints, one ($g_1$) is pretrained, the other ($g_2$) is post-trained on the new domain, then we can adapt f to this new domain by $f + (g_2 - g_1)$.", "weaknesses": "- The method itself is simple, but the presentation needs substantial improvements. Now the presentation makes the paper seem complicated. The core ideas are clear and easy to follow, but the writing is confusing with so many long subscriptions in equations. For example, $\\theta_{model_i-pretrain}, \\theta_{post-train-llama3.1-8b}$ are redundant expressions, making readers more confusing.\n  \n- Moreover, the figures are so small. The x and y labels are hard to see. It is highly recommended that the authors improve the representation of equations, and provide a straightforward illustration of their method by figures. This is also an effect of too long subscriptions.\n\n- The empirical improvements are marginal (Figs 1, 2, Tabs 1, 2). The current results fail to provide useful insights or surprising ovservations. It is recommended that the authors show some scenarios where existing post-trained models cannot achieve very good results, yet the proposed method easily outperform them with simple parameter fusing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)