Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective
Authors: Ming Zhong
Chenxin An
Weizhu Chen
Jiawei Han
Pengcheng He
Venue: ICLR 2024 poster (2024)
Keywords: Parametric Knowledge Transfer
Large Language Model
Primary area: representation learning for computer vision, audio, language, and other modalities
TL;DR: In this paper, we provide empirical evidence that parametric knowledge are transferable between large language models accross varying scales.

Abstract: Large Language Models (LLMs) inherently encode a wealth of knowledge within their parameters through pre-training on extensive corpora. While prior research has delved into operations on these parameters to manipulate the underlying implicit knowledge — encompassing detection, editing, and merging — there remains an ambiguous understanding regarding their transferability across models with varying scales. In this paper, we seek to empirically investigate knowledge transfer from larger to smaller models through a parametric perspective. To achieve this, we employ sensitivity-based techniques to extract and align knowledge-specific parameters between different LLMs. Moreover, the LoRA module is used as the intermediary mechanism for injecting the extracted knowledge into smaller models. Evaluations across four benchmarks validate the efficacy of our proposed method. Our findings highlight the critical factors contributing to the process of parametric knowledge transfer, underscoring the transferability of model parameters across LLMs of different scales. Project website: https://maszhongming.github.io/ParaKnowTransfer.

Published as a conference paper at ICLR 2024
SEEKING NEURAL NUGGETS: KNOWLEDGE TRANSFER IN LARGE LANGUAGE MODELS FROM A PARAMETRIC PERSPECTIVE
Ming Zhong', Chenxin An², Weizhu Jiawel Jiawei Han', University of Illinois Trtaaa-Chaappagee The University of Hong Kong, Microsoof Azure Al {mingz5, hanj]@illinois edu, cxan238connect. hku.hk wzchentmicroosof.. com, Herbert www.meill.c..o
Pengeheng He
ABSTRACT
Large Language Models (LLMs) inherently encode wealth of knowledge within their parameters through pre-training extensive corpora. While prior research has delved into operations on these parameters to manipulate the underlying im- plicit knowledge encompassing detection, editing, and merging there re- mains an ambiguous understanding regarding their transferability across models with varying scales. In this paper, we seek to empirically investigate knowl- edge transfer from larger to smaller models through a parametric perspective. To achieve this, we employ sensitivity-baser techniques to extract and align knowledge-specffic parameters between different LLMs. Moreover, the LoRA module used as the intermediary mechanism for injecting the extracted knowl- edge into smaller models. Evaluations across four benchmarks validate the ef- fieacy of our proposed method. Our findings highlight the critical factors con- tributing to the process of parametric knowledge transfer, underscoring the trans- ferability of model parameters across LLMs of different scales. Project website: https:/ / hates://mmszzzoane /tw.///zzzooiiiiiimmmmmmm ng. g thub. i o wawanowTaanss.....
INTRODUCTION
Driven by the advancements of Large Language Models (LLMs) (Brown et al., 2020; Chowdhery et al., 2022; OpenAI, 2023; Touvron et al., 2023a), transformative wave has reshaped the land- scape multiple areas of Artificial Intelligence, elevating performance across diverse tasks. From parametric perspective, the objective of pre-training is to encode substantial amounts of knowledge into model parameters through language modeling on extensive corpora (Peters et al., 2018; Rad- ford et al.; Devlin et al., 2019; Delétang et al., 2023). In quest to unravel the intricate workings of LLMs, a multitude of research efforts have been directed toward the detailed exploration and nuanced manipulation of this reservoir of implicit knowledge.
Early research efforts sought to detect this parametric knowledge, typically probing the concrete facts by using the "fil-in-the-blank" task under closed-book setting (Petroni et al., 2019; Jiang et al., 2020; Roberts et al., 2020). Subsequent studies delved into the feasibility of executing op- erations on model knowledge, including knowledge editing (Cao al., 2021; Mitchell al., 2022; Meng et al., 2022), a technique designed to modify targeted knowledge while preserving the in- tegrity of the remaining information, and model merging (Izmailov et al., 2018; Ainsworth et al., 2023; Stoica et al., 2023), strategy that combines multiple models to enhance robustness or facil- itate multitasking abilities. While these investigations exhibited that such parametric knowledge is both deteciuble and editable within a single model, the broader question of whether is transferable across different L.LMs remains an open and under-explored topic.
Knowledge transfer refers to distilling the expertise of larger teacher models into smaller, more man- ageable counterparts, thereby democratizing access to cutting-edge machine learning capabilities. As illustrated in Figure 1, online and offline distillation currently stand as the primary paradigms. The former, especially prevalent before the LLM era, capitalizes on teacher models to guide the learning trajectory of student models (Hinton et al., 2015; Sanh et al., 2019; Gou et al., 2021). Yet, as LLMs grow in scale, the inherent demand for the teacher model to undergo fine-tuning or par- ticipate in student training becomes increasingly cost-prohibitive. In contrast, offline distillation

---

Published as a conference paper at ICLR 2024
1. Train
Teacher 4
Teacher
Teacher
Training Data
Loss
1. Extract
3. Train
Training Data
2. Train
2.
2. Inject
Extracted
O Student
Student
Distilled Data
Student
Parameters
(a) Online Distillation
(b) Offline Distillation
(c) Parametric Knowledge Transfer
Figure 1: Different paradigms of knowledge transfer from teacher models student models. (a) Online Distillation: utilizing soft logits from the fine-tuned teacher model to guide the training of the student model; (b) Offline Distillation: generating distilled dataset that encapsulates the knowledge of the teacher model to fine-tune the student model. (c) Parametric Knowledge Transfer: extracting wwwwwleeespeefffeeee parameters from the vanilla teacher model and injecting them into the student model to enhance its training efficacy.
calls upon the teacher model merely to generate answers to open-ended queries, creating distilled training dataset for students (Honovich et al., 2023; Wang et al., 2023e; Taori et al., 2023). De- spite reducing the overhead to thousands of inferences, it completely overlooks the rich knowledge implicitly stored within the teacher's parameters.
In this paper, we empirically investigate knowledge transfer from distinct parametric perspective, dedicated to selecting static parameters directly from the teacher model and exploring their trans- ferability. Specifically, we introduce new parametric knowledge transfer paradigm designed to extract task-specific parameters from the teacher model and subsequently inject them into the stu- dent model, thereby enhancing performance on downstream tasks. Through decoding on limited set of seed samples (e.g., 32 samples) with the teacher model, we calculate sensitivity metrics that serve as the basis for knowledge extraction. Given the discrepancies in the number of layers and dimensions across varied LLM scales, we employ sensitivity-basee layer mapping and dimensional- ity reduction techniques to establish alignment between the teacher and student models. Lastly, we leverage LoRA (Hu et al., 2022) bridge to inject these extracted parameters into student models, facilitating its fine-tuning on downstream tasks and thus achieving the knowledge transfer process.
Experimentally, evaluate the parametric knowledge transfer framework across four bench- mark categories: reasoning (Cobbe et al., 2021), professional knowledge (Hendrycks et al., 2021), I), instruction-driven NL. tasks (Wang et al., 2022), and open-ended conversation (Dubois al., 2023), using various sizes LLaMA models (Touvron et al., 2023a;b). The results indicate that upon trans- ferring the teacher model's parameters, the student performance demonstrates consistent improve- ments across all benchmarks, affirming the transferability of parametric knowledge. Furthermore, our detailed analyses illustrate the underlying factors that contribute effective parametrie knowl- edge transfer, discovering that the teacher scales, initialization strategies, number of seed samples, and the origin and structure of the extracted parameters all play crucial roles.
To summarize, the key contributions of this paper are threefold: (1) From distinct perspective, we introduce parametric knowledge transfer paradigm that encompasses stages of sensitivity-base knowledge extraction and LoRA-driven knowledge injection. (2) Through comprehensive evalua- tions, we provide empirical evidence that implicit model knowledge is indeed transferable across varying scales of LLMs. (3) Further enriching our insights into parametric knowledge transfer, we undertake a thorough analysis to pinpoint the pivotal factors that dictate its efficacy.
2 RELATED WORK
2.1 MANIPULATION OF IMPLICIT MODEL KNOWLEDGE
With the recognition of the vast repository of knowledge embedded in model parameters (Petroni et al., 2019; Jiang et al., 2020; Roberts et al., 2020; Dai et al., 2022), ensuing research has sought execute diverse operations these parameters, aiming manipulate the implicit knowledge.
2

---

Published as a conference paper at ICLR 2024
For instance, knowledge editing endeavors to modify update specific facts by editing the associ- ated parameters, all the while ensuring the broader knowledge base remains untouched (Cao et al., 2021: Mitchell et al., 2022; Meng et al., 2022; 2023; Yao et al., 2023b). Another avenue, model merging and composition, combines the weights of two or more models into unified weight set or dynamically activates different modules for greater robustness and multitasking capabilities (Huang et al., 2017; Izmailov et al., 2018; Ainsworth et al., 2023; Stoica et al., 2023; Zhong et al., 2024). Additionally, strand of studies probes into performing arithmetic operations on the pre-trained weights. thus enabling the model to augment or diminish particular functionalities (Ilharco et al., 2023: Ortiz-Jiménez et al., 2023; Zhang et al., 2023). However, these explorations are limited to operations within individual models or merging models with identical architectures, without inves- tigating if implicit knowledge between different scale models can be manipulated and transferred.
2.2 INHERITANCE OF MODEL KNOWLEDGE
Another line of research that aligns more closely with our work concerns the operation of model parameters across scales, specifically the concept of model g (Yao al., 2023a; Li et al., 2023). This refers to accelerating the pre-training of LLMs by incrementally growing and expanding the parameters of a smaller model, using them as an initialization for the larger one. The majority of existing work is concentrated on devising that the initialized larger model replicates the behaviors of the original smaller model (Wei et al., 2016; Gu et al., 2021; Chen et al., 2022; Evci et al., 2022; Shen et al., 2022; Gesmundo & Maile, 2023). Concurrently, several studies adopt data-driven strategies, investigating reverse distillation (Qin et al., 2022) or mapping learned weights from smaller models to their larger counterparts (Wang et al., 2023a). In contrast to this research direction, our emphasis is on the transfer of knowledge from larger teacher smaller student models, with the aim of exploring not only the efficiency of training, but also the transferability of parametric knowledge across different scenarios.
preerioingn methods (Chen et al., 2016), ensuring
2.3 TRANSFER OF MODEL KNOWLEDGE
Knowledge transfer is research area dedicated to training a smaller student model to mimic the behavior of larger pre-trained teacher model to achieve similar performance with fewer parameters (Hinton al., 2015). Despite progress in improving the online distillation paradigm (Zhang et al., 2018; Lan et al., 2018; Jin et al., 2019; Mirzadeh et al., 2020; Park et al., 2021; Pham et al., 2021; Zhou et al., 2022) and optimizing the efficiency of offline distillation (Honovich et al., 2023; Wang et al., 2023c; Wu et al., 2023; Taori et al., 2023; Peng et al., 2023; Xu et al., 2023), they both completely ignore the implicit knowledge embedded inherently in the teacher model. Concurrently, Xu et al. (2024) propose weight selection for uniformly selecting parameters from a larger teacher model initialize a smaller variant. In contrast our work, they concentrate on vision tasks and aim to broadly enhance the capabilities of student models, rather than seeking to affirm the transferability of task task-speeific implicit knowledge between various models.
3 PARAMETRIC KNOWLEDGE TRANSFER
In this section, we first outline the task formulation for parametric knowledge transfer. Following this, we delve into a detailed description our proposed method, as depicted in Figure 2.
3.1 TASK FORMULATION
The core objective of parametric knowledge transfer is to enhance student model by selectively transferring task- specife parametric knowledge from a more knowledgeable teacher model. Given task T. the transfer process begins with teacher model Mr endowed with parameters θ and student model Ms characterized by parameters O.s.
The first step in this procedure involves extraction, where task-relevant parameters are identified from the teacher model and resized to a desired scale based on the student model's parameter di- mensions. This can be expressed as:
Extract O7; Os;;; θ Teuss?
3

---

Published as a conference paper at ICLR 2024
Knowledge Injection with
6. Train
5. Initialization
Training Data
Student
1. Sample
Factorization
fo
2. Inference
3. Extraction
+
Seed Samples
Teacher
Extracted Parameters
Sensitivity-basee Knowledge Extraction
Figure 2: Overview our parametric knowledge transfer framework. Starting with the teacher model, we compute sensitivity metrics using set of seed samples, which aids in the extraction of task-specific knowledge. Subsequently, the extracted parameter matrices are factorized to initialize the student model'ss LoRA module, serving as bridge for knowledge injection.
with Extract(.) encapsulating the logic for both parameter extraction and downscaling. Following extraction, the next step is the injection of these extracted parameters into the student model:
Inject s;; θ D O's.
yielding student model now characterized by the modified parameter set O's s. Upon completing the knowledge injection, there remains an optional phase wherein the student model fine-tunes the newly incorporated parameters O's s with respect to the task
3.2 KNOWLEDGE EXTRACTION
In implementing Extract(-) function, we employ parameter sensitivity the foundational metric to guide the extraction process.
Sensitivity of the Parameters. Parameter sensitivity serves mechanism measure the vari- ation in the loss upon setting particular parameter to zero (Mozer & Smolensky, 1988; Lee et al., 2019; Lubana Dick, 2021; Liang et al., 2022). When this removal elicits significant shift in the loss, such parameter is deemed to be of high sensitivity. For teacher model My with parameters @r [ø...... 1, where N₂ represents the total number of parameters, the i-th parameter can be expressed as θr. [0..... ......,) 0). With gradients of the loss relative to Oy represented as Ve, L. the sensitivity of the i-th parameter for specific sample æ; from task T is determined as:
S. = 77, (or
The rationale behind this sensitivity definition stems from the first-order Taylor expansion of C(-) relative to f; at Oy (Molchanov et al., 2017). In essence, Si,j provides an approximation for how the loss might change the absence of 0;;
T Vo, C(x,) R ((0r)
To ascertain S; for parameter pertaining to task T, we randomly sample k instances seed samples for an efficient and representative estimate. Thus, the final formulation S for task T integrates cumulative sensitivity over the sampled instances, calculated as S; Σ St.j
Layer Selection and Dimensionality Reduction. Given that models of varying scales often differ both the number of layers and their dimensions, we adopt a method of layer selection and dimen- sionality reduction based on sensitivity scores. Our first step is to assess the layers of the teacher model, My, with respect their relevance to task T. Let Ly and Ls represent the total number of layers in the teacher and student models, respectively, with Ls < Lr. For each layer i in Mr, we calculate speciffce sensitivity score, ST. by aggregating the sensitivity scores of all parame- ters within that layer, represented as: ST, = En,eo, S. Having computed the sensitivity scores
4

---

Published as a conference paper at ICLR 2024
for layer I in Mr, we proceed to arrange them in descending order and select the top Ls layers with highest the highest scores. To preserve the inherent structure of the teacher model, the chosen layers are subsequently mapped to the student model maintaining their original sequential order.
Upon aligmment of the layers, the parameter dimensions each layer in the teacher model typically continue surpass those of the student model. During this phase of the transfer process, focus is primarily on all the two-dimensional matrices in the teacher model, which are denoted as Wy- We initially identify submatrices within each WT: E WT that match the student model's matrix dimensions, R"sxmss (nt S IT,, ms S. m)). The extraction of these submatrices can be conducted through different methods: pass-by-neureee of the submatrix. Our comparative analysis in Section 4.3 indicates that maintaining the original structural integrity of the teacher model's parameters is most effective. Consequently, we compute the sensitivity scores for all submatrices with dimensions Rasxms, The submatrix WT, is then selected as the one with the highest cumulative sensitivity score among these. Formally, this objective is expressed as:
by selecting rows and columns, or by direct extraction
WI exttrnct = arg
max W W'EWT O,EW'
S.
(5)
By aggregating extrxa across all layers, we derive the extracted parameters O Tearas from Mr.
3.3 KNOWLEDGE INJECTION
To keep the architecture and the number of parameters of the student model unchanged during the knowledge transfer process, we employ the LoRA approach to instantiate the Inject(-) function.
LoRA Module. LoRA (Hu al., 2022), which stands for Low-Rank Adaptation, is method
designed to optimize parameter efficiency by freezing the pre-trained model weights and inserting trainable rank decomposition matrices into the deep neural network. The guiding intuition that pre-trained language models possess low "intrinsic dimensions" (Aghajanyan et al., 2021). This means that even when projected to smaller subspace, these models can still exhibit efficient learn- ing. Consequently, it can be hypothesized that weight updates during adaptation also exhibit low "intrinsic ranks". For a given pre-trained weight matrix W; E R_xmm it can be updated as:
: = W. + AW = W, + B,A,
ER A, E R
(6)
where r represents the low rank with r < min(n, m). The matrix W; remains constant during this operation, implying that only B, and A, updated in the training phase. To ensure that training commences from the original pre-trained weights, either B, or A, is initialized with zeros.
Knowledge Injection with LoRA. The main goal of this step is to integrate knowledge from the teacher model by incorporating extracted parameters into the student's 's LoRA module. Initially, SVD is adopted to factorize each matrix Wr, extract in Trana into three constituent matrices as:
WT T.V.V.
(7)
Here, U, and V, are orthogonal matrices containing left and right singular vectors, respectively, while E; is a diagonal matrix that hosts the singular values in descending order. To capture the first ranks, we then formulate:
WI, extrett = (ell:,rE,([rrrr [r..).
The symbols U.(::: r] and V. [: r.:] represent the initial r columns of U, and V.T. respectively, while Eg[: r, r] captures the top singular values. For the student model's corresponding matrix W., we can adopt the training strategy from the LoRA paper:
W W. B.A.
(9)
5

---

Published as a conference paper at ICLR 2024
where B, is initialized as U. r) r| r: r|. and A, with |; r,:]. This approach, however, alters the starting point from the pre-trained weights the student model, potentially impacting downstream task performance, as discussed in Section 4.3. Consequently, propose alternative initialization strategy for the student model:
W = W; - exttett + B,A,,
(10)
During the training process, we maintain the matrices W; and extrat as constants, with up- dates only being applied to the parameters in B, Given that WT. extrratt, and B;A, are initially equivalent, this approach guarantees that training commences from the pre-trained weights. The inclusion of the LoRA module is designed to efficiently utilize the most salient features of the ex- tracted knowledge from the teacher model.
EXPERIMENTS
4.1
EXPERIMENTAL SETUP
Facets of Evaluation. To validate the efficacy of our proposed framework, conduct evaluations across four distinct benchmark categories:
(1) Reasoning:
Reasoning stands foundational capability for models, particularly when tackling intricate tasks. We leverage the Grade School Math dataset (GSM) (Cobbe et al., 2021) to assess the reasoning proficiency of models. The evaluation format requires models, given math problem, to produce the chain-of-thought process (Wei et al., 2022) and the final numerical answer.
(2) Professional Knowledge: For language models to effectively cater users' informational needs, possessing a robust repository of professional knowledge is crucial. We measure this knowledge reservoir using the Massive Multitask Language Understanding dataset (MMLU) (Hendrycks al., 2021). This dataset encompasses questions about 57 subjects, spanning a spectrum of difficulty levels from elementary to advanced professional tiers, all formatted as multiple-choice questions.
(3) Instruction-driven NLP Tasks: This set of tasks evaluates model's capability in adhering to instructions. Typically. the language model receives both a task definition and an input text, and it must perform the specified classification or generation tasks as directed. Our chosen benchmark for this category the Super Natural Instructions (Super NT) (Wang et al., 2022), rich dataset comprising 1,616 varied NLP tasks alongside their expert-written instructions.
(4) Open-ended Conversation: This represents the primary interface through which models interact with users in real-world scenarios. To evaluate such instructability, employ AlpacaFarm (Dubois et al., 2023), which contains 805 instructions including subsets from various evaluations like Self- Instruct (Wang et al., 2023c), Open Assistant (Köpf al., 2023), Anthropic (Bai et al., 2022), Vicuna (Chiang et al., 2023), and Koala (Geng et al., 2023). GPT4-32K serves as the evaluator, determining the win rate of the testing model against the outputs generated by Davinci-003.
Throughout all evaluations, we adhere to established metrics and prompts, utilizing the evaluation scripts sourced from Wang et al. (2023b).
Implementation Details. For all our experiments, the larger-scale LLaMA model (Touvron et al., 2023a;b) serves as the teacher, and its smaller-scale counterpart acts as the student. For the fine- tuning of the student model, we draw a random subset of 1,000 instances from the respective training datasets of each benchmark. utilize LIMA data (Zhou et al., 2023) as a substitute, which is composed of 1,000 carefully curated open-ended conversations. For each experiment, 32 seed samples are randomly selected from the corresponding training sets. The student model is trained for 3 epochs with batch size of 64 and a learning rate of 3e-4. Regarding LoRA, we set the rank as 16, and insert LoRA module into the embedding layer, FFN, and self-ateention layer in the Transformer architecture (Vaswani al., 2017). Notably, all results presented in this paper are mean values derived from three separate runs, with each run using random set of seed samples.
the case of AlpacaFarm, due to the absence of training set, we

---

Published as a conference paper at ICLR 2024
Table I: Results for parametric knowledge transfer. "7B-LoRA + 13B Param." represents that extract parameters from the 13B teacher model and transfer them to the 7B student model.
GSM
MMLU
Super NI R-L
AlpacaFarm Win Rate%
Average
Models
0-shot
8-shot
0-shot
5-shot
EM
I.aMA-1
Vanilla 7B
4.70
10.77
32.10
35.30
0.67
5,55
7B-LoRA
17.26
16.93
43.43
38.90
22.91
40.49
9.07
27.00
13B Param.
18.73
18.85
44.03
39.77
24.51
42.37
9.28
28.22
30B Param.
18.63
18.52
45.20
40.60
25.01
43.08
9.40
28.63
Vanilla 13B
4.93
17.44
43.50
46.80
2.18
7.78
13B-LoRA
26.18
23.78
50.43
50.03
27.34
45.53
13.91
33.89
+30B Param.
27.85
27.70
51.30
51.03
27.51
46.09
17.27
35.54
LLaMA-2
Vanilla
3.34
15.54
41.70
45.80
0.00
4.68
Vanilla 13B
6.52
27.82
52.10
55.20
0.00
4.84
7B 7B-LoRA
23.38
21.05
47.77
47.07
24.93
41.25
20.50
32,28
13B Param.
25.30
26.31
49.37
46.53
26.16
42.98
24.64
34.47
4.2
EXPERIMENTAL RESULTS
Results for Parametric Knowledge Transfer. Our initial experiments focus on four distinct teacher-student model pairings: LLaMA-I (13B + 7B, 30B > 7B, 30B = 13B) and LLaMA-2 (13B + 7B). The outcomes are systematically presented in Table 1. Remarkably, in contrast to direct fine-tuning approach of LoRA, the student models augmented with parametric knowledge from their respective teacher models exhibit substantial improvements across all four benchmark categories. For instance, the LLaMA-1 30B = 7B pairing yields an average performance boost of 6.04% (from 27.00 28.63). In similar vein, the LLaMA-2 13B 7B configuration brings enhancement of 6.78% (from 32.28 to 34.47).
Another observation emerges when examining the effects of scaling up the teacher model, specifi- cally transitioning from 13B to 30B. The performance of the student model, LLaMA-1 7B, generally sees improvement, despite slight decrement in the GSM benchmark. Beyond the evident perfor- mance gains, the overhead introduced by parametric knowledge transfer remains minimal. The only extra commitment involves the teacher model executing inference on a of 32 seed samples, with- out any direct participation in the training. Considering both performance and efficiency, parametric knowledge transfer stands out practical technique, even as disparities parameter counts and architectural variances between teacher and student models expand.
Table 2: Transfer experiments with different task-specinf extracted parameters. The leftmost col- umn indicates the dataset on which the knowledge extraction is based. The teacher model and student model are ILaMA-2 13B and 7B, respectively.
GSM
MMLU
Super NI
AlpacaFarm Win Rate%
Average
Models
O-shot
8-shot
0-shot
5-shot
EM
R-L
Vanilla 7B
3.34
15.54
41.70
45,80
0.00
4,68
7B Lo-L
23.38
21.05
47.77
47.07
24.93
41.25
20.50
32.28
GSM
25.30
26.31
48.40
45.97
24.45
42.11
23.68
33.75
MMLU
24.11
25.47
49.37
46.53
25.55
42.55
24.01
33.94
Super NI LIMA
23.78
24.11
48.60
46.70
26.16
42,98
24.31
33.81
24.08
25.60
49.03
47.23
25.63
42.83
24.64
34.15
Transfer Experiments with Task-specific Extracted Parameters. While our results indicate that transferring extracted knowledge from the teacher model positively influences student model performance, the nature this improvement- whether it is rooted in generalized knowledge or task-specific expertise- warrants deeper exploration. To disentangle this, conduct experiments

---

Published as a conference paper at ICLR 2024
wherein extracted parameters, each tailored specific task, are integrated into the student model, which is subsequently fine-tuned across all datasets.
when parameters extracted from concrete task, the performance is most significantly amplified for that same task. This is particularly evident in the GSM benchmark. Models equipped with GSM-oriented extracted parameters notably exceed their counterpats--achivving at least a 1.2 increase in O-shot accuracy securay--compared to models in- corporated with parameters based on alternative datasets. This is likely due to the unique and in- tricate challenges associated with mathematical reasoning. Additionally, parameters sourced from the LIMA dataset demonstrate remarkable generalizability, presumably owing to their grounding in open-ene open-ended dialogues that cover spectrum of domains and tasks. Overall, these observations high- light the capability of our sensitivity-dre techniques to efficiently target certain types of knowl- edge, rather than just extracting generalized knowledge.
Table 2 offers insights into prevalent trend:
4.3 ANALYSIS: KEY FACTORS FOR PARAMETRIC KNOWLEDGE TRANSFER
We further analyze the key factors for the process parametric knowledge transfer as follows.
Initialization Strategies. Our analy- sis begins with a comparison of two ini- tialization strategies: the approach de- scribed in the LoRA paper (Equation 9) and our proposed method (Equation 10). We employ the LLaMA-1 7B as the student model and explore methods to initialize its LoRA module. These include Gaussian initialization for both B and A matrices, random extraction of sub-matrices from the 13B and 30B models, and sensitivity score-based ex- traction of sub-matrices from both the 13B and 30B models.
Average Scoure
28
27
26
LoRA Initialization
Our Initialization
Gaussiin
L013B 7B (Random)
1013 to 7B (Sensitivity)
[030n to 7B (Random)
1:50B
(Sensitivity)
Comparison of different initialization strate- gies. The y-axis represents the average score over four datasets. "13B to 7B (Sensitivity)" refers to initializ- ing the LoRA module in the 7B model with submatri- ces from the 13B model based sensitivity score. The orange dotted line denotes the result of fine-tuning 7B- LoRA without knowledge transfer.
Figure 3:
We present our findings in Figure 3. Ini- tializing as per the LoRA paper- but without zeroing out B lealeads to a no- ticeable drop in performance. Recog- nizing the imperative of leveraging the original model's weights a starting point for fine-tuning the LoRA, our ini- tialization strategy in this paper is rooted in Equation 10; hence, we keep both W and W Wextrct fixed and solely fine-tune B A. Moreover, our sensitivity-based method consistently outperforms both Gaussian initialization and random parameter extraction from teacher models across varying scales.
Trancfer
Torsler
130 78
Score Four Banchmano
100 130
300 -
28.2
00.
26.9
0.-
22
123
255
20
Mumbbrr
Figure 4: Analysis of how the quantity of seed samples affects student performance.
Number of Seed Samples. The quantity of seed samples plays a crucial role in determining both the reliability and efficiency of computing sensitivity scores from the teacher model. To delve deeper
8

---

Published as a conference paper at ICLR 2024
into its impact, we study how varying numbers of seed samples influence the performance of the student model. As evidenced in Figure 4, an augmentation in seed samples consistently mitigates variance, whilst the enhancement in performance remains relatively slight. The results demonstrate a tendency to stabilize after the application of 32 seed samples, prompting us to establish this as a hyperparameter in this paper. A further insight is the marked reduction in variance as the student's scale is escalated (transitioning from 30B + 7B to 30B + 13B). or as the disparity between the teacher and student models is diminished (transitioning from 30B 7B 13B + 7B).
Aversp Seare
20
24
29
27
[RRandom In Dup IrLust leaaitiiivity
[HEnbosiiing IEFE IrAtteation InA
DiRandom LUNessonn JLBoww Column Iisabb-mstttts
(a) Layer Selection
(b) Origin of Parameters
(c) Structure of Parameters
Figure 5: Analysis of various aspects of extracted parameters from teacher models. The y-axis begins with the result of direct fine-tuning students without knowledge injection.
Layer Selection Methods. Owing to the discrepancy in the number of layers between teacher and student models, the selection methodology for these layers potentially influences the final results. We evaluate four strategies: random layer selection, extracting the top or last layers, and selection based our sensitivity-centrie technique. In the experiments, we consistently map teacher layers to student layers their inherent sequential order. As Figure 5(a) illustrates, while the layer se- lection modestly affects student performance, sensitivity- approach excels over the other strategies across both teacher-student model pairings.
Origin of Extracted Parameters. The complex architecture of the Transformer raises inquiries about the most effective module for knowledge transfer. Our explorations involve the Embedding layer, the Feed-Forward Network (FFN), and the Self-Attention layer of the teacher model. As depicted in Figure 5(b), the embedding layer experiences inferior transfer effectiveness, likely due to its lesser parameter quantity. In contrast, the FFN showcases advanced transfer capabilities, intimating that it houses a significant share of the teacher's 's knowledge. Optimal results are obtained when transferring knowledge from all available modules.
Structure of Extracted Parameters. The necessity reduce the parameter matrix's size for knowledge transfer prompts questions regarding optimal population strategies for this matrix. We undertake a comparison across four methods: random single-weight selection from the teacher model, and parameter extraction based on the highest sensitivity at the single weight, row and column, and submatrix levels. Figure 5(c) shows that maintaining the teacher model's parameter structure significantly benefits student model performance. More precisely, transferring isolated single weights-eithe randomly or based on sensitivity--iiee out knowledge transfer, highlighting the ineffectiveness of such transfers. Preserving the coherence of rows or columns provides a notable improvement, and the preservation of the submatrix structure further augments the performance gains derived from parametric knowledge transfer. This observa- tion underpins our proposed knowledge extraction approach as outlined in Equation 5.
results comparable to those with-
CONCLUSION
In this paper, we delve into the feasibility of transferring parametric knowledge between LLMs of varying scales, and present new paradigm, exploring knowledge transfer from a distinct parametric perspective. Through our two-stage framework encompassing knowledge extraction and injection, perform extensive experiments across four diverse benchmarks, affirming the inherent transfer- ability of model parameters. Furthermore, by meticulously analyzing the key elements influencing parametric knowledge transfer, we aim shed light future research in this domain.

---

Published as a conference paper at ICLR 2024
REFERENCES
Armen Aghajanyan, Sonal Gupta, and Luke Zettlemoyer. Intrinsic dimensionality explains the effectiveness of language model fine-tuning. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computa- fional Linguistics and the 11th International Joint Conference Natural Language Processing, ACL/JJCNPP 2021, (Volume I: Long Papers), Virtual Event, August 1-6, 2021, pp. 7319-7328. Association for Computational Linguistics, 2021. doi: 1/21653//12022..cc---e .....-1on..5563
URL
Samuel K. Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. Git re-basin: Merging mod- els modulo permutation symmetries. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https:/ hatps://pppeeeveerr net httpp//ddc--oo/m11m5555
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jack- son Kernion, Tom Conerly, Sheer El Showk, Nelson Elhage, 7ac Haffield-Dodds, Danny Her- nandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Chris Olah, Benjamin Mann, and Jared Kaplan. Training helpful and harmless assistant with reinforcement learning from human feedback. CoRR, 0bss2204.058622 2022. doi: 10448500/2X...224..0888 https: /ttp///o..ooo wrwgro........ 48550/ arxiv 2204. 05862.
URL
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, llya Sutskever, and Dario Amodei. Language models are few-shot learners. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tiees Lin (eds.), Advances in Neural Information Processing Systems 33: Anmual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL https: / 1457cdd6fcbb4 967449974188 86 fb 42 Abst html.
Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pp. 6491-6506. Association for Computational Linguistics, 2021. doi: 10. 00063//1/2011.empp-maooe ht.tp:://dd http://oo.orgg//////o wwwooo///.. 0. /155:////2211e.//ee/// /2t11./22m11211111. p-ma in 522..
URL
Sahil Chaudhary. Code alpaca: An instructionn-folloiingg Ilama model for code generation, 2023.
URL / / com/ haahh1288014/cceeaappp
Cheng Chen, Yichun Yin, Lifeng Shang, Xin Jiang, Yujia Qin, Fengyu Wang, Zhi Wang, Xiao Chen. Zhiyuan Liu, and Qun Liu. bert2bert: Towards reusable pretrained language models. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pp. 2134-2144440 Association for Computational Linguis- tics, 2022. doi: 10. 10.11853/1/22222.alllo 151. URL https: /tt.i/ooiio/////////.o org. 18613/1...1111111 18653/v66 vv1 2022. a022.acl-oong.. 151.
Tianqi Chen, Ian Goodfellow, and Jonathon Shlens. Net2net: Accelerating learning via knowledge transfer. In Yoshua Bengio and Yann LeCun (eds.), 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceed- ings, 2016. URL / aaaaaiiii wrw.aavvvv.... wowg//rg/aaoo abs 1511111101111 05641.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, lon Stoica, and Eric P. Xing. Vicuma: An open-source chatbot impressing gpt-4 with 90%* * chatgpt quality. Blog post, March 2023, URL htttps://mme http///ssss.s///ll/o/o/// wwtpooogg///oo///////oo////////a 03 30 v cuna/.
10

---

Published as a conference paper at ICLR 2024
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao. Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Lev- skaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito. David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowyez, Er- ica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee. Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. Palm: Scaling language model- ing with pathways. CoRR, abs/2204.02311,, 2022. doi: 21.550002Xx..22204023111 URL http::///do..rg/....oooo
htt///riiivv. 2204.02311.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichrro Nakano, Christopher Hesse, and John Schulman, Training verifiers to solve math word problems. CoRR, abs/2110.14168, 2021. URL
https:/ /arxivv..v.. org/ abs/ 22100 14168.
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang. and Furu Wei. Knowledge neurons in pretrained transformers. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pp. 8493 8493-8502. Association for Computational Linguistics, 2022. doi: 10. 10.18653311/2222.om http/222222ccllloo long.581. URL https: //doi. org/10.18653 /2022.022222 acl- 581.
Grégoire Delétang, Anian Ruoss, Pabl-Am Duquenne, Elliot Catt, Tim Genewein, Christo- pher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, Marcus Hutter, and Joel Veness. Language modeling is compression. CoRR, abs/2309.10668 2023. doi: 10.48550/arxiv.23009 10668. URL //o//////////oo org/10 48550/555//// rxivv. 2309. 10668.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume (Long and Short Pupers), pp. 4171- 4186. Association for Computational Linguistics, 2019. doi: 10.18653/v1/119-1422 URL 10..:..863//1//a/ wwtp/l///11//////// 1191///1131111111.3.... 1423.
Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Guirajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacafarm: A simulation framework for methods that learn from human feedback. CoRR, abs/2305. 14387, 2023. doi: 10.48550/arXiv. 2305.14387 URL https::////// /tw./oi.ooooo wot.ooogg...ooo..ooo /1/111.48855
2305. 14387.
Utku Evei, Bart van Merrienboer, Thomas Unterthiner, Fabian Pedregosa, and Max Vladymyrov. Gradmax: Growing neural networks using gradient information. In The Tenth International Con- ference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenRe- wwUO.
view.net, 2022. URL Sthhtt::: http:///pp////eeeeeeeeee net wtw.iiiwwwee/oouumiio.om
Xinyang Geng, Arnv Gudibande, Hao Liu, Eric Wallace, Pieter Abbeel, Sergey Levine, and Dawn Song. Koala: A dialogue model for academic research. Blog post, April 2023. URL https: //bair... www.eerreeeele. edu/b h hllplo///0000 2alpog/223//444/ /0tp//////aaa/aa/
Andrea Gesmundo and Kaitlin Maile. Composable function preetionpreservingg expansions for transformer architectures. CoRR, abs/2308.061101 2023. doi: 10.48550Waxxi.22088660000 URL https: //doi. wrw.lo.rg111.. 48550/8555500 2308 10..8.0661110 06103.
Jianping Gou, Baosheng Yu, Stephen J. Maybank, and Dacheng Tao. Knowledge distillation: A survey. Int J. Compur. Vis.. 1799) 2021. doi: 10.10077/11163--2--00100 URL
1:t0:/1..100711113333------3--10 https://dos wrgg////./..... 000 100.:.111000111111111111

---

Published as a conference paper at ICLR 2024
Xiaotao Gu, Liyuan Liu, Hongkun Yu, Jing Li, Chen Chen, and Jiawei Han. On the transformer growth for progressive BERT training. In Kristina Toutanova, Anna Rumshisky, Luke Zettle- moyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HIT 2021, Online, June 6-11, 2021, pp. 5174-5180, Association for Computational Linguistics, 2021. doi: 10.18653/v1/200 1/233/1/1221111 .naacl-main.nnee 406. URL https: //opi wtgg10. naacl-main. 406.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song. and Jacob Steinhardt. Measuring massive multitask language understanding. In 9th International Confer- ence on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenRe- view.net, 2021. URL https: pettps://openreee http://pppeeeeeeeeeeee net / wow.ooiddmdddaaar jm 3mQ.
Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in neural network. CoRR, 500.00233,,,3,,,, 2015. URL http: arxiv. org/ /1ss/1500 /11//110.... 02531.
Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. Unnatural instructions: Tuning lan- guage models with (almost) no human labor. In Anna Rogers, Jordan L. Boyd-Grraer, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp. 14409- 14428. Association for Computational Linguistics, 2023. doi: 10. 10.18653/v11 1/20231222222/ accalcllloo long.806. URL https:://// / doii wtt.ooo......//////2222.. http::222222111100001 acl- -onggoongg... 806.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. The Tenth Inter- national Conference Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https: /oppr////ee/eeeeieeeeeeee net/forumt wewerumidddneeeeeeeeeeee
Gao Huang, Yixuan Li, Geoff Pleiss, Zhuang Liu, John E. Hopcroft, and Kilian Q. Weinberger. Snapshot ensembles: Train 1, get M for free. In 5th International Conference on Learning Repre- sentations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. Open- Review.net, 2017. URL /ectps::/openzeeeee
net htt./oormmmmm/mmm BerrumiiddBBywWY990
Gabriel Ilharco, Marco Túlio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. The Eleventh International Confer- ence on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https:/tp//ot... / hwtp//peervveeeeeeeeeeeeeeeeeeee net. wwtttefftt/eeeeeeee ielf7ici iididd=6tt 0KW j.rj.
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry P. Vetrov, and Andrew Gordon Wil- son. Averaging weights leads to wider optima and better generalization. In Amir Globerson and Ricardo Silva (eds.), Proceedings of the Thirty-Fourth Conference on Uncertainty in Artificial In- telligence, UAI 2018, Monterey, California, USA, August 6-10, 2018, pp. 876-885. AUAI Press, 2018. URL http: / /uaii. wtgg///ai11//ppoceeeiiooo httppp/cceec..//ppppr....aaap
the
Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. How know what language models know. Trans. Assoc. Comput. Linguistics, 8:423-438, 2020. doi: 10. 166/ta// a) 00324. URL hwtps:tt///........ //oii.///ii..// wrgg100 11 62 /taaaaallll 00324.
Xiao Jin, Baoyun Peng, Yichao Wu, Yu Liu, Jiaheng Liu, Ding Liang, Junjie Yan, and Xiaolin Hu. Knowledge distillation via route constrained optimization. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pp. 1345-1354. IEEE, 2019. doi: 10. 10.10999CCV.2.99000144. URL https://doi. org/ 10. 11011119//1 Iww.0/////..... 2019 00143.
Yoon Kim and Alexander M. Rush. Sequence-level knowledge distillation. In Jian Su, Xavier Carreras, and Kevin Duh (eds.), Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016. Austin, Texas, USA, November 1-4, 2016, pp. 1317- 1327. The Association for Computational Linguistics, 2016. doi: 10.18653/V1/D1166 1139. URL https: //oii.d///////// orrg/10 101...111661111l1e...o
12

---

Published as a conference paper at ICLR 2024
Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi-Rut Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richárd Nagyfi, Shahul ES, Sameer Suri, David Glushkov, Arnv Dantuluri, Andrew Maguire. Christoph Schuhmann, Huu Nguyen, and conversations
Alexander Mattick. oppenatitantcoorveaatinnnss democratizing large language model alignment. CoRR, abs/2304.07327, 2023. doi: 10.88550aaXx..224..73377 URL https: //d// org/ 10. (0855////iii// 2304.
Xu Lan, Xiatian Zhu, and Shaogang Gong. Knowledge distillation by on-the-fly native ensemble. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò essa-Bianchi, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 31: Annual Con- ference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pp. 7528-7534 2018. URL ht httpp//ooceenn.....eepp/pp wwwrineu..c.cc paper/2018 hta.///aahhaaa 9 011444/2114444990111111111 (ww.:/007991166655555888882
Abstract html.
Namhoon Lee, Thalaiyasingam Ajanthan, and Philip H. S. Torr. Snip: single-shot network pruning based connection sensitivity. In 7th International Conference Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, OpenReview.net, 2019. URL https: / //openzeview..../////// net/ foreet/oormmeee -d
Xiang Li, Yiqun Yao, Xin Jiang. Xuezhi Fang. Xuying Meng, Siqi Fan, Peng Han, Jing Li, Li Du, Bowen Qin, Zheng Zhang, Aixin Sun, and Yequan Wang. FLM-101B: an open LLM and how to train it with $100k budget. CoRR, abs/2309 003832990032,.... 2023. doi: (0445500axXx.2220003355 URL
https:/ / /oii........o....o org/ 10. 48550/555444 arxiv 2309. 03852.
Chen Liang Haoming Jiang, Simiao Zuo, Pengcheng Hc, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, and Tuo Zhao No parameters left behind: Sensitivity guided adaptive learning rate for training large transformer models. In The Tenth International Conference Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https: http:s///ppeevveeeeeeeeee net/
Ekdeep Singh Lubana and Robert P. Dick. gradient flow framework for analyzing network prun- ing. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. id=rumy7QmLUue.
net/ forum?
Kevin Meng, David cating and
Bau, Alex Andonian, and Yonatan Belinkov. Lo- editing factual associations in GPT. In NeurIPS, 2022. wccppaper_fillsspppeeeeee http://papert 82 -tt--bbbbbbbllll---tttt--eee-e-e
URL
Kevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. Mass- editing memory in transformer. In The Eleventh International Conference on Learning Repre- sentations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https: //openzeview net/pdf3 2 id=MkbcAHIXQyS..
Seyed-Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa, and Has- san Ghasemzadeh. Improved knowledge distillation via teacher assistant. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-eccond Innovative Appli- cations Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium Edu- cational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, URL https:
2020, pp. 5191-5198. AAAI Press, 2020. doi: 20.1609/aaai.334104.4 / //doi. org/10. 2009/aaa1 +341044 .4..344.56....
Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. Memory- based model editing at scale. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference on Machine Learning, ICMI. 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 15817-15831. PMLR, 2022. URL https: peettts:/ppoccee /ttpp/pecccggs.see/eeppppp mww.inn..s.....m..mmm.mmm......m press/ wtt.m//mmeeeeeeeeeeeeeee html.
Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. Pruning convolutional neural networks for resource efficient inference. In 5th International Conference Learning
13

---

Published as a conference paper at ICLR 2024
Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings.
OpenReview.net, 2017. URL https: wptp//eiee.eeeeee////e/ee httpss//peeneeeiei.... net http:/reewa..eeeeooommmoeee-eoo. formm?id=SJGcii55g.
Michael C Mozer and Paul Smolensky.
Skeletonization: A technique for trimming the fat from a network via relevance assessment. Advances in neural information processing systems, 1, 1988. URL htwps::/p//.cccc.t. htt.tppss//prrcceednggsssss
willes/paperr1// html.
hash/07e cddccaaacacc 67844494771111147777777777 0t00:///0000/0220200000022333660
OpenAL GPT-4 technical report. CoRR, abs/2303.08774, 2023. doi: 10.485504axXiv22333.7777
URL http:::too///...oo 00t 08774
Guillermo Ortiz-Jiménez, Alessandro Favero, and Pascal Frossard. Task arithmetic in the tangent space: Improved editing of pre-trained models. CoRR, 12s/2,355.1227, 2023. doi: 10.48550/ .2005. 12827.
atXiv.2305.12877 URL https: //to///o..//// //tp://rg/1/.......//// 4 48550/arXXi
Dae Young Park, Moon-Hyun Cha, Changwook Jeong, Daesin Kim, and Bohyung Han. Learn- ing student-friendly teacher networks for knowledge distillation. In Marc' Aurelio Ran- zato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neu- ral Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pp. 13292-13303, 2021. URL https: /ttp:////ccceeeeegeeccooo newrips. ctt pttppppppp///////////ep/ hash/
-bsstooaaaaeaabaaaaat........a.. html.
Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with GPT-4. CoRR, Abs/2304.032771, 2023, doi: 10:48550/r2X..23344.3777 URL https:
//doi. wwwoorr/////. /0/ 48550 (0155///axivv 2304. 03277.
Matthew E. Peters, Mark Neumann, Mohit lyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In Marilyn A. Walker, Heng Ji, and Amanda Stent (eds.), Proceedings of the 2018 Conference of the North American Chupter of the Association for Computational Linguistics: Human Language Technologies, NAACI-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), pp. 2227-2237. Association for Computational Linguistics, 2018. doi: 10.18653/v1/n133 1202 URL https: / //doi. org. 10.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. Language models as knowledge bases? In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.). Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pp. 2463-2473. Association for Computational Linguistics, 2019. doi: (0.18653/v1/D19-111 1250. URL https: //doi. org/ 10.
Hieu Pham, Zihang Dai, Qizhe Xie, and Quoe V. Le. Meta pseudo labels. In IEEE Confer- ence on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pp. 11557-11568. Computer Vision Foundation IEEE, 2021. doi: 10.1109/CVPR4447.. 0021.01139, hwt..httpss://peeccceee wew.rnacccesss..eeeeeeee tww.cf..ecccccc com http///ceeee/c//////eeeeee// Pham_Mettaa
waslabels_cccPP CVPR___C________________ C12__V22R22_2222 pacpllpaperrrt.. html.
Yujia Qin, Yankai Lin, Jing Yi, Jiajie Zhang, Xu Han, Zhengyan Zhang, Yusheng Su, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. Knowledge inheritance for pre-trained language models. In Marine Carpuat, Marie-Cathering de Marneffe, and Iván Vladimir Meza Ruíz (eds.), Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pp. 3921-3937. Association for Computational Linguistics, 2022. doi: 10.18653/w1/202.. maacl-main.21 URL https: //oiii////o//// oot. 1 /0t./55//v/2222 naacl wwaecmmmain... 288.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving lan- guage understanding by generative pre-training. URL https://coe www.ai..peaiiccoommm http://aaunuueuuuueerrsaaaaaaaa wttpeeuuudddrssinsnrn.nn
paper.pdf.
14

---

Published as a conference paper at ICLR 2024
Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge you pack into the parameters of language model? In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pp. 5418-5426. Association for Computational Linguistics, 2020. doi: 111865//11/22000mmmmmmmmaa 18653/v1 20063//12202..emm.pp
URL nittps://do org/10.
437.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR, abs/1910.01100 108, 2019. URL http://
waxiv.orrge abs /1910. 01108.
Sheng Shen, Pete Walsh, Kurt Keutzer, Jesse Dodge, Matthew E. Peters, and Iz Beltagy. Staged training for transformer language models. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference on Machine Learning, ICML. 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Pro- ceedings of Machine Learning Research, pp. 19893-19908 PMLR, 2022. URL https: //proceedings mlr. wall.ppesss prwsspresssee.........e
html.
George Stoica, Daniel Bolya, Jakob Bjorner, Taylor Hearn, and Judy Hoffmman. Zipit! merging models from different tasks without training. CoRR, 000./22355.0033, 2023, doi: 10.48550/arXiv. 2305.03053. URL https: /tap::://ddd /doi. /ttp///.../////ii...../ 2305, 03053.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-oolloinn llama model.
https: /ttl:://gitttie github. wtw/ttttttt/eeeeeee tatttttsu-laaa hww.ilaabstaaffrddooo swwaforrdaaaaaaetoooo 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Ar- mand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. CoRR, abs/2302. 13971, 2023a. doi: 1044850WarX...2322.13777 URL 2302 13971.
https:://do http://ti..og/...//// R.L10.485500
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lec. Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet. Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine- tuned chat models. CoRR, abs/2307 0998888 2023b. doi: 10.48550/arXiv.2307 230700988.... URL attps://dol www.o.org/....... 48550/ arxiv 2307. 09288.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp- 2017. URL https: http:/::/pcccccpeeeeeeeee neurips wwtpp/ppp///..../..e.cc /17hhhhh/hhhh/ 3f5ee243547deee 3abeee/433eeeee1oooooooooeeeeeee
Abstract html.
Peihao Wang, Rameswar Panda, Lucas Torroba Hennigen, Philip Greengard, Leonid Karlinsky, Rogério Feris, David Daniel Cox, Zhangyang Wang, and Yoon Kim. Learning to grow pretrained models for efficient transformer training. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023a. URL httt::://operevieen net stet/pdffdd==cDRsS...
15

---

Published as a conference paper at ICLR 2024
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjaaaa Arunkumar, David Stap. Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Ishan Purohit, Ishani Mondal, Jacob Ander- son, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal, Maitreya Patel, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy A, Sumanta Patro, Tanay Dixit, and Xudong Shen. Superrnatturalinstritiiosss Generalization via declarative instructions on 1600+ NLP tasks. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.). Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pp. 5085- 5109. Association for Computational Linguistics, 2022. doi: 10. 10.18663/1122222.mmmff mww.mmma........... URL https::/dd/t org/ 1eegg/1..16553. v1 2222 emn p-ma in..in..... 340.
Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannanch Ha- jishirzi. How far can camels go? exploring the state of instruction tuning on open resources. CoRR, 045555 2023b. doi: 11445550/rXxi.2230.0077777 URL https: /doi......./// org/ 10. 40.48550/arxi1 2306. 04751.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In Anna Rogers, Jordan L. Boyd- Graber, and Naoaki Okazaki (eds.), Proceedings of the 61 An- mual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp. 13484-13508, Association for Computational Lin- guisties, 2023c. doi: 10.18653/v1/2002 aonns//o//2.....lloongg.... 754. URL, https: / //wii www//rg///00 18653/ v1 v1/2023. acl - long. 754.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In NeurlPS, 2022. URL http::/papper nips. wwtpppccppppiiaeeeeee seraape__iles/pppeeee /paper/ 2022 2022/hash/ 0055091114444444111111117 aace34laaa44-aaa Abttaaaattttttttt-ooooeeeeeeee html.
Tao Wei, Changhu Wang, Yong Rui, and Chang Wen Chen. Network morphism. In Maria- Florina Balcan and Kilian Q Weinberger (eds.), Proceedings of the 33nd International Con- ference on Machine Learning, ICML. 2016, New York City, NY, USA, June 19-24, 2016, vol- ume 48 of JMLR Workshop and Conference Proceedings, pp. 564-572. JMLR.org, 2016. URL http:/ /ptppeeeeinggsssce wwwiiinngsmmrr.ee. www.s.pres//ee
weil6.htm..
Minghao Wu, Abdul Waheed, Chiyu Zhang, Muhammad Abdul-Mageed, and Alham Fikri Aji. Lamini-Im: A diverse herd of distilled models from large-scale instructions. CORR, 0bs/2304.14402, 2023. doi: 10.48550/arXi..2344..4444 14402. 48550/arXiv 2304 14402.
URL /zt./////......./oooooo
Canwen Xu, Daya Guo, Nan Duan, and Julian J. McAuley. Baize: An open-source chat model with parameter-efficient tuning on self-chat data. CoRR, abs/2304.01196, 2023, doi: 10.48550/arXiv. 2304.01196. URL https:/ / /ooi. org/10
2304. 01196.
Zhiqiu Xu, Yanjie Chen, Kirill Vishniakov, Yida Yin, Zhiqiang Shen, Trevor Darrell, Lingjic Liu, and Zhuang Liu. Initializing models with larger ones. International Conference on Learning Representations (ICLR), 2024.
Yiqun Yao, Zheng Zhang, Jing Li, and Yequan Wang. 2x faster language model pre-training via masked structural growth. CoRR, abs/2305.02869 2023a. doi: 10.48550/axxi.22205.2222 URL https::////... //t///doo /ooi..... org/10. 0etg//1..85588 118550/arxis. 2305. 02869.
Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems, methods, and opportunities. CoRR, doi: 10.48500aaXii.2255..1122 URL https: hopp:///oo. //ooi. org/10. 2tt15//axxiv............
0bbs22305.13172 2023b.
4 48550/arXis 2305.13177
16

---

Published as a conference paper at ICLR 2024
Jinghan Zhang, Shiqi Chen, Junteng Liu, and Junxian He. Composing parameter-cfficien modules with arithmetic operations. CoRR, abs/2306.14870, 2023, doi: 10.48550/axXwv2336..4533 URL 48550/
Ying Zhang, Tao Xiang. Timothy M. Hospedales, and Huchuan Lu. Deep mu- tual learning. In 2018 IEEE Conference on Computer Vision and Pattern Recogni- tion, CVPR 2018, Salt Lake City, UT. USA, June 18-22. 2018, pp. 4320-4322 Com- doi: 10.1109/CVR.22018 00454. URL http::/openaccesst wtr.reessstteeeee.eee com/ content_ccpr__21110 Dew_ee/wtbeelliieeennnoooorPPPP, 20perpaperrrr__pppppppppp_pp_rrr ptmaap...mll.......
puter Vision Foundation / IEEE Computer Society, 2018.
Ming Zhong, Yelong Shen, Shuohang Wang, Yadong Lu, Yizhu Jiao, Siru Ouyang, Donghan Yu, Jiawei Han, and Weizhu Chen. Multi-lora composition for image generation. arXiv preprint arXiv:2402.1 16843, 2024.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis. Luke Zettlemoyer, and Omer Levy. LIMA: less is more for alignment. CoRR, abs/2305.1 1226, 2023. doi: 10.48550/arXiw2230555 11206. URL httph:tp/////c /tte:i///o/ii// org/10.... 48550 a 2305 11206.
Wangchunshu Zhou, Canwen Xu, and Julian J. McAuley. BERT learns to teach: Knowledge dis- tillation with meta learning. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.). Proceedings of the 60th Annual Meeting of the Association for Computational Linguis- fies (Volume I: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pp. 7037-7049. Association for Computational Linguistics, 2022. doi: 10.18653/1/2222..cll1o /tt:://dooi..e http:p///ooo..orr///...6////////
URL
485.
17

---

Published as a conference paper at ICLR 2024
A
APPENDIX
A.1
VISUALIZATION FOR PARAMETRIC KNOWLEDGE
Sensitivity Score
Sensitivity Score
Layers
Mathematical Reasoning (a) GSM
Code Generation (b) Alpaca Code
Sensitivity Score
Sensitivity Score
Layer
Layers
Instruction-driven NLP Tasks
Open-ended Conversation
(c) Super NI
(d) LIMA
Figure 6: Visualization of parametric knowledge across different layers for four distinct task cate- gories. Darker shades represent higher sensitivity scores for each layer.
In our exploration, we also attempt to visualize the parametric knowledge intrinsic to different task categories. MMLU is omitted from the set tasks, given its encompassing knowledge from multi- ple domains, and introduce code generation (Chaudhary, 2023) additional task for analysis. LLaMA-1 30B serves as the teacher model, and we base our findings on 32 randomly selected seed samples, illustrating the sensitivity scores layer by layer. During the visualization process, sub-
18

---

Published as a conference paper at ICLR 2024
ject each parameter matrix min-max normalization, ensuring that sensitivity scores fall within the [0, 1] range. The insights from Figure 6 reveal that the distribution of parametrie knowledge across layers varies considerably among tasks. For instance, mathematical reasoning predominantly en- gages the bottom layer, instruction-driven NLP tasks concentrate on the bottom and middle layers, open-ended conversations
centered around the middle and upper layers, while code gener- ation appears to draw from all layers. This further emphasizes the efficacy of our sensitivity-bae knowledge extraction method in pinpointing task-specific parametric knowledge, thereby aiding the subsequent transfer processes between diverse models.
A.2
EXTENDED EXPERIMENT ON LLAMA-2 70B TO 7B
Table 3: Results for parametric knowledge transfer on LLaMA-2, with additional 70B to 7B transfer results included.
Models
GSM (0-shot)
MMLU (0-shot)
Super NI (R-L)
AlpacaFarm
Average
Vanilla 7B
3.34
41.70
Vanilla 13B
6.52
52.10
4.84
7B-LoRA
23.38
47.77
41.25
20.50
33.23
+ 13B Param.
25.30
49.37
42.98
24.64
35,57
+ 70B Param.
26.16
49.60
43.65
24.27
35.92
As shown in Table 3, we additionally exhibit the ILaMA-2 70B to 7B experiment for reference. Due the differences in attention architectures between LLaMA-2 70B and 7B models (with the 70B employing grouped-query attention and the 7B using multi-head attention), ric knowledge transfer from 70B to 7B to the FFN and embedding layers, which account for 0.36% trainable parameters. In contrast, in the transfer from the 13B to 7B model, we also include the atten- tion module, increasing the percentage of trainable parameters to 0.61% %. Nevertheless, the transfer from the 70B to the 7B model demonstrates greater performance gains than transferring from 13B to 7B. This implies that our parametric knowledge transfer approach becomes increasingly effective the teacher model scales up, even in the presence of architectural differences beyond the number of layers and dimensions.
restrict the paramet-
A.3 ANALYSTS OF THE NUMBER OF PARAMETERS
Table 4: Analysis of the effect of the number of parameters on the performance of parametric knowl- edge transfer.
Transfer Module
LoRA
Params.
13B to 7B
30B to 7B
Embedding
128
0.137%
27.33
27.52
Attention
16
0.248%
27.53
27.92
FFN
16
0.343%
27.87
28.23
All Modules
4
0.152%
27.93
28.37
Modules
0.304% 8
28.17
28.48
All Modules
16
0.608%
28.22
28.63
All Modules
32 1.216%
28.19
28.54
All Modules
64
2.432%
28.27
28.60
Our analysis of the origin of parameters (see Figure 5(b)) involves discussion of parameter amounts. For example, transferring combination of the embedding layer, FFN, and attention modules, which constitute 0.608% of the model's parameters, yields the best results. In contrast, transferring single module, like the FFN alone which accounts for 0.343% trainable parameters, leads to relatively poor performance.
To further investigate the effect of the number of parameters, extend the experiments at LLaMA- 13B to 7B and 30B to 7B by introducing comparisons with different LoRA r (intrinsic rank). The results of the average score the four datasets are listed in Table 4. For the transfer involving
19

---

Published as a conference paper at ICLR 2024
only the embedding layer, set LoRA at 128, ensuring that the number of trainable parameters remains comparable. We observe that increasing LoRA beyond 16 does not significantly enhance maintain LoRA at 16 for the
the results when transferring all modules. Consequently, periments presented in this paper. Our analysis indicates that the origin of the parameters has a more pronounced impact on the effectiveness of parametric knowledge transfer than the number of parameters transferred.
A.4 COMPARISON WITH DISTILLATION METHODS
The parametric knowledge transfer paradigm in this paper is fundamentally distinct from traditional distillation methods, characterized by the following differences:
Purpose and Focus: Rather than proposing better distillation methods, our study is fo- cused on exploring the transferability of implicit knowledge embedded in static parameters. While prior research has explored the detectability and editability of parametric knowledge, its transferability remains less explored. Our experiments provide empirical evidence in the knowledge transfer scenario, where student models show improved performance after ceiving task-specific knowledge from the teacher model, as shown in Tables and 2.
Process and Efficiency: Our approach differs from standard knowledge distillation, which typically requires fine-tuning or the direct involvement of the teacher model in student model training - a computationally intensive process. contrast, our parametric knowl- edge transfer involves extracting task-specific parameters from the vanilla teacher model and integrating them into the student model. This method, requiring only 32 inferences from the teacher model, offers significant efficiency advantage, especially in the context of LLMS.
Despite these differences, provide the results of the distillation methods reference in the knowledge transfer scenario. Table I contains the results for LLaMA-1 13B to 7B and 30B to 7B, where KD refers vanilla knowledge distillation (Hinton et al., 2015) and SeqKI is sequence-level knowledge distillation (Kim & Rush, 2016).
Table 5: Results for parametric knowledge transfer L. LLaMA-11 additonnlll including results from knowledge distillation methods.
Models
GSM (0-shot)
MMLU (0-shot)
Super NI (R-L)
AlpacaFarm
Average
Vanilla 7B
4.70
32.10
5.55
Vanilla
4.93
43.50
7.78
7B-LoRA
17.26
43.43
40.49
9.07
27.56
to 7B (KD)
17.69
43.57
42.08
9.32
28.17
13B to 7B (SeqKD)
17.86
43.33
41.91
9.36
28.12
13B to7B (Ours)
18.73
44.03
42.37
9.28
28.60
30B 7B (KD)
17.81
44.10
41.96
9.48
28.34
30B to 7B (SeqKD)
17.99
43.97
42.40
9.61
28.49
30B 7B (Ours)
18.63
45.20
43.08
9.40
29.08
All models fine-tuned with LoRA, using identical training data and hyperparameters. In this paper, we attempt to explore the evidence that knowledge in static parameters can be transferred between different LLMs, and knowledge transfer is the scenario in which find and provide pirical evidence. Our focus is not on proposing to find better methods in this scenario.
A.5 TRADE-OFF DISCUSSION BETWEEN PERFORMANCE AND RUNNING COST
Considering that users may have varying computational resources in practical application scenarios, we discuss the trade-off's between performance and running costs as follows:
Experimental details:
We conduct comparisons for two knowledge transfers: LLaMA-1 13B to 7B and 30B to 7B.
20

---

Published as a conference paper at ICLR 2024
Table Experimental results for the trade-off discussion between the performance and the running cost.
Transfer
Structure of Ext. Para.
Methods for Ext. Para.
Score
Inference Time
Memory
13B to 7B
Submatrix
Random
27.83
39 min
205 G
13B to 7B
Submatrix
Sensitivity
28.22
39 min
205
13B to 7B
Single Weights
Random
27.06
39 min
205 G
13B to 7B
Single Weights
Sensitivity
27.27
39 min
205
30B 7B
Submatrix
Random
28.11
82 min
510 G
30B to 7B
Submatrix
Sensitivity
28.63
82 min
510 G
30B 7B
Single Weights
Random
27.12
82 min
510
30B to 7B
Single Weights
Sensitivity
27.36
82 min
510
For the structure of extracted parameters, "submatrix" extraction involves directly taking sub-matrices from the teacher model's larger matrix that aligns with the student model's s dimensions. In contrast, "single weights" extraction means taking individual weights from the teacher model's larger matrix and arranging them into smaller matrices, maintaining their original order, to initialize the student model's LoRA module.
We compare random selection with our sensitivity score-based method for choosing pa- rameters.
The "Score" column represents the average performance across four benchmarks.
The running cost compared on CPU for 32 seed examples from GSM dataset, using teacher models of 13B and 30B. We perform backpropagation for each inference and based our experiments on fp32. Given that users may have limited GPU memory real-world applications, we conduct these experiments CPUs.
Key observations:
The sensitivity score-based method propose consistently outperforms random extrac- tion.
Extracting parameters via submatrices is significantly more effective than by single weights. This aligns with our discussion in Section 4.3 about the importance of main- taining the structural integrity parameters for successful parametric knowledge transfer.
Scaling up the teacher model from 13B to 30B consistently enhances performance comes with about 2. Lxthe runtime and 2.5x the memory usage.
We can observe that the performance of transferring from 30B to 7B (Submatrix + Random) is comparable to (slightly lower than) 13B 7B (Submatrix + Sensitivity). Thus, for applications with resource constraints, opting to randomly extract sub-matrices directly from a larger teacher model presents viable alternative, considering it reduces inference time.
21


Reviews:
[{"review_id": "jyUABcngvL", "reviewer": "Reviewer_Df53", "summary": "This paper proposes a method to transfer knowledge from big to small models by sharing the parameter. To be specific, it first detects and extracts a submatrix of parameters in large models, and the submatrix has the same size as the small model. Then, it decomposes the matrix into the Lora module and adds this module with the parameters of a small model. On Five datasets, this method brings about 2 points improvement compared to the original small model.", "questions": "1. I would like to see results from llama2 70b to 7b.\n\n2. In Formula 3, you use Taylor expansion to approximate the change of masking some parameters. But mask parameters are not trivial. Why can 1-order Taylor expansion approximate it?\n\n---I have read the whole rebuttal and raise my score from 5 to 6.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper proposes a novel method that transfers knowledge from large models to small models by sharing parameters.", "weaknesses": "1. The experiments are limited. This paper only tests on a few datasets for LLMs, does not discuss which amount of parameters is the best for transferring, and does not compare with other distillation methods, and does not compare with other methods about intrinsic dimension.\n\n2. The application of this method is limited. The paper does not experiment on different models. Can the knowledge transfer across architectures with this method?  This paper also assumes that the small model is already fine pre-trained. Can we transfer to the random-initialized model?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PmZFO2RVtf", "reviewer": "Reviewer_ccZi", "summary": "The authors propose to better understand knowledge transfer between LLMs by identifying/extracting/aligning knowledge-specific parameters between teacher/upstream and student/downstream LLMs.\nCompared to offlne/online distillation methods which transfer knowledge via synthetic datasets / teacher model outputs (offline) or use teacher activations as a loss/regularizer for the student (online), the proposed parametric knowledge transfer method seeks to identify and re-scale task-specific parameters in the teacher model and inject them into the student model via LoRA.\nThe authors use the gradient of parameters with respect to the loss across a number of sampled instances as a parameter set sensitivity measure, and extract parameters via 1) identifying the top LAYERS by task sensitivity; 2) extracting student matrix-size submatrices from the top teacher layer matrices to transfer; and 3) decompose the extracted parameters and fine-tune the student model with LoRA initialized from the decomposition.\nThe authors demonstrate that their method out-performs vanilla LoRA and fine-tuning of smaller student models across a variety of reasoning tasks.", "questions": "Q1 - Can the authors provide more detail about 3.2 - how exactly the submatrix (with highest cumulative sensitivity) is extracted from the larger-parameter teacher model layer to create a mapping matrix in the student model's parameter set? And why in particular extracting a submatrix (with arbitrary possible location in the larger 2D teacher layer matrix) works?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "S1 - The paper targets an important fundamental question about how knowledge is stored/represented across language models (e.g. are representations aligned / similarly structured across different pre-training or architectural regimes).\n\nS2 - The authors evaluate on a good variety of knowledge-rich tasks (reasoning, MMLU/professional QA, instruction tasks, and open dialog) to demonstrate significant improvements over vanilla LoRA and fine-tuning. It would be interesting to see a wider range of model sizes being compared or comparing models pre-trained on different corpora (e.g. LLaMA-1/2 cross-transfer, or something like GPT-J). \n\nS3 - There is a solid discussion of different factors that may affect parametric knowledge transfer here (layer selection, parameter location [ffn/self-attention], structure in transferred parameters, and number of seed samples)", "weaknesses": "W1 - The technical details (Section 3) are succintly written but could be written in a more clear and understandable way (e.g. summarize how standard LoRA is modified in knowledge injection). Certain details are explained in a more accessible manner in e.g. 4.3 but should be present in the main technical spec.\n\nW2 - The diagrams can be more clearly framed with larger font and clear colors. In particular, the legend for Figure 3 is very difficult to read.\n\nW3 - In 4.3, it seems that random submatrix initialization from the 30B model is copmarable to sensitivity-extracted submatrices from the 13B model. This merits further discussion about what the trade-offs are between just using random sub-matrices from larger models vs. the sensitivity method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A2QeG9L5VN", "reviewer": "Reviewer_NWu9", "summary": "Large Language Models (LLMs) inherently encapsulate a vast reservoir of knowledge within their parameters, acquired through pre-training on extensive textual data. The authors investigate the possibility of transferring parametric knowledge across Large Language Models (LLMs) of different sizes. They introduce a novel approach, examining knowledge transfer from a unique parametric angle. Employing a two-stage framework involving knowledge extraction and injection, they conduct thorough experiments on four varied benchmarks, confirming the natural transferability of model parameters. Additionally, through detailed analysis of the pivotal factors impacting parametric knowledge transfer, they provide valuable insights for future research in this field.", "questions": "Given the rapid evolution of LLMs, do you foresee the long-term viability and adaptability of the proposed knowledge transfer approach?", "limitations": "", "rating": 8, "confidence": 1, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The transferability of parametric knowledge is an important and timely topic that this paper addresses. Filling this gap is crucial in understanding the broader applicability and potential democratization of cutting-edge machine learning capabilities.\n\nThe paper introduces a novel parametric knowledge transfer paradigm. It moves beyond traditional methods like online and offline distillation and presents a unique approach focused on selecting specific static parameters from teacher models and injecting them into student models.\n\nThe experiments are conducted across various benchmark categories and LLM sizes. The results consistently demonstrate improvements in student performance after transferring teacher model parameters, providing strong empirical support for the proposed method.\n\nThe paper provides in-depth analysis of crucial factors such as teacher scales, initialization strategies, number of seed samples, and the nature of extracted parameters. This enriches the understanding of parametric knowledge transfer and its underlying dynamics.\n\nThe paper is clear and concise in writing.", "weaknesses": "How does your method compare with the existing distillation and pruning methods? It would have been great to see some other methods applied for comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)