Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
Authors: Xize Liang
Lin Yang
Jie Wang
Yiyang Lu
Runyu Wu
Hanzhu Chen
Jianye HAO
Venue: ICML 2025 poster (2025)
Keywords: curriculum learning
large language models
mixed data
Primary area: deep_learning->large_language_models

Abstract: The multi-domain fine-tuning of large language models (LLMs) confronts a notorious trade-off among abilities across domains. Existing studies attribute this trade-off to the conflicts between samples rooted in inherent semantics. Recent approaches attempt to mitigate these conflicts through the empirical investigation or heuristic strategies. However, without a fundamental understanding of interactions between samples, they yield only marginal improvements, while incurring substantial trial-and-error costs. To address this challenge, we move beyond empirical studies by modeling interactions between samples as their influence on each other's loss, estimated using gradients. Intriguingly, we find that these interactions **evolve throughout training** rather than being purely determined by inherent semantics. Building on this insight, we propose **EV**olving **I**nteraction-guided **C**urriculum (**EVIC**), which iteratively selects samples that positively influence the overall dataset for training. By dynamically adapting the training curriculum to prioritize samples that contribute the most to the model training, EVIC effectively mitigates conflicts and improves the sample efficiency. Extensive experiments on a mixed dataset covering coding, math, and general tasks with several model architectures show that EVIC significantly outperforms all baselines across diverse capabilities.

Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
Xize Liang? Lin Yang²
Jie Wang'
Yiyang Lu'
Runyu Wu
Hanzhu Chen Jianye Hao 23
Abstract
1. Introduction
Large language models (LLMs) have demonstrated remark- able capabilities in various fields, such as general instruction following (Yang et al., 2024b; Xu et al., 2024a; Dong et al., 2024), code generation (Jiang et al., 2024; Hou et al., 2024; Hui et al., 2024), and mathematical reasoning (Ahn et al., 2024; Imani et al., 2023; Luo et al., 2023). Building on these successes, cutting-edge LLMs are striving to integrate diverse capabilities across various domains, aiming better address read-world tasks and advance the realization of gen- eral artificial intelligence (Dubey et al., 2024; Yang et al., 2024a; Liu et al., 2024a; Adler et al., 2024).
The multi-domain fine-tuning of large language models (LLMs) confronts notorious trade-off' among abilities across domains. Existing studies attribute this trade-off to the conflicts between samples rooted in inherent semantics. Recent approaches attempt to mitigate these conflicts through the empirical investigation or heuristic strategies. However, without fundamental un- derstanding of interactions between samples, they yield only marginal improvements, while incur- ring substantial trial-and-eree costs. To address this challenge, we move beyond empirical studies by modeling interactions between samples as their influence on each other's loss, estimated using gradients. Intriguingly, we find that these inter- actions evolve throughout training rather than being purely determined by inherent semantics. Building on this insight, we propose EVolving Interaction-guicn Curriculum (EVIC), which it- cratively selects samples that positively influence the overall dataset for training. By dynamically adapting the training curriculum prioritize sam- ples that contribute the most to the model training. EVIC effectively mitigates conflicts and improves ratio. Extensive ex-
However, the multi-domani fine-tuning of LLMs still con- fronts significant challenges. In particular, training a model that excels across all domains extremely difficult, as im- provements in one capability often come at the expense of others (Wang et al., 2023a; Leybzon & Kervadec, 2024; Mueller et al., 2022). Existing studies attribute this issue to inherent conflicts between samples, arising from their dif- ferent semantics (Wu et al., 2024; Dong et al., 2023; Wang et al., 2023b; Ge et al., 2023). A common alternative is to train specialized models for each domain and then in- tegrate them (Feng et al., 2024; Zhou et al., 2024; Wang et al., 2024), but this inevitably incurs considerable com- putational costs. Furthermore, from the perspective of data resources, conflicts between samples hinder the effective use of high-quality data, which is especially valuable due to its scarcity and high acquisition costs (Li et al., 2023a; Liu et al., 2024b; Li et al., 2023b). Therefore, it is highly desir- able explore strategies to fully leverage mixed datasets for the enhancement of LLMs' multi-domain capabilities.
the serformance-to-sampee periments on a mixed dataset covering coding, math, and general tasks with several model archi- tectures show that EVIC significantly outperforms all baselines across diverse capabilities.
Existing efforts for this problem rely empirical investi- gation or heuristic strategies to manage training samples. Based on empirical insights gained from experiments on a mixed dataset containing samples from mathematics, code, and general domains, Dong et (2023) propose Dual-stage Mixed Fine-tuning (DMT), which trains on full amounts of specialized (i.e., math and code) data first and then trains general data with small amount of specialized data included. Such experience- approaches may be useful in certain scenarios but tend to be inefficient, requiring nu- merous experiments to discover suitable training strategies.
Work done during Xize Liang 's research internship at Huawei. <xizeliaag @ ust.l.sttc.du.cn. 'MoE Key Laboratory of Brain- inspired Intelligent Perception and Cognition, University of Sci- ence and Technology of China "Noah's Ark Lab, Huawei Technolo- gies 'College of Intelligence and Computing, Tianjin University. Correspondence to: Jie Wang <jiewangx www.aangg@ustc..cu........
Proceedings of the 4 International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Boosting Multi-Domain Fine- Tine-Tuning of Large Language Models through Evolving Interactions between Samples
ing the training curriculum to prioritize samples that contribute the most to the model training.
0.20
copla
0:15
0.35
0.10
0.30
0.05
0.25
mah
We conduct extensive experiments on mixed dataset containing 182, 166 samples covering the domains of mathematical reasoning, code generation, and general instruction following with Mistral-7B, LIama-3. 1-B. and Qwen2.5-14H5 14BB The evaluation results on GSM8K, HumanEval, and AlpacaEval 2.0 show that EVIC out- performs all baselines across diverse capabilities.
0.00
0.50
0.05
-.05
Qetiral
-0.10
-0.19
-0.15
code
ganeral
0230
ma2h
gencral
(a) Warm-Up
(b) Epoch 1
Figure 1. The interaction matrices computed after training Mistral- 7B for warm-up and one epoch. We randomly select 15 samples from cach domain generate the hcatmap. The indices in the figure, ordered from smallest to largest, correspond to samples from coding, math, and general domains. We divide the elements of both matrices by 10 and clip them to the range of (-0.2, 0.2).
2. Preliminaries
Supervised Fine-Tuning (SFT). An LLM To (with param- eters 0) generates a response y = [. ymm to the query x = [xi....,,ffl where the tokens (rz) and (y, ( come from a predefined vocabulary. in an autoregressive paradigm. Specifically, the model samples yj from the con- ditional probability distribution xa(- X1:j-1)) where V1:0 is null and y1:5-1 = Iy1..... My-1 for j = 2..... m. We can decompose the conditional probability To(y | x) into TH (y x) = 11 T x, Y1y-- We fine- tune the model on dataset D [(x(1, (00)) ) with the loss function C(:; D) = N E E(0, x(0) (0) =
Alternatively, Mixture-of-Skills (MoS) employs reward sig- nals based on heuristic metries such as transferability and difficulty to train scorer network, which is then used to as- sign sampling probabilities to training samples. Compared to experience-driven methods, heuristic-basee based methods of- fer more guided curriculum strategies but also lead to only marginal improvements due to the lack of a fundamental understanding of interactions between samples.
N Σm log (y x (1).
To address this challenge, we move beyond empirical stud- ies and propose novel LLM fine-tuning framework named EVolving Interaction-guide Curriculum (EVIC). Specifi- cally, we model the interactions between samples as their influence on each other's ' loss, estimated using (Adam) gradi- ents. Notably, we find that these interactions evolve during the training process (as shown in Figure 1), rather than being purely determined by inherent semantics. Inspired by this intriguing observation, EVIC periodically updates the interactions between each pair of samples and selects the samples that positively influence the overall dataset for training. By dynamically adapting the training curriculum to prioritize samples that contribute the most to the model train- ing, EVIC effectiivly mitigates conflicts and significantly improves the performance- -t--pppe ratio. That is, given the sample batch size, EVIC achieves higher performance with fewer training steps compared to existing methods.
Vanilla Curriculum Learning (CL). The key idea of vanilla CL is to train the model on a sequence of tasks or samples that gradually increase in difficulty (Bengio et al., 2009; Soviany et al., 2022). The model is initially presented with simpler samples, allowing it to learn funda- mental concepts before moving to more complex tasks.
Formally, for the dataset D = {(x("), y i== J cur- riculum consists of a sequence of subsets (De) K= where D2 = (x y ($) fen and the difficulty increases with k. Then, the model learns (Dx) in sequence. The vanilla CL approaches use various metrics to measure difficulty, such as sample complexity, model uncertainty, or domain- specific heuristics.
(())
Fine- Mone-Tuning with Adam. The Adam optimizer (Kingma, 2014) is widely used for fine-tuning LLMs, which combines the advantages of both adaptive gradient algorithms and momentum methods. Starting from the initial parameter By. the update rule of the Adam optimizer is
The main contributions of this paper are as follows.
We find that the interactions between samples esti- mated with (Adam) gradients evolve during the training process, providing insights into enhancing LLM fine- tuning on multi-domain datasets through progressive data management.
0: = 0--1 - 11-111 (((-))()-
r(0--)= = Vv. +e
mi
for t 2 1, where
We propose the EVIC framework, which effectively mitigates conflicts between samples and achieves supe- rior performance-tor sosmall ratio by dynamically adapt-
I-8 (3 11mm-1 + (1 C() (t2
m.
0,
(1= 0)
2

---

Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
and
(Xia al., 2024a), we use the Johnson-LLndeenstaaus (JL) transformation project the (Adam) gradients into 8192- dimensional space, which preserves inner products within an acceptable margin of error (see Appendix C.1 for details), Formally, we denote an (Adam) gradient g after projection as g E 1R81922 Then, for the dataset D = {(x(() ( } we use an interaction matrix Int(O,) E N X to represent the interactions between samples, where
Ve 1- (82v.-1 + (1- Vo (1 1) 0,
(1=0)
represent the first and second moment estimates, respec- tively, and Bi, B2 are hyperparameters that control the ex- ponential decay rates of these moment estimates. Here, we follow the notation I' from (Xia et al., 2024a) to denote the Adam gradient. We omit the batch of samples B, in (05(0,-1; B,) and 11,,-
Int (0)(i
Val (1ix())))) I (0Ex)))))
1; Be) for the simplicity of nota- tion, unless we want to emphasize B,.
projected gradicmt of 2(1)
projected Adam arudiont of .()
(2)
3. Evolving Interaction-guideed Curriculum
where Int(O.)(i,) 1) 2 indicates that 8j promotes the learn- ing of 8;, while Int(O.)li. i] <0 indicates the opposite. By the linearity of the inner product, the estimated influence of 8j the overall dataset is given by the j-th row sum of Int(O.), i.e., Σ Int(O.)(j..).
We propose EVolving Interaction-guider Curriculum (EVIC), novel approach with minimal manual design cost, sample-level data management, and superior performance- to-sample ratio. Specifically, we start from the perspective of learning dynamics and model the inter-sample interac- tions as their influence on each other's loss, estimated using (Adam) gradients (Section 3.1). Then, we observe that these interactions evolve throughout the training process, rather than being purely determined by intrinsic semantics (Sec- tion 3.2). Building on this insight, we periodically update the estimation of inter-sample interactions during training and select samples that positively interact with the overall dataset for subsequent training (Section 3.3).
3.2. Observations of Inter-Sample Interactions
Assumptions underlying existing approaches. Studies based on learning dynamics typically compute gradients and related quantities (e.g., sample importance) using early- stage checkpoints in the training process (Xia al., 2024a; Pruthi et al., 2020). Research on fine-tuning LLMs on multi- domain datasets, such as (Dong et al., 2023), generally treats all samples from the same domain as a whole, focusing on data management at the domain granularity. These practices imply two underlying assumptions as follows.
3.1. Modeling Interactions between Samples
The learning dynamics provides an effective way to un- derstand and quantify how samples influence each other during the training process. Inspired by (Xia et al., 2024a), which uses loss- dynamics to estimate the similarity between training and test samples, model the interac- tions between training samples as their influence on each other's loss. Consider two samples g(()) = ,x(0,y())) and 5(5) (x(6), y(,)),, ). the influence of أاا) on the loss of g(i) at parameter 0, can be estimated by the Taylor expansion as
1. sample interactions remain largely unchanged throughou the training process.
2. Samples within the same domain mutually enhance each other's 's learning.
However, our observations suggest that these two custom- ary assumptions do not always hold. On a mixed dataset composed of CodeAlpaca, GSM8K- (the three datasets correspond to domains of code generation, mathematical reasoning, and general instruction following, respectively. See Section 4. for details), we first randomly select 5% of the samples and perform warm-up training for Llama-3.1 1-BB to adapt it from the pre-training distribution to the SFT distribution. After that, we conduct one epoch of training on the entire dataset. We compute the interaction matrices as in Eq. (2) and randomly sample 15 samples from each domain to plot the interaction heatmap, as shown in Figure/ 1. From the heatmap, we make the following interesting observations.
and Alpaca-GPTT GPT4
(022 (02)ix() )0)) -e =(0::x(0)
2s Vol ( ((xxx),,(0)) T (04f) - 0.)
V (0ix00,)0 r (0rxx() x(ii)
= 7/
(1)
gradient of g(1)
Adam gradient of 6(3)
where 0+1 is the parameter after learning ((d) from O, and T) is the learning rate at the 1-th step.
Random gradient projection. For LLMs, the high di- mensionality of (Adam) gradients imposes a prohibitive computational overload on computing Eq. (1). Following
1. The signs and absolute values of the interactions be- tween many pairs of samples change significantly dur-
3

---

Boosting Fine- Fine-Tuning of Large Language Models through Evolving Interactions between Samples
Int (8 11..8 2
Code
Math
General
Interaction Matrix
Model Training
Multi-domain Dataset
Sample Selection and Fine-Tuning
(od(Ax y00) ((oxx()))
X
projected gradients
Gradients
Adam Gradients
projected Adam gradients
Interaction Matrix
Computing (Adam) Gradients
Updating Inter-Sample Interactions
Figure 2. Framework of EVIC. Given multi-domain mixed dataset, EVIC first obtain (Adam) gradients for the entire dataset in each stage. Then, EVIC computes the inter-sample interaction matrix using the (Adam) gradients. After that EVIC selects samples that positively interact with the entire dataset for training.
ing training, indicating that the interactions evolve dy- namically over the course of training.
Algorithm The EVIC framework
Input: dataset D
{s(1) N number of iterations M,
2. For some sample pairs (ss, 85). sign (Int a)) is not necessarily equal to sign(In IIt (0.) which means that the interaction relationship are asymmetric.
base model Tbase
Warm-Up and Initialization:
3: Do + Randomly selected 5% of samples from D
4: Train Tbase on Do to form ne,
3. Conflicts also exist among samples within the same domain, while some samples from different domains mutually promote each other's 's learning, suggesting that managing samples the sample-level may be more effective than at the devel.-level.
5: To +
for m 1 I...., M do
Interaction Matrix Computation:
Compute JL-projected (Adam) gradients for D)
Compute Int(Orm-,) according to Eq. (2)
10: Sample Selection:
3.3. Dynamic Interaction-guided Curriculum Learning
11:
Dm
(())
EN Int(@ 0f
The above intriguing observations inspire to select ples that have positive influence the overall dataset in staged manner during training, thus leveraging the dy- namic nature of interactions between samples. We summa- rize the EVIC framework Algorithm 1. Specifically, we first randomly select 5% of the samples (for ablation study on the proportion, please see Section 4.3) from the multi-domain dataset for warm-up training (Lines 2-5 in Al- gorithm 1). Then, compute the interaction matrix for the entire dataset (Lines 7-9 in Algorithm I) and select samples corresponding to non non-negative-sun rows for training (Lines 10-14 14 in Algorithm 1). After that, we update the interac- tion matrix, select samples, train the model, and repeat this process until the end of the training process.
12: Shuffle Dm
13: Model Training:
14: T0rm fine-tune To on D
15: end for
16: Output: 39TM
Discussion. Although our framework does impose constraint requiring every sample to be selected, periments show that nearly all samples are learned at least once. We provide an intuitive explanation for this, and leave
a more rigorous analysis of this phenomenon for our fu- ture work. First, because many inter inter- interactions change their signs during the training process, pairs of sam-

---

Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
Specifically, we compute the average metric by
Table 1. Statistics of the datasets and benchmarks for each
domain, and seftings for model inference and evaluation.
AVG = 3 (ACC+ P@T + P@10 T.C-WR WR
Domain
Dusin Dresin
Doest Diees Temp Seed
+
2
2
Code
CodeAlpaca 20,022
HumanEval
164
0.3 1-10
Math GSM8K RFT 110,142
GSM8K-test
1,319
0.0
We use vLLM (Kwon et al., 2023) for efficient model infer- ence. For more details, please see Appendix A.4.
General Alpaca-GPT4 52,002 AlpacaEvval 2.0 805
0.7 1-10
Baselines and Hyperparameters. We compare our pro- posed EVIC framework with the baselines as follows.
ples that are initially in conflict may gradually become less conflicting or even mutually promoting as the training pro- gresses. Second, empirically, gradients at parameters near local optima tend to have smaller magnitudes. Thus, even if a pair of samples remains in conflict throughout training, the absolute value of their interaction will diminish as one of the samples is learned. This causes the contribution of these conflicting pairs to the overall influence on the dataset to decrease, thereby increasing the likelihood of selecting samples that have not yet been learned.
Multi-Tass Mast Learning (MTL) trains the multi-domain dataset without any sample management strategies. MTL belongs to progressive CL methods as categorized in (So- viany et al., 2022).
Dpmaif-Specific Learning (DSL) trains only on the separate dataset for each domain, which reflects the upper limit of the model's specialized ability on each dataset. We use DSLeode: DSLmath. and DSL to represent the versions of DSL training only on code, math, and general samples, respectively.
4. Experiments
Dual-stage Mixed Fine-tuning (DMT, Dong et al. (2023)) trains on full amounts of specialized (i.e., code and math) samples first and then trains on general-domain samples with a small proportion of the specialized sam- ples included.
4.1. Experimental Settings
Tasks and Datasets. We focus on improving the comprehen- sive capabilities of LLMs in code generation, mathematical reasoning, and general instruction following, which are the three most attention-grabbee
abilities of LLMs at present.
Mixture-of-Skille (MoS, Wu et al. (2024)) employs rein- forcement learning with reward signals based on heuristic metrics such as transferability and difficulty to train a scorer network, and then use it to assign sampling proba- bilities to samples. MoS belongs to balanced CL methods as categorized in (Soviany et al., 2022).
We use three datasets for each domain: CodeAlpaca (Chaud- hary, 2023) for code generation, RSM8K-RFT (Cobbe et al., 2021; Yuan et al., 2023) for mathematical reasoning, and Alpaca-GPT4 (Peng et al., 2023) for general instruction following. For details of the datasets, see Appendix A.
Evaluation. We evaluate the performance of code genera- tion, mathematical reasoning, and general instruction fol-
Besides, we also add vanilla CL and self-pased CL as our additional baselines in Appendix B.2.
lowing of all models on HumanFyal (Chen et al., 2021), GSMSK-test (Cobbe et al., 2021), and AlpacaEval 2.0 (Dubois et al., 2024), respectively. The statistics of the datasets and benchmarks for each domain, and the settings for model inference and evaluation presented in Table 1.
Model Training, We conduct all the model training once with LoRA (Hu et al., 2021) implemented in the LLaMA- Factory framework (Zheng et al., 2024). We run DSL until its test performance no longer improves. For MTL, DMT, and MoS, we align their training process to three epochs, as their performance shows only marginal gains beyond three epochs. Because EVIC selects different samples at each stage, we measure its training duration in terms of iterations' 1 and training steps (with a batch size of 128), and ensure that the total training steps of EVIC do not exceed those of the baseline methods to maintain fairness. We run all experiments on 8 NVIDIA A100 GPUs (80GB). For more details of our baselines, hyperparameters, and the training process, see Appendix A.2.
Specifically, for HumanEval, we perform model inference with temperature of 0.3 and random seeds ranging from to 10, and report the mean and standard deviation of Pass @ and @ass@ 10 of all models. For GSMSK-test, we employ the greedy decoding with a temperature of 0.0 and report the accuracies. For AlpacaEval 2.0, we perform model in- ference with temperature of 0.7 and random seeds ranging from 1 to 10. We employ GPT the default evaluator of AlpacaEval-as the judge to compare the outputs of all mod- els with those of 4PT-4-Turbo, using the sength-controlled win rate and the win rate as metrics. In addition to the indi- vidual metrics for each domain, we also report their average.
An iteration refers performing one sample selection phase and training the model on the selected samples for one epoch.
5

---

Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
Table Performance of models trained on the multi-domain dataset with different methods, evaluated on multiple benchmarks. The bold font indicates the best result and an underline indicates the second- best result. Please note that DSL is not involved in the performance comparison and only serves as a reference for the upper limit of each ability.
Model
Mistral-7B-v0.3
Llama-3.1-8P
Qwen2.5-14B
Dont
GSMBK
HumanBval
AlpacuEval
GSM8K
HumanEval
AlpacaEvval
GSM8K
HumanEval
AlpacuEval
Metric
ACC
P@1
P0 LC-WR
WR
AVG
ACC
P@1
P@ 10 LC-WR WR
AVG
ACC
P@1
P@ 10 LC-WR
WR
AVG
DSLmath
59.1
0.2
144
0.6 0.7
20.2
68.3
6.7 19.0 0.9 L.
27.4
85.0
57.5
37..0.o81....1 81,0 3.8
3.3
52.6
DS
0.1
40.5
7.5 3,8+03
18.4
13.7
8.1+04
3.9
26.6
60.4
71.6+as
89.2-14
10.6
4.8-04
49.5
DSL
37.6
11.6ees33.9:z2
11.6 7.0
23.2
44.1
6.9+a4 23,0+21
12.5 6.6eas
22.9
73.6
3.1 13,4-15
19.3
10.8
32.3
MTL
56.2
21.31ox33.9
8.6
5.6
30.3
65.4
46.1
66.3
9.9
5.1 1uo
43.0
74.4
66.0
84.8
17.2 113
8.4
54.2
DMT
55.5
3.1
10.5
ou 6.7
24.2
65.3
Ho.3+oo69.55
12.9
6.9
43.5
77.9
55.6+11
114 18.1
8.9
54.6
MoS
56.9
43.2+4):3.2+-8
9.0=11
5.400s
33.3
65.1
44.5
10.7
5.9
43.4
79.6
66.7+0o
84.9-12
16.8
8,5
56.0
EVIC
57.1
37.8:0:56.9
9.8=05 5.9
* 37.4
65.8
69.9 11.6
6.2
44.3
79.7
69.1 86.1 17.9
9.2
* 57.0
Models. We conduct supervised fine-tuning with three pop- ular base Moodels-Mistral-715
learning in multi-domain datasets.
(Jiang et al., 2023), Llama-3.1-8B (Dubey et al., 2024), and Qwen2.5-14B (Yang et al., 2024a). We use pretrained models instead of instruction-tuned versions to prevent the models from having already seen some samples in our mixed dataset. For more details of the models, see Appendix A.3.
DMT exhibits overall stronger general capabilities than EVIC and MoS, which may be attributed its focus on the general data in the final training stage. However, DMT may lead to unsatisfactory coding performance, especially when code data is scarce (in our experiments, the amount of code data is one-fifth of the math data and less than half of the general data). This is because the small portion of code data added in the final stage cannot fully counteract the decline in coding ability caused by improvements in other capabilities.
4.2. Main Results
EVIC boosts the multi-domain fine-tuning of LLMs. We present the performance of different methods on multiple benchmarks in Table 2. Please note that DSL is not involved in the performance compurison and only serves as a refer- ence for the upper limit of each ability. From the table, we make the following observations:
MoS exhibits stronger coding ability than DMT, possi- bly because it can adaptively adjust data sample prob- abilities to handle imbalanced domain distributions. However, it still underperforms EVIC. We speculate that an important reason for this performance gap is that the heuristic reward signals in MoS do not directly relate to the model's optimization process, the inter- actions between samples do.
None of the baselines outperform all others across all metrics, regardless of the base model architecture. This confirms the existence of conflicts between abilities and samples from different domains.
Our proposed EVIC outperforms all baseline methods in terms of average performance across all metrics. Specifically, EVIC exceeds the best-performing base- line by four points on Mistral-7B and around one point Qwen2.5-14B. Furthermore, EVIC achieves either the highest or the second-highest results in the individ- ual metrics for each domain.
EVIC yields higher performance-to-sampl ratio. Table presents the number of training steps taken by different methods to achieve the performance shown in Table 2, with batch size of 128. For EVIC, we also report the number of training steps per iteration. As can be seen from the table, given the same batch size, EVIC achieves higher performance with fewer training steps compared to all the baselines. Specifically, the performance-to-sampp of EVIC is least 1.29 times that of MTL, that is, EVIC takes most 77.7% of the training steps taken by MTL to outperform all baselines across diverse capabilities, Even more remarkably, for Qwen2.5-148 these two numbers are 2.11 and 47.4%, highlighting the advantage of our approach in fine-tuning large-scate scale LLMs.
ratio
Although simple and straightforward. MTL sometimes achieves decent performance. Specifically, its aver- age performance on Mistral-7B surpasses that of DMT, and its accuracy for the mathematical reasoning and Pass @1 for coding on Llama-3. 1-BBBBBBBBBBBBBB-. exceed those of both DMT and MoS. Nevertheless, MTL remains over- all inferior to EVIC as well as DMT and MoS, indicat- ing the necessity of data management or curriculum
EVIC covers almost all samples. Table reports the sam-
6

---

Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
Table
Sample coverage rate (%) EVIC across different domains and over the entire dataset in each iteration.
Model
Mistral-7B-v0.3
Llama-3. 1-8B
Qwen2.5-144
Domain
Code
Math
General
Total
Code
Math
General
Total
Code
Math
General
Total
Iteration
68.99
45.37
40.27
46.51
83.42
86.09
71.75
81.70
80.73
72.62
55.85
68.72
Iteration
93.58
90.75
92.08
91.44
87.59
94.52
80.34
89.71
97.78
97.40
99.10
97.92
Iteration 3
97.49
96.11
95.78
96.17
99.86
99.96
99.87
99.92
Iteration 4
98.97
99.07
98.50
98.90
Table The number of training steps taken by different meth- ods to achieve the performance in Table [p and [DEvIc-stemm respectively represent the number of warm-up steps and the number of steps in the mith iteration. Devic|. [DDMT], [Dtusl, and DMTI. respectively represent the number of steps for EVIC (in total), DMT (3 epochs), MoS epochs), and MTL (3 epochs). We use batch size of 128 for EVIC and all baselines.
Table 6. Performance of Tlama-3.1-88 trained with non-
iterative variant of EVIC at each iteration. EVICiter and
EVICnon-ttern represent the standard version EVIC and the non-iterative variant for iterations, respectively.
Dose
GSM8K
HvmanEval
2
Metric
ACC
P@1
P@
LC-WR
AVG
EVIC
62.2
43.1
68.3
10.5
5.3
41.9
Model
Mistral
Llama3.1
Qwen2.5
EVIC EV1Cnou-teer2
63.4
45.2
65.2
10.6
5.1
42.2
EVICnon-tter
64.1
45.1
68.9
11.4
5.4
43.2
Dwarm-upl
72
72
72
DEVIC-Berl
661
1162
978
FVTC
65.8
46.3
69.9
11.6
6.2
*4.33
DEVIC-ne:2
873
959
973
DEVIC-ter3
654
1010
DBVIC-Meed
1,058
Pertorr Reffommat nce
-13
Sample Efficiency Ratio
DMTL (for
epochs)
4,269
4,269
4,269
DDMT (for 3 epochs)
>4,269
>4,269
>4,269
DMssl (for 3 epochs)
4,269
4,269
4,269
36
DEVIc (in total)
3,318
3,203
2,023
-1.1
Mveraba e
34
D DDMT
<1.00
<1.00
<1.00
D DMus
1.00
1.00
1.00
Average Performance
D Devicl
1.29
1.33
2.11
Sample Efficiency Ratio
5
20
100
Propartion of Warm-Up Samples (%)
Table Performance of Mistral-7B trained with the iterative variant of EVIC at each iteration. EVICier and EVICuon-iterm represent the standard version of EVIC and the non-iterative variant for m iterations, respectively.
Figure 3. Average performance across all benchmarks of Mistral-
7B trained with EVIC with different warm-up sample proportions (5%, 20%, and 100%). We default to 5% proportion for the warm-up data in our main experiments. The "sample efficiency ratio" refers to DacrL/DRvic) (defined in Table 4).
D
GSMBK
HumanEval
AlpacaFval 2
Metric
ACC
P@
P@10 10
LC-WR
WR
AVG
DVICaumsierrl
51.7
18.5
30.5
8.0
4.5
27.5
50.7
28.4
47.6
8.9
5.1
31.9
non-ten-
54.3
28.0
46.3
10.1
6.1
33.2
positively influences the overall dataset, we can conclude that nearly all samples are presented to the model at an appropriate time. This iterative adaptation of the training curriculum, which prioritizes samples that contribute most to the model training, effectively enhances the model's ss di- verse capabilities (as shown in Table 2) and significantly ratio (as shown in Table
non-tent
52.2
27.2
48.2
12.0
6.8
33.1
52.8
26.3
45.7
11.2
6.4
32.5
EVTC
57.1
37.8
56.9
9.8
5.9
*37.4
ple coverage rate EVIC across domains in each iteration, defined as the proportion samples that have been learned at least once, excluding the warm-up stage. From the ta- ble we find that, although the EVIC framework does not enforce every sample to be selected at least once, nearly 90% of the samples are learned within two iterations, and this proportion exceeds 96% within three iterations. Consid- ering that EVIC selects sample for training only when it
improves performance--o-sanpee 4). For an intuitive explanation of this feature, please see the discussion in Section 3.3.
4.3. Ablations
Iterative interaction computation is necessary. We con- duet ablations with Mistral-7B and 1.lma-3.1188 to verify

---

Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
the necessity of the iterative interaction computation in the EVIC framework. In the experiments, compare EVIC with its non-iterative variant, which only selects samples after the warm-up phase for multiple epochs of learning, without updating the iteration matrix periodically during the training process. Specifically, as shown in Table 3, the Mistral-7B and Llama-3. 1-BB models trained with the non- iterative variant are exposed to approximately 46% and 81% of all the samples, respectively. To align their total training steps with those of the standard version of EVIC shown in Table 4, we train Mistral-7B and 1.ama-3.1 1-8B using the non-iterative variant for five and three epochs, respectively,
Xu et al., 2024b; Wang al., 2023b; Ge et al., 2023).
Recently, some pioneering studies have explored this prob- lem. Dong et al. (2023) conduct extensive experiments and propose an empirical guideline for fine-tuning LLMs on mixed datasets that include math, code, and general sam- ples: to train on the full amount of math and code data, and then train on the general data with a small proportion of math and code samples included. This may work for some model architectures datasets, but the lack of theo- retical supports limits its reliability. Wu et al. (2024) use reinforcement learning to adjust focus dynamically on var- ious sub-datasets based on their current learning process. Others apply curriculum learning, using prior criteria such as prompt length, attention scores, and loss values to rank sample difficulty and design learning path from simple to complex tasks (Kim & Lee, 2024), but they do not consider conflicts between samples and are therefore not applicable to multi-domain mixed datasets. In contrast to the afore- mentioned methods that rely on empirical investigation or heuristic strategies, we move beyond empirical practices by modeling inter-sample interactions between samples as their influence on each other's loss, estimated using gradi- ents. Our approach, which delves deeper into the model training process, allows for more refined management of training samples and provides a reliable training curriculum, thereby improving the model's multi-domain performance and enhancing performance-tor -o-smll ratio.
The results are presented in Tables 5 and 6. As can be seen, the performance of the non-iterative variant improves as training progresses. However, despite with similar train- ing steps, its results remain significantly lower than those of the standard version of EVIC. In addition, we find that the performance of the non-iterative variant for Mistral-7B begins to decline after the fourth iteration. Although its performance on AlpacaEval 2.0 exceeds that of the stan- dard version at the fourth iteration, its average performance throughout the training process is least four points lower than that of the standard EVIC. Thus, we can conclude that: (1) The iterative computation of interactions between sam- ples and the corresponding sample selection are necessary for the effectiveness of EVIC. (2) The effective use of as many samples as possible contributes to the improvement of model performance, which further highlights the importance of EVIC's s superiority in the performance- sopormaanne-t-smmppl to-sle ratio.
Another line of research focuses on goal-directed fine- tuning, which aims to select data from multi-domain datasets relevant to specific capabilities, rather than enhane- ing the model's overall abilities. (Cao et al., 2023; Wu et al., 2024; Zhang et al., 2023; Xia et al., 2024b). Xia et al. (2024b) selects samples for training by predicting the similarity between training and test samples. In addi- tion, some methods rely solely on instruction information to identify relevant tasks (Lee et al., 2024), while others propose self-guided approach where the student LLM gen- erates task-specific input-output pairs and fine-tunes itself in multiple stages (Zhao et al., 2024). In contrast to the goal of the aforementioned lorks-leveraging mixed datasets to enhance the model's specialized abilities, we focus on improving the comprehensive multi-domain capabilities of LLMs. Furthermore. from technical perspective, we con- sider the interactions between training samples, which the aforementioned works do not take into consideration.
5% of the samples for warm-up is sufficient. In the main experiments, we default to using 5% of all samples as warm- up data to help the model adapt to the transition from the pretraining distribution to the SFT distribution. To evaluate the effect of this proportion, we conduct an ablation study with -B- as shown in Figure 3. Specifically, we report the average performance of the model across all benchmarks and I / DEvic| (defined in Table 4) with the proportions of 5% 20%. and 100%. As shown in the figure, increasing the proportion of warm-up samples leads to a slight performance improvement (less than 2%). but at the cost of a significant decrease in the performance-tor -oo sample ratio. Therefore, a 5% warm-up sample proportion is sufficient for EVIC to achieve satisfactory performance while maintaining a high performance-to-samye ratio.
5. Related Work
6. Conclusion and Future Work
Despite the impressive performance of LLMs in special- ized domains, research on training them on multi-domain datasets remains in its early stages (Wang et al., 2023a; Ley- bzon& & Kervadec, 2024; Mueller et al., 2022; Chen al., 2025; 2024). A key challenge stems in the confficts between samples from different sources and domains (Su et al., 2024;
In this paper, we propose EVIC, a novel approach to multi- domain fine-tuning of large language models, which ad- dresses the notorious trade-off between domain-specific abilities. By modeling the evolving interactions between training samples, we go beyond heuristic or empirical meth-
8

---

Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
ods, providing deeper understanding of how these interac- tions influence the model's learning process. Our approach not only improves performance-lo-sampll ratio but also mitigates domain conflicts, resulting in significant enhance- ments across various capabilities. Extensive experiments on a mixed dataset covering math, coding, and general instraction-follovingg tasks, using Mistral-7B, Llama-3. 8B, and Qwen2.5- 14B, demonstrate that EVIC outperforms all baseline methods, highlighting its effectiveness in hancing multi-domain performance.
Chaudhary, S. Code alpaca: An instractionnfollowinn llama model for code generation. GitHub repository, 2023.
Chen, H., Shen, X., Lv, Q., Wang. J., Ni, X., and Ye, J. Sac-kg: Exploiting large language models as skilled au- tomatic constructors for domain knowledge graph. In Proceedings of the 62nd Annual Meeting of the Associ- ation for Computational Linguistics (Volume 1: Long 4360, 2024.
Papers), pp.
Chen, H., Shen, X., Wang, J., Wang, Z., Lv, Q., He, J., Wu, R., Wu, F., and Ye, J. Knowledge graph finetuning en- hances knowledge manipulation in large language models. The Thirteenth International Conference on Learning Representations, 2025.
Our future work will further explore the theoretical expla- nations behind EVIC, including the dynamics driving its ef- fectiveness in improving multi-domain capabilities, the the- oretical foundations of its superior performance-to-sapee ratio, and why EVIC can nearly cover all samples without imposing the constraint that every sample must be learned.
Chen, M.. Tworek, J., Jun, H.,, Yuan, Q.. Pinto, D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021.
Acknowledgments
This work was supported in part by National Key R&D Program of China under contract 2022ZD01 19801, National Nature Science Foundations of China grants U23A20388 and 62021001. This work was supported in part by Huawei as well. We would like to thank all the anonymous reviewers for their insightful comments.
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M.. Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems. arXiv preprint arXiv: 141688 2021.
Dong. G., Yuan, H., Lu, K., Li, C., Xue, M., Liu, D., Wang, W., Yuan, Z., Zhou, C., and Zhou, J. How abilities in large language models are affected by supervised fine-tuning data composition. arXiv preprint arXiv:2310.05499. 2023.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Dong, G., Lu, K., Li, C., Xia, T., Yu, B., Zhou, C., and Zhou, J. Self-play with execution feedback: Improv- ing instruction-followii models. arXiv preprint arXiv:2406.13542 2024.
capabilities of large language
References
Adler, B., Agarwal, N., Aithal, A., Anh, D. H., Bhattacharya, P., Brundyn, A., Casper, J., Catanzaro, B., Clay, S., Co- hen, J., et al. Nemotron-4 340b technical report. arXiv preprint ar Xrxiv:2406.111 1/704, 2024.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21785 2024.
Ahn, J., Verma, R., Lou, R., Liu, D., Zhang, R., and Yin, W. Large language models for mathematical reasoning: Pro- gresses and challenges. arXiv preprint arXiv:2402.00157, 2024.
Dubois, Y., Galambosi, B., Liang, P., and Hashimoto, T. B. Length-controlled alpacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.00 2404.04475 2024.
Feng, W., Hao, C., Zhang, Y., Han, Y., and Wang, H. Mixture-of- loras: An efficient multitask tuning for large language models. arXiv preprint arXiv:2403.03433 2024.
Bengio, Y., Louradour, J., Collobert, R., and Weston, J. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pp. 41-48, 2009.
Ge, Y., Hua, W., Mei, K., Tan, J., Xu, S., Li, Z., Zhang, Y., et al. Openagi: When Ilm meets domain experts. Advances in Neural Information Processing Systems, 36: 5539-5566 2023.
Cao, Y., Kang. Y., Wang, C., and Sun, L. Instruction min- ing: Instruction data selection for tuning large language models. arXiv preprint arXiv:2307. 000,0, 2023.

---

Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang. K., Li, L.,, Luo, X., Lo, D., Grundy, J., and Wang, H. Large language models for software engineering: A systematic literature review. ACM Transactions on Software Engineering and Methodology, 33(8):1-79, 2024.
Leybzon, D. and Kervadec, C. Learning, forgetting, remem- bering: Insights from tracking IIm memorization during training. In Proceedings of the 7th BlackboxNLP Work- shop: Analyzing and Interpreting Neural Networks for NLP,, pp. 43-57, 2024.
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106. 09685, 2021.
Li, D., Nguyen, H. I.., and Zhang, H. R. Identification of negative transfers in multitask learning using surrogate models. arXiv preprint arXiv:2303. /4582, 2023a.
Li, M., Zhang, Y., Li, Z., Chen, J., Chen, L.,, Cheng, N., Wang, J., Zhou, T., and Xiao, J. From quantity to quality: Boosting Ilm performance with self self-guided data selection for instruction tuning. arXiv preprint arXiv:2308. 1.Niv:2308./2023 /2032, 2023b.
Hui, B., Yang, J., Cui, Z., Yang. J., Liu, D., Zhang. L., Liu, T., Zhang, J., Yu, B., Lu, K., et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409. 12186, 2024.
Imani, S., Du, L., and Shrivastava, H. Mathprompter: Math- ematical reasoning using large language models. arXiv preprint arXiv:2303.05398, 2023.
Li, X., Zhang, T., Dubois, Y., Taori, R., Gulrajani, I., Guestrin, C.,, Liang. P., and Hashimoto, T. B. Alpacaeval: An automatic evaluator of instruction-folloiing models, 2023c.
Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. I., Bressand, F,, Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. arXiv preprint arXiv:2310.0682. 2023.
Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al. Deepseek- v3 technical report. arXiv preprint arXiv:2411 24.::241221999 19437, 2024a.
Jiang, J., Wang, F., Shen, J., Kim, S., and Kim, S. A survey on large language models for code generation. arXiv preprint arXiv:2406.005: 00515, 2024.
Liu, Y., Tao, S., Zhao. X., Zhu, M., Ma, W., Zhu, J., Su, C., Hou, Y., Zhang, M., Zhang, M., et al. Coachlm: Automatic instruction revisions improve the data quality in Ilm instruction tuning. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), pp. 5184-5197. IEEE, 2024b.
Johnson, W. B. Extensions of lipshitz mapping into hilbert space. In Conference modern analysis and probability, 1984, pp. 189-206, 1984.
Kim, J. and Lee, J. Strategic data ordering: Enhancing large language model performance through curriculum learning. arXiv preprint (xXiv:2405.074900 2024.
Luo, H., Sun, Q., Xu, C., Zhao, P., Lou, J., Tao, C., Geng, X., Lin, Q., Chen, S., and Zhang, D. Wizard- math: Empowering mathematical reasoning for large lan- guage models via reinforced evol-instruct. arXiv preprint arXiv:2308.09553 2023.
Kingma, D. P. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.690,0 2014.
Mueller, D., Andrews, N., and Dredze, M. Do text-to- text multi-task learners suffer from task conflict? arXiv preprint arXiv:22 2.06645, 2022.
Kwon, W. Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Sym- posium Operating Systems Principles, pp. 61 611-626, 2023.
Peng, B., Li, C., He, P., Galley, M., and Gao, J. Instruc- tion tuning with gpt-4. arXiv preprint arXiv:2304.03277, 2023.
Pruthi, G., Liu, F., Kale, S., and Sundararajan, M. Estimat- ing training data influence by tracing gradient descent. Advances in Neural Information Processing Systems, 33: 19930, 2020.
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Soviany, P., Ionescu, R. T., Rota, P., and Sebe, N. Cur- riculum learning: A survey. International Journal of Computer Vision, 130(6):1526
Lee, C., Han, J., Ye, S., Choi, S. J., Lee, H., and Bae, K. Instruction matters: A simple yet effective task selection for optimized instruction tuning of specific tasks. In Pro- ceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 18620-18642 2024.
2022.
Su, Z., Zhang, J., Qu X., Zhu, T., Li, Y., Sun, J., Li, J., Zhang, M., and Cheng, Y. Conflicthank: A benchmark
10

---

Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
for evaluating the influence of knowledge conflicts in Ilm. arXiv preprint arXiv:2408. /2076, 2024.
for large language models: A survey, arXiv preprint arXiv: 2rXh::230810079 10792, 2023.
Wang, M., Adel, H., Lange, I.., Strötgen, J., and Schütze, H. Rehearsal-free modular and compositional con-
Zhao, C., Jia, X., Viswanathan, V., Wu, T., and Neu- big, Self-guide: Better task-specific instruction fol- lowing via self-synthetic finetuning. arXiv preprint 2rXiv:207.12874 12874, 2024.
tinual learning for language models. arXiv preprint OrXiv:2404.00730 2024.
Wang. Y., Ivison, H., Dasigi, P., Hessel, J., Khot, T., Chandu, K., Wadden, D., MacMillan, K., Smith, N. A., Beltagy. I., et al. How far can camels go? exploring the state of instruction tuning on open resources. Advances in Neu- ral Information Processing Systems, 36:74764-74786, 2023a.
Zheng. Y., Zhang, R., Zhang, J., Ye, Y., Luo, Z., Feng, Z., and Ma, Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv preprint arX1v:2403.13372. 2024.
Zhou, Y., Song, L.., Wang, B., and Chen, W. Metagpt: Merging large language models using model exclusive task arithmetic. arXiv preprint arXiv:2406. 1185, 2024.
Wang, Z., Zhong, W., Wang, Y. Zhu, Q. Mi, F., Wang, B., Shang, L., Jiang, X., and Liu, Q. Data management for large language models: survey. CoRR, 2023b.
Wu, M., Vu, T.TT., Qu, I.., and Haffari, G. Mixture-of- skills: Learning to optimize data usage for fine-tuning large language models. arXiv preprint arXiv:2406.088 2024.
Xia, M., Malladi, S., Gururangan, S., Arora, S., and Chen, D. Less: Selecting influential data for targeted instruction tuning. arXiv preprint arXiv:2402.0433. 2024a.
Xia, M., Malladi, S., Gururangan, S., Arora, S., and Chen, D. Less: Selecting influential data for targeted instruction tuning. arXiv preprint 40Xi::240..04333, 2024b.
Xu, C., Sun, Q. Zheng, K., Geng, X., Zhao, P., Feng, J., Tao, C., Lin, Q., and Jiang, D. Wizardlm: Empowering large pre-trained language models to follow complex in- structions. In The Twelfth International Conference Learning Representations, 2024a.
Xu, R., Qi, Z., Guo, Z.,, Wang, C., Wang, H., Zhang, Y., and Xu, W. Knowledge conflicts for Ilms: A survey, arXiv preprint ar arXiv:2403.083/9 2024b.
Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412. 24. 15/15, 2024a.
Yang. R., Song, L., Li. Y., Zhao, S., Ge, Y., Li, X., and
Shan, Y. Gpt4tools: Teaching large language model to use tools via self-instruction. Advances in Neural Information Processing Systems, 36, 2024b.
Yuan, Z.,, Yuan, H., Li, C., Dong, G., Lu, K., Tan, C., Zhou, C. and Zhou, Scaling relationship on learning math- ematical reasoning with large language models. arXiv preprint ar arXiv:2308.0882 2023.
Zhang, S., Dong, L., Li, X., Zhang, S., Sun. X., Wang, S., Li, J., Hu, R., Zhang, T., Wu, F, et al. Instruction tuning

---

Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
A. More Details about Experiments
A.1. Tasks and Datasets
To enhance the capabilities of large models in code generation, mathematical reasoning, and instruction following, we use three distinct datasets (i.e., CodeAlpaca, GSM8k-RFT, and GPT4-Alpaca) to train our models in these respective domains.
CodeAlpaca' (Chaudhary, 2023) is designed for training code generation models and contains 175,000 programming challenges in Python. Each challenge tests model's ability to generate syntactically correct and meaningful code, ranging from simple algorithms to more complex tasks.
GSM8K-RFT' (Cobbe et al., 2021) is employed to evaluate mathematical reasoning, containing over 8,000 arithmetic and algebraic problems that require multi-step reasoning. This dataset tests model's ability to solve complex numerical tasks including problem solving and logical deduction.
GPT4-Alpaca' (Peng et al., 2023) is utilized for instruction following and consists of 1,750 human-writteh tasks across multiple domains. Model is required to generate correct responses based on diverse instructions, assessing its capacity to understand and execute complex instructions.
A.2. Baselines, Hyperparameters, and Model Training
Baselines. We compare our proposed EVIC framework with the following baselines.
Multi-Task Learning (MTL) trains on the mixed dataset without any sample management strategies.
Domain-Specific Learning (DSL) trains only on the separate dataset for each domain, which reflect the upper limit of the model's specialized ability on each dataset. We use DSLcode. DSLmath, and DSLgen to represent the versions of DSL training only on code, math, and general samples, respectively.
Dual-stage Mixed Fine-tuning (DMT, Dong et al. (2023)) trains on full amounts of specialized (i.e., code and math) samples first and then trains on general-domain samples with a proportion of & of the specialized samples included. Dong et al. (2023) recommend to search for an optimal ic and thus explore its performance limits.
1/256 in their paper, but we conduct experiments using k = 1/64, 1/128, 1/256
Mixture-of-Skilk (MoS, Wu et al. (2024)) employs reinforcement learning with reward signals based on heuristic metrics such as transferability and difficulty to train a scorer network, and then use it to assign sampling probabilities to samples. We MoS with both of the metrics explore the upper limit of performance.
Hyperparameters. We use a learning rate of 2 x 10-5, the cosine learning rate scheduler, a batch size of 128 for all the methods. For LoRA, we use a rank of 128, = 512, a dropout ratio of 0.1, and learn LoRA parameters for all attention matrices.
Model Training. We conduct all the model training with LoRA (Hu al., 2021) implemented in the LLaMA- -aatoraatoryy framework (Zheng al., 2024). We run DSL until its test performance no longer improves. For MTL, DMT, and MoS, we align their training process to three epochs, as their performance shows only marginal gains beyond three epochs. Because EVIC selects different samples at each stage, we measure its training duration in terms of iterations and training steps (with batch size of 128), and ensure that the total training steps of EVIC do not exceed those of the baseline methods to maintain firness. We run all experiments on NVIDIA A100 GPUs (80GB).
prcetps://gitin con http://hh.....oaaaaa1111////eee.
*https: :// thub. com/ Instructii on oo- Toni hh- ch- 4/GGT-4-1111
"An iteration refers to performing one sample selection phase and training the model on the selected samples for one epoch.
12

---

Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
A.3. Models
We conduct supervised fine-tuning with three popular base models--iistra-7B--0.333 (Jiang et al., 2023), Llama-3.1-87) (Dubey al., 2024), and Qwen2.5-14B" (Yang al., 2024a). We pretrained models instead of instruction-tumed versions to prevent the models from having already some of the samples in our mixed dataset.
Mistral-7B-v0.3 is an open-source language model developed by Mistral Al. With 7 billion parameters, it excels in performance, and is designed for efficiency and versatility.
1lama-3.1-80 is large-scale language model by Meta, featuring billion parameters. It excels in tasks such as language understanding, generation, and translation across various domains.
Qwen2.5-14B is robust language model developed by Qwen AI, offering 14 billion parameters. It focuses on enhancing generative tasks, including content creation and problem-solving, delivering highly accurate and context-aware outputs.
A.4. Evaluation
For models trained on CodeAlpaca, GSM8k-RFT and GPT4-Alpaca, we evaluate their performance on the HumanEval, GSM8k-test and AlpacaEval benchmarks.
HumanEval (Chen et al., 2021) consists of 164 programming problems that cover a range of common tasks in the Python programming language. Each problem requires the model to generate a valid Python code solution based on a given problem description. The difficulty of the problems ranges from basic algorithmic tasks to moderately complex programmiing challenges. We perform model inference with temperature of 0.3 and random seeds ranging from to 10, and report the mean and standard deviation of Pass and Pass@ 10 of all models.
GSM8K (Cobbe et al., 2021) test set a subset the GSM8K dataset, consisting of more than 8,000 mathematical problems. These problems are designed evaluate models' math reasoning abilities, particularly in solving arithmetic and algebraic tasks that require multi-step reasoning. We employ greedy decoding with temperature of 0.0 and report the accuracies evaluate models with GSM8k test set.
AlpacaEval (Li et al., 2023c) consists of 800 instruction tasks across various domains, such as information retrieval, task execution, reasoning, and classification. Each instruction is human-written, and the model's task to generate the correct response based on The dataset tests the model's ability to understand and follow diverse instructions. We perform model inference with temperature of 0.7 and random seeds ranging from to 10. We employ GPT-4-the default evaluator of AlpacaEval-an the judge compare the outputs of all models with those of GPT-4, using the length-controlled win rate and the win rate as the metrics.
B. More Experiments
B.1. Number of times different samples are selected
We report the numbers of times different samples being selected during the training of Llama-3.1-8B and Mistral-7B in Figure 4.
B.2. More Curriculum Learning Baselines
The baselines in our initial submission (DMT and MoS) belong to progressive CL and balanced CL methods as categorized (Soviany et al., 2022). Additionally, we add vanilla CL and self-paced CL as baselines. As shown Tables 7 and EVIC, DMT, and MoS significantly outperform other CL. baselines, underscoring the importance of CL. methods tailored for multi-domain fine-tuning.
wtt./mistraaaaiii "https: /tatps: c 1amm-111ammmmmma-a55 3-18 http://aaaceeeoeeeeeeeeeeeeee 5-14B
'https:
13

---

Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
Serand
2000
1009
1200
(a) Code (Mistral-7B-vv.3)
(b) Math (Mistra-7B--O.3..
(c) General (Mistral-B--vo)
Total (Mistral-7B-v0.33
2000
(c) Code (Llama-3.1-88)
(f) Math (1.ama-3.1188)
(g) General (Llama-3.
(h) Total (1lama-3.1-8B)
Figure 4. Number times different samples are selected.
Table Performance of Mistral-7B-v0.3 trained the multi-domain dataset with different methods, evaluated multiple benchmarks. Vanilla CL. uses query length to measure the difficulty of samples and trains the model an "easy-to-hard" paradigm accordingly. Self Self-p CL measures the difficulty samples using model's loss value the sample and also adopts the "casy-to-hard" manner.
Drest
GSM8K
HumanEval
AlpacaEval 2
Metric
ACC
P@ 1
P@ LC-WR WR
AVG
MTL
56.2
21.3:0433.9e13
8.6
5.6
30.3
DMT (progressive CL)
55.5
3.1 13.7
10.5
6.7
24.2
MoS (balanced CI.)
56.9
28.61::43.2au
9.0gi
33.3
Vanilla CL
47.9
3.1
13.2
8.3
5.1
20.9
Self-paced CL.
48.4
3.5
11.9
7.9
4.0
20.7
EVIC
57.1
37.8
9.8ans
5.9
37.4
Table 8. Performance of Llama-3.1-8B trained on the multi-domain dataset with different methods, evaluated on multiple benchmarks. Vanilla CL. uses query length measure the difficulty samples trains model an "casy-to-hard"" paradigm accordingly. Self-pased CL measures the difficulty of samples using the model's loss value the sample and also adopts the "easy-to-hard" manner.
D
GSMBK HumanEval
AlpacaFval
2
Metric
P@
P@
LC-WR WR
AVG
MTL
65.4
66.1::66..31a0
9,9 4,1
43.0
DMT (progressive CL)
65.3
41.3+os69.55
12.9
6.9
43.5
MoS (balanced CL)
56.9
28.6:443..2.
9,0
5,4
33.3
Vanilla CL
59.3
3.4
15.9
9.6
5.8
25.6
Self CL
56.6
2.3
11.0
9.5
5.2
23.5
EVIC
65.8
46.31n.69.9.cc
11.6
6.2
44.3
14

---

Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples
C. Mathematical Derivations
C.1. Introduction Johnson-Lindenstrauss (JL) Transformation
The Johnson- Linnest-Linddsssrruusss (JL) Transformation((Jonnsoo, 1984) is a method for reducing the dimensionality of high- dimensional data while approximately preserving the pairwise distances and inner products. The transformation projects data points into lower-dimensional subspace and ensures that the Euclidean distance between any two points remains the same within specified error bound.
In practice, to reduce the computational cost on Eq. (I), we employ the JL transformation to project the (Adam) gradients into an 8192-dimensional space. following (Xia et al., 2024a). Specifically for a given parameter 9 and two samples (+) (x())y' (()) and و(ي) = (x()), yli)), we can compute the ddddimensional projections of the gradient of g(() and the
Adam gradient of y(3 by
Vel (1:x(())) xx(1)) = II, Voe (0;x(() yy)))
(3)
i (01xx(), y(ii = I
y()).
(4)
with 1I,, III E R Pxd drawn from a Rademacher distribution. the equations, represents the dimension of the gradient, Vol (f1;)((), y()) represents the projected gradient of s(0) and f (0,: y()) represents the projected Adam gradient of sl)) In our method, we choose d = 8192 following (Xia et al., 2024a).
15


Reviews:
[{"review_id": "UXcS2hbfM1", "reviewer": "Reviewer_q9CU", "summary": "This work presents a curriculum learning method to improve the multi-domain fine-tuning of LLMs. Specifically, the idea is to model the Adam gradient interaction between examples and select the example with the best total benefit on learning other examples. This whole process starts with a warmup stage with around 5% of the data and an iterative process using the aforementioned method. In the evaluation on three different datasets, this method is shown to be better than simple multi-task learning and some other baselines in term of jointly learning all the tasks.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer q9CU,\n\nThank you for your valuable review. We respond to each comment as follows and sincerely hope that our response can properly address your concerns.\n\nTables can be found in **q9CU.md** in **https://anonymous.4open.science/r/ICML25-EVIC-D5E8**\n\n# Claims And Evidence\n\n> C1: There is no curriculum learning (CL) baseline in the comparison. It is imporatant to compare EVIC with some basic CL methods.\n\n**Res:** Apologies for the lack of clarity in our paper. The baselines in our initial submission (DMT and MoS) belong to **progressive CL** and **balanced CL** methods as categorized in [1,2]. We will refine this in the paper.\n\nAdditionally, we have added **vanilla CL** and **self-paced CL** as baselines. Thus, our experiments have covered **four CL method categories**. **Tables q9CU-1 and q9CU-2** in the anonymous link show that EVIC, DMT, and MoS significantly outperform other CL baselines, underscoring the importance of CL methods tailored for multi-domain fine-tuning.\n\n> C2: EVIC seems to be computationally expensive due to the gradient computation for every sample when computing the interaction matrix.\n\n**Res:** We humbly believe the additional computational cost of the interaction matrix is **acceptable and worthwhile**. Additionally, to reduce costs and improve efficiency, we provide some ideas on implementations and methodological improvements.\n\n1. EVIC’s gradient computation and total cost are **less than twice** that of MTL, which is accpetable and worthwhile for real-world applications. In multi-domain LLM fine-tuning, **performance bottlenecks are more challenging than efficiency bottlenecks**, as extending training time often fails to overcome performance limits. Our proposed EVIC effectively improves the performance via iterative interaction estimation, thereby making a significant contribution.\n\n2. However, the efficiency and cost concerns you mentioned are also important. For full dataset gradient computation, we parallelize computations across eight GPUs. **We would like to point out that there is a trade-off between efficiency and performance.** To reduce costs, we could use historical gradients of some samples without recomputing them, but this would result in inaccurate interaction estimates and thus decrease training performance. In our initial submission, **we prioritize performance over efficiency**, but in efficiency-critical scenarios, using historical gradients could be explored. \n\n# Methods And Evaluation\n\n> M1: Why computing interactions with \"Adam\" gradients? Does the momentum heavily influence the calculation in Sec. 3.1? How much of the evolution of interactions is due to the momentum term?\n\n**Res:** Because the interactions are modeled as influences between samples during \"training\" and LLMs are usually trained with Adam optimizer, the \"Adam\" gradients naturally appears in the computation of interactions. In a nutshell, our interactions are **defined based on** Adam gradients the corresponding momentum.\n\nWe further add experiments to demonstrate the rationale behind computing interactions based on Adam gradients, where we compare EVIC with its variant that computes interactions using the inner product of the original gradients. Please see **Table q9CU-3** in the anonymous link for details due to rebuttal length constraints.\n\n# Experimental Designs Or Analyses\n\n> M2: I can't find how $M$ and the total length of each iteration is determined in this paper.\n\n**Res:** We provide the relevant information in Lines 248-252 (right column) in our initial submission but acknowledge it lacks detail. We will revise it as follows.\n\nSpecifically, we set the number of iterations $M$ to be as large as possible while ensuring that the total training steps of EVIC do not exceed those of baselines to maintain fairness. Thus, $M=4,3,2$ for Mistral, Llama, and Qwen experiments, respectively. As for the length of each iteration, it refers to the number of training steps required to learn all selected samples in this iteration, i.e., $|D_m|/{\\rm batch size}$.\n\n# Other Comments Or Suggestions\n\n> S1: Adding some qualitative examples of how the models select examples will make it more intuitive to understand how EVIC works.\n\n**Res:** Thanks for your valuable suggestion. We have added some qualitative examples in Figure q9CU-1 in the anonymous link, but we are not sure if this is exactly what you want. **Could you please clarify your suggestion in more details so that we can better follow it?**\n\n> S2: Definitions of LC-WR and WR should be added when first introduced.\n\n**Res:** We will add the definitions based on [3]. Due to the rebuttal length constraints, we cannot provide the definitions here. We would greatly appreciate your understanding.\n\n> S3: Two repetitive references.\n\n**Res:** We will make modifications accordingly.\n\n---\n\n[1] Curriculum Learning: A Survey.\n\n[2] A Survey on Curriculum Learning.\n\n[3] Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators.", "claims_and_evidence": "This work presents the EVIC method for multi-domain fine-tuning. The experiments are quite convincing in demonstrating the performance improvement with EVIC and how it outperforms MTL and some other baselines. However, I have two main concerns about the evaluation:\n\n1. There is no curriculum learning baseline in the comparison. I understand that curriculum learning methods are usually not specially designed for multi-domain fine-tuning. However, it is still really important to compare EVIC with some basic curriculum learning methods given how similar these methods are on a high level. This can also serve as ablations to demonstrate the effectiveness of the sample interaction modeling part.\n\n2. The EVIC method seems to be really computationally expensive due to the need to compute the gradient for every example when computing the interaction matrix. This makes scaling this method to large datasets difficult. Additionally, this makes all the questions around sample efficiency questionable. From a practical perspective, the more important value that practitioners care about is how many times you compute the gradient on any example, and I don't see an efficiency advantage on that from EVIC.", "methods_and_evaluation_criteria": "The method makes sense on a high level, but I wonder why the authors compute sample interactions with \"Adam\" gradients. Momentum plays a huge role in Adam, and I feel it also heavily influences the calculation in Sec. 3.1. Do the authors have any thoughts or have the authors done analyses on this point? Additionally, lines 159-160 mention that the interaction evolves over the course of the training. How much of this evolution is due to the momentum term?", "theoretical_claims": "No theoretical claims.", "experimental_designs_or_analyses": "I can't find how important hyperparameter like M and the total length of each iteration is determined in the paper. These are very important hyperparameters for this method.", "relation_to_broader_scientific_literature": "Multi-domain fine-tuning is a practically important but relatively underexplored field. This paper introduces a novel algorithm on this based on prior literature of curriculum learning and sample influence modeling.", "essential_references_not_discussed": "No essential major references.", "other_strengths_and_weaknesses": "n/a", "other_comments_or_suggestions": "1. If the authors can add some qualitative examples of how the models select examples, it will make it more intuitive to understand how this method works.\n2. Definitions of LC-WR and WR should be added to the paper when they are first introduced.\n3. There are two repetitive references to Xia et al., 2024.", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3T0iA3OnYw", "reviewer": "Reviewer_JcLd", "summary": "This paper leverages a way to estimate the training data samples' influence on each other by leveraging the gradients of Adam and projecting it into a lower dimensional space from Xia et al. 2024 [1] and iteratively using this computation in order to select samples to train in the multi-domain fine-tuning setting. They demonstrate that \n\n[1] Xia, Mengzhou, et al. \"Less: Selecting influential data for targeted instruction tuning.\" arXiv preprint arXiv:2402.04333 (2024).\n\n### Update after rebuttal\nGiven the authors' additional experiments provided, I am inclined to raise my score to 4. I stand corrected in that the authors do show that their method can on average improve scores across different benchmarks. However, I recommend that these two points should be addressed for the updated camera ready/next version.\n1. Random seeds should be computed across training runs, not LLM evaluation. Yes I do agree that there is variability in LLM evaluations, but to show consistency in performance improvement, the training run seeds are much more important.\n2. The plot of the training iterations/samples vs performance that the authors provided is a great start, although it lacks details about which dataset it was conducted on. Because the novelty of this paper is not the derivation for computing influence of training samples but applying that to multi-domain finetuning, this kind of analysis of how the method is sample efficient compared to other methods seems very important.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer JcLd,\n\nThank you for your valuable review. We respond to each comment as follows and sincerely hope that our response can properly address your concerns.\n\nFigures and Tables can be found in **JcLd.md** in **https://anonymous.4open.science/r/ICML25-EVIC-D5E8**\n\n# Other Strengths And Weaknesses\n\n> W1: It would be poignant to clarify the contribution of this paper is not to derive the influence computation using Adam gradients.\n\n**Res:** We will accordingly modify our presentation.\n\n> W2: Often times they observe marginal improvement and sometimes no improvement (within the std dev). For example, in the Qwen and Mistral experiments.\n\n**Res:** We confidently yet humbly believe **EVIC significantly outperforms existing methods** for the following reasons:\n\n1. Our focus is on the **overall (AVG) performance improvement** in multi-domain fine-tuning. EVIC outperforms the second-best method in terms of AVG by 4.1, 0.8, and 1.0 on Mistral, Llama, and Qwen, respectively. We believe this improvement is **significant**, as **even a 0.5 gain on a multi-domain datasets with inter-sample conflicts is challenging**.\n\n2. We respectfully speculate that you may find EVIC's improvement in some single domains marginal. However, please note that **EVIC consistently ranks first or second across all domains**, which is also a remarkable and challenging achievement.\n\n> W3: Table 3 does not provide a holistic picture of how the evaluation metrics are evolving over training across different methods of multi-domain finetuning. Could you actually provide a more informative version of Table 3 where we can visualize the evaluation metrics over the training?\n\n**Res:** Due to the deletion of intermediate results and checkpoints for some baselines, we need to retrain them. With limited computational resources, the experiment is still in progress. **We will complete it before the discussion deadline and provide more informative tables or figure as suggested.** We would greatly appreciate your understanding.\n\n# Other Comments Or Suggestions\n\n> S1: Can this method be adopted to calculate the influence between test samples and training samples, and use the scores to select training samples?\n\n**Res:** The approach of calculating the influence between test and training samples is orthogonal to and compatible with our method. However, it faces challenges in multi-domain fine-tuning of LLMs. When test samples conflict (e.g., the evaluation is across different domains), even positively influencing training samples may still conflict with each other, hindering the performance. This brings us back to the core challenge of \"how to conduct multi-domain fine-tuning.\"\n\n> S2: How are the standard deviations in Table 2 calculated? How many runs of the experiments were done?\n\n**Res:** We provide relevant information in Lines 254-264 (left column) of our initial submission but acknowledge it lacks detail. We will revise it as follows. \n\nSpecifically, all training is conducted once using the same random seed due to limited resources. For inference:\n- HumanEval: We run model inference with a temperature of 0.3, using random seeds from 1 to 10. Then, we report the mean and standard deviation (std dev) of Pass@1 and Pass@10 using Numpy.\n- GSM8K-test: We use greedy decoding with a temperature of 0.0 (so there is no std dev) and report accuracies.\n- AlpacaEval 2.0: We run inference with a temperature of 0.7, using random seeds from 1 to 10. We use the AlpacaEval library, with GPT-4 as the judge, to compare model outputs against GPT-4 Turbo outputs, and report the mean and std dev of (length-controlled) win rate using Numpy.\n\n> S3: How expensive is computing the influence at each iteration of EVIC?\n\n**Res:** The cost of calculating influence (interactions) in each iteration is mainly dominated by the gradient computation for all samples, which is rougthly equivalent to performing one epoch of MTL. We humbly believe the additional computational cost of the interaction matrix is **acceptable and worthwhile**. For more details, please see our response to **C2 of Reviewer q9CU** due to the rebuttal length constraints.\n\n# Questions For Authors\n\n> Q1: Please see W3.\n\n> Q2: How does the frequency or sample selection size of each iteration change the dynamics of EVIC?\n\n**Res:** The higher the frequency and the fewer samples learned per iteration, the more accurate the gradient computation and interaction matrix estimation become, leading to better EVIC performance. However, excessively high frequency would result in numerous full-dataset gradient computations and extremely high costs. Therefore, in our initial submission, we simply set the frequency and sample selection size as follows.\n- Sample selection size: All samples with a non-negative row sum in the interaction matrix are selected, with no limit on the number.\n- Iteration frequency: After all samples selected in the previous iteration have been learned, the process moves on to the next iteration.", "claims_and_evidence": "The claims are sound.", "methods_and_evaluation_criteria": "The proposed methods and evaluation are sensible for this problem.", "theoretical_claims": "Yes.", "experimental_designs_or_analyses": "Yes, the experimental setup seems sound.", "relation_to_broader_scientific_literature": "This relates to the language modeling community in investigating how to build models that can excel in different domains when trained altogether.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "Strengths\n- The paper is clearly written. The problem and the proposed solution (iterative solution to estimate influence and select data samples) are clearly described. A caveat here is that the contribution of the paper isn't to derive the method using Adam gradients to calculate the influence function and it would be poignant to clarify the contributions of this paper vs leveraging insights from other works. \n- The experiments are somewhat comprehensive across different benchmarks and models.\n\nWeaknesses\n- The results are generally positive, but often times they observe marginal improvement and sometimes no improvement (within the std dev) over other methods at all. For example, in the Qwen and Mistral experiments. \n- The sample efficiency is an important point for this method, but Table 3 doesn't necessarily provide a holistic picture of how the evaluation metrics are evolving over the training across the different methods of multi-domain finetuning. It would be very helpful to have that as I mention in the questions below.", "other_comments_or_suggestions": "1. A curiosity is whether one can adopt the approach of leveraging test samples to calculate their similarity like in Cao et al. 2023; Wu et al. 2024) in this proposed method to calculate the influence between the test samples vs the training samples and then use the scores to select the training samples. This could verify whether influence computation and sample selection indeed are effective, by leveraging privileged information about the test samples.\n2. How are the standard deviations in Table 2 calculated? How many runs of the experiments were done?\n3. How expensive is computing the influence at each iteration of EVIC?", "questions_for_authors": "1. Could you actually provide a more informative version of Table 3 where we can visualize the evaluation metrics over the training (x-axis being the % of total data seen and y-axis the evaluation benchmarks, for example)? It would be important to see the training dynamics and the sample efficiency that way.\n2. The authors argue that iterative computation is necessary, but how does the frequency or sample selection size of each iteration change the dynamics of EVIC?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4Ms9Ixdv06", "reviewer": "Reviewer_zLAH", "summary": "The authors propose EVolving Interaction-guided Curriculum (EVIC), a training technique that aims to improve the performance of LLM multi-domain fine-tuning. EVIC iteratively finds the most “helpful” samples in the training set (those that are likely to have helpful influence on the model’s overall loss), then trains on just this helpful subset. The authors conduct experiments of fine-tuning GPT-4 to code, math and general reasoning domains, comparing EVIC to four prior works.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer zLAH,\n\nThank you for your valuable review. We respond to each comment as follows and sincerely hope that our response can properly address your concerns.\n\nFigures and Tables can be found in **zLAH.md** in **https://anonymous.4open.science/r/ICML25-EVIC-D5E8**\n\n# Methods And Evaluation Criteria\n\n> M1: Do you find that the interactions eventually stabilize and evolve less? How does the time to stabilization compare with the model convergence time?\n\n**Res:** We do not observe the stability of the interaction matrix. Taking Llama-3.1-8B and Mistral-7B-v0.3 as examples, the Frobenius norm of the interaction matrix after the warm-up phase, 1 iteration, 2 iterations, and 3 iterations is as follows.\n\n- Llama-3.1-8B: 9e12 -> 5e12 -> 6e12 -> 4e12\n- Mistral-7B-v0.3: 9e12 -> 2e12 -> 4e12 -> 2e12\n\n> M2: How asymmetric are the interactions? Is there any chance that the asymmetry is just due to random noise? Do you have any other hypothesis as to why these interactions are asymmetric?\n\n**Res:** Taking Llama-3.1-8B and Mistral-7B-v0.3 as examples, their interaction matrices consistently have more than **99.7%** of sample pairs $(i,j)$ satisfying $Int(i,j) \\neq Int(j,i)$. We further analyze the proportion of pairs $(i,j)$ with opposite influence directions during training, i.e., those satisfying $sign(Int(i,j)) \\neq sign(Int(j,i))$. The percentages after the warm-up phase, after 1 iteration, 2 iterations, and 3 iterations are as follows.\n\n- Llama-3.1-8B: 19.3% -> 17.9% -> 16.7% -> 17.4%\n- Mistral-7B-v0.3: 22.1% -> 21.5% -> 21.8% -> 21.1%\n\nThe asymmetry of the interaction matrix arises from the difference between each sample's original gradient $Grad$ and Adam gradient $Adam$. Since $Int(j,i) = \\langle Adam(j), Grad(i) \\rangle$, $Int(i,j) = \\langle Adam(i), Grad(j)\\rangle$, we naturally have $Int(j,i) \\neq Int(i,j)$.\n\n> M3: (1) Have you tried EVIC with initial portions other than 5%? (2) Is EVIC's performance highly dependent on which samples are initially selected?\n\n**Res:** (1) Yes, please see Figure 3 in Section 4.3.\n\n(2) The standard EVIC randomly samples 5% of the dataset for warm-up. To investigate the impact of warm-up samples, we have added additional experiments only using samples from code and general domains for warm-up, denoted as CodeWarmUp and GeneralWarmUp. The results are shown in **Tables zLAH-1 and zLAH-2** in the anonymous link. As shown, unbalanced warm-up sample distribution will reduce model performance.\n\n> M4: It would be interesting to see a histogram of how often samples are chosen as helpful during training. For instance, a bar for samples being never chosen, a bar for samples being chosen once/twice, etc.\n\n**Res:** We have added figures in the anonymous link following your suggestion.\n\n> M5: Will EVIC negatively impact the performance of model on the rarer domain when training on domains with datasets of varing sizes?\n\n**Res:** Imbalanced datasets can indeed hinder the learning of rarer domains. Fine-tuning on such imbalanced datasets is another important challenge in multi-domain finetuning [1,2,3]. To enhance the learning of rare domains, we can weight the interaction matrix computation in EVIC based on the sample size of each domain, increasing the probability of selecting samples from rare domains.\n\nExtending EVIC to imbalanced datasets is a new direction, and we will continue to explore this in future work. If you are interested, we would be glad to discuss this topic further.\n\n# Experimental Designs Or Analyses\n\n> E1 There are a strong number of baselines and related works included in the evaluation and the evaluation appears to be well-designed by including tasks of varying sizes. However, the evaluation could be expanded and strengthened by including more models beyond GPT-4 and additional domains/datasets.\n\n**Res:** Thanks for the positive feedback on our experimental design.\n- **Regarding the models:** Our experiments use three of the most popular and widely-used base models---Mistral-7B-v0.3, Llama-3.1-8B, and Qwen2.5-14B.\n- **Regarding the domains/tasks:** Since DMT is experience-baesd and cannot be directly applied to other domains (as the authors of DMT do not provide relevant experience), we **follow DMT's setting** using mixed dataset containing code, math, and general domains to **ensure fairness**. If you have any other domains or datasets of interest, we would be happy to hear from you through a rebuttal response. **While our computational resources are limited, we will make every effort to conduct additional experiments to follow your suggestions.**\n\n# Other Strengths And Weaknesses\n\n> W1 The \"firness\" in Line 252 should be \"fairness\".\n\n**Res:** Thanks for pointing this out. We will correct this typo.\n\n---\n\n[1] Mixture-of-Skills: Learning to Optimize Data Usage for Fine-Tuning Large Language Models\n\n[2] Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges\n\n[3] Unsupervised Cross-Lingual Representation Learning at Scale", "claims_and_evidence": "Claims are supported", "methods_and_evaluation_criteria": "- Lines 159-160 (and other places): when examining the interactions among samples over time, do you find that the interactions eventually stabilize and evolve less? How does that time to stabilization compare with the model convergence time?\n- Line 163: How asymmetric are these interactions? Is there any chance that the asymmetry is just due to random noise or do you have any other hypothesis as to why these interactions are asymmetric?\n- EVIC begins with a “warm-up” phase, where 5% of samples are randomly selected to start training with. Because 5% is such a small proportion, is the method’s performance highly dependent on which samples are initially selected? Have you tried EVIC with initial portions other than 5%?\n- It would be interesting to see a histogram of how often samples are chosen as helpful during training: for instance, a bar for samples never chosen, a bar for samples chosen once, twice, etc.\n- If you are training on domains with datasets of varying sizes, will EVIC negatively impact the performance of the model on the rarer domain? For instance, suppose you are training a model to perform n different tasks, where n-1 of the datasets have 10000 samples and the nth dataset has 100 samples. Will the strategy of focusing only on the samples that appear most helpful overall result in neglecting the performance of the smaller dataset?", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "- EVIC performs well in the evaluations compared to prior works (Table 2)\n- There are a strong number of baselines and related works included in the evaluation and the evaluation appears to be well-designed by including tasks of varying sizes. However, the evaluation could be expanded and strengthened by including more models beyond GPT-4 and additional domains/datasets.", "relation_to_broader_scientific_literature": "Multi-domain training is a useful and important area of ongoing research, and EVIC appears to perform favorably compared to prior works in this area", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "One typo: line 252 “firness” instead of “fairness”", "other_comments_or_suggestions": "Overall, the method is intuitive and offers good performance. However, the analysis and evaluations could be improved by including more tasks and comparing to prior works. These factors together have contributed to my score.", "questions_for_authors": "All questions are asked in the prior sections", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces EVIC, a curriculum learning strategy for multi-domain LLM fine-tuning that iteratively selects beneficial training samples based on estimated gradient interactions. The method is well-motivated and addresses the important challenge of balancing performance across diverse tasks. Reviewers generally found the paper clear (JcLd) and the method intuitive (zLAH), acknowledging its novelty (q9CU). The empirical results demonstrate improved average performance across multiple domains and models compared to several baselines, including standard multi-task learning and other curriculum learning approaches (all reviewers, JcLd raised score following rebuttal).

**Strengths:**
* **Novelty and Approach:** Proposes an intuitive and novel iterative method (EVIC) using gradient interactions for multi-domain fine-tuning curriculum (q9CU, zLAH).
* **Improved Average Performance:** Demonstrates consistent average performance gains across diverse domains (code, math, general) and models (Mistral, Llama, Qwen) compared to MTL and relevant CL baselines (q9CU, JcLd, zLAH).
* **Clarity:** The paper is generally well-written, clearly presenting the problem and the proposed EVIC method (JcLd).
* **Reasonable Experimental Setup:** Experiments cover multiple standard models and benchmarks relevant to multi-domain tuning (q9CU, JcLd, zLAH).

**Concerns:**
* **Computational Cost (q9CU, JcLd):** EVIC requires computing gradients for all samples multiple times per epoch to build the interaction matrix, making it significantly more costly than standard fine-tuning.
    * *Address:* Authors argued cost is <2x MTL and justifiable for performance gains, provided training plots showing faster convergence to better results. (q9CU acknowledged explanation; JcLd accepted plots/efficiency argument).
* **Sample Efficiency Framing & Statistical Robustness (q9CU, JcLd):** Initial "sample efficiency" claims needed refinement given the computational cost. Reported standard deviations are based on multiple *evaluation* runs, not multiple *training* runs, limiting conclusions about training stability/robustness.
    * *Address:* Authors promised to rephrase efficiency claims (e.g., "performance gain per sample") and clarify std dev reporting. (q9CU acknowledged; JcLd accepted performance but noted std dev caveat).
* **Contribution Clarity (JcLd):** Initial presentation could better distinguish the paper's contributions versus leveraged methods (Xia et al., 2024).
    * *Address:* Authors promised clarification in the revision. (JcLd acknowledged).
* **Scope and Scalability (zLAH):** Potential challenges with highly imbalanced datasets were noted; evaluation could potentially be broader.
    * *Address:* Authors acknowledged imbalanced data as future work and justified current scope based on resources/baseline fairness. (zLAH acknowledged).
* **Method Details (q9CU, zLAH):** Initial questions regarding Adam gradient rationale, interaction dynamics, and warm-up sensitivity required clarification.
    * *Address:* Authors provided ablations, data, and explanations addressing these points. (q9CU, zLAH acknowledged).

Overall, the paper presents a promising direction for multi-domain fine-tuning with empirical evidence for improvement. While concerns about computational cost and statistical reporting remain, the overall reviewer feedback following the discussion justifies acceptance, conditioned on incorporating the promised revisions.
Decision: Accept (poster)