IDEAL: Data Equilibrium Adaptation for Multi-Capability Language Model Alignment
Authors: Chenlin Ming
Chendi Qu
Qizhi Pei
Zhuoshi Pan
Yu Li
Xiaoming Duan
Lijun Wu
Conghui He
Venue: ICLR 2026 Poster (2026)
Keywords: Data Re-weighting
LLM SFT
Primary area: optimization

Abstract: Large Language Models (LLMs) have achieved impressive performance through Supervised Fine-tuning (SFT) on diverse instructional datasets. When training on multiple capabilities simultaneously, the mixture training dataset, governed by volumes of data from different domains, is a critical factor that directly impacts the final model's performance. 
Unlike many studies that focus on enhancing the quality of training datasets through data selection methods, few works explore the intricate relationship between the compositional quantity of mixture training datasets and the emergent capabilities of LLMs. 
Given the availability of a high-quality multi-domain training dataset, understanding the impact of data from each domain on the model's overall capabilities is crucial for preparing SFT data and training a well-balanced model that performs effectively across diverse domains. 
In this work, we introduce IDEAL, an innovative data equilibrium adaptation framework designed to effectively optimize volumes of data from different domains within mixture SFT datasets, thereby enhancing the model's alignment and performance across multiple capabilities. 
IDEAL employs a gradient-based approach to iteratively refine the training data distribution, dynamically adjusting the volumes of domain-specific data based on their impact on downstream task performance. By leveraging this adaptive mechanism, IDEAL ensures a balanced dataset composition, enabling the model to achieve robust generalization and consistent proficiency across diverse tasks. 
Experiments across different capabilities demonstrate that IDEAL outperforms conventional uniform data allocation strategies, achieving a comprehensive improvement of approximately 7\% in multi-task evaluation scores.

Published as a conference paper at ICLR 2026
IDEAL: DATA EQUILIBRIUM ADAPTATION FOR MULTI-CAPABILITY LANGUAGE MODEL ALIGNMENT
Chenlin Ming', Chendi Qu', Xiaoming Duan', Qizhi Pei", Zhuoshi Pan', Yu Li² Conghui He?, Lijun Wu
'Shanghai Jiao Tong University Shanghai Artificial Intelligence Laboratory
TTinngua University *Renmin University of China
{mc12019c11457 xduan @ @s hau.e.u.ccm Mmulijun]@ppilas org.
ABSTRACT
Large Language Models (LLMs) have achieved impressive performance through Supervised Fine-tuning (SFT) on diverse instructional datasets. When training on multiple capabilities simultaneously, the mixture training dataset, governed by volumes of data from different domains, is critical factor that directly impacts the final model's 's performance. Unlike many studies that focus on enhaneing the qual- ity of training datasets through data selection methods, few works explore the intri- cate relationship between the compositional quantity of mixture training datasets and the emergent capabilities of LLMs. Given the availability of high-quality multi-domain training dataset, understanding the impact of data from each domain the model's overall capabilities is crucial for preparing SFT data and training a well-balanced model that performs effectively across diverse domains. In this work, we introduce IDEAL, innovative data equilibrium adaptation framework designed to effectively optimize volumes of data from different domains within mixture SFT datasets, thereby enhancing the model's s alignment and performance across multiple capabilities. IDEAL employs gradient- approach to itera- tively refine the training data distribution, dynamically adjusting the volumes of Nomain-specific data based on their impact on downstream task performance. By leveraging this adaptive mechanism, IDEAL ensures a balanced dataset compo- sition, enabling the model to achieve robust generalization and consistent profi- ciency across diverse tasks. Experiments across different capabilities demonstrate that IDEAL outperforms conventional uniform data allocation strategies, achiev- ing a comprehensive improvement of approximately 7% multi-tase evaluation scores. Our code is available at athttps:///iii com ng bot./DEAT
INTRODUCTION
Recent advancements in LLMs have demonstrated their remarkable ability master diverse capa- bilities Dong et al.
| (2024); Zhang et al. (2024b); Hu et al. (2023); Mecklenburg et al. (2024); Li et al. (0225) through Supervised-Fine-tunne
(SFT) on instruction-aliigeed datasets Liu et al. (2023):
et al. 2023); Agarwal et al. (2024); Wang al. (223). By training on heterogeneous tasks such as mathematical reasoning Luong et al. (2024); Guo et al. (2025)
; Pei et al. (2025); Pan et al. 2025
code generation DDhaernee et al. (2022) Si et al. (2024). and creative writing Wang al. (024);
Gódez-Rodriguez & Williams (202)); Franceschelli & Musolesi (2024), models like GPT-4 Ope
nAI et al. (2024). Claude Anthropii (2023), and LLaMA-3 3 Grattafiori al. (2024) achieve promis- ing performance across various domains. However, empirical studies reveal that naively merging datasets for multi-objective fine-tuning often degrades performance compared single-task spe- cialization Tunstall et al.) [heen (2024): Dong et al. (2024). To mitigate the aforementioned issue, a common approach to adjust the training data distribution Xie al.] (2024); Ye al.] (2024), thereby regulating the volume of data from each domain within the mixed dataset. However, critical challenges persist: the optimal mixture proportions of these domains are poorly understood and how to adjust the optimal mixture proportions is not clear. While heuristic solutions such as manual data
"Corresoonding author

---

Published as a conference paper at ICLR 2026
reweighting or rule-based curriculum learning Bengio al. (2009) exist, they suffer from scalabil- ity limitations and suboptimal task balance. Prior attempts to automate data allocation, including pretraining-centir methods Xie al. (2024); Ye et al. 2024), fail to address the unique dynam- ies of SFT, where data-task alignment directly governs cross- interference. Consequently, a principled framework for resolving data conflicts in multi-capability SFT remains open problem.
To address this challenge, we propose IDEAL, novel framework that dynamically aligns SFT data mixtures with model capabilities. IDEAL employs the second-order information to opti- mize the data mixture ratios iteratively. Un- like previous works using the influence func- tion for data sample selection Xia et al. (2024); Zhang et al. (2024a), we employ a domain- specific hyperparameter
Data-praportion Space
Cosel
loss
regulate the volume of training data for each domain. To scale our method to large model parameter sizes and en- sure computational efficiency, we optimize the approach for efficiently computing the Hessian matrix and incorporate upsampling and down- sampling strategies, significantly speeding up the process of adjusting training data distribu- tions. Through iterative optimization of the data volume for each domain, the model trained on the optimized dataset exhibits a significant reduction in loss, as shown in Fig. [ It's important to clarify that balanced dataset does not simply mean having equal amounts data from each domain, rather finding an optimal dis- tribution where all capabilities can develop properly. This model-aware mechanism adapts to the LLM's evolving training dynamics,, ensuring equilibrium between data efficiency and task balanc- ing. Crucially, our framework operates without costly hyperparameter sweeps, enabling scalable multi-capability SFT with theoretical guarantees.
Random Iniial Sihustion * Oytimizedd Silutioon
IDEAL
Figure 1: IDEAL adjusts data mixture proportions to optimize model performance, leading to de- crease in loss.
Extensive experiments validate IDEAL's effectiveness across diverse capability combinations, in- cluding reasoning, mathematics, code generation, and coarruction-olooiingg. On multiple bench- marks, IDEAL outperforms uniform data blending 7% average and even surpasses single-task SFT baselines after 2 iterations, demonstrating its ability to resolve data conflicts while amplifying cross-task synergies. Further analysis delves into sensitivity study on the IDEAL algorithm'' per- formance with the selection of hyperparameters and its robustness on extended benchmarks and do- mains, all of which help in comprehensively evaluating the proposed method. These results establish data equilibrium adaptation -error -errr
critical lever for training generalist LLMs, advancing beyond the paradigm of conventional SFT. IDEAL offers a practical, mathematically grounded method for LLM data optimization, bridging innovation with real-world applicability.
trial-and-ene
2 RELATED WORK
Data Mixing. Techniques like data mixing, alternatively referred as data re- re-weighting, center fining the distribution of characteristics within a pre-prepared dataset, which aims to enhance mod- els' multi-capability performance. Conventional methods typically depend on approaches based on the ratio of tokens Touvron et al. (003)); (Shen et al.) (2024); Liu et al.] (2025). Some methods also focus on data selection to change the distribution of training data by selecting high-quality training data|Parmaa et al. (2024); Chung et al. (2023); Engstrom et al. (2024); Xia et al. ; Mekala et al. 2024); Kang et al. (2024) Xu al. (2023b). the LLM era, data mixing has evolved into learning- based approaches by using optimization algorithms some smaller well-trained models Chen et al. (0024); Shao et al. 2024):; Yu et al. (2024). Wu Wu et al. (2024) employs a reinforcement learning approach train scorer network for data sampling probability allocation, constituting a bi-level data selection framework. Dong Dong et al. (2024) investigates the DMT training procedure, man- ually designing mixing ratios demonstrate multi-domain training challenges. DoReMi Xie et al. (2024) employs group distributionally robust optimization to fine-tune a small proxy model. After determining the optimal proportions, DoReMi transfers them to larger LLM,, significantly reduc-
2

---

Published as a conference paper at ICLR 2026
ing training costs. DOGE Fan et al. (2023) builds on this approach by re-weighting the training dataset for specific target domains, also utilizing a trained proxy model to determine the final do- main weights. Data Mixing Law Ye et al. (2024) introduces mixing proportion law based on extensive experimentation. The mixing proportion law reveals the relationship between data propor- tions and the corresponding validation loss, providing a reference for adjusting the data distribution. However, these methods require costly global weight searches and often disregard the continuity of data distributions. Our work addresses these gaps by gradient-guided iterative refinement, enabling efficient adaptation.
Data Selection. Data quality is decisive factor for model performance Albalak et al. (2024). Data selection techniques are widely used to identify beneficial data from massive datasets for model training, particularly in the pre-training phase Gu al. (2022). LESS LESS[Xia al. leverages first- order gradient information to compare data gradients with target gradients, selecting data that aligns more closely with the target distribution. Gu al. Gu et al. (2024) model the training process as a dynamical system, applying Pontryagin's Maximum Principle (PMP) Pontryygin (2018) to optimize data selection for training. Fan et al. Fan et al. (025) focuses on domain similarity to address dimensional collapse, introducing the DiSF method to enhance data diversity in training. Liu et al. [Liu et al. (2024) proposes SelectIT, which utilizes the internal uncertainty signals of LLMs to select high-quality data, eliminating the need for external resources. However, most data selection methods assume the availability of large-scale, reliable data and have not yet been extended to the SFT phase, where high-quality data is often scarce.
Inftuence Function. The Influence Function Hampel 1974) has undergone decades of development and has attained widespread recognition due methods, the influence function, as noted in Koh & Liang (0117), serves as a crucial link between model performance and the training dataset. It casts light on the inner workings of the black- predictor, demystifying the often opaque process of the model behavior. Despite the high cost of computational resources Grosse al. 2023): [yyas et al. (2022): Zhang al. (2024)), many
remarkable interpretability. In learning-basee
gradient-based approximation methods are used to explore the relationship between the performance of the large language model and the training set [Engstrom et al. (024); Park et al. (2023); Pruthi it al. (2020). Liang Liang et al. (2025) uses an iterative train-filtee filter process and focuses instance- level data filtering. MATES Yu et al. 2024) utilizes a proxy model to estimate the influence score for selecting the pre-training dataset. However, influence- based methods are overly focused on individual data points and lack the ability to handle mixed datasets effectively. While they are well- suited for filtering data per-instance basis, they are not efficient for tasks that require adjusting the overall distribution of a dataset.
METHODOLOGY
3.1 PROBLEM FORMULATION
We first give the necessary notations and problem formulation. Assume base model Mu param- eterized with 0 E RMM exists, and our objective is to leverage SFT to develop an instruction-tume model based on Mo that excels across multiple domains. To facilitate the optimization of the whole training dataset Dir, we partition the training data into distinct categories based on their respective domains, such as math, code, and so on, as Di...., Dn-- Let N be the total number of training samples N |Derl [Di] + ... + [D.l. where [Di represents the cardinality of the domain- specific training dataset D.. The learning objective is to optimize model parameters by minimizing the training cost function, with optimal parameters expressed as:
0* arg min L(Dir, 0) arg min 0
Σ 8), N
1
0
where is the parameter of the model, and L is the parameterized empirical cost function. We employ small, independent reference dataset Dref to rigorously validate its performance across diverse domains. This dataset is strictly excluded from the training process to ensure unbiased and fair assessment. The cost of the well-traine model on Dref serves as comprehensive perfor- mance metric across multiple tasks, and our ultimate objective is to minimize the reference loss: L(D O*).
3

---

Published as a conference paper at ICLR 2026
3.2 OPTIMIZATION OF MULTI-DOMAIN TRAINING DATASET
When simply mixing the data from multiple domains together, like Dtr Di U.... D for SFT, the instruction-tuned model is easy to show imbalanced capabilities (as shown in Ta. Although each domain-specific training set has been individually validated for effectiveness, a straightforward mixture of these datasets could result in an inappropriate training data distribution. The unbalanced blending of data volumes across domains may introduce data conflicts, causing the trained model fail in mastering specific domain tasks. One direct approach is expand the training data (e.g., adding new training data) for underperforming capabilities while reducing the data volume for other domains. However, determining the precise amount of data to add or remove remains an unclear problem, which often lacks scientific basis and frequently relies on the empirical judgment of engineers. Therefore, introducing a principled approach for determining the data mixture ratio is crucial task. Besides, according to Muennighoff's s findings Muennighoff et al. (2023), repeating existing data up to four times yields a performance improvement that is largely equivalent to in- troducing the same amount of new data, which further supports the rationality of modifying the multi-domain data mixture ratio. Building on this insight, we introduce parameter B E R to control the proportion of repeated data relative to the original domain-specific training set, thereby dynamically adjusting the volume of data from different domains. To achieve this, we redefine the learning function and optimal parameters as:
1
arg min [(Di,,, 0) + E (.C(D.)))
0"
(1)
N
B.
0
Note that 3r is a real number that can be either positive or negative; when 3; is positive, it indicates the need to repeat existing data, whereas when it is negative, it signifies the requirement to reduce the volume of the corresponding domain -ppeciic data, Finding the optimal proportion /3 is equal to solving the following bi-level optimization problem:
min BES
((3) == L(Dres,
s.l. Eq. 0
Our objective is to understand the impact of 3 on the model's performance on Dref and find the best optimization direction for B. By applying the chain rule, we determine the impact of a specific Bg E [BI...., .B.. on the model's performance [(Dres, 0*) as:
02(3) (aL(Dres, 0*) a0*
03, - (
(3)
03,*
Assumption The empirical risk function C is twice differentiable with respect to 0. S, defined as the domain of 3 E R", is compact and convex set.
This assumption is readily applicable to the general training of LLMs where cross-entropy is used as the loss function. With this assumption in place, we can derive the following lemma.
Lemma 1 Suppose Assumption I holds. The impact of a specific B, on the optimal model parame- ters O* trained on the whole training set can be explicitly expressed as Eq. 4
a0
V2 0-) + B.V'
VL(D,,0*)
(4)
03,
For a comprehensive derivation and detailed proof, please refer to Appendix B
Looking back into the initial training configuration, we employ the straightforwardly combined dataset to refine the model Mo into M1, thereby obtaining the optimized model parameters 0*.
With the initial state 3 (0.... 0). we can therefore obtain the influence of B on the validation set
Dref as Eq. ( by substituting Eq. (4) into Eq.
0Q(3)
[D((Dre. 00
42
(5)
03
3 ()......))
4

---

Published as a conference paper at ICLR 2026
3.3 EFFICIENT CALCULATION
Evaluating the inverse of the Gauss-Newton Hessian in Eq. 5 for a large language model presents formidable challenge. In this section, we employ approximation methods and acceleration tech- niques to achieve efficient computations. According to K-FAC theory Martens & Grosse (2015); Ueno al. (2020); Zhang et al. (2024a), we use the Kronecker product to accelerate the inverse Hessian-vector products (iHVP) computation. Considering minimal direct dependencies between the gradients of different MLP layers, we decompose the Hessian matrix H into a simple block- diagonal approximation according to different layers. Specifically, we utilize the hook function to obtain the gradients of each layer. The gradient with respect to layer / can be expressed as DO = vec(DW") x'' & 8'. Here, DO' represents the gradient of the parameters in layer I, which vectorized as vec DW*). The term x' denotes the input to layer and represents the error gradient propagated back to this layer. The symbol @ indicates the outer product operation, which combines z' and to compute the gradient update for the parameters in layer Thus, the block matrix of the layer in the Hessian matrix H is:
H = E(DD' DO'T) = E(a' s 88888
B(x'x") E(6' x' = 8
Decomposing the Hessian matrix into components corresponding different MLP layers makes it feasible to calculate its inverse for a large language model with an enormous number of parameters. To further release the memory pressure,
the eigendecompositions of X and A/:
X -xxAx A' QAiAA Q
By leveraging Eq. (6) and Eq.
7.. we obtain more accurate expression for H:
H ~ (Qx @ Qa)A(Qx @ QQ)),
Aa [[Qx & Qa)DD)
in which A captures the variances of the pseudo-gradient projected onto each eigenvector of the K-FAC approximation. We then identify the 'important" MLP layers by choosing those with lower variances, as these layers exhibit enhanced stability. Reducing the number of calculation layers can significantly alleviate the storage pressure. However, will also lead a relatively small magnitude of the final result. To address the above issue, we introduce a dynamic scaling vector % which linearly scales the maximum absolute value in the calculated 33 to predefined value m. We update the 3 values shown in Eq. 9
Q =
aQ(33
3 == - ④ 0 = max abs( (x)
m
(9)
0/
(0(.....0))
3.4 IDEAL ALGORITHM
Our method maximizes the effectiveness of data and enhances the model's capabilities by adjusting the quantity ratio of the training set, thereby catalyzing the synergistic effect among data from different domains. Our approach leverages trained model to assess and op- timize the distribution of the train- ing set. After calculating B, we iteratively refine the training set composition, ultimately deriving enhanced model from the founda- tional base model.
Algorithm 1 IDEAL Algorithm
Require: Initial model Mo. initial training set D
{Di, D,, maximum iterations T (or stop criteria).
1: for t = to T do
2: Train Mo on Dir, until optimal, resulting in Mr;
3: Test the performance of Me
4: Compute /3 following Eq. equation 9
5: Update domain-specific data: Dr.41 + (1+ B.)
6: Update training set: Der,++ + Σ Di,411
7: if stopping criteria meet then
8: break
The IDEAL algorithm begins with an initial model My and train- ing set Dr, partitioned into domain- specific subsets (Di,.., D.). For
9: end if
10: end for
5

---

Published as a conference paper at ICLR 2026
each iteration t, the model Mo is trained on the current training set Dtr. until convergence, result- ing in an updated model Mr, as shown in Line 2 Next, the performance of Me is evaluated on the reference dataset to assess its multi-task capabilities. Based on this evaluation, the parameter 3 is computed using the update rule defined in Eq. (2). In Line 5 the domain-specific data vol- umes are then adjusted proportionally, with each subset Di,++1 updated as + B.,,,,, We employ upsampling and downsampling to respectively increase or decrease the data volume of the dataset. The random sampling approach, on one hand, conserves computational resources and enhances processing speed, while simultaneously minimizing the potential impact of selection methods on the performance of IDEAL. The updated training set Der,t+1 is reconstructed by aggregating the adjusted domain-specific subsets, as shown in Line The process repeats until the stopping crite- ria are satisfied, ensuring an optimal and balanced dataset composition for training the final model. This iterative approach allows IDEAL to adaptively refine the dataset, enhancing model performance across diverse tasks.
EXPERIMENTS
4.1 EXPERIMENT SETUP
Dataset Preparation. We select four key domains: reasoning, mathematics, coding, and instruction- following to represent the model's diverse capabilities. These domains are among the most widely studied and representative areas of focus in the field, encapsulating critical aspects of model per- formance and alignment. In Section 4.22 we firstly evaluate models on BigBench Hard(BBH) Suz- gun et al. (2022), GSM8K Cobbe et al. (202I). HumanEval Chen et al. (2021). and IFEval Zhou jet al. (203)) benchmarks, which are widely used in previous works. In Section 4.33 we expand the evaluation benchmarks by adding MATH Hendrycks et al. (202T). ARC_C et al. 2018),
MBPP Austin et al. (2021b). and TruthfulQA Lin et al. (2021) benchmarks. Appendix A provides detailed explanation of the benchmarks selected for our experiments, along with comprehensive description of the specific preparation process for our training datasct.
We choose the LLama3. 1-BB Grattafiori et al. (024)) our base model .Mo to adopt full fine-tuning. In all fine-tuning training experiments, we set the batch size to 256 and the maximum learning rate 2 x 10 with cosine decay schedule. We train every model in 1 epoch and 3 epochs on 8 x A100 GPUs and evaluate the result by using the OpenCompass platform Con- tributors 2023), , respectively. The parameters mentioned above may not be optimal. However, ensure that all our experiments use the same set of training parameters for consistency. Morcover, each experiment repeated 5 times, and the average value is calculated to avoid the interference of random phenomena.
Training Setting:
Baseline. We compare the performance of our IDEAL with other data training strategies as follows. (1) Specific SFT, which only uses specific domain training data for SFT. (2) Joint SFT, where the different capability data directly combined. (3) Random, we randomly sample different data scales for each capability. (4) DoReMi Xie et al. (022)), which uses the group distributionally robust optimization (Group DRO) steps to generate new domain weights. (5) DOGE Fan et al.) 2023), which determines the data proportions between domains by minimizing the discrepancy in backpropagation gradients. For our IDEAL, we set the parameter m = 0.15. In order to accelerate calculation speed, sample the training set with a sample factor = 0.5.
4.2
MAIN RESULTS
In this section, we select individual benchmarks from four universal domains--atthematics cod- ing, reasoning, and instruction following- evaluation, specifically GSM8K, HumanEval, BBH, and IFEval. Additionally, we configure two experimental settings with epochs set to I and 3, respec- tively. Training for I epoch is widely adopted due to its shorter training duration and lower risk of overfitting. Training for 3 epochs is aligned with practical applications and can achieve better SFT effects. These settings allow us to thoroughly assess the robustness and adaptability of our method across varied evaluation criteria and training durations. The main results shown in the Table. [[]
Initially, we train four specific SFT models using the training sets corresponding to their respective capabilities. After dopain- training, the base model demonstrates a 10% improvement in

---

Published as a conference paper at ICLR 2026
Table
Performance comparison of different baselines.
Benchmark
Mathematics
Coding
Reasoning
Instruction
Overall
Methods
Dataset
Acc(#Size)
Pass@ ((#Size)
Averago(#Size)
Average
Average
Epoch=1
Base Model
56.41(-)
27.44(
62.13(
12.22
39.55
Mathematics
65.81(10.0k)
0.00(0.0k)
35.94(0.0k)
22.54(0.0k)
31,07
Coding
48.14(0.0k)
37.20(5.3k)
2.99().0k)
19.66(0.0k)
27.00
Specific
Reasoning
61.87(0.0))
0k)
60.19(6.5k)
26.70(0.0k)
39.02
Instruction
57.39(0.0k)
(6.95(0.0k)
61.87(0.0k)
22.47(2.0k)
47.17
Joint
Do
66.62(10.0k)
41.26(5.3k)
72.92(6.5k)
38.36(2.0k)
$4.79
Random
candon
63.84(3.5k)
43.90(7.4k)
75.11 1.O))
39.7033.0k
entom
63.23(1.7k)
40.85(9 Ok)
74.56(12.0K)
38.21(1.96 (1.9k)
DoReMi
(DokkMn))
65.96(5.0k)
41,63(8 Ok)
73.44(9.7k)
34.26(1.0k)
$3.82
D ((DORMM)
64.82(5.3k)
43,90(12 12.0k)
73.79(4.8k)
38. 18.1601.55 ..5k)
55.17
D (DOGE)
64.82(11.5k)
40.02(8.00 Ok)
74.99(3.2k)
39.02(1.06 LOk)
54.71
DOGE
P200GB
67.10(10.00)
42.24(12.0K)
73.59(1.6k)
30.53(0.5k)
53.37
DDAL
(DDAL))
68.01(9.5k)
(6.51(6.0K)
72.82(6.8K)
39.78(2.1k)
56.28
2(DEAL)
67.55(9.0k)
50.61(7.1k)
74.29(7.3k)
39.03(1.9k)
57.87
Epoch=3
Mathematics
74.45(10.0K)
0.00(0.0K,
30.82(0.0k)
21.73(0.0k)
31.75
Coding
49.43(0.0k)
41.46(5.3k)
41.93(0.0k)
28.74(0.0k)
Specific
Reasoning
62.09(0.0k)
4.27(0.0k)
65.6916.5k
24.86(0.08
39.23
Instruction
58.91(0.0k)
40.85(0.0k)
60.90(0.0k)
38.77(2.06 (2k))
49.86
Joint
Do
68.66(10.0k)
39.64(5.3k)
(6.11(
38.99(2.0k)
55.35
Random
DJ
gandon
69.75(9.0k)
45.12(7.5kk
73.20(9.0k,
35.81(3.0k)
55.97
D
64.82(1.7k)
43.90(9.3k)
70.60(5.0k,
39.73(4.0k)
54.76
D (DoR)M))
70.96(5.0k)
40.85(8.0k)
72.38(9.0k)
36.00(1.5kk
$5.05
DoReMi
D DEDOROMO
68.01(6.3k)
(1.5k(10.0k)
70.32(4.0k)
42.16(3.0k)
56.25
DOGE
(DOGE)
70.17(L1.0k)
40.02(8.08 Ok)
73.44(4.0k)
40.21(1.k)
$5,96
D Dapoor
69.10(9.6k)
42.24(12.0k)
73.59(1.4k)
42.53(1.06
56.87
D (UDBAL))
70.17(10.0k)
42.07(6.1k)
74.71(6.8k)
45.25(2.1k)
$8.05
IDEAL
DAUDBAL))
71.19(11.0k)
46.34(7.0k)
73.97(7.2k)
45,40(2.2k)
59.23
performance or maintains its previously high scores. We further explain the specific SFT perfor- mance in Appendix E By evaluating these specific SFT models, we validate the reliability of the quality of our training datasets. Subsequently, we integrate all four training sets into Do and train Joint SFT model, further enhancing generalization capabilities across multiple domains. Based on Table L analyze key findings that demonstrate approach 's effectiveness.
Suboptimal initial data distribution. Although the Joint SFT model demonstrates improved per- formance over the Specific SFT models across all datasets except HumanEval, we aim avoid the "bucket effect" where the overall model performance is constrained by its weakest component. The random baselines also exhibit similar suboptimal performance. Although randomly determining the proportions of the four datasets could potentially yield different training outcomes, it lacks stability.
IDEAL achieves optimal balance in iterations. By incrementally refining data ratios, IDEAL surpasses Joint SFT on all metrics and stabilizes performance across benchmarks, notably achieving improvement in the Epoch=1 setting) without compromising other tasks, fulfilling efficiency and stability requirements. We use bold and underline to denote the best and second-best performance scores, respectively. From the perspective of training data volume, IDEAL significantly increases the data amount in the Coding domain after iteration, while slightly reducing the data volume the Mathematics and Instruction domains. Compared DoReMi and DOGE, the changes in data volume by IDEAL are more stable. We argue that the key to the stable performance optimization of the model trained by IDEAL lies searching for a better distribu- tion around the initial distribution. Comparing the first and second rounds of the IDEAL method, the recond-round model shows a slight decline Mathematics and Instruction but achieves a sub- stantial improvement in performance in the Coding domain. This demonstrates IDEAL's ability to directionally enhance model performance.
HumanEval improvements(+9))
Performance Degradation in HumanEval with Extended Training. Although models trained for three epochs show improved performance across most domains, their capability on HumanEval is inferior the optimal scores achieved with epoch=1. This trend is also observed in the results of

---

Published as a conference paper at ICLR 2026
the Specific SFT models trained for three epochs, where the upper limit of scores on the HumanEval benchmark decreases as the number of training epochs increases. We infer that suboptimal training data distribution introduces conflicts among the data, and extended training exacerbates this issue, thereby inhibiting the model's ability to enhance performance in certain dimensions.
Training different Epochs showcases different optimization priorities. Training models over multiple epochs aims to leverage more data information and gradient updates. The IDEAL method controls the size of training sets across domains through the parameter 3. where positive values of B favor larger quantities, thereby providing more gradient information for that domain (even without introducing new data). In the epoch=3 setting, models are more likely to memorize existing infor- mation and thus require additional data and gradient updates. In Table [ all four domains exhibit a demand for increased training data volume. contrast, during the epoch=1 phase, IDEAL tends to locally adjust the data volume of each domain to achieve optimal performance. This selective adjustment highlights IDEAL's ability to fine-tune the dataset composition based on the model's training configuration, ensuring more balanced and effective training process.
4.3 EXTENDED EXPERIMENT
We extend the TrustAl training set and double the number of benchmark evaluation criteria to en- sure more comprehensive and robust assessment of the model's performance. We also choose the Llama3.1 8B as our base model. To establish a clear upper bound on the performance achievable with the available data, we train domain-speciie recorded as Specific SFT (FULL) in Table 5) We also introduce an additional experimental con- trol group, Do which includes all data from five training domains, comprising approximately 66k training samples, to investigate whether data volume plays a more critical role in model per- formance. We also conduct other extended experiments about model size and model architecture in Appendix D
models using the entire domain-speciie dataset,
In this experiment, we adopt a balanced initial dataset Do. where 5k samples are randomly selected from each of the five domain-specific datasets to form a new dataset. Unlike the imbalanced do- main distribution in Section 4.2 this balanced setup ensures a more equitable representation of each domain, making it easier to achieve consistent performance across tasks. Compared to DOIFULL), Do contains fewer data and is proper subset of it, allowing for a direct comparison to analyze the trade-off between data volume and distribution in multi-domain training. Additionally, this config- uration presents greater challenge for IDEAL, rigorously testing its robustness and adaptability in handling uniformly distributed dataset.
Table 2: Results of the extended experiment
Benchmark
Mathemuties
Coding
Reasoning
Instruction
TrustAl Truthfulqa
Overall
Methods
Dataset
GSM8K
MATH
HvmanEval
MBPP
BBH
ARC.C
IFEval
Average
Epocha3
Base
56.41
1.10
27.44
5,00
62.13
34.24
12.22
45.80
30.54
Mathematics
75.21
37.70
0.00
0.20
35.04
65.42
22.33
43.48
34,92
Coding
6.75
4.74
40.24
43.00
51.00
33.22
35.15
25.65
29.97
Specific
Reasoning
33.81
11.06
0.00
0.80
56.63
64.07
26.95
34,06
28.42
Instruction
37.60
12.60
33.54
43.18
TrustAl
16.00
8.74
7.32
48.60
26.50
70.85
55.95
17.06
31.38
Joint
2b
69.83
30.94
42.68
44,20
69.32
72.88 88
47,20
55.22
$4,03
30.85
70.39
72.32
53.94
DEAL
D 2EDDAL
2 DiDDAAL))
70.51
69.45
30.20
30.46
44.51
44.51
44.00
45.60
70.58
72.12
74.92
73.90
47,80
47.78
55.94
53.94
55.20
$4.58
Data Volume Matters Little. conclude that increasing data volume does not necessarily lead to improved model performance, even when the data quality is high. Past experience suggests that during the SFT phase, the volume of training data is not the critical factor; rather, the quality of the training data is the key determinant of model performance. Building on this, we add a further observation: in multi-domain joint training, data volume is not the sole concern; the distribution of training data across domains is equally as important as data quality in influencing the final model performance. This highlights the significance of balanced data allocation in achieving optimal results.
Comparing the results of Joint SFT and Joint SFT (FULL), we
IDEAL Demonstrates Robustness. In both Section 4.2 and the experiment this section, IDEAL consistently demonstrates strong capability in optimizing data distribution, leading to improved
8

---

Published as a conference paper at ICLR 2026
domain-specific scores for the model. Similarly, whether targeting individual capabilities average performance, IDEAL achieves measurable enhancements within two iterations. However, as the number of evaluation targets increases, the reference set for validating average performance becomes more generalized, potentially losing the unique characteristics of individual datasets. This leads to less significant improvements in average performance; for instance, shown in Table [] IDEAL's average score increases by only 2.1% %, compared to the 7% improvement observed in Section 4.2
SENSITIVITY STUDY
In this section, delve further into the hyperparameters and other factors that may influence the performance of IDEAL. In Section we explore the optimal value of the parameter m. In Section 5.2 we clarify through experiments that the effectiveness of IDEAL not solely due changes in data volume.
5.1
SENSITIVITY TO THE SELECTION OF IN EQ- P
Strataver.
Jeen 571
2237
20
(a) Mathematics
(b) Reasoning
(c) Instruction
(d) Coding
Figure Models' performance on different domains with varying m values.
As shown in Lemma B is essentially small perturbation around O. The upper bound of the scale m plays a vital role in determining the magnitude of the adjustment B. To explore the optimal range for B, we carry out experiments on three different settings for the choice of 10.1,0.15, 0.3]. Results on DI(IDEAL) in Sec. 4.2 after training epoch are shown in Figure
The choice of m should be neither large too small. When 0.1, limited adjustments yield marginal gains due to insufficient data changes. Conversely, m = 0.3 causes unstable capability fluctuations drastic shifts deviate from the original distribution. Based on our experiments, recommend m = 0.15 as the optimal value, balancing moderate data adjustments and distribution integrity. This configuration achieves the highest average performance by enabling controlled yet impactful updates to data proportions.
5.2 SENSITIVITY TO THE VOLUME OF TRAINING DATA
Table 3: A comparison of performance across different baselines under the same data volume.
Benchmark
Mathematics
Coding
Reasoning
Instruction
Overall
Methods
Dataset
Acc(#Size)
Pass@ (#Size)
Average(#Size)
Average(#Size)
Average
Epoch=1
DoReMi
DacDoReMii
64.82(5.3k)
13.90(12.00
73.79(4.8k)
38.16(1.5k)
55.17
D D2(DereM))
65.21(5.6k)
43.90(12.88 8k)
73.89(5.1k)
38.45(1.6k)
55.36
D2(Do
67.10(10.0k) 0k)
42.24(12.0k)
73.59(1.66 61.kk
30.53(0.5k)
53.37
DOGE
Dacocce
67.73(10.0k)
42.78(12.88) 8k)
73.60(1.7k)
31.12(0.5k)
53.81
IDEAL
DacmeAL
67.55(9.0k)
50.61(7.1k)
74.29(7.33)
39.03(1.9%)
57.87
The IDEAL method automatically adjusts the quantity of training data based on the calculated / but does not control the total volume of training data. In the epoch=l setting of Section 4.2 the total volume of training data in DZuDEAL) is expanded compared to the reweighting methods of DoReMi and DOGE. To ensure fair comparison, volume of DOGE and DoReMi with that of IDEAL and generate new training datasets D2DoReMii and D2DOGE) We evaluate the models on these datasets and summarize results in Table
apply the upsampling method align the training data

---

Published as a conference paper at ICLR 2026
IDEAL outperforms other re-weighting methods not because of increase in data volume, but rather due to its ability to optimize the distribution of training data across domains more effectively and targeted. Although both DoReMi and DOGE exhibit improvements after upsampling (DoReMi increases from 55.17 to 55.36, and DOGE improves from 53.37 to 53.81), 1,, they still fall significantly short of IDEAL,, which achieves average score of 57.87. This highlights IDEAL's superior ability to optimize data distribution and improve model performance across diverse domains.
CONCLUSION
We propose data equilibrium adaptation framework, IDEAL, which effectively optimizes dataset proportions for SFT. IDEAL an iterative method that dynamically adjusts training data distribu- tions and uses a simple and efficient manner, relying solely on existing datasets without the need for additional data or external resources. Experiments demonstrate IDEAL's robustness and effective- ness in diverse settings, consistently improving the multi-task performance of LLMs. Our frame- work provides scalable and practical solution for preparing SFT training data, ensuring balanced and enhanced capabilities in LLMss for real-world applications.
ETHICS STATEMENT
This study did not involve any unethical data collection, experimental procedures, practices. All analyzes were conducted using publicly available or ethically obtained data, and no sensitive per- sonal information was collected. The research design and methods adhere to established ethical standards for academic research, ensuring that no harm is caused to individuals, groups, or the envi- ronment.
REPRODUCIBILITY STATEMENT
The paper fully discloses all the information needed to reproduce the main experimental results to the extent that it affects the main claims and/or conclusions. We release our code at hhtps:: 7/github. com mi not IDEAL and our training dataset details are shown in Appendix A.
ACKNOWLEDGMENT
This work is completed during Chenlin Ming's internship the Shanghai Artificial Intelligence Laboratory and is supported by the Shanghai Artificial Intelligence Laboratory.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Ishika Agarwal, Krishna Killamsetty, and Lucian et al. Popa. Delift: Data efficient language model instruction fine tuning. arXiv preprint wwXiv:2411.0422. 2024.
Alon Albalak, Yanai Elazar, and Sang Michael et al. Xie. A survey on data selection for language models. arXiv preprint arXiv:2402.16827 2024.
Anthropic. The claude model family: Opus, sonnet, http:://wwwcchhhnttrrrpp com/ de@ba 9 cca.t cbab
(acobocc688576777//8 /Mode Caro_Clludee
3. p. 2023. Accessed 18 May 2024.
Jacob Austin, Augustus Odena, and Maxwell Nye al. Program synthesis with large language models, 2021a.
wwtp:xiivvrrrreooooooooo http//bs//2000 .77322
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry Quoc Le, et al. Program synthesis with large language models. arXiv preprint (08.072, 2021b.
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference machine learning, pp. 41-48, 2009,
Mark Chen, Jerry Tworek, and Heewoo Jun et al. Evaluating large language models trained on code. 2021.
Xuxi Chen, Zhendong Wang, and Daouda al. Sow. Take the bull by the horns: Hard sample- reweighted continual training improves Ilm generalization. arXiv preprint arXiv:2402.142700 2024.
Hyung Won Chung, Noah Constant, and Xavier et al. Garcia. Unimax: Fairer and more effective
language sampling for large-scale multilingual pretraining, arXiv preprint arXiv:2304001151, 2023.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try are, the ai2 reasoning challenge. arXiv preprint
2018.
Karl Cobbe, Vineet Kosaraju, and Mohammad et al. Bavarian. Training verifiers to solve math word problems. arXiv preprint arXiv:2 /0. 10.:2//////4/// /4/68, 2021.
OpenCompass Contributors. Opencompass: A universal evaluation platform for foundation models. http:///gg.huui.cooo/eeeeeoaa hctpp//pccccopassss/pocooooaaaa 2023.
https::/github. wwt.
Enrique Dehaerne, Bappaditya Dey, and Sandip al. Halder. Code generation using machine learn-
ing: A systematic review. leee Access, 10:82434-8244 2022.
Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou. How abilities in large language models are affected by supervised fine-tuning data composition. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 177-198, 2024.
Logan Engstrom, Axel Feldmann, and Aleksander Madry. Dsdm: Model-aware dataset selection
with datamodels. arXiv preprint -xXiv:2401.129222 2024.
Simin Fan, Matteo Pagliardini, and Martin Jaggi. Doge: Domain reweighting with generalization estimation. arXiv preprint arXiv:2310.1553 2023.
Ziqing Fan, Siyuan Du, Shengchao Hu, Pingjie Wang, Li Shen, Ya Zhang, Dacheng Tao, and Yan- feng Wang. Combatting dimensional collapse in Ilm pre-training data via diversified file selection. arXiv preprint arXiv:2504.20644 2025.
Giorgio Franceschelli and Mirco Musolesi. On the creativity of large language models. Al & SOCIETY, pp. 1-11, 2024.
11

---

Published as a conference paper at ICLR 2026
Suyash Fulay, William Brannon, Shrestha Mohanty, Cassandra Overney, Elinor Poole- Daale,Dayan,,aa Deb Roy, and Jad Kabbara. On the relationship between truth and political bias language models. arXiv preprint wrXh:2240.05523,, 2024.
Tao Ge, Xin Chan, and Xiaoyang al. Wang. Scaling synthetic data creation with 1,000,000,000 personas. arXiv preprini arXiv:2406.2000 2024.
Carlos Gómez-Rodríguez and Paul Williams. A confederacy of models: A comprehensive evaluttton of Ilms on creative writing. arXiv preprint urXiv:2310.0845:: 2023.
Aaron Grattafiori, Abhimanyu Dubey, and Abhinav Jauhri al. The llama 3 herd of models, 2024. URL htttps:/aaxiv..oot
abs/ /oles/2440 /t/.//2272.2222 21783
Roger Grosse, Juhan Bae, and Cem et al. Anil. Studying large language model generalization with influence functions. arXiv preprint arXiv:2308. 033996,, 2023.
Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. Ppt: Pre-trained prompt tuning for few- shot learning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 8410 8423, 2022.
Yuxian Gu, Li Dong, Hongning Wang. Yaru Hao, Qingxiu Dong, Furu Wei, and Minlie Huang. Data selection via optimal control for language models. arXiv preprint arXiv:2410.07064, 2024.
Daya Guo, Dejian Yang, and Haowei et al. Zhang. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
Frank R Hampel. The influence curve and its role in robust estimation. Journal of the american statistical association, 69(346):3833392 1974.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.038:2 2021.
Linmei Hu, Zeyi Liu, and Ziwang et. Zhao. A survey of knowledge enhanced pre-traing pre-tedieed language models. IEÉE Transactions on Knowledge and Data Engineering, 2023.
Andrew Hyas, Sung Min Park, and Logan et al. Engstrom. Datamodels: Predicting predictions from training data. arXiv preprint arXiv:2202.0063 2022.
Feiyang Kang, Hoang Anh Just, and Yifan et al. Sun. Get more for less: Principled data selection for warming up fine-tuning in 1lms. arXiv preprint arXiv:2405.02774, 2024.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1894. PMLR, 2017.
Yu Li, Qizhi Pei, Mengyuan Sun, Honglin Lin, Chenlin Ming Xin Gao, Jiang Wu, Conghui He, and Lijun Wu. Cipherbank: Exploring the boundary of I1m reasoning capabilities through cryptogra- phy challenges. arXiv preprint arXiv:2504. 19093, 2025.
Xize Liang, Lin Yang. Jie Wang, Yiyang Lu, Runyu Wu, Hanzhu Chen, and Jianye Hao. Boost- ing multi-domain fine-tuning of large language models through evolving interactions between samples. In Forty-second International Conference on Machine Learning, 2025.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human 2021.
falsehoods. arXiv preprini
Liangxin Liu, Xuebo Liu, Derek F Wong. Dongfang Li, Ziyi Wang, Baotian Hu, and Min Zhang. Selectit: Selective instruction tuning for Iims via uncertainty-aware self-reftection. Advances in Neural Information Processing Systems, 37:97800 97825, 2024.
Qian Liu, Xiaosen Zheng, and Niklas Muennighoff al. Regmix: Data mixture regression for language model pre-training, 2025.
org abs 2200 01492
Wei Liu, Weihao Zeng, and Keqing et al. He. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. arXiv preprint arXiv:2312. /5685, 2023.
12

---

Published as a conference paper at ICLR 2026
Keming Lu, Hongyi Yuan, and Zheng et al. Yuan. # instag: Instruction tagging for analyzing supervised fine-tuning of large language models. In The Tweifth International Conference on Learning Representations, 2023.
Trung Quoe Luong, Xinbo Zhang, and Zhanming et al. Jie. Reft: Reasoning with reinforced fine- tuning. arXiv preprint 2xXv:2401.08867, 2024.
James Martens and Roger Grosse. Optimizing neural networks with kronecker-factored approximate curvature. In International conference machine learning, pp. 2408-2417. PMLR, 2015.
Nick Mecklenburg, Yiyou Lin, and Xiaoxiao et al. Li. Injecting new knowledge into large language models via supervised fine-tuning, arXiv preprint arXiv:2404.00213, 2024.
Dheeraj Mekala, Alex Nguyen, and Jingbo Shang. Smaller language models are capable of selecting instruction-tumat tunning training data for larger language models. arXiv preprint arXiv:2402. 10430, 2024.
Niklas Muennighoff, Alexander Rush, and Boaz et al. Barak. Scaling data-constrained language models. Advances in Neural Information Processing Systems, 36:50358-50376, 2023.
OpenAl, Josh Achiam, and Steven Adler et al. Gpt-4 technical report, 2024. URL https:// www.v.org/abb///oo
Zhuoshi Pan, Yu Li, Honglin Lin, Qizhi Pei, Zinan Tang, Wei Wu, Chenlin Ming, H Vicky Zhao,
Conghui He, and Lijun Wu. Lemma: Learning from errors for mathematical advancement in llms. arXiv preprint arXiv:2503. 2025.
Sung Min Park, Kristian Georgiev, and Andrew al. Ilyas. Trak: Attributing model behavior at scale. arXiv preprint 1rX1v22303.41186 2023.
Jupinder Parmar, Shrimai Prabhumoye, and Joseph et al. Jennings. Data, data everywhere: A guide for pretraining dataset construction. arXiv preprint arXiv:2407.06380, 2024.
Qizhi Pei, Lijun Wu, Zhuoshi Pan, Yu Li, Honglin Lin, Chenlin Ming, Xin Gao, Conghui He, and Rui Yan. Mathfusion: Enhancing mathematic problem-solving of Ilm through instruction fusion. arXiv preprint 2in::2223..2111 2025.
Lev Semenovich Pontryagin. Mathematical theory of optimal processes. Routledge, 2018.
Garima Pruthi, Frederick Liu, and Satyen et al. Kale. Estimating training data influence by tracing gradient descent. Advances in Neural Information Processing Systems, 33:19920-19930, 2020.
Yunfan Shao, Linyang Li, and Zhaoye et al. Fei. Balanced data sampling for language model training with clustering. arXiv preprint 40X.v:2402.15526 2024.
Ming Shen. Rethinking data selection for supervised fine-tuning. arXiv preprint arXiv:2402.06094, 2024.
Zhiqiang Shen, Tianhua Tao, and Liqun Ma et al. Slimpajama-de: Understanding data combinations for IIm training, 2024.
100...18110 1081
Chenglei Si, Yanzhe Zhang, and Zhengyuan al. Yang. Design2code: How far are we from au- tomating front-end engineering? arXiv e-prints, pp. arXiv-2403, 2024.
Mirac Suzgun, Nathan Scales, and Nathanael al. Schärli. Challenging big-bench tasks and whether chain-of- -htuhht can solve them. arXiv preprint arXiv:2210.09261. 2022.
Hugo Touvron, Thibaut Lavril, and Gautier et al. Izacard. Llama: Open and efficient foundation language models. arXiv preprint arXiv: 13971, 2023.
Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Shengyi Huang, Kashif Rasul, Alvaro Bartolome, Alexander M. Rush, and Thomas Wolf. The Alignment Handbook. URL
https: / wtt.:a/......gggggggggte.c. wwtpc/////aaaaeeeeeeeeemmeemeeee www.gnntttannbaee
13

---

Published as a conference paper at ICLR 2026
Yuichiro Ueno, Kazuki Osawa, and Yohei et al. Tsuji. Rich information is affordable: A systematic performance analysis second-order optimization using k-fac. In Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 2145-2153, 2020.
Tiannan Wang, Jiamin Chen, and Qingrui et al. Jia. Weaver: Foundation models for creative writing. arXiv preprint arX1v.2401.17268, 2024.
Yufei Wang, Wanjun Zhong, and Liangyou al. Li. Aligning large language models with human: A survey, arXiv preprint arXiv:2307. 2023.
Minghao Wu, Thuy Vu, Lizhen Qu, and Reza Haf. Mixture-of-skills: Learning optimize data usage for fine-tuning large language models. In Proceedings of the 2024 Conference on Empirical Methodds in Natural Language Processing, pp.
2024.
Mengzhou Xia, Sadhika Malladi, and Suchin al. Gururangan. Less: Selecting influential data for targeted instruction tuning. arXiv preprint arXiv:2402. 04300200433,,. 2024.
Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy S Liang, Quoe V Le, Tengyu Ma, and Adams Wei Yu. Doremi: Optimizing data mixtures speeds up
language model pretraining. Advances in Neural Information Processing Systems, 36, 2024.
Can Xu, Qingfeng Sun, and Kai al. Zheng. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304. 2023a.
Hu Xu, Saining Xie, Po-Yao Huang, Licheng Yu, Russell Howes, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feichtenhofer. Cit: Curation training for effective vision-language data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15 15180-15189, 2023b.
Jiasheng Ye, Peiju Liu, Tianxiang Sun, Yunhua Zhou, Jun Zhan, and Xipeng Qiu. Data mixing laws: Optimizing data mixtures by predicting language modeling performance. arXiv preprint arXiv:2403.1699 2024.
Zichun Yu, Spandan Das, and Chenyan Xiong. Mates: Model-aware data selection for efficient pretraining with data influence models. arXiv preprint arXiv:2406.0M: 06046, 2024.
Chi Zhang, Huaping Zhong, and Kuan et al. Zhang. Harnessing diversity for important data selection in pretraining large language models. arXiv preprint arXiv:2409. 16986, 2024a.
Hengyuan Zhang, Yanru Wu, and Dawei al. Li. Balancing speciality and versatility: coarse to fine framework for supervised fine-tuning large language model. arXiv preprint arXiv:2404. 10306, 2024b.
Jeffrey Zhou, Tianjian Lu, and Swaroop Mishra al. Instraction-ollowiing evaluation for large language models, 2023. URL https: wwt./aiww...ga/aaaaaa.o/../goa.t 2311. 07911
14

---

Published as a conference paper at ICLR 2026
A DATASET AND TRAINING INFORMATION
Reasoning: We initially select BigBench Hard (BBH) Suzgun et al. (202) as the benchmark to evaluate the reasoning capabilities of our model. BBH is widely recognized benchmark designed to test a model's s ability to handle complex and diverse reasoning tasks, making it an ideal choice for assessing our model's comprehensive reasoning skills. In section D further employ the challenge part of A12's Reasoning Challenge (ARC) Clark et al. (2018) as our out-of-distribution evaluator. To further enhance the quality of the training dataset, we utilize the official BBH dataset as a foundation and employ GPT-4OpenAl et al. (2024) to regenerate the corresponding answers. This process allows us refine and improve the quality of the dataset, ensuring that the training examples are both accurate and high-quality.
We select GSM8K Cobbe et al. (2021) as the benchmark to evaluate the mathemat- ical reasoning capabilities of our model. GSM8K is a highly regarded dataset specifically designed to test models on a wide range of math problems, including arithmetic, algebra, and word problems. By covering diverse mathematical scenarios, GSM8K serves as a comprehensive tool for evaluating both the precision and depth of our model's mathematical understanding. In section [ we further add the MATH Hendrycks et al. (2021) benchmark. We start with the official GSM8K dataset and leverage the GPT-4 to generate the corresponding thtin-tf- (CoT) solutions. This approach allows us to refine the reasoning steps and enhance the clarity and accuracy of the solutions.
Mathematies:
We choose HumanEval Chen et al. (202I) as the benchmark to assess the coding ca- pabilities of our model. HumanEval is well well-knoww dataset specifically designed to evaluate model's ability to understand, generate, and execute code. It provides a set of programming tasks that require not only syntactic correctness but also semantic understanding, logical reasoning, and problem-solving skills. We also add a new evaluator called the Mostly Basic Python problems (MBPP) Austin al. (2021a) benchmark to more fairly evaluate the model's coding capability. However, due to the lack of an official training dataset for HumanEval, we construct a high- high-quality training set by sampling from the Tulu-Code dataset Ge et al. (2024), which contains 35k data.
Coding:
mnstruction-followinge We select IFEval Zhou et al. (2023) as the benchmark to evaluate our model's s instruction-followin abilities. IFEval is designed to test model's capacity to understand and execute diverse instructions, making it ideal for assessing alignment with user intent across various scenarios. Due to the limited size of the official IFEval training set, we enhance it by sampling additional data from WizardLM Evol-Instruct data Xu et al. (2023a). This combination creates a richer and more diverse training set, enabling our model to better generalize and excel in instruction-folloiing tasks.
TrustAl:
The reliability and truthfulness of large models are widely recognized as critical areas of focus. For evaluating model trustworthiness, we adopt TruthfulQA Lin et al. (202T) as our bench- mark. Since TruthfulQA does not provide an open-source training dataset, we curated a new dataset by expanding the TruthGen Fulay et al. (2024) dataset, and used the GPT-4 to regenerate CoT an- swers.
The reference dataset serves validation set, independent from the training data, and is used to assess the model's generalization performance. Importantly, the reference dataset strictly excluded from training to ensure unbiased evaluation. The performance of the model on this dataset gives us a comprehensive measure of the model's 's ability to generalize across all the tasks it was fine-tuned on. This is critical for validating that the IDEAL framework not only optimizes the training data distribution but also results in improved performance on unseen data. Our reference dataset consists of 500 representative samples selected from the training set, with any duplicates removed to ensure that the reference dataset is both unseen data, allowing for the evaluation of the model's s generalization ability, and completely distinct from the test data, thus preventing any potential data leakage.
Reference Dataset:
Evaluation Metrie: For GSM8K, we adopt the 'accuracy' metric. For the HumanEval benchmark, we use the "pass@1 I' metric to evaluate the probability that the code generated by the model in single attempt successfully compiles. For the BBH benchmark, we consider the naive average metric to evaluate the average score of the model across multiple test capabilities in BBH. In IFEval, we Inst-level-strict-acee acc-s--a-c prompt-
adopt four metrics, namely comppp--oee--striit-co-CP...P.. (ocel-loose-cc(PP--ac))))) and inst-levellloos--cccL--aeee to comprehensively evaluate the model's
15

---

Published as a conference paper at ICLR 2026
1 + Σ D
arg min [(Der, 0) Σ B,C(D;, -
-013) -
(12)
0
capabilities in detail. The P-s-acc metric assesses the accuracy of the model's responses at the prompt level with strict criteria, while the I-s-acc evaluates the accuracy at the instance level with strict standards. On the other hand, the P-I-ace and I-I-acc use more relaxed criteria for evaluation the prompt and instance levels, respectively. In our extended experiment, we use the "accuracy' metric in MATH, ARC.C, and TruthfulQA. For MBPP, we adopt the 'score' metric to evaluate models' coding performance. After obtaining these values, we calculate their average enable quick comparison among different models experimental results, which provides straightforward way to gauge the overall performance of the model in a more comprehensive manner.
B THEORETICAL ANALYSIS
B. PROOF OF LEMMA
Proof Since assume in Eq. we can obtain the following:
that the model is trained to optimality the training set Dar.
VL(Dir 0*) Σ (V")
(10)
for any fixed B. Then calculate the derivative of both sides with respect to By. We obtain:
a0*
B L 8r) 30
V2L
03,
03,
(11)
VVL(D,,
which implies Eq.
B.2 GUARANTEE FOR INVERTIBLE HESSIAN
According to Assumption we have V C the Hessian matrix of the empirical risk function with respect 0. When assuming L(D, 0) strictly convex to f. Hessian matrix V is invertible. Now consider the possibly-noncee nonconvex case. When optimizing near local minimum, the nonconvex problem can be locally approximated by a strictly convex optimization problem, allowing an invert- ible Hessian. In practice, once we obtain a parameter by solving the inner problem Eq. equation | we can form a convex quadratic approximation of the objective around by adding a damping term. Then problem Eq. turns into Eq.
where 0. Now the invertible term Eq. changes to (V2L(Drr, 0-) AT) providing meaningful result.
C ESTIMATION ERROR ANALYSIS
In the process of implementing the IDEAL method, several factors may lead to estimation inaccura- cies, which can potentially affect the overall performance and reliability of the proposed approach.
Sub-optimality in Model Training. The IDEAL method assumes that the model is trained to optimality on the training set D,r as per Equation Eq. (T). However, practical scenarios, to prevent overfitting, models balance is struck to obtain sub-optimal parameters that ensure good generalization across different domains. When the model is trained to its full potential, the gradients and Hessian-related calculations used in method, such as those in Lemma for calculating 8s.,, may not accurately represent the true behavior of the model at its optimal state. This deviation from the ideal training
typically not trained to reach the globally optimal parameters. Instead,
0g'
16

---

Published as a conference paper at ICLR 2026
condition can introduce errors in the determination of the optimal mixing ratio for the training datasets.
Methodological Errors from K-FAC for Hessian Matrix Computation. To enable efficient cal- culation of the influence function, we rely on the K-FAC theory to decompose the Hessian matrix. As described in Section 3.3 approximate the Hessian matrix H by decomposing into block- diagonal form according to different MLP layers. While this approximation significantly acceler- ates the inversion of the second-order gradient matrix, it inevitably introduces methodological errors.
The block chook-li approximation, where H' x B(x'x' @ (E(' simplifies the complex struc- ture of the true Hessian matrix. This simplification means that the calculated influence function may deviate from the exact value. However, Martens Martens Grosse (2015) provides detailed anal- ysis of estimation approximations and derives theoretical upper bound for the approximation error in our Eq. equation
| xx 0 [E('))l<(("" 88' (8 (
Here x(.) denotes the cumulant of its argument. Martens also includes extensive empirical validation of the estimation error. When inverting the approximated Hessian matrix to calculate 80 in Eq. 44
these errors can propagate through the subsequent calculations of OC(D.or,07 in Eq. 8,
Experimental Errors due Random Sampling for Accelerated Computation. To expedite the computational process, we resort to random sampling from the training set. Although the law of large numbers assures that the mean of large number of independent and identically distributed random samples converges the expected value of the population, there is still possibility of introducing random biases. our method, when calculating expectations such as those in the de- composition of the Hessian matrix, the of sampled data instead of the entire dataset lead to errors. These random biases potentially result in inaccurate final results.
D SCALABILITY EXPERIMENT
D.1 MODEL SCALABILITY
Table 4: Results of the model scalability(@wen2.55 7B) experiment
Renchmark
Mathematics
Coding
Reasoning
Instruction
TrustAl
Overall
Methods
GSM8K
MATH
HumanEval
ARC.C
IFEval
Truthfulqa
Average
Epoch=3
Mathematics
83.85
71.98
75.61
58,80
27.51
38.31
37.39
39.42
54. 10
Specific
Coding
82.41
65.74
78.05
2.20
69.17
40.00
52.88
60.00
56.30
Reasoning
44.28
59.50
73.78
59.40
58.06
28. 14
42.68
$2.90
52.34
Instruction
85.60
49.14
78.05
41.20
68.77
33.56
49.18
62.17
58.45
Joint
Db
90.90
72.76
81.71
61.60
78.85
32.88
52.58
62.46
66.71
IDEAL
DICUBAL)
89.73
72.56
81.90
62.34
80.25
38.98
52.88
DammoaL
91.20
72.76
81.71
62.18
81.93
44.27
55.47
61.90
68.93
To present IDEAL's universal on different model structures, we conduct similar experiment in Ta- ble 5 on Qwen2.5 7B model. IDEAL achieved an average performance gain of +2.22 on Qwen2.5 7B, with a remarkable + 11.39 improvement specifically on ARC-C, effectively addressing the lim- itations of simple Joint SFT training. These results collectively demonstrate IDEAL's cross-model generalization capability.
D.2 MODEL SIZE SCALABILITY
To explore the adjustment scale to larger smaller parameter regimes, we conduct extended exper- iments in Table 5 on smaller (Llama3.2 IB) and larger (Llama3.1 70B) variants. On a IB model, IDEAL achieves achivess a +0.77 average improvement over uniform mixing. Although the gain is lower than that observed on the 8B model (+1.26), we argue that this stems from the limited capacity of the IB base model, which inherently constrains the effectivvness of IDEAL. On 70B model, IDEAL still yields a +3.81 improvement, slightly outperforming the +1.26 gain on 8B. This confirms that our K-FAC implementation scale capability in practice.
17

---

Published as a conference paper at ICLR 2026
Table 5: Results of the model size scalability experiment
Benchmark
Mathematics GSMSK
Codling
Reasoning
Instruction
TrustAl Truthfulqu
Overall
Methods
Dataset
MATH
HvmanEval
MBPT
BBH
ARC.C
IFEval
Average
Epoch=3
Db
32.48
14.50
7.13
3.61
47.32
45.81
23.67
32.70
25.90
Llama3.2 1B
D (amoo))
32.67
14.82
6.93
3,70
46,91
44.78
25.32
33.00
26.02
POIDRAL)
7.54
4.10
33.00
26.67
69.83
30.94
42.68
44,20
69.32
72,88
47.20
55.22
54.03
Llama3.1 8B
D PHEBRAL)
69.45
30.46
44,51
44.00
70.58
73,90
47.80
55.94
54.58
DOIDBAL)
70.51
30.20
44.51
45.60
72.12
74,92
47.78
55.94
55,20
90.36
45.22
69.35
70.10
92.90
84,10
60.83
62.54
71.92
Llama3.1
72.13
72.60)
92.71
73.66
D PaIEnCAL)
91.20
49.25
76.33
76.50
92.47
86.77
69.90
63.42
75.73
E EXPLANATIONS OF DATA CONFLICTS AND DATA SYMBIOSIS
In the experiment conducted in Section 4.2 where the model is trained for one epoch, the per- formance of the Specifie SFT models exhibit phenomena that warrant further explanation. In this section, provide a detailed analysis of the results of the Specific SFT models trained for one epoch, as presented in Table [ and offer pertinent interpretations of the observed phenomena.
The Training set of instructions includes diverse data types. In the context of Specific SFT, only the model trained on the instruction dataset demonstrates comprehensive capabilities, while other models exhibit significant performance degradation in specific domains. Upon examining the data content, we find that the selected training set included a variety of data types, such as mathematics, reasoning, coding, and more, to enhance the model's cestruction-oollwwee of the data content enables the model to perform well across different domains. As result, the trained model excels in all four selected domains and even outperformed models trained on domain- specific datasets in the areas of Coding and Reasoning. This phenomenon is further validated in subsequent experiments involving training for epochs.
ability. The high diversity
Reasoning data enhances instruction-follvviii and reasoning capabilities. The Reasoning Spe- cific SFT model trained on the BBH dataset demonstrates strong mnstuuction-folloing capabilities. We hypothesize that the improvement in reasoning ability enables the model to better analyze user- provided instructions and engage in simple logical reasoning, thereby generating outputs that more closely align with expected results. The enhancement in reasoning ability also leads to improved performance in mathematics, which intuitive given that mathematical reasoning can be viewed a symbolic variant of logical reasoning. Mathematical proficiency still relies the assistance of logical reasoning skills. This observation underscores the interconnectedness of reasoning and following performance.
mathematical abilities in enhancing the model's overall
Coding Data Appears Detrimental to Capabilities in Other Domains. The model trained using the Coding dataset for SFT exhibits improved performance in the Coding domain, but its perfor- mance in other domains declines. We hypothesize that the exclusive focus on code-specific formats during training may cause the model to lose its ability to handle normal text-based dialogue data, thereby negatively impacting its performance in non-code-relates domains. This suggests that while specialized training in coding enhances the model's 's coding proficiency, may come the cost of its generalizability and effectiveness in broader contexts.
Based on the phenomena and explanations discussed above, we can draw the following conclu- sions: data from different domains exerts either positive negative influences on capabilities in other domains, indicating the presence of Data Conflicts and Data Symbiosis. When mixed datasets used for training to achieve balanced improvements across multiple domains, the com- position of the training dataset becomes a critical factor. This highlights the importance of carefully selecting and balancing data types optimize the model's 's performance across diverse domains.
18

---

Published as a conference paper at ICLR 2026
F SUPPLEMENTARY INFORMATION ABOUT 3
In this section, we present the computed 33 values calculated in Appendix D along with the data composition distributions for both iterations of the IDEAL framework.
Table 6: values and training data distribution
Benchmark
Dataset
Mathematics
Coding
Reasoning
Instruction
TrustAl
D ((IDE)))
7.34
84.30
15.74
18.02
-5.00
B (x107))
D2(IDEAL)
4.24
73.24
13.45
15.74
-4117
DI(IEAA))
4.9k
5.5k
4.9k
4.9k
4.7k
#Size
D2(DEAL)
4.9k
6.3k
5.0k
5.1k
4.7k
The base model uses uniform initial training data distribution, while IDEAL demonstrates stability and consistency fine-tuning data quantities. The trends of changes across the five training domains remain consistent over two iterations and scale proportionally. As analyzed in Section 5.1 the |3 value ensures relatively stable model capabilities across all domains.
limiting
G SUPPLEMENTARY INFORMATION ABOUT WALL-CLOCK
In this section, we present our real running time of the scalability experiments in Appendix D Since we employed an 8xA100 GPU configuration, the system can complete two full iterations of IDEAL training within 12 hours.
Table 7: Wall-clock in Appendix D
Phases
EFLOPs
Actual
SFT-0
0.824
8h
IDEAL-1
3.164
32h
SFT-1
0.835
8h
IDEAL-2
3.159
32h
SFT-2
0.837
8h
Total
8.819
88h
H DECLARATION OF LLM USAGE
The paper only used LLMs for polishing grammar and language. No LLMs were involved in the design of the methodology, experimental process, data analysis, any other core component of the research.
19


Reviews:
[{"review_id": "KNXVxs7TZp", "reviewer": "Reviewer_c1Z3", "summary": "This work explores a widely discussed but yet inconclusive research question: how to achieve the optimal ratio of data from different domains in the SFT dataset for LLMs. This work proposes an iterative data ratio optimization method, using a gradient-based approach to iteratively optimize the training data distribution. It dynamically adjusts the amount of domain-specific data based on its impact on the performance of downstream tasks. Experiments across different capabilities show that IDEAL outperforms the traditional uniform data allocation strategy.", "questions": "1. Would considering the quality of data samples during the dataset sampling process lead to further improvements in accuracy?\n2. Please refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. A classic and intriguing research problem, accompanied by an innovative solution.\n2. This work provides some theoretical proofs under the premise of a well-formalized description.", "weaknesses": "1. Dynamic data curation has already been explored in some existing works [1], and it is recommended to introduce the related studies in the discussion.  \n2. Although the method is optimized for efficiency, the computational overhead is still relatively large, as shown in Table 7. This somewhat undermines the applicability of the method.\n3. According to the experimental results in Table 1, at epoch=3, the performance of IDEAL is actually worse than its performance at epoch=1. This, to some extent, makes the method face more hyperparameter tuning challenges in practical application scenarios.\n\n[1] CiT: Curation in Training for Effective Vision-Language Data. ICCV 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "j7iNXq1vxD", "reviewer": "Reviewer_grmy", "summary": "This paper introduces IDEAL (Data Equilibrium Adaptation), a framework aimed at optimizing the mixture proportions of data from different domains when performing multi-capability Supervised Fine-Tuning (SFT) on Large Language Models (LLMs). The authors argue that naive data mixing often leads to suboptimal performance and propose a principled approach to find a better \"data equilibrium.\" IDEAL uses bi-level optimization theory to derive how changes in the proportion of data from each domain affect the model's loss on a separate reference dataset. This involves estimating gradients that depend on the Hessian matrix of the training loss. To make this computationally feasible for LLMs, the framework employs approximations like K-FAC. The IDEAL algorithm iteratively refines the data proportions by performing SFT, calculating these gradients, updating the proportions, and then resampling the dataset for the next iteration. Experiments across several capability domains suggest that IDEAL leads to improved multi-task performance compared to uniform data mixing.", "questions": "1. The K-FAC approximation involves selecting 'important' MLP layers based on variance . Could the authors provide more details on how these layers are selected and how many are typically used? Does this selection significantly impact the results?\n\n2. In Table 1 (Epoch=1), the 'Specific SFT' model trained only on 'Instruction' data outperforms the 'Specific SFT' models trained on 'Coding' and 'Reasoning' data on their own respective benchmarks (Coding: 46.95 vs 37.20, Reasoning: 61.87 vs 60.19). This seems counter-intuitive. Could the authors offer an explanation, perhaps related to the composition of the 'Instruction' dataset mentioned in Appendix E ?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper addresses the critical and practical challenge of determining optimal data mixtures for multi-capability SFT, moving beyond simple heuristics.\n\n2. The proposed IDEAL framework is grounded in optimization theory (bi-level optimization), providing a principled, gradient-based approach to adapt data proportions based on their impact on a reference set performance.\n\n3. The work incorporates practical considerations for LLMs by using techniques like K-FAC to approximate the Hessian, making the theoretically complex approach computationally tractable, and demonstrates empirical improvements over baseline mixing strategies.", "weaknesses": "1. The computational overhead remains extremely high, potentially limiting practical utility. Despite approximations, the iterative nature of IDEAL, requiring a full SFT cycle per iteration plus complex gradient calculations involving Hessian approximations, makes it very resource-intensive. The appendix reveals the total time is an order of magnitude higher than standard SFT.\n\n2. The method's effectiveness heavily relies on the accuracy of Hessian approximations (like K-FAC), which can introduce significant errors. The paper acknowledges this possibility but does not quantify the impact of these approximation errors on the reliability of the calculated gradients ($\\beta$ updates). Inaccurate gradients could lead to suboptimal or unstable optimization\n\n3. The framework's optimization target and granularity may be suboptimal. IDEAL optimizes based on performance on a chosen reference set $\\mathcal{D}_{ref}$, making the results sensitive to this set's representativeness. Furthermore, it only adjusts quantities at the domain level via sampling, ignoring potential quality variations within a domain's data. More granular, sample-level weighting might be more effective.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2LzQ4Xiy96", "reviewer": "Reviewer_QVGQ", "summary": "The paper proposes IDEAL, a framework for optimizing the distribution of multi-domain training data during supervised fine-tuning (SFT) of large language models. The method uses gradient-based optimization with second-order information to iteratively adjust domain-specific data volumes, claiming approximately 7% improvement over uniform data allocation strategies.", "questions": "The paper claims IDEAL \"ensures a balanced dataset composition\" but provides no formal guarantees.\nThe assertion that data volume matters little (Section 4.3) contradicts extensive prior work and is based on limited evidence.\nThe \"data conflicts and data symbiosis\" discussion in Appendix E lacks rigorous empirical support.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Important Problem Selection. The paper addresses a genuinely important challenge in LLM training - how to balance multiple capabilities during supervised fine-tuning. This is a practical problem that many practitioners face, and finding principled solutions has real value.\n2. Systematic Approach. Rather than relying on heuristics or manual tuning, IDEAL provides a systematic, gradient-based framework for optimizing data distributions. The mathematical formulation, while not novel, is reasonably rigorous and provides a clear optimization objective.\n3. Relative Comprehensive Experimental Setup. The paper evaluates across multiple diverse domains (mathematics, coding, reasoning, instruction-following) Includes both single-epoch and multi-epoch training scenarios", "weaknesses": "1. Limited Technical Novelty. The core contribution relies on well-established techniques (influence functions, K-FAC approximation) applied to data mixing. The formulation in Eq. (1) introducing β parameters for data repetition is straightforward, and the bi-level optimization problem (Eq. 2) follows standard approaches. The use of influence functions for data weighting has been extensively explored in prior work, making the technical contribution incremental.\n2. Experiment results and setup is hard to follow. What models are used in table1 and table 2? No statistical significance testing is provided for the reported improvements. The baseline comparisons are limited, missing recent strong methods in data selection and mixing such as regmix, autoscale and icons.\n regmix: https://arxiv.org/abs/2407.01492\nautoscale: https://arxiv.org/abs/2407.20177\nicons: https://arxiv.org/abs/2501.00654\n\n\n\n3. Computational Efficiency Concerns. Despite claims of efficiency, the method requires: Multiple full training iterations (88 hours total as shown in Table 7). Complex Hessian approximations that may not scale well. The K-FAC approximation introduces significant estimation errors (acknowledged in Appendix C but not properly addressed)\n\n4. Theoretical Limitations. Assumption 1 about twice differentiability and convexity is unrealistic for modern LLMs. The paper acknowledges in Appendix B.2 that the Hessian may not be invertible in practice, requiring ad-hoc damping terms. The connection between the local perturbation analysis and global optimality is not established\n\n5. Experimental Design Issues. The reference dataset of only 500 samples is too small to reliably estimate performance across diverse capabilities. The choice of m=0.15 appears arbitrary despite the sensitivity analysis. The random sampling with factor σ=0.5 introduces additional variance not properly controlled for\n\n6. Presentation and Clarity Issues. The paper lacks clear ablation studies isolating the contribution of different components.The \"data equilibrium\" terminology is not well-justified theoretically. Important details are relegated to appendices, making the main paper incomplete.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CPLoZJX7br", "reviewer": "Reviewer_i18D", "summary": "This paper proposes IDEAL, a framework for adaptive data proportion optimization in multi-domain supervised fine-tuning (SFT) of large language models. The method formulates the allocation of training data from different domains as a bi-level optimization problem, where the validation loss is minimized with respect to domain-wise mixing coefficients. IDEAL leverages influence functions and second-order (Hessian-based) approximations, implemented efficiently through layerwise K-FAC techniques, to update data proportions. Experiments on multiple benchmarks covering domains such as mathematics, code, reasoning, and instruction following suggest that IDEAL improves average performance over uniform or heuristic data mixing baselines. Ablation studies explore hyperparameter sensitivity and approximation robustness.", "questions": "Please see Weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The problem of domain proportioning in multi-domain fine-tuning is relevant and timely for large-scale LLM adaptation.\n- The paper provides a reasonably clear mathematical formulation of the bi-level optimization problem and explains how influence-function approximations can guide proportion updates.\n- The method is relatively easy to integrate into existing SFT workflows and scales to medium-sized LLMs with standard K-FAC approximations.", "weaknesses": "I have the following concerns. *If the authors could properly address them during the rebuttal phase, I am willing to raise my score.*\n- The novelty is quite limited. IDEAL largely combines elements that have appeared in prior works such as DoReMi, DOGE, and influence-based data reweighting. The paper does not provide a clear conceptual or algorithmic distinction beyond rephrasing the same principles in a unified framework.\n- The method depends heavily on strong approximations (block-diagonal K-FAC, mini-batch stochastic estimation), but the effect of these approximations is not quantitatively analyzed. There is no estimate of how much the inverse-Hessian-vector product deviates from the true gradient, which raises doubts about the claimed stability.\n- The empirical evaluation, though broad, does not test IDEAL under difficult or realistic conditions such as domain shift, heterogeneous modalities, or extreme imbalance. The results are confined to a few text-based domains and mid-size models, limiting generality.\n- Some important recent baselines [1,2,3] are missing. The paper would benefit from either a comparison with these methods or a discussion of their relevance.\n- The sensitivity analysis is shallow. Although the paper examines one hyperparameter $m$, it does not analyze the dependence on other key parameters or propose principled ways to set them, which undermines practical applicability.\n\n[1] Mixture-of-Skills: Learning to Optimize Data Usage for Fine-Tuning Large Language Models. EMNLP 2024.\n\n[2] How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition. ACL 2024.\n\n[3] Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples. ICML 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)