OptimSyn: Influence-Guided Rubrics Optimization for Synthetic Data Generation
Authors: Zhiting Fan
Ruizhe Chen
Tianxiang Hu
Ru Peng
Zenan Huang
Haokai Xu
Yixin Chen
Jian Wu
Junbo Zhao
Zuozhu Liu
Venue: ICLR 2026 Poster (2026)
Keywords: LLMs
data synthetic
instruction tuning
Primary area: foundation or frontier models, including LLMs

Abstract: Large language models (LLMs) achieve strong downstream performance largely due to abundant supervised fine-tuning (SFT) data that imparts problem-solving capabilities. However, as applications expand, high-quality SFT data in knowledge-intensive verticals (e.g., humanities and social sciences, medicine, law, finance) is exceedingly scarce: expert curation is costly, privacy constraints are strict, and label consistency is hard to guarantee. Recent work turns to synthetic data, typically prompting a teacher model over domain documents and filtering with handcrafted rubrics. Yet, rubric design is expert-dependent and rarely transfers across domains; moreover, prevalent heuristic optimization follows a brittle loop (write rubric $\rightarrow$ synthesize $\rightarrow$ train $\rightarrow$ inspect $\rightarrow$ guess tweaks) that lacks reliable, quantitative feedback about a rubric's true contribution to downstream performance.
We argue for assessing synthetic data quality through its causal impact on the target model, using this feedback to guide data generation. Inspired by classic influence functions, we repurpose an optimizer-aware estimator that uses gradient information to quantify each synthetic sample's contribution to the objective of a given target model on specific tasks. Our analysis reveals a gap: although synthetic and real samples may be close in embedding space, their influence on learning can differ substantially. Building on this insight, we propose an optimization-based synthetic data framework that adapts rubrics with target-model feedback. Instead of manually engineering domain rubrics, we supply lightweight guiding text and delegate rubric generation to a rubric-specialized model conditioned on the task; crucially, rubric (and data) selection is supervised by estimated downstream impact rather than proxy formality. Empirically, the framework yields consistent gains across domains (HSS and health), target models (e.g., Qwen and Llama families), and data generators, demonstrating broad generalization and engineering portability without task-specific tuning.

Published as a conference paper at ICLR 2026
OPTIMSYN, YN: INFLUENCE- RUBRICS OPTI- MIZATION FOR SYNTHETIC DATA GENERATION
Zhiting Fan 1:2 Ruizhe Chen', Tianxiang Hu', Ru Peng', Zenan Huang Haokai Xu' Yixin Chen', Jian Junbo Zhao 1. Zuozhu Liu
'Zhejiang University, IIccusiion AI, Ant Group, Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence {zhiting.28 23, zuozhuling li inti..nn.. zju. edu. cn
ABSTRACT
Large language models (LLMs) achieve strong downstream performance largely due to abundant supervised fine-tuning (SFT) data that imparts problem-solving capabilities, However, as applications expand, high-quality SFT data in knowledge-intensiv domains (e.g.. humanities and social sciences, medicine, law, finance) is exceedingly scarce: expert curation is costly, privacy constraints are strict, and label consistency is hard to guarantee. Recent work turns to synthetic data, typically prompting a generator over domain documents and filtering with handcrafted rubrics. Yet, rubric design is expert-dependent and rarely transfers across domains; moreover, prevalent heuristic optimization follows a brittle loop (write rubric synthesize + train + inspect + guess tweaks) that lacks reliable, quantitative feedback about rubric's true contribution downstream perfor- mance. We argue for assessing synthetic data quality through its training utility on the target model, using this feedback guide data generation. Inspired by classic influence estimations, we repurpose an optimizer-aware estimator that uses gradi- ent information of a given target model on specific tasks. Our analysis reveals gap: although syn- thetic and real samples may be close in embedding space, their influence on learn- ing can differ substantially. Building on this insight, we propose an optimization- based synthetic data framework that adapts rubrics with target-model feedback. Instead of manually engineering domain rubrics, we supply lightweight guiding text and delegate rubric generation to rubric-specialized model conditioned on the task; crucially, we employ influence score reward and optimize the rubric generator with reinforcement learning. Empirically, the framework yields consis- tent gains across domains (HSS and health), target models (e.g., Qwen and Llama families), and data generators, demonstrating broad generalization and engineer- ing portability without task-specific tuning.
quantify each synthetic sample's contribution to the objective
INTRODUCTION
Large language models (LLMs) now excel across a wide range of downstream tasks (Achiam et al,, 2023 Comanici al. 2025 Guo et al.) 2025), due in small part training on vast and diverse corpora. In particular, supervised fine-tuning (SFT) has endowed LLMs with strong instruction- following and problem-solving capabilities (Peng et al. 2023 Maeng al. 2017). However, as LLMs are deployed in increasingly specialized scenarios, high-quality real SFT data has become acutely scarce especially in knowledge-intene intensive verticals such as the humanities and social sci- ences, medicine, law, and finance. The barriers are both practical and structural: domain expertise costly and limited, privacy constraints are stringent, and large-scale manual annotation difficult to standardize, expensive to procure, and hard to keep consistent.
Synthetic data has recently emerged as promising way to alleviate this bottleneck (Wang et al.) 2022 Maeng et al. 2017). Most current pipelines bootstrap from raw materials (e.g.. documents), query a teacher model to produce question-answer pairs, and then filter (or steer) generations with
Coorreponning Author

---

Published as a conference paper at ICLR 2026
Mow
-20
(a) Influences W.L.t gains.
(b) Distributions in embedding space. (c) Distributions in gradient space.
Figure 1: Visualization of influence score as an effective feature for assessing sample utility. (a) Relationship between validation influence scores and training performance. (b) Distribution of three validation sets in the embedding space of Qwen3-8B-Bar (c) Distribution of three validation sets in the loss-gradient embedding space, showing closer alignment and shared optimization directions.
pre-defined rubrics (rules prompts) (Penedo et al., 2024: Jiang al. 2025 Yue et al. 2024
Yet this paradigm faces two fundamental limitations. (i) Limited transferability. Rubric design is largely expert-driven and domain-speciffe rubrics that work in one field often fail to generalize to another, undermining the universality of synthesis strategies. (ii) Heuristic optimization. The common loop- "handeraft rubrics + synthesize data train model + inspect outcomes - propose revisions"--ee on tacit experience and provides little quantitative feedback. Human observers cannot reliably attribute downstream performance changes to specific rubric choices, making the process slow, brittle, and uncertain.
To address these limitations, we propose to directly quantify the training utility of each synthetic example for a given target model and task using gradient-based influence estimator (Xia et al.) 2024]. Concretely, we approximate the contribution of candidate QA pair to the validation objec- tive under the same update rule used during fine-tuning (Adam), yielding an influence signal that aligned with the actual optimization dynamics. Preliminary analyses (Figure I reveal a marked gap between representation proximity and training impact: synthetic and real samples can be close embedding space yet diverge substantially their estimated influence, which helps explain why seemingly on-distribution samples may still underperform during SFT. Moreover, dataset-level in- fluence aggregates exhibit a strong positive correlation with held-out accuracy, validating influence reliable proxy for synthetic data quality. These observations motivate replacing heuristic rubric engineering with model-impact supervision grounded gradients.
Building this insight, we introduce prompter (the policy model) optimization framework that treats rubric construction as a learnable component driven by target-model feedback. Starting from minimal guiding text, rubric generator proposes seed document-specinn conditioned on the seed document and rubric, synthesizes QA pair; and the target model supplies verifiable reward that combines lightweight validity checks with the optimizer-aware influence score. We then update the policy using GRPO algorithm (Guo et al., (202),, thereby closing the synthesis-training loop and explicitly maximizing expected downstream improvement. Empirically, this framework yields consistent gains across knowledge-intensit inteeni-iiniivee domains (the humanities and social sciences, and medicine and health), transfers across target model families and scales, and robust to the choice of data generator, turning rubric design from brittle, expert-crafted heuristics into a portable, model-aligned optimization problem. The contributions of this paper are threefold:
rubrics; generator model,
1. Optimizer-aware influence estimation. We adapt classical influence- cestuence-estimation ideas to modern synthetic-data pipelines and derive practical, optimizer-aligne estimator of each synthetic sample's 's training utility.
2. From heuristics to supervision. We replace heuristic rubric engineering with model- impact objective by training a rubric generator to maximize estimated downstream benefit (compatible with RL or gradient-based updates), yielding more effective synthetic data.
2

---

Published as a conference paper at ICLR 2026
(a) RL Workflow
KL
Reference Model
Mubrica
Sym@4
Group Computation
Rubricsy
Syme4
Reward Foomat latluenee
Seed Doc
Prompter
Rubricsg
SynOdG
Score
Generator
(b) Influence Score Computation
projection
Warmiu
* Gradient
Features
projection
Initial QA Target Gradient Features
Vol Set
The proposed RL framework for synthetic data generation with optimizer-aware rewards. (a) RL workflow. Starting from a seed document, a prompler instantiates G seed-specific rubrics, and generator produces synthetic QA pair. (b) Reward computation. We employ the influence score for each synthetic QA pair as reward, followed by policy update.
Figure 2:
3. Transferable and efficient across domains. Empirically, the learned rubrics are model- and lask-conditioned reducing reliance domain expertise and enabling deployment in privacy-constrained, knowledge-intensive fields where manual curation infeasible.
METHOD
2.1 OVERVIEW
Given set of seed data S (s.) the goal of data synthesis is construct dataset contain- ing synthetic question-answer pairs {(Q,, N Existing methods typically instruct teacher model-ood often a strong external model (e.g., an API) with carefully designed rubrics to gener- ate such data. However, rubric-based synthesis is constrained by human intuition, highly domain- specific, and thus poorly transferable across tasks and models. To overcome these limitations, our framework employs dedicated rubric generator that produces targeted rubric B₃ for each seed datum S, and a specified target model. We then condition the teacher model on (B;, Si) to gener- a synthetic pair A4). In the following, detail how we quantify the training utility of synthetic pair (Q,, A:) using optimizer-aware influence estimates and how this feedback drives the learning of the rubric generator.
2.2 ESTIMATING THE INFLURNCE OF SYNTHETIC DATA
While human-crafted rubrics appear reasonable, there is no guarantee that they actually lead to ef- fective dataset generation. promising yet underexplored alternative is adopt more principled generation process that does not rely solely on human intuition. Influence functions (Pruthi et al.) 2020), which approximate training dynamics via first-order analysis, have been widely used es- timate the impact of individual training examples on held-out performance. Prior work (Xia et al.) 2024) has shown their effectiveness selecting data that benefits downstream tasks. This naturally motivates use of influence functions principled guide for dataset generation.
Preliminaries. The classical influence function (Koh & Liang 2017) quantifies how individual training points affect a model's parameters and, consequently, its predictions. TracIn (Pruthi et al.) 2020) pursues the same goal with scalable, first-order, trajectory-basee estimator: it accumulates gradient inner products of training checkpoints to approximate influence. Because requires only per-example gradients, learning rates, and saved checkpoints, TracIn is practical at LLM scale. Since contemporary LLMs are typically optimized with Adam (Adam et al.) 2014). we adopt an Adam-compatible variant (Xia et al. 2024). Given training sample its influence evaluation
3

---

Published as a conference paper at ICLR 2026
Algorithm 1 Influence-Guider Data Synthesis Framework
Require: Seed data set S, Generator model T, Target model M, Policy model Ti, Validation set Daa, Number of updates K. Rollout group size G, Reference model Teet 4- To;
1: for k = 1 to K do
Sample a mini-batch of seed data [..))m= X. 3: for each seed data S; E S do
4:
Generate G rubrics Bt., ~ T-(-)S) for = 1.. for each rubric B.,, do
5:
6:
Ohtain (Qi,,, A.,) T(,, Ba,))
Estimate influence score IF Adam QQ, A) according to Eq.
8:
Compute trajectory reward Ii(T) Valid(Q, IF(Q. A) A Valid(Q, end for
9:
10:
Aggregate rewards and compute group-normalizee advantages:
11:
ÁLe = (R(Ti) き Σ5=1 I((T))) V V (R(T;) + 2 R(Tx))* 2 +
12:
Aggregate loss over tokens:
13:
I(0) - _ Ein Fe Elal min (0)) Ace, clip(ri.)())) Aio]
BKI(TO || Tmd)
14:
Update policy end for
16:
end for
17:
return Optimized prompter T
sample z' can be represented as:
Inf Adam (2, 2')
Σ Tt cos cos((v((z':00)) ((2,
where f) is the average learning rate epoch (out of T total epochs) and 0g is the checkpoint after epoch T requires the moment statisties (m, v) which depend past gradients, with details provided in Appendix B
Why gradients, not embeddings? Empirically (Fig. Ib and Ic]. we observe two phenomena: (i) synthetic sets whose gradient distributions are closer to the validation distribution yield better down- stream performance; (ii) the same trend does not hold in embedding space, where proximity often fails to predict gains. This explains why aesthetically "high-quality" samples may not train well: they align semantically but steer optimization in suboptimal directions. Let IF(Q, A) denote the Adam-compatible influence score (Eq, [1] of the synthetic pair (Q, A) with respect to the validation set. We further quantify this by correlating IF averages with held-out accuracy (Fig. Taa: IF exhibits strong positive correlation, validating
reliable proxy for synthetic data quality.
Takeaway. Influence functions provide a principled, model-aware signal for judging the training utility of synthetic examples, turning data synthesis from heuristic rubric engineering into model- centric optimization. Building on this, we close the loop with an RL framework in which a rubric generator acts as the agent (replacing ad-hoc human design), explores the rubric space, and receives influence-based rewards that estimate each synthesized sample's contribution to downstream im- provement. This alignment of generation with measured model impact yields more useful data with less domain-speent -seecffc handcraftng, better sample efficiency, and a clear path to automatic refinement across tasks and domains.
2.3 INFLUENCE- GUIDED REINFORCEMENT LEARNING
We train the rubric generator policy that explores the rubric space and receives influence-based feedback from the target model. Formally, given a seed datum S. the policy L.M va produces rubric B Ta(- [S); teacher model then conditions on (S, B) to synthesize pair (Q, A); the target model supplies a scalar reward that scores the training utility of A). A trajectory is {S, B, (Q, and the objective is maximize E,~r, [R()))
Verifiable, influence- rewards. We combine automatic validity checks with an optimizer- aware influence estimate. Let Valid A) (0, 1) be conjunction of lightweight verifiers (for-
4

---

Published as a conference paper at ICLR 2026
matting, non-triviality, safety). The reward is:
R(r) = Valid A) 4) (1 Valid A)):
(2)
where À O penalizes invalid generations. This yields a verifiable signal that is aligned with downstream improvement while discouraging degenerate outputs.
Policy optimization. We use clipped policy-gradien objective (GRPO/PPO-stt with group- relative baselines for variance reduction. For each seed we sample G rubrics, producing trajecto- ries (T:) Denote by Ta,(1) the 1-th token and by <i its prefix. The objectíve is
1ra
1 (0) = G
Σ
min (()( Are. clip(ra..()), 1 e, 1+e) Acs] - K Trae); (3)
with importance ratio (0) = TT(Te,()) Ti Tr,ce) / Told
ri,c)), and group-normalized ad- (e, (T,
vantage A. (R(T,) - Eド= R(T,)) / Ej= (R(T,) Ei= R(Tx)) + 6, where the clipping parameter, 3 controls KI. trust region against a reference policy uee (e.g., the SFT model), and 6 > 0 stabilizes normalization. We add entropy regularization to encourage exploration.
This RL loop closes the synthesis-training feedback cycle: the policy learns rubrics that system- atically maximize measured training impact, yielding synthetic data that is not only plausible by heuristic criteria but empirically helpful for learning.
EXPERIMENT
3.1
EXPERIMENTAL SETUP
Models The teacher LLM (also denoted as generator), unless otherwise specified, is Qwen3-235B.
Instruct (Yang et al. 2025), which generates synthetic supervised fine-tuning (SFT) data; in ablation studies, we substitute it with GPT-4.1 (Achiam et al. (2023) and Gemini-2.5-Pro (Comanici et al.) 2025) to evaluate the sensitivity of our pipeline to the teacher model choice. The target LLM is by default Owen3-8B-Base trained the synthesized data; we further examine in two settings: (Dubey et al. (202)) and (ii) Qwen variants-Qwent 4B- 4B-Base, Qwen3-8B-Bass and Qwen3- tiven3-14-BBaa 14B-Base BiiiBas--using the same learned prompter. Finally, the prompter base model is initialized from Bwvn3-88-IInstruee which serves as the base model for rubric optimization throughout training.
(i) replacing the target with Llama3-8B-Ban
Training Dataset We apply our SFT data synthess framework two Humanities and Social Sciences and Medical and Health. For HSS, we curate seed docu- ments from books: specifically, we select 223 humani- ties and social sciences books with high public ratings on Goodreads, with the domain distribution shown in Fig. 3 (left). Texts are segmented by table-of-contents chapters, and we remove boilerplate, front/back mat- ter, and other non-textual or noisy content. For the medical domain, we draw seed documents from the pretraining corpora released with the Meditron large language model (Chen et al., 2023) and from a sub- set of open-access PubMed abstracts; the distribution is shown in Fig. 3 (right). Because rubric generation requires feeding each seed document to the prompter model and is constrained by the model's input window, we retain only documents shorter than 8k tokens. After filtering, the HSS and medical corpora contain 18,665 and 26,435 seed documents, respectively. Further details of the raw sources are provided in Appendix D.3
2010
4
Vidkal
Figure 3: Domain distribution of seed documents used to synthesize question- answering pairs: Humanities and Social Sciences and Medical and Health.
Validation Set Construction When constructing the validation set, we follow two principles: (i) there is no overlap information leakage with the SFT training data, and (ii) the validation data
5

---

Published as a conference paper at ICLR 2026
Table I: Results across two domains. All metries are accuracy (higher better). Across both HSS and Medical domains, our method consistently upgrades Qwen3-8B- -e-a--BBBBsse beyond widely used open SFT corpora, often rivaling or surpassing Swenn-8B-Instruct
Humanilies and Social Sciences (HSS)
Model
MMLU pro
Super GPQA
HLE
BBH
HellaSwar wag
DROP
Qwen3-8B- Swen3-8B-Base
22.83
20.77
5.70
35.62
35.71
45.73
Qwen3-8B- Swwn3-8B-Instruee Instrutttct
49.87
23,44
4.66
85.21
78.71
80.73
MAGACorpus
39.90
21.18
8.29
58.21
72.88
62.53
Bonito
39.37
21.36
5.70
45.78
83.00
50.95
Conder
41.99
23.29
6.22
74.21
59.32
41.52
Cosmospedia
27.30
20.03
5.18
55.63
79.33
49.70
Longwriter
30.45
12.02
6.22
34.89
65.09
23.62
Openhermes
49.56
24.60
3.63
73.64
73,41
61.62
Synthquestion
51.71
23.99
5.18
74.24
80.20
61.88
WebrPro
50.39
22,85
7.77
68.32
77.60
58.77
Wildchat
52.76
22.11
6.74
76.32
73.41
54.31
Ours
56.96
26.07
7.85
75.65
78.08
65.42
Medical and Health
Model
MMLU pro
Super GPQA
HLE
Health Bench
PubMed
MedQA
Qwen3-8B-BB
45.97
28.06
10.36
70.06
65.90
$1.45
Owen3-8B-Instrue
60.39
37.16
7.21
87.70
65.70
$7.09
ChatDoctor
16.48
21.22
6.31
85,40
21.91
MedQA
13.45
34.61
9.91
30.46
71.70
53.41
Medical-ol
60.51
27.26
8.11
70.14
73.90
58.75
Medical-RI -Distill
57.33
35.28
4.95
80.38
73.40
57.81
ReasonMed
55.00
28.01
14.41
51.76
79.80
23.56
Ours
56.97
38.28
10.81
74.82
80.70
58.75
come from public benchmark splits that share the same distribution the downstream evaluations. In the HSS domain, we use the dev split of MMLU-pro (History) as the validation set; in the Medical domain, we use the official MedQA test split as the validation set. To ensure that the datasets used for synthetic data evaluation do not leak information from out-of-domain benchmarks, we follow the official AllenAI Tülu decontamination toolkit, implemented on Elasticsearch, to conduct leakage checks; details are provided in Appendix D.
Evaluation Benchmarks We evaluate models on benchmarks in two domains. Because the hu- manities and social sciences (HSS) lack dedicated evaluation suites, extract HSS- subsets from three comprehensive datasets: MMLU-pro (History) (Wang et al.) 2024), SuperGPQA (His- tory) (Du al.) 2025). and Humanity's Last Exam (Phan et al. (025) (Humanities Social Science). We include Big Bench Hard (Kazemi et al.) 2025). HellaSwag (Zellers et al. (019). and DROP gauge performance on complex comprehension and commonsense reasoning. For the medical domain, we assess capabilities using MMLU-pm -pr (Health), SuperGPQA (Health), Humanity's 's Last Exam (Biology Medicine). HealthBench (Consensus) (Arora al.) (022). PubMedQA (Jin al.) 2019), and MedQA (Koh & Liang, 2017). Our evaluation adopts 5-shot for MMLU-pro and one- shot for PubMedQA and MedQA, with answer Accuracy as the primary metric. For benchmarks HIE and HealthBench that require an LLMM-as-Judge, we use GPT-4.1 as the judge.
Baselines the humanities and social sciences, our baselines comprise Owen3-8B-Insteee (with- out additional fine-tuning) and models fine-tuned eight SFT datasets. The datasets span three cat- egories: (i) human-authored-
semi-automatically Chihhhhuhhrrr--WiChhhaa (Zhao et al. (022) and OpenHermes (Teknium) 2023); (ii) (Hao et al., 2025), WebR Pro (Jiang et al.) 2025
Bonito (Nayak et al., (024),, and SynthQuestions (Zhu et al.) 2025); and (ii) fully automated synthetic Conder (Cao al. 2025). In the medical and health domain,
basclines include
Qwen3-8BB Swee3-8B--nnstenn Instruct and models fine-tuned on five SFT datasets: the wallworld clinical dialogue
corpus ChatDoctor (Li et al. 2023), the multiple-choice dataset MedQA (Singhal al. 2025

---

Published as a conference paper at ICLR 2026
Hlmuniterr and Social Sciences
Medocal and Heaalle
(t) Qwen3-4B
(b) Qheen 3-8
(c) Queen31148 148
(d) Llama3-Ne 8b
Figure 4: The performance our framework across different target model scales and families.
Dataset
Total samples
Token mean
Token std
MTLD
HDD
LongWriter
6,000
1823.45
452.20
80.63
0.8954
WildChat
529,428
289.59
344.77
52.70
0.9188
Condor
20,000
428.79
35.70
101.48
0.8650
Cosmopedia
50,000
773.85
44.94
111.52
0.8843
Openhermes 2.5
1,001,551
236.14
249.04
106.43
0.8887
SynthQuestions
2,500
634,60
17.88
137.0
0.8584
OptimSyn (Ours)
25,875
196.49
34.69
133.82
0.9241
Table 2: Statistics of different datasets (Medical).
and three LLM-distilled medical reasoning datasets Medical-ol (Chen et al. 22024. Medical-RI- Distill (Chen et al. 2024), and ReasonMed (Sun et al., 2025). We conduct a more detailed analysis of the data characteristics of the baseline datasets, such as data scale and diversity. For our pro- posed dataset and the previously used datasets, we report their statistical properties, including the number of samples, the mean and standard deviation input/output lengths, as well as the lexical diversity of inputs and outputs (measured by MTLD and HDD), shown in the Table 2 A detailed description of the data characteristics the baseline dataset can be found in Appendix D.2
Implementation Details For influence estimation, first warm up the target model using 10% of the synthetic data generated by the initialized prompter and generator. The resulting model pro- vides reference gradients for computing influence scores, which are subsequently incorporated into the RI stage. For reinforcement learning, we adopt GRPO (Guo et al., 2025) as the optimization algorithm, where the reward function integrates the influence score with a format-consistency com- ponent. Training is performed with batch size of 256, learning rate of 1x 10 rollout temperature 1.5, rollout size n 5, and one epoch updates. À is set to 0. Processing approximately 20K samples takes about 10 hours. All experiments conducted 8x H200 GPUs. Additional imple- mentation details are provided the Appendix D.4
3.2 MAIN RESULTS
Superiority over Human-Annotation and Synthetic-Date Baselines. We present results for the two domains in Table D Our rubric-guided synthesis consistently enhances the target base model across both Humanities & Social Sciences and Medical domains, surpassing strong open SFT base- exceeds Deliberate-reasonee
lines and its instruct model. On HSS tasks, it rivals achieving a +27.2% relative gain on HLE (0.0785 vs. 0.0570), thereby demonstrating that structured, group-aware data synthesis can effectively distill reasoning ability withou test-time reasoning. In
teachers,

---

Published as a conference paper at ICLR 2026
BQver Beleeeeenn
NHLLE
Suges CPQA
Morlls Sears
Medga
NINLE
GPQA
BBLE
BM1
Bellongg
(a) Performance in med domain.
(c) Performance in HSS domain.
(b) IF distribution in med domain.
(d) IF distribution HSS domain.
Figure 5: The performance of our framework across different generators.
the medical domain, our method narrows the gap on MMLU_PRO and SUPER_GPQA while deci- sively outperforming baselines on HLE and HEALTHBENCH. The most significant improvements appear on tasks aligned with domain rubrics, underscoring the strength of our prompter-centric RL framework in producing high-quality, domain-grounded supervision. Overall, the approach consis- tently upgrades the same 8B backbone beyond widely used SFT corpora, often matching or surpass- ing inference-time "thinking" models on reasoning-cente metrics.
Generalization across Target Model Scales and Families. To test whether our training recipe is target-agnostic, we replace the backbone with Qwen3-(4B, 8B, 14B) and Llama3-8B and evaluate on two domain suites-/lumanitee & Social Sciences and Medical & Health, with results shown in Figure 4 Across all four backbones, the plots exhibit consistent improvements for Ours than for the Base and Instruct variants on most tasks within both domain suites. The gains persist when moving from small to larger models (4B - 14B), while remaining substantial at the smallest scale, suggesting that method does not rely on model capacity to be effective. Moreover, the improve- ments transfer from the Qwen3 family to Llama3 with comparable margins, despite architectural and pretraining differences. Taken together, these results demonstrate that the proposed method confers family- and scale-robust benefits, supporting the claim that our method yields broadly applicable performance gains for diverse target LLMs.
Robustness to the Choice of Generator. We ablate the synthetic-da data generator by swapping Qwen3-235B, 235B, 1, and Gemini-2.5-PPr while keeping the target model fixed. The results are provided in Figure 5a and 5c] Across both domains-Medicall & Health and Humanities & Social Sciences-model. trained with our method consistently outperform the base model on nearly all benchmarks, indicating that the gains are not tied to any single generator.
3.3 ANALYSIS
Consistent Enhancement of the Influence Distribution in Synthetic Data. We further visualize the distribution of influence scores produced by our method, as shown in Figure 5d and Figure 5b) The left panel shows that our approach aligns the distribution of synthetic data influence scores more closely with that of the validation data. The right panel demonstrates that, even when substituting different generators,
method consistently shifts the distribution toward higher average influence scores. Taken together, these results indicate that the proposed reward signal effectively selects and amplifies high-utility synthetic examples in generator-agnostt fashion.
8

---

Published as a conference paper at ICLR 2026
domain relevance
Contex window
Clarity
TO UCS
Soundness
Correctnes Type cons is tercyyyyy
ensures
short
reflect L
comp letee Leteness ture
Por sags Lity
cricalll
for Etoon Eson density:
domain relevance
(a) Distributions of the embeddings of rubrics.
(b) Word cloud of rubrics (c) Word cloud of rubrics (d) New feehffreyency without our method.
with our method.
terms in Figure bc
Figure 6: Effects of the Proposed Method on Rubrics
Effeets of the Proposed Method on Rubries. We examine how the prompter-centric RL reshapes the learned rubrics. Figure visualizes vigualiiassssulizess t-SNE embeddings of rubric texts in the generator space be- fore and after RL: R..t-Rs occupy a broader, more structured region, whereas pre-RL rubrics cluster in narrow band, indicating increased coverage and diversity aligned with the seed data dis- tribution. From our case analyses, we observe that the post-RL rubrics are more tightly grounded in the source documents, which in turn explains their broader coverage and clearer structure in the embedding space. We hypothesize that such more detailed rubrics better steer the generator to synthesize data that conforms the specification and task requirements. The word clouds in Fig- ure 6c reveal a qualitative shift from generic imperatives (e.g., focus, align, short) to domain- and quality- attributes (e.g., critical, clarity, completeness, patpattnnssssppaablliii.. information density, logical soundness). Figure 6d further quantifies these changes by highlighting newly introduced terms and their normalized frequency deltas, confirming that our method promotes specific, actionable criteria rather than vague instructions. Detailed case studies of the rubrics and question-anse answer pairs are provided in Appendix E.S and E.9
Predictive Relationship Between Influence Scores and Downstream Accuracy. We investigate whether per-example Influence Scores (IF), computed on the synthetic training pool, are predic- tive of downstream supervised fine-tuning (SFT) performance. Specifically, we repeatedly sample random subsets of size 2K or 4K from the generated synthetic dataset, compute the aggregate IF of each subset, and fine-tune the model exclusively on that subset. Figure shows that higher-IF subsets consistently yield higher test accuracy under both training budgets. Quadratic regressions (dashed curves) summarize the trend with strong goodness of fit (R²2 = 0.57 for 2K and R²² = 0.54 for 4K), revealing a clear positive correlation with mild saturation the high-IF end. These re- sults demonstrate that IF serves as reliable proxy for the utility of synthetic examples, and that prioritizing higher-IF data can improve downstream accuracy even with limited training budgets.
Effects of the Rubries on Synthetic Data. We ablate the number of rubric rollouts per seed, G. to assess how exploration affects synthetic data quality. Figure 8 shows RL training curves for G {5, 10, 15] larger G reaches higher reward and exhibits reduced variance, suggesting that broader rubric exploration stabilizes IF-driven optimization. Figure reports downstream accuracy on multiple benchmarks: increasing consistently improves performance. Together, these results support that diverse, targeted amplify the utility of synthetic data.
by higher rollout counts-bott stabilize training and
4 RELATED WORKS
As LLMs improve, synthetic data is increasingly used to offset annotation costs for in- struction tuning. A first line of work converts readily available document corpora into SFT-style dialogue data: WebR (Jiang et al.) (205) casts webpages as dual-view recon- struction (web-as-instruction / web-as-response with ciidence-oostrramee refinement), enabling seedless, label-free instruction-tuning corpora; MH2 (Yue et al., 2024) scales via web-recall -Rl - cetemiintttion-tfnneeeeee -refinement pipeline; Bonito maps unlabeled text plus task attributes to instruction via meta-template conditional generators, supporting
-decontamination

---

Published as a conference paper at ICLR 2026
Accurary: 2K dk samples
6.54
+G-13
63-
6.1
Rovnnd
0:35
18
15
Figure 7: Influence score vs. accu- racy.
Figure 8: Accuracy across
Figure 9: Training curves
different group sizes.
across different group sizes.
zero-shot cross-task/domain adaptation. A complementary line focuses on iteratively improving the synthesized data: Evol-Instruct (Xu et al. 2024) evolves seed instructions to increase dif- ficulty and produce matched responses; Condor (Cao al. 2025) organizes a World Knowl- edge Tree and applies self-reflection refinement knowledge-driven, self-reflective pipeline; (Li et al. (024a) employs indlurign exploration by feeding student-model failures back to the teacher to synthesize targeted hard examples; and Montessori-Instruuc (Li et al. 2024b) contrasts beneficial vs. non-beneficial samples and uses Direct Preference Optimization to bias the teacher toward the most instructive data.
in two-stage
Compared to prior work, our approach (i) replaces heuristic filters with a training- siinn--li-llllneeee objective that directly optimizes downstream improvement; (ii) learns rubrics from model feedback rather than expert priors, enabling transfer across domains without bespoke rule engineering: and (iii) leverages influence-based analysis reveal the mismatch between embedding similarity and training influence, clarifying failure modes of earlier pipelines and guiding principled corrections. By elevating the role of the target model's own training signal in the synthesis loop, our approach provides principled route to scalable SFT in data-scarce, high-stakes domains. It turns rubric design from brittle, expert-driven craft into a learnable component that aligns synthetic data gen- eration with what actually improves the model.
5 CONCLUSION
We introduced a synthesis-training framework that replaces heuristic rubric design with model-impact supervision. its core is an optimizer-aware gradient-based influence estimator that quantifies each synthetic QA pair's contribution to the target model's 's objective, revealing a marked mismatch between embedding similarity and training impact and motivating gradient-aligned selec- tion. Leveraging this signal, we cast rubric construction as a learnable policy within prompter- centric RL loop, optimized with GRPO objectives under verifiable rewards that blend lightweight validity checks with influence scores. This closes the synthesis-training feedback cycle and grounds rubric learning in downstream impact rather than surface heuristics. Empirically, our approach con- sistently improves performance across domains, model families, and generators, rivaling surpass- ing standard SFT baselines. Influence scores correlate strongly with realized accuracy under fixed budgets, validating them as reliable proxy for synthetic data quality and highlighting the portability of our prompter-centric optimization framework.
6 LIMITATIONS
Our prompter is optimized via reinforcement learning using scalar rewards computed after a sepa- rate generator synthesizes QA pairs conditioned on the sampled rubrics. This introduces an indirect predit-assignmed path from the modget-model feedback to the prompter: the reward reflects both the generator's 's stochasticity and the downstream influence estimation, but gradients cannot propa- gated through the generation step. As result, policy-gradient updates (e.g., GRPO) can exhibit high variance and training instability, especially when the rollout group size is small.
10

---

Published as a conference paper at ICLR 2026
AKNOWLEDGEMENT
This work is supported by the National Key R&D Program China (Grant No. 203404C)30 the "Pioneer" and "Leading Goose" R&D Program Zhejiang (Grant No. 2025C01 128), the Na- tional Natural Science Foundation of China (Grant No. 62476241), and the Natural Science Foun- dation of Zhejiang Province, China (Grant No. LZ23F020008).
ETHICS STATEMENT
All authors attest that they have read and will abide by the ICLR Code of Ethies. Our study develops an afflence-alignee prompter-centrie framework for rubric-guided synthetic data generation and evaluates it on knowledge-intensin domains. Below we summarize ethical considerations specific to this work.
Human subjects and IRB. This work does not involve human participants, user studies, or the collection of personal data. No IRB approval was required.
Data sources, licensing, and privacy. Seed documents are drawn from publicly available sources in the Humanities Social Sciences and Medical domains; preprocessing removes boilerplate and noisy content, and the generator is instructed to paraphrase rather than reproduce lengthy excerpts verbatim. We release only prompts/rubrics and synthesized QA pairs, not the original documents. We screen out personally identifiable information (PII) and sensitive attributes to the best of our ability and respect the licenses and terms of of all sources. We will honor takedown requests for inadvertently included material.
Safety, misuse, and domain constraints. Although our method targets data quality, synthetic outputs in medical other high-stakes settings may be misinterpreted as professional advice. Our models and datasets are research artifacts and not intended for deployment in clinical, legal, fi- nancial, or safety-critical contexts. The rubric pipeline includes format/safety checks (e.g., refusal of unsafe instructions) and excludes harmful topics during synthesis. We recommend downstream users apply additional content filters and human review.
Use of external APIs and compliance. When large closed models are used as generators or judges, all interactions comply with provider terms of service; no user data or proprictary content is transmitted beyond the required prompts. We report which systems used and how (teacher, judge) in the paper.
Conflicts of interest and sponsorship. The authors declare conflicts of interest related this submission. No external sponsor influenced the problem formulation, experiments, or reporting.

---

Published as a conference paper at ICLR 2026
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Iige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint (rXii:2303.08772 2023.
Kingma DP Ba Adam et al. A method for stochastic optimization. arXiv preprint arXiv: 14)2(6), 2014.
Rahul K Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quiñonero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, et al. Health- bench: Evaluating large language models towards improved human health. arXiv preprint arXiv:2505.08775, 2025.
Maosong Cao, Taolin Zhang, Mo Li, Chuyu Zhang, Yunxin Liu, Haodong Duan, Songyang Zhang, and Kai Chen. Condor: Enhance IIm alignment with knowledge-driver data synthesis and refine- ment. arXiv preprint 2.Xii:2501.122: 2025.
Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang. Jianye Hou, and Benyou Wang. Huatuogpt-ol, towards medical complex reasoning with Ilms, 2024. URL
https://arxiss wwtp:///////////....////
httpo://arxii.orrr/aa///22...
Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, et al. Meditron-70b: Scaling medical pretraining for large language models. arXiv preprint arXiv:2311.16079, 2023.
Gheorghe Comanici, Eric Bieber, Mike Schackermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.0626/ 2025.
Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, King Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, et al. Supergpqa: Scaling Ilm evaluation across 285 graduate disciplines. arXiv preprint arXiv:2502. 14739, 2025.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The llama 3 herd of models. arXiv e-prints, pp. arXiv 2407, 2024.
Yuxian Gu, Li Dong, Hongning Wang, Yaru Hao, Qingxiu Dong, Furu Wei, and Minlie Huang. Data selection via optimal control for language models. arXiv preprint arXiv:2410.07066 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, arXiv preprint arXiv:2501.12944 2025.
Xintong Hao, Ke Shen, and Chenggang Li. Maga: Massive genre-audience reformulation to pre- training corpus expansion. arXiv e-prints, pp. arXiv-2502, 2025.
Yuxin Jiang, Yufei Wang, Chuhan Wu, Xinyi Dai, Yan Xu, Weinan Gan, Yasheng Wang. Xin Jiang, Lifeng Shang, Ruiming Tang, et al. Instruction-tuning data synthesis from scratch via web recon- struction. arXiv preprint arXiv:2504.1555
2025.
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. Pubmedqa: A dataset for biomedical research question answering. arXiv preprint 17Xiv:1909006146 2019.
Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, San- ket Vaibhav Mehta, Lalit K Jain, Virginia Aglictti, Disha Jindal, Peter Chen, et al. Big-bench extra hard. arXiv preprint arXiv:2502.19180 2025.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1885-1894, PMLR, 2017.
12

---

Published as a conference paper at ICLR 2026
Qintong Li, Jiahui Gao, Sheng Wang, Renjie Pi, Xueliang Zhao, Chuan Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Forewarndd forearmed: Leveraging Iims for data synthesis through failure-ing -indurig exploration. arXiv preprint arXiv:2410.16730 2024a.
Xiaochuan Li, Zichun Yu, and Chenyan Xiong. Montessori-insttrutt: Generate influential training data tailored for student learning. arXiv preprint arXiv:2410. 00044100.4205 14208, 2024b.
Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. Chatdoctor: A medical chat model fine-tuned on large language model meta-ai (Ilama) using medical domain knowledge. Cureus, 15(6). 2023,
Kiwan Maeng, Alexei Colin, and Brandon Lucia. Alpaca: Intermittent execution without check- points. Proceedings of the ACM on Programming Languages, 1(00PSLLA))-30 2017.
Nihal V Nayak, Yiyang Nan, Avi Trost, and Stephen H Bach. Learning to generate instruction tuning datasets for zero-shot task adaptation. arXiv preprint 24Xi:2402218334,, 2024.
Guilherme Penedo, Hynek Kydlfček, Anton Lozhkov, Margaret Mitchell, Colin A Raffel, Leandro Von Werra, Thomas Wolf, et al. The fineweb datasets: Decanting the web for the finest text data at scale, Advances Neural Information Processing Systems, 37:308 1 1-30849, 2024.
Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4, arXiv preprint arXiv:2304.03277, 2023.
Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al. Humanity's 's last exam. arXiv preprint UrXiv:2501.14249 2025.
Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Advances in Neural Information Processing Systems, 33: 19920-19930, 2020.
Kashun Shum, Yuzhen Huang, Hongjian Zou, Qi Ding, Yixuan Liao, Xiaoxin Chen, Qian Liu, and Junxian He. Predictive data selection: The data that predicts is the data that teaches. arXiv preprint arXiv:2503.0003 2025.
Karan Singhal, Tao Tu, Juraj Goutwis, Rory Sayres, Ellery Wulezyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, et al. Toward expert-level medical question answering with large language models. Nature Medicine, 31(3):943-99 950, 2025.
Yu Sun, Xingyu Qian, Weiwen Xu, Hao Zhang, Chenghao Xiao, Long Li, Yu Rong, Wenbing Huang, Qifeng Bai, and Tingyang Xu. Reasonmed: A 370k multi-agent generated dataset for advancing medical reasoning. urXiv preprint arXiv:2506.099 09513, 2025.
Teknium. Openhermes 2.5: An open dataset of synthetic data for generalist IIm assistants, 2023.
URL httpttttt///hgggitt face co/ dat aatttseesss......... wtt.sstttttettteeeeeeeeeeeeeeeee wtwp.riiii//eeeeeeeeeeeeeeee
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannanch Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint a2 /2. 10560, 2022.
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi- task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266-9522 2024.
Alexander Wettig, Aatmik Gupta, Saumya Malik, and Danqi Chen. Qurating: Selecting high-quality data for training language models. arXiv preprint arXiv:2402.09733) 2024.
Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. Less: Se- lecting influential data for targeted instruction tuning. arXiv preprint arXiv:2402.0433 2024.
Sang Michael Xie, Shibani Santurkar, Tengyu Ma, and Percy S Liang. Data selection for language models via importance resampling. Advances in Neural Information Processing Systems, 36: 34201-34227, 2023.
13

---

Published as a conference paper at ICLR 2026
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. Wizardlm: Empowering large pre-trained language models to follow com- plex instructions. In The Twelfth International Conference Learning Representations, 2024.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint 05t::2505500388, 2025.
Zichun Yu, Spandan Das, and Chenyan Xiong. Mates: Model-aware data selection for efficient pretraining with data influence models. Advances in Neural Information Processing Systems, 37: 108735-108759, 2024.
Xiang Yue, Tianyu Zheng, Ge Zhang. and Wenhu Chen. Mammoth2: Scaling instructions from the web. Advances in Neural Information Processing Systems, 37:90629 906600, 2024.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can ma- chine really finish your sentence? arXiv preprint arXiv:1905.078300 2019.
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: Im chatgpt interaction logs in the wild. arXiv preprint arXiv:2405.01470. 2024.
Chiwei Zhu, Benfeng Xu, Xiaorui Wang, and Zhendong Mao. From real to synthetic: Synthesizing millions of diversified and complicated user instructions with attributed grounding. arXiv preprint arXiv:2506.03368, 2025.
Xinlin Zhuang, Jiahui Peng, Ren Ma, Yinfan Wang, Tianyi Bai, Xingjian Wei, Qiu Jiantao, Chi Zhang, Ying Qian, and Conghui He. Meta-rater: A dineessoaman data selection method for pre- pre-trainmn training language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume Long Pupers), pp. 18856108966 2025.
14

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS
In the preparation of this manuscript, GPT-5 was employed solely for language refinement purposes, including grammar checking and stylistic polishing. All conceptualization, methodological design, experiments, analyses, and interpretations were conducted by the authors. The LLM was not used to generate novel content, ideas, or results, but only to assist in improving clarity, readability, and consistency of expression. Specifically includes: INTRODUCTION; 2 METHOD; 4 RELATED WORKS; CONCLUSION.
B INFLUENCE ESTIMATION
Influence estimation Koh & Liang 2017); Pruthi et al. (2020) aim to quantify how an individual training example perturbs both the learned parameters and the loss target (validation/test) point. In the large- language model (LLM) regime, computing exact influences is prohibitively expensive; consequently, prior work Pruthi et al. (2020); Xia et al. (2024) adopts a first-order approximation to the training dynamics to estimate the influence of training datapoint on held-out performance.
Per-step influence. Let H denote the model parameters at step and E(:; 0*) the loss. A first-ordrr Taylor expansion of the loss on a validation example z' gives
f(z'; 01+) (()':0') + (Vol(z':0") f 8¹)
(4)
Assuming SGD with batch size and learning rate The if z is used at step then gc+1 Hr -Mc Vol(2;0'). and equation yields the per-step influence
(l+1) ((2';0") = (Vol(z;0", Vol(z';0')
(5)
(For mini-batches, replace V Vol(z; 0+) with the batch gradient.)
Trajectory influence (SGD). Aggregating equation over epochs results in trajectory-level score:
(2fsCD(Z,2)) Σ Th (Vol(z';0)) vollx.))
where îJ the average learning rate in epoch (out N total epochs) and 0, is the checkpoint after epoch i.
Extension to Adam. Instruction tuning typically uses Adam (Adam et al. 2014 Let the per-coordinate update direction be
8+1
T(2, 8*)
gt+1 - gt = - -
(7)
V het1 +e
with elementwise operations and
m 2+1 = Bmmm + (1 -1- (1- vol(z;8'),
++ 1 = B2 + (1 - -2) Vol(=; 00)2
m i+1 m 1+1 /11 3(+1),
1+ o' (1-1
A fist-oorer expansion analogous to equation <n<(Vo(=z (")) [(=, 0')) Because the magnitude of varies with the adaptive moments, use scale-robust cosine similarity when accumulating across epochs, and define
gives the stepwise effect
Inf Adam (2,z)
Σ The cos (Vol(= r(2,00)
(8)
Computing I' requires the moment statistics (m, v). which depend on past gradients; following our procedure in 1.3,,3, we obtain them from short warmup before scoring.
15

---

Published as a conference paper at ICLR 2026
C RELATED WORKS: SYNTHETIC DATA GENERATION
With the continued improvement of LLM capabilities, synthetic data has been widely used to allevi- ate the high annotation costs in instruction tuning and dialogue scenarios, many studies convert read- ily available document corpora into SFT-style dialogue samples using LLMs. WebR (Jiang et al.) 2025) treats web pages dual-view reconstruction target: first, page content is converted into structured instructions and corresponding responses; second, the page is regarded as the response to infer latent instructions, after which the initial responses are refined under evidence constraints. This enables the construction of high-quality instruction-tuning corpora without human intervention or seed data. MHmmoTH2 (Yue et al.] (024) proposes scalable pipeline comprising web recall, traction of instruction-rrespnse pairs, decontamination, and refinement. Bonito (Nayak et al. 2024) maps unlabeled text and task attributes into instruction-response samples via meta-template- driven conditional task generators, turning domain documents into trainable data and enabling zero-shot cross-task and dross-domain adaptation. A complementary line of work focuses on iteratively im- proving the synthesized data. Evol-Instruct (Xu et al., 2024) evolves seed instructions in depth and breadth to progressively increase task difficulty and generate matched responses, illustrating the po- tential of self-evolving synthesis. Condor (Cao et al. (025) constructs a World Knowledge Tree to ensure topic coverage and difficulty stratification and introduces a self-reflection refinement mecha- nism that conducts structured self-evaluation of candidate answers along strengths, weaknesses, and suggestions before rewriting, yielding a two-stage knowledge-dris driven and self-reflective synthesis framework. Li et al. (2024a) adopts failure-inducing exploration by feeding cases that the student model still fails on back to the teacher to synthesize targeted hard examples. Montessori-Instrun (Li et al. 2024b) forms pairs of beneficial and non-beneficial data and applies Direct Preference Opti- mization to train the teacher's preference toward examples that are most instructive for downstream performance.
Current data selection strategies can be broadly divided into two categories:(1) Static criteria (human-authored rubrics LLMM-a-Judge): Emphasize interpretability and control. Representa- tive approaches include: QuRating (Wettig et al. 2024). which uses an LLM to score texts along dimensions such as quality, expertise, factualityknowwlddee, and educational value to guide sample selection; DSIR (Xie et al.) (023), which applies importance resampling to large-scale pretraining subset selection; and Wib (Penedo et al., 2024), which performs large-scale deduplication and filtering and uses distributional visualizations and ablations to validate how data quality affects per- formance, thereby providing a tooling stack for curating high-quality general-purpose purpose corpora. (2) Dynamic feedback structive for the model. Representative approaches include: LESS (Xia et al. 2024), which selects instruction samples via influence estimates and low-rank gradient similarity; PreSelect (Shum al.) 2025), which uses predictability as lightweight scoring function; PDS (Gu et al. 2024). which for- mulates data selection and MATES (Yu et al. 2024), which learns a data-influence model that adapts over training to match the most effective samples for the current stage.
poodell/task-drivenn Directly selects samples using signals for what is most in-
optimal-control problem to characterize selection-traiie training dynamics;
D EXPERIMENT DETAILS
D. DECONTAMINATIN PROTOCOL.
We follow the official AllenAI T'ulu decontamination toolkit, implemented on Elasticsearch, and use three complementary matching strategies to mitigate train-test leakage. In text mode (without n- grams), we issue strict match-phrase queries over the validation index (used for influence-function computation) and flag any example that contains an identical contiguous span to the evaluation prompt, thus capturing verbatim duplicates. In n-gram mode, we tokenize each evaluation instance, construct 5-grams, retrieve validation documents containing these 5-grams, and compute an over- lap score as the fraction of query tokens covered by matched n-grams; validation examples whose maximum overlap with any test item exceeds a threshold are removed, which detects high-lexical- overlap near duplicates. pre-trained embedding model (NV- ONV-Embed-v2) -22,, perform kNN search over the stored embeddings using Elasticsearch's 's vector index, and treat the highest similarity to any test item as a semantic contamination score, removing validation examples above similarity threshold; this step identifies
vector mode, we encode both the validation and evaluation texts with
16

---

Published as a conference paper at ICLR 2026
Method
MMI.Uprr
Super GPQA
HLE
PubMed
MedQA
Health Bench
Text-match
0.000
0.000
0.000
0.000
0,000
0.000
5-gram (0.3)
0.002
0.004
0.000
0.005
0.009
0.001
Embedding (0.85)
0.000
0.001
0.000
0.001
0.009
0.002
Table 3: Estimated contamination scores medical and health benchmarks under different decon- lamination modes.
Method
MMLU-pro
Super GPQA
HLE
BBH
HellaSwag
DROP
Text-match
0.000
0.000
0.000
0.000
0.000
0.000
5-gram (0.3)
0.006
0.004
0.001
0.000
0.000
0.000
Embedding (0.85)
0.003
0.001
0.000
0.000
0.000
0.000
Table 4: Estimated contamination scores on general reasoning benchmarks under different decon- tamination modes.
paraphrased or semantically equivalent variants that purely lexical methods may miss. The detailed results are shown in Table Table
D.2 BASELINE ANALYSIS
We conduct more detailed analysis of the data characteristics of the baseline datasets, such as data scale and diversity. For our proposed dataset and the previously used datasets, we report their statis- tical properties, including the number of samples, the mean and standard deviation of input/output lengths, as well as the lexical diversity of inputs and outputs (measured by MTLD and HDD), as shown in the table below. As can be seen, existing SFT datasets already cover very wide range in terms of these statistics. Compared with other datasets, the size, average length, and diversity metries of OptimSyn all lie within this existing range. other words, OptimSyn is better viewed as more "efficciett" data mixture obtained by re-organizing and filtering samples within the existing statistical range, rather than simply relying on "more data" or "longer texts". More importantly, when we relate these statistics to the downstream evaluation results (Table we do not observe monotonic trend that "larger scale longer length leads to better performance": under the same tar- get model and training configuration, some datasets with larger size or longer outputs do not achieve the best performance, whereas several moderately sized but higher-quality corpora yield more sta- ble results instead. This suggests that basic corpus statistics alone are insufficient explain the performance gains brought by our method.
D.3 SOURCE OF TRAINING DATASET
Our seed documents for synthesizing data in the humanities and social sciences were collected from books. The list of book titles is as follows:
A Black Jurist in a Slave Society Antonio Pereira Rebouças and the Trials of Brazilian Citizen- ship; A History of Intellectual Property in 50 Objects; A History of Modern Africa 1800 to the Present Wiley Blackwell Concise History of the Modern World; A Taste of Irrationality Sample Chapters From Predictably Irrational and Upside of Irrationality; A Case for Sometimes Tube- feeding Patients in Persistent Vegetative State; Adult Attachment A Concise Introduction to Theory and Research; Advanced Programming in the UNIX Environment 3rd Edition; Advanced Signal Integrity for Highspeed Digital Designs; Advocating for Self Womens Decisions Concerning Con- traception; African History A Very Short Introduction; Algorithm Design Monograph; American Criminal Justice Policy An Evaluation Approach to Increasing Accountability and Effectiveness; American Grace How Religion Divides and Unites Us; An Analysis of Ernest Gellners Nations and Nationalism; An Analysis of Marcel Mauss The Gift The Form and Reason for Exchange in Archaic Societies; An Analysis of Robert A Dahls Who Governs Democracy and Power in an American City; An Analysis of Robert E Lucas Jrs Why Doesnt Capital Flow from Rich to Poor Countries; An Organizers Tale Speeches; Assessments in Forensic Practice A Handbook; Bargaining with the
17

---

Published as a conference paper at ICLR 2026
Devil When to Negotiate When to Fight; Basic Political Writings: Bertrand Russell and the Na- ture of Propositions A History and Defence of the Multiple Relation Theory of Judgement; Beyond Reason Using Emotions You Negotiate: Blackness in Britain: Challenging Behaviour in Demen- tia A Personcentred Approach; Changing Minds The Art and Science of Changing Our Own and Other Peoples Minds; Child Soldiers A Reference Handbook; Children and Play Understanding Childrens Worlds; Cite Them Right The Essential Referencing Guide 12th Edition; Cognitive Be- havioral Therapy for PTSD A Case Formulation Approach; Cognitive Behavioral Treatment for Generalized Anxiety Disorder From Science to Practice; Collaborative Intelligence Using Teams to Solve Hard Problems; Collective Action and Exchange A Gametheoretic Approach to Contempo- rary Political Economy; Companion Encyclopedia of Medicine in the Twentieth Century; Computer Architecture A Quantitative Approach; Concise Guide to APA Style The Official APA Style Guide for Students; Confronting The Internets Dark Side Moral And Social Responsibility On The Free Highway; Contemporary Debates in Cognitive Science Contemporary Debates in Philosophy; Cur- ing Their IIIs Colonial Power and African Illness; Dependency and Development in Latin America; Descriptive Physical Oceanography Sixth Edition An Introduction; Development Microeconomics; Dignity and Daily Bread New Forms of Economic Organizing among Poor Women in the Third World and the First; Divine Hiddenness and Human Reason; Economics and Policy Issues in Cli- mate Change; Economics of Regulation and Antitrust Fifth Edition The MIT Press; Effective Treat- ments for PTSD Third Edition Practice Guidelines from the International Society for Traumatic Stress Studies; European Legal History A Cultural and Political Perspective; Euthanasia Ethics and the Law from Conflict to Compromise; Exchange Rate Regimes Fixed Flexible or Something in Between; Experiences of Depression A Suudy Phenomenology; Factfulness Ten Reasons Were Wrong About the World and Why Things Are Better Than You Think; Fatal Invention How Science Politics and Big Business Recreate Race in the Twentyfirst Century; Feeding the World An Eco- nomic History of Agriculture 1 1800-2000; Feminism Literature and Rape Narratives Violence and Violation; Feminism Unmodified Discourses on Life and Law; Feminist Judgments From Theory to Practice; Fiasco The American Military Adventure in Iraq 2003 to 2005; Formulation in Psy- chology and Psychotherapy Making Sense of Peoples Problems; Free Innovation; From the Great Recession Labour Market Recovery Issues Evidence and Policy Options; Fandumentals of De- velopmental Psychology; Geographies of Development An Introduction to Development Studies; Getting It Wrong How Canadians Forgot Their Pust and Imperilled Confederation; HBRs 10 Must Reads on Managing Strategy; Hackers Delight; Helping People Win at Work A Business Philosophy Called Dont Mark by Paper Help Me Get an A; Her Place at the Table A Womans Guide to Nego- tiating Five Key Challenges to Leadership Success; Hindu Worldviews Theories of Self Ritual and Realit;; How To Do Your Research Project 3rd Edition; In Command of History Churchill Fighting and Writing the Second World War; Infertility and Patriurchy The Cultural Politics of Gender and Family Life in Egypt; Influence New and Expanded The Psychology of Persuasion; Innovation and Entrepreneurship Practice and Principles: Institutions Institutional Change and Economic Perfor- mance; Introduction to Econometrics Global Edition; Introduction to Meta-Analysis; Introductory Econometrics A Modern Approach MindTap Course List; Islam and Global Dialogue Religious Pluralism and the Pursuit of Peace; John P Kotter on What Leaders Really Do; Judgment How Winning Leaders Make Great Calls; Just Health Meeting Health Needs Fairly; Key Concepts in Geography; Law Legitimacy and the Rationing of Health Care A Contextual and Comparative Per- spective; Leadership on the Line Staying Alive Through the Dangers of Leading: Lean In Women Work and the Will to Lead: Leviathan and the Air Pump Hobbes Boyle and the Experimental Life; Linear Algebra Done Right; Macleods Clinical Examination Ebook; Making Babies Is There a Right to Have Children; Making Democracy Work Civic Traditions in Modern Ital;; Manias Panics and Crashes A History of Financial Crises; Marshall and the Marshallian Heritage Essays in Honour of Tiziano Raffaelli; Mastering Leadership A Vital Resource for Health Care Organizations: Math with Bad Drawings Illuminating the Ideas That Shape Our Reality; Microeconomics 9th Global Edition; Modeling Foundations of Economic Property Rights Theory An Axiomatic Analysis of Eco- nomic Agreements Studies in Economic Theory 23; Modern Classics Making of the English Working Class Penguin Modern Classics; Modern Environmentalism An Introduction; Money Well Spent A Strategic Plan for Smart Philanthropy; Mostly Harmless Econometrics; National Insecurity Amer- ican Leadership in an Age of Fear; New Museum Theory and Practice An Introduction; Notes on the Theory of Choice; Oil Palm A Global History; One Economics Many Recipes Globalization Institutions and Economic Growth; Operating Systems In Depth Design and Programming: Ordi- nary and Partial Differential Equations With Special Functions Fourier Series and Boundary Value Problems; Organizational Trust A Reader; Paul Samuelson on the History of Economic Analysis Se-
18

---

Published as a conference paper at ICLR 2026
lected Essays; Personality and Intelligence at Work Exploring and Explaining Individual Differences at Work; Philosophical Issues in Psychiatry Explanation Phenomenology and Nosology: Philosophy of Psychology Contemporary Readings: Presidential Leadership and the Creation of the American Era; Psychiatry as Cognitive Neuroscience Philosophical Perspectives; Psychological Assessment and T'herapy with Older Adults; Psychology The Science of Mind Behaviour; Publication Manual of the American Psychological Association The Official Guide to APA Style; Raising Cando Kids Giving Children the Tools to Thrive in a Fastchanging World; Randomized Algorithms; Real Es- tate Development Principles and Process; Reconstructing Educational Psychology; Religion and Politics in the United States; Religion as Make Believe A Theory of Belief Imagination and Group Identity; Religion Violence Memory and Place; Reseurch Methods and Statistics in Psychology 7th Edition; Rethinking Informed Consent in Bioethics; Reveille for Radicals; Rewriting the Soul Mul- tiple Personality and the Sciences of Memory; Risk Ambiguity and Decision; Rule of Experts Egypt Technopolitics Modernity; Science and the Practice of Medicine in the Nineteenth Century; Seeing Like State How Certain Schemes to Improve the Human Condition Have Failed; Selfcare Science Nursing Theory and Evidencebased Practice; Senior Leadership Teams What It Takes to Make Them Great: Slave Country American Expansion and the Origins of the Deep South; Small Differences That Matter Labor Markets and Income Maintenance in Canada and the United States; Small States in World Markets Industrial Policy in Europe; Smart Choices A Practical Guide Making Better Life Decisions; Social Research; State Responsibility The General Part; Strategic Leadership and Management in Nonprofit Organizations Theory and Practice; Strategic Management for Nonprofit Organizations Theory and Cases; Strategy Safari A Guided Tour Through the Wilds of Strategic Management; Studies in Tudor and Stuart Politics and Government Papers and Reviews 1946-1972 Vol I; Supervision in the Helping Professions Supervision in Context; The 5 Elements of Effective Thinking; The American Political Economy Macroeconomics and Electoral Politics; The Cambridge Companion to Shakespeare Cambridge Companions to Literature; The Cambridge Companion to Victorian and Edwardian Theatre Cambridge Companions to Literature; The Cambridge Handbook of Personality Psychology; The Economy of the Word Language History and Economics; The Far Enemy Why Jihad Went Global Second Edition; The Greatest Benefit to Mankind A Medical His- tory of Humanity from Antiquity the Present; The IBS Elimination Diet and Cookbook The Low FODMAP Plan for Fating Well and Feeling Great; The Little Black Book of Neuropsychology A Syn- drome Based Approach; The Lucifer Effect Understanding How Good People Turn Evil; The Moral Economy of the Peasant Rebellion and Subsistence in Southeast Asia; The Mystery of Capital Why Capitalism Triumphs in the West and Fails Everywhere Else; The Politics of Life Itself Biomedicine Power and Subjectivity in the Twenty First Century; The Postcolonial Studies Reader; The Proba- bilistic Method; The Protestant Ethic and the Spirit of Capitalism; The Quest Energy Security and the Remaking of the Modern World; The Routledge Companion to Islamic Philosophy Routledge Philosophy Companions; The Routledge Handbook of Memory Activism; The SAGE Handbook of Social Geographies: The Social History of English Seamen 1485-1649; The United Nations and Changing World Politics Revised and Updated with a New Introduction; The Western Medical Tra- dition 1800-2000 The Wiley Blackwell Handbook of Individual Differences; The Architecture of the Mind Massive Modularity and the Flexibility of Thought; The Balanced Scorecard Translating Strategy into Action; The Blank Slate The Modern Denial of Human Nature; The Emergence of In- dustrial America Strategic Factors in American Economic Growth since 1870; The Female Frontier A Comparative View of Women on the Prairie and the Plains; The Filter Bubble What the Internet is Hiding from You; The Fortune at the Bottom of the Pyramid Eradicating Poverty Through Profits; The Global Interior Mineral Frontiers and American Power; The Missing Peace The Inside Story of the Fight for Middle East Peace; The Political Determinants of Health; The Prize The Epic Quest for Oil Money Power; The Psychological Complex Social Regulation and the Psychology of the In- dividual; The Slave Ship A Human History; To Vote or Not to Vote The Merits and Limits of Rational Choice Theory; Treatment of Obsessive Compulsive Disorder; Unified Growth Theory; Varieties of Capitalism The Institutional Foundations of Comparative Advantage: West from Appomattox The Reconstruction of America After the Civil War; Why David Sometimes Wins Leadership Organiza- tion and Strategy in the California Farm Worker Movement; Why Nations Fail The Origins of Power Prosperity and Poverty; Why We Disagree About Climate Change Understanding Controversy In- action and Opportunity; Wind Energy Renewable Energy and the Environment; Working Bodies Interactive Service Employment and Workplace Identities; Yangzi Waters Transforming the Water Regime of the Jianghan Plain in Late Imperial China China Studies 44; The Courage Create
19

---

Published as a conference paper at ICLR 2026
D.4 IMPLEMENTATION DETAILS
We conduct all experiments on H22200 GPUs. The framework consists of three stages: warm-up, reinforcement learning (RL), and final SFT training.
Warm-up. the beginning of training, the initialized prompter synthesizes a set of rubrics, which guide GENERATOR to produce an initial batch of SFT data. We then warm up the target model qwen3 1iren-88bb -b--aseeeeeeeeeeeeeeeeeeeeeeeeee by fine-tuning it with LoRA adapters and the Adam optimizer, using 10% of the synthetic data with batch size 16, learning rate 1xx110-5 and training for one epoch. The resulting reference signals for influence score estimation. Each training
model provides loss gradients example is assigned an influence score, which is normalized within the batch and used as the reward signal to update the prompter via reinforcement learning (RI.).
Reinforcement Learning, During RL, a generator is deployed on 8xH200 GPUs to synthesize QA pairs conditioned on seed documents and prompter rollouts. Each synthetic instance is scored by comparing the update direction (from Adam on the generated sample) with validation gradients of the target model. The similarity then normalized via min-max scaling to [0, 1], forming the primary reward signal. We adopt GRPO as the training algorithm, where the overall reward com- bines the influence score with consstecon rewards. Training uses batch size 256, learning rate 1 x 10 rollout temperature 1.5, rollout size 5, and one epoch of updates. Processing 20K samples requires roughly 10 hours.
Final SFT. After RL, the refined prompter generates an extended synthetic dataset, which is then used to conduct supervised fine-tuning (SFT) on qwen3- 10n3--8-0 8b-base as the target model. This en- sures that the model benefits both from high-quality abrrc--uided data and from the RL-enhanced prompter optimization.
E EXPERIMENTAL RESULTS
E.1 BASELINE
Since Monressor- Instrucct requires training the teacher model, we re-implement it by replacing its original teacher with Qwen-30B-A3B, so that all influence-based baselines and method share the same strong teacher. Therefore, we argue that our work is not a simple application of existing influence-function tools to new task, but rather re-design of how influence-based signals used under the realistic constraint of "synthetic data generation + a strong but frozen teacher": instead of "training the model", we shift "training an evaluator, which in turn guides data synthesis". This usage pattern has been rarely discussed systematically in prior work and, in our opinion, consti- tutes the main conceptual novelty of our method. Under this unified setting, the results in Table D show a clear hierarchy among different ways of exploiting influence- Instruct brings only marginal or even negative changes compared with the base model on several benchmarks (e.g., HLE and Health Bench), while the intermediate variant Monressor (inter 2) can leverage influence scores more effectively and yields consistent gains across almost all tasks. Our method further improves over these basclines most general- and domain-speciff benchmarks: achieves the best performance on MMLU-pro, Super GPQA, HLE, Health Bench, and PubMed, and remains competitive on MedQA, slightly below Monressor (inter 2) but comparable Monressor- Instruct. Taken together, these findings support our design choice of using influence-basee signals to steer synthetic data construction via learned evaluator, and demonstrate that this re-purposed pipeline can provide more robust and efficient improvements than directly optimizing the teacher model itself.
signals. Monressor-
E.2 ABLATION ON FEEDBACK MECHANISMS.
We conduct ablation study that directly compares several representative feedback mechanisms under a unified setup, where we only vary the signal used to optimize the rubrics and select syn- thetic samples. Concretely, consider (i) QuRating Wettig et al. (2024), which uses LLM--bssed pairwise quality scores along multiple dimensions (style, expertise, factuality, educational value); (ii) SineWeb-Edu-see proxy Penedo et al. (202)), which measures the "educational / knowledge
20

---

Published as a conference paper at ICLR 2026
Method
MMI.Uppro
Super GPQA
HLE
Health Bench
PubMed
MedQA
Base
0.4597
0.2806
0.1036
0.7006
0.6590
0,5145
Monressor-Instruue
0,4672
0.2728
0.0946
0.6453
0.7027
0.5204
Monressor (inter 2)
0.5028
0.3105
0.1047
0.6893
0.7458
0.5469
Ours
0.5369
0.3549
0.1096
0.7062
0.7859
0.5206
Table 5: Performance comparison of different influence- methods.
Method
MMLU-pro
Super GPQA
HLE
Health Bench
PubMed
MedQA
Base-Model
0.4597
0.2806
0.1036
0.7006
0.6590
0.5145
QuRating
0,4954
0.3032
0.1118
0.7106
0.7356
0.5534
FineWeb-Edu
0.4682
0,2892
0.1064
0.7340
0.7044
0.5628
PRRC
0.5317
0.3472
0.0914
0.7556
0.7568
0.5247
Ours
0.5697
0.3828
0.1081
0.7482
0.8070
0.5875
Table 6: Comparison of different reward / feedback mechanisms.
density" of each candidate sample by its similarity to the FineWeb-Edu distribution; (iii) PRRC / Meta-rater Whhuang et al. (2025), which aggregates multi-dimensional quality assessments (Profes- sionalism, Readability, Reasoning, Cleanliness) into a single proxy-reward proxy-validation score; and (iv) Ours (Influence), which uses an fflueece-baset signal aligned with the target validation loss. Results are shown in Table
six medical and health benchmarks, three proxy-based feedback mechanisms significantly improve over the base model, indicating that downstream reward proxy-validation signals are effective. However, when averaging across all benchmarks, our influence- the best overall performance. While proxy signals rely heuristic properties of each sample (c.g., teacher scores, similarity to high-quality corpora, or performance on a small validation subset), our influence signal is derived from optimization perspective and explicitly approximates how small upweighting of a sample would change the target validation loss, thereby aligning data selection more tightly with the final training objective.
feedback achieves
E.3 ANALYSIS
Impact of data scale. We compare the effect of different amounts of synthetic data on model train- ing. For the 8B base model, adding small amount of synthetic data (5k-10k) already yields clear improvements over the base model, and performance continues to increase up to around 25k-30k examples on most benchmarks; beyond this point, enlarging the dataset 40k brings diminishing or even slightly negative returns. In contrast, for the 14B base model, performance on MMLU-pro, Su- per GPQA, PubMed and MedQA improves more steadily as the synthetic data scale grows, with the best overall results obtained at 40k examples, although individual tasks such as Health Bench peak earlier. These trends suggest that the optimal data scale is both model-size- and task- dependent::: smaller models quickly saturate and can even be hurt by excessive synthetic data, whereas larger models are better able to exploit larger-scale synthetic corpora. The specific results are shown in Figure 10
E.4 VALIDATION SET OF SCORE CONSTRUCTION AND ANALYSIS.
When constructing the validation set, we follow two principles: (i) no overlap or information leak- age with the SFT training data, and (ii) the validation data should come from public benchmark splits that share the same distribution as the downstream evaluations. the dev split of MMLU-pro (History) as the validation set, while reporting test performance on the official test splits of MMLU-pro, SuperGPQA, and HLE. In the Medical domain, we use the official MedQA test split the validation set for computing influence scores and guiding synthetic data selection, whereas evaluation is conducted on the Medical & Health suite including MMLU-pro (Health), SuperGPQQ (Health), HLE, Heachh PubMedQA, and MedQA. This setup is
the HSS domain, we use
21

---

Published as a conference paper at ICLR 2026
03
Aag
May "laa
Figure 10: The impact of training data volume on model performance.
Scale
MMLU-pro
Super GPQA
Health Bench
PubMed
MedQA
HLE
T-test
8B-base
0.4597
0.2806
0.7006
0.6590
0,5145
0.1036
0.0112
100
0.5236
0.3016
0.6759
0,6946
0.5141
0.0942
0.0046
300
0.5423
0.3096
0.6837
0.6480
0.4908
0.1006
0.0224
500
0.5318
0.3426
0.6995
0.7682
0.5738
0.0913
0.0023*
800
0.5721
0.3739
0.7958
0.7835
0.5729
0.0982
0.9158
1000
0.5613
0.3663
0.7831
0.7961
0.6063
0.1057
0.7644
1273
0.5697
0.3828
0.7482
0.8070
0,5875
0.1081
Table 7: Effect of validation set size on downstream performance (8B model).
sistent with prior work such as LESS Xia et al. (2024). where an in-distribution public split is used purely as a validation set compute gradients and influence scores, but is never included SFT training, thereby preserving evaluation fairness while providing representative task signal. We further analyze how the choice of validation set affects model performance the Medical domain. Keeping the SFT data and training configuration fixed, we sample subsets of different sizes (100 300 500 800 1000 full 1273 examples) from the MedQA test set as validation set D and use them to compute influence scores. As shown in Table 7 enlarging the validation set from 100 to 800 examples leads to steady improvements on downstream benchmarks, while the gains beyond 800 examples become marginal. Even very small validation sets (100 300 examples) already yield models that significantly outperform the 8B base model on average, but the perfor- mance variance across different random subsets is noticeably higher. To mitigate the instability of small validation sets, we explore two diversity-aware sampling strategies with a fixed valida- tion size of 100: (i) Embedding-Baseed pairs into an embedding space and selects a subset with more uniform coverage in this space; and which leverages the EvalTree capability decomposition choose samples that
which uses Dreen-SB-Embeddeee
map candidate QA
(ii EvalTree-Ban cover broad range of skills, especially those where the model is relatively weak. 'Table g shows that both strategies substantially improve over random sampling: with only 100 validation exam- ples, Embedding-Basse and the EvalTree-Basee variant already approaches the performance obtained with the full 1273- example validation set. These results indicate that, long diversity and capability coverage properly controlled, our influence-bart based method is not overly sensitive to validation set size, and a small but carefully constructed validation set can still provide high-quality supervision for influence estimation.
and EvalTree-Baud Based sampling outperform Random most benchmarks,
E.5 TRAINING EFFICIENCY COMPARISON.
To further assess training efficiency, we compare our method with baseline datasets under fixed compute budget. The dataset statistics are summarized in Table Since all models are trained using same SFT framework imated by the total number of tokens in each dataset. As reported in Figure II under comparable compute budgets, the OptimSyn-bain based model consistently achieves the best performance among all methods, demonstrating superior training efficiency.
identical hyperparameters, the required training FLOPs can be approx-
22

---

Published as a conference paper at ICLR 2026
Strategy
MML.Uprr
Super GPQA
Health Bench
PubMed
MedQA
HLE
Random
0.5236
0.3016
0.6759
0.6946
0,5141
0.0942
Simbedding-Based Based
0.5428
0.3138
0.6954
0.7284
0.5576
0.1036
EvalTree-
0.5601
0.3596
0.7434
0.7392
0.5327
0.0968
All
0.5697
0.3828
0.7482
0.8070
0.5875
0.1081
Table 8: Comparison of different validation sampling strategies (100-sample subsets vs. using all validation data).
Chat
1916
convrnt
Laiverse
cavvrre covvent
can try?
01.00g
Figure 11: Comparison of SFT results under the same computational budget.
E.6 INSTRUCTIONS FOR PROMPTER
System Prompt for Prompter to Generate a Rubrics
You are an expert the {domain) field. Your core task instruct the "Generator"
LLM to produce single, high-quality Question-Answering (QA) data point based on the provided {domain] content.
Figure 12: Instruction template for generating QA pair.
23

---

Published as a conference paper at ICLR 2026
Prompt Template for Prompter to analyze quality dimensions
There is a given document: (document)
Objective:
Guide the model to generate corresponding questions and answers based on the document provided above. This QA data will be used to fine-tune downstream Target LLM, aiming to significantly enhance its comprehension, analysis, and creative capabilities in [domain] tasks.
Instructions:
1. Randomly select one question type from the following set: [Multiple choice, Fill-in-the- blank, Short answer, Essay].
2. Based on the characteristics of the document, analyze which quality dimensions should be prioritized for the question and answer under the selected type (with the ultimate goal of enabling the model to perform well in {domain} downstream tasks).
The analysis may include, but is not limited to: the document's topic, style, purpose, com- plexity level, target audience, linguistic style, and knowledge density. Broad quality dimen- sions (e.g., scope, clarity, creativity, relevance, reasoning depth, factual accuracy) can be used, but their importance and descriptions should be adapted to match the specific charac- teristics of the document.
least three additional dimensions beyond the above examples should considered such as completeness. format compliance, tone aligament, safety, neutrality, information density. instructional precision. or adaptability to ambiguous inputs.
When defining and prioritizing quality dimensions, you may draw examples such as: Prompt-related: clarity, unambiguity, exccutability, conciseness, focus, language and gram- mar correctness, format appropriateness, domain relevance, bias avoidance, temporal valid- ity, contextual coherence.
Response-related correctness, completeness, relevance, logical soundness, conciseness, fluency, credibility of information sources, creativity, information value density, neutrality, safety.
Prompt-Response alignment: consistency, logical linkage, coverage of all sub-questions, appropriate disambiguation, handling of potentially incorrect prompts, compliance with con- straints.
parsability, consistent encoding, absence of PIT, diversity across samples, low noise, length suitability, copyright compliance.
Technical/trainabbiity aspects:
Avoid using fixed templates or rigid numerical scales. Do not provide specific content of the Q-A; instead, focus on evaluating which dimensions matter most and why.
The standards must be actionable to ensure "Generator" can accurately understand synthesize the required high-quality data.
Output Format:
("Prompt-relatec""
"Response-related":
"Prompt-Response ali alignment": "Fechiical/trainaalliiy aspects"; ...)
Figure 13: Prompt template for analyzing quality dimensions before QA generation.
24

---

Published as a conference paper at ICLR 2026
Dataset
Tokens (M)
Num.
ChatDoctor
246.3
100K
MedQA
15.7
12.7k
Medical-ol
557.9
19.7k
Modical-R1-Distiee
579,4
22k
ReasonMed
645.4
370k
OptimSyn
125.7
26.6k
Table 9: Data Statistics.
E.7 INSTRUCTIONS FOR GENERATOR
System Prompt for Generator to Generate QA pair
You are an expert in the field of {domain). will now provide you with document. Based on this document, you need to synthesize ONLY pair of high-qualit question answer (QA) data, which will be used for downstream task training.
Figure 14: Instruction template for generating QA pair.
Prompt Template for Generator generate QA
There is given document: {doc}
Specific requirements generating QA data:
1. Generate ONLY QA pair.
2. Question Quality Standard: (Prompt-related).
3. Answer Quality Standard: (Response-related)
4. Prompt-Response alignment:
alignment).
5. Technical/trannabliiyy aspects: (Technica//rainaabliyy aspects).
6. For multiple-chotee questions, necessary generate a "think step by step" reasoning process in the answer, ultimately leading to the conclusion.
7. All generated questions and answers must be closely related to the given content. However, do indicate that you can see content or use expressions like "according the docu- ment." Do not mention the document at all.
8. The output format must be JSONL, with the specific structure follows:
"messages":
"system", ""rose""""ytte"", [ ["role"; "user", "content "ebeen"QQueee Questtion, ""role":"assistar "content " :Answer)
("role":
"content"; "You "You are a helpful assistents ")
Figure 15: Prompt template for generating QA data in the HSS and Medical domain.
25

---

Published as a conference paper at ICLR 2026
E.8 EXAMPLES OF RUBRICS
There is given document: (doc)
Generate ONLY QA pair.
2. Question Quality Standard:
"Clarity": "Questions must be clear and unambiguous, explicitly referring specific categories of medieval hospituls, such small hostels/hospices, large civic hospituls, hospitals established by crusading orders, to ensure precise aligument with structured descriptions the document.",
"Questions should focus core themes such religious character, the service recipients, drivers of expansion, to maintain high relevance to the Humanities and Social Sciences domain.".
"Domain Relevance":
process of medicalization,
"Format Appropriatenes"" "For short-answer questions, prompts should specify the required response format, such briefly explaining the main function expansion reason of particular type of hospital, facilitate evaluation." "
3. Answer Quality Standard:
"Correctness": "Answers must accurately reflect the definitions the document, for example correctly describing how type game uses power, to ensure factual reliability.". "Completeness": "Responses should address key aspects of the game type, including its objective, primary participants, and operating mechanisms." ",
"Relevance": "Answers must avoid introducing unrelated theories or cases that not pertain to organizational political games, so as to maintain focus."
).
4. Fompp-Response alignment:
"Consistency": "The answer must remain consistent with the question's focus, such as clearly differentiating the roles and objectives of two game types when making comparisons." "Format Matching": "For multiple-choice questions, the answer should correspond to the provided options and avoid adding new, unlisted information." ",
"Handling Ambiguous Inputs": the question vague, answer should explicitly reference the name of specific game type from the document to ensure clarity."
Technical/rainaailityy aspects: {
"Parsability:: "QA pairs must use structured, machine-readable formats (such as JSON) to facilitate automated processing and training.".
"Copyright Compliance": "Answers should not reproduce the verbatim but instead paruphrase original language while preserving meaning.", "Length Suitubility": "Responses should be concise, idcally within 200 tokens, to prevent truncation during model training."
6. For multiple-chote questions, it is necessary to generate a "think step by step" reasoning process in the answer, ultimately leading to the correct conclusion.
7. All generated questions and answers must be closely related to the given content. However, do not indicate that you can see content use expressions like "based the document," not mention the document at all.
8. The output format must be JSONL, with the following structure:
"msssaees" [("role": "system", "content":
"you are a helpfid assistent." "), ("role": "user", "content": Question). ("role": "assistant", "content": Answer]]
Figure 16: An example of generated rubrics for data synthesis.
26

---

Published as a conference paper at ICLR 2026
There is a given document: {doc}
1. Generate ONLY QA pair.
2. Question Quality Standard: Questions should have high clarity and strong domain relevance, focusing directly on the concepts described in the source document about medieval hospitals including their religious foundations, structural variations, medicalization process, and historicul expunsion. For example, they might compare differences between small hostels histel//hospices und large civic hospitals, analyze the role of the Crusades hospital network expansion, examine how institutions such as Sta Maria Nuova evolved to serve both indigent and higher-class patients. Avoid vague wording that could obscure distinctions between civic and religious institutions or the nature of services provided. Each question should clearly set the theme and scope and, when applicable, identify relevant historical categories. For short-answer formats, prompts may indicate the expected response structure (c.g.. "Summarrzze how Sta Maria Nuova transitioned from hospice to medicalized hospital in two sentences"), Idcally, questions should require synthesis across the entire text rather than retrieval of single sentence.
Answer Quality Standard: Answers should accurately reflect the text's historical details, distinguishing between different hospital types (religious vs civic, small hospices vs large urban hospitals). and highlighting contextual factors such as the influence of Crusading orders and non- military brotherhoods. Responses must be logically coherent, factually correct, and address all relevant aspects mentioned in question, including chronology, institutional roles, and socio- economic reach. Information density can be enhanced by providing key examples from the text (c.g.. St ceonard's s in York, St Bartholomew's 's Paris Hôtel Dieu, Sta Maria Nuova's
medicalization). Answers must rely comprehensive reading of the entire passage, not not isolated lines, and preserve the distinctions made in the text. Clear differentiation between similar cases will help maintain alignment with the text's historical framework.
4. Prompt-Response alignment: Strong alignment is shown when the answer directly ties the question's 's focus and cites concrete examples from the text that illustrate structural, functional, and historical contrasts. If the prompt requests a comparison of hospital lypes or factors driving expansion, the answer should deliver that comparison with historical examples (e.g., Crusader-built hospitals Order of the Holy Spirit network). Completeness involves covering all subcomponents in the question (such as stakeholders, mechanisms, intended outcomes) and presenting them in the expected structure and tone. Matching quality should consider whether the depth, detail selection, and contextual fit align with the question's intent. Logical consistency must be maintained, avoiding conflation between distinct institutions or time periods.
5. Technical/trainabllig aspects: Paraphrasing should replace verbatim copying to maintain meaning while improving linguistic flcxibility. Responses should avoid excessive length to stay within model context limits. A neutral, academic tone should be maintained, factually describing hospital foundations, structures, and development without subjective evaluation. These standards guide the generation of high-quality question-answer pairs that are clearly aligned with the theme of medieval hospitals, historically accurate, and evaluable in training contexts. 6. For multiple-choice questions, it necessary generate "think step by step" reasoning process in the answer, ultimately leading to correct conclusion.
7. All generated questions and answers must be closely related to the given content. However, do not indicate that you see the content use expressions like "hased on the document." Do not mention the document all.
8. The output format must be JSONI., with the following structure:
"messages": [!"role": "system", "content": "you are helpfid assistent." ("role": "user". "content": Question). ("role": "assistant". "content": Answer]]
Figure 17: An example of generated rubrics for data synthesis.
27

---

Published as a conference paper at ICLR 2026
E.9 EXAMPLES SYNTHETIC QA PAIRS
System prompt: You are a helpful assistant.
Briefly compare the differences between large medieval civie hospitals and small hostels/hospices in terms of religious background, medical staff allocation, and drivers of expansion.
Question:
Answer: Large civic hospitals were typically established in urban centers, with a strong religious background, but supported by city economies and municipal funding. They gradually incorporated professional doctors, pharmacists, and auxiliary staff, achieving high degree medicalization, and catered the poor while also providing private rooms for upper-class patients. In contrast, small hostels hospices were mostly tied to local religious charity traditions, limited in scale, and had almost resident medical staff. Their expansion was driven mainly by charitable donations from local religious orders or parishes, rather than systematic public funding.
Figure 18: An example of synthetic QA pairs.
System prompt: You are a helpful assistant.
Question: Analyze how religious character and medicalization process medieval hospitals were manifested different types of institutions (such as small bostels/hospices, large civic hospitals, and hospitals established by crusading orders), and compare their differences medical staff allocation, social service recipients, drivers of expansion.
Answer: Medieval hospitals generally possessed strong religious character. Small hostels hospices were essentially attached charitable religious traditions, primarily serving pilgrims with lodging and basic care in their early stages, and lacked resident medical personnel. In contrast, large civic hospitals such as Paris's 's Hôtel Dieu and Florence's Santa Maria Nuova gradually introduced dedicated doctors and pharmacists, signaling significant progress in medicalization. These institutions not only treated the poor but also provided private rooms reserved for upper-class patients, reflecting diversity in their social service recipients. Hospitals established by crusading orders such the Knights Hospitaller, the Knights Templar, and the Teutonie Knights were mostly located in Mediterranean and German-speaking regions, primarily serving the needs of military forces and local populations ulong their routes, with expansion driven by military campaigns and religious missions rather than urban public service demands. The growth of small religious hostels was constrained by funding and local religious resources, while large civic hospituls relied on urban populations and municipal support, and the network of crusader hospitals was sustained by the transregional organizational strength of the orders. Overall, these three types of institutions were similar their pervasive Christian ethos but differed sharply the scale of medical staff deployment, the breadth their service recipients, and the factors driving their expansion.
Figure 19: An example of synthetic QA pairs.
28

---

Published as a conference paper at ICLR 2026
E.10 EXAMPLES OF SEED DATA
Title / Chapter: HOSPITALS
Medieval hospitals were religious foundations through and through. Those planted in the West had originally been small and mainly for pilgrims; their late medieval successors were often impressive. St Lonnrdd's York had 225 sick and poor 1287; still larger were the civic hospitals of Milan, Siena and Paris. In Florence alone, city of some 30,000 inhabitants, there were over thirty foundations by the fifleenth century. Some had only ten beds, others hundreds. In England hospitals and almshouses totalled almost five hundred by 1400, though few were any significance. London's Si Bartholomew's dates from 1123 and St Thomas's from around 1215. A1 Bury St Edmunds six hospitals were endowed between 1150 and 1260 to cater for lepers. pilgrims, the infirm and the aged. in'nSmall hospitals were essentially hostels or hospices lacking resident medical assistance, but physicians were in attendance by 1231 at the Paris Hötel Dieu, next to Notre Dame, and Sta Maria Nuova in Florence 'the sick and poor', this 'first hospital among Christians', as one Florentine patriot called expanded by 1500 to a medical staff of ten doctors, a pharmacist and several assistants, including female surgeons. Although catering largely for the indigent, it had eight private rooms 'reserved for the sick of the higher classes'. Within hospital walls Christian ethos was all-pervasive. in'nin hospital expansion the Crusades played their part, since crusading orders such as the Knights of St John of Jerusalem (later the Knights of Malta), the Knights Templar, and the Teutonic Knights built hospitals throughout Mediterranean and military brotherhoods, such as the Order of the Holy Spirit, were also running infirmaries from Alsace to Poland, while the Order of St John of God appeared in Spain in the sixteenth century, building insane asylums and putting up about hospitals the New World.
gradually medicalized: from twelve beds in 1288 for
speaking lands. By fourteenth century non-
Figure 20: An example of seed data.
29


Reviews:
[{"review_id": "1pckfrMYMD", "reviewer": "Reviewer_tUAj", "summary": "Previous studies on synthetic data generation typically rely on expert-designed, domain-specific rubrics, which often suffer from limited transferability and scalability. This paper proposes a novel paradigm: instead of handcrafting rubrics, it advocates evaluating synthetic data quality based on its actual training utility on the target model. This feedback is then used to iteratively refine the rubrics themselves, forming a closed loop between data synthesis and downstream performance. \n\nThe resulting QA pairs are optimized to maximize estimated downstream benefit under these learned rubrics. Experiments in two low-resource domains—Humanities & Social Sciences (HSS) and Medical & Health (MH)—demonstrate that the learned rubrics are both model- and task-conditioned, reducing reliance on domain expertise.", "questions": "please refer to \"Weaknesses\"", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe key insight—replacing heuristic rubric design with a model-impact-driven objective—is interesting. Training a rubric generator to maximize downstream utility offers a principled and adaptive alternative to manual engineering.\n2.\tEvaluation in HSS and MH domains, where high-quality SFT data is scarce, showcases the method’s practical value and generalizability beyond standard benchmarks.", "weaknesses": "1.\tHow do the number and length of rubrics evolve throughout reinforcement learning? Could the observed improvements be primarily driven by the generator's capacity or the diversity of synthesized QA pairs, rather than the rubrics themselves? What mechanism controls the ratio between rubric guidance and QA pairs? Additionally, have you analyzed the diversity of the generated data and its correlation with specific rubrics?\n2.\tCould the authors provide more details about the lightweight verification rules (e.g., formatting correctness, non-triviality, safety filters)? \n3.\tAre there notable differences in data statistics (e.g., size, average length, complexity) between prior datasets and the proposed one? Such differences may confound the observed gains; a comparative analysis would strengthen the claims.\n4.\tIn the 14B model results, the performance gain in the MH domain appears less pronounced than in HSS. Have the authors explored whether increasing the volume of synthetic data improves outcomes in MH, suggesting a data-hungry regime?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XyllAYhRcY", "reviewer": "Reviewer_WCjb", "summary": "This paper proposes a framework to unify the influence-based reward into model training to guide the synthetic data generation.\nThe motivation stems from the observation that current synthetic data pipelines rely heavily on handcrafted rubrics and heuristic feedback loops, which are expensive, brittle, and poorly correlated with true downstream model performance. The authors introduce an influence-function–based estimator that quantifies each synthetic sample’s contribution to the fine-tuned model’s objective. This influence signal is then used to guide rubric optimization and synthetic data selection via a lightweight optimization loop, replacing manual rubric engineering with model-informed adaptation.", "questions": "1. Can the authors provide quantitative results about efficiency?\n2. Have the authors compared their influence-based signal with alternative feedback mechanisms?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. Motivation is clear. Synthetic data generation requires a lot rubric engineering and hard to scaling.\n2. Frameworks is good. Put influence functions into RL framework to make the whole procedure automatic. \n3. Good performance. The improvements are consistent and statistically meaningful.", "weaknesses": "1. Limited Novelty and Conceptual Depth. While the paper presents a plausible framework, the underlying technical innovation appears incremental. The idea of integrating quantitative signals to guide synthetic data generation is intuitive and has been explored under different names (e.g., data valuation, reward-guided synthesis, active selection). The contribution primarily involves adopting influence-based rewards for rubric optimization, which builds directly on prior influence-function research rather than introducing a fundamentally new mechanism. As a result, the work feels more like a thoughtful application of existing tools than a conceptual breakthrough.\n2. Scalability and Efficiency Unclear. Traditional influence functions are notoriously computationally expensive. The paper claims to deploy an “optimizer-aware estimator” that scales efficiently, but does not provide sufficient empirical evidence or runtime comparisons to support this claim. Since the method is applied per-sample, a clear analysis of runtime overhead and system-level feasibility is crucial to evaluate its practicality for real-world synthetic data pipelines.\n3. Questionable Necessity of the Influence Signal. The paper argues that influence estimation provides a better feedback signal for synthetic data quality, but the empirical justification is not fully convincing. The improvement margins over simpler proxy metrics (e.g., downstream reward, embedding-similarity heuristics, or small proxy validation sets) are modest. Without ablation or sensitivity studies, it remains unclear whether the influence-based signal is truly superior or merely another heuristic choice. A comparison against alternative feedback mechanisms would strengthen the paper’s argument.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PQRfg7nhHd", "reviewer": "Reviewer_MqSM", "summary": "The paper introduces OptimSyn, a method that optimizes rubrics for synthetic SFT data by directly maximizing each rubric’s training impact on a chosen target model. The approach replaces heuristic or embedding-similarity–based rubric design with an optimizer-aware influence objective aligned to Adam, ensuring the signal reflects actual learning dynamics. Experiments span multiple domains and model families, showing consistent gains over strong SFT and heuristic-rubric baselines.", "questions": "* Could you explain how validation data is selected? How selection of validation data impacts the downstream target model training performance?\n* When computing influence score as in equation (1), are gradients taken with respect to the full parameter set used for training? Are there any analysis or experiments on runtime and memory complexity for scaling the model size from 8B to larger models, what gradient-feature compression method could be used to reduce the computation and storage?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper identifies a core mismatch between embedding proximity and true training utility, motivating an influence-aligned objective that targets what actually updates the model.\n* The empirical coverage is broad, including HSS/medical domains and cross-family targets/generators, which strengthens external validity.", "weaknesses": "* The approach requires a dedicated validation set to estimate influence, which some tasks may not have; when available, the estimator’s reliability can be highly sensitive to how that set is constructed, risking leakage and brittleness.\n* The paper defines the reward function with hyper-parameter $\\lambda$. Sensitivity analysis for $\\lambda$ is needed to assess the robustness of the reward choice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ER9quieLH4", "reviewer": "Reviewer_a9XS", "summary": "This paper proposes a method to generate specialized rubrics to condition synthetic data generation. The authors optimize these rubrics via an RL pipeline based on GRPO, where the reward is defined using validity checks and an influenced based score. The intuition is that a rubric is good if the generated synthetic data conditioned on that rubric maximize the influence on evaluation samples.", "questions": "How do you ensure no contamination between the validation sets used for influence scoring and the benchmarks you report?\nDid you run any overlap analysis between the seeds and the eval benchmarks?\n\nCan you provide token/step parity vs each baseline in Table 1?\n\nHow does the method scale beyond 20k samples ?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Relevance of the problem: the problem tackled in this paper is an important one, as high quality SFT data is scarce in some domains.\n\nNovelty: the use of Adam-aware influence estimation in the context of synthetic data seems to be novel.\n\nPerformance: the reported downstream performance is competitive with the baselines, as it often beats both the base and instruct Qwen model on HSS tasks, and narrows the gap with medically-oriented baselines on medical tasks.", "weaknesses": "Details about the validation dataset: since rewards are computed using a validation set from the same topics as the benchmarks (HSS/medical), the authors should provide more details about how this validation dataset is constructed. It is important that there is no leakage between the validation and the test dataset.\n\nSeeds and test datasets: seeds come from Goodreads books and Meditron/PubMed. It is important to check for overlap vs. MMLU-pro, SuperGPQA, PubMedQA, etc. to verify that performance gains just do not come from data contamination.\n\nComparison with the baselines: the paper does not report matched data volumes, number of fine-tuning steps, or compute budgets per baseline. This means improvements could come from getting more tuning tokens rather than better data quality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)