ICLR 2025

Published as a conference paper at ICLR 2025
MONTESSORI-INSTRUCT:
GENERATE INFLUENTIAL
TRAINING DATA TAILORED FOR STUDENT LEARNING
Xiaochuan Li♦∗, Zichun Yu✦, Chenyan Xiong✦
♦School of Software, Tsinghua University
✦Language Technologies Institute, Carnegie Mellon University
li-xc20@mails.tsinghua.edu.cn
zichunyu@andrew.cmu.edu
cx@cs.cmu.edu
ABSTRACT
Synthetic data has been widely used to train large language models, but their gener-
ative nature inevitably introduces noisy, non-informative, and misleading learning
signals. In this paper, we propose MONTESSORI-INSTRUCT, a novel data synthesis
framework that tailors the data synthesis ability of the teacher language model
toward the student language model’s learning process. Specifically, we utilize local
data influence of synthetic training data points on students to characterize students’
learning preferences. Then, we train the teacher model with Direct Preference
Optimization (DPO) to generate synthetic data tailored toward student learning pref-
erences. Experiments with Llama3-8B-Instruct (teacher) and Llama3-8B (student)
on Alpaca Eval and MT-Bench demonstrate that Montessori-Instruct significantly
outperforms standard synthesis methods by 18.35% and 46.24% relatively. Our
method also beats data synthesized by a stronger teacher model, GPT-4o. Further
analysis confirms the benefits of teacher’s learning to generate more influential train-
ing data in the student’s improved learning, the advantages of local data influence
in accurately measuring student preferences, and the robustness of Montessori-
Instruct across different student models. Our code and data are open-sourced at
https://github.com/cxcscmu/Montessori-Instruct.
1
INTRODUCTION
Synthetic training data is highly effective in various applications of large language models (LLMs) (Lu
et al., 2023), spanning from general pretraining (Allal et al., 2024; Zhou et al., 2024), instruction-
tuning (Tong et al., 2024) to domain-specific scenarios such as mathe...