Teaching According to Talents! Instruction Tuning LLM s with Competence-Aware Curriculum Learning (2025)

Abstract: Efficient instruction tuning aims to enhance the ultimate performance of large language models (LLMs) trained on a given instruction dataset. Curriculum learning as a typical data organization strategy has shown preliminary effectiveness in instruction tuning. However, current curriculum tuning methods suffer from the curriculum rigidity, since they rely solely on static heuristic difficulty metrics. These methods fail to adapt to the evolving capabilities of models during training, resulting in a fixed and potentially sub-optimal learning trajectory. To address the issue, **C**ompetence-**A**ware **M**ulti-**P**erspective c**U**rriculum in**S**truction tuning framework termed **CAMPUS** is proposed. CAMPUS offers several advantages: (1) Dynamic selection for sub-curriculum. (2) Competency-aware adjustment to the curriculum schedule. (3) Multiple difficulty-based scheduling. Extensive experiments prove the superior performance of CAMPUS, compared to other state-of-the-art baselines for efficient instruction tuning.

## introduction instruction tuning @xcite aligns large language models (llms) @xcite with human preferences, enhancing their effectiveness across realworld tasks. some studies @xcite have highlighted the importance of instruction data management, including data quality curation and training strategy. this has fueled interest in efficient instruction tuning @xcite , which aims to maximize llm performance trained on a given instruction dataset. among various strategies, optimizing training order has emerged as a key research focus. curriculum learning @xcite , as a typical data order organization strategy that mimics the learning process of human education, has * equal contribution. ‡ corresponding author. shown preliminary effectiveness in instruction tuning. the essential idea is to train llms with instruction data in a progression from easy to difficult, thereby accelerating model convergence and achieving a higher performance upper bound. the key to curriculum learning lies in the design of difficulty metrics for instruction data. for instance, tree-instruct @xcite treats instruction data as a semantic tree and measures difficulty based on the number of tree nodes. referring to the expert-designed educational framework, @xcite utilizes chatgpt to synthesize the instruction dataset corgi with increasing difficulty, and then train the llms sequentially. despite these advancements, previous difficulty metrics are often static and predefined with human intuition. these metrics fail to gauge the difficulty variations of the same data for different llms or at different learning stages. consequently, as shown in figure 1 , curriculum instruction tuning based on them lacks the flexibility to adjust curriculum for different llms or as llms evolve. the rigid curriculum schedule result in a sub-optimization in instruction tuning. as educator john dewey said, "if we teach today's students as we taught yesterday's, we rob them of tomorrow," underscoring the necessity of adaptable learning strategies in human cognition. however, it remains unclear for us how to customize a "suitable" instruction curriculum for a specific llm. in light of the above issues, we seek to develop a data order organization strategy that customizes a dynamic curriculum for different llms or llm at different training phases, based on which we can improve the training efficiency of instruction dataset. to this end, a competence-aware multi-perspective curriculum instruction tuning framework termed campus is proposed. initially, campus independently sorts instruction data using multiple difficulty metrics, each representing a curriculum schedule developed from distinct perspectives. at each training step, cam-pus assesses the llm competence and inherent data difficulty, selecting the sub-curriculum from the various schedules that poses the minimum perplexity, i.e., sub-curriculum that can be relatively easily comprehended and internalized by current llm. campus is compatible with arbitrary quantities of difficulties. besides the conventional heuristic difficulty, we introduce competence-aware difficulty based on a lightweight scoring model, which is trained with adversarial learning. it considers both llm parameters and instruction data as joint inputs to assess the instruction data's difficulty for the current llm. as the llm's training progresses, the competence-aware difficulty consistently evolves, allowing the curriculum based on the reward model to dynamically adjust. compared to existing curriculum instruction tuning methods, campus offers the following advantages: (1) dynamic selection for sub-curriculum: it continuously evaluates the learning state and progress of the model, selecting the most appropriate sub-curriculum to match the current learning needs. (2) competency-aware adjustment to the curriculum schedule: campus adjusts the curriculum schedule based on real-time assessments of the model's competencies, ensuring that the curriculum aligns with the model's evolving strengths and weaknesses. (3) multiple difficulty-based scheduling: unlike methods that rely on a single static metric, campus utilizes multiple difficulty metric, holding a more comprehensive perspective of the data difficulty. llms from different families are equipped with campus to demonstrate their effectiveness, including bloomz and llama. in experiments, we utilized a mixed instruction dataset derived from code alpaca, gsm8k, and sharegpt as the orig-inal training dataset d. various types of efficient instruction methods, including data selection and training order optimization, are served as baseline models to enhance training efficiency with d. we conduct an extensive evaluation on three benchmarks: gsm8k for mathematical reasoning, hu-maneval for coding, and mt-bench for general language understanding. campus consistently outperforms state-of-the-art methods by an average of 7.0%. additionally, as a pluggable training strategy, campus can be integrated with other data selection methods to further enhance the efficiency of instruction tuning.