RAISE : Reinforced Adaptive Instruction Selection For Large Language Models (2025)

Abstract: Instruction tuning of large language models (LLMs) benefits more from a handful of high-quality examples than from hordes of low-quality ones. Existing selection methods typically rely on static, heuristic quality scores and are executed only once before training. Consequently, they neither adapt to the changing state of the model nor target downstream objectives, leaving substantial room for optimization. We propose RAISE (**R**einforced **A**daptive **I**nstruction **SE**lection), a *dynamic*, *task-driven* framework that integrates selection into every training step. At each step, RAISE estimates the expected contribution of each candidate instruction to task performance and admits only the most helpful. By modeling this process as sequential decision making, we optimize the selector with reinforcement learning, yielding an interpretable policy specialized for the target task. Extensive experiments show that RAISE reaches comparable or better results than full-data training while updating only 1% of the steps, demonstrating both high efficacy and significant computational savings.

## introduction large language models (llms) have achieved remarkable progress in recent years, demonstrating exceptional capabilities in general language understanding @xcite and generation @xcite @xcite @xcite @xcite . a critical factor enabling these advancements is instruction fine-tuning @xcite @xcite , a process that aligns pretrained models with human intentions by training them on task-specific instructions. while existing efforts predominantly focus on scaling instruction datasets * equal contribution. considering a dynamic, task-aware approach to instruction selection, we introduce the concept of an instruction's dynamic value-its impact on the final model performance when used for gradient updates at time step t (we put detailed descriptions of dynamic value and example in appendix c). this dynamic value, which depends on both the training step and the task objective, serves as a quality measure that replaces fixed heuristic metrics and provides strong interpretability. dynamic instruction selection can be modeled as a sequential decision-making process aiming to maximize the model's performance after t steps. obviously, the optimal selection strategy is to select those instructions that have the most dynamic value at each step. based on this idea, we propose raise (reinforced adaptive instruction selection), a dynamic, non-heuristic, task-driven instruction selection framework. at its core is an acquisition function-a trainable mlp (sample-wise scorer) that estimates the dynamic value of each instruction and is optimized to maximize the final model performance. by leveraging a fully trained acquisition function to guide instruction selection, raise consistently produces high-performing models. moreover, its task-driven design allows the acquisition function to be flexibly adapted to various tasks through adjustments in the validation set and performance metrics. due to the sequential decisionmaking nature of raise, we employ reinforcement learning (rl) @xcite to optimize the acquisition function, treating each training process of llm as an episode in the rl setting. to promote diversity , raise groups instructions into multiple classes via k-means @xcite and ensures balanced sampling from each group in every training batch. these classes constitute diversity constraint. figure 1 shows the framework for raise, which considers both the score of the acquisition function and the diversity constraint when selecting instructions. in summary, our contributions are as follows: ## method in this section, we describe our method for dynamic instruction selection. a learnable acquisition function is trained to estimate the dynamic value of each instruction, ensuring adaptive and diversity-aware selection throughout the training process. we formally define the problem of dynamic instruction selection ( § 3.1), introduce the training framework of our selection policy ( § 3.2), and describe the state fusion mechanism that combines training state and data features ( § 3.3). we then present the instruction selection algorithm ( § 3.4) and the policy optimization algorithm for improving the selection policy ( § 3.5). ## analysis in this section, we further investigate how raise selects instructions by examining two core modules: state fusion and diversity-constrained selection. finally, we analyze the distribution of data selected by raise at different stages of training. we report results for two versions of llama-3.2 (3b and 1b) and qwen-2.5-3b. "0%" denotes the base model and "100%" denotes full alpaca dataset, and otherwise we select 1% of the data or equivalent number of training steps. "avg" denotes the average metric across these three benchmarks. "avg.q" denotes the additional performance gain achieved by each method, relative to the improvement obtained by using the 100% data baseline. it is computed as: avg.q(•) = avg(•)-avg(100%) avg(100%)-avg(0%) . bold numbers denotes the best performing on its column. ## conclusion in this paper, we present raise, a dynamic instruction selection method that adaptively selects beneficial instructions for llm fine-tuning. raise employs a task-objective-driven acquisition function and a cluster-based diversity mechanism to identify high-utility data. our experiments on multiple benchmarks demonstrate that raise outperforms static selection baselines, achieving strong performance while using only a small fraction of training steps. we hope this work inspires further research on adaptive data selection and fine-tuning strategies for large language models.