L ima C ost: Data Valuation for Instruction Tuning of Large Language Models (2025)

Abstract: Instruction tuning (IT) is an effective approach for aligning large language models (LLMs) with human intentions. There is ongoing discourse regarding the data quality for IT. As an effort to find the robust criteria of data quality for IT, we introduce LimaCost, a data quality measure that exhibits a strong correlation with model performance. LimaCost utilizes LIMA dataset, which effectiveness in IT has already been validated by several previous works. LimaCost then estimates the value of a given data by estimating how many LIMA data points might be needed to approximate its gradient. Our experiments reveal that LimaCost enables effective data selection that derive high alignment performance. We demonstrate that selecting data based on high LimaCost proves to be more effective than existing data selection strategies.

## introduction instruction tuning (it) is one of the practical strategy for large language models (llms) to attain human-interactive capability @xcite @xcite . to achieve more effective alignment, recent studies have highlighted the importance of acquiring high-quality it data over a large quantity @xcite @xcite . then, what data should we choose for it? numerous studies have sought a satisfactory answer to this question @xcite @xcite ), but we have yet to find a definitive answer. these attempts include quantifying data quality using frontier llms like chatgpt @xcite @xcite , or establishing explicit quality criteria and manually designed data based on those standards @xcite @xcite . ## related works it has garnered significant attention with the advent of llms @xcite @xcite . specifically, research over recent years has established a consensus that the quality of instruction tuning is crucial for ensuring alignment @xcite @xcite @xcite . this research trajectory continues to evolve, with a rich discourse on defining high-quality data and strategies for obtaining it @xcite @xcite . approaches include using model internal knowledge to select high-quality data @xcite and employing gpt evaluations to establish quality benchmarks @xcite @xcite . notably, research presented by lima @xcite demonstrated that carefully designed 1,000 pieces of data (experts manually devised heuristics indicative of high-quality data @xcite ) can yield a significantly high instruction-following capability. moreover, several recent works figured out that selectively training on the small fraction of selected alpaca data points show more effective than employing the entire 52k dataset @xcite @xcite . however, we find that these studies have not yet provided compelling explanations. the quality scores evaluated by frontier llms do not show a strong correlation with the performance of the trained models @xcite . additionally, most other quality scores rely heavily on humandefined heuristics @xcite @xcite or remain at the assumption stage that they might benefit model training @xcite . in response, we propose a quality metric that is more directly linked to model performance and allows for objective valuation of each data point. ## limacost limacost aims to quantify the influence of each data points on the model during it process. in other words, limacost serves as a quality measure for each data point z, with higher limacost(z) indicating higher quality. specifically, we measure the change of the model parameters when trained on each data point (i.e. also can be expressed as a gradient), and count the amount of lima data needed to estimate that change (i.e. gradient). to accomplish this, we define the following processes. ## experimental settings to assess the effectiveness of limacost, we select a subset of 1k instructions with the highest limacost scores from a large it dataset. we train llms using this selected data and evaluate their alignment performance. this approach allows us to verify the impact of limacost-selected data on alignment tuning. we assume that the model's performance correlates directly with data quality. to ensure rigorous validation, we performed a hyperparameter search based on the lima dataset. this search aimed to identify the optimal settings for training the it model using 1k general data, thereby achieving the best performance. we then applied these optimal hyperparameters consistently across all experiments. detailed training configurations and evaluation setups are provided in the appendix b and c. ## conclusion in this paper, we propose limacost, a data quality measure that leverages existing well-established assets. limacost quantifies the quality of data based on the amount of lima data required to estimate parameter changes learned from each data point. through comparison with various data selection baselines, we demonstrate that our proposed method achieves superior alignment performance. by analyzing data with high and low limacost, we identify factors that influence alignment performance. we plan to refine future research to enhance data quality assessment accuracy by utilizing existing assets.