Low-Confidence Gold: Refining Low-Confidence Samples for Efficient Instruction Tuning (2025)

Abstract: The effectiveness of instruction fine-tuning for Large Language Models is fundamentally constrained by the quality and efficiency of training datasets. This work introduces Low-Confidence Gold (LCG), a novel filtering framework that employs centroid-based clustering and confidence-guided selection for identifying valuable instruction pairs. Through a semi-supervised approach using a lightweight classifier trained on representative samples, LCG curates high-quality subsets while preserving data diversity. Experimental evaluation demonstrates that models fine-tuned on LCG-filtered subsets of 6K samples achieve superior performance compared to existing methods, with substantial improvements on MT-bench and consistent gains across comprehensive evaluation metrics. The framework’s efficacy while maintaining model performance establishes a promising result for efficient instruction tuning.

## introduction large language models (llms) have been trained to follow instructions by specific supervised response data after pre-training stage. many instruction finetuning (ift) @xcite datasets emerge to realize various downstream tasks, for example: mathematic calculation, sentence analysis, haiku writing and etc, aiming to strengthen the ability of llms in instruction following. to save vast human costs for data annotation, most of studies introduce other teacher llms (e.g. text-davinci-003 @xcite ) to align the best instructions with corresponding responses. however, ift datasets (e.g. alpaca_52k @xcite , magpie @xcite ) suffer from misleading content and poor quality, resulting in the bottleneck of post-training performance, even though teacher models replenish the missing parts of context and instruction pairs. this highlights the need for effective data filtering methods that iden-figure 1 : we target to select complex and quality samples confidence ranking for benefiting llm training. alpagasus @xcite proposed a model-based approach that introduces proprietary llms to score data quality in multiple facets, replacing human annotation by taking advantage of the automated pipeline. however, this leads to datasets that are likely biased by the preference for redundant and limited responses @xcite , which potentially deteriorates the diversity of the original data. ge et al., 2024 emphasizes the necessity of diversity and therefore proposed clustering and ranking to select subsets of data. further, superfiltering @xcite gains more insights in small opensource llm that scores the instruction following ability of alpaca_52k. although the instruction score provides an efficient and simple criterion for data selection, it does not consistently correlate with both the quality and diversity of data. consequently, improvements in performance may not always be guaranteed. to address these challenges, we propose a novel data filtering framework, low-confidence gold (lcg) for efficient instruction tuning that signif-icantly reduces computational costs while maintaining model performance. our approach, shown in 1, innovatively seeks to identify high-value instruction data through classification tasks. specifically, we develop a lightweight classification model trained on centroid subsets that effectively categorizes instruction-response pairs, and leverage lowconfidence predictions to curate challenging examples most beneficial for instruction tuning. another perspective is that, since the common instruction tuning data are lack of annotations and labels, we adopt the manner of semi-supervised learning, to construct pseudo-labels as our training groundtruth, as well as getting inspired quality data from affordable yet effective models. through extensive experiments on the alpaca_52k dataset, we demonstrate that our filtered subsets achieve comparable or better performance when fine-tuning various open-source language models, while requiring only a fraction of the original data. our main contributions are threefold: ## conclusion in this paper, we proposed low-confidence gold (lcg), a novel data filtering framework that combines cluster-centric pseudo-labeling with earlystopped classifier training for efficient instruction tuning. through extensive experiments, we demonstrated the strong performance across multiple benchmarks and base models, validating the effectiveness of our semi-supervised learning paradigm in maintaining both data quality and diversity for instruction tuning.