ICML 2025

Large Language Models are Demonstration Pre-Selectors for Themselves
Jiarui Jin 1 2 * † Yuwei Wu 3 * Haoxuan Li 4 Xiaoting He 5 Weinan Zhang 1 Yiming Yang 3 Yong Yu 1
Jun Wang 6 Mengyue YangB 7
Abstract
In-context learning (ICL) with large language
models (LLMs) delivers strong few-shot per-
formance by choosing few-shot demonstrations
from the entire training data. However, exist-
ing ICL methods, which rely on similarity or di-
versity scores to choose demonstrations, incur
high computational costs due to repeatedly re-
trieval from large-scale datasets for each query.
To this end, we propose FEEDER (FEw yet Es-
sential Demonstration prE-selectoR), a novel pre-
selection framework that identifies a core sub-
set of demonstrations containing the most repre-
sentative examples in the training data, tailored
to specific LLMs. To construct this subset, we
introduce the “sufficiency” and “necessity” met-
rics in the pre-selection stage and design a tree-
based algorithm to identify representative exam-
ples efficiently. Once pre-selected, this represen-
tative subset can effectively replace the full train-
ing data, improving efficiency while maintain-
ing comparable performance in ICL. Additionally,
our pre-selected subset also benefits fine-tuning
LLMs, where we introduce a bi-level optimization
method that enhances training efficiency without
sacrificing performance. Experiments with LLMs
ranging from 300M to 8B parameters show that
FEEDER can reduce training data size by over 20%
while maintaining performance and seamlessly in-
tegrating with various downstream demonstration
selection strategies in ICL.
* Equal contributions: Jiarui Jin and Yuwei Wu. † Work done
during Jiarui Jin’s visit at University College London. 1Shanghai
Jiao Tong University 2Xiaohongshu Inc. 3Carnegie Mellon Uni-
versity 4Peking University 5No Affiliation 6University College
London 7University of Bristol. Correspondence to: Mengyue Yang
<mengyue.yang@bristol.ac.uk>.
Proceedings of the 42 nd ...