Learning to Rank for In-Context Example Retrieval
Authors: Yuwen Ji
Luodan Zhang
Ambyerhan
Haoran Que
Lei Shi
Wang Chao
Yue Zhang
Venue: NeurIPS 2025 poster ()
Keywords: few-shot learning
in-context learning
Primary area: applications
TL;DR: The paper provides a novel approach to ICL example selection/ordering through the extension of DPO.

Abstract: Recent advances in retrieval-based in-context learning (ICL) train the retriever using a classification objective, which categorizes in-context examples (ICEs) into the most useful and the rest based on absolute scores. However, during inference, ICEs are retrieved by score ranking rather than classification  — The classification training objective deviates from this test scenario. Hence, in this paper, we propose a novel algorithm that trains a retrieval model by ranking formulation, where the preference rankings between ICEs are given by comparing the likelihood of the LLM generating the correct answer conditioned on each exemplar. By learning to rank, we motivate the retriever to automatically learn diverse rationales why specific examples are more useful for ICL decisions. This addresses the issue that classification models poorly capture broader utility. Experimental results demonstrate the top-1 performance of our proposal across 9 NLP tasks, with ablation studies and case studies further validating the effectiveness of our design. *The code can be found in: https://github.com/2022neo/SeDPO_NIPS25*

Learning to Rank for In-Context Example Retrieval
Yuwen Ji, 1.2+ 1i,
Luodan Zhang, 2-
Ambyer Han,
Haoran Que,*
Lei Shi,
Wang Chao,
Yue Zhang
Zhejiang University
Westlake University
Amap, Alibaba Group "Peking University 'Beihang University {zhangluodan, zhangyue, jiyuwen)@westllake edu.cn
Abstract
Recent advances retrieval-based in-context learning (ICL) train the retriever using classification objective, which categorizes in-context examples (ICEs) into the most useful and the rest based on absolute scores. However, during inference, ICEs are retrieved by score ranking rather than classification The classification training objective deviates from this test scenario. Hence, this paper, we propose novel algorithm that trains retrieval model by ranking formulation, where the preference rankings between ICEs are given by comparing the likelihood of the LLM generating the correct answer conditioned to rank, we motivate the retriever to automatically learn diverse rationales why specific examples more useful for ICL decisions. This addresses the issue that classification models poorly capture broader utility. Experimental results demonstrate the top-1 performance of our proposal across NLP tasks, with ablation studies and case studies further validating the effectiveness of our design. comm/22022no0/5600001122222
each exemplar. By learning
The code can be found
Introduction
Large Language Models (LLMs) 2 3 have shown their versatility addressing diverse problems through in-context learning (ICL), which be viewed few-shot learning. ICL
allows providing few in-context examples (ICEs) to guide LLMMs in generating predictions for test inputs without parameter updates. When there is large set of labeled data, selecting the most useful few examples can improve ICL performance. To this end, existing methods (8, 9 fine-tune LMs as dense retrievers, typically in two steps: (1) Scoring set of examples the ICL LLM; (2) Training the retriever model on scored data to align with the ICL LLM.
4
5 6
by one group by group using
Dominant approaches train the retriever using classification objective [10], categorizing ICEs into the most useful and the rest based on absolute scores. However, during inference, ICEs are retrieved by score ranking rather than classification The classification training objective deviates from this test scenario, which leads to limitations. Taking math problems as an example, when there are no reference answers for test query, recommending other useful ICEs, e.g., relevant formulas, will be more valuable. What we care about ultimately is not the absolute classification of examples, but the relative orders conditioned by the test input which sets of examples are more useful.
We formulate in-context example retrieval as an information retrieval (IR) task by adopting a learning- to-rank (LTR) objective. LTR has been widely used in information retrieval (Τ]. naturally capturing the broader utility across examples with ranking formulation. However, it remains underexplored in
"Equal contribution. "Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Sample small pool
Training retriever
Q Testing
LLM
P(
Xirain)
Corpus
LTR provides alter-
Imput
x
Learning to rank (ours)
native useful ICEs
2.9
2.7
0.1
repel
P(
Xtrain
When no "EJZ" class for X
LLM scored data
Pointwise classification
Figure I: Motivating framework of existing work and proposal (in red). E.g., when there reference answers (EJZ) for test input, LTR recommends alternative concepts or formulas (DEF).
ICL. As depicted in Figure ! using partial order knowledge between scored examples, retriever can retrieve broader range of useful examples with ranking formulation, improving ICL performance.
To train the retriever, we propose novel algorithm that aligns the preference ranking of ICEs, by comparing the likelihood of the LLM generating the correct answer conditioned each exemplar. By LTR, we motivate the retriever to automatically learn various rationales why ICES are more useful for ICL decisions. This addresses the issue that classification models poorly capture broader utility. To learn how to rank training examples, we adapt direct preference optimization (12) (DPO), integrating the Sequential Example relaxation [3], so derive trainable pairwise ranking objective, while implicitly conforming to global preference order constraints. We thus name this algorithm SeDPO.
Experimentally, method constantly ranks in top-1 across 9 NLP tasks, with an improvement of up to 18% over the best-performing glassffication-based baseline, achieving the SOTA results. Ablation study further confirms the usefulness of ranking formulation and its complementary strength existing paradigms. We summarize our key contributions as follows:
We propose to learn preference ranking orders for ICL example retrieval.
We introduce an extension of DPO for training retriever with pairwise ranking formulation.
We demonstrate that SeDPO significantly outperforms existing state-of-the-art retrieval methods for in-context example retrieval.
2
Related work
Retrieval-based ICL. Two primary types of retrievers commonly used for sample selection
in ICL.. The first type consists of off-the-shelf retrievers derived from heuristic criteria [1] 15] 16
24 25]. However, the assumption that model performance is always correlated with heuristic criteria is not reliable. Consequently, other approaches train a retriever on corpus using LLM's feedback as select ICEs that the LLM truly prefers. EPR (26) encodes and selects few-shot examples independently, with both queries and examples being encoded as vectors, enabling rapid retrieval through DPR (dense passage retrieval) (7 during inference. Following EPR's 's paradigm, CEIL 28 and UPRISE (29 are proposed to tackle various aspects of NLP tasks.
18 19 20
21
22
23
Observing that the above methods overlook the interaction of examples that are used together as one few-shot prompt, the latest advancements achieve the SOTA performance by modeling the sequential order between examples in each prompt, such RetICL 30 Active Example Selection (31, and Se as Markov Decision Processes (MDP) and optimize models through reinforcement learning (RL). However, they suffer from training instability. examples in each prompt directly on the input text, achieving closed-loop optimization and the SOTA performance, making it the most suitable baseline in this class. Neither of these methods considers ranking orders of examples across prompts, which we address in this paper.
[3]. Among these, RetICL and Active Example Selection formalize the problem
contrast, Se² models sequential order among

---

Recent ICL papers have considered aspects that are complementary to our work: (1) BESC 32 focuses on the internal ordering of ICEs within each prompt, using a contrastive loss to incrementally construct optimal sequences with dynamic lengths step-by-step. By contrast, our work emphasizes the ranking of different prompts. (2) CASE [33 prioritizes the efficiency of the selection process, framing it as a "top-m arm identification" problem with absolute training rewards. We, however, prioritize the quality of retrieved examples, formulating selection as a "learning to rank" problem with pairwise training rewards. (3) CLG [34] is a task-level selection method for few-shot scenarios, where scalability is critical-it selects a fixed set of examples for all test queries via gradient matching. Our work, by contrast, is optimized for few-shot scenarios where query-specific utility of each example is essential. We leave the exploration of these complementary aspects for future work.
Preference-aware ICL. Example retrieval methods considering the preference ranking orders of ICL examples are limited. UDR [55] additionally fits mini-batch rank indices with LambdaRank loss 36.. which may conflict with global rank indices, as inserting a new scored example between two originally scored examples alters the local rank indices. This discrepancy amplifies as the example pool scales, Another line is RL-ICL (37), which develops self-retrievable LLM with PPO [38 by modeling ICL performance as a reward signal. Due to differences in experimental and retrieval setups, we focus our comparison on mainstream retriever-based methods. A few ICL methods [39 40) consider the preference order as an evaluation metric. In contrast, our method can efficiently learn the ranking orders of scored examples over the entire corpus.
3 Method
We first formalize the problem of ICL for LLM (Section 3.1) and subsequently use it to label ranking data (Section 3.2). Finally, we use the ranked data to train one different retriever (Section 3.3).
3.1 In-context learning with sequential example retrieval
In-context learning [4]] is key capability of LLMs where the model learns from a few examples provided in the input prompt to perform a task without parameter updates. Following the previous definition [13 30 given a test sample (I, v). the LLM predicts ÿ based on examples and input x as:
i = LLM(eR @ c @ ... De 3 x.),
(I)
where (==(xx: Us ys)k=1 is an example drawn from corpus C, consisting of an input-output pair. K is the shot number and @ is the concatenation. The retrieval objective is to seek a set of examples in C for
test input v, putting them into a sequential order (ei,..., -1- ex) = [[I:]] following (42), aiming
make 9 match the label y. All candidate K-shot sequences are denoted as E {ElI:K] }. Notably,
we specify in Eq. (1) that ordered examples are input right-to-left. As long as the ICL templates remain consistent during training and inference, the left-to-right input order is also supported.
Retriever model. We consider a class of methods based on dense passage retrieval (DPR) as our retriever model. DPR consists of query encoder E-rqur()) and an example encoder Eexampte (-), often initialized with a pretrained text-encoder such as BERT-base-uncased 43 The retrieval score Orerriever of example e for test input x conditioned on is computed sim(e, c)) Fexample (e) Exmpur (c(Br) x)).
Retrieval process. Following [13, we use the same beam search during inference to retrieve ē[1:K] for fair comparison. We first encode and index all examples eg E C using the trained Ee (-). Given a test input x, we encode with Einpar()) and retrieve examples with the highest retrieval scores E Fexample( (.) Fampun (-). We set default beam size = 3, the same as in [13] We also draw = 1 for brief illustration in Figure Cc.. These examples are then concatenated with the current inputs as new context sequences. The retriever scores accumulated into the sequences' scores. This process is repeated, encoding inputs and secking examples to maintain the w highest scoring candidate sequences until each sequence contains K examples. The sequence with highest accumulated scores is chosen as e[I,k- Motivated by left-to-right generation in autoregressive models, this retrieval process lets later retrieved ICES observe previous retrieved ones, being aware of sequential relationships.
Training task. To convert LTR for ICL into a formal goal, we design our training task as follows: (1) A scoring function éLIM takes (e[1:K] ; Ir, y) as input and evaluates the ICL performance for each [[1:K] E E through L.M.. In this way, we obtain an ICL. performance ranking conditioned on æ. (2) Since the true answer y of test input x is unavailable during inference, another scoring function
3

---

(a) Prepare ranking orders for training
(b) Train on ranking orders
(c) Inference w/ retriever
Sequential retrieval
LLM
X
Given
I.LM
Input embedding
Retriever
* Inner product
LLM
Corpus embeddings
Policy ranking orders
Policy update
Sampled examples
Sequentially score and rank with I.I.M
Ground truth ranking orders
ScDPO loss
Figure 2:
(a) Prepare preference data D. (b) Train the retriever on D with SeDPO loss to align with partial order instead the top-scored. (c) Inference with dense passage retrieval (DPR).
Prewiever is introduced, which takes (e(u:)).2) x) as input to score each eji:k] E E. This produces a retriever-based ranking conditioned on x. (3) To ensure that higher-scored retrieved e[I:K] can lead to better ICL performance, our goal is to train the retriever by aligning the Pretrievr--ased -aased ranking with the duaM-based ranking. Notably, previous methods widely use classification objective, categorizing [[1:K] with the best ICL. performance as positive, and the rest as negative, which is unlike our goal.
3.2 Labeling data with ranking orders
Definition of ranking orders. We focus on Multiple-Choice Question (MCQ) tasks 44 by default. MCQ is a question format in which respondents (i.e., LLM) are asked select only the correct answers from the choices offered as a list. How much the LLM's prediction ý matches the ground truth y E g can be measured by a task-specific scoring function S(; :), which can be formulated:
LH y [e]:k]] E x))
ΦLLM( x, y) = Sxcq(quıR) ① @x, 22 Ev LH(y'len.x) @ 0z)
(2)
where )' or P'r is outpur/grround-ruuh label space, LH is per token conditional likelihood of the LLM. For brevity, we denote [[1:k] @ x = (ex @ ... @ e @ x) w.r.t Eq. (T). As higher scores indicate that LLMs are more likely to output grouhd-truth answers, we can define partial order for ranking as 11 > <1:xq]= i if Sxcq(e @ x, 3) > SMcq(e'1.K) @ x, y) for all E Vet given æ. The partial order can be a total order by weighted aggregating Suaq for all y E Ygr but we focus on the partial order domain knowledge isn't required. In scored MCQ data, there may be multiple or no examples that guide LLMs in predicting correctly, requiring exploration beyond top-scored examples.
Scored data construction. Since the corpus C can form (ICI))((CII K)! possible en-k]. scoring all [[] would be computationally prohibitive. Following [13,, we employ a greedy algorithm to selectively score the data for fair comparison, ultimately obtaining sequentially scored data D. The constructed queries in D are marked by tilde. For each (z, y) E C, we denote the test input æ without ICE as To. and sample L examples from corpus C as B(r)). We score the examples in B(ū,) with frozen LLM and Eq. ( and repeatedly resample the examples in B(ūg) that cannot be ranked. Finally, we select an example from scored set ((ūa)) as c, based its rank:
exp(-rank)
p(rank)
1rank exp(-rank)
from which the higher-scored example is more likely to be selected and the diversity is preserved.
fx+1 c. ① T. is iteratively updated for next round scoring, until the Khshot data are all constructed.
The L scored examples in each B(Tx) are gathered with corresponding 7e to form D (Fig. 2 (a)).
Ranked data construction. Given Zy, we have L scored examples, forming (b) pairs of partial order. Empirically, we consider that two types of preference play a dominant role: (a) Given (x, y). examples in different pairs are non-overlapping, so to enhance the diversity of training data. (b) " and e have a larger score margin, where their discrepancy is easier to learn. Therefore, we select T' examples with the highest scores as the preferred ones and randomly match them with bottom-T dispreferred candidates, to construct sequential preference data D out of D. We discuss it in ablation.

---

3.3 Algorithm for learning orders
We devise a novel RL algorithm for training a retriever, as directly applying existing RL algorithms such as DPO [1] on ranked data (Sec. (.2) is prohibitive: (1) DPO requires knowing the probability distribution of specific retrieval actions over the entire corpus (i.e.. policy), but the retriever model only outputs retrieval scores for specific ICEs; (2) optimizing retrieval actions over the entire corpus is very expensive and hard to scale. We innovatively address these challenges with (a) reparameterization of
retriever score into policy model and (b) sequential relaxation of e E [[1:K] in this section.
Policy of retriever model. To train a retriever with ranked data using DPO, we must convert the retriever score into policy, to model the probability that retrieved ICES are ranked in the order of ICL performance. We denote C {eiJick the corpus excluding the selected ICEs. Based on retrieval scores ++)),, the policy π of selecting en:k] from all candidates without replacement is:
exp(sim (e()))))) = II E LeleCtfes)cce exp(simme (s. (lfeite)))
K
(4)
k=1
Direct preference optimization (DPO) [2] is one of the most popular RL methods for preference ranking orders modeling. Instead of learning explicit reward model, DPO reparameterizes the reward function r using a closed-form expression with the optimal policy. By initializing policy model xo and reference model Tref with Eq. (4), the reward function is as follows
r(z, (l1:k) = 3 log [e((n=)]1=) + 3 log ((x)
(5)
Tref (ee:(((1
where 3 controls the KL-divergence constraint policylreference models, and Z(x) is the partition function [12]. By incorporating this reward formulation into the Bradley Terry ranking objective, P(efii]] > f/1x]/zz) (r((z, fi.k]) r(z, <(]x))), DPO expresses the probability of preference
data with the policy model, yielding the following loss for triple (2r, (r,e(iix) (i. (Ux)))
Traf (e (x) 3 log To To(c (x)) )
(e
SoPo(ToiTreel = - (( log
( [Bog
(6)
where fii.]] and efa.x; denote the preferred and dispreferred prompts conditioned by input æ. During training, we update the mo while freezing Tref- By plugging Eq. (4) into Eq. (6), we have:
LDPO Tres) -a (aa [Iogo a (B. folæ, (B-folz, (ix)) B fo(2, o c(2 B (Yw mi)
fo et
((1n)))
ss (e())ck))) am (1 (c/)
j je (w,1)
Ee Eoceere//ie exp(simo(e.)). a (c) (<()))
log
i E (w.1)
1=1 Ee.ec exp(sim.or hie)))
However, the estimation of the policy denominator (i.e., 7) necessitates repeated re-embedding of the entire corpus for each sampling step. This representation results in prohibitively high computational overhead. To tackle this, we consider relaxation by modeling the sequential relation among ex within the same prompt C[:,k), motivated by left-to-right generation in autoregressive models.
Sequential LLM preference alignment. Under the sequential assumption, the ICL, performance of the prompt given retrieved e solely influenced by the performance of the retrieved ICEs, i.e., Cick- Therefore, the optimal ex E [[I:k] can be explored sequentially:
max SMcQ @ e el + vx v) + {exl max Swcooee @ + @ K {
(8)
from which sequential preference data D can be constructed from D as:
{(ee > e .... @ z U ((e" > + = {(e) > 1
1K
(9)
5

---

where the conditional tern is abbreviated as žg-1- Based on this, the policy can be factorized sequentially into T(q I * (ecx/#x-1) 1). Here, x (< ( is as follows:
exp(sim(ex,æx_ (R-1))
((=)) E Ee,echfesiic exp(sim 1
(10)
by plugging Eq. (10) into Eq. (6). obtain same form Eq. with Tor -
CSE.DPO(TO: mos) = (5n 1.8 ADD [log (3- fo(ūx 6x) 8 fo - fe( eg) = simo (cx Tx-1) sim,er (c x2-1), j (w,
30]],
(11)
The new objective function is tractable by implicitly considering the global partition function (i.e., the denominator in Eq. (10).. Training signals (c.g.. ex .e() can be readily generated from scored data. We train the retriever on D with Eq. [1). The resulting retriever discern the preferred ICEs for varying context input. Note that the partial order learned by our bi-encoder satisfies the transitivity: ea e lx: and e² ea1ar then e > el|,, ensuring our retriever inkinn-informative. The proof of the transitivity can be found in Appendices.
4 Experiments
We take the SOTA method Se 13 our base, which is classification- based method, and implement SeDPO on top of it. We first compare the ICL performance with SOTA retrievers (main results); validate key components in SeDPO (ablation); then provide extra experiments for deeper analysis.
4.1 Experimental settings
Task and dataset. We use total of 9 tasks across 4 distinct categories, including Paraphrase: MRPC [45), PAWS [46), QQP 47 Coreference: WSC [48]; Reading: MultiRC [49], BoolQ (50),
AGNews [5T]: NLI: MNLI-m/mm (52). The preprocessing and evaluation for all datasets the same as Se The description of each dataset is provided in the appendices.
Implementation details. For fair comparison, we follow the hyperparameter setting of Se" and use GPT-Neo-2.7B (53) the scoring and inference LLM for most of the experiments. The encoders in retriever model are initialized with "BERT-bbaseuncasee
43 The shot number = 3. We set T = 20, ensuring that D is sourced from the same training data for fairness; retriever is fine-tuned for 6 epochs each category. the best checkpoint is chosen based on retrieval accuracy of validation set, and evaluated using task-specific metric on the test set. Refer appendices for more details.
Additionally, Sea incorporates two data augmentation techniques: (1) positive chosen Se reserves only the data where the selected representative examples can guide LLM in correctly predicting: (2) in-batch rejection - in each training batch, the negative sample set for each input is extended with examples from other inputs for diversity. Following Se?, we reserve the data where e" guides LLM correctly predicting; for each input, the rejected instance uniformly sampled from combined set of original el and examples from other inputs. We analyse the techniques in ablation study.
Baselines. All model comparisons are fair using the size, detailed in the appendices. We repeat each algorithm 10 times and report the average performance. The off-the-shelf baselines include: Random involves sampling K demonstrations randomly: B.M25 (1 is the commonly used sparse retriever that finds exemplars based on textual similarity; SBERT 2 a dense retriever by computing sentence embedding, we take "paraphrase- waaeee--mneeeettteteeeeeee as its encoder.
For fair comparison, we re-trained the learning-basee basclines aligning our task settings (c.g., with the same scored dataset): UPRISE [29] estimate the usefulness of each example separately: Se² [3] improves upon UPRISE by sequentially retrieving representative examples, making it the most competitive and accessible alternative for comparison; UDR (35) uses LambdaRank loss to inject ranking information, while estimating the usefulness of each example separately. We also study the integration of UDR's 's LambdaRank loss with Se² in the ablation experiments.

---

Table 1: Main results various tasks. The best results and the second-best highlighted. The
Avg. of all metrics for tasks within the same category with significant improvements is marked by f.
Paraphrase
Coreference
MRPC
PAWS acc
QQP
WSC
Avg.
ncc
acc
acc/Arg.
Zeroshot
46.11:0.0
45.3
51.810.0
48,4:1000
42.110.0
46.7=0.0
59.610.0
Random
66.8+13.0
79.514.1
50.113.8
40.6:14.8
50,917.8
57.623.8
48.318.2
BM25
$7.8:10.0
69.110.0
48.910.0
54.810..0
55.410.0
$7.2+0.00
52.410.0
SBERT
56.4+-0.0
66.9+0.00
49.4+0.0
51.2++0.0
56.24+0.0
56.0+=0.0
46.2+0.0
UDR
65.911..6
75.41:..5
51.811.2
74.1=11
67.912.44
67.0+1.2
52.014.7
UPRISE
74.0+10.8
83.340.1
49.120.0
71.0111.
69.810.11
69.410.2
46.5+2.2
Se?
77.6:t0.4
85,410.3
54.710.1
75.5#0.1
72.8+0.0
73.2+00.
55.1
ScDPO
7..910.9
85.6
73.0
77.62:.66
75.010.2
77.9-0.6
62.5=0.2
Reading
Natural Language Inference (NLI)
MultiCC
BoolQ
AGNews
MNLI-m
MNLI-mm
Avg.
Avg.
fl
Zeroshot
57.111.00
54.610.0
38.410.0
50.010.0
35.2-0.0
36,410.00
35.810.0
Random
57.7122.5
54.8.6.7
25.8.11.1
46.1
34.2.3.00
34.913.9
34.611..6
BM25
46.5110.0
60.310.0
81.710.0
62.810.0
35.3-0.0
35.611..0
35.5.1.00
SBERT
49.3=#0.0
58.1=0.0
84.7+0.0
64.0#t0.0
37.3=0.0
37.3+0.0
37.3=0.0
UDR
55.3++3.1
54.6=1.9
88.5+1.0
66.1+009
62.7=1.5
65.0+t1.3
63.8+1.4
UPRISE
55.4.10.2
61.510.1
90.6.10.8
69.2100.1
68.5-0.1
70.3.0..3
69.410.2
Se²
52.1::2.3
63.620.2
90.8110.3
68.8100.7
69.4-0.2
70.410.11
69.910.2
SeDPO
60.3:10.4
6..611.7
91.0110.2
72.010.6'
70.6_0.1
72.0
71.3:10.2
4.2
Main results
Table details the experimental results on MCQ tasks, where generative LLMs are known to need improvement [29]. On each task, mark the best results in bold and underline the second-best. The Avg. column represents the mean performance for each category, with significant improvements marked by 1 (confidence level is 99%) over the best alternative. SeDPO outperforms all other methods in all categories with an avg improvement of up to 4.7%, and we also have several findings.
First, random sampling does not lead to sizable gains compared to zeroshot. In contrast, BM25 and SBERT have significant gain over random sampling and zeroshot. This demonstrates the necessity of providing related examples for LLMs in downstream tasks. In addition, finetuning- based retrievers perform better than off-the-shelf retrievers, highlighting the effectiveness using LLM feedback.
Second, among the finetuning-based retrievers, Se* outperforms UPRISE by using sequential rela- tionships between examples, showing the importance of capturing sequential relationships between examples. Though UDR considers local ranking regularization, the performance gain is limited on average compared to Se². In particular, our proposed SeDPO achieves the best performance in all categories, significantly exceeding Se² by at least 1.4% to maximum of 7, 7.4%. This demonstrates that learning preference orders rather than the representative examples, significantly enhances the ICL performance on MCQ tasks and complements previous advancements.
Third, for downstream tasks such PAWS. WSC, and MultiRC, SeDPO achieves substantial improvement of up to 18.3%. We speculate that retrievers trained to learn representative patterns fail capture the true preferences LLMs on challenging data, they show significant improvement compared to zero-shot. Particularly, Se² underperforms random selection in both MultiRC and WSC tasks, indicating that these tasks require the capture of more diverse query-dependent ICE patterns. Se² learns from top-scored ICEs, thus insufficient to generalize to test input. In contrast, SeDPO consistently outperforms random selection by 2.9% to 3.9% and surpasses Se by 7.4% 8.2%. This demonstrates the superiority of our proposal in challenging tasks.
4.3 Ablation studies
We compare original framework with its variants altering each time different component.
7

---

Table 2: Ablation results Paraphrase task
using GPT-Neo-2.7BB, trained on 3-shot.
8
80
Sea
Paraphrase
70
Avg. 65065
77.9
w/o positive chosen
77.9
60
w/ top-1 chosen
70.8
55
in-batch rejection
73,9
50
w/ random preference
57.6
0.03
B (Se-DPO)
w/ LambdaRank (UDR)
72.9
w/ RoBERTa
85.7
Figure 3: Performance of SeDPO on Paraphrase category using GPT-Neo-2.7B with different B. The dashed line represents the results of Se".
ScDPO (8 = 0.02) o Se
79.0
Se² ScDPO (3 0.02)
74.8
Table 3:
The average textual/semantic diversity of selected ICEs, as well as the average performance when the input order of ICEs randomized. We take the main results on Paraphrase as our base.
SeDPO
Se*
UPRISE
UDR
SBERT
BM25
Random
Textual Diversity
53.3%
49.0%
46.7%
54.3%
49.7%
46.0%
61.4%
Semantic Diversity
40.7%
39.0%
37.3%
40.3%
25.3%
29.0%
46.0%
Random order (Best of 5)
78.2%
73.5%
70.9%
68.4%
57.3%
58.1%
Random order (Worst of 5)
77.1%
72.5%
68.6%
66.3%
55.1%
57.0%
Complementary strength. We test the complementary strength SeDPO and Se". In Table 2 SeDPO (5=0.02) o Se² denotes initializing the retriever's weights of Se with weights of trained SeDPO, and vice versa. Training Se² model using SeDPO's s trained weights can enhance ICL performance. We surmise that the greedy data construction may not be globally optimal for fully learning preference ranking order, there is still room for improvement. Conversely, initializing ScDPO with Se* weights leads suboptimal results. This asymmetry likely arises from ScDPO's inherent KL-divergence constraint, which preserves the retriever's pretrained knowledge base. Se may overfit to top-scored, hindering SeDPO's ability to use the retriever's pretrained knowledge.
Effect of components. As shown in Table 2 replacing our original preference data with randomly selected pairs of the same size led poor results (w/ random preference). Since T < (9). random local observations struggle to learn reliable partial orderings. Furthermore, using local representative examples (w/ top-1 chosen) the only chosen in preference data and pairing them randomly with bottom-T examples, while performing better than random selection, still falls short of our original design. This demonstrates the effectiveness of reserving diverse (e" e') with a larger
margin.
Notably, we also investigate the impact of the two data augmentation techniques used in Se² (detailed in Sec. [...p. The positive chosen does not enhance our results, indicating that learning orders rely more on the discrepancy between chosen/rejected samples rather than the absolute quality of the chosen ones. In contrast, in-batch rejection increases data diversity and improves our results by 4%.
Influence of embeddings. This paper uses "BERT-base "eser--aae-unccsed as the encoder model for fair comparison. Nowadays there many better text embedding models available. To further show the influence the embeddings. we replaced the retriever backbone of SeDPO with "RoBERTa-base" (54), which known outperform "BERRT-aas-unncaeed" results are shown as w/ RoBERTa in Table
[4] across range of NLP tasks. The SeDPO benefits from stronger embeddings as expected.
Diversity of retrieved examples. Table presents the diversity metries corresponding to the main results Table [ where higher values indicate lower qimrl-similarity and a richer context of retrieved ICEs. Both ScDPO and UDR achieve high diversity in selected ICEs, exceeding other retrievers (except random) by at least 3.6% in textual/semantic diversity. This demonstrates that incorporating ranking signals enhances diversity. The poor ICL performance of randomly selected ICES highlights the importance of selecting relevant ICEs. In addition, though UDR's 's diversity comparable to SeDPO's. s, its ICL performance lags behind Se² and UPRISE. This indicates that ScDPO better trades off diversity with ICL utility and successfully retrieves diverse yet useful ICEs.

---

Table Transferability on shot number and model size. The average performance Paraphrase.
Inference Model
Method
1-shot
3-shot
6-shot
9-shot
12-shot
15-shot
Average
BM25
57.6
58.5
58.8
58.7
59.3
60.1
58.8
SBERT
57.9
57.5
59.6
58.6
58.3
58.5
GPT-2-XL-115B (0-shot=39.6)
UPRISE
69.2
69.4
69 .8
69.8
70.0
70.2
69.7
Sea
73.9
72.9
72.9
72.8
72.8
72.7
73.0
ScDPO
75.0
78.9
79.5
79.2
79.0
79.2
78.5
BM25
57.1
57.2
58.9
59.5
59.0
59.4
58.5
SBERT
56.6
56.0
59.4
58.9
59.8
58.4
58.2
GPT-Neo-2.7B
UPRISE
69.4
69.7
69.5
69.2
69.2
69.3
69.4
(0-shot=46.7)
Sea
73.5
73.2
73.1
73.0
72.8
72.6
73.0
SeDPO
77.6
77.9
78.0
77.9
78.2
78.1
78.0
BM25
68.6
73.2
74.7
75.1
75.6
76.7
74.0
SBERT
68.3
73.0
75.1
75.4
76.1
73.5
Llama3-8B-Instru
UPRISE
70.9
75.3
76.4
76.6
76.9
77.0
75.5
(0-shot=56.4)
Sea
71.9
76.7
78.0
78.0
77.9
77.9
76.7
SeDPO
71.9
77.4
78.5
79.3
80.2
80.3
77.9
BM25
78.4
80.7
82.2
81.7
81.8
82.2
81.2
SBERT
78.7
80.3
81.2
81.7
81.7
82.7
81.1
Llama3.3-708
UPRISE
77.3
80.2
81.3
80.5
80.8
81.3
80.2
(0-shot=67.0) 67.6)
Sea
81.0
82.2
81.9
81.9
81.1
SeDPO
78.6
81.0
82.3
82.9
83.2
83.2
81.9
Impact of ICE ordering. We empirically analyze the impact of randomizing the input order of ICE on paraphrase performance. The results in Table suggest that randomizing the ICE order exhibits limited potential for enhancing sequential approaches. We attribute this that SeDPO and Se² already optimize the input order of ICEs to extent by sequential example selection.
Impact of (. Table 3 shows the Paraphrase performance of ScDPO using GPT-Neo-2.7B with different B. Too large too small | can lead to dominant negligible constraint of KL divergence, resulting in performance degradation. SeDPO shows promising improvements when /3 ranges from tune between 0.001 and 2 across different categories.
0.01 to 0.1,
Impact T. Table in Appendices shows the extra ablation results for different T, following the setup of Table 2 As increases, the performance of SEDPO improves because a broader preference ranking learned. However, when 7 too large, the performance gain decreases due to more low-confidence L.M rankings, leading to the same conclusion as in Table (w/ random preference).
4.4 Analysis
Transferability. As LLM scales, aligning the LLM preference under different shot numbers K of ICES is time-consuming and resource-intensive. We thus explore the effectiveness of retrievers, as LLMs and example numbers vary. Specifically, in our main experiments (Table [T. we trained retrievers for each category using 3-shot data and GPT-Neo-2277. We then evaluated these retrievers in unseen inference settings, where the varied from to 15, and the inference L.LMs included and I.lama.3-700 (2). Table illustrates that ScDPO consistently outperforms baseline retrievers in settings. Notably, though ScDPO is trained on 3-shot, its performance improves as the K increases and significantly outperforms 0-shot by at least 10% on all LLMs. We also find that as the model size LLMs increases, the gap between different ICL methods decreases. This indicates that larger LLMs are smarter and can suboptimal examples; on the other hand, smaller LLMs (not that smart) rely more on high-quality examples, is also mentioned in [56]. However, the in-context examples provided by SeDPO are useful for both small and large L.Mss, demonstrating strong transferability.
GPT-2-XI. [55] G.TT-Neo-2.7B,
using
Math inference. We compare ScDPO and Se² with several reasoning- retrievers 57 on AUQA [59], MCQ task requiring math inference. Table 9 in Appendices shows that SeDPO outperforms Se²., aligning with our main discovery. Though RGER [57] built for sessoninn--fcussee
58 28
tasks, SeDPO surpasses RGER [57 without being geared toward reasoning capabilities.
More in-depth analysis. further study the performance boundaries SeDPO, we provide extra results in Appendices, including O-shot performance human-labeled data, impact of preference

---

0-shot Test Input 1
0-shot Test Input
Please tell me those questions are the same. Why rightt "right" and wrong is right and what is wrong?
Do these two sentences mean the same thing? the 2002 study, the margin of error ranged from 4.4 percentage points, hts margin of of of minus three to four percentage points.
Yes
Se"
Se'
Example1: What are some mind blowing drones that most people don't 't know about? What
Example1: The benchmark 10-year note US10YT + RR rose 11 32 for a yield of percent. The benchmank 10-year Bund yield down 5.2 basis points at 3.50 percent. No
mind blowing drones that most people don't 't know? Yes Example2: Which are the best for the ultimate bass, within Rs. 1000? Which the Best earphones under Rs 10007 Yes
best earphones
Example2: The Nasdaq composite indexincreased 10.73, or percent, to 1,514.77. The Nasdaa Composite inder, full of technology stocks, was falefy up around 18 points. No.
Example3: When was the time you true happiness? And for what reason? When was the last time you felt happy?Because can't remeber. Yes
Americans eager for service, whicch intended to block about 80 percent of telemarketing calls, The free service was originally intended to block about 80 percent of telemarketer calls, No
Example3: He urged patience
Yes
No
ScDPO
SeDPO
Example1: Which is the hest camera smartphone
Example1: Romanian port authorities said two ships that sank during World War had resurfaced water
under 20k India? Which best carnera
river traffic. authorities in Romania said two ships that sank during World War it but resurfaced due low water levels could block river traffic. Yes
and
smartphone under 15k india? No
Example2: What is the best Android smertphone for under $4007 Which is the best Android smartphone under 1800?? No
Example2: Defence lawyers sought
those charges tnrown claiming
the Ball court had no jurisdiction over those crimes, Defonce lawyers saught to have those charges thrown out, claiming the Bali court frearing Mr Samadra's did not have jurisdiction over those crimes, Yes.
Example3: Which has better exclusives, PS4 Xbox One? As of March 2014 which is the better gaming console on the market, Sony's 's P54 Micresoft's Xbox
Example3: In 2001, the number of death row inmetes nationally fell. for the first time in generation. 2001, the number people on death row dropped for the first time decade. Yes
No
Yes
Figure 4: Two cases on Paraphrase where SeDPO helps LLM correctly infer, but Sea does not.
dataset construction using various LLMs, detailed math inference, analysis of full list order, proof of transitivity, cost of constructing training data, discussion of Eq. (2) in open-ended QA setting.
Case study. In Figure 4 we analyze two cases from the Paraphrase, to intuitively compare the effectiveness of ScDPO and Se". The answers of the examples are marked with underscores. Specifi- cally, the task focuses on whether two sentences in the test input are synonymous. Se²'s examples emphasize surface patterns: in the left case, the two sentences have similar word compositions; in the right case, there are changes in percentage numbers. In contrast, SeDPO captures the taskerelated nuances preferred by LLMs: it selects examples based on the extent of difference between the two sentences and marginalizes causal information using broader contexts such as historical, legal, and social. This demonstrates how learning preference ranking orders gives broader causal relationship between examples, which improves the ICL performance. More cases can be found in the appendices.
5 Conclusion
We considered learning to rank for in-context example retrieval, introducing ScDPO, simple yet effective method. Unlike dominant methods that focus on representative examples, SeDPO captures the global preference orders through a pairwise ranking formulation. We effectively address the issue that classification bessi retrievers poorly capture broader utility. Extensive experiments demonstrate superiority. Additional experiments, discussions, and proofs appear in the Appendices.
Limitations. Our research focuses on improving retriever training but shares existing frameworks' structural limitations. First, we mainly use Ne7--Neo-2.7B for fair comparison, where shot number analysis is constrained by sequence length: this can be improved by recent input/prompt compression [60]. Second, results are affected by inherent biases [61] in retriever models and LLMs, requiring fair and interpretable strategies (a promising direction). Third, permutation-be based example retrieval is underexplored. Notably, SeDPO needs dominant preference identification to reduce computational costs and redundant interference; while clear for MCQ tasks, open-ended question challenges remain.
Broader Impacts. Learning-to-ram for ICE retrieval boosts LLM's ICL performance but carries negative societal risks. Without safeguards, it may retrieve/prioritiee biased, misleading, or harmful reinforcnnii unfair decisions (e.g., employment/legal consultation) or spreading disinfor- mation-and could be misused to get adversarial examples manipulating L.MMs. Mitigation requires rigorous fairness audits, retrieved example filters, and controlled access for high-stakes deployments.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by the National Natural Science Foundation China (NSFC) Key Program under Grant Number 62336006, the National Key R&D Program of China (202IYFB3500700) and the Beijing Science and Technology Plan Project.
References
[1] Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. Large language models: A survey. arXiv preprint wrXiv:2402.06190 0699, 2024.
[2] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Daalle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The Ilama 3 herd of models. arXiv preprint arXiv:2407221783 2024.
[3] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint 144497 2024.
[4] Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. Chain-of of-thought prompting elicits reasoning in large language models. Advances neural information processing systems, 35:24824-24888
2022.
[5] Siyun Zhao, Yuqing Yang, Zilong Wang, Zhiyuan He, Luna K Qiu, and Lili Qiu. Retrieval augmented generation (rag) and beyond: A comprehensive survey on how to make your llms use external data more wisely. arXiv preprini urXiv:2409. 14924, 2024.
[6] Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, et al. A survey on in-context learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1107-1128, 2024.
[7] Rishabh Agarwal, Avi Singh, Lei Zhang, Bernd Bohnet, Luis Rosias, Stephanie Chan, Biao Zhang, Ankesh Anand, Zaheer Abbas, Azade Nova, et al. Many-shot in-context learning. Advances in Neural Information Processing Systems, 37:76930-76666, 2025.
[8] Shangyu Wu, Ying Xiong, Yufei Cui, Haolun Wu, Can Chen, Ye Yuan, Lianming Huang. Xue Liu, Tei-Wei Kuo, Nan Guan, et al. Retreeval-augmented generation for natural language processing: A survey. arXiv preprint arXiv:2407. 13193, 2024.
[9] Alon Albalak, Yanai Elazar, Sang Michael Xie, Shayne Longpre. Nathan Lambert, Xinyi Wang, Niklas Muennighoff, Bairu Hou, Liangming Pan, Haewon Jeong, al. survey data selection for language models. arXiv preprint arXiv:2402. 16827, 2024.
[10] Man Luo, Xin Xu, Yue Liu, Panupong Pasupat, and Mehran Kazemi. In-context learning with retrieved demonstrations for language models: A survey. arXiv preprint arXiv:2401.11624 2024.
[11] Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Zhicheng Dou, and Ji-Rong Wen. Large language models for information retrieval: A survey. arXiv preprint 2xXv:22308.07107, 2023.
[12] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
[13) Haoyu Liu, Jianfeng Liu, Shaohan Huang, Yuefeng Zhan, Hao Sun, Weiwei Deng, Furu Wei, and Qi Zhang. sk2: Sequential example selection for in-context learning. arXiv preprint a40iv:2402. /3874, 2024.
[14] Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786, 2021.
11

---

[15] Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, and Lingpeng Kong. Self-adaptive in- in-context learning: An information compression perspective for in-context example selection and ordering. arXiv preprin! arXiv:2212. 10375, 2022.
[16] Xi Ye, Srinivasan Iyer, Asli Celikyilmaz, Ves Stoyanov, Greg Durrett, and Ramakanth Pasunuru. Complementary explanations for effective in-context learning, arXiv preprinl 1.Xiv:2211113892
2022.
[17] Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, et al. Selective annotation makes language models better few-shot learners. arXiv preprint arXiv:2209.0 02977999999. 2022.
[18] Tai Nguyen and Eric Wong. In-context example selection with influences. arXiv preprint arXiv:2302. 1042, 2023.
[19] Yanda Chen, Chen Zhao, Zhou Yu, Kathleen McKeown, and He He. On the relation between sensitivity and accuracy in in-context learning. arXiv preprint (xXiv:2299006611 2022.
[20] Shizhe Diao, Pengcheng Wang, Yong Lin, Rui Pan, Xiang Liu, and Tong Zhang. Active prompting with chain-of-thought for large language models. arXiv preprint 12X1v:2302.12246, 2023.
[21] Stephen Robertson, Hugo Zaragoza, et al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends@ in Information Retrieval, 3(4):333-333 2009.
[22] Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke Zettlemoyer, and Marjan Ghazvininejad. In-context examples selection for machine translation. arXiv preprint arXiv:2212.02437, 2022.
[23] N Reimers. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint 1908 10084, 2019.
[24] Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A Smith, Luke Zettlemoyer, and Tao Yu. One embedder, any task: cntrucctionfinetunee text embeddings. arXiv preprint arXiv22212.09741, 2022.
[25] Young-Jun Lee, Chae-Gyyn Lim, and Ho-Jin Choi. Does gpt-3 generate empathetic dialogues? a novel in-context example selection method and automatic evaluation metric for empathetic dialogue generation. In Proceedings of the 29th International Conference Computational Linguistics, pages 669-683, 2022.
[26] Ohad Rubin, Jonathan Herzig, and Jonathan Berant. Learning to retrieve prompts for in-context learning. arXiv preprint arXiv:21 2021.
[27] Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. arXiv preprint 200::20000004906, 2020.
[28] Jiacheng Yc, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong. Compositional ex- emplars for in-context learning. In International Conference on Machine Learning, pages 39818-39833 PMLR, 2023.
[29] Daixuan Cheng, Shaohan Huang, Junyu Bi, Yuefeng Zhan, Jianfeng Liu, Yujing Wang, Hao Sun, Furu Wei, Denvy Deng, and Qi Zhang. Uprise: Universal prompt retrieval for improving zero-shot evaluation. arXiv preprint arXiv:2303.088 085188 2023.
[30] Alexander Scarlatos and Andrew Lan. Reticl: Sequential retrieval of in-context examples with reinforcement learning, arXiv preprint arxiv:2305. 2023.
[31] Yiming Zhang, Shi Feng, and Chenhao Tan. Active example selection for in-context learning. arXiv preprint 2..iv:221..04486, 2022.
[32] Xiang Gao, Ankita Sinha, and Kamalika Das. Learning to search effective example sequences for in-context learning. arXiv preprint arXiv:2503.0800 2025.
12

---

[33] Kiran Purohit, V Venktesh, Sourangshu Bhattacharya, and Avishek Anand. Sample efficient demonstration selection for in-context learning. arXiv preprint arXiv:2506.08607, 2025.
[34] Jianfei Zhang, Bei Li, Jun Bai, Rumei Li, Yanmeng Wang, Chenghua Lin, and Wenge Rong. Selecting demonstrations for many-shot in-context learning via gradient matching. arXiv preprint arXiv:2506. 04579, 2025.
[35] Xiaonan Li, Kai L.v, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, and Xipeng Qiu. Unified demonstration retriever for in-context learning. arXiv preprint arXiv:2305.04520, 2023.
[36]
Christopher JC Burges. From ranknet to lambdarank to lambdamart: An overview. Learning, 1 11(23-581):11,29 2010.
[37] Quanyu Long, Jianda Chen, Wenya Wang, and Sinno Jialin Pan. Large language models know what makes exemplary contexts. arXiv preprint arXiv:2408.07500 2024.
[38] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint S.Xiv:1777.063477 2017.
[39] Marie Al Ghossein, Emile Contal, and Alexandre Robicquet. Iclerb: In-context learning embedding and reranker benchmark. arXiv preprint arXiv:2411. 2024.
[40] Feifan Song. Yuxuan Fan, Xin Zhang, Peiyi Wang, and Houfeng Wang. ledpo: Effectively borrowing alignment capability of others via in-context direct preference optimization. arXiv preprint arXiv:2402.099.3 2024.
[41] Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning. arXiv preprint arXiv:2301.00234 2022.
[42] Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering. arXiv preprint arXiv:2007.01282. 2020.
[43] Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume I (long and short papers), pages 4171-41888 2019.
[44] Nikitas Karanikolas, Eirini Manga, Nikoletta Samaridi, Eleni Tousidou, and Michael Vassi- lakopoulos. Large language models versus natural language understanding and generation. In Proceedings of the 27th PCI '23, page 7778299, New York, NY, USA, 2024. Association for Computing Machinery.
Conference on Progress in Computing and Informatics,
[45] William B. Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP2005), 2005.
[46] Yuan Zhang, Jason Baldridge, and Luheng He. PAWS: Paraphrase adversaries from word scrambling. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1298- 1308, Minncapolis, Minnesota, June 2019. Association for Computational Linguisties.
[47] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Tal Linzen, Grzegorz Chrupala, and Afra Alishahi, editors, Proceedings of the 2018 EMNLP Workshop BlackboxNIPP: Analyzing and Interpreting Neural Networks for NIP, pages 353-355, Brussels, Belgium, November 2018. Association for Computational Linguistics.
[48] Pengcheng He, Xiaodong Liu, Weizhu Chen, and Jianfeng Gao. A hybrid neural network model for commonsense reasoning. Simon Ostermann, Sheng Zhang, Michael Roth, and Peter Clark, editors, Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing, pages 13-21, Hong Kong, China, November 2019, Association for Computational Linguistics.
13

---

[49] Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. Looking beyond the surface: A challenge set for reading comprehension over multiple sentences. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume I (Long Papers), pages 252-262,, 2018.
[50] Christopher Clark, Kenton Lee, Ming- Ming-WWii Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. In NAACL, 2019.
[51] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015.
[52] Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Marilyn Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pupers), pages 1112-1122, New Orleans, Louisiana, June 2018. 8. Association for Computational Linguistics.
[53] Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, et al. Gpt-neox- 20b: An open-source 2022.
autoregressive language model. arXiv preprint
[54] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:190.111692. 2019.
[55] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, al. Language models are unsupervised multitask learners. OpenAl blog. 18):9, 2019.
(56) Liang Wang, Nan Yang, and Furu Wei. Learning to retrieve in-context examples for large language models. FACI. 2024, July 2023.
[57] Yukang Lin, Bingchen Zhong, Shuoran Jiang, Joanna Siebert, and Qingcai Chen, Reasoning graph enhanced exemplars retrieval for in-context learning. arXiv preprint 2rXiv:2409.11144 2024.
[58] Jing Xiong, Zixuan Li, Chuanyang Zheng, Zhijiang Guo, Yichun Yin, Enze Xie, Zhicheng Yang, Qingxing Cao. Haiming Wang, Xiongwei Han, al. Dq-lore: Dual queries with low rank approximation re-ranking for in-context learning. arXiv preprint arXiv:2310.02954 2023.
[59] Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by ratio- nale generation: Learning to solve and explain algebraic word problems. arXiv preprint arXiv: 7775.04145 2017.
[60] Tao Ge, Hu Jing. Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei. In-context autoencoder for context compression in a large language model. In The Twelfth International Conference on Learning Representations, 2024.
[61] Yu Fei, Yifan Hou, Zeming Chen, and Antoine Bosselut. Mitigating label biases for in-context learning. arXiv preprint arXiv:2305.19) /9148, 2023.
[62] Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V Le. Finetuned language models are zero-shot learners. In International Conference on Learning Representations, 2022.
[63] Feifan Song, Bowen Yu, Minghao Li, Haiyang Yu, Fei Huang, Yongbin Li, and Houfeng Wang. Preference ranking optimization for human alignment. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18990-18998, 2024.
[64] Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. Beir: heterogenous benchmark for zero-shot evaluation of information retrieval models. arXiv preprint arXiv:2 104. 08663, 2021.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly state the claims made.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We mention the limitations of the work in "Conclusion" section and provide more details in the appendices.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification:
Proofs related to our theoretical results can be found in the supplemental
material.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All the information needed to reproduce our main results can found the "Experiments" section or the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
16

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Data, code, and instructions all available in the supplemental material. We will open-source all materials when the paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nnis.cc
aublic/guudees/Codeebbbmsss
caapssioooolyyyy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips. wwtps/cccpblccggiuiieeeee
www.ssssiicggooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We specify all the training and test details in the "Experiments" section; more details can be found in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the statistical significance of the main comparison in the "Experi- ments" section.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Full information on the computer resources is available in the supplemental materials.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We have carefully followed the Ethies Guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We mention the impacts of the work in "Conclusion" section and provide more details in the appendices.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
18

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We conduct experiments using compliant released data/models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Please refer to the supplemental materials.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URI.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide the assets introduced in the paper in the supplemental materials.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: We describe the usage of LLMs experiments and provide more details in appendices.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hutpe://hearipssccc//ootereeeeep for what should or should not be described.
21

---

Appendices
.1 Datasets details
MRPC:
A paraphrase task with 3,668 training examples and 408 test examples, evaluated using Accuracy and F1.
PAWS: A paraphrase task with 49,401 training examples and 8,000 test examples, evaluated using Accuracy.
QQP: paraphrase task with 363,846 training examples and 40,430 test examples, evaluated using Accuracy and F1.
WSC: coreference task with 554 training examples and 104 test examples, evaluated using Accuracy.
MultiRC: A reading comprehension task with 27,243 training examples and 4,848 test examples, evaluated using F1.
BoolQ: A reading comprehension task with 9,427 training examples and 3,270 test examples, evaluated using Accuracy.
AGNews: A reading comprehension task with 120,000 training examples and 7,600 test. examples, evaluated using Accuracy.
MNI.Im//mm: A natural language inference task with 392,702 training examples and 9,815/9,832 test examples for m/mm, evaluated using Accuracy.
We list the detailed datasets' statistical information above. To convert datasets into natural language instructions, we follow previous practice [13 using the instruction template of FLAN (62]. Each task dataset corresponds to approximately seven templates. All datasets are publicly available under open licenses (e.g., CC-BY, CC-BY-SA, or research-only terms). Datasets are all for MCQ tasks and are widely used in relevant work without offensive content, which is in line with our purpose of use.
.2 Model details
BERT-bbas-uncaseed has approximately 110 million parameters and is released under the Apache License 2.0.
GPT-Neo-2.7B consists of 2.7 billion parameters and distributed under the MIT License.
Llama3-8B-Instrue features 8 billion parameters and is licensed under the Meta Llama Community License.
-llama33-70B contains 70 billion parameters and is also governed by the Meta Llama Community License.
The documentation for the artifacts publicly available, refer to their citations in main paper.
Implementation details
Table lists the overall hyperparameters. For fair comparison, we follow the hyperparameter setting of Se : we use GPT-Nco-2.7B 7B (53) as the scoring and inference LLM for most experiments; both encoders were initialized with "BERT-base-uncaser
[43]; up 10k data points are randomly selected for each task to construct the training data and example pool for each category while maintaining class balance in classification tasks; sample size L set to 50 by default, depending on the configuration of Se"; the shot number K=3; retriever is fine-tuned for 6 epochs each category, the best checkpoint is chosen based on retrieval accuracy using the validation set, and evaluated using task-specific metric on the test set. For the hyperparameters of SeDPO, we set T 20 and ensure that our preference data sourced from the training data of Se² for data fairness. / takes values between 0.001 and 2.
Number of samples per batch. Note that in mini- batch training. for each top-scored positive example considered by Se?, it is necessary to simultaneously consider T low-scored negative examples and negative examples randomly sampled from the corpus, as shown in parentheses of of samples per batch for Se in Table S SeDPO generates T positive and negative example pairs for each input, but each positive example only needs to consider one negative example, as shown in parentheses of We trained the retriever with 8 threads in
the number of samples per batch for SeDPO in Table 5
22

---

Table Hyperparameter settings.
Hyperparameter
Assignment
Hyperparameter
Assignment
shot-number
Max sequence length
512 for retriever
Optimizer
Adam
2048 for LLMs
Number of epochs
6
per GPU Max learning rate # of samples per batch for ScDPO
le-5
Preference 8
0.001
8+32*(++1)
<
# of samples per batch for Se
1322(
Adam epsilon
le-8
Warmup steps
1000
Adam beta weights
0.9, 0.999
Learning rate decay
linear
Weight decay
0.0
Learning rate scheduler
warmup linear
data-distributed manner on
To speed up the training process of Se², we considered 32 positive examples in each mini-batch, along with their dependent negative examples, resulting in total of (2*(1+²))) examples used per mini-batch. Since SeDPO needs to allocate additional memory to the reference model, we only consider using 8132*(++1) examples per mini-batch in ScDPO. The number of samples per batch can be adjusted according to the experimental environment.
Since the training algorithm does not alter the model architecture, the total number of parameters remains 220M, consistent with Se Briefly put, design is compatible with existing fine-tuning- based retrievers using DPR 27 or its sequential version, no further inference or data load is introduced. SeDPO leaves the original framework's token flux unchanged [33] each task takes about 7/9 hours in the scoring/training phase. To allay concerns that the improved ICL performance might stem from differences in backbone models, we detail settings of related methods in Table for reference only.
Table 6: The setting of the related methods.
Finetuned Retriever (Number of Parameters)
Scoring L.M
BM25 SBERT
X
paraphrase-mpnnt--assee (1 109)
UPRISE Se² SeDPO
V
2 BBERR-baaedd-uncase (201
GPT-Nee-22.75
2 B-BBRR--aedd-unaaaaaee (2*110M)
GPT-Neo-2.77 2.7B
2 "-BBRR-basddunm (2*110M)
GPT-Neo-2.7B
.4 Diversity calculation
Textual diversity in Table 3 is measured by Levenshtein Edit distance:
dist(s1.
divrextual (81, 82) sum(len(s)). len((2))
(12)
where i and $2 are two sentences, dist(...)) is Levenshtein Edit distance, len(.) denotes the number of characters in sentence. Semantic diversity in Table 3 is measured by SBERT 23
cos (((1), E(s2))
diviemantic ($ s2)
(13)
where FE()) is the sentence embedding encoded by SBERT and cos(,,) :) is the cosine similarity between two embeddings. We take "paraphrase-Minn MiniL.M-L--v22 as the encoder.
Table The co-reference performance using preference data generated by different LLMs.
G.T-Neo-2.7B Llama3 1e--3-8B-nnnee Instrutt DeepSeek-RI
SeDPO Se 2
62.5%
59.6%
53.8%
55.1%
54.8%
53.8%
.5 Analysis of preference labeling
will be valuable study the reliability of the preferences generated by artificial intelligence and their impact on performance. To this end, we conduct comparative experiments dataset
23

---

construction using various LLMs in two settings: (a) To study the reliability of the likelibood-based scoring method, we construct training data through -lama3-&B-IIsteeee compare with the prompt-engineer-baset scoring method, we use DeepSeek-RI as an agent to simulate a human annotation pipeline following T The overall results on the co-reference task are shown in Table 7 Specifically, to align with the results in Table [ the test time L.MM is still GPT-Neo-2.7B. The generalization ability of the retriever on different test LLMs has been analyzed in Table 4 The results show that imitating human sorting cannot super-enhance ICL performance. We recommend using the output probability of the LLM for the desired answer to rank ICEs.
[2] using Eq. 2 (b) To
Table 8: Prompt used for R1 ranking.
Prompt
You have question inside <question> tags, and you have correct answer inside <answer> tags. Your task is to determine which examples inside <demonstrations> tags are more conducive to obtaining the correct answer to the question.
<answer> <lanswer>
In the tags, the examples arranged in the form of "ID: example". You sort all the examples based on their usefulness and return their sorted IDs in the form of a number list that can be parsed by ISON in the <output> tags. More useful examples should be the front of the list.
Discussion on full list order. Recent advances 63 in Reinforcement Learning from Human Feedback show the potential of modeling full list order. The policy in this kind of method requires estimating the partition function by knowing all actions. For L.Mss, the partition function means being able to observe the entire vocabulary to model the probability of the next word, and this is achievable. However, applying such a formulation to retrieval substantially more challenging - it involves re-embedding the entire corpus at each sampling step, which leads to prohibitive computational costs. In contrast, SeDPO, the pairwise approach, circumvents this by implicitly estimating the partition function, i.e., the denominator of retrieval policy in Eq. (4). This makes pairwise algorithm feasible.
Table 9: The performance on mathematical inference tasks.
Base LLM
Base Shot Number
AUQA-3shot
AUQA-8shot
CEIL. 28
Jlama2-7B-chee
8
22.83
DQ-LoRe 58
Llama2-7B: -B-chat
8
25.20
RGER [57]
Llama2-7B-chat
25.59
Sea
GPT-Neo-2.7B
3
24.80
22.44
ScDPO
GPT-Nco-2.7B
3
25.59
27.56
.6 Mathematical inference
The results on inference tasks are shown in Table Due to variations in the language models, tasks, instruction templates, training and testing datasets, as well as evaluation metrics used by different methods, and due to limitations in computational resources, it is hard to include all related work in the comparison. So we also collected the settings and reported performance of (57) for reference only, Note that only the AUQA (59) task adopted in RGER [57] belongs to MCQ and is compatible with our scoring framework. The results show SeDPO still outperforms Se² on inference tasks, aligning with our main discovery. Notably, though RGER is explicitly built for sessoinngffccusee tasks, SeDPO achieves comparable effectiveness without being geared toward reasoning capabilities.
.7 0-shot retrieval performance
BEIR a robust and heterogeneous evaluation benchmark for information retrieval, aiming to assess the 0-shot retrieval capabilities on human-labeled document of retrieval models. We
24

---

Table 10: The zero-shot retrieval performance on tasks of BEIR.
webis-touche2022
fiqa
scidocs
arguana
nq
Avg.
DPR-BERT
0.0000
0.0002
0.0025
0.0601
0.0012
0.0128
Sea
0.0000
0.0000
0.0032
0.0287
0.0010
0.0066
SeDPO
0.0000
0.0000
0.0032
0.0701
0.0013
0.0149
evaluate nDCG@ @ 10 following BEIR in Table 10 While both Se² and SeDPO are fine-tuned on DPR-BERT, they show no sigmificant O-shot retrieval gains, aligning with BEIR's s observation about dense retrievers' generalization limitations human-labcled documents. Furthermore, Table analyzed, SeDPO and Se² s task pes--preeeeence specialization can compromise semantic similarity modeling, which may be detrimental content-similarity tasks in BEIR. Addressing this requires strategies such scaling and task-aware prompting, which presents promising research direction.
3
.8 Extra analyse
Discussion of ΦLLM in open-ended QA setting. Our paper focused on MCQ setting, with theoretical soundness. We also empirically analyze open-ended QA as follows. For open-ended QA, the set y becomes impractically large, making Eq. (2) intractable to compute. We have experimented with sampling y values for y directly from the model; however, the results were unsatisfactory. Unlike MCQs, open-ended QA lacks a clear way to quantify and normalize the quality gap between good and bad answers, making reliable supervision difficult.
Proof of transitivity. By the definition of (Section 3.2
* If e. > e² x, then for all y E Vgt. SMco @ @r,,) > Sxco @ @r,y). (1)
If > ef x, then for all y Va.. SMco(e" @ @x,y) Swcole @x,y). (2)
Note that (. ) is scalar-valued function, and its outputs are real numbers. The ">" relation on the real numbers is transitive: for any real numbers a, b, c, if and c, then >
Applying this transitivity to (1) and (2) for each y E Fgr
For all y E Vgi. (eco(e" @z, y) Smco @x,g) and Smcole @z. g) Suco @a, y) implies SMco(e" @ y) > Smco(e" @r,y). (3)
By the definition of again, (3) implies en e x. Thus, the relation is transitive. Q.E.D.
Cost of constructing training data. We provide in Table II the average cost of constructing preference data for all tasks. The scoring batch size is 10, using GPT-Neo-2.7B as the ICL model. For each v, we only sample 7 = 20 preference pairs. This means the number of processed preference pairs is less than the number of scored entries. For instance, NLI, the total time for constructing scored data is 794x greater than that for preference data:
Table 11: Cost of constructing training data.
Table 12: Impact of T
Steps to construct trainset Score data (Se2) Rank scored data (SeDPO)
Speed 0.0241 s/entry
Method
T=1
T=10
T=20
T=30
SeDPO (8=0.02)
66.1
74.5
77.9
79.9
0.0007 s/pair
.9 More case studies
In addition, provide more interesting cases in Table 13 experiments are considered.
Various finetuning- based baselines of our
Table 13: More case studies on different tasks.
Task: Paraphrase (MRPC)
25

---

Test Input: "And it's going be a wild ride," said Allan Hoffenblum, Republican consultant. "Now the rest is just mechanical," said Allan Hoffenblum, Republican consultant. Please tell me if the sentences above mean the same. UPRISE:
Examplel: His wife, who he married in a first ever space wedding by a space phone during his lengthy mission, waited in Moscow. His wife Yekaterina Dmitriyeva, whom he married in first ever space wedding by a space phone during his daunting mission, was waiting for him in Moscow. Please tell me if the sentences above mean the same. Yes.
Example2:
The main change, said Jim Walton, CNN's president, is fundamental shift in the way CNN collects its news. The main change, said CNN president Jim Walton, was a fundamental shift the way the network collected its news. Please tell me if the sentences above mean the same. Yes.
Example3: "There were," said board member and Nobel-prize winning Stanford physicist Douglas Osheroff, "some extremely bad decisions." Board member Douglas Osheroff, Nobel-prize winning Stanford physicist, said: "There were some extremely bad decisions." Please tell me the sentences above mean the same. Yes.
Test Input Answer: Yes. Se
Examplel: The Nets and the Spurs are crossing new frontiers of offensive ineptitude while causing their high-scoring American Basketball Association forefathers to cringe. The Nets and the San Antonio Spurs are crossing new frontiers of offensive ineptitude while embarrassing their high- scoring ABA forefathers. Please tell me if the sentences above mean the same. Yes.
Example2: The girl turned up late Sunday
convenience store in East Palo Alto, about 30 miles from her home. The girl turned up late Sunday night at an East Palo Alto convenience store about 30 miles from her home. Please tell me if the sentences above mean the same. Yes.
Example3: "There were," said board member and Nobel-prize winning Stanford physicist Douglas Osheroff, "some extremely bad decisions." Board member Douglas Osheroff, Nobel-prize winning Stanford physicist, said: "There were some extremely bad decisions." Please tell me the sentences above mean the same. Yes.
Test Input Answer: Yes.
SeDPO:
Examplel: He was sentenced in June more than seven years in prison for securities fraud, perjury and other crimes. He was sentenced to more than seven years in prison after pleading guilty to charges including securities fraud. If the first sentence is true, the second one also true? No.
Example2: He really left us with smile on his face and no last words, daughter Linda Hope said, "He really left us with a smile on his face and no last words... gave us each kiss and that was it." she said. If the first sentence is true, the second one also true? No.
Example3: This Palm OS smart phone is the last product the company will release before becomes part of palmOne. This was almost certainly its last full quarter before the company becomes part of Palm. the first sentence is true, is the second one also true? No.
Test Input Answer: No. V
Task: Paraphrase (QQP)
Test Input: What is teeth scaling? How is teeth scaling done? Do those questions have the same meaning?
UPRISE:
Example1: How much time is needed to master Quantum Mechanics from scratch? How much time required to master Quantum mechanics from scratch? Do those questions have the same meaning? Yes.
Example2: Why was the holocaust slow? Why did the holocaust happen so slowly? Do those questions have the same meaning? Yes.
Example3: Why can't 't we make artificial blood? Why we can't make artificial blood? Do those questions have the same meaning? Yes.
Test Input Answer: Yes.
26

---

Se²:
Examplel: The best of 2016: Which are the best Bollywood movies in 2016? Which Bollywood movie you like the most 2016? Do those questions have the same meaning? Yes.
Example2: How does starving help to lose weight? How can starving yourself to lose weight adversely affect your health? Do those questions have the same meaning? Yes.
Example3:
Why are so few drugs with promising animal trials tested in humans? How much do clinical trials for drugs cost? Do those questions have the same meaning? No.
Test Input Answer: Yes.
SeDPO:
Example1: Is PC gaming better than console gaming? Is PC gaming better? Please tell me if those questions are the same. Yes.
Example2: Should people over 95 not be allowed to vote? Should people over 93 not be allowed to vote? Please tell me if those questions are the same. Yes.
Example3: Which hp laptop is best for a graphic design/gamer? Which is best HP or Dell laptop? Would you say that these questions are the same? No.
Test Input Answer: No. V
Task: Paraphrase (PAWS)
Test Input: Do these mean the same? Wilbur was born on March 1921 in North Caldwell, New Jersey and grew up in New York City. Wilbur was born in North Caldwell, New Jersey March 1, 1921, and grew up in New York City.
UPRISE:
Examplel: Are these paraphrases? Born in Gosforth, Northumberland, he moved to the south as a boy Wiseton Estate, near Retford, Nottinghamshire, when his father found jobs there. Born in Retford, Nottinghamshire, he moved as boy to Wiseton Estate, near Gosforth, Northumberland, when his father found jobs there. No.
Example2: Do these mean the same? J.David Spurlock was born on November 18, 1959 in Memphis, Tennessee. He moved to Dallas, Texas in 1973. David Spurlock was born on 18 November 1959 in Dallas, Texas, and moved Memphis, Tennessee 1973. No.
Example3: Do these mean the same? Joe was born Somerville, Massachusetts on March 27, 1929 and grew up in Quincy, Massachusetts. Joe was born March 27, 1929 in Quincy, Massachusetts, where grew up in Somerville, Massachusetts. No.
Test Input Answer: NO. x
Sea:
Examplel: Are these paraphrases? The following sound changes from Proto-Celtic to Welsh, Cornish and Breton are summarised in the regular consonant table. The regular consonantal sound changes from Proto-Celtic to Welsh, Cornish and Breton summarised in the following table. No.
Example2: Do these two sentences from wikipedia have the same meaning? Kennell was born in Colorado Springs, Colorado, and spent her early years between the Rockies and Dunedin, Florida. Kennell born in Dunedin, Florida, and spent her early years between the Rockies and Colorado Springs, Colorado. No.
Example3: Do these two sentences from wikipedia have the same meaning? Robert Maass was born East Orange, New Jersey, to study German immigrants Hedwig and Clara Maass. Robert Maass was born in East Orange, New Jersey, to German immigrants Hedwig and Clara Maass. Yes.
Test Input Answer: No. SeDPO:
Examplel: Do these two sentences from wikipedia have the same meaning? Several of these names were chosen to correspond to their international equivalents in rough chess, and not as literal translations of the Japanese names. These names were chosen to correspond their international counterparts in the rough chess and not as literal translations of Japanese names. Yes.
27

---

Example2: Do these two sentences from wikipedia have the same meaning? Due to the results obtained in the previous round, Kevin Gleason received + 30kg, Gianni Morbidelli 20kg and Pepe Oriola 10kg. Due to the results of the previous round Kevin Gleason received + 30kg, Gianni Morbidelli + 20kg and Pepe Oriola + 10kg. Yes.
Example3: Do these two sentences from wikipedia have the same meaning? When combined for joint or coalition operations, it was known coalition operations. When combined for joint or coalition operations, was known as a joint or employed air operations center for coalition operations. Yes.
common employed air operations centre for
Test Input Answer: Yes. /
28


Reviews:
[{"review_id": "6lT8xbuwHh", "reviewer": "Reviewer_R6FR", "summary": "This paper introduces **SE-DPO**, a novel learning-to-rank algorithm for in-context example (ICE) retrieval in large language models (LLMs). Unlike classification-based methods, SE-DPO aligns retrieval scores with LLM preferences by optimizing pairwise rankings of ICEs, leveraging Direct Preference Optimization (DPO) with sequential relaxation. Experiments across 9 NLP tasks show SE-DPO outperforms state-of-the-art baselines by up to 18%, particularly in challenging tasks like PAWS and MultiRC. Ablation studies confirm the importance of ranking formulation and diversity in ICE selection. The method also demonstrates strong transferability across model sizes and shot numbers. Limitations include reliance on dominant preferences and potential biases, addressed via safeguards discussed in the broader impacts section.", "questions": "1. The impact of different embedding models on the results. I am not clear about whether the influence of embeddings on the results is good or bad. This paper uses \"BERT-base-uncased\" as the encoder model, but nowadays there are many better text embedding models available.\n2. Preference Labeling Stability: The paper constructs ranked data by comparing LLM likelihoods, but how sensitive is SE-DPO to noise or inconsistencies in these scores? If the LLM’s confidence in ranking ICEs is low (e.g., near-tie scores), does the method degrade?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths:\n1. **Effective Ranking Formulation:** SE-DPO outperforms classification-based methods by explicitly optimizing pairwise rankings of in-context examples (ICEs), leading to significant performance gains (up to 18%) across multiple NLP tasks.  \n2. **Strong Generalization:** The method demonstrates robust transferability across different LLM sizes and shot numbers, showing consistent improvements even when tested on unseen inference settings.  \n\n## Weaknesses: \n1. **Computational Overhead:** The sequential ranking process and preference alignment introduce additional complexity, making training more resource-intensive compared to simpler classification-based retrievers.  \n2. **Bias and Fairness Risks:** Like other retrieval-based methods, SE-DPO may inherit biases from LLM-generated preferences, potentially reinforcing harmful or misleading examples without careful mitigation.", "was_revised": true, "final_justification": "I appreciate the authors' response and would like to keep my positive score.", "rebuttal": "Thank you very much for your constructive feedback. We sincerely appreciate your time and effort in reviewing our work and would like to address all of your concerns. If you find our responses helpful, we would be truly grateful if you could consider adjusting your rating accordingly :)\n\n**W1/W2. Fairness, overhead, and discussion clarity.**\nWe have discussed several limitations of our method—including fairness—in the appendix. Since our algorithm does not introduce inference-time overhead, the additional computational cost remains affordable. We will revise the conclusion section to better highlight these points.\n\n\n**Q1: Text encoder fairness and ablation results (lines 183–188).**\nAs described in the paper, both SEDPO and all trainable baselines use BERT-base-uncased to ensure fair comparison (lines 183–188, Table 2). \nTo further show the influence of the embedding model, we conducted additional ablation experiments with RoBERTa-base:\n\n| Encoder | Paraphrase-Avg |  Paraphrase-MRPC (ACC/F1) | Paraphrase-PAWS (ACC) | Paraphrase-QQP (ACC/F1) | \n|--------|---------|---------|---------|---------|\n| BERT-base-uncased |  77.9  | 77.9/85.6 | 73.0 | 77.6/75.0 |\n| RoBERTa-base | 85.7 |  86.3/90.5 | 88.1 | 83.6/ 80.0 |\n\nRoBERTa-base is known to outperform BERT-base-uncased across a range of NLP tasks, and as shown above, SEDPO benefits from stronger embeddings as expected. We will include these results in Table 2.\n\n\n**Q2: Robustness to noisy preferences (lines 134 and 242, Table 2).** \nAs shown in Table 2 and discussed in lines 134 and 242, replacing our original preference data with randomly sampled preference pairs of the same size significantly degrades performance. This effect is more pronounced than the degradation seen in cases of close-score (low-confidence) preferences. These results suggest that SEDPO is sensitive to preference noise, and that low-confidence LLM rankings can negatively affect learning. We will revise the relevant sections to make this insight clearer and more prominent."}, {"review_id": "SZrH7Alr9t", "reviewer": "Reviewer_TQpj", "summary": "This paper proposes a ranking retrieval method, where the preference rankings between ICEs are given by comparing the likelihood of the LLM generating the correct answer conditioned on each exemplar.", "questions": "1. Could you explain more on Eq. (3)? As currently written, $p(rank)=\\frac{exp(-rank)}{exp(-1)+exp(-2)+..+exp(-L)}$, appears to be a fixed distribution over ranks, independent of the specific sample $e_c$, assuming $L$ is fixed. There are no sample-dependent variables in the equation, which makes it unclear how this relates to individual examples. \nIs this meant to represent a normalized ranking distribution averaged over multiple sampling rounds for each $e_c$? Clarifying this would improve the reader's understanding.\n\n2. In line 152, the formulation of $Z(x)$ is not defined. It would be helpful to either include its explicit definition or provide a citation that explains it, to enhance clarity and reproducibility.", "limitations": "This paper has discussed the limitations.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength:\n\n1. The proposed method achieves significant improvement by adopting GPT-Neo-2.7B.\n2. A wide range of datasets is adopted during the evaluation.\n\nWeakness:\n\n1. The main motivation of the paper is based on the claim that \"dominant approaches train the retriever as a classifier\" (line 22) and that \"learning-to-rank (LTR) remains underexplored in in-context learning (ICL)\" (lines 29–32). However, this motivation is not fully convincing, as several prominent ICL methods—such as UDR [1], PromptPG [2], and LLM-R [3]—already employ ranking-based retrievers, where samples with the highest ranking scores are selected. As such, the paper’s premise that LTR is underexplored appears overstated. Furthermore, the proposed method does not present a substantial novelty over these existing approaches, and the experimental section lacks a direct comparison with other ranking-based ICL methods (only UDR is included).\n\n[1] Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, and Xipeng Qiu. Unified demonstration retriever for in-context learning.\n\n[2] Lu P, Qiu L, Chang K W, et al. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning[J]. ICLR 2023.\n\n[3] Wang L, Yang N, Wei F. Learning to retrieve in-context examples for large language models[J]. arXiv preprint arXiv:2307.07164, 2023.\n\n2. According to Table 4, the performance gains of the proposed method diminish as the model size increases. Specifically, the improvements for LLaMA3-8B and LLaMA3.3-70B are relatively modest—only 1.2% and 0.7%, respectively. This suggests that the proposed method may not generalize well to larger language models, raising questions about its scalability and broader applicability.\n\n3. Some of the equations or notations are not clearly explained, which confuses the readers. Please refer to the questions for details.\n\n4. This paper emphasizes its contribution over the rankings between ICEs. Will this paper also consider the inner order of the ICEs within each prompt? According to the implementation details (line 185), the shot number $K$ is only 3. Given such a small number, does the inner ordering of ICEs meaningfully affect performance? Clarifying whether and how this aspect was considered would strengthen the experimental analysis.", "was_revised": true, "final_justification": "I would like to raise my score to 3.", "rebuttal": "Thank you very much for your time and effort in reviewing our work. We sincerely appreciate your thoughtful comments and would like to address the concerns raised. If you find our responses helpful, we would be truly grateful if you could consider adjusting your score accordingly :)\n\n\n**W1. Clarifying our contributions.** We believe there may be some misunderstandings regarding the uniqueness of our work. As described in both the introduction and Figure 1, our study focuses on the ranking formulation among ICEs (i.e., different $K$-shot prompts), which is a central contribution highlighted in both the introduction and Figure 1. In contrast, prior works primarily adopt classification-based approaches:\n- **PromptPG** focuses on classification-based retrieval and does not explicitly investigate the internal or external ordering of ICEs. While it notes that ordering can affect model performance, this is mentioned only in the analysis section and not as a core research objective.\n- **LLM-R** similarly does not study the preference order among ICEs. The “ranking” mentioned in its analysis refers to a screening mechanism within a classification formulation.\n- **UDR** introduces a local ranking regularizer into the classification formulation, but still fundamentally operates in a classification framework. We have provided a detailed analysis of UDR’s limitations in lines 68, 203–205, and 218–220, along with supporting experiments in Tables 1 and 2.\n\n\n**W2. Clarification on Table 4 and additional experiments.** Table 4 in our paper aims to demonstrate that the performance improvement of our proposed method becomes more pronounced as the model size decreases (lines 274–276). This experiment is designed to explore the boundary of transferability and to help identify suitable application scenarios for SEDPO. We would like to respectfully note that smaller models remain a highly active research area. To further support this point, we provide below an additional evaluation on **GPT-2 XL (1.5B)**, following the same setup as in Table 3:\n\n| Method | 1-shot |3-shot |6-shot |9-shot |12-shot |15-shot |Avg. |\n|--------|---------|---------|---------|---------|---------|---------|---------|\n| UPRISE | 69.15 | 69.41 | 69 .84 | 69.82 | 69.97 | 70.16 | 69.72 |\n| BM25 | 57.61 | 58.47 | 58.81 | 58.67 | 59.31 | 60.06 | 58.82 |\n| SBERT | 57.93 | 57.52 | 58.98 | 59.56 | 58.57 | 58.29 | 58.48 |\n| Se2 | 73.90 | 72.89 | 72.90 | 72.80 | 72.80 | 72.70 | 73.00 |\n| SEDPO| 75.03 | 78.93 | 79.46 | 79.17 | 78.99 | 79.19 | 78.46 | \n\nSEDPO outperforms the second best by up to 5.46%, which demonstrates better transferability on smaller models. We hope this additional evidence further supports the generalizability and effectiveness of our method. If space allows, we’ll add more small-LLM results to Table 4.\n\n**W3.** Kindly refer to our responses to Q1 and Q2 for clarifications.\n\n**W4. Transferability on shot number and intra-ICE ordering.** Table 4 presents the transferability of different methods across 1 to 15-shot settings. While our primary focus is on the ordering between ICEs, we also analyze the effect of intra-ICE ordering through additional experiments. This analysis is provided in Table 3 and discussed in lines 259–262.\n\n**Q1. Greedy sampling strategy (Section 3.2)** We appreciate your attention to the selection process for constructing $K$-shot prompts $e_{[1:K]}$ for training. As described in Section 3.2, considering all non-repetitive combinations is computationally infeasible (line 122). Therefore, we use a greedy sampling approach guided by Eq. (3) to select each example $e_c$ within $e_{[1:K]}$. When selecting $e_1$, we randomly sample $L$ examples (Eq. (2)), rank them using the LLM score, and then sample from these candidates based on the probabilities given in Eq. (3). This approach, which takes into account the importance and diversity of examples, is expected to be more effective than uniform sampling, as cited in line 123.\n\n\n**Q2. Clarifying $Z(x)$ in Eq. (5).**\nThank you for raising this point. The partition function $Z(x)$ is introduced in Eq. (5), and reference of DPO \\[4]—which provides a detailed explanation—is cited in the paragraph preceding the formula. To enhance clarity for readers, we will consider adding more explicit references near the $Z(x)$ itself."}, {"review_id": "KeQ2UH2YOU", "reviewer": "Reviewer_Cqhk", "summary": "This paper proposes an approach to selecting and ranking examples for ICL. The approach is based on a ranking formulation based on DPO and sequentially choosing examples. The paper adapts DPO to this setting and uses a relaxation to derive a tractable algorithm. Experiments on multiple choice questions (MCQ) show SOTA performance compared to several baselines.", "questions": "* Eq (2): this is defined for a small \\mathcal{Y}, as in your MCQ setting. How can \\phi_{LLM} be defined for general outputs? How about sampling y's from the model for \\mathcal{Y}?\n* Line 173: can you include a proof for transitive closure?\n* Can you comment on the cost of constructing the training data?\n* How does performance vary with different values of T?\n\nPlease detail relationship to these related papers:\n* https://arxiv.org/pdf/2503.08030\n* https://www.arxiv.org/pdf/2506.08607 (made public after NeurIPS submission)\n* https://www.arxiv.org/pdf/2506.04579 (made public after NeurIPS submission)\n\nMinor / Typos\n* SE-DPO appears in lines 6 and 34, but defined only in line 39.\n* Eq (1): why are examples ordered from K to 1? This is a bit strange.\n* Line 90: the notation E_e(e) and E_x(x) is confusing. Consider something like E_{example}(e) and E_{input}(x).\n* Line 92: “we first encodes and indexes” => “we first encode and index”\n* Line 125: The notation \\tilde{x}_1\\in\\mathcal{C} is confusing since \\mathcal{C} is a corpus of (x,y) examples (line 83) while x represents inputs only.\n* Line 146: “we denotes” => “we denote”", "limitations": "See weaknesses/questions above.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n* Identifying mismatch between train and test in ICL, and using a ranking formulation for training to resolve it.\n* The empirical evaluation seems comprehensive and well executed.\n* The empirical results show SOTA performance compared to strong baselines.\n\nWeaknesses:\n* Generalizability beyond MCQ.\n* Potentially computationally expensive dataset construction.", "was_revised": true, "final_justification": "After reading through the authors response and the other reviews, I would like to keep my \"borderline accept\" recommendation.\n\nThe authors did a good job in the rebuttal addressing some of the questions. The additional experimental results on scalability to larger models and the effect of the number of compared examples T are a great addition and are helpful in understanding the performance of the proposed approach. The authors also clearly differentiated their work from other previous and concurrent work: their focus on formulating the retriever training as a direct learning-to-rank problem with a pairwise preference objective distinguishes their work from others. The authors also adequately addressed my original concerns about computational cost of training data construction.\n\nI think that the main limitation of the paper is its restricted application to MCQ. In their rebuttal, the authors confirmed that their attempts to apply the method to open-ended question answering were unsatisfactory. Furthermore, novelty seems to be OK but not outstanding: the core idea is applying DPO to the problem of in-context example retrieval. While the application is novel and the engineering to make it work is sound, it mostly cleverly combines existing components. The work is high-quality, but it builds upon established paradigms.\n\nIf there was an option for a 4.5 score I would choose that, but don't feel it's a 5 paper.", "rebuttal": "We are grateful for your thoughtful suggestions and will incorporate additional details into the paper accordingly. If you find our responses helpful, we would sincerely appreciate it if you could consider raising your score :)\n\n**Q1. On general output and the size of $\\mathcal{Y}$:** \nIf the general output refers to open-ended QA, the set $\\mathcal{Y}$ becomes impractically large, making Equation (2) intractable to compute. We have experimented with sampling $y$ values for $\\mathcal{Y}$ directly from the model; however, the results were unsatisfactory. Unlike MCQs, open-ended QA lacks a clear way to quantify and normalize the quality gap between good and bad answers, making reliable supervision difficult.\n\n\n**Q2. Proof of transitivity:**  \nBy the definition of $\\succ$ (lines 117-118):  \n- If $e^a \\succ e^b|x$, then for all $y \\in {\\mathcal{Y}}\\_{\\text{gt}}$ , $\\mathrm{S}\\_\\mathrm{MCQ}(e^a \\oplus x,y) > \\mathrm{S}\\_\\mathrm{MCQ}(e^b \\oplus x,y)$. (1)  \n- If $e^b \\succ e^c|x$, then for all $y \\in \\mathcal{Y}\\_{\\text{gt}}$, $\\mathrm{S}\\_\\mathrm{MCQ}(e^b \\oplus x,y) > \\mathrm{S}\\_\\mathrm{MCQ}(e^c \\oplus x,y)$. (2)  \n\nNote that $\\mathrm{S}\\_\\mathrm{MCQ}(\\cdot)$ is a scalar-valued function, and its outputs are real numbers. The \">\" relation on the real numbers is transitive: for any real numbers $a, b, c$, if $a > b$ and $b > c$, then $a > c$.  \n\nApplying this transitivity to (1) and (2) for each $y \\in \\mathcal{Y}_{\\text{gt}}$:  \nFor all $y \\in \\mathcal{Y}\\_{\\text{gt}}$, $\\mathrm{S}\\_\\mathrm{MCQ}(e^a \\oplus x,y) > \\mathrm{S}\\_\\mathrm{MCQ}(e^b \\oplus x,y)$ and $\\mathrm{S}\\_\\mathrm{MCQ}(e^b \\oplus x,y) > \\mathrm{S}\\_\\mathrm{MCQ}(e^c \\oplus x,y)$ implies $\\mathrm{S}\\_\\mathrm{MCQ}(e^a \\oplus x,y) > \\mathrm{S}\\_\\mathrm{MCQ}(e^c \\oplus x,y)$. (3)  \n\nBy the definition of $\\succ$ again, (3) implies $e^a \\succ e^c|x$.  \nThus, the relation $\\succ$ is transitive.  \nQ.E.D.\n\n\n**Q3. Cost of constructing training data:** \nWe provide below the average cost of constructing preference data for all tasks. Experiments were conducted on 8 V100-32GB GPUs with a scoring batch size of 10, using GPT-Neo-2.7B as the ICL model. For each $x$, we only sample $T = 20$ preference pairs. This means *the number of processed preference pairs is less than the number of scored entries*. For instance, on NLI, the total time for constructing scored data is 794× greater than that for preference data:\n\n| Steps to construct training data | Speed |\n|--------|---------|\n| Construction of scored data (Se2) | 0.0241 s/entry |\n| Constructing preference data from (SEDPO) | 0.0007 s/pair |\n\n\n**Q4.** \nWe provide below the extra ablation results on Paraphrase for different $T$, following the setup of Table 2.\n\n\n| Method | T=1 | T=10 | T=20 | T=30 |\n|--------|---------|---------|---------|---------|\n|SE-DPO (β=0.02) | 66.1 | 74.5 |  77.9 | 79.9 |\n\nAs T increases, the performance of SEDPO improves because a broader preference ranking is learned. However, when T is too large, the performance gain decreases due to more low-confidence LLM rankings. As we have discussed this in lines 131-136 and lines 241-243, low-confidence LLM rankings can negatively affect learning --- there is a trade-off.\n\n**Minor/Typos.** Thank you for your careful reading. We will thoroughly correct all typos and minor issues in the revised version."}, {"review_id": "BjyIP6V2cv", "reviewer": "Reviewer_ESxb", "summary": "The paper considers the problem of selecting in-context learning examples to improve a language model's performance.\nIt formulates in-context example (ICE) retrieval as a learning-to-rank problem and uses preference orderings, rather than classification.\n\nConcretely, given a prompt $x$, the goal is to select examples $e_1, e_2, \\dots, e_K$ from a corpus and add them to a problem.\nIt trains a retriever that selects the top $w$ examples in one batch given the currently selected examples.\nIt repeats this operation until $K$ examples are selected.\n\nIt uses the likelihood of the correct response given a sequence of examples as reward signals.\nTo generate preference labels, it compares the likelihoods of different ICE sequences.\nIt trains the retriever using a variation of DPO, to prefer sequences that lead to higher rewards.\n\nEmpirically, the method outperforms prior retrieval-based ICL methods on 9 NLP tasks.", "questions": "Some questions are in the weaknesses section above.\n\nAs state-of-art language models support much longer context lengths, does this generalize to larger number of examples (allowing selecting more than $K$ examples during inference)?", "limitations": "The limitations are discussed in Section .9 in the appendix.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n**Novelty.**\nUsing a DPO-based pairwise loss to train a retriever is novel and inspirational.\nDPO is known to be stable and is a reasonable choice for training a retriever to achieve better model outputs.\n\n**Empirical performance.**\nThe method is empirically evaluated in multiple NLP domains and outperforms multiple baseline algorithms, including Se2 and UPRISE.\n\n### Weaknesses\n\n**Missing baselines in empirical evaluation.**\nOne existing work that is closely relevant is Active Example Selection ([31] in the paper).\nThe paper claims that this method \"suffer from training instability\" (line 63),\nand \"(not) considers ranking orders of examples across prompts\" (line 66).\n\nIf I understand this work correctly, Active Example Selection uses an RL approach to select examples one by one.\nSo it does consider the order of examples.\nAlso, this baseline algorithm is not evaluated empirically, so it's unclear if the proposed algorithm is indeed advantageous.", "was_revised": true, "final_justification": "The authors have clarified the contributions and baseline method selection. It shows convincing results that ranking-based ICE selection outperforms score-based ICE selection. However, as other reviewers noted, the results are only evaluated on multi-choice question benchmarks, which makes the contributions relatively limited.\n\nI have adjusted the score to 4.", "rebuttal": "Thank you very much for your time and effort in reviewing our work. We would like to address some of the comments that may stem from misunderstandings. If you find our response helpful, we would sincerely appreciate it if you could consider raising your score :)\n\n**W1/W2. Clarification of our contributions:**  \nAs described in both the introduction and Figure 1, our key contribution lies in modeling the ranking among ICEs—for example, comparing and ranking different K-shot prompts. In contrast, the Active Example Selection (AES) method (citation [31]) is not closely-related to our work. AES focuses on sequentially selecting examples to construct a K-shot prompt, emphasizing the internal ordering within ICEs. Our baseline, Se2, is the most advanced follow-up to AES, following the same paradigm for intra-ICE ordering but without the need for active learning. The following shows the results of AES on related tasks  (taken from Se2 [13] in paper).\n\n| Method | PAWS (ACC) | AGNews (ACC) | MNLI-m (ACC) | MNLI-mm (ACC) |\n|---|---|---|---|---|\n| AWS  | 51.7 | 78.2 | 43.2 | 29.5 |\n| SE-DPO | 73.0 | 90.7 | 70.6 | 72.0 |\n\nConsidering both relevance and performance, AES is not suitable as our baseline. We will highlight the evidence in the related work section.\n\n\n**Q. Clarification on inference results:** \nWe have provided inference experiments with $K$ ranging from 1 to 15, using various LLMs, as shown in Table 4. A detailed analysis of these results can be found in lines 268–276 of the paper. Our empirical study (Table 3) demonstrates that SEDPO can generalize to a larger number of examples."}]

Author rebuttal:

Meta-review: I recommend accepting this paper, "Learning to Rank for In-Context Example Retrieval." The authors propose SE-DPO, a novel algorithm that trains a retrieval model using a ranking formulation rather than classification, where preference rankings between in-context examples (ICEs) are determined by comparing LLM likelihoods of generating correct answers. The approach outperforms state-of-the-art baselines across 9 NLP tasks, with particularly strong results for smaller models. While the performance gains diminish somewhat for larger LLMs, the authors have demonstrated that few-shot learning with their method still significantly outperforms zero-shot approaches, even for larger models. The reviewers noted the technical soundness and empirical effectiveness of the approach, and the authors provided thorough responses addressing concerns about novelty, transferability, and computational requirements. This work makes a valuable contribution to improving in-context learning through more effective example selection.
Decision: Accept (poster)