Active Ranking without Strong Stochastic Transitivity
Anonymous Author(s)
AffiliationAddress email
Abstract
Ranking from noisy comparisons is of great practical interest in machine learning.
In this paper, we consider the problem of recovering the exact full ranking fora list of items under ranking models that do not assume the Strong Stochastic
Transitivity property.
We propose a δ -correct algorithm, Probe-Rank, that activelylearns the ranking of the items from noisy pairwise comparisons.
We prove a samplecomplexity upper bound for Probe-Rank, which only depends on the preferenceprobabilities between items that are adjacent in the true ranking.
This improvesupon existing sample complexity results that depend on the preference probabilitiesfor all pairs of items.
Probe-Rank thus outperforms existing methods over a largecollection of instances that do not satisfy Strong Stochastic Transitivity.
Thoroughnumerical experiments in various settings are conducted, demonstrating that ProbeRank is significantly more sample-efficient than the state-of-the-art active rankingmethod.
Introduction
Ranking from noisy comparisons has a wide range of applications including voting [6, 4], identifyingthe winner/full ranking of teams in sport leagues, ranking players in online gaming systems [14],crowdsourcing services [5], web search [7], and recommendation systems [1, 20].
In practice,comparisons usually contain certain levels of “noise”.
For example, duels in a game are not alwayswon by the more proficient player, and preferences between movies/restaurants can also vary amongdifferent individuals.
The presence of noise is commonly studied using a probabilistic comparisonmodel [10, 22], where an item has a certain probability to win the comparison over another (pairwise)or a group of items (listwise).
We are interested in estimating the total ranking.
To guarantee that the ranking is consistent with thepreference probabilities, it is often assumed [12, 18, 10, 22] that if i ranks higher than j , then i winsa comparison between j with probability p
i,j > 12 .
It is clear that the closer p i,j is to 1 / 2 , the moredifficult it becomes to compare i and j .
This assumption is referred to as Weak Stochastic Transitivity(WST) .
A more strict assumption, Strong Stochastic Transitivity (SST) , is also often made [8, 21, 23].
SST requires items that have closer ranks to be more difficult to compare, i.e., if i ≻ j ≻ k , thenp i,k ≥ p i,j > 12 .
Formal definitions of WST and SST are stated in Section 2.
However, SST can be too strong for scenarios where preference probabilities are not based oncomparing a single quantifiable attribute.
For instance, in sports, match outcomes are usually affectedby team tactics.
Team k may play a tactic that counters team i , resulting in a higher winning rateagainst team i compared with team j .
Furthermore, items usually have multidimensional features andpeople may compare different pairs based on different features.
A close pair in the overall ranking isthus not necessarily harder to compare than a pair that has a large gap.
For example, when comparingcars, people might compare a given pair based on their interior design and another pair based onperformance.
As another example, in an experiment with games of chance with different probabilities of winning and payoffs [27], it was observed that “people chose between adjacent gambles accordingto the payoff and between the more extreme gambles according to probability, or expected value.”
Motivated by such applications, in this paper, we are interested in the problem of recovering the fullranking of n items under a more general setting, where only WST holds, while SST is not assumedto hold.
We focus on only pairwise comparisons as they are easier to obtain and less prone to errorin practice.
Furthermore, as many applications [5, 19] allow interactions between users/annotators,we consider comparisons collected in an adaptive manner.
Our goal is to use as few comparisons aspossible and achieve a high confidence.
Existing algorithms [18, 22] can not avoid comparing every item i with the item i ∗ who is the mostsimilar to i , i.e., (cid:12)(cid:12)
p
i,i ∗ − 12 (cid:12)(cid:12) =
min j ̸ = i { (cid:12)(cid:12) p i,j − 12 (cid:12)(cid:12) } .
Further, [22] pointed out that comparing itempairs that are adjacent in the true ranking are necessary.
When SST holds, adjacent pairs are also themost difficult pairs to distinguish, existing methods thus achieve sample-efficiency.
For example, the
Iterative-Insertion-Ranking (IIR) algorithm proposed in [22] maintains a preference tree and performsranking by inserting items one after another.
During the insertion process, every item is possible tobe compared with every other items (and thus the most similar one), depending on the relative orderof insertion and the true ranking.
IIR was shown to be sample complexity optimal under SST andsome other conditions.
However, when SST does not hold, comparing nonadjacent items harms the performance.
Consideran extreme scenario where the true ranking is 1 ≻ 2 ≻ 3 and p 1 , 2 = p 2 , 3 = 0 .
8 , p 1 , 3 = 12 + 2 − 10 .
If item 1 is directly compared to item 3, then it takes Θ (cid:0) 2 20 (cid:1) comparisons 1 .
For instance, in IIR,this can happen during the insertion process of item 3 when item 1 happens to be the root of thepreference tree.
On the other hand, a simple fix exists as we can let the three pairs to be comparedsimultaneously.
The comparisons between items 1 and 2, items 2 and 3 will terminate much earlierand provide us with the information 1 ≻ 2 , 2 ≻ 3 , which is enough to recover the total ranking.
Therefore, it is important to devise an algorithm whose sample complexity will not be harmed when
SST fails to hold.
Contribution.
In this paper, we propose an active algorithm, termed Probe-Rank , that ranks nitems based on pairwise comparisons.
Probe-Rank is a maxing-based algorithm, i.e., it ranks itemsby performing n − 1 steps of maxing.
We show that as long as the WST condition is satisfied, withprobability at least 1 − δ , Probe-Rank returns the correct ranking after conducting at most
[Equation]
comparisons, where (cid:101) ∆ i =
min
j : j and i are adjacent (cid:12)(cid:12) p
i,j − 12 (cid:12)(cid:12) .
Probe-Rank is the first algorithm whosesample complexity only depends on comparison probabilities of adjacent items instead of all pairsof items [26, 18, 22, 28].
Theoretical analysis and numerical experiments under various settingsare provided and show that Probe-Rank is more efficient than the state-of-the-art when comparingnonadjacent items is more difficult than comparing adjacent items.
We also present a preliminaryanalysis on the sample complexity lower bound in the worst case scenario when SST does not hold.
Preliminaries
Notation Without loss of generality, let [ n ] = { 1 , 2 , . .
. , n } denote the set of n items.
We writep ∼ Uni ( a, b ) to denote that p is sampled uniformly at random from the interval ( a, b ) , and use Ber ( p )to denote a Bernoulli random variable which equals 1 with probability p .
We use ( i, j ) to denote theunordered item pair, i.e., ( i, j ) =
( j, i ) .
Comparisons between items are probabilistic.
Whenevertwo items i, j are compared, i (respectively, j ) is preferred with probability p
i,j (respectively, p j,i ),independent of any other quantities.
For all i ̸
= j , p i,j + p j,i = 1 .
A probabilistic comparison modelover [ n ] is thus defined by the set of probabilities P = { p i,j } 1 ≤ i<j ≤ n .
In this paper, asymptotic notation including O ( · ) , Ω( · ) , Θ( · ) are defined in the standard sense, with(cid:101)
O ( · ) , (cid:101) Ω( · ) , (cid:101)
Θ( · ) denoting corresponding weaker forms by allowing logarithmic factors.
Problem setup We assume that there exists a total ordering ‘ ≻ ’ over [ n ] such that σ 1 ≻ σ 2 ≻· · · ≻ σ n for some permutation σ of [ n ] .
The permutation σ is referred to as the true ranking.
Twoitems are called adjacent if they are adjacent in σ , i.e., one ranks right next to the other.
To ensurethat the true ranking σ is consistent with comparisons, we also assume that i has a higher rank than jif and only if p i,j > 12 .
In other words, if an item i is more preferred than j in σ , then i has a betterchance to win the comparison with j .
This assumption is known as Weak Stochastic Transitivity(WST) .
A more strict assumption, Strong Stochastic Transitivity (SST) , is also frequently adopted.
Inaddition to WST, SST assumes that whenever i ≻ j ≻ k , p i,k ≥ max ( p i,j , p j,k ) .
In this paper, weassume only WST and our goal is to recover the true ranking σ with a given confidence level δ bytaking pairwise comparisons and minimize the sample complexity.
Problem instances are uniquelydetermined by the permutation σ representing the true ranking and the comparison probabilities P .
Definition 1 ( δ -correct algorithm) .
An algorithm is said to be δ -correct if for any input instance, withprobability at least 1 − δ , it returns a correct result in finite time.
It is clear that the closer p i,j is to 12 , the more difficult it becomes to obtain the ordering between iand j .
Therefore, the probability gap ∆ i,j , defined as ∆ i,j = (cid:12)(cid:12) p i,j − 12 (cid:12)(cid:12) , provides a characterizationof the ranking task difficulty and will be used as a parameter for measuring sample complexities ofalgorithms.
For instance, Farrell [11] shows that for any δ -correct algorithm A ,
[Equation]
where T A [∆] is the expected number of samples taken by A on two items with probability gap ∆ .
Further, for each item i , we define ∆ i = min
j : j ̸ = i ∆ i,j , the minimum probability gap between item iand any other item j , and
[Equation]
the minimum probability gap between i and its adjacent items in the true ranking.
Note that ∆ i ≤ (cid:101) ∆ iby definition and the equality holds when SST is satisfied.
Related work
The problem of ranking under coherent probabilistic comparisons dates back to 1994 [12].
Feigeet al.
[12] studied the comparison model assuming that i ≻ j ⇔ p i,j = 12 + ∆ for some known ∆ .
It was shown that any δ -correct algorithm finds the true ranking with at least Θ( n ∆ − 2 log ( n/δ ))comparisons in the worst case.
Later in [18], a δ -correct algorithm TOP was proposed to rank thetop-
k elements by assuming only the existence of a total ranking (WST).
The state-of-the-art IIRalgorithm was proposed in [22], as discussed in Section 1.
A comparison of related algorithms arepresented in Table 1.
[Table]
The
[Table]
Luce model
[Table]
[Equation]
[Table]
[Equation]
Ranking or maxing have also been widely studied under more strict assumptions, e.g., SST, RST 2 and
STI 3 and are usually in the probably approximately correct (PAC) setting [8, 9, 10, 21, 23, 24, 26, 29].
In particular, [21, 23, 24, 26] considered parametric comparison models such as the multinomiallogit (MNL) model.
Note that parametric models are often more restrictive and can imply SST/STIconditions.
In PAC setting, the goal is to find an ϵ -ranking r 1 ≻ r 2 ≻ · · · ≻ r n such that p r i ,r j > 12 − ϵfor all i < j .
Although ϵ -rankings become closer to the true ranking as ϵ goes to 0, it is pointedout by [22] that PAC ranking algorithms can not be easily extended to the case when ϵ = 0 .
Among all, [10] is the most relevant work to this paper.
In [10], PAC ranking and maxing werestudied for both SST and WST settings.
For WST, an instance-independent lower bound Θ( n 2 ) wasproved, and a brute-force algorithm which compares each pair to an accuracy of ϵ and thus conducts O (( n 2 /ϵ 2 ) log( n/δ )) comparisons was proposed.
Note that in this paper, we are aiming at recoveringthe exact ranking instead of an ϵ -ranking.
A trivial extension of the brute-force algorithm can lead tosample complexity (cid:101) O (cid:16)
[Table]
(cid:17) , which is substantially worse than our proposed algorithm.
Another line of research is to not assume the existence of a coherent ranking, i.e., the comparisonprobabilities p
i,j can take any values in (0 , 1) as long as p i,j + p j,i = 1 .
In such scenarios, rankingscan be defined and derived based on various criteria including Borda score [13, 16, 25] and Copelandscore [3, 30].
The ranking problem has also been studied from a heterogeneous perspective [28, 15],where queries are made by multiple agents with different comparison probabilities.
More broadly,the problems of ranking, maxing or selection can be formulated in the context of dueling bandits.
Acomprehensive survey can be found in [2].
Proposed algorithm
In this section, we propose a δ -correct algorithm for exact ranking of all problem instances thatsatisfy the WST condition.
As mentioned previously, our algorithm is designed to take into accountsituations where nonadjacent items can be more difficult to compare than adjacent items.
To avoid spending unnecessary samples on item pairs with small probability gaps, we propose asubroutine named Successive-Comparison (SC) (see Subroutine 1).
SC uses a parameter τ forcontrolling to what extent should the comparison lasts.
Specifically, SC compares the given item pairfor a fixed number b τ = (2 /ϵ 2 τ ) log(1 /δ τ ) times with an accuracy level ϵ τ = 2 − τ and confidencelevel δ τ = 6 δ/ ( τ 2 π 2 ) .
If the empirical probability that i (respectively, j ) wins is over 1 / 2 by morethan ϵ τ / 2 , then SC returns i (respectively, j ) as the more preferred item.
Otherwise, SC will return‘unsure’ to inform us that more samples are needed.
For two items i and j , SC ( i, j, δ, τ ) will be called successively with τ increasing by 1 at a time.
Weshow in Appendix A that after τ gets large enough such that ϵ τ ≤ ∆ i,j , the correct ordering betweeni and j will be returned with high probability. Comparison i, j,
SC
Partial order preserving graph During the ranking process, we maintain a directed graph T tostore the partial orders we have obtained from SC instances so far.
The graph T is initialized with nnodes V 1 , . .
. , V n and no edge exists between any two nodes.
Nodes V 1 , V 2 , . .
, V n represent items1 , 2 , . . . , n , respectively.
In our algorithm, T is involved with three types of operations, edge update ,node removal and maximal set selection .
Every time an instance of SC returns a pairwise order, e.g.,i ≻ j , we add a directed edge from V i to V j , written as T
=
T ∪ ( i ≻ j ) .
Moreover, we also completeall edges in the transitive closure of the existing edges.
In other words, if the edge between V i and
V j induces a directed path from V k 1 to V k 2 , then a directed edge from V k 1 to V k 2 is also added to T .
By completing the transitive closure, we can avoid comparing pairs whose ordering can be inferredfrom current knowledge and keep T acyclic.
In the ranking process, we only run comparisons on item pairs that are not connected by edges and hence no contradictions in orderings will be returnedby SC .
By removing node V
i , we remove V i and all edges of V i from T .
The maximal elements of
T are the nodes which do not have any incoming edges.
Since edges represent comparison resultsreturned by SC , maximal elements correspond to items that have not lost to any other items.
Notethat since T is acyclic, maximal elements always exist.
Algorithm Next, we establish our ranking algorithm Probe-Rank (see Algorithm 2).
Probe-Rank finds the trueranking by performing maxing for n − 1 rounds.
In every round t , subroutine Probe-Max returns anitem in S t as the most preferred item (the maximum), where S t denotes the set of remaining unrankeditems right before round t .
The strategy of Probe-Max is to repeatedly apply SC on all item pairs.
Forevery item pair ( i, j ) , we initialize a global variable τ
i,j as the probing parameter for all SC instancesthat run over i, j .
The graph T storing obtained partial orders is also viewed as a global variable.
Parameters τ i,j and graph T will be accessed and altered in Probe-Max.
In Probe-Max ( S, δ ) (see Subroutine 3), SC instances are performed only on items that are possible tobe the actual maximum.
Let U be the set of maximal elements in T .
By definition, every item in Uhas not lost to any other item in S yet.
Assuming all previous comparison results are correct, to findthe actual maximum, it suffices to focus on items in U .
We use S 2 to denote the set of all unordereditem pairs in S , i.e., S 2 = { ( a, b ) : a, b ∈ S, a ̸ = b } .
All ‘legitimate’ pairs that can potentiallyprovide us with information about the maximum item in S are thus
[Equation]
where ( i, j ) / ∈ T means that nodes V i and V j are not connected in T .
While U contains more than oneitems, Probe-Max keeps applying SC on item pairs in P .
If an item in U loses a comparison, then weremove it from U .
In every iteration of the while loop, the pair ( i ∗ , j ∗ ) in P with the smallest τ valueis chosen (break ties arbitrarily) and SC ( i ∗ , j ∗ , δ, τ i ∗ ,j ∗ ) is performed.
Note that the τ value increasesby one after each call of SC .
To start with item pairs with small τ values guarantees that we do notmiss any useful information that can be obtained by paying only a small amount of comparisons.
Subroutine 3 Probe-Max ( S, δ )
1: Input: set of unranked items S , SC confidence level δ 2: Let U be the set of maximal elements according to T 3: while | U | > 1 do 4:
Let P = { ( i, j ) : ( i ∈ U or j ∈ U ) , ( i, j ) ∈ S 2 , ( i, j ) / ∈ T } ; 5: ( a, b ) = argmin
( x,y ) ∈
P τ x,y ; Ans =
SC ( a, b, δ, τ a,b ) ; τ a,b = τ a,b + 1 ; 6: if Ans is not ‘unsure’ then w, l = Ans ; T = T ∪ ( w ≻ l ) ; 7: if | U | > 1 and l ∈ U then U =
U \ { l } ; 8: end if 9:
We provide an example demonstrating the ranking process.
Example 1.
Consider items { 1 , 2 , 3 , 4 } with true ranking 1 ≻ 2 ≻ 3 ≻ 4 .
Figure 1 shows the statusof T, U, S t throughout the ranking process.
In particular, we assume the pairwise comparison resultsare all correct and returned in order 1 ≻ 2 , 2 ≻ 4 , 1 ≻ 3 , 2 ≻ 3 , 3 ≻ 4 .
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
Upper bound on the sample complexity of Probe-Rank
In this section, we provide a sample complexity upper bound for the proposed algorithm Probe-Rank.
Theorem 2.
Let δ > 0 be an arbitrary constant.
For all problem instances satisfying the Weak
Stochastic Transitivity (WST) property, with probability at least 1 − δ , Probe-Rank returns the trueranking of n items and conducts at most
O
[Equation]
(5) comparisons, where (cid:101) ∆ i is defined as in (3) .
The proof of Theorem 2 is deferred to Appendix A.
By the preceding theorem, the sample complexity of Probe-Rank is upper bounded by the sum ofterms ( (cid:101) ∆ i ) − 2 (log log( (cid:101) ∆ i ) − 1 + log( n/δ )) with an additional multiplicative factor of n .
Recall from
Section 2 that the term ( (cid:101) ∆ i ) − 2 (log log( (cid:101) ∆ i ) − 1 + log( n/δ )) can be viewed as a lower bound on thenumber of comparisons that is needed for obtaining the order between i and its adjacent items withconfidence level δ/n .
Theorem 2 thus suggests that in Probe-Rank, every item is compared until itcan be distinguished from its neighbors and no further.
This matches with our intuition that onlycomparisons between adjacent items are necessary, and a single nonadjacent pair being extremelyhard to distinguish should not harm the overall sample complexity.
In contrast, sample complexitiesof existing algorithms are determined by the smallest probability gap between items, which can leadto a substantially large amount of unnecessary comparisons.
However, Probe-Rank achieves the dependence on (cid:101) ∆ i instead of ∆ i at the cost of an additionalmultiplicative factor of n .
Intuitively, because we have zero prior information about which items areadjacent and which are not, Probe-Rank pays Θ ( n ) attempts for each item i in order to ‘identify’ itsneighbors and get the ordering feedback.
We compare Probe-Rank with the state-of-the-art IIR algorithm.
Let C ( Probe ) and C ( IIR ) denotethe sample complexities of two algorithms.
From Table 1 and Theorem 2,
[Equation]
noting that from the proofs, the sample complexity upper bounds are both tight in the worst case.
Under WST with no other conditions assumed, ∆ i ≤ (cid:101) ∆ i .
In particular, when (cid:101) ∆ i / ∆ i = Θ( √ n ) for alli , then C ( Probe ) and C ( IIR ) are of the same asymptotic order with respect to n ; if (cid:101) ∆ i / ∆ i = ω ( √ n ) ,then Probe-Rank is asymptotically more sample-efficient than IIR.
These phenomena are also reflectedin our numerical experiments in Section 6 (see Figure 3).
Remark.
It is worth noting that IIR is optimal if the more strict assumption SST as well as someother conditions are made, as shown in [22].
When SST holds, (cid:101) ∆ i = ∆ i .
Probe-Rank thus suffersfrom an addition factor of n .
This case is also included in our numerical experiment (see Figure 2(a)).
Experiments
In this section, we present numerical experiments demonstrating the practical performance of ProbeRank.
We compare Probe-Rank with the IIR algorithm, which was shown to outperform all the otherbaseline algorithms both theoretically and numerically [22].
We study different settings where SST is satisfied, not guaranteed, or violated, but WST alwaysholds, which is consistent with our theory.
Specifically, we want to rank n items with the trueranking σ 1 ≻ σ 2 ≻ · · · ≻ σ n , where n varies over [10 , 100] .
The probabilistic comparison modelp ij is generated in different ways to satisfy different assumptions.
Note that ∆ and ∆ d are tuningparameters in all the following settings.
• SST :
SST is satisfied.
p ij is generated from the MNL model, where p σ i ,σ j = (exp( s σ
i − s σ j )+1) − 1 ,and s σ 1 , . . .
, s
σ n is a decreasing sequence where s σ i = 100∆ d · ( n +1 − i ) n .
• WST : SST does not necessarily hold.
Let p i,j ∼ Uni ( 12 + ∆ d , 1) for all items i ≻ j .
• NON-SST :
SST does not hold.
For adjacent items, we have p σ i ,σ i +1 ∼ Uni (cid:0) 12 + ∆ d , 1 (cid:1) .
Otherwise,we have p σ i ,σ j ∼ Uni (cid:0) 12 + ∆ d 10 , 12 + ∆ d (cid:1) for j > i + 1 .
• ADJ-ASYM : SST does not hold.
This setting is used to verify the asymptotic analysis in Section 5. For adjacent items, we set p σ i ,σ i +
= 12 + ∆ d .
Otherwise, we set p σ i ,σ j = 12 + ∆ d n α for j > i + 1 .•
ADJ-CNST :
SST does not hold.
For adjacent items, we set p σ i ,σ i +
= 12 + ∆ .
Otherwise p σ i ,σ j =12 + ∆ d for j > i + 1 .
Here ∆ > ∆ d .
All experiments are averaged over 100 independent trials.
For each trial, the ground truth ranking σis generated uniformly at random and the comparison probabilities are assigned accordingly.
Theconfidence level δ is fixed to be 0 .
Throughout the experiment, every trial for every algorithmsuccessfully recovered the correct ranking.
We use internal clusters of intel “Skylake” generation CPUs.
Each job contains a single model typefor item numbers ranging from 10 to 100 with a step size of 10.
Models are generated from a jobunique random seed shared among the two algorithms.
Most jobs with sample complexity smallerthan 10 7 terminate in 3 minutes.
For ∆ d = 0 . 1 under the ADJ-ASYM model, 3 hours are needed dueto high sample complexity.
Due to the space limit, more detailed experimental setups and thoroughablation studies can be found in Appendix E.
[Figure]
Performance comparison Figure 2 with y-axis in log-scale shows comparison of IIR and ProbeRanking under the SST , WST and NON-SST settings.
The parameter ∆ d is set to be 0 .
It can be seenthat under the SST and WST settings (Figures 2(a), 2(b)), Probe-Rank consumes less samples than IIRfor small n .
As n gets larger, however, IIR becomes more sample-efficient due to that Probe-Rankhas an additional factor of n in its sample complexity compared with IIR for instances satisfy SST.
However, under the NON-SST setting where SST does not hold, Probe-Rank has a clear advantageover IIR, as shown in Figure 2(c).
Dependence on n and the probability gaps Following Theorem 2, we verify that the samplecomplexity of Probe-Rank is lower than IIR when the number of items n gets larger.
We use the
[Figure]
where nonadjacent items can be muchmore difficult to compare.
In particular, we choose α = 1 (see Figure 3(a))and
α = 1 / 2 (see Figure 3(b)).
Itcan be seen from Figure 3(a) that asthe number of items n gets larger, thegap between the two curves also getslarger.
This matches our analysis thatwhen (cid:101) ∆ i / ∆ i = ω ( √ n ) , then the sample complexity of IIR is of higher order than that of Probe-Rank.
When(cid:101) ∆ i / ∆ i = Θ( √ n ) , Figure 3(b) showsthat the gap between the two sample complexities varies little as n increases.
Our analysis alsosuggests that sample complexities of two algorithms are of the same order.
Furthermore, we show through the NON-SST and ADJ-CNST settings that when the probability gapsof nonadjacent item pairs decrease, the advantage of our algorithm will be more and more prominent.
[Figure]
In Figure 4, we fix n = 80 and let ∆ dvary.
Clearly, Probe-Rank has an advantage over IIR in both settings.
In particular, Figure 4(b) shows the comparisonof two algorithms in the ADJ-CNST setting with the probability gaps betweenadjacent items ∆ fixed as 0 . 4 .
As theprobability gap between nonadjacentitems ∆ d varies from 0 .
01 to 0 .
4 , it canbe seen that the sample complexity of
Probe-Rank does not vary much.
However, the sample complexity of IIR hasa positive correlation with 1∆ 2 d .
Thisnumerical result matches our analysisthat Probe-Ranking is not affected bythe comparison probability of nonadjacent items, which does not hold for IIR.
Discussion on the lower bound
In this section, we provide some insights about the lower bound for pairwise ranking by proposing aconjecture for a particularly hard instance that satisfies the WST condition.
This problem instance I WST is constructed as follows.
For all i ≻ j ,
[Equation]
where c and ∆ are constants and n − 10 can be replaced by any other quantity that is smaller than n − 2 .
By a reduction, any δ -correct algorithm that finds the maximum item for I WST can be constructed tosolve Problem 3, described below.
Therefore, a lower bound on the sample complexity for Problem 3will imply a lower bound of the same order for the maxing (and thus, ranking) problem for I WST .
This lower bound is also a worst-case lower bound for ranking under WST.
In the following, weprovide an analysis for Problem 3.
The reduction technique will be deferred to Appendix D.
Problem 3.
Consider vectors q 1 , q 2 , . .
. , q n .
Each q i is of length n .
Let q i,k denote the k -th entry ofq i .
Every entry of q 1 is Ber (1 / 2) .
For i > 1 , the i -th entry of q i , q i,i , is Ber (1 / 2 − 2∆) , and otherentries are all Ber (1 / 2) .
Given the n vectors, but randomly permuted, the agent is asked to predictwhich vector is q 1 , i.e., the one with all entries Ber (1 / 2) by sampling the n 2 entries.
All entries areindependent.
We conjecture that any δ -correct algorithm for Problem 3 has a sample complexity at least
[Equation]
We start from viewing it as a hypothesis testing problem.
Let p 1 , . .
. , p n be the vectors afterpermutation.
Consider that the agent is asked to determine if p 1 satisfies hypothesis H 0 :
[Equation]
or H j , in which the j -th entry is biased:
[Equation]
Suppose the hypothesis testing algorithm
A is δ -correct and stops within T rounds of interactions.
We denote A ( T ) as the output at the T -th round, which is either
(accept H 0 ) or 1 (reject H 0 ).
Forany given j ̸ = 1 , by the Bretagnolle–Huber inequality, we have
[Equation]
where P 0 is the probability measure under H 0 , and P A 0 is the probability measure of the canonicalbandit model under H 0 .
In fact, we have the divergence decomposition lemma [17, Lemma 15.1]:
[Equation]
where E 0 denotes the expectation under H 0 ; E 0 [ N k ( T )] denotes under H 0 , the expected number ofqueries for the entry p 1 ,k within T rounds.
; P 0 ,k , P j,k are the Bernoulli distributions specified by p 1 ,kunder H 0 , H j , respectively.
The second equality is due to the fact that the only difference between
H 0 and H j is that the j -th entry has different Bernoulli distributions.
Combining the two inequality above gives:
[Equation]
Since KL ( Ber (1 / 2) ||
Ber (1 / 2 − x ))
<
(4 x ) 2 for all x < 2 / 9 , we get E
[ N j ( T )] = Ω(∆ − 2 log(1 /δ )) .
Thus, the total expected number of queries unde H 0 will be Ω( n ∆ − 2 log(1 /δ )) .
In Problem 3, there are in total n vectors.
To identify which vector satisfies H 0 requires at least Ω( n )attempts, with each attempt costs Ω( n ∆ − 2 log(1 /δ )) .
Thus, we conjecture any δ -correct algorithmrequires Ω (cid:0) n 2 log(1 /δ ) /
∆ 2 (cid:1) queries.
Conclusion and future work
In this paper, we studied the problem of exact ranking under the most general assumption WST.
We proposed a δ -correct algorithm Probe-Rank, and derived an instance-wise upper bound on itssample complexity.
The upper bound shows that the performance of Probe-Rank only depend onthe comparison probabilities of adjacent items and thus improves existing results when SST doesnot hold.
Numerical results also suggest that our ranking algorithm outperforms state-of-the-art.
Adiscussion over the lower bound for pairwise ranking is also provided.
We propose a conjecture thatin the worst case, any algorithm has sample complexity n times the number of comparisons neededfor comparing all adjacent items.
However, it remains an open problem whether our conjecture holdsand will be left to future work.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] We mentioned that the pro- posed algorithm has an additional factor of n compared with state-of-the-art.
We also illustrated that the proposed lower bound is a conjecture.
(c) Did you discuss any potential negative societal impacts of your work?
[No] We believe that our work will not cause any negative societal impact.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes](b) Did you include complete proofs of all theoretical results?
[Yes] Due to space limitation,a part of the proofs is deferred to in Appendices A and D.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex-
perimental results (either in the supplemental material or as a URL)?
[Yes] The code is provided as supplementary material for the review process for anonymity.
Upon acceptance of the paper, it will be uploaded online for public access with GPLv
license.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes]
(c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]4.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
We do not use
any existing assets.
For algorithms, we implement our version of algorithms in cited
resources.
For dataset, it is generated purely by code.
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A
[-By the union bound and Lemma 
-]