Sample Complexity Bounds for Active Ranking from Multi-wise Comparisons

1 Introduction . 1.1 Background and motivation . Ranking from comparisons is a class of fundamental problems that underpin many areas in machine learning , and has found various applications in problems involving crowd-sourcing , social choices , recommendation , and searching . In such ranking problems , there is a hidden ranking among multiple items to be recovered , where items may refer to candidates , products , movies , advertisements , etc . In this paper , we study ranking from multi-wise comparisons . A multi-wise comparison refers to a query on m items about the most preferred one ( the winner feedback ) or the full ranking ( the full-ranking feedback ) of these items . These comparisons may be deterministic or non-deterministic ( i.e. , noisy or they may return incorrect results ) . The noise comes from the uncertain nature of humans , the lack of information , or the underlying physics . In this paper , we focus on two goals . One is to find the top-k items ( ranking or ordering these items are not necessary ) , and the other is to find the full ranking . Our focus is on active ranking ( e.g. , [ 4–7 , 10 , 17–19 ] ) , where “ active ” means that after each comparison , the learner can adaptively choose the next items to be compared according to past observations 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . and comparison results . Active ranking can be viewed as active learning for ranking problems and the comparisons refer to the samples . The opposite of active ranking is passive ranking , where the learner first obtains a set of comparison results and then recovers a ranking from there . Active ranking can greatly reduce the sample complexity in many scenarios , e.g. , if all comparisons return correct results with probability 23 , passively ranking n items needs Ω ( n 2 ) comparisons [ 3 ] , while active ranking only needs O ( n log n ) 1 comparisons [ 9 , 18 ] . Most existing works have focused on ranking from pairwise comparisons . In contrast , we focus on ranking from multi-wise ( or m-wise ) comparisons . The pairwise comparisons can be viewed as multi-wise comparisons with m = 2 . One motivation is that in many scenarios , multi-wise comparisons are more common . For instance , in video streaming websites or e-shopping apps , customers or users are normally presented more than two options , and the choices made by users can be viewed as multi-wise comparisons that suggest their preferences over these options . Studying ranking from multi-wise comparisons is useful for these types of applications . Besides , ranking from multi-wise comparisons may also reduce the cost of the learning process . In some applications , conducting the comparisons may be expensive . For instance , to find the candidates that are most preferred by the voters , people may need to do a series of surveys . Each survey contains a query about the preference order of a voter . In this application , the cost of finding the voter and asking this voter to fill the survey could far outweigh the cost of filling the survey itself . This means that the cost of conducting a multi-wise comparison is almost the same as that for a pairwise comparison , and how to reduce the number of samples by multi-wise comparisons becomes more interesting . Thus , it is not only interesting but also significant to study whether and to what degree we can reduce the sample complexity for ranking by using multi-wise comparisons . We focus on a non-parametric model , where each comparison returns the correct result with a certain probability q > 12 and an arbitrary incorrect result otherwise . When q = 1 , the comparisons are deterministic and always return correct results , and when q < 1 , we say the comparisons are nondeterministic or noisy . This differs from parametric models2 that assume that each item holds a value representing the users ’ preference on this item . In parametric models , the comparisons may provide more information than the setting in this paper3 , and thus , the conclusion drawn under parametric models can not be directly applied to this paper . 1.2 Problem formulation . Assume that there are n items , indexed by 1 , 2 , 3 , ... n , that form the item set [ n ] 4 . We further assume that these items have a unique unknown true ranking r1 ≻ r2 ≻ · · · ≻ rn , where for any items i and j , notation i ≻ j means that item i ranks higher ( or is more preferred ) than item j . Assume that we can compare at most m items at a time . The comparisons can be either deterministic or non-deterministic . We also assume that the comparisons are independent across time , items , and sets , which is standard in the literature ( e.g. , [ 5–7 , 10 , 11 , 13 , 17–22 ] ) . Here , we note that the independence is based on the assumption that the hidden parameters and ranking are some fixed values . When the comparisons are deterministic , the comparisons always return the correct results , i.e. , the best item under the winner feedback model or the true ranking of the compared items under the full-ranking feedback model . In the deterministic case , our goal is to find the exact top-k items or the true ranking . When the comparisons are non-deterministic , we assume that they return the correct results with a certain fixed probability q ∈ ( 12 , 1 ) . In this case , it is infeasible to rank the items with 100 % confidence , and thus , our goal is to find the ranking with confidence 1− δ for some δ ∈ ( 0 , 12 ) . We focus on the case where q = 23 . We note that this does not lose much generality . When q > 2 3 , the 1All log in this paper are natural log unless explicitly noted . 2For instance , parametric models can be the Bradley-Terry-Luce ( BTL ) model [ 2 ] , the Plackett-Luce ( PL ) model [ 15 ] , or the multinomial logit ( MNL ) model [ 14 ] . 3For instance , given three items i , j , k with i being the best and k being the worst , in the setting of this paper with q = 0.7 , the multi-wise comparison returns the best item i with probability 0.7 , and return j or k with some unknown probabilities , which does not provides information about the orders of j and k. In contrast , in a parametric model , the comparison over { i , j , k } would return i , j , or k with probability 0.7 , 0.2 , and 0.1 , respectively , which not only provides information about the best one but also information for ordering j and k. 4For any positive integer l , we define [ l ] : = { 1 , 2 , 3 , ... , l } . algorithms and sample complexity bounds in this paper can be directly applied . When q < 23 , we can use repeated comparisons to simulate one comparison with correct probability at least 23 . To see this , consider a set over which a comparison returns the correct result with probability 12 +∆ > 1 2 . By comparing this set for ⌈ 12∆2 log 3⌉ times , the item or permutation that occurs most often is the correct result with probability at least 23 ( by Hoeffding Inequality ) . Thus , we can use the above method to substitute the comparisons in the algorithms designed for the case where q = 23 while only introducing an additional 1∆2 factor on the sample complexity . This ( 12 +∆ ) m-wise comparison model can also be justified in many scenarios that use an iterative subroutine to conduct m-wise comparisons over a set in a smaller time-scale . Here , we give a simple example . We use pi , S to denote the probability that item i wins the comparison performed on set S ( assuming |S| ≤ m ) and we use i∗ to denote the best item of S. If pi∗ , S ≥ pi , S +∆ for any item i ̸= i∗ and any set S , then by repeatedly comparing S for Θ ( 1∆2 log |S| ) times , one can find the best item of S with confidence 23 . We can use the above procedure as a subroutine in each iteration of our algorithms , and get the algorithms for this case while only introducing an additional logm factor to the sample complexities . In more general cases , we can use similar tricks but the additional factors may vary . 1.3 Main results . The main results of this paper are summarized in Table 1 . We note that due to space limitation , all proofs in this paper are left to the supplementary material . 2 Related works . When m = 2 and the comparisons are deterministic , the top-k selection problem becomes the classical pairwise k-selection problem , which requires Θ ( n ) comparisons [ 1 ] , and the full ranking problem becomes the classical pairwise sorting problem , which requires Θ ( n log n ) comparisons . Thus , ranking from multi-wise comparisons can also be viewed as extensions of these foundational problems . Surprisingly , these extensions have not been well understood . For top-k selection from non-deterministic pairwise comparisons , the authors of [ 9 ] showed that Θ ( n log kδ ) comparisons are necessary and sufficient to reach confidence level 1− δ . For probably approximately correct ( PAC ) 5 top-k selection with error tolerance ϵ > 0 , the authors of [ 5 , 7 ] proved a Θ ( nϵ2 log 1 δ ) bound for k = 1 , and the authors of [ 16 , 18 ] proved a Θ ( n ϵ2 log k δ ) bound for k ≤ n 2 . For full ranking from non-deterministic pairwise comparisons , an early work was [ 9 ] , whose authors proved that when the comparisons of all pairs have the same noise level , then to get the full ranking with confidence 1 − δ , Θ ( n log nδ ) comparisons are necessary and sufficient . The authors of [ 17 ] 5The PAC setting means that we want to find the ranking approximately with an error tolerance ϵ and a confidence 1− δ , where the ϵ tolerance means that for any two items i and j , item i is viewed to rank higher than item j if item i wins the comparisons over item j with probability 1 2 − ϵ or higher . extend the above results to the case where the comparisons on different pairs have different noise levels , and also obtained the Θ ( n log nδ ) bounds . The authors of [ 5–7 ] showed the Θ ( n ϵ2 log n δ ) order-wise tight upper and lower bounds for PAC full ranking with error tolerance ϵ > 0 . Thus , we can see that for m = 2 ( i.e. , ranking from pairwise comparisons ) , people have already established optimal sample complexity bounds for top-k selection and full ranking . However , the corresponding multi-wise ranking problems have been relatively under-explored . Most of these works have focused on parametric models such as the PL model and the MNL model . Under the MNL model , the work in [ 4 ] showed that under certain cases , one can get an m-reduction from m-wise comparisons over ranking from pairwise comparisons . However , for most cases , using m-wise comparisons does not reduce the sample complexity ( ignoring constant factors ) . For multi-wise full ranking under the MNL model , the works in [ 16 , 17 ] showed that the lower bound is the same as that using pairwise comparisons , i.e. , there is no reduction by using m-wise comparisons . Under the list-wise PL model with certain types of feedback , we may achieve up to m-reduction in the sample complexity according to [ 19 ] . In contrast to the above works , we do not assume any parametric model , and instead use a nonparametric model where each comparison returns a correct result with a certain fixed probability q . As noted in Section 1.1 , the results drawn from the parametric models can not be directly applied to this non-parametric model . Multi-wise ranking under this non-parametric setting is even less under-explored in the literature . The most related work to this paper is [ 17 ] , where the authors showed a Θ ( n lognlogm ) bound for full ranking from the winner feedback , which solves one of the eight cases ( top-k selection or full ranking , winner feedback or full-ranking feedback , deterministic feedback or non-deterministic feedback ) that is studied in this paper and shown in Table 1 . We will study the rest seven cases in this paper . 3 Top-k selection from winner feedback

Summary: This paper considers the problem of ranking from multi-wise comparisons where the goal is either to find the set of top-k items or to find a complete ranking over the items. The paper considers a model for multi-wise comparisons where the feedback is either-- (1) deterministic where the correct result is returned; or (2) probabilistic where the feedback is correct with probability larger than 1/2. The paper assumes that there is an underlying ordering over items and considers two models of feedback-- (1) winner feedback where the best item in a set is returned; (2) full-ranking feedback where a complete ordering over arms is returned. The paper proposes lower and upper bounds on sample complexity for each of these 8 problem settings and outlines the reduction in sample complexity due to the use of multi-wise comparisons.