Original: Active Learning with Label Comparisons
Revised: Active Learning with Label Comparisons

Original: Abstract
Revised: Abstract

Original: Supervised learning typically relies on manual annotation of the true labels.
Revised: Supervised learning typically relies on manual annotation of the true labels.

Original: When there are many potential classes, searching for the best one can be prohibitive for a human annotator.
Revised: When there are many potential classes, searching for the best one can be prohibitive for a human annotator.

Original: On the other hand, comparing two candidate labels is often much easier.
Revised: On the other hand, comparing two candidate labels is often much easier.

Original: We focus on this type of pairwise supervision and ask how it can be used effectively in learning, and in particular in active learning.
Revised: We focus on this type of pairwise supervision and ask how it can be used effectively in learning, and in particular in active learning.

Original: We obtain several insightful results in this context.
Revised: We obtain several insightful results in this context.

Original: In principle, finding the best of k labels can be done with k − 1 active queries.
Revised: In principle, finding the best of k labels can be done with k − 1 active queries.

Original: We show that there is a natural class where this approach is sub-optimal, and that there is a more comparison-efficient active learning scheme.
Revised: We show that there is a natural class where this approach is sub-optimal, and that there is a more comparison-efficient active learning scheme.

Original: A key element in our analysis is the “label neighborhood graph” of the true distribution, which has an edge between two classes if they share a decision boundary.
Revised: A key element in our analysis is the “label neighborhood graph” of the true distribution, which has an edge between two classes if they share a decision boundary.

Original: We also show that in the PAC setting, pairwise comparisons cannot provide improved sample complexity in the worst case.
Revised: We also show that in the PAC setting, pairwise comparisons cannot provide improved sample complexity in the worst case.

Original: We complement our theoretical results with experiments, clearly demonstrating the effect of the neighborhood graph on sample complexity.
Revised: We complement our theoretical results with experiments, clearly demonstrating the effect of the neighborhood graph on sample complexity.

Original:  INTRODUCTION
Revised:  INTRODUCTION

Original: Supervised learning is a central paradigm in the empirical success of machine learning in general, and deep learning in particular.
Revised: Supervised learning is a central paradigm in the empirical success of machine learning in general, and deep learning in particular.

Original: Despite the recent advances in unsupervised learning, and in particular self-training, large amounts of annotated data are still required in order to achieve high accuracy in many tasks.
Revised: Despite the recent advances in unsupervised learning, and in particular self-training, large amounts of annotated data are still required in order to achieve high accuracy in many tasks.

Original: The main difficulty with supervised learning is, of course, the manual effort needed for annotating examples.
Revised: The main difficulty with supervised learning is, of course, the manual effort needed for annotating examples.

Original: Annotation becomes particularly challenging when there are many classes to consider.
Revised: Annotation becomes particularly challenging when there are many classes to consider.

Original: For example, in a text summarization task, we can ask an annotator to write a summary of the source text but this will likely not result in the “best” summary
Revised: For example, in a text summarization task, we can ask an annotator to write a summary of the source text, but this will likely not result in the “best” summary.

Original: We could also present the annotator a summary and ask for feedback (e g. is it good) but the quality could be difficult to judge in isolation.
Revised: We could also present the annotator a summary and ask for feedback (e.g. is it good) but the quality could be difficult to judge in isolation.

Original: We could also ask the annotator to select the best summary  of acandidates set  (e.g. produced by a language model), but this could be taxing if not infeasible when there are many candidates.
Revised: We could also ask the annotator to select the best summary out of a set of candidates (e.g. produced by a language model), but this could be taxing if not infeasible when there are many candidates.

Original: Motivated by the above scenario, previous works [e.g., see
Revised: Motivated by the above scenario, previous works [e.g., see

Original: Stiennon et al., 2020, Ouyang et al., 2022, for a recent application to large language models] have considered an alternative, and arguably natural, form of supervision: “Label
Revised: Stiennon et al., 2020, Ouyang et al., 2022, for a recent application to large language models] have considered an alternative, and arguably natural, form of supervision: “Label

Original: Comparisons”.
Revised: Comparisons”.

Original: Instead of presenting many potential labels to the annotator (e.g., candidate text summaries), we only present two candidates and ask the annotator to choose the better one.
Revised: Instead of presenting many potential labels to the annotator (e.g., candidate text summaries), we only present two candidates and ask the annotator to choose the better one.

Original: For example, when summarizing Snow White, we can ask to compare the summaries “A story about an evil step-mother” and “A story about a girl  driven to the forest by an evil step-mother and lives with dwarves”.
Revised: For example, when summarizing Snow White, we can ask to compare the summaries “A story about an evil step-mother” and “A story about a girl who is driven to the forest by an evil step-mother and ends up living with dwarves”.

Original: Most annotators would easily choose the latter as a better summary.
Revised: Most annotators would easily choose the latter as a better summary.

Original: Label comparisons clearly require a much lighter cognitive load than considering all alternatives, and thus have high potential as an annotation mechanism.
Revised: Label comparisons clearly require a much lighter cognitive load than considering all alternatives, and thus have high potential as an annotation mechanism.

Original: However, our theoretical understanding of this mechanism is fairly limited.
Revised: However, our theoretical understanding of this mechanism is fairly limited.

Original: While there has been work on learning to rank, which also uses comparisons, the goal of label comparisons is typically not to learn a complete ranking, but rather to build a model that outputs optimal predictions.
Revised: While there has been work on learning to rank, which also uses comparisons, the goal of label comparisons is typically not to learn a complete ranking, but rather to build a model that outputs optimal predictions.

Original: Here we set out to analyze label comparisons from this perspective, and  obtain several surprising results and a new algorithm.
Revised: Here we set out to analyze label comparisons from this perspective, and we obtain several surprising results and a new algorithm.

Original: Our key question is what is the best way to learn with label comparisons.
Revised: Our key question is what is the best way to learn with label comparisons.

Original: We assume that during learning we can only ask an annotator for label comparisons and not, for example, for the ground-truth label of the input, which we refer to as an argmax query.
Revised: We assume that during learning we can only ask an annotator for label comparisons and not, for example, for the ground-truth label of the input, which we refer to as an argmax query.

Original: We then ask how one can design algorithms that make effective use of such queries, and what is the corresponding query complexity.
Revised: We then ask how one can design algorithms that make effective use of such queries, and what is the corresponding query complexity.

Original: Namely, how many queries are needed to achieve a given test error.
Revised: Namely, how many queries are needed to achieve a given test error.

Original: Perhaps, the most natural way of using comparisons is simply for finding the argmax label, which can be done via k − 1 active queries.
Revised: Perhaps the most natural way of using comparisons is simply for finding the argmax label, which can be done via k − 1 active queries.

Original: However, as we shall see, this is a suboptimal approach.
Revised: However, as we shall see, this is a suboptimal approach.

Original: The first question we ask is whether access to comparisons is more informative than access to the argmax.
Revised: The first question we ask is whether access to comparisons is more informative than access to the argmax.

Original: If we know all (cid:0) k 2 (cid:1) comparisons for x , we can also infer the argmax and so it would seems like the answer to this should be in the affirmative.
Revised: If we know all (cid:0) k 2 (cid:1) comparisons for x , we can also infer the argmax and so it would seem like the answer to this should be in the affirmative.

Original: Our first result shows that in the PAC setting, this is in fact not the case, and that knowing all comparisons may result in the same sample complexity as knowing only the argmax.
Revised: Our first result shows that in the PAC setting, this is in fact not the case, and that knowing all comparisons may result in the same sample complexity as knowing only the argmax.

Original: The intuition for this negative result is that for 1D classifiers, the informative points are those that lie close to the decision boundaries between classes, and the argmax label for these points can also be used to find the boundaries, so that comparisons do not provide further advantage.
Revised: The intuition for this negative result is that for 1D classifiers, the informative points are those that lie close to the decision boundaries between classes, and the argmax label for these points can also be used to find the boundaries, so that comparisons do not provide further advantage.

Original: The negative result above may seem to suggest that comparisons are only useful for inferring the argmax.
Revised: The negative result above may seem to suggest that comparisons are only useful for inferring the argmax.

Original: However, we show that in the case of active learning, comparisons can be used more effectively.
Revised: However, we show that in the case of active learning, comparisons can be used more effectively.

Original: We consider the setting where the active learner can choose which label comparison queries to request for a given input x (including not requesting any queries at all).
Revised: We consider the setting where the active learner can choose which label comparison queries to request for a given input x (including not requesting any queries at all).

Original: A natural approach here is to take a “standard” active learning algorithm based on argmax queries, and implement it using pairwise comparisons, by using k −active comparisons for each input x to obtain the argmax.
Revised: A natural approach here is to take a “standard” active learning algorithm based on argmax queries, and implement it using pairwise comparisons, by using k −active comparisons for each input x to obtain the argmax.

Original: This strategy results in an algorithm that asks γ ( k − 1) comparisons, where γ is the number of argmax queries used.
Revised: This strategy results in an algorithm that asks γ ( k − 1) comparisons, where γ is the number of argmax queries used.

Original: Here we show that one can in fact do better than simulating argmax, by asking the “right” comparisons in an active fashion.
Revised: Here we show that one can in fact do better than simulating argmax, by asking the “right” comparisons in an active fashion.

Original: These beneficial comparisons are closely related to the “Label Neighborhood Graph” (see Figure 2) where labels are neighbors if they share a decision boundary.
Revised: These beneficial comparisons are closely related to the “Label Neighborhood Graph” (see Figure 2) where labels are neighbors if they share a decision boundary.

Original: We show that it is sufficient to ask queries only about neighbor pairs in this graph.
Revised: We show that it is sufficient to ask queries only about neighbor pairs in this graph.

Original: Thus, if this graph is sparse, active learning can be implemented with fewer queries.
Revised: Thus, if this graph is sparse, active learning can be implemented with fewer queries.

Original: In particular, for linear classifiers in R , each class has at most two neighbors, and thus the neighborhood graph is very sparse, and our proposed active learning approach is highly effective.
Revised: In particular, for linear classifiers in R , each class has at most two neighbors, and thus the neighborhood graph is very sparse, and our proposed active learning approach is highly effective.

Original: Taken together, our results demonstrate the richness of the label-comparison setting, and the ways in which its query complexity depends on the structure of the data.
Revised: Taken together, our results demonstrate the richness of the label-comparison setting, and the ways in which its query complexity depends on the structure of the data.

Original:  RELATED WORK
Revised:  RELATED WORK

Original: Several lines of works have addressed alternative modes of supervision for multi-class learning.
Revised: Several lines of works have addressed alternative modes of supervision for multi-class learning.

Original: Bandit Feedback:
Revised: Bandit Feedback:

Original: In this setting (e.g., Kakade et al.
Revised: In this setting (e.g., Kakade et al.

Original: Crammer and Gentile [2013]) the learner only observes whether its predicted class is correct or not.
Revised: Crammer and Gentile [2013]) the learner only observes whether its predicted class is correct or not.

Original: On the one hand, this feedback is stronger than label comparisons, because positive bandit feedback implies knowledge of the argmax.
Revised: On the one hand, this feedback is stronger than label comparisons, because positive bandit feedback implies knowledge of the argmax.

Original: On the other hand, label comparisons provide more information than bandit feedback, because comparisons provide knowledge about relative ordering of non argmax labels.
Revised: On the other hand, label comparisons provide more information than bandit feedback, because comparisons provide knowledge about relative ordering of non-argmax labels.

Original: Maxing from pairwise comparisons.
Revised: Maxing from pairwise comparisons.

Original: Maximum selection (maxing) from noisy comparisons is well-studied problem.
Revised: Maximum selection (maxing) from noisy comparisons is well-studied problem.

Original: Falahatgar et al.
Revised: Falahatgar et al.

Original: [2018] give an overview of known results under various noise models.
Revised: [2018] give an overview of known results under various noise models.

Original: Differently, we show that, for multiclass learning, using comparisons to first learn the global structure of the problem is more efficient than only using them for maxing.
Revised: Here, we show that for multiclass learning, using comparisons to first learn the global structure of the problem is more efficient than only using them for maxing.

Original: Daskalakis et al.
Revised: Daskalakis et al.

Original: [2011] consider maxing in partially ordered sets, where some pairs may be incomparable, which is interesting to explore in our setting.
Revised: [2011] consider maxing in partially ordered sets, where some pairs may be incomparable, which is interesting to explore in our setting.

Original: Dueling Bandits: In online learning, learning from pairwise comparisons is studied under the dueling bandits setting [Saha et al., 2021, Dudík et al., 2015], in which the learner “pulls” a pair of arms and observes the result of a noisy comparison (duel) between them.
Revised: Dueling Bandits: In online learning, learning from pairwise comparisons is studied under the dueling bandits setting [Saha et al., 2021, Dudík et al., 2015], in which the learner “pulls” a pair of arms and observes the result of a noisy comparison (duel) between them.

Original: The objective in these cases is to minimize the regret w.r.t a solution-concept from the social choice literature, e.g. the Condorcet winner [Yue et al., 2012], Borda winner, Copeland winner, or theVon
Revised: The objective in these cases is to minimize the regret w.r.t a solution-concept from the social choice literature, such as the Condorcet winner [Yue et al., 2012], Borda winner, Copeland winner, or the

Original:  Neuman winner [Dudík et al., 2015].
Revised: Von Neuman winner [Dudík et al., 2015].

Original: The focus on such regret minimization objectives is principally different from ours, since our primary goal is to minimize the number of queries made, rather than minimizing an online loss.
Revised: The focus on such regret minimization objectives is principally different from ours, since our primary goal is to minimize the number of queries made, rather than minimizing an online loss.

Original: Active Learning with rich supervision: Several works have explored alternative forms of supervision.
Revised: [Table] with rich supervision: Several works have explored alternative forms of supervision.

Original: Balcan and
Revised: Balcan and

Original: Hanneke [2012] explore class-conditional queries, where the annotator is given a target label and a pool of examples, and must say whether one of the examples matches the target label.
Revised: Hanneke [2012] explore class-conditional queries, where the annotator is given a target label and a pool of examples, and must say whether one of the examples matches the target label.

Original: Several works [Kane et al., 2017, Hopkins et al., 2020, Xu et al., 2017] have studied comparison queries on instances, where the annotator receives two inputs x 1 , xand reports which one is more positive (for binary classification).
Revised: Several works [Kane et al., 2017, Hopkins et al., 2020, Xu et al., 2017] have studied comparison queries on instances, where the annotator receives two inputs x 1 , xand reports which one is more positive (for binary classification).

Original: Ben-Eliezer et al.
Revised: Ben-Eliezer et al.

Original: [2022] study active learning of polynomial threshold function in d = 1 using derivative queries (e.g., is a patient getting sicker or healthier?).
Revised: [2022] study active learning of polynomial threshold functions in d = 1 using derivative queries (e.g., is a patient getting sicker or healthier?).

Original: Our supervision is conceptually different from all of these, as it compares between several labels on the same sample x .
Revised: Our supervision is conceptually different from all of these, as it compares between several labels on the same example x .

Original: Learning Ranking as a Reward Signal: A recent line of work demonstrated that pairwise label-comparisons elicited from humans can be used to improve the performance of
Revised: Learning Ranking as a Reward Signal: A recent line of work demonstrated that pairwise label-comparisons elicited from humans can be used to improve the performance of

Original: LLMs.
Revised: LLMs.

Original: Stiennon et al.
Revised: Stiennon et al.

Original: [2020] collect a dataset of human comparisons between summaries of a given text, and use it to obtain better summarization policies, and
Revised: [2020] collect a dataset of human comparisons between summaries of a given text, and use it to obtain better summarization policies, and

Original: Ouyang et al.
Revised: Ouyang et al.

Original: [2022] extend this idea to aligning LLMs with user intent.
Revised: [2022] extend this idea to aligning LLMs with user intent.

Original: Our focus here is to understand the theoretical properties of such label comparisons, which we expect will result in more effective ways of collecting and using such comparisons.
Revised: Our focus here is to understand the theoretical properties of such label comparisons, which we expect will result in more effective ways of collecting and using such comparisons.

Original:  PRELIMINARIES
Revised:  PRELIMINARIES

Original: Multi-class learning .
Revised: Multi-class learning .

Original: Let X ⊂
Revised: Let X ⊂

Original: R
Revised: R

Original: d denote the feature space and Y denote the label space, consisting of k classes.
Revised: d denote the feature space and Y denote the label space, consisting of k classes.

Original: We use D to denote an (unknown) distribution on X and H to denote a class of target functions, f : X → R k .
Revised: We use D to denote an (unknown) distribution on X and H to denote a class of target functions, f : X → R k .

Original: In this work, our focus is on a realizable setting in which the target function is some (unknown) f
Revised: In this work, our focus is on a realizable setting in which the target function is some (unknown) f

Original: ⋆ ∈ H .
Revised: ⋆ ∈ H .

Original: For x ∈ R d and a class i ∈
Revised: For x ∈ R d and a class i ∈

Original: [ k ] , f i ( x ) is the score assigned to class i on instance x .
Revised: [ k ] , f i ( x ) is the score assigned to class i on instance x .

Original: Given a target function f
Revised: Given a target function f

Original: ⋆ , the loss of a candidate classifier f is the standard (multiclass) 0-1 loss: L ( f ) =
Revised: ⋆ , the loss of a candidate classifier f is the standard (multiclass) 0-1 loss: L ( f ) =

Original: Pr x ∼D [arg max i ∈ [ k ] f i ( x ) ̸ = arg max i ∈
Revised: Pr x ∼D [arg max i ∈ [ k ] f i ( x ) ̸ = arg max i ∈

Original: [ k ] f ⋆i ( x )] .
Revised: [ k ] f ⋆i ( x )] .

Original: Since we are interested in how the difficulty of learning scales with the number of classes k , we will explicitly parameterize hypothesis classes in terms of k , (cid:8)
Revised: Since we are interested in how the difficulty of learning scales with the number of classes k , we will explicitly parameterize hypothesis classes in terms of k , (cid:8)

Original: H k (cid:9) k ∈ N .
Revised: H k (cid:9) k ∈ N .

Original: For example, the class of homogeneous linear classifiers 1 over k ∈ N classes in dimension d is H k,d lin =
Revised: For example, the class of homogeneous linear classifiers 1 over k ∈ N classes in dimension d is H k,d lin =

Original: (cid:8) h ( · ; W ) :
Revised: (cid:8) h ( · ; W ) :

Original: W ∈ R k × d (cid:9) , where h ( x ; W ) =
Revised: W ∈ R k × d (cid:9) , where h ( x ; W ) =

Original: W x ∈ R k .
Revised: W x ∈ R k .

Original: Supervision Oracles.
Revised: Supervision Oracles.

Original: Pertinent to this work is a distinction between two types of access to the target multiclass function: argmax (i.e. label) queries and label-comparison queries .
Revised: Pertinent to this work is a distinction between two types of access to the target multiclass function: argmax (i.e. label) queries and label-comparison queries .

Original: Definition 3.1 (Supervision Oracles) .
Revised: Definition 3.1 (Supervision Oracles) .

Original: Given a target function f
Revised: Given a target function f

Original: ⋆ : X → R k , we define the following oracles:
Revised: ⋆ : X → R k , we define the following oracles:

Original: [Equation]
Revised: [Equation]

Original: In the rest of the manuscript we simply use A f ⋆ to denote the supervision oracle, where it’s understood that if it receives an input x it invokes the argmax oracle and if it receives a triplet x , j 1 , j 2 it invokes the comparisons oracle.
Revised: In the rest of the manuscript we simply use A f ⋆ to denote the supervision oracle, where it’s understood that if it receives an input x it invokes the argmax oracle and if it receives a triplet x , j 1 , j 2 it invokes the comparisons oracle.

Original:  PASSIVE LEARNING
Revised:  PASSIVE LEARNING

Original: We define the sample and query complexities of PAC learnability using both argmax and label-comparisons supervision.
Revised: We define the sample and query complexities of PAC learnability using both argmax and label-comparisons supervision.

Original: We begin with the usual passive learning setup, and differentiate between the situation in which every example arrives with its argmax (i.e., the standard PAC setup), and where every example arrives with all the (cid:0) k 2 (cid:1) pairwise label comparisons (essentially, the total order on the classes).
Revised: We begin with the usual passive learning setup, and differentiate between the situation in which every example arrives with its argmax (i.e., the standard PAC setup), and where every example arrives with all the (cid:0) k 2 (cid:1) pairwise label comparisons (essentially, the total order on the classes).

Original: Definition 4.1 (Sample complexity of passive learning with argmax supervision) .
Revised: Definition 4.1 (Sample complexity of passive learning with argmax supervision) .

Original: Fix a distribution D over X and a target function f
Revised: Fix a distribution D over X and a target function f

Original: ⋆ : X → R k .
Revised: ⋆ : X → R k .

Original: Let D f ⋆ denote the distribution on X × Y in which a sample ( x , y ) ∼ D f ⋆ is generated by drawing x ∼ D and taking y =
Revised: Let D f ⋆ denote the distribution on X × Y in which a sample ( x , y ) ∼ D f ⋆ is generated by drawing x ∼ D and taking y =

Original: A f ⋆ ( x ) .
Revised: A f ⋆ ( x ) .

Original: We say that the sample complexity of passively learning a class (cid:8) H k (cid:9) k ∈ N is m H : (0 , 1) × N → N if there exists a learning algorithm with the following property: for every distribution D on X , for every k ∈ N , for every f ⋆ ∈ H k , and for every ε ∈ (0 , 1) , given m ≥ m H ( ε, k )
Revised: We say that the sample complexity of passively learning a class (cid:8) H k (cid:9) k ∈ N is m H : (0 , 1) × N → N if there exists a learning algorithm with the following property: for every distribution D on X , for every k ∈ N , for every f ⋆ ∈ H k , and for every ε ∈ (0 , 1) , given m ≥ m H ( ε, k )

Original: i.i.d samples from D f ⋆ , the algorithm returns an hypothesis h s.t w.p at least 7 / 8 , L D ( h ) ≤ ε .
Revised: i.i.d samples from D f ⋆ , the algorithm returns an hypothesis h s.t w.p at least 1 − 1 / 15 , L D ( h ) ≤ ε .

Original: Definition 4.2 (Sample complexity of passive learning with label-comparisons.) .
Revised: Definition 4.2 (Sample complexity of passive learning with label-comparisons.) .

Original: Fix a distribution D over X and a target function f
Revised: Fix a distribution D over X and a target function f

Original: ⋆ : X → R k .
Revised: ⋆ : X → R k .

Original: Let D f ⋆ denote a distribution on
Revised: Let D f ⋆ denote a distribution on

Original: X ×{± 1 } k 2 where a sample ( x , { b ij } ki,j =1 ) is generated by drawing x ∼ D and for i, j ∈ [ k ] , taking b ij =
Revised: X ×{± 1 } k 2 where a sample ( x , { b ij } ki,j =1 ) is generated by drawing x ∼ D and for i, j ∈ [ k ] , taking b ij =

Original: A f ( x ; i, j ) .
Revised: A f ( x ; i, j ) .

Original: We say that the sample complexity of passively learning a class (cid:8) H k (cid:9) k ∈ N is m H : (0 , 1) × N → N if there exists a learning algorithm with the following property: for every distribution D on X , for every k ∈ N , for every f ⋆ ∈ H k , and for every ε ∈ (0 , 1) , given m ≥ m H ( ε, k )
Revised: We say that the sample complexity of passively learning a class (cid:8) H k (cid:9) k ∈ N is m H : (0 , 1) × N → N if there exists a learning algorithm with the following property: for every distribution D on X , for every k ∈ N , for every f ⋆ ∈ H k , and for every ε ∈ (0 , 1) , given m ≥ m H ( ε, k )

Original: i.i.d samples from D f ⋆ , the algorithm returns an hypothesis h s.t w.p at least 7 / 8 , L D ( h ) ≤ ε .
Revised: i.i.d samples from D f ⋆ , the algorithm returns an hypothesis h s.t w.p at least 1 − 1 / 15 , L D ( h ) ≤ ε .

Original: Note that in the latter setting, the learner receives strictly more information about every example: the argmax can always be inferred from the total order on the classes.
Revised: Note that in the latter setting, the learner receives strictly more information about every example than in the argmax supervision setting. Namely, the argmax can always be inferred from the total order on the classes.

Original: We will therefore consider label-comparisons as helpful in this setup if knowing all comparisons results in improvement to the sample complexity.
Revised: We will therefore consider label-comparisons as helpful in this setup if knowing all comparisons results in improvement to the sample complexity.

Original: Our first result is negative: in general, label-comparisons may not be helpful in the passive regime.
Revised: Our first result is negative: in general, labelcomparisons may not be helpful in the passive regime.

Original: Theorem 4.3.
Revised: Theorem 4.3.

Original: Any algorithm that PAC learns H k, 2 lin must use m H ( ϵ, k )
Revised: Any algorithm that PAC learns H k, 2 lin must use m H ( ϵ, k )

Original: ∈
Revised: ∈

Original: Ω( k/ϵ ) samples, irrespective of whether it has access to argmax or label-comparison supervision.
Revised: Ω( k/ϵ ) samples, irrespective of whether it has access to argmax or label-comparison supervision.

Original: Proof.
Revised: Proof.

Original: For regular PAC learning (with argmax supervision), the standard approach for lower bounding the sample complexity is to lower bound the Natarajan dimension [Natarajan, 1989].
Revised: For regular PAC learning (with argmax supervision), the standard approach for lower bounding the sample complexity is to lower bound the Natarajan dimension [Natarajan, 1989].

Original: To extend this result to the setting of Definition 4.2, we employ  the dimension introduced in Daniely and Shalev-Shwartz [2014], which we will refer to as the
Revised: To extend this result to the setting of Definition 4.2, we employ a suitable variant of the dimension introduced in Daniely and Shalev-Shwartz [2014]. Following

Original:  Daniely-Shwartz dimension.
Revised: Brukhim et al. [2022], we refer to it as the Daniely-Shwartz dimension.

Original: It provides a tighter lower bound on the sample complexity, and it is also easier to adapt to our label comparison setting.
Revised: It provides a tighter lower bound on the sample complexity, and it is also easier to adapt to our label comparison setting.

Original: To emphasize the difference between functions mapping x ∈ X to a single class y ∈ [ k ] and functions mapping x ∈ X to a total order over the k classes, we will denote the former with f and the latter with f ∇ (and likewise for hypotheses classes).
Revised: To emphasize the difference between functions mapping x ∈ X to a single class y ∈ [ k ] and functions mapping x ∈ X to a total order over the k classes, we will denote the former with f and the latter with f ∇ (and likewise for hypotheses classes).

Original: We write arg max f ∇ ( x ) ∈ [ k ] for the class ranked first in the total order f ∇ ( x ) .
Revised: We write arg max f ∇ ( x ) ∈ [ k ] for the class ranked first in the total order f ∇ ( x ) .

Original: Definition 4.4.
Revised: Definition 4.4.

Original: Given a set { x 1 , . . .
Revised: Given a set { x 1 , . . .

Original: , x n } ⊂ X , we say that f ∇ and g ∇ are x i -close if:
Revised: , x n } ⊂ X , we say that f ∇ and g ∇ are x i -close if:

Original: [Equation]
Revised: [Equation]

Original: With this we can define a variant of Definition 12 in Daniely and Shalev-Shwartz [2014] for the case of extra supervision.
Revised: With this we can define a variant of Definition 12 in Daniely and Shalev-Shwartz [2014] for the case of extra supervision.

Original: Definition 4.5 (The Daniely-Shwartz dimension for label comparisons.) .
Revised: Definition 4.5 (The Daniely-Shwartz dimension for label comparisons.) .

Original: A set { x 1 , . . . , x n } is shattered by H ∇ if there exists a  subset of functions H ′∇ ⊂ H ∇ with the following property: for every f ∇ ∈ H ′∇ and for every i ∈ [ n ] , there exists
Revised: A set { x 1 , . . . , x n } is shattered by H ∇ if there exists a finite subset of functions H ′∇ ⊂ H ∇ with the following property: for every f ∇ ∈ H ′∇ and for every i ∈ [ n ] , there exists

Original: g ∇ ∈
Revised: g ∇ ∈

Original: H ′∇ such that f ∇ , g ∇ are x i -close.
Revised: H ′∇ such that f ∇ , g ∇ are x iclose.

Original: The Daniely-Shwartz dimension of H ∇ , dim ( H ∇ ) , is the maximal cardinality of a shattered set.
Revised: The Daniely-Shwartz dimension of H ∇ , dim ( H ∇ ) , is the maximal cardinality of a shattered set.

Original: Daniely and Shalev-Shwartz [2014] prove that the sample complexity of  learning a class H  is Ω( dim ( H ) /ε ) .
Revised: In the Supplementary Material we prove that the sample complexity of passively learning a class H with labelcomparisons (Definition 4.2) is Ω( dim ( H ) /ε ) .

Original: Thus, our objective is to prove that dim ( H 2 ,k lin ) ∈ Ω( k ) .
Revised: Thus, our objective is to prove that dim ( H 2 ,k lin ) ∈ Ω( k ) .

Original: To show this, we will construct a shattered set of size k for H 2 , 2 k lin .
Revised: To show this, we will construct a shattered set of size k for H 2 , 2 k lin .

Original: Consider 2 k labels of the form ( b, i ) , where b ∈ { 0 , 1 } and i ∈ { 1 , ..., k } .
Revised: Consider 2 k labels of the form ( b, i ) , where b ∈ { 0 , 1 } and i ∈ { 1 , ..., k } .

Original: Partition the numbers 1 , ..., 3 k to k triples: { 1 , 2 , 3 } , { 4 , 5 , 6 } , . . .
Revised: Partition the numbers 1 , ..., 3 k to k triples: { 1 , 2 , 3 } , { 4 , 5 , 6 } , . . .

Original: { 3 k − 2 , 3 k − 1 , 3 k } .
Revised: { 3 k − 2 , 3 k − 1 , 3 k } .

Original: We claim that k middle points, S = { 2 , 5 , . . .
Revised: We claim that k middle points, S = { 2 , 5 , . . .

Original: 3 k − 1 } are shattered by H 2 , 2 k lin .
Revised: 3 k − 1 } are shattered by H 2 , 2 k lin .

Original: Showing this requires defining a subset F of H 2 , 2 k lin with the property of Definition 4.5.
Revised: Showing this requires defining a subset F of H 2 , 2 k lin with the property of Definition 4.5.

Original: To define each function f ∇ ∈ F we will use an equivalent parametrization of linear classifiers in 1d as 1NN classification.
Revised: To define each function f ∇ ∈ F we will use an equivalent parametrization of linear classifiers in 1d as 1NN classification.

Original: i.e., each total order in H 2 , 2 k lin is parameterized by c ∈ R 2 k , where the total order h ( x ; c ) is the one implied by sorting the classes according to the distance of their centers c to x .
Revised: i.e., each total order in H 2 , 2 k lin is parameterized by c ∈ R 2 k , where the total order h ( x ; c ) is the one implied by sorting the classes according to the distance of their centers c to x .

Original: See Figure 1 for an illustration.
Revised: See Figure 1 for an illustration.

Original: With this parameterization in mind, F consists of all functions which satisfy the following: for each i ≤ k , the centers corresponding to labels (0 , i ) and (1 , i ) are located in the i ’th triplet, and exactly one of them is located in the middle of the triplet, on the point 3 i − 1 .
Revised: With this parameterization in mind, F consists of all functions which satisfy the following: for each i ≤ k , the centers corresponding to labels (0 , i ) and (1 , i ) are located in the i ’th triplet, and exactly one of them is located in the middle of the triplet, on the point 3 i − 1 .

Original: By construction, |F| = 4 k
Revised: By construction, |F| = 4 k

Original: (for each of the k triplets we need to specify which of the two centers is located in the middle of the triplet, and whether to locate the other center on the left or on the right of it).
Revised: (for each of the k triplets we need to specify which of the two centers is located in the middle of the triplet, and whether to locate the other center on the left or on the right of it).

Original: To see that S is shattered, consider f ∇ ∈ F and a point 3 i − 1 ∈ S .
Revised: To see that S is shattered, consider f ∇ ∈ F and a point 3 i − 1 ∈ S .

Original: W.l.o.g, assume that the center located on 3 i −is (0 , i ) .
Revised: W.l.o.g, assume that the center located on 3 i −is (0 , i ) .

Original: We define g ∇ ∈ F based on the location of the center of (1 , i ) , which by definition of F , could be either to the right (on 3 i ) or to the left (on 3 i − 2 ).
Revised: We define g ∇ ∈ F based on the location of the center of (1 , i ) , which by definition of F , could be either to the right (on 3 i ) or to the left (on 3 i − 2 ).

Original: In the first case, g ∇ is obtained by shifting both centers one unit to the left: in g ∇ the center (0 , i ) is located on 3 i − 2 and the center (1 , i ) is located on 3 i − 1 .
Revised: In the first case, g ∇ is obtained by shifting both centers one unit to the left: in g ∇ the center (0 , i ) is located on 3 i − 2 and the center (1 , i ) is located on 3 i − 1 .

Original: In the second case, g ∇ is obtained by shifting both centers one unit to the right: in g ∇ the center (0 , i ) is located on 3 i and the center (1 , i ) is located on 3 i − 1 .
Revised: In the second case, g ∇ is obtained by shifting both centers one unit to the right: in g ∇ the center (0 , i ) is located on 3 i and the center (1 , i ) is located on 3 i − 1 .

Original: By the definition of F , g ∇ ∈ F .
Revised: By the definition of F , g ∇ ∈ F .

Original: Crucially, f ∇ and g ∇ are { 3 i − 1 } -close
Revised: Crucially, f ∇ and g ∇ are { 3 i − 1 } -close

Original: (Definition 4.4): moving from f ∇ to g ∇ the center located on 3 i − 1 (and therefore the argmax) has changed, but the total order for every other point in S is remained unchanged, per the requirement of Definition 4.4.
Revised: (Definition 4.4): moving from f ∇ to g ∇ the center located on 3 i − 1 (and therefore the argmax) has changed, but the total order for every other point in S is remained unchanged, per the requirement of Definition 4.4.

Original: This proves S is shattered, and so dim ( H 2 ,k lin )
Revised: This proves S is shattered, and so dim ( H 2 ,k lin )

Original: ∈ Ω( k ) , as required.
Revised: ∈ Ω( k ) , as required.

Revised: Remark.

Revised: An interesting open question is whether this negative result can be extended to other classes (e.g. linear classifiers in higher dimensions).

Revised: Technically, this requires lower bounding the the DS dimension of the class, as we did here for H 2 ,k lin .

Revised: We conjecture that for d ≫ 1 the negative result can be extended in a distribution-specific manner (e.g., restricting to distributions with properties such as margin and sparsity); see the discussion in the Supplementary Material, where we report experimental results for the passive learning setting.

Original:  ACTIVE LEARNING
Revised:  ACTIVE LEARNING

Original: Next, we consider the active learning setting.
Revised: Next, we consider the active learning setting.

Original: Specifically, we focus on pool-based active learning, where the learner has access to unlabeled samples and can decide which queries to ask the oracle for (including not asking any queries).
Revised: Specifically, we focus on pool-based active learning, where the learner has access to unlabeled samples and can decide which queries to ask the oracle for (including not asking any queries).

Original: The performance of the algorithm is now measured in terms of the query complexity, namely the number of queries it makes to the labeling oracle in question:
Revised: The performance of the algorithm is now measured in terms of the query complexity, namely the number of queries it makes to the labeling oracle in question.

Original: Definition 5.1 (Query complexity of active learning.) .
Revised: Definition 5.1 (Query complexity of active learning.) .

Original: The query complexity of actively learning a class (cid:8) H k (cid:9) k ∈ N is q H : (0 , 1) × N
Revised: The query complexity of actively learning a class (cid:8) H k (cid:9) k ∈ N is q H : (0 , 1) × N

Original: → N if there exists a function m H : (0 , 1) × N → N and a learning algorithm with the following property: for every distribution D on X , for every k ∈
Revised: → N if there exists a function m H : (0 , 1) × N → N and a learning algorithm with the following property: for every distribution D on X , for every k ∈

Original: N , for every f ⋆ ∈ H k , and for every ε ∈ (0 , 1) , given m ≥ m H ( ε, k ) i.i.d samples from D and at most q H ( ε, k ) queries to A f ⋆ , the algorithm returns an hypothesis
Revised: N , for every f ⋆ ∈ H k , and for every ε ∈ (0 , 1) , given m ≥ m H ( ε, k ) i.i.d samples from D and at most q H ( ε, k ) queries to A f ⋆ , the algorithm returns an hypothesis

Original: h s.t w.p at least 7 / 8 , L D ( h ) ≤ ε .
Revised: h s.t w.p at least 7 / 8 , L D ( h ) ≤ ε .

Original: We refer to q H as the query complexity of learning H with argmax supervision or with label-comparison supervision, depending the oracle A f ⋆ .
Revised: We refer to q H as the query complexity of learning H with argmax supervision or with label-comparison supervision, depending the oracle A f ⋆ .

Original: We note that every active learning algorithm that uses argmax queries can always be simulated using comparison queries: in the adaptive setting (where the choice of query to ask at time t can depend on the previous answers), k − 1 label-comparison queries suffice to implement a “tournament” that reveals the argmax.
Revised: We note that every active learning algorithm that uses argmax queries can always be simulated using comparison queries: in the adaptive setting (where the choice of query to ask at time t can depend on the previous answers), k − 1 label-comparison queries suffice to implement a “tournament” that reveals the argmax.

Original: This provides a generic way to use the label-comparison oracle: simply request the label-comparison queries necessary for a “regular” active learner.
Revised: This provides a generic way to use the label-comparison oracle: simply request the label-comparison queries necessary for a “regular” active learner.

Original: We therefore say that comparisons are useful for active learning if the number of label-comparison queries required to learn a class H is strictly lower than the number of label-comparison queries required to simulate the best active learner that uses argmax queries to learn H . 
Revised: We therefore say that comparisons are useful for active learning if the number of label-comparison queries required to learn a class H is strictly lower than the number of label-comparison queries required to simulate the best active learner that uses argmax queries to learn H . Let C denote the set of all the learned binary classifiers. C ) Definition 5.

Original: Interestingly, the distinction between passive and active learning is important.
Revised: Interestingly, the distinction between passive and active learning is important.

Original: Our main result is that when the learner is allowed to decide which queries to request, labelcomparisons are helpful: we provide a learning algorithm that uses label comparisons more efficiently than simply using them to implement the best “regular” active learner.
Revised: Our main result is that when the learner is allowed to decide which queries to request, labelcomparisons are helpful: we provide a learning algorithm that uses label comparisons more efficiently than simply using them to implement the best “regular” active learner.

Original: Theorem 5.2.
Revised: Theorem 5.2.

Original: The label-comparison query complexity for active learning H k, 2 lin is ˜ O
Revised: The label-comparison query complexity for active learning H k, 2 lin is ˜ O

Original: ( k · log 1 ε ) , whereas the query complexity of simulating the best argmax active learner is ˜Ω( k 2 · log 1 ε ) .
Revised: ( k · log 1 ε ) , whereas the query complexity of simulating the best argmax active learner is ˜Ω( k 2 · log 1 ε ) .

Original: The proof of Theorem 5.2 will employ a specific multiclass to binary reduction that uses the concept of the label neighborhood graph of the target classifier.
Revised: The proof of Theorem 5.2 will employ a specific multiclass to binary reduction that uses the concept of the label neighborhood graph of the target classifier.

Original: Intuitively, two classes

Original: Let C denote the set of all the learned binary classifiers.

Original: Return f ( G,C ) (Definition 5.4). i and j are considered neighboring if they share a decision boundary; i.e., there are two arbitrarily close points in R d , where for one the argmax is i and for the other the argmax is j .
Revised: Intuitively, two classes i and j are considered neighboring if they share a decision boundary; i.e., there are two arbitrarily close points in R d , where for one the argmax is i and for the other the argmax is j .

Original: See Figure 2 for an example of the label neighborhood graph of a linear classifier in d = 2 .
Revised: See Figure 2 for an example of the label neighborhood graph of a linear classifier in d = 2 .

Original: Definition 5.3 (Label Neighborhood graph) .
Revised: Definition 5.3 (Label Neighborhood graph) .

Original: For a  function f : R d → R k , the neighborhood graphG
Revised: Fix a continuous function f : R d → R k . The neighborhood graph

Original:  = G ( f ) is an undirected graph on k vertices, with an edge between vertices i ∈
Revised: G = G ( f ) is an undirected graph on k vertices, with an edge between vertices i ∈

Original: [ k ] and j ∈ [ k ] if and only if there exists x ∈ R d for which for every r ∈ [ k ] , f i ( x ) =
Revised: [ k ] and j ∈ [ k ] if and only if there exists x ∈ R d for which for every r ∈ [ k ] , f i ( x ) =

Original: f j ( x ) ≥ f r ( x ) .
Revised: f j ( x ) ≥ f r ( x ) .

Original: To simplify notation, we use ( i, j ) ∈ G to refer to an edge in G , and e ( G ) for the total number of edges.
Revised: To simplify notation, we use ( i, j ) ∈ G to refer to an edge in G , and e ( G ) for the total number of edges.

Original: The degree of i ∈ [ k ] is the number of neighbors i has in G .
Revised: The degree of i ∈ [ k ] is the number of neighbors i has in G .

Original: We next define NbrGraphM2B (Neighborhood Graph
Revised: We next define NbrGraphM2B (Neighborhood Graph

Original: Multiclass-to-Binary), a procedur for actively learning a multiclass classifier f using a neighborhood graph G (see
Revised: Multiclass-to-Binary), a procedure for actively learning a multiclass classifier f using a neighborhood graph G (see

Original: Algorithm 1).
Revised: Algorithm 1).

Original: Given as input a binary active learning algorithm and a neighborhood graph, it uses comparison queries to learn a binary classifier for distinguishing every pair of neighboring classes i, j in G .
Revised: Given as input a binary active learning algorithm and a neighborhood graph, it uses comparison queries to learn a binary classifier for distinguishing every pair of neighboring classes i, j in G .

Original: It then aggregates these into a multi-class classifier using the following scheme:
Revised: It then aggregates these into a multi-class classifier using the following scheme:

Original: Definition 5.4 (Binary to multiclass aggregation.) .
Revised: Definition 5.4 (Binary to multiclass aggregation.) .

Original: Fix ( G, C ) , where G is a neighborhood graph and C = { h ij } ( i,j ) ∈ G,i<
Revised: Fix ( G, C ) , where G is a neighborhood graph and C = { h ij } ( i,j ) ∈ G,i<

Original: j is a collection of binary classifiers, one for every edge in G .
Revised: j is a collection of binary classifiers, one for every edge in G .

Original: The graph-based aggregation of ( G, C ) is a function f
Revised: The graph-based aggregation of ( G, C ) is a function f

Original: ( G,C ) :
Revised: ( G,C ) :

Original: X → R k defined as follows:f
Revised: X → R k defined as follows:

Original: [Equation]
Revised: [Equation]

Original: Namely, the label of x is the class in [ k ] that won the largest fraction of “duels” against its neighbors in the graph G .
Revised: Namely, the label of x is the class in [ k ] that won the largest fraction of “duels” against its neighbors in the graph G .

Original: An important component in analyzing NbrGraphM2B is the following lemma.
Revised: An important component in analyzing NbrGraphM2B is the following lemma.

Original: It establishes that when invoked w.r.t the true neighborhood graph G ⋆ , if the binary classifiers are sufficiently accurate, then so is the resulting multiclass classifier.
Revised: It establishes that when invoked w.r.t the true neighborhood graph G ⋆ , if the binary classifiers are sufficiently accurate, then so is the resulting multiclass classifier.

Original: See  Supplementary C for  proof.
Revised: See the Supplementary Material for the proof.

Original: Lemma 5.5.
Revised: Lemma 5.5.

Original: Fix a distribution D on X and a classifier
Revised: Fix a distribution D on X and a classifier

Original: W ⋆ .
Revised: W ⋆ .

Original: Fix ( G, C ) .
Revised: Fix ( G, C ) .

Original: If G = G ( W ⋆ ) and every h ij ∈ C has error at most ε/ e ( G ) , then f ( G,C ) has error at most ε .
Revised: If G = G ( W ⋆ ) and every h ij ∈ C has error at most ε/ e ( G ) , then f ( G,C ) has error at most ε .

Original: From this, we obtain the following upper bound on the query complexity of learning H k,d lin using label comparisons.
Revised: From this, we obtain the following upper bound on the query complexity of learning H k,d lin using label comparisons.

Original: Corollary 5.6.
Revised: Corollary 5.6.

Original: If the target neighborhood graph G ⋆ is known, the label-comparison query complexity of learning
Revised: If the target neighborhood graph G ⋆ is known, the label-comparison query complexity of learning

Original: H k,d lin is O
Revised: H k,d lin is O

Original: ( e ( G ⋆ ) · q b ( ε/ e ( G ⋆ )) , where q b ( γ ) is the query complexity of active learning in the binary case (i.e. k = 2 ).
Revised: ( e ( G ⋆ ) · q b ( ε/ e ( G ⋆ )) , where q b ( γ ) is the query complexity of active learning in the binary case (i.e. k = 2 ).

Original: Corollary 5.6 suggests that label-comparisons will be useful when (i) the target neighborhood graph is sparse (has low degree), and (ii) it can be learned with relatively few labelcomparisons.
Revised: Corollary 5.6 suggests that label-comparisons will be useful when (i) the target neighborhood graph is sparse (has low degree), and (ii) it can be learned with relatively few labelcomparisons.

Original: We are now ready to prove Theorem 5.2: we will show that for learning H k, 2 lin (the class for which we demonstrated comparisons are not useful in the passive setting), both these conditions hold.
Revised: We are now ready to prove Theorem 5.2: we will show that for learning H k, 2 lin (the class for which we demonstrated comparisons are not useful in the passive setting), both these conditions hold.

Original: Hence, comparisons indeed provide a provable gain over argmax supervision.
Revised: Hence, comparisons indeed provide a provable gain over argmax supervision.

Original: Proof of Theorem 5.2 : We will begin by instantiating the bound from Corollary 5.6 for d = 1 .
Revised: Proof of Theorem 5.2 : We will begin by instantiating the bound from Corollary 5.6 for d = 1 .

Original: Consider the degree of the neighborhood graph.
Revised: Consider the degree of the neighborhood graph.

Original: Using the equivalent parameterization of linear classifiers in d = 1 (see Figure 1), it follows that every class i ∈ [ k ] has at most 2 neighbors: exactly the preceding and succeeding classes in the sorted order of the classes.
Revised: Using the equivalent parameterization of linear classifiers in d = 1 (see Figure 1), it follows that every class i ∈ [ k ] has at most 2 neighbors: exactly the preceding and succeeding classes in the sorted order of the classes.

Original: Thus, for every f ⋆ ∈ H k, 2 lin , e ( G ( f ⋆ ))
Revised: Thus, for every f ⋆ ∈ H k, 2 lin , e ( G ( f ⋆ ))

Original: =
Revised: =

Original: O ( k ) .
Revised: O ( k ) .

Original: Second, active learning in d = 1 is well-understood: unlike higher dimensions, the distribution-free query complexity of active learning for two classes is q b ( γ )
Revised: Second, active learning in d = 1 is well-understood: unlike higher dimensions, the distribution-free query complexity of active learning for two classes is q b ( γ )

Original: = log( 1 γ ) using binary search over R [Dasgupta, 2004].
Revised: = log( 1 γ ) using binary search over R [Dasgupta, 2004].

Original: Plugging both ofthese facts into the upper bound of Corollary 5.6, we conclude that the query complexity for learning H k, 2 lin when the target neighborhod graph is known is O ( k · log kε ) . It remains to reason about the number of label comparisons required to learn G ⋆ (i.e., the sorted order of the classes). We claim that O ( k log k ) label-comparison queries suffice for this purpose. Consider Algorithm 2.
Revised: Plugging both of

Original: It receives a sample of points n from D and uses exactly 2 k + n log n label comparisons to infer the sorted order of the classes they belong to.

Original: Thus, under the assumption that this procedure is correct, applying it to O ( k ) samples (enough to guarantee that w.p at least 7/8, every label that has mass under D is also represented in S ) indeed learns G ⋆ , and uses O ( k log k ) label comparisons to do so.

Original: To prove correctness of the algorithm it suffices to prove that the learned total order is identical to the true total order.

Original: Input: n i.i.d samples from D , x 1 , . . .
Revised: Input: n i.i.d samples from D , x 1 , . . .

Original: , x n .
Revised: , x n .

Original: Output: A neighborhood graph G .
Revised: Output: A neighborhood graph G .

Original: Set x L = min i x i and x R = max i x i .
Revised: Set x L = min i x i and x R = max i x i .

Revised: Use a comparison sorting procedure to obtain a total order over the k classes, i 1 ≻ · · · ≻ i k .

Original: for i < j ∈ [ k ] do Determine the partial order between i and j : i appears before j if and only if (i)
Revised: Every time the sorting procedure requires the comparison between classes i, j ∈ [ k ] , determine that i appears before j if and only if (i)

Original: A f ( x L , i, j ) =and A f ( x R , i, j ) = 0 , or (ii)
Revised: A f ( x L , i, j ) = and A f ( x R , i, j ) = 0 , or (ii)

Original: A f ( x L , i, j ) =  and A f ( x R , i, j )= 1 .
Revised: A f ( x L , i, j ) = 1 and A f ( x R , i, j )

Original: end for Use a sorting procedure to obtain a total order over the k classes, i 1≻ · · · ≻ i k .
Revised: = 1 .

Original: Define a neighborhood graph G with an edge between i and j iff classes are consecutive in the learned total order.
Revised: Define a neighborhood graph G with an edge between i and j iff classes are consecutive in the learned total order.

Original: Return G .
Revised: Return G .

Original: Indeed, first note that if A f ( x L , i, j )  =
Revised: these facts into the upper bound of Corollary 5.6, we conclude that the query complexity for learning H k, 2 lin when the target neighborhod graph is known is O ( k · log kε ) . Next, we turn to the question of learning G ⋆ using labelcomparison queries. Towards this, consider Algorithm 2. The algorithm receives a sample of n =

Original: A f ( x R , i, j )then the “losing” class is not an actual class, in that none of the n points belong to it.
Revised: O (1 /ε ) points from D and uses exactly 2 k log k label comparisons to return a neighborhood graph G .

Original: So we can safely place it last in the total order (effectively ignoring this class), and it will not effect the labels of x 1 ,  . . . , x n . 
Revised: As we claim below the graph G will be identical to G ⋆ , except for possibly a set of edges pertaining to classes outside S whose overall probability under D is smaller than ε . The key observation behind Algorithm 2 is that we can use exactly two label-comparison queries to infer whether a class i appears before a class j , as long as both classes are “represented” in S . 3 We can therefore use a total of 2 k log k queries to infer the total order of all the “represented” classes.

Original: On the other hand, if A f ( x L , i, j )̸ =
Revised: It remains to argue why O (1 /ε )

Original: A f ( x R, i, j ) , then their values directly determine the order of the classes: class i appears before class j if and only if A f ( x L , i, j ) = 1 and A f ( x R , i, j )
Revised: samples suffice to guarantee that with high probability, classes that are not “represented” by S have mass at most ε . To see this, fix D on R and denote F ( z ) = Pr x ∼D [ x < z ] . Let z be such that F ( z )

Original: = 0 .
Revised: = ε .

Original: We can now combine both parts (using Algorithm 2 to learn G ⋆ , and then using Algorithm NbrGraphM2B ) to conclude that the overall query complexity of learning H k, 2 lin is
Revised: We are interested in the number of samples n required to guarantee that Pr S ∼D n [min( S ) > z ] < δ . Now, Pr S [min( S ) >

Revised: z ] =

Original: O ( k log k
Revised: ((1 − F ( z )) n = (1 − ε ) n ≤ exp( − n · ε ) And exp( − n · ε ) ≤ δ ⇐⇒ n ≥ 1 ε log 1 δ . Similarly, the same number of samples can be used to bound the the "tail" beyond max( S ) .

Revised: Union-bounding over both events yields the required result.

Revised: To summarize, the full procedure for actively learning H k, 2 lin is to run NbrGraphM2B with the neighborhood graph G that is returned by Algorithm 2.

Revised: Combining Lemma 5.5 and the analysis of Algorithm 2, we conclude that this procedure has an overall unlabeled sample complexity of O (1 /ε ) , and an overall query complexity of O ( k log k

Original: + k log kε )
Revised: + k log kε )

Original: =
Revised: =

Original: ˜ O ( k · log 1 ε ) , as required.
Revised: ˜ O ( k · log 1 ε ) .

Original: To conclude the proof of Theorem 5.2, it remains to lower bound the complexity of learning with argmax queries.
Revised: To conclude the proof of Theorem 5.2, it remains to lower bound the complexity of learning with argmax queries.

Original: We will prove that Ω( k log k log kε ) argmax queries are needed.
Revised: We will prove that Ω( k log k log kε ) argmax queries are needed.

Original: This will imply that simulating any argmax active learning requires at least ˜Ω( k 2 · log 1 ε ) label-comparisons.
Revised: This will imply that simulating any argmax active learning requires at least ˜Ω( k 2 · log 1 ε ) label-comparisons.

Original: We will prove this via the label revealing task [e.g., see Kane et al., 2017], where the goal is to reveal the correct labels of a given (realizable) sample of n points, and show that O ( k log k log n ) argmax queries are needed to reveal all n labels.
Revised: We will prove this via the label revealing task [e.g., see Kane et al., 2017], where the goal is to reveal the correct labels of a given (realizable) sample of n points, and show that O ( k log k log n ) argmax queries are needed to reveal all n labels.

Original: Towards this, fix n points and consider a tree that denotes the run of an active learning algorithm (with nodes being the queries asked and the children the possible answers).
Revised: Towards this, fix n points and consider a tree that denotes the run of an active learning algorithm (with nodes being the queries asked and the children the possible answers).

Original: Note that the number of unique labelings corresponds to the number of leaves in the tree and the query complexity corresponds to the depth of the tree, which we denote q .
Revised: Note that the number of unique labelings corresponds to the number of leaves in the tree and the query complexity corresponds to the depth of the tree, which we denote q .

Original: The number of ways to arrange n points into k classes in 1d is k ! (cid:0) nk − 1 (cid:1) ( k ! options for ordering the classes and then (cid:0) n k − 1 (cid:1) options for locating the thresholds).
Revised: The number of ways to arrange n points into k classes in 1d is k ! (cid:0) nk − 1 (cid:1) ( k ! options for ordering the classes and then (cid:0) nk − 1 (cid:1) options for locating the thresholds).

Original: Since the degree of the tree is k for argmax queries, it must be that the k q ≥ k !
Revised: Since the degree of the tree is k for argmax queries, it must be that the k q ≥ k !

Original: (cid:0) nk − 1 (cid:1) , which implies 3 a lower bound q ≥ O ( k log k · log n ) .
Revised: (cid:0) nk − 1 (cid:1) , which implies 4 a lower bound q ≥ O ( k log k · log n ) .

Original: Together, this concludes the proof of Theorem 5.2.
Revised: Together, this concludes the proof of Theorem 5.2.

Original: □
Revised: □

Original: Algorithm

Original: Input: Label neighborhood graph G , buffer size R , steps

Original: T , confidence parameter τ , learning rate η , comparison oracle A f ⋆ .

Original: Output: classifier h ( · ; W ) , number of comparisons q .

Original: Initialize W (0) , L = 0 , q = 0 , b = 0 .

Original: for t = 1 , 2 , . . .

Original: , T do

Original: Sample x ∼ D .

Original: Sample ( i, j ) uniformly from the edges of G .

Original: if (cid:12)(cid:12)(cid:12) h

Original: i ( x ; W ( t − 1) ) − h j ( x ; W ( t − 1) )

Original: (cid:12)(cid:12)(cid:12) <

Original: τ then

Original: Obtain oracle comparison c = 2( A f ∗ ( x , i, j ) − 0 . 5)

Original: L += log(1 + e − c

Original: ( h i ( x ; W ) − h j ( x ; W )) ) .

Original: q += 1 , b

Original: += 1 .

Original: end if if b ≥ r then

Original: Update W ( t ) ← W ( t − 1) − η · ∂L∂ W

Original: Clear buffer: L = 0 , b = 0 .

Original: end if end for

Original: Our analysis suggests that when we can efficiently learn
Revised: Our analysis suggests that when we can efficiently learn

Original: G ⋆ and it is sparse, label-comparisons provide a gain over argmax queries.
Revised: G ⋆ and it is sparse, label-comparisons provide a gain over argmax queries.

Original: We showed this when d = 1 , and it is natural to ask to what happens for d > 1 .
Revised: We showed this when d = 1 , and it is natural to ask to what happens for d > 1 .

Original: This requires addressing both the question of what is the binary active learning primitive that we use, as well as the questions of sparsity and learning the graph.
Revised: This requires addressing both the question of what is the binary active learning primitive that we use, as well as the questions of sparsity and learning the graph.

Original: See Supplementary A for discussion of these aspects.
Revised: See Supplementary ?? for discussion of these aspects.

Original:  A GENERAL PURPOSE ACTIVE LEARNING ALGORITHM
Revised:  A GENERAL PURPOSE ACTIVE LEARNING ALGORITHM

Original: The approach of Algorithm 1 is to explicitly learn e ( G ) binary classifiers and aggregate them into a single classifier, that is not in H k,d lin .
Revised: The approach of Algorithm 1 is to explicitly learn e ( G ) binary classifiers and aggregate them into a single classifier, that is not in H k,d lin .

Original: For simplicity of optimization, we will prefer to work with models in H k,d lin .
Revised: For simplicity of optimization, we will prefer to work with models in H k,d lin .

Original: To do so, in Algorithmwe present the NbrGraphSGD algorithm, a natural variation which can work directly with such models.
Revised: To do so, in Algorithmwe present the NbrGraphSGD algorithm, a natural variation which can work directly with such models.

Original: It works as follows: we first initialize a multiclass model h ( · ; W ) :
Revised: It works as follows: we first initialize a multiclass model h ( · ; W ) :

Original: R d → R k (e.g. W ∈ R k,d for a linear model, but h can also be a neural network).
Revised: R d → R k (e.g. W ∈ R k,d for a linear model, but h can also be a neural network).

Original: For every data point x , we sample an edge ( i, j ) in the graph G .
Revised: For every data point x , we sample an edge ( i, j ) in the graph G .

Original: This edge is a candidate label comparison.
Revised: This edge is a candidate label comparison.

Original: To decide whether to query it or not, we evaluate the difference in logits between labels i and j .
Revised: To decide whether to query it or not, we evaluate the difference in logits between labels i and j .

Original: If this difference is smaller than τ we query the pair ( i, j ) and add a binary cross entropy term that encourages the logit difference to have the correct sign.
Revised: If this difference is smaller than τ we query the pair ( i, j ) [Table] do if (cid:12)(cid:12)(cid:12) h i ( x ; W ( t − 1) ) − h j ( x ; W ( t − 1) )

Original: Once we accumulate sufficiently many comparisons, we perform an update step.
Revised: (cid:12)(cid:12)(cid:12) < τ then Obtain oracle comparison c = 2( A f ∗ ( x , i, j ) − 0 . 5) L += log(1 + e − c

Original: k
Revised: ( h i ( x ; W ) − h j ( x ; W )) ) . q += 1 , b += 1 . end if if b ≥ r then

Original: [Equation]
Revised: Update − η ∂L∂ W

Original: (cid:16)(cid:2)
Revised: Clear end if end for and add a binary cross entropy term that encourages the logit difference to have the correct sign.

Original: [Equation]
Revised: Once we accumulate sufficiently many comparisons, we perform an update step.

Original: The remaining practical question is which graph G to use.
Revised: The remaining practical question is which graph G to use.

Original: Recall that G ⋆ has an edge ( i, j )
Revised: Recall that G ⋆ has an edge ( i, j )

Original: iff ∃ x where j was the 2nd best label
Revised: iff ∃ x where j was the 2nd best label

Original: and i was the argmax.
Revised: and i was the argmax.

Original: For 1d, we showed this could be learned from data effectively.
Revised: For 1d, we showed this could be learned from data effectively.

Original: We leave the general case open, and consider here practical recipes for G .
Revised: We leave the general case open, and consider here practical recipes for G .

Original: The simplest approach is to base G on prior knowledge regarding which classes are expected to be neighbors (e.g., via distances on their word embeddings, or other co-occurrence statistics).
Revised: The simplest approach is to base G on prior knowledge regarding which classes are expected to be neighbors (e.g., via distances on their word embeddings, or other co-occurrence statistics).

Original: Another practical case is when first and second best labels are available without the x values (e.g., consider asking individuals what are their first and second most favorite products, without keeping user info).
Revised: Another practical case is when first and second best labels are available without the x values (e.g., consider asking individuals what are their first and second most favorite products, without keeping user info).

Original: Note that in this case, we will receive evidence of edges only for x values sampled from D .
Revised: Note that in this case, we will receive evidence of edges only for x values sampled from D .

Original: This corresponds to an empirical notion of the neighborhood graph, which we define below.
Revised: This corresponds to an empirical notion of the neighborhood graph, which we define below.

Original: Definition 6.1 (Empirical Label Neighborhood graph) .
Revised: Definition 6.1 (Empirical Label Neighborhood graph) .

Original: For a target function f
Revised: For a target function f

Original: ⋆ : R d → R k , the neighborhood graph
Revised: ⋆ : R d → R k , the neighborhood graph

Original: G D ( f ) is an undirected graph on k vertices, where there is an edge between vertices i and j if and only if there exists x ∈ R
Revised: G D ( f ) is an undirected graph on k vertices, where there is an edge between vertices i and j if and only if there exists x ∈ R

Original: d whose probability under D is non-zero, and for which for every r ∈ [ k ] , f ⋆ ( x ) i
Revised: d whose probability under D is non-zero, and for which for every r ∈ [ k ] , f ⋆ ( x ) i

Original: =
Revised: =

Original: f ⋆ ( x ) j ≥ f ⋆ ( x ) r .
Revised: f ⋆ ( x ) j ≥ f ⋆ ( x ) r .

Original: By definition, G ⋆ D ⊆ G ⋆ .
Revised: By definition, G ⋆ D ⊆ G ⋆ .

Original: One might hope that the discarded edges will not impact accuracy w.r.t D .
Revised: One might hope that the discarded edges will not impact accuracy w.r.t D .

Original: However, in the worst-case this is not true.
Revised: However, in the worst-case this is not true.

Original: Specifically, in proving Lemma 5.5 we used the fact that when B is given by the true binary classifiers (i.e. h ij = W ⋆
Revised: Specifically, in proving Lemma 5.5 we used the fact that when B is given by the true binary classifiers (i.e. h ij = W ⋆

Original: i − W ⋆ j ), the aggregated classifier f
Revised: i − W ⋆ j ), the aggregated classifier f

Original: ( G ⋆ ,C ) has perfect accuracy on D .
Revised: ( G ⋆ ,C ) has perfect accuracy on D .

Original: This may fail for G ⋆ D : f ( G ⋆ D ,C ) may err on examples supported in D ; See 
Revised: This may fail for G ⋆ D : f ( G ⋆ D ,C ) may err on examples supported in D ; See the

Original: Supplementary Figure 6 for an example.
Revised: Supplementary Material for an example.

Original: In Section 7 we observe that the performance of both graphs is comparable.
Revised: In Section 7 we observe that the performance of both graphs is comparable.

Original:  EXPERIMENTS
Revised:  EXPERIMENTS

Original: In this section we evaluate our label-comparisons algorithm
Revised: In this section we evaluate our label-comparisons algorithm

Original: NbrGraphSGD on synthetic as well as real data.
Revised: NbrGraphSGD on synthetic as well as real data.

Original: We consider the online active learning scenario.
Revised: We consider the online active learning scenario.

Original: At each round t ∈ [ T ] = { 1 , ..., T } , the learner receives a batch of points drawn i.i.d.
Revised: At each round t ∈ [ T ] = { 1 , ..., T } , the learner receives a batch of points drawn i.i.d.

Original: according to D and must decide which queries to request from the oracle A f ⋆ (including not requesting any queries).
Revised: according to D and must decide which queries to request from the oracle A f ⋆ (including not requesting any queries).

Original: We compare the following methods:
Revised: We compare the following methods:

Original: • NbrGraphSGD ( G ) :
Revised: • NbrGraphSGD ( G ) :

Original: This is our algorithm which takes as inputgraph a  G and, for a given x , only considers labels pairs in G as possible pairs to query.
Revised: This is our algorithm which takes as input a graph G and, for a given x , only considers label pairs in G as possible pairs to query.

Original: For the given x , we iterate over all ( i, j ) ∈ G .
Revised: For the given x , we iterate over all ( i, j ) ∈ G .

Original: For each pair we check if | W i x − W j x | is smaller than a fixed threshold.
Revised: For each pair we check if | W i x − W j x | is smaller than a fixed threshold.

Original: If it is, we query this pair.
Revised: If it is, we query this pair.

Original: We consider different versions of NbrGraphSGD ( G ) , that use different graphs.
Revised: We consider different versions of NbrGraphSGD ( G ) , that use different graphs.

Original: • PassiveTour : This baseline uses label comparisons to simulate a standard argmax-based active learning algorithm
Revised: • PassiveTour : This baseline uses label comparisons to simulate a standard argmax-based active learning algorithm

Original: [Joshi et al., 2009].
Revised: [Joshi et al., 2009].

Original: Namely, for each x , we evaluate the logits W y x and query x if the difference between the first and second best logits is below some threshold.
Revised: Namely, for each x , we evaluate the logits W y x and query x if the difference between the first and second best logits is below some threshold.

Original: In the standard argmax setting, we would have requested the label of x .
Revised: In the standard argmax setting, we would have requested the label of x .

Original: With label comparisons, we need to do this using k − 1 active comparisons.
Revised: With label comparisons, we need to do this using k − 1 active comparisons.

Original: Namely, we perform a tournament between labels to reveal the maximizer.
Revised: Namely, we perform a tournament between labels to reveal the maximizer.

Original: ActiveTour :
Revised: ActiveTour :

Original: It may seem wasteful to ask for k − 1 comparisons as above, since we may be sufficiently confident in some of these comparisons.
Revised: It may seem wasteful to ask for k − 1 comparisons as above, since we may be sufficiently confident in some of these comparisons.

Original: We thus consider an “active tournament” algorithm: whenever the current model is sufficiently confident in a given pair in the tournament, we take the model’s answer, and do not query for it.
Revised: We thus consider an “active tournament” algorithm: whenever the current model is sufficiently confident in a given pair in the tournament, we take the model’s answer, and do not query for it.

Original: Evaluation.
Revised: Evaluation.

Original: In online active learning, the quality of an algorithm is measured by its accuracy after T rounds, and the total number of comparisons requested within these T rounds.
Revised: In online active learning, the quality of an algorithm is measured by its accuracy after T rounds, and the total number of comparisons requested within these T rounds.

Original: We use a linear teacher model to simulate the comparison oracle (Definition 3.1), and measure accuracy as the categorical accuracy 4 on the test set, w.r.t the teacher’s argmax.
Revised: We use a linear teacher model to simulate the comparison oracle (Definition 3.1), and measure accuracy as the categorical accuracy 5 on the test set, w.r.t the teacher’s argmax.

Original: We use an accumulating buffer mechanism to control for the number of parameter updates across methods (each method accumulates the requested comparisons until the buffer is full, and only then performs a gradient update).
Revised: We use an accumulating buffer mechanism to control for the number of parameter updates across methods (each method accumulates the requested comparisons until the buffer is full, and only then performs a gradient update).

Original: 7.1 SYNTHETIC DATA
Revised: 7.1 SYNTHETIC DATA

Original: In this section we validate our theoretical findings from
Revised: In this section we validate our theoretical findings from

Original: Sections 4 and 5 on synthetic data.
Revised: Sections 4 and 5 on synthetic data.

Revised: Specifically, for d ∈ N 5 Specifically, use Top-K accuracy, where K = 0 .

Revised: 1 · k .

Original: Specifically, for d ∈ N and ˆ k ∈ N we consider D to be the uniform distribution on a unit sphere in R d and draw a random linear target model
Revised: [Figure] Sp a r s i t y d=5, G*_D d=5, G* d=7, G*_D d=7, G* and ˆ k ∈ N we consider D to be the uniform distribution on a unit sphere in R d and draw a random linear target model

Original: W ⋆ ∈ R ˆ k,d .
Revised: W ⋆ ∈ R ˆ k,d .

Original: This yields a multiclass classifier with k ≤ ˆ k distinct decision regions (“effective classes”).
Revised: This yields a multiclass classifier with k ≤ ˆ k distinct decision regions (“effective classes”).

Original: We draw data from D and divide it into training and test sets.
Revised: We draw data from D and divide it into training and test sets.

Original: Sparsity of the neighborhood graph.
Revised: Sparsity of the neighborhood graph.

Original: We begin by computing the sparsity level of both the true neighborhood graph G ⋆ = G ( W ⋆ ) and the empirical neighborhood graph
Revised: We begin by computing the sparsity level of both the true neighborhood graph G ⋆ = G ( W ⋆ ) and the empirical neighborhood graph

Original: G ⋆ D = G D ( W ⋆ ) , where the latter is computed w.r.t the training set.
Revised: G ⋆ D = G D ( W ⋆ ) , where the latter is computed w.r.t the training set.

Original: We define the sparsity level as the number of edges in G , divided by (cid:0) k 2 (cid:1) (i.e,.
Revised: We define the sparsity level as the number of edges in G , divided by (cid:0) k 2 (cid:1) (i.e,.

Original: the number of edges in a complete graph).
Revised: the number of edges in a complete graph).

Original: In Figure 3 we plot the sparsity level as a function of k and d , as averaged over 25 random target models.
Revised: In Figure 3 we plot the sparsity level as a function of k and d , as averaged over 25 random target models.

Original: We see that the empirical sparsity level tracks the true sparsity level, and that for a fixed dimension d , both decrease with the number of effective classes k .
Revised: We see that the empirical sparsity level tracks the true sparsity level, and that for a fixed dimension d , both decrease with the number of effective classes k .

Original: This confirms that we expect the sparsity to “kick in” when k ≫ d .
Revised: This confirms that we expect the sparsity to “kick in” when k ≫ d .

Original: Comparisons of Active Learning Methods.
Revised: Comparisons of Active Learning Methods.

Original: We next compare the different baselines described above.
Revised: We next compare the different baselines described above.

Original: For
Revised: For

Original: NbrGraphSGD ( G ) we consider multiple variations, that use different versions of the graph G .
Revised: NbrGraphSGD ( G ) we consider multiple variations, that use different versions of the graph G .

Original: In Figure 4 we report the performance of NbrGraphSGD relative to several natural baselines.
Revised: In Figure 4 we report the performance of NbrGraphSGD relative to several natural baselines.

Original: First, it can be seen that the active tournament outperforms the passive one, suggesting that indeed some tournament queries can be avoided.
Revised: First, it can be seen that the active tournament outperforms the passive one, suggesting that indeed some tournament queries can be avoided.

Original: Yet NbrGraphSGD outperforms the tournament baselines, indicating that tourna4 Specifically, use Top-K accuracy, where K = 0 . 1 · k . [Figure] [Figure]
Revised: Yet NbrGraphSGD outperforms the tournament baselines, indicating that tournament comparisons are generally not the optimal approach.

Original: Sp a r

Original: s

Original: i t y d=5, G*_D d=5, G* d=7, G*_D d=7, G*

Original: [Figure]

Original: ment comparisons are generally not the optimal approach.

Original: Within the NbrGraphSGD methods, using the true graph (either G ∗ or G ∗ D ) provides the best performance, indicating that the graph plays an important role in active learning efficacy [Figure] and that NbrGraphSGD can use this structure.
Revised: Within the NbrGraphSGD methods, using the true graph (either G ∗ or G ∗ D ) provides the best performance, indicating that the graph plays an important role in active learning efficacy, and that NbrGraphSGD can use this structure.

Original: 7.2 REAL DATA
Revised: 7.2 REAL DATA

Original: The QuickDraw dataset [Ha and Eck, 2017], is a collection of 50 million drawings across 345 categories, contributed by players of the game “Quick, Draw!”.
Revised: The QuickDraw dataset [Ha and Eck, 2017], is a collection of 50 million drawings across 345 categories, contributed by players of the game “Quick, Draw!”.

Original: We use the bitmap version of the dataset, which contains these drawings converted from vector format (keystrokes) into 28x28 grayscale images.
Revised: We use the bitmap version of the dataset, which contains these drawings converted from vector format (keystrokes) into 28x28 grayscale images.

Original: We randomly select 70 , 000 examples from this large data and use 60 , 000 as our training set and the rest as the test set.
Revised: We randomly select 70 , 000 examples from this [Figure] .r.t G ⋆ D (green) against the baselines on the QuickDraw dataset.

Revised: In the plot titles, d denotes the dimension of lowdimensional projection of the data and edges is e ( G ⋆ ) , the number of edges in the true neighborhood graph of W ⋆ .

Revised: large data and use 60 , 000 as our training set and the rest as the test set.

Original: We train a linear teacher on the data after randomly projecting it into R d .
Revised: We train a linear teacher on the data after randomly projecting it into R d .

Original: We then use the resulting model W ⋆ to implement the label-comparison oracle (see
Revised: We then use the resulting model W ⋆ to implement the label-comparison oracle (see

Original: Definition 3.1).
Revised: Definition 3.1).

Original: We denote the true graph of W ⋆ (Definition 5.3) as G ⋆ and the empirical graph of W ⋆ (Definition 6.1, as computed w.r.t the training set) as G ⋆ D .
Revised: We denote the true graph of W ⋆ (Definition 5.3) as G ⋆ and the empirical graph of W ⋆ (Definition 6.1, as computed w.r.t the training set) as G ⋆ D .

Original: We begin by comparing the performance of NbrGraphSGD w.r.t G ⋆ D with the same baselines from Section 7.1.
Revised: We begin by comparing the performance of NbrGraphSGD w.r.t G ⋆ D with the same baselines from Section 7.1.

Original: We explore the relationship between the sparsity of the true graph G ⋆ and the performance of NbrGraphSGD w.r.t G ⋆ D as a function of the dimension d and k 
Revised: We explore the relationship between the sparsity of the true graph G ⋆ and the performance of NbrGraphSGD w.r.t G ⋆ D as a function of the dimension d and k = 345 .

Original: In  5 we report the query complexities left and d = right).
Revised: In Figure 5 we report the query complexities w.r.t d = 5 (left) and d = 15 (right).

Original: Note that this is a realizable learning task since the models are measured in terms of their accuracy w.r.t the teacher’s predictions, and the teacher is also a linear model.
Revised: Note that this is a realizable learning task since the models are measured in terms of their accuracy w.r.t the teacher’s predictions, and the teacher is also a linear model.

Original: In line with our theoretical results from Section 5, we observe that the gain from using our method (over e.g the passive or active tournament baselines) is smaller when the true neighborhood graph is less sparse.
Revised: In line with our theoretical results from Section 5, we observe that the gain from using our method (over e.g. the passive or active tournament baselines) is smaller when the true neighborhood graph is less sparse.

Original:  CONCLUSIONS
Revised:  CONCLUSIONS

Original: We studied the setting where annotators are asked to provide only pairwise label comparisons.
Revised: We studied the setting where annotators are asked to provide only pairwise label comparisons.

Original: We believe this is a natural setting as it is both easy for humans to provide, and still results in sufficient information for learning.
Revised: We believe this is a natural setting as it is both easy for humans to provide, and still results in sufficient information for learning.

Original: Our results provide several key characterizations of how this information should be gathered and used.
Revised: Our results provide several key characterizations of how this information should be gathered and used.

Original: We show that, perhaps counter-intuitively, there are cases for which having all the class comparisons per training point does not yield a sample complexity advantage over just receiving the one true class label.
Revised: We show that, perhaps counter-intuitively, there are cases for which having all the class comparisons per training point does not yield a sample complexity advantage over just receiving the one true class label.

Original: On the other hand, in the active setting, we show that comparisons can be used in an effective way that goes beyond obtaining the argmax training labels.
Revised: On the other hand, in the active setting, we show that comparisons can be used in an effective way that goes beyond obtaining the argmax training labels.

Original: Many interesting open questions remain.
Revised: Many interesting open questions remain.

Original: First, our focus was on linear classification, and it would be interesting to generalize the result to other classes (such as neural networks).
Revised: First, our focus was on linear classification, and it would be interesting to generalize the result to other classes (such as neural networks).

Original: Second, one can consider a mixture of comparisons and true-labels, since the latter may be easy to obtain in some instances, and hence query-complexity should count these cases differently.
Revised: Second, one can consider a mixture of comparisons and true-labels, since the latter may be easy to obtain in some instances, and hence query-complexity should count these cases differently.

Original: Finally, here we assumed that annotators can provide answers to all queries.
Revised: Finally, here we assumed that annotators can provide answers to all queries.

Original: In practice, some queries may not be answerable (e.g., labels are too “close” or both are equally bad), and it would be interesting to extend the formalism and practical algorithm to these cases.
Revised: In practice, some queries may not be answerable (e g., labels are too “close” or both are equally bad), and it would be interesting to extend the formalism and practical algorithm to these cases.

Original: A ACTIVE LEARNING COMPLEXITY FOR d >

Original: Our analysis in the main text suggested that when we can efficiently learn G ⋆ and it is sparse, label-comparisons provide a gain over argmax queries.

Original: We showed this when d = 1 , and it is natural to ask to what happens for d > 1 .

Original: This requires addressing both the question of what is the binary active learning primitive that we use, as well as the questions of sparsity and learning the graph.

Original: We discuss these two points below.

Original: Regarding binary active learning, for d > 1 , active learning is known to no longer provide asymptotic benefits over passive learning in the distribution-free setting [Dasgupta, 2004].

Original: Hence for d > 1 it’s natural to focus on distribution-specific learning.

Original: Namely, we will understand q b ( · ) to be the query complexity of active learning the binary class H 2 ,d lin under structural assumptions on the target distribution.

Original: For example, Balcan and Long [2013] prove that for log-concave distributions, q b ( γ ) =

Original: Θ( d log 1 γ ) . Regarding sparsity, we note that linear classifiers in d = 1 are maximally spase. Interestingly, linear classifiers in d = 2 also admit sparse graphs. By definition, the decision regions of h ( · ; W ) ∈ H k, 3 lin are convex polyhedrons.
Revised: Acknowledgements

Original: By Steinitz’s theorem, these are 3-connected planar graphs, and so a corollary of Euler’s Formula implies

Original: that e ( G ) ≤ 3 k − 6 .

Original: In Section 7.1 we experimentally evaluate the sparsity of random linear teachers as a function of k, d , observing that the graphs are becoming sparse as k ≫ d .

Original: This is related to a known result in computational geometry Dwyer [1991], that the expected number of edges in a Voronoi regions is linear, when points are sampled uniformly from the sphere (see also Aurenhammer and Klein [2000]).

Original: We leave open the question of whether the graph learning procedure of Algorithm 2 has an efficient analogue in d > 1 .

Original: B DEMONSTRATION OF G ∗ D MISSES

Original: In the main text, we discuss the empirical neighborhood graph G ∗ D , which is a subset of the true graph G ∗ .

Original: In Figure 6 below we show an example where G ∗ D will result in erroneous classification, because it will miss important boundaries points.

Original: C PROOF OF LEMMA 5.

Original: Fix D and W ⋆ .

Original: We will use an important observation: if f ⋆ is f ( G,C ) w.r.t G = G ( W ⋆ ) and C given by the true binary classifiers (i.e. h ij =

Original: W ⋆

Original: i − W ⋆ j = h ⋆ij ), then L D ( f ⋆ ) = 0 . I.e, for every x ∈ R d with argmax y = arg max i ∈ [ k ] W ⋆ i
Revised: We thank Ami Wiesel for contributing many ideas throughout the development of this work and for helpful feedback on this manuscript.

Original: x , the following holds: ∀ j :

Original: [Equation]

Original: Now, fix ( G, C ) satisfying the conditions in the theorem statement.

Original: We can upper bound L ( f ( G,C ) ) as follows:

Original: [Equation]

Original: As required.