Original: Improved Algorithms for Neural Active Learning
Revised: Improved Algorithms for Neural Active Learning

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress email
Revised: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: We improve the theoretical and empirical performance of neural-network(NN)-based active learning algorithms for the non-parametric streaming setting.
Revised: We improve the theoretical and empirical performance of neural-network(NN)-based active learning algorithms for the non-parametric streaming setting.

Original: Inparticular, we introduce two regret metrics by minimizing the population loss thatare more suitable in active learning than the one used in state-of-the-art (SOTA)related work.
Revised: Inparticular, we introduce two regret metrics by minimizing the population loss thatare more suitable in active learning than the one used in state-of-the-art (SOTA)related work.

Original: Then, the proposed algorithm leverages the powerful representationof NNs for both exploitation and exploration, has the query decision-maker tailoredfor k -class classiﬁcation problems with the performance guarantee, utilizes the fullfeedback, and updates parameters in a more practical and efﬁcient manner.
Revised: Then, the proposed algorithm leverages the powerful representationof NNs for both exploitation and exploration, has the query decision-maker tailoredfor k -class classiﬁcation problems with the performance guarantee, utilizes the fullfeedback, and updates parameters in a more practical and efﬁcient manner.

Original: Thesecareful designs lead to a better regret upper bound, improving by a multiplicativefactor O (log T ) and removing the curse of both input dimensionality and thecomplexity of the function to be learned.
Revised: Thesecareful designs lead to a better regret upper bound, improving by a multiplicativefactor O (log T ) and removing the curse of both input dimensionality and thecomplexity of the function to be learned.

Original: Furthermore, we show that the algorithmcan achieve the same performance as the Bayes-optimal classiﬁer in the longrun under the hard-margin setting in classiﬁcation problems.
Revised: Furthermore, we show that the algorithmcan achieve the same performance as the Bayes-optimal classiﬁer in the longrun under the hard-margin setting in classiﬁcation problems.

Original: In the end, we useextensive experiments to evaluate the proposed algorithm and SOTA baselines, toshow the improved empirical performance.
Revised: In the end, we useextensive experiments to evaluate the proposed algorithm and SOTA baselines, toshow the improved empirical performance.

Original:  Introduction
Revised:  Introduction

Original: The Neural Network (NN) is one of the indispensable paradigms in machine learning and is widelyused in multifarious supervised-learning tasks [22].
Revised: The Neural Network (NN) is one of the indispensable paradigms in machine learning and is widelyused in multifarious supervised-learning tasks [21].

Original: As more and more complicated NNs are developed, the requirement of the training procedure on the labeled data grows, incurring signiﬁcantcost of label annotation.
Revised: As more and more complicated NNs are developed, the requirement of the training procedure on the labeled data grows, incurring signiﬁcantcost of label annotation.

Original: Active learning investigates effective techniques on a much smaller labeleddata set while attaining the comparable generalization performance to passive learning [17].
Revised: Active learning investigates effective techniques on a much smaller labeleddata set while attaining the comparable generalization performance to passive learning [16].

Original: In thispaper, we focus on the k -class classiﬁcation problem in the streaming setting of active learning with
Revised: In thispaper, we focus on the k -class classiﬁcation problem in the streaming setting of active learning with

Original: NN models.
Revised: NN models.

Original: At every round, the learner receives an instance and is compelled to decide on-the-ﬂywhether or not to observe the label associated with this instance.
Revised: At every round, the learner receives an instance and is compelled to decide on-the-ﬂywhether or not to observe the label associated with this instance.

Original: This problem seeks to maximizethe generalization capability of learned NNs in a sequence of rounds, such that the model has robustperformance on the unseen data from the same distribution [38].
Revised: This problem seeks to maximizethe generalization capability of learned NNs in a sequence of rounds, such that the model has robustperformance on the unseen data from the same distribution [37].

Original: In active learning, given access to the i.i.d.
Revised: In active learning, given access to the i.i.d.

Original: generated instances from a distribution D , supposethere exist a class of functions F that formulate the mapping from instances to theirs labels.
Revised: generated instances from a distribution D , supposethere exist a class of functions F that formulate the mapping from instances to theirs labels.

Original: In theparametric setting, i.e., F has ﬁnite VC-dimension [24], existing works [23, 11, 7] have shown that theactive learning algorithms can achieve the convergence rate of e
Revised: In theparametric setting, i.e., F has ﬁnite VC-dimension [23], existing works [22, 11, 7] have shown that theactive learning algorithms can achieve the convergence rate of e

Original: O (1 / p N ) to the best population lossin F , where N is the number of label queries.
Revised: O (1 / p N ) to the best population lossin F , where N is the number of label queries.

Original: In the non-parametric setting, recent works [33, 34]provide the similar convergence results while suffering from the curse of input dimensionality.
Revised: In the non-parametric setting, recent works [32, 33]provide the similar convergence results while suffering from the curse of input dimensionality.

Original: Unfortunately, most of NN-based approaches to active learning do not come with the performanceguarantee, despite having powerful empirical results.
Revised: Unfortunately, most of NN-based approaches to active learning do not come with the performanceguarantee, despite having powerful empirical results.

Original: The ﬁrst performance guarantee for neural active learning has been established in a recent workby [45], and the analysis is for over-parameterized neural networks with the assistance of Neural
Revised: The ﬁrst performance guarantee for neural active learning has been established in a recent workby [44], and the analysis is for over-parameterized neural networks with the assistance of Neural

Original: Tangent Kernel (NTK).
Revised: Tangent Kernel (NTK).

Original: We carefully investigate the limitations of [45], which turn into the main motivations of our paper.
Revised: We carefully investigate the limitations of [44], which turn into the main motivations of our paper.

Original: First, [45] transforms the classiﬁcation problem into a multi-armed banditproblem [50], to minimize a pseudo regret metric.
Revised: First, [44] transforms the classiﬁcation problem into a multi-armed banditproblem [49], to minimize a pseudo regret metric.

Original: Yet, on the grounds that they seek to minimizethe conditional population loss on a sequence of given data, it is dubious that the pseudo regret usedin [45] can explicitly measure the generalization capability of given algorithms (see Remark 2.1).
Revised: Yet, on the grounds that they seek to minimizethe conditional population loss on a sequence of given data, it is dubious that the pseudo regret usedin [44] can explicitly measure the generalization capability of given algorithms (see Remark 2.1).

Original: Second, the proposed algorithm in [45] has an unknown prior complexity term S that reﬂects thecomplexity of function to be learned, brought by the NTK approximation (Lemma 1 in [45]).
Revised: Second, the proposed algorithm in [44] has an unknown prior complexity term S that reﬂects thecomplexity of function to be learned, brought by the NTK approximation (Lemma 1 in [44]).

Original: Toestimate the magnitude of S , the authors introduce another complex model selection algorithm andmake strong assumptions regarding the generated models.
Revised: Toestimate the magnitude of S , the authors introduce another complex model selection algorithm andmake strong assumptions regarding the generated models.

Original: Third, the training process for NN modelsis not efﬁcient, as [45] uses vanilla gradient descent and starts from randomly initialized parametersin every round.
Revised: Third, the training process for NN modelsis not efﬁcient, as [44] uses vanilla gradient descent and starts from randomly initialized parametersin every round.

Original: Fourth, although [45] removes the curse of input dimensionality d , the performanceguarantee strongly suffers from another introduced term, the effective dimensionality e d , which can bethought of as the non-linear dimensions of Hilbert space spanned by NTK.
Revised: Fourth, although [44] removes the curse of input dimensionality d , the performanceguarantee strongly suffers from another introduced term, the effective dimensionality e d , which can bethought of as the non-linear dimensions of Hilbert space spanned by NTK.

Original: In the worse case, themagnitude of e d can be an unacceptably large number and thus the performance guarantee collapses.
Revised: In the worse case, themagnitude of e d can be an unacceptably large number and thus the performance guarantee collapses.

Original: 1.1 Main contributions
Revised: 1.1 Main contributions

Original: In this paper, we propose a novel algorithm, I-NeurAL
Revised: In this paper, we propose a novel algorithm, I-NeurAL

Original: ( I mproved Algorithms for Neur al A ctive
Revised: ( I mproved Algorithms for Neur al A ctive

Original: L earning), to tackle the above limitations.
Revised: L earning), to tackle the above limitations.

Original: Our contributions can be summarized as follows: (1)
Revised: Our contributions can be summarized as follows: (1)

Original: We expand the binary classiﬁcation studied in [45] to the k -class classiﬁcation problem.
Revised: We expand the binary classiﬁcation studied in [44] to the k -class classiﬁcation problem.

Original: Further,we introduce two new regret metrics to minimize the population loss, which can directly reﬂect thegeneralization capability of NN-based algorithms.
Revised: Further,we introduce two new regret metrics to minimize the population loss, which can directly reﬂect thegeneralization capability of NN-based algorithms.

Original: (2) I-NeurAL has a neural exploration strategy witha novel component to decide whether or not to query the label, coming with the performance guarantee.
Revised: (2) I-NeurAL has a neural exploration strategy witha novel component to decide whether or not to query the label, coming with the performance guarantee.

Original: I-NeurAL does not have unknown complexity terms and the hyper-parameters are straightforwardand intuitive.
Revised: I-NeurAL does not have unknown complexity terms and the hyper-parameters are straightforwardand intuitive.

Original: (3) I-NeurAL is designed to support mini-batch Stochastic Gradient Descent (SGD).
Revised: (3) I-NeurAL is designed to support mini-batch Stochastic Gradient Descent (SGD).

Original: Inparticular, at every round, I-NeurAL does mini-batch SGD starting with the parameters of the lastround, i.e., with warm start, which is more efﬁcient and practical compared to [45].
Revised: Inparticular, at every round, I-NeurAL does mini-batch SGD starting with the parameters of the lastround, i.e., with warm start, which is more efﬁcient and practical compared to [44].

Original: (4) Without anynoise assumption on the data distribution, we provide the performance guarantee of I-NeurAL forover-parameterized neural networks.
Revised: (4) Without anynoise assumption on the data distribution, we provide the performance guarantee of I-NeurAL forover-parameterized neural networks.

Original: Compared to [45], ﬁrst, we remove the curse of both the inputdimensionality d and the effective dimensionality e d ; second, we improve the regret by a multiplicativefactor log( T ) , where T is the number of rounds; third, our regret bound does not have any unknowncomplexity terms such as S ; (5) under a hard-margin assumption on the data distribution, to the bestof our knowledge, we provide the ﬁrst performance analysis that NN models can achieve the samegeneralization capability as Bayes-optimal classiﬁer after a ﬁxed number of label queries; (6) weconduct extensive experiments on real-world data sets to demonstrate the improved performanceof I-NeurAL over state-of-the-art baselines including the closest work [45] which has not providedempirical validation of their proposed algorithms.
Revised: Compared to [44], ﬁrst, we remove the curse of both the inputdimensionality d and the effective dimensionality e d ; second, we improve the regret by a multiplicativefactor log( T ) , where T is the number of rounds; third, our regret bound does not have any unknowncomplexity terms such as S ; (5) under a hard-margin assumption on the data distribution, to the bestof our knowledge, we provide the ﬁrst performance analysis that NN models can achieve the samegeneralization capability as Bayes-optimal classiﬁer after a ﬁxed number of label queries; (6) weconduct extensive experiments on real-world data sets to demonstrate the improved performanceof I-NeurAL over state-of-the-art baselines including the closest work [44] which has not providedempirical validation of their proposed algorithms.

Original: 1.2 Related Work
Revised: 1.2 Related Work

Original: Active learning has been extensively studied and applied to many essential applications [42].
Revised: Active learning has been extensively studied and applied to many essential applications [41].

Original: Bayesianactive learning methods typically use a probabilistic regression model to estimate the improvement ofeach query [28, 39].
Revised: Bayesianactive learning methods typically use a probabilistic regression model to estimate the improvement ofeach query [27, 38].

Original: In spite of effectiveness on the small or moderate data sets, the Bayesian-basedapproaches are difﬁcult to scale to large-scale data sets because of the batch sampling [41].
Revised: In spite of effectiveness on the small or moderate data sets, the Bayesian-basedapproaches are difﬁcult to scale to large-scale data sets because of the batch sampling [40].

Original: Anotherimportant class, margin algorithms or uncertainty sampling [32], obtains considerate performanceimprovement over passive learning and is further developed by many practitioners [18, 27, 36, 12].
Revised: Anotherimportant class, margin algorithms or uncertainty sampling [31], obtains considerate performanceimprovement over passive learning and is further developed by many practitioners [17, 26, 35, 12].

Original: Margin algorithms are ﬂexible and can be adapted to both streaming and pool settings.
Revised: Margin algorithms are ﬂexible and can be adapted to both streaming and pool settings.

Original: In thepool setting, a line of works utilize the neural networks in active learning to improve the empiricalperformance [35, 40, 5, 15, 29, 43].
Revised: In thepool setting, a line of works utilize the neural networks in active learning to improve the empiricalperformance [34, 39, 5, 14, 28, 42].

Original: However, they do not provide performance guarantee for NNbased active learning algorithms.
Revised: However, they do not provide performance guarantee for NNbased active learning algorithms.

Original: From the theoretical perspective, [48, 19, 6, 8, 49] provide theperformance guarantee with the speciﬁc classes of functions and [25, 20] present the theoreticalanalysis of active learning algorithms with the surrogate loss functions for binary classiﬁcation.
Revised: From the theoretical perspective, [47, 18, 6, 8, 48] provide theperformance guarantee with the speciﬁc classes of functions and [24, 19] present the theoreticalanalysis of active learning algorithms with the surrogate loss functions for binary classiﬁcation.

Original: However, their performance guarantee is restricted within hypothesis classes, i.e, the parametricsetting.
Revised: However, their performance guarantee is restricted within hypothesis classes, i.e, the parametricsetting.

Original: In contrast, our goal is to derive an NN-based algorithm in the non-parametric setting thatperforms well both empirically and theoretically.
Revised: In contrast, our goal is to derive an NN-based algorithm in the non-parametric setting thatperforms well both empirically and theoretically.

Original: To the best of our knowledge, [45] is the only neuralactive learning algorithm with such performance guarantee, of which limitations are discussed above.
Revised: To the best of our knowledge, [44] is the only neuralactive learning algorithm with such performance guarantee, of which limitations are discussed above.

Original: As [45] is the closest related work to our paper, we emphasize the differences of our techniques from[45] throughout the paper.
Revised: As [44] is the closest related work to our paper, we emphasize the differences of our techniques from[44] throughout the paper.

Original: We introduce the problem deﬁnition and proposed algorithms in Section
Revised: We introduce the problem deﬁnition and proposed algorithms in Section

Original: 2 and Section 3 respectively.
Revised: 2 and Section 3 respectively.

Original: Then, we provide performance guarantees in Section 4 and empiricalresults in Section 5, ending with the conclusion in Section 6.
Revised: Then, we provide performance guarantees in Section 4 and empiricalresults in Section 5, ending with the conclusion in Section 6.

Original:  Problem Deﬁnition
Revised:  Problem Deﬁnition

Original: In this paper, we study the streaming setting of active learning in the k -class classiﬁcation problem.
Revised: In this paper, we study the streaming setting of active learning in the k -class classiﬁcation problem.

Original: Let X denote the input space over R d , Y = { 1 , 2 , . .
Revised: Let X denote the input space over R d , Y = { 1 , 2 , . .

Original: . , k } represent the label space, and D be someunknown distribution over X ⇥ Y .
Revised: . , k } represent the label space, and D be someunknown distribution over X ⇥ Y .

Original: At round t 2 [ T ] = { 1 , 2 , . . .
Revised: At round t 2 [ T ] = { 1 , 2 , . . .

Original: , T } , an instance x t is drawn from the 96 marginal distribution
Revised: , T } , an instance x t is drawn from the 96 marginal distribution

Original: D X
Revised: D X

Original: and accordingly y t is drawn from the conditional distribution D Y| x t .
Revised: and accordingly y t is drawn from the conditional distribution D Y| x t .

Original: Here, 97 y t can be thought of as the index of the class that x t belongs to.
Revised: Here, 97 y t can be thought of as the index of the class that x t belongs to.

Original: Inspired by [45], we ﬁrst transformx t into k context vectors representing the k classes respectively: x t, 1 = ( x > t , 0 > , . . . , 0 > ) > , x t, 2 =( 0 > , x > t , . . . , 0 > ) > , . . .
Revised: Inspired by [44], we ﬁrst transformx t into k context vectors representing the k classes respectively: x t, 1 = ( x > t , 0 > , . . . , 0 > ) > , x t, 2 =( 0 > , x > t , . . . , 0 > ) > , . . .

Original: , x t,k
Revised: , x t,k

Original: = ( 0 > , 0 > , . . .
Revised: = ( 0 > , 0 > , . . .

Original: , x > t ) > and x t,i 2 R dk , 8 i 2 [ k ] .
Revised: , x > t ) > and x t,i 2 R dk , 8 i 2 [ k ] .

Original: In accordance withcontext vectors, we construct the k label vectors representing the k possible prediction: y t, 1 =(1 , 0 , . . . , 0) > , y t, 2 = (0 , 1 , . . . , 0) > , . . .
Revised: In accordance withcontext vectors, we construct the k label vectors representing the k possible prediction: y t, 1 =(1 , 0 , . . . , 0) > , y t, 2 = (0 , 1 , . . . , 0) > , . . .

Original: , y t,k
Revised: , y t,k

Original: = (0 , 0 , . . . , 1) > and y t,
Revised: = (0 , 0 , . . . , 1) > and y t,

Original: i 2 R k , 8 i 2 [ k ] .
Revised: i 2 R k , 8 i 2 [ k ] .

Original: Thus, y t,y tis the ground-truth label vector for x t .
Revised: Thus, y t,y tis the ground-truth label vector for x t .

Original: Under the non-parametric setting of active learning, we deﬁne an unknown function h to formulatethe conditional distribution D Y| x t : X k !
Revised: Under the non-parametric setting of active learning, we deﬁne an unknown function h to formulatethe conditional distribution D Y| x t : X k !

Original: [0 , 1] , such that
Revised: [0 , 1] , such that

Original: [Equation]
Revised: [Equation]

Original: which is subject to
Revised: which is subject to

Original: [Equation]
Revised: [Equation]

Original: i )
Revised: i )

Original: = 1 .
Revised: = 1 .

Original: For simplicity, we consider the k -class classiﬁcation problemwith 0-1 loss.
Revised: For simplicity, we consider the k -class classiﬁcation problemwith 0-1 loss.

Original: Given x t , i.e., x t,i , i 2 [ k ] , let b i be the index of the class predicted by some hypothesisf and thus y t, b
Revised: Given x t , i.e., x t,i , i 2 [ k ] , let b i be the index of the class predicted by some hypothesisf and thus y t, b

Original: i is the prediction.
Revised: i is the prediction.

Original: Then, we have the following loss:
Revised: Then, we have the following loss:

Original: [Equation]
Revised: [Equation]

Original: where is the indicator function.
Revised: where is the indicator function.

Original: Given the number of rounds T , at each round t 2 [ T ] , the learner receives an instance x t drawn i.i.d.from D X .
Revised: Given the number of rounds T , at each round t 2 [ T ] , the learner receives an instance x t drawn i.i.d.from D X .

Original: Then, the learner needs to make a prediction y t, b i , and at the same time, decide on-the-ﬂywhether or not to query the label y t,y t where y t is drawn i.i.d.
Revised: Then, the learner needs to make a prediction y t, b i , and at the same time, decide on-the-ﬂywhether or not to query the label y t,y t where y t is drawn i.i.d.

Original: from D Y| x t .
Revised: from D Y| x t .

Original: As the goal of active 112 learning tasks is often to minimize the population loss [38], we introduce the following two regretmetrics.
Revised: As the goal of active 112 learning tasks is often to minimize the population loss [37], we introduce the following two regretmetrics.

Original: Deﬁnition 2.1 (Latest Population Regret) .
Revised: Deﬁnition 2.1 (Latest Population Regret) .

Original: Given the data distribution D , the number of rounds T ,the Latest Population Regret is deﬁned as
Revised: Given the data distribution D , the number of rounds T ,the Latest Population Regret is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where y T,i ⇤ is the prediction the Bayes-optimal classiﬁer would make on instance x T ,
Revised: where y T,i ⇤ is the prediction the Bayes-optimal classiﬁer would make on instance x T ,

Original: i.e., i ⇤ =
Revised: i.e., i ⇤ =

Original: arg max i 

Revised: arg max i

Original: [ k ] h ( x T,i ) for y T,i ⇤ .
Revised: [ k ] h ( x T,i ) for y T,i ⇤ .

Original: Deﬁnition 2.2 (Cumulative Population Regret) .
Revised: Deﬁnition 2.2 (Cumulative Population Regret) .

Original: Given the data distribution D , the number of rounds
Revised: Given the data distribution D , the number of rounds

Original: T , the Cumulative Population Regret is deﬁned as:
Revised: T , the Cumulative Population Regret is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where y t,i ⇤ is the prediction the Bayes-optimal classiﬁer would make on instance x t , i.e., i ⇤ =
Revised: where y t,i ⇤ is the prediction the Bayes-optimal classiﬁer would make on instance x t , i.e., i ⇤ =

Original: arg max i 

Revised: arg max i

Original: [ k ] h ( x t,i ) for y t,i ⇤ .
Revised: [ k ] h ( x t,i ) for y t,i ⇤ .

Original: R T measures the performance at the last round T only, and R T measures the overall performance in
Revised: R T measures the performance at the last round T only, and R T measures the overall performance in

Original: T rounds combined.
Revised: T rounds combined.

Original: Therefore, the goal of this problem is to minimize R T or R T , or both.
Revised: Therefore, the goal of this problem is to minimize R T or R T , or both.

Original: At thesame time, we also aim to minimize the following expected query cost:
Revised: At thesame time, we also aim to minimize the following expected query cost:

Original: [Equation]
Revised: [Equation]

Original: where I t is the indicator of the query decision in round t such that I t
Revised: where I t is the indicator of the query decision in round t such that I t

Original: = 1 if y t is observed; I t = 0 ,otherwise.
Revised: = 1 if y t is observed; I t = 0 ,otherwise.

Original: Remark 2.1.
Revised: Remark 2.1.

Original: Minimizing R T or R T shows the generalization capability of the learned hypothesis onthe distribution D .
Revised: Minimizing R T or R T shows the generalization capability of the learned hypothesis onthe distribution D .

Original: However, the problem deﬁned in [45] is to minimize the cumulative conditionalpopulation regret as follows:
Revised: However, the problem deﬁned in [44] is to minimize the cumulative conditionalpopulation regret as follows:

Original: [Equation]
Revised: [Equation]

Original: As E y t ⇠ D Y| x t [ L ( y t, b i , y t,y t ) | x t ] is the population loss conditioned on x t , unfortunately, e
Revised: As E y t ⇠ D Y| x t [ L ( y t, b i , y t,y t ) | x t ] is the population loss conditioned on x t , unfortunately, e

Original: R T onlymeasures the performance of the learned hypothesis on the collected data { x t } Tt =1 , and e
Revised: R T onlymeasures the performance of the learned hypothesis on the collected data { x t } Tt =1 , and e

Original: R T cannotdirectly measure the accuracy of the hypothesis on unseen data instances.
Revised: R T cannotdirectly measure the accuracy of the hypothesis on unseen data instances.

Original: Although e
Revised: Although e

Original: R T follows theregret deﬁnition in multi-armed bandits [50], it is fair to say that e
Revised: R T follows theregret deﬁnition in multi-armed bandits [49], it is fair to say that e

Original: R T may not be a good metric inactive learning.
Revised: R T may not be a good metric inactive learning.

Original:  Proposed Algorithms
Revised:  Proposed Algorithms

Original: In this section, we elaborate on the proposed algorithm I-NeurAL (Algorithm 1).
Revised: In this section, we elaborate on the proposed algorithm I-NeurAL (Algorithm 1).

Original: In contrast to thedirectly comparable work [45], I-NeurAL has the following novel and advantageous aspects: (1)
Revised: In contrast to thedirectly comparable work [44], I-NeurAL has the following novel and advantageous aspects: (1)

Original: I-NeurAL incorporates a neural-based exploration strategy (Line 6) inspired by recent advances inbandits [10] to solve the exploitation-exploration dilemma in the decision for whether or not to querylabels; (2) I-NeurAL includes a novel component (Line 11) to decide whether or not to query labelsin the k -class classiﬁcation problem, which does not have the unknown complexity term S as in [45],thus avoiding the complicated model selection procedure (Algorithm 2 in [45]); (3) I-NeurAL infersand exploits the feedback of all the contexts (Lines 12-17), instead of only utilizing the feedback ofthe chosen context in [45]; (4) I-NeurAL conducts mini-batch SGD based on the parameters of thelast round (Algorithm 2), which is more practical, as opposed to conducting vanilla gradient descentfrom the initialization at every round in [45].
Revised: I-NeurAL incorporates a neural-based exploration strategy (Line 6) inspired by recent advances inbandits [10] to solve the exploitation-exploration dilemma in the decision for whether or not to querylabels; (2) I-NeurAL includes a novel component (Line 11) to decide whether or not to query labelsin the k -class classiﬁcation problem, which does not have the unknown complexity term S as in [44],thus avoiding the complicated model selection procedure (Algorithm 2 in [44]); (3) I-NeurAL infersand exploits the feedback of all the contexts (Lines 12-17), instead of only utilizing the feedback ofthe chosen context in [44]; (4) I-NeurAL conducts mini-batch SGD based on the parameters of thelast round (Algorithm 2), which is more practical, as opposed to conducting vanilla gradient descentfrom the initialization at every round in [44].

Original: Next, we will present the details of I-NeurAL.
Revised: Next, we will present the details of I-NeurAL.

Original: Exploitation Network f 1 .
Revised: Exploitation Network f 1 .

Original: Given x t,i , i 2 [ k ] , to learn the unknown function h (Eq.
Revised: Given x t,i , i 2 [ k ] , to learn the unknown function h (Eq.

Original: (2.1)), we use afully-connected neural network f 1 with L -depth and m -width:
Revised: (2.1)), we use afully-connected neural network f 1 with L -depth and m -width:

Original: [Equation]
Revised: [Equation]

Original: where W 
Revised: where W 11 2 R

Original: [Equation]
Revised: [Equation]

Original: kd , W
Revised: kd , W

Original: [Equation]
Revised: [Equation]

Original: 2 R m ⇥ m , for 

Revised: 2 R m ⇥ m , for

Original:  l  L  1 , W 1 L 2 R 1 ⇥ m , ✓ 1 =[ vec ( W 11 ) > , . .
Revised:  l  L  1 , W 1 L 2 R 1 ⇥ m , ✓ 1 =[ vec ( W 11 ) > , . .

Original: , vec ( W 1 L ) > ] > 2 R p 1 , and  is the ReLU activation function  ( x ) = max { 0 , x } .
Revised: , vec ( W 1 L ) > ] > 2 R p 1 , and  is the ReLU activation function  ( x ) = max { 0 , x } .

Original: In round t , given x t,i , i 2 [ k ] , f 1 ( x t,i ; ✓ 1 t  1 ) is assigned to learn h ( x t,i ) .
Revised: In round t , given x t,i , i 2 [ k ] , f 1 ( x t,i ; ✓ 1 t  1 ) is assigned to learn h ( x t,i ) .

Original: Based on the fact 152 h ( x t,i ) =
Revised: Based on the fact 152 h ( x t,i ) =

Original: E y t ⇠ D Y| x t [1  L ( y t,i , y t,y t )] , it is natural to regard 1  L ( y t,i , y t,y t ) as the label fortraining f 1 .
Revised: E y t ⇠ D Y| x t [1  L ( y t,i , y t,y t )] , it is natural to regard 1  L ( y t,i , y t,y t ) as the label fortraining f 1 .

Original: Note that we take the basic fully-connected network as an example for the sake ofanalysis in over-parameterized networks and f 1 can be easily replaced with more complicated modelsdepending on the tasks.
Revised: Note that we take the basic fully-connected network as an example for the sake ofanalysis in over-parameterized networks and f 1 can be easily replaced with more complicated modelsdepending on the tasks.

Original: Exploration Network f 2 .
Revised: Exploration Network f 2 .

Original: In addition to the network f 1 , we assign another network f 2 to exploreuncertain information contained in incoming instances.
Revised: In addition to the network f 1 , we assign another network f 2 to exploreuncertain information contained in incoming instances.

Original: First, we carefully design the input of f 2 toincorporate the context vectors of the instance and the discrimination-ability of f 1 , to learn the errorbetween the Bayes-optimal probability h ( x t,i ) and the prediction f 1 ( x t,i ; ✓ 1 ) .
Revised: First, we carefully design the input of f 2 toincorporate the context vectors of the instance and the discrimination-ability of f 1 , to learn the errorbetween the Bayes-optimal probability h ( x t,i ) and the prediction f 1 ( x t,i ; ✓ 1 ) .

Original: Deﬁnition 3.1 (Derivative-Context (DC) Embedding) .
Revised: Deﬁnition 3.1 (Derivative-Context (DC) Embedding) .

Original: Given the exploitation network f 1 ( · ; ✓ 1 t  1 )and an input context x t,i , its DC embeding is deﬁned as
Revised: Given the exploitation network f 1 ( · ; ✓ 1 t  1 )and an input context x t,i , its DC embeding is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where O x t,i f 1 is the partial derivative of f 1 ( x t,i ; ✓ 1 t  1 ) with respect to x t,i  .
Revised: where O x t,i f 1 is the partial derivative of f 1 ( x t,i ; ✓ 1 t  1 ) with respect to x t,i . ( x t,i ) is normalized so that k  ( x t,i ) k 2 = 1 .

Original: Note that  ( x t,i  )  is normalized so that k  ( x t,i ) k 2 = 1 .
Revised: Note that the input for f 2 in [10] is the gradientwith respect to ✓ 1 , denoted by O ✓ 1 f 1 ( x t,i ; ✓ 1 t  1 ) 2 R p 1 . Its dimensionality is much larger than O x t,i f 1 ( x t,i ; ✓ 1 t  1 ) in Deﬁnition 3.1, may causing signiﬁcant computation cost.

Original: Given the input  ( x t,i ) , similarly, we choosethe fully-connected network to build f 2 :
Revised: Given the input  ( x t,i ) , similarly, we choose the fully-connected network to build f 2 :

Original: [Equation]
Revised: [Equation]

Original: where W 21 2 R m ⇥ 2 kd , W 2 l 2 R m ⇥ m , for 2  l  L  1 , W 2 L = R 1 ⇥ m and ✓ 2 =[ vec ( W 21 ) > , . . , vec ( W 2 L ) > ] > 2 R p 2 . In round t , given x t,i , 8 i 2 [ k ] , f 2 is to predict h ( x t,i ) 
Revised: [Equation]

Original: Input: T (number of rounds) f 1 , f 2 (neural networks), ⌘ 1 , ⌘ 2 (learning rate),  (exploration parameter), b (batch size),  (conﬁdence level) 1: Initialize ✓ 10 , ✓ 20 ; b ✓ 1 0 = ✓ 10 ; b ✓ 2 0 =
Revised: Input: T (number of rounds) f 1 , f 2 (neural networks), ⌘ 1 , ⌘ 2 (learning rate),  (exploration parameter), b (batch size),  (conﬁdence level) 1: Initialize ✓ 10 , ✓ 20 ; b ✓ 1 0 = ✓ 10 ; b ✓ 2 0 =

Original: ✓2: H 10 = ; ; H 20 = ; 3: for t = 1 , 2 , . .
Revised: ✓2: H 10 = ; ; H 20 = ; 3: for t = 1 , 2 , . .

Original: , T do 4:
Revised: , T do 4:

Original: Observe instance x t 2 R d and build x t,i , 8 i 2 [ k ] 5: for each i 2 [ k ] do 6: f ( x t,i ; ✓ t  1 ) =
Revised: Observe instance x t 2 R d and build x t,i , 8 i 2 [ k ] 5: for each i 2 [ k ] do 6: f ( x t,i ; ✓ t  1 ) =

Original: ⇣ f 1 ( x t,i ; ✓ 1 t  1 ) Exploitation Score + f 

Revised: ⇣ f 1 ( x t,i ; ✓ 1 t  1 ) Exploitation Score + f

Original: (  ( x t,i ); ✓ 2 t  1 ) Exploration Score ⌘ 7: end for 8: b i = arg max i 2 [ k ] f ( x t,i ; ✓ t  1 ) 9: i  = arg max i 2 ([ k ] \{ b i } ) f
Revised: (  ( x t,i ); ✓ 2 t  1 ) Exploration Score ⌘ 7: end for 8: b i = arg max i 2 [ k ] f ( x t,i ; ✓ t  1 ) 9: i  = arg max i 2 ([ k ] \{ b i } ) f

Original: ( x t,i ; ✓ t  1 ) 10:
Revised: ( x t,i ; ✓ t  1 ) 10:

Original: Predict y t, b i 11:
Revised: Predict y t, b i 11:

Original: I t = {| f ( x t, b i ; ✓ t  1 )  f ( x t,i  ; ✓ t  1 ) | < 2   t } 2 { 0 , 1 } ;  t = q 2 t + ⇣ c 1 3 L p 2 t ⌘ + q 2log
Revised: I t = {| f ( x t, b i ; ✓ t  1 )  f ( x t,i  ; ✓ t  1 ) | < 2   t } 2 { 0 , 1 } ;  t = q 2 t + ⇣ c 1 3 L p 2 t ⌘ + q 2log

Original: c 2 tk ) /  ) t 12: if I t = 1 then 13:
Revised: c 2 tk ) /  ) t 12: if I t = 1 then 13:

Original: Query x t and observe y t 14: for i 2 [ k ] do 15: r 1 t,i =
Revised: Query x t and observe y t 14: for i 2 [ k ] do 15: r 1 t,i =

Original: 1  L ( y t,i , y t,y t ) (deﬁned in E.q. (2.2)) 16: r 2 t,i =
Revised: 1  L ( y t,i , y t,y t ) (deﬁned in E.q. (2.2)) 16: r 2 t,i =

Original: r 1 t,i  f 1 ( x t,i ; ✓ 1 t  1 ) 17: end for 18: else 19: for i 2 [ k ] do 20: r 1 t,i = 1  L ( y t,i , y t, b i ) 21: r 2 t,i =
Revised: r 1 t,i  f 1 ( x t,i ; ✓ 1 t  1 ) 17: end for 18: else 19: for i 2 [ k ] do 20: r 1 t,i = 1  L ( y t,i , y t, b i ) 21: r [Equation]

Original: r 1 t,i  f 1 ( x t,i ; ✓ 1 t  1 ) 22: end for 23: end if 24:
Revised: [Equation] = [Equation] [Equation]

Original: H
Revised:

Original: [Equation]
Revised: [Equation]

Original: t =
Revised: ( x

Original: H
Revised: [Equation] [Equation] ; ✓

Original: [Equation]
Revised: [Equation]

Original: t  1[ { ( x t,i , r 1 t,i ) , i 2 [ k ] } 25:
Revised: t [Equation] 1 ) 22: end for 23: end if 24:

Original: H
Revised: H

Original: [Equation]
Revised: [Equation]

Original: t =
Revised: H

Original: H 2 t  1 [ { ( x t,i , r 2 t,i ) , i 2 [ k ] } 26: ✓ 1 t , ✓ 2 t =
Revised: [Equation] [ k ] } 26: ✓ 1 t , ✓ 2 t =

Original: Mini-Batch-SGD-Warm-Start ( f 1 , f 2 , H 1 t , H 2 t  ) 27: end for f{+where W 21 2 R m ⇥ 2 kd , W 2 l 2 R m ⇥ m , for 
  l  L  1 , W 2 L = R+} 1  ( x t,i; ✓ 1 t  1 ) for exploration.
Revised: Mini-Batch-SGD-Warm-Start ( f 1 , f 2 , H 1 t , H 2 t , b ) 27: end for {+where W 21 2 R m ⇥ 2 kd , W 2 l 2 R m ⇥ m , for 
  l  L  1 , W 2 L = R+} 1 ⇥ m and ✓ 2 =[ vec ( W 21 ) > , . . , vec ( W 2 L ) > ] > 2 R p 2 . In round t , given x t,i , 8 i 2 [ k ] , f 2 is to predict h ( x t,i ) f

Revised: 1 ( x t,i ; ✓ 1 t  1 ) for exploration.

Original: Because h ( x t,i )  f 1 ( x t,i ; ✓ 1 t  1 ) =
Revised: Because h ( x t,i )  f 1 ( x t,i ; ✓ 1 t  1 ) =

Original: E y t ⇠ D Y| x t [1  L ( y t,i , y t,y t )
Revised: E y t ⇠ D Y| x t [1  L ( y t,i , y t,y t )

Original: f 1 ( x t,i ; ✓ 1 t  1 )] , we regard 1  L ( y t,i , y t,y t )  f 1 ( x t,i ; ✓ 1 t  1 ) as the label for training f 2 .
Revised: f 1 ( x t,i ; ✓ 1 t  1 )] , we regard 1  L ( y t,i , y t,y t )  f 1 ( x t,i ; ✓ 1 t  1 ) as the label for training f 2 .

Original: To sum up, in round t , given x t,i , 8 i 2 [ k ] , the prediction b i ( y t, b i ) is made based on the sum ofexploitation and exploration scores, i.e., f 1 ( x t,i ; ✓ 1 t  1 ) + f 2 (  ( x t,i ); ✓ 2 t  1 ) (Lines 5-10).
Revised: To sum up, in round t , given x t,i , 8 i 2 [ k ] , the prediction b i ( y t, b i ) is made based on the sum ofexploitation and exploration scores, i.e., f 1 ( x t,i ; ✓ 1 t  1 ) + f 2 (  ( x t,i ); ✓ 2 t  1 ) (Lines 5-10).

Original: Query Decision-maker (Line 11) .
Revised: Query Decision-maker (Line 11) .

Original: A label query is made when I-NeurAL is not conﬁdent enoughto discriminate the Bayes-optimal class from other classes.
Revised: A label query is made when I-NeurAL is not conﬁdent enough todiscriminate the Bayes-optimal class from other classes.

Original: 2   t (  t is also deﬁned in Lemma10.3) can be thought of as a conﬁdence interval for the distance between the optimal class andsecond optimal class, where  is the hyper-parameter to tune the sensitivity of the decision-maker inpractice.
Revised: 2   t (  t is also deﬁned in Lemma 7.3)can be thought of as a conﬁdence interval for the distance between the optimal class and secondoptimal class, where  is the hyper-parameter to tune the sensitivity of the decision-maker in practice.

Original: Given any   1 ,  2 (0 , 1) , with probability at least 1   , based on our analysis (Lemma10.5), E ( x t ,y t )
Revised: Given any   1 ,  2 (0 , 1) , with probability at least 1   , based on our analysis (Lemma 7.5), E ( x t ,y t )

Original: ⇠ D [ L ( y t, b i , y t,y t )] =
Revised: ⇠ D [ L ( y t, b i , y t,y t )] =

Original: E ( x t ,y t )
Revised: E ( x t ,y t )

Original: ⇠ D [ L ( y t,i ⇤ , y t,y t )] when I t = 0 , i.e., I-NeurAL suffersno regret.
Revised: ⇠ D [ L ( y t,i ⇤ , y t,y t )] when I t = 0 , i.e., I-NeurAL suffers noregret.

Original: Thus, we use y t, b i as the pseudo-label in this case and we have the following update rules.
Revised: Thus, we use y t, b i as the pseudo-label in this case and we have the following update rules.

Original: Utilize Full Feedback (Lines 14-25) .
Revised: Utilize Full Feedback (Lines 14-25) .

Original: Different from the bandit setting where the learner can onlyobserve the reward of the selected context, we can infer the rewards of all contexts in active learning, aswe know the speciﬁc class of the current instance.
Revised: Different from the bandit setting where the learner can onlyobserve the reward of the selected context, we can infer the rewards of all contexts in active learning, aswe know the speciﬁc class of the current instance.

Original: Thus, for each x t,i , i 2 [ k ] , r 1 t,i = 1  L ( y t,i , y t,y t )is regarded as the "reward" of x t,i , predicted by f 1 , and r 2 t,i =
Revised: Thus, for each x t,i , i 2 [ k ] , r 1 t,i = 1  L ( y t,i , y t,y t )is regarded as the "reward" of x t,i , predicted by f 1 , and r 2 t,i =

Original: r 1 t,i  f 1 ( x t,i ; ✓ 1 ) is regarded as the"residual reward" of x t,i , predicted by f 2 . In summary, in round t , when I t = 1 , y t,y t is observed to
Revised: r 1 t,i  f 1 ( x t,i ; ✓ 1 ) is regarded as the

Original: 1: Deﬁne L 1 [( x , r 1 ); ✓ 1 ] =
Revised: 1: Deﬁne L 1 [( x , r 1 ); ✓ 1 ] =

Original: ( r 1  f 1 ( x ; ✓ 1 )) 2 /2:
Revised: ( r 1  f 1 ( x ; ✓ 1 )) 2 /2:

Original: Uniformly draw a set b
Revised: Uniformly draw a set b

Original: H
Revised: H

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: , s.t., | b
Revised: , s.t., | b

Revised: H

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: b 3: b ✓
Revised: b 3: b ✓

Original: [Equation]
Revised: [Equation]

Original:  =
Revised: t =

Original: b
Revised: b

Original: [Equation]
Revised: [Equation]

Original: b
Revised: b

Original: P
Revised: P

Original: [Equation]
Revised: [Equation]

Original: O
Revised: O

Original: [Equation]
Revised: [Equation]

Original:  , r
Revised: [( x , r

Original: [Equation]
Revised: [Equation]

Original: ); b ✓
Revised: ); b ✓

Original: [Equation]
Revised: [Equation]

Original:  ] 4: Deﬁne L 2 [(  ( x ) , r 2 ); ✓ 2 ] =
Revised:  [Equation] ] 4: Deﬁne L 2 [(  ( x ) , r 2 ); ✓ 2 ] =

Original: ( r 2  f 2 (  ( x ); ✓ 2 ))
Revised: ( r 2  f 2 (  ( x ); ✓ 2 ))

Original: Uniformly draw a set b
Revised: Uniformly draw a set b

Original: H
Revised: H

Original: [Equation]
Revised: [Equation]

Original: | b
Revised: b

Original: H
Revised: H

Original: [Equation]
Revised: [Equation]

Original: b 6: b ✓
Revised: b 6: b ✓

Original: [Equation]
Revised: [Equation]

Original: = b ✓
Revised: = b ✓

Original: [Equation]
Revised: [Equation]

Original: t
Revised:

Original: [Equation]
Revised: [Equation]

Original:  
Revised:  ⌘

Original: [Equation]
Revised: [Equation]

Original: b
Revised: b

Original: P (  ( x ) ,r 1 )b
Revised: P (  ( x ) ,r 1 )b

Original: H 2 t
Revised: H 2 t

Original: O ✓ 2 L 2 [(  ( x ) , r 2 ); b ✓ 2 t  1 ] 7: ⌦ t =
Revised: O ✓ 2 L 2 [(  ( x ) , r 2 ); b ✓ 2 t  1 ] 7: ⌦ t =

Original: ⌦ t  1 [ { ( b ✓ 1 t , b ✓ 2 t ) } 8: Return ( ✓ 1 t , ✓ 2 t ) uniformly from⌦ t update r 1 t,i and r 2 t,i  ; when I t = 0 , y t, bi is regard as the pseudo-label to obtain r 1 t,i and r 2 t,i , 8 i 2 [ k ] .
Revised: ⌦ t  1 [ { ( b ✓ 1 t , b ✓ 2 t ) } 8: Return ( ✓ 1 t , ✓ 2 t ) uniformly from t "residual reward" of x t,i , predicted by In summary, in round , when = is t,i and t i ; when I t = 0 , y t, b

Revised: i is regard as the pseudo-label to obtain r 1 t,i

Revised: 2 t,i , 8 i 2 [

Original: Therefore, we have the training data H 1 t for f 1 and H 2 t for f 2 .
Revised: Therefore, we have the training data H 1 t for f 1 and H 2 t for f 2 .

Original: Mini-Batch SGD with Warm-Start (Algorithm 3) .
Revised: Mini-Batch SGD with Warm-Start (Algorithm 3) .

Original: Unlike [45] that uses vanilla gradient descent fromrandomly initialized parameters in each round, causing unnecessarily expensive computation, weextend the training procedure to mini-batch SGD with warm start, i.e., we incrementally train theparameters ✓ t starting from the parameters of the last round ✓ t  1 in each round t .
Revised: Unlike [44] that uses vanilla gradient descent fromrandomly initialized parameters in each round, causing unnecessarily expensive computation, weextend the training procedure to mini-batch SGD with warm start, i.e., we incrementally train theparameters ✓ t starting from the parameters of the last round ✓ t  1 in each round t .

Original: Algorithm 1 depicts the workﬂow of I-NeurAL.
Revised: Algorithm 1 depicts the workﬂow of I-NeurAL.

Original: Lines 1-2 initialize the parameters where eachentry of W l is drawn from the normal distribution N (0 , 2 /m ) and each entry of W L is drawnfrom N (0 , 1 /m ) for both f 1 and f 2 .
Revised: Lines 1-2 initialize the parameters where eachentry of W l is drawn from the normal distribution N (0 , 2 /m ) and each entry of W L is drawnfrom N (0 , 1 /m ) for both f 1 and f 2 .

Original: H 10 , H 20 store the historical data for f 1 and f 2 respectively.
Revised: H 10 , H 20 store the historical data for f 1 and f 2 respectively.

Original: Ineach round t , Line 4 builds the k contexts for the observed instance x t , and Lines 5-7 calculate theexploitation-exploration score for each context.
Revised: Ineach round t , Line 4 builds the k contexts for the observed instance x t , and Lines 5-7 calculate theexploitation-exploration score for each context.

Original: b i (Line 8) is the index of the optimal-predicted classand thus y t, b i is the prediction.
Revised: b i (Line 8) is the index of the optimal-predicted classand thus y t, b i is the prediction.

Original: i  (Line 9) is the index of the second optimal-predicted class, whichis used to decide whether to make a query.
Revised: i  (Line 9) is the index of the second optimal-predicted class, whichis used to decide whether to make a query.

Original: Line 11 is our decision component.
Revised: Line 11 is our decision component.

Original: When I t = 1 , itshows that we are not conﬁdent enough about our prediction, so that we make a query for x t andobserve the rewards for each context (Lines 12-17).
Revised: When I t = 1 , itshows that we are not conﬁdent enough about our prediction, so that we make a query for x t andobserve the rewards for each context (Lines 12-17).

Original: When I t = 0 , based on our analysis, with highconﬁdence, the prediction y t, b i matches the one predicted by the Bayes-optimal classiﬁer.
Revised: When I t = 0 , based on our analysis, with highconﬁdence, the prediction y t, b i matches the one predicted by the Bayes-optimal classiﬁer.

Original: Hence, weconsider y t, b i as the label and observe the reward for all contexts (Lines 18-23).
Revised: Hence, weconsider y t, b i as the label and observe the reward for all contexts (Lines 18-23).

Original: In the end, we updatethe networks f 1 and f 2 , based on the collected data (Lines 24-26).
Revised: In the end, we updatethe networks f 1 and f 2 , based on the collected data (Lines 24-26).

Original:  Regret Analysis
Revised:  Regret Analysis

Original: In this section, we provide the regret analysis of I-NeurAL in the over-parameterized neural networks.
Revised: In this section, we provide the regret analysis of I-NeurAL in the over-parameterized neural networks.

Original: First, we need the standard normalization and separateness restricted to the input instances.
Revised: First, we need the standard normalization and separateness restricted to the input instances.

Original: Assumption 4.1.
Revised: Assumption 4.1.

Original: For any t 2 [ T ] , k x t k 2 = 1 and k x t  x t 0 k 2  ⇢ , 8 t 0 

Revised: For any t 2 [ T ] , k x t k 2 = 1 and k x t  x t 0 k 2  ⇢ , 8 t 0

Original: [ T ] where ⇢ 2 (0 , 1) .
Revised: [ T ] where ⇢ 2 (0 , 1) .

Original: This is a mild assumption widely used in the analysis of over-parameterized neural networks [2, 21, 4],and has become standard in neural active learning [45] and neural bandits [50, 10].
Revised: This is a mild assumption widely used in the analysis of over-parameterized neural networks [2, 20, 4],and has become standard in neural active learning [44] and neural bandits [49, 10].

Original: Note that [45] hasthe equivalent assumption that the smallest eigenvalue of Neural Tangent Kernel (NTK) needs to bepositive.
Revised: Note that [44] hasthe equivalent assumption that the smallest eigenvalue of Neural Tangent Kernel (NTK) needs to bepositive.

Original: Then, we have the following main result without any assumption on the data distribution D .
Revised: Then, we have the following main result without any assumption on the data distribution D .

Original: Theorem 4.1.
Revised: Theorem 4.1.

Original: Given the number of rounds T , for any  2 (0 , 1) ,  > 1 , ⇢ 2 (0 , O ( 1 L )) , supposem  poly ( T, k, L, ⇢  1 , e p log( Tk/  ) ) , ⌘ 1 = ⌘ 2 = T 5  2 m .
Revised: Given the number of rounds T , for any  2 (0 , 1) ,  > 1 , ⇢ 2 (0 , O ( 1 L )) , supposem  poly ( T, k, L, ⇢  1 , e p log( Tk/  ) ) , ⌘ 1 = ⌘ 2 = T 5  2 m .

Original: Then, with probability at least 1   over theinitialization of ✓ 10 , ✓ 20 , Algorithm 1 achieves the following regret bound:
Revised: Then, with probability at least 1   over theinitialization of ✓ 10 , ✓ 20 , Algorithm 1 achieves the following regret bound:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: and at the same time N T  O ( T ) .
Revised: and at the same time N T  O ( T ) .

Original: Theorem 4.1 provides the regret bound of I-NeurAL for R T and R T respectively, R T  O ( p log T p T )and R T  O ( p T log T ) .
Revised: Theorem 4.1 provides the regret bound of I-NeurAL for R T and R T respectively, R T  O ( p log T p T )and R T  O ( p T log T ) .

Original: As [45] only provides the regret bound for e
Revised: As [44] only provides the regret bound for e

Original: R T , to show the advantages of
Revised: R T , to show the advantages of

Original: I-NeurAL, we also provide the following lemma for fair comparison.
Revised: I-NeurAL, we also provide the following lemma for fair comparison.

Original: Lemma 4.1.
Revised: Lemma 4.1.

Original: Given the number of rounds T , for any  2 (0 , 1) ,  > 1 , ⇢ 2 (0 , O ( 1 L )) , supposem  poly ( T, k, L, ⇢  1 , e p log( Tk/  ) ) , ⌘ 1 = ⌘ 2 = T 5  2 m .
Revised: Given the number of rounds T , for any  2 (0 , 1) ,  > 1 , ⇢ 2 (0 , O ( 1 L )) , supposem  poly ( T, k, L, ⇢  1 , e p log( Tk/  ) ) , ⌘ 1 = ⌘ 2 = T 5  2 m .

Original: Then, with probability at least 1   over theinitialization of ✓ 10 , ✓ 20 , Algorithm 1 can achieve the following regret bound:
Revised: Then, with probability at least 1   over theinitialization of ✓ 10 , ✓ 20 , Algorithm 1 can achieve the following regret bound:

Original: [Equation]
Revised: [Equation]

Original: and at the same time N T  O ( T ) .
Revised: and at the same time N T  O ( T ) .

Original: Comparison with [45] .
Revised: Comparison with [44] .

Original: Lemma 4.1 shows that I-NeurAL can achieve the regret bound of samecomplexity for e
Revised: Lemma 4.1 shows that I-NeurAL can achieve the regret bound of samecomplexity for e

Original: R T as R T .
Revised: R T as R T .

Original: Under the same assumption in the over-parameterized neural networks,without any assumption on D , Theorem 1 in [45] (i.e., the lower-noise condition with exponent↵ = 0 , and k = 2 is ignored in the binary classiﬁcation) achieves the following regret bound:
Revised: Under the same assumption in the over-parameterized neural networks,without any assumption on D , Theorem 1 in [44] (i.e., the lower-noise condition with exponent↵ = 0 , and k = 2 is ignored in the binary classiﬁcation) achieves the following regret bound:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original:  T (log det( I + H ) + S 2 )) where H is the NTK matrix [26, 4], S = p h > H  1 his a complexity term, and h =
Revised: p T (log det( I + H ) + S 2 )) where H is the NTK matrix [25, 4], S = p h > H  1 his a complexity term, and h =

Original: ( h ( x 1 , b i ) , . . .
Revised: ( h ( x 1 , b i ) , . . .

Original: , h ( x T, b i )) >
Revised: , h ( x T, b i )) >

Original: 2 R T .
Revised: 2 R T .

Original: Note that I-NeurAL and [45] havethe same trivial label complexity O ( T ) in this difﬁcult case.
Revised: Note that I-NeurAL and [44] havethe same trivial label complexity O ( T ) in this difﬁcult case.

Original: According to the deﬁnition of effectivedimension e d in [50], the above regret bound obtained by [45] can be represented by:
Revised: According to the deﬁnition of effectivedimension e d in [49], the above regret bound obtained by [44] can be represented by:

Original: [Equation]
Revised: [Equation]

Original: Remark 4.1.
Revised: Remark 4.1.

Original: Theorem 4.1 and Lemma 4.1 do not depend on e d .
Revised: Theorem 4.1 and Lemma 4.1 do not depend on e d .

Original: The effective dimension e d was ﬁrstintroduced in [44] and then used in [50], which can be thought of as the non-linear dimensions in the NTK kernel space.
Revised: The effective dimension e d wasﬁrst introduced in [43] and then used in [49], which can be thought of as the non-linear dimensionsin the NTK kernel space.

Original: However, e d can be p = m + mkd + m 2 ( L  1) in the worst case, i.e., e d  T(see details in Appendix 10.4).
Revised: However, e d can be p = m + mkd + m 2 ( L  1) in the worst case, i.e.,e d  T (see details in Appendix 9).

Original: Eq.(4.4) has the term O ( e d ) and thus the regret bound obtained by[45] can explode due to e d .
Revised: Eq.(4.4) has the term O ( e d ) and thus the regret bound obtainedby [44] can explode due to e d .

Original: This is because the analysis of [45] closely depends on NTK, i.e., toapply Conﬁdence Ellipsoid bound (Theorem 2 in [1]) to the NTK approximation.
Revised: This is because the analysis of [44] closely depends on NTK, i.e., toapply Conﬁdence Ellipsoid bound (Theorem 2 in [1]) to the NTK approximation.

Original: This procedureinevitably bind their regret bound to the determinant of NTK that can have a very large magnitude.
Revised: This procedureinevitably bind their regret bound to the determinant of NTK that can have a very large magnitude.

Original: In contrast, Eq.(4.3) does not have the term e d , because our analysis does not depend on the NTKapproximation and I-NeurAL directly utilizes the property of over-parameterized neural networks,i.e., the convergence error [2] and the generalization concentration bound [13, 10].
Revised: In contrast, Eq.(4.3) does not have the term e d , because our analysis does not depend on the NTKapproximation and I-NeurAL directly utilizes the property of over-parameterized neural networks,i.e., the convergence error [2] and the generalization concentration bound [13, 10].

Original: These two termsare independent of e d , which paves the way for I-NeurAL to remove the curse of e d .
Revised: These two termsare independent of e d , which paves the way for I-NeurAL to remove the curse of e d .

Original: Remark 4.2.
Revised: Remark 4.2.

Original: Theorem 4.1 and Lemma 4.1 improve the regret by a multiplicative factor
Revised: Theorem 4.1 and Lemma 4.1 improve the regret by a multiplicative factor

Original: O (log T )over [45].
Revised: O (log T )over [44].

Original: Note that the analysis of [45] is built for binary classiﬁcation and thus k = 2 in Theorem4.1 and Lemma 4.1.
Revised: Note that the analysis of [44] is built for binary classiﬁcation and thus k = 2 in Theorem4.1 and Lemma 4.1.

Original: This improvement stems from the different analysis workﬂow of I-NeurAL from[45].
Revised: This improvement stems from the different analysis workﬂow of I-NeurAL from[44].

Original: Again, our analysis does not rely on NTK approximation and it is built on the convergence andgeneralization bound of wide neural networks.
Revised: Again, our analysis does not rely on NTK approximation and it is built on the convergence andgeneralization bound of wide neural networks.

Original: Remark 4.3.
Revised: Remark 4.3.

Original: Theorem 4.1 and Lemma 4.1 do not have the intractable complexity term S .
Revised: Theorem 4.1 and Lemma 4.1 do not have the intractable complexity term S .

Original: S comesfrom the approximation of h using the NTK function (Lemma 1 in [45]), reﬂecting the complexityof h .
Revised: S comesfrom the approximation of h using the NTK function (Lemma 1 in [44]), reﬂecting the complexityof h .

Original: S usually is intractable because we do not know the smallest eigenvalue of H and thus S isdifﬁcult to bound.
Revised: S usually is intractable because we do not know the smallest eigenvalue of H and thus S isdifﬁcult to bound.

Original: To ﬁnd the proper substitute value for S , [45] has to use a complex model selectionalgorithm.
Revised: To ﬁnd the proper substitute value for S , [44] has to use a complex model selectionalgorithm.

Original: In contrast, I-NeurAL does not have this concern because the analysis of I-NeurAL doesnot use the NTK approximation. 
Revised: In contrast, I-NeurAL does not have this concern because the analysis of I-NeurAL doesnot use the NTK approximation. Remark 4.4.

Revised: Our proof workﬂow of Theorem 4.1 and Lemma 4.1 is inspired by [10].

Revised: Comparedto [10], we provide the ﬁrst regret bound supporting mini-batch SGD with warm-start and a moregeneric generalization bound (Lemma 7.3) that holds for every arm (class).

Revised: Moreover, we carry outthe performance analysis of query decision-maker (Lemma 7.5), which is a new addition.

Original: For the label complexity, N T has the trivial O ( T ) complexity which is the same as Theorem1 in [45] (with the exponent ↵ = 0 ).
Revised: For the label complexity, N T has the trivial O ( T ) complexity which is the same as Theorem1 in [44] (with the exponent ↵ = 0 ).

Original: Because we have to consider the worst case where theunique Bayes-optimal class does not exist, i.e., given x t,i , i 2 [ k ] , there does not exist i
Revised: Because we have to consider the worst case where theunique Bayes-optimal class does not exist, i.e., given x t,i , i 2 [ k ] , there does not exist i

Original: ⇤ such thath ( x t,i ⇤ ) >
Revised: ⇤ such thath ( x t,i ⇤ ) >

Original: h ( x t,i ) , 8 i 2 [ k ] \ { i ⇤ } .
Revised: h ( x t,i ) , 8 i 2 [ k ] \ { i ⇤ } .

Original: Therefore, we provide the following analysis and show that R T 255 and N
Revised: Therefore, we provide the following analysis and show that R T 261 and N

Original: T can be upper bounded by constants as long as there exists a unique Bayes-optimal class forthe input instances, described by the following mild margin assumption.
Revised: T can be upper bounded by constants as long as there exists a unique Bayes-optimal class forthe input instances, described by the following mild margin assumption.

Original: Assumption 4.2 ( ✏ -margin) .
Revised: Assumption 4.2 ( ✏ -margin) .

Original: In round t 2 [ T ] , given an instance x t and the label y t , then x t has the✏ -Unique optimal class if there exists ✏ > 0 such that
Revised: In round t 2 [ T ] , given an instance x t and the label y t , then x t has the✏ -Unique optimal class if there exists ✏ > 0 such that

Original: [Equation]
Revised: [Equation]

Original: where i ⇤ = arg max
Revised: where i ⇤ = arg max

Original: i 2 [ k ] h ( x t,i ) is the Bayes-optimal class and i  = arg max i 2 ([ k ] \{
Revised: i 2 [ k ] h ( x t,i ) is the Bayes-optimal class and i  = arg max i 2 ([ k ] \{

Original: i ⇤ } ) h ( x t,i ) is 260 the second Bayes-optimal class.
Revised: i ⇤ } ) h ( x t,i ) is 266 the second Bayes-optimal class.

Original: Theorem 4.2.
Revised: Theorem 4.2.

Original: Suppose the instances drawn from D satisfy Assumption 4.2.
Revised: Suppose the instances drawn from D satisfy Assumption 4.2.

Original: Then, given thenumber of rounds T , for any  2 (0 , 1) , ⇢ 2 (0 , O ( 1 L )) ,  > 1 , ✏ 2 (0 , 1) , suppose m poly ( T, k, L, ⇢  1 , e p log( Tk/  ) ) , ⌘ 1 = ⌘ 2 =
Revised: Then, given thenumber of rounds T , for any  2 (0 , 1) , ⇢ 2 (0 , O ( 1 L )) ,  > 1 , ✏ 2 (0 , 1) , suppose m poly ( T, k, L, ⇢  1 , e p log( Tk/  ) ) , ⌘ 1 = ⌘ 2 =

Original: T 5  2 m .
Revised: T 5  2 m .

Original: Then, with probability at least 1   over theinitialization of ✓ 10 , ✓ 20 , Algorithm 1 achieves the following regret bound:
Revised: Then, with probability at least 1   over theinitialization of ✓ 10 , ✓ 20 , Algorithm 1 achieves the following regret bound:(

Original: [Equation]
Revised: [Equation]

Original: where
Revised: else .

Original: [Equation]
Revised: [Equation]

Revised: where ⇠ = 2 log 2 p 6(  +1) ✏ + log( O ( k ))

Revised: + 94 ( O ( L 2 ) + 49 )  log  .

Original: Remark 4.4.
Revised: Remark 4.5.

Original: Theorem 4.2 provides the upper bound for R T that is independent of T .
Revised: Theorem 4.2 provides the upper bound for R T that is independent of T .

Original: Whenother parameters are ﬁxed, this indicates R T is upper bounded by a constant when T !
Revised: Whenother parameters are ﬁxed, this indicates R T is upper bounded by a constant when T !

Original: + 1 .
Revised: + 1 .

Original: Moreover, the analysis of R T indicates that I-NeurAL can achieve the same performance as Bayesoptimal classiﬁer with high conﬁdence after a ﬁxed number of rounds (i.e. T > 48(  +1) 2 ✏⇠ ).
Revised: Moreover, the analysis of R T indicates that I-NeurAL can achieve the same performance as Bayesoptimal classiﬁer with high conﬁdence after a ﬁxed number of rounds (i.e. T > 48(  +1) 2 ✏⇠ ).

Original: Tothe best of our knowledge, this is the ﬁrst analysis for neural active learning to show that an NNbased algorithm can achieve the performance as the Bayes-optimal classiﬁer after a ﬁxed numberof rounds.
Revised: Tothe best of our knowledge, this is the ﬁrst analysis for neural active learning to show that an NNbased algorithm can achieve the performance as the Bayes-optimal classiﬁer after a ﬁxed numberof rounds.

Original: In Theorem 1 of [45] ( with the exponent ↵ !
Revised: In Theorem 1 of [44] ( with the exponent ↵ !

Original: + 1 equivalent to Assumption 4.2),e
Revised: + 1 equivalent to Assumption 4.2),e

Original: R T  O ( e d log(1 + T ))
Revised: R T  O ( e d log(1 + T ))

Original: q ( e d log(1 + T )
Revised: q ( e d log(1 + T )

Original: + S 2 ) that still is dependent on T because NTK dependson T and e d .
Revised: + S 2 ) that still is dependent on T because NTK dependson T and e d .

Original:  Experiments
Revised:  Experiments

Original: In this section, we evaluate I-NeurAL on public classiﬁcation data sets compared with state-of-theart (SOTA) baselines.
Revised: In this section, we evaluate I-NeurAL on public classiﬁcation data sets compared with state-of-theart (SOTA) baselines.

Original: Due to the space limit, we only report the main results here and leave theimplementation details and parameter sensitivity in the Appendix 7.
Revised: Due to the space limit, we only report the main results here and leave theimplementation details and parameter sensitivity in the Appendix 10.

Original: We report the experimental results on the following six data sets: Phishing 1 , IJCNN [37], Letter [16],
Revised: We report the experimental results on the following six data sets: Phishing 1 , IJCNN [36], Letter [15],

Original: Fashion [46], MNIST [31] and CIFAR-10 [30].
Revised: Fashion [45], MNIST [30] and CIFAR-10 [29].

Original: In each round, one instance is randomly drawn fromthe data set and the algorithm is compelled to make prediction on it.
Revised: In each round, one instance is randomly drawn fromthe data set and the algorithm is compelled to make prediction on it.

Original: Then, the regret is 1 if theprediction does not match the label; the regret is 0 , otherwise.
Revised: Then, the regret is 1 if theprediction does not match the label; the regret is 0 , otherwise.

Original: At the same time, if the algorithmdecides to observe the label, it costs one query budget.
Revised: At the same time, if the algorithmdecides to observe the label, it costs one query budget.

Original: As the algorithm may abusively make labelqueries, we restrict the query budget to 3% of the total number of instances in the data set for faircomparison.
Revised: As the algorithm may abusively make labelqueries, we restrict the query budget to 3% of the total number of instances in the data set for faircomparison.

Original: The compared baselines are described as follows.
Revised: The compared baselines are described as follows.

Original: (1) Random :
Revised: (1) Random :

Original: The NN classiﬁer queries the labelwith a ﬁxed probability p until the query budget is exhausted; (2) Margin : The NN classiﬁer queriesthe label when the predicted probability is lower than a threshold.
Revised: The NN classiﬁer queries the labelwith a ﬁxed probability p until the query budget is exhausted; (2) Margin : The NN classiﬁer queriesthe label when the predicted probability is lower than a threshold.

Original: These two baselines are used in[20].
Revised: These two baselines are used in[19].

Original: (3) NeuAL-NTK-F (Algorithm 1 in [45] : This model makes predictions based on the frozen
Revised: (3) NeuAL-NTK-F (Algorithm 1 in [44] : This model makes predictions based on the frozen

Original: NTK approximation coming with an Upper-Conﬁdence-Bound(UCB)-based exploration strategy.
Revised: NTK approximation coming with an Upper-Conﬁdence-Bound(UCB)-based exploration strategy.

Original: NeuAL-NTK-D (Algorithm 3 in [45]) :
Revised: NeuAL-NTK-D (Algorithm 3 in [44]) :

Original: The prediction is made based on the NN classiﬁer with a
Revised: The prediction is made based on the NN classiﬁer with a

Original: UCB while the NTK is updated accordingly.
Revised: UCB while the NTK is updated accordingly.

Original: (5) ALPS [20] : Given a class of pre-trained hypotheses,the hypothesis minimizing the logistic loss of labeled and pseudo-labeled data is chosen to makepredictions and the label query is based on the disagreement of different hypotheses.Results .
Revised: (5) ALPS [19] : Given a class of pre-trained hypotheses,the hypothesis minimizing the logistic loss of labeled and pseudo-labeled data is chosen to makepredictions and the label query is based on the disagreement of different hypotheses.

Original: The regret comparison on six data sets is shown in Table 1 and Figure 1.

Original: I-NeurALconsistently outperforms all baselines across all data sets.

Original: In particular, I-NeurAL surpasses the

Original: Phishing
Revised: Phishing

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: Results .

Original: [Figure] [Table] [Figure]
Revised: The regret comparison on six data sets is shown in Table 1 and Figure 1.

Original: [Table]
Revised: I-NeurALconsistently outperforms all baselines across all data sets.

Original: [Figure] [Table] [Figure] best baseline by 31.3%, 45.6%, 52.2% on IJCNN, MNIST, Fashion respectively.
Revised: In particular, I-NeurAL surpasses thebest baseline by 31.3%, 45.6%, 52.2% on IJCNN, MNIST, Fashion respectively.

Original: Since NeuALNTK-F uses frozen NTK approximation, the new knowledge of each round is barely utilized bythe neural network and thus it turns into the worst baseline.
Revised: Since NeuALNTK-F uses frozen NTK approximation, the new knowledge of each round is barely utilized bythe neural network and thus it turns into the worst baseline.

Original: NeuAL-NTK-D updates the networkparameters with gradient descent and queries the label based on the uncertainty estimation.
Revised: NeuAL-NTK-D updates the networkparameters with gradient descent and queries the label based on the uncertainty estimation.

Original: However,its upper conﬁdence bound is still based on the conﬁdence ellipsoid.
Revised: However,its upper conﬁdence bound is still based on the conﬁdence ellipsoid.

Original: Instead, I-NeurAL leveragesthe representation power of neural networks for both exploitation and exploration.
Revised: Instead, I-NeurAL leveragesthe representation power of neural networks for both exploitation and exploration.

Original: ALPS maintainsa class of pre-trained hypotheses and tries to make the best decisions based on these hypotheses.
Revised: ALPS maintainsa class of pre-trained hypotheses and tries to make the best decisions based on these hypotheses.

Original: Nevertheless, the model parameters are ﬁxed before the online active learning process.
Revised: Nevertheless, the model parameters are ﬁxed before the online active learning process.

Original: Hence, ALPSis not able to take the new knowledge obtained by queries into account and its performance is highlyrestricted by the hypothesis class.
Revised: Hence, ALPSis not able to take the new knowledge obtained by queries into account and its performance is highlyrestricted by the hypothesis class.

Original: Although Margin algorithm is simple and straightforward, it exhibitsgreat empirical performance in practice.
Revised: Although Margin algorithm is simple and straightforward, it exhibitsgreat empirical performance in practice.

Original: This observation is consistent with other studies [47] [20].
Revised: This observation is consistent with other studies [46] [19].

Original: However, Margin algorithm does not incorporate the exploitation portion and the query criterion isnot adaptive to difference instances, thus still outperformed by I-NeurAL.
Revised: However, Margin algorithm does not incorporate the exploitation portion and the query criterion isnot adaptive to difference instances, thus still outperformed by I-NeurAL.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we introduce two regret metrics and propose a novel neural-based algorithm (I-NeurAL)tailored for the streaming setting of non-parametric active learning.
Revised: In this paper, we introduce two regret metrics and propose a novel neural-based algorithm (I-NeurAL)tailored for the streaming setting of non-parametric active learning.

Original: We carefully design its exploration strategy, query decision-maker, update rules, and training procedure, which lead to both thetheoretical and empirical improvement compared to SOTA [45].
Revised: We carefully design its exploration strategy, query decision-maker, update rules, and training procedure, which lead to both thetheoretical and empirical improvement compared to SOTA [44].

Original: In the regret analysis, we removethe dependence on either the input dimensionality or effective dimensionality, and improve over[45] by a multiplicative factor of O (log T ) .
Revised: In the regret analysis, we removethe dependence on either the input dimensionality or effective dimensionality, and improve over[44] by a multiplicative factor of O (log T ) .

Original: On the other hand, we empirically show that I-NeurALconsistently achieves better accuracy under the same query budget than the strong baselines includingthe SOTA work [45] and [20].
Revised: On the other hand, we empirically show that I-NeurALconsistently achieves better accuracy under the same query budget than the strong baselines includingthe SOTA work [44] and [19].