Original: Semi-supervised Active Linear Regression
Revised: Semi-supervised Active Linear Regression

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Labeled data often comes at a high cost as it may require recruiting human labelersor running costly‘ experiments.
Revised: Labeled data often comes at a high cost as it may require recruiting human labelers or running costly‘ experiments.

Original: At the same time, in many practical scenarios, onealready has access to a partially labeled, potentially biased dataset that can helpwith the learning task at hand.
Revised: At the same time, in many practical scenarios, one already has access to a partially labeled, potentially biased dataset that can help with the learning task at hand.

Original: Motivated by such settings, we formally initiate astudy of semi-supervised active learning through the frame of linear regression.
Revised: Motivated by such settings, we formally initiate a study of semi-supervised active learning through the frame of linear regression.

Original: Here, the learner has access to a dataset X 2 R ( n un + n lab ) ⇥
Revised: Here, the learner has access to a dataset X ∈ R ( n un + n lab ) ×

Original: d composed of n ununlabeled examples that a learner can actively query, and n lab examples labeled apriori.
Revised: d composed of n un unlabeled examples that a learner can actively query, and n lab examples labeled a priori.

Original: Denoting the true labels by Y 2 R n un + n lab , the learner’s objective is to ﬁndb  2 R d such that,
Revised: Denoting the true labels by Y ∈ R n un + n lab , the learner’s objective is to ﬁnd (cid:98) β ∈ R d such that,

Original: [Equation]
Revised: [Equation]

Original: while querying the labels of as few unlabeled points as possible.
Revised: while querying the labels of as few unlabeled points as possible.

Original: In this paper,we introduce an instance dependent parameter called the reduced rank, denoted
Revised: In this paper, we introduce an instance dependent parameter called the reduced rank, denoted

Original: R X , and propose an efﬁcient algorithm with query complexity
Revised: R X , and propose an efﬁcient algorithm with query complexity

Original: O ( R X / ✏ ) .
Revised: O ( R X /(cid:15) ) .

Original: Thisresult directly implies improved upper bounds for two important special cases: ( i )active ridge regression, and ( ii ) active kernel ridge regression, where the reducedrank equates to the statistical dimension , sd  and effective dimension , d  of theproblem respectively, where   0 denotes the regularization parameter.
Revised: This result directly implies improved upper bounds for two important special cases: ( i ) active ridge regression, and ( ii ) active kernel ridge regression, where the reducedrank equates to the statistical dimension , sd λ and effective dimension , d λ of the problem respectively, where λ ≥ 0 denotes the regularization parameter.

Original: Finally,we introduce a distributional version of the problem as a special case of the agnosticformulation we consider earlier; here, for every X , we prove a matching instancewise lower bound of ⌦ ( R X / ✏ ) on the query complexity of any algorithm.
Revised: Finally, we introduce a distributional version of the problem as a special case of the agnostic formulation we consider earlier; here, for every X , we prove a matching instancewise lower bound of Ω( R X / (cid:15) ) on the query complexity of any algorithm.

Original:  Introduction
Revised:  Introduction

Original: Classiﬁcation and regression are the cornerstone of machine learning.
Revised: Classiﬁcation and regression are the cornerstone of machine learning.

Original: Often these tasks are carriedout in a supervised learning manner - a learner collects many examples and their labels and thenruns an optimization algorithm to minimize a loss function and learn a model.
Revised: Often these tasks are carried out in a supervised learning manner - a learner collects many examples and their labels, and then runs an optimization algorithm to minimize a loss function and learn a model.

Original: However, the processof collecting labeled data comes at a cost, for instance when a human labeler is required, or ifan expensive experiment needs to be run for the same.
Revised: However, the process of collecting labeled data comes at a cost, for instance when a human labeler is required, or if an expensive experiment needs to be run for the same.

Original: Motivated by such scenarios, two popularapproaches have been proposed to mitigate these issues in practice: Active Learning [24] and
Revised: Motivated by such scenarios, two popular approaches have been proposed to mitigate these issues in practice: Active Learning [24] and

Original: Semi-Supervised Learning [30].
Revised: Semi-Supervised Learning [30].

Original: In Active Learning, the learner carries out the task by adaptivelyquerying the labels of a small subset of characteristic data points.
Revised: In Active Learning, the learner carries out the task by adaptively querying the labels of a small subset of characteristic data points.

Original: On the other hand, semi-supervisedlearning is motivated by the fact that learners often have access to massive amounts of unlabeleddata in addition to some labeled data, and algorithms leverage both to carry out the learning task.
Revised: On the other hand, semi-supervised learning is motivated by the fact that learners often have access to massive amounts of unlabeled data in addition to some labeled data, and algorithms leverage both to carry out the learning task.

Original: Active learning and Semi-supervised learning have found numerous applications in areas such as textclassiﬁcation [26], visual recognition [17] and foreground-background segmentation [15].
Revised: Active learning and Semi-supervised learning have found numerous applications in areas such as text classiﬁcation [26], visual recognition [17] and foreground-background segmentation [15].

Original: In practice, learners often have access to smaller labeled datasets and larger unlabeled datasets, chosenpoints of which can be submitted for manual labeling.
Revised: In practice, learners often have access to smaller labeled datasets and larger unlabeled datasets, chosen points of which can be submitted for manual labeling.

Original: This motivates the semi-supervised active regression (SSAR) formulation  here, the learner is provided an a priori labeled data  as well as an unlabeled dataset X un 2 R n un ⇥ d , points of which can be iteratively submitted to anexternal agent to be labeled at unit cost.
Revised: This motivates the semi-supervised active regression (SSAR) formulation - here, the learner is provided an a priori labeled data X lab ∈ R n lab × d , as well as an unlabeled dataset X un ∈ R n un × d , points of which can be iteratively submitted to an external agent to be labeled at unit cost.

Original: In the linear regression framework, the objective of thelearner is to then minimize the squared-loss k X   Y k 22 while making as few label queries (of pointsin X un ) as possible.
Revised: In the linear regression framework, the objective of the learner is to then minimize the squared-loss (cid:107) Xβ − Y (cid:107) 22 while making as few label queries (of points in X un ) as possible.

Original: Here X is the overall dataset, X lab [ X un represented as a matrix in R ( n un + n lab ) ⇥ d ,while Y 2 R n un + n lab denotes the corresponding labels.
Revised: Here X is the overall dataset, X lab ∪ X un represented as a matrix in R ( n un + n lab ) × d , while Y ∈ R n un + n lab denotes the corresponding labels.

Original: The learner seeks to ﬁnd a regression functionb  2 R d such that,
Revised: The learner seeks to ﬁnd a regression function (cid:98) β ∈ R d such that,

Original: [Equation]
Revised: [Equation]

Original: Likewise, the ✏ -query complexity of an algorithm is deﬁned as the number of unlabeled points in X unthat must be queried in order to return a b  such that eq.
Revised: Likewise, the (cid:15) -query complexity of an algorithm is deﬁned as the number of unlabeled points in X un that must be queried in order to return a (cid:98) β such that eq.

Original: (2) is satisﬁed with constant probability.
Revised: (2) is satisﬁed with constant probability.

Original: In the special case when no labeled data is provided (i.e. X lab is empty), SSAR reduces to activelinear regression, which has been studied in several prior works [9, 5, 7, 8, 6].
Revised: In the special case when no labeled data is provided (i.e. X lab is empty), SSAR reduces to active linear regression, which has been studied in several prior works [9, 5, 7, 8, 6].

Original: SSAR includes several other important problems as special cases.
Revised: SSAR includes several other important problems as special cases.

Original: For some  > 0 , consider thesetting where the labeled dataset X lab is the set of standard basis vectors scaled by a factor of p  ,with corresponding labels as 0 .
Revised: For some λ > 0 , consider the setting where the labeled dataset X lab is the set of standard basis vectors scaled by a factor of √ λ , with corresponding labels as 0 .

Original: The loss of a learner b  on this dataset exactly corresponds to the ridgeregression loss on the unlabeled dataset, k X un [Equation]   [Equation]{+(cid:98) β − Y un (cid:107) 
+}
Revised: The loss of a learner (cid:98) β on this dataset exactly corresponds to the ridge regression loss on the unlabeled dataset, (cid:107) X un {+(cid:98) β − Y un (cid:107) 
+}

Original: un k 22 +  k  k 22 , where Y un are the labels of points49 in X un .
Revised: + λ (cid:107) β (cid:107) 22 , where Y un are the labels of points in X un .

Original: Thus, active ridge regression is captured as a special case of SSAR.
Revised: Thus, active ridge regression is captured as a special case of SSAR.

Original: On the other hand, it turns out that active kernel ridge regression is also captured as a special caseof SSAR.
Revised: On the other hand, it turns out that active kernel ridge regression is also captured as a special case of SSAR.

Original: Here, data points lie in a reproducing kernel hilbert space (RKHS) [21, 25, 20] that ispotentially inﬁnite dimensional, but the learner has access to a kernel representation of the N datapoints.
Revised: Here, data points lie in a reproducing kernel hilbert space (RKHS) [21, 25, 20] that is potentially inﬁnite dimensional, but the learner has access to a kernel representation of the N data points.

Original: Denoting the kernel matrix by K 2 R N ⇥ N and the labels of the points as Y 2 R N , theobjective in kernel ridge regression is to minimize the loss k K   Y  

Revised: Denoting the kernel matrix by K ∈ R N × N and the labels of the points as Y ∈ R N , the objective in kernel ridge regression is to minimize the loss (cid:107) Kβ − Y (cid:48) (cid:107)

Original: [-k 
-] +  k  k 2 K .
Revised: [-k 
-] + λ (cid:107) β (cid:107) 2 K .

Original: This can be viewedas a special case of the SSAR framework by choosing the labeled dataset as X lab = p  p K 2 R N ⇥ Nwith each point labeled by 0 , and the unlabeled dataset X un = K 2 R N ⇥ N with labels Y [-
-]
Revised: This can be viewed as a special case of the SSAR framework by choosing the labeled dataset as X lab = √ λ √ K ∈ R N × N with each point labeled by 0 , and the unlabeled dataset X un = K ∈ R N × N with labels Y [-
-](cid:48) ∈ R N .

Original: 2 R N .

Original: In this work, the main question we ask is,
Revised: In this work, the main question we ask is,

Original: Can we design algorithms for SSAR with instance-optimal query complexity?
Revised: Can we design algorithms for SSAR with instance-wise optimal query complexities?

Original: Our ﬁrst main contribution is to propose an instance dependent parameter called the reduced rank ,denoted R X , that can be used to upper bound the worst-case optimal query complexity of SSAR.
Revised: Our ﬁrst main contribution is to propose an instance dependent parameter called the reduced rank , denoted R X , that can be used to upper bound the worst-case optimal query complexity of SSAR.

Original: Intuitively, R X measures the relative importance of the points at which labels are known, X lab , in thecontext of the overall dataset.
Revised: Intuitively, R X measures the relative importance of the points at which labels are known, X lab , in the context of the overall dataset.

Original: Formally, the reduced rank is deﬁned as:
Revised: Formally, the reduced rank is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: We propose an efﬁcient polynomial time algorithm for SSAR with ✏ -query complexity bounded by
Revised: We propose an efﬁcient polynomial time algorithm for SSAR with (cid:15) -query complexity bounded by

Original: O ( R X / ✏ ) .
Revised: O ( R X /(cid:15) ) .

Original: In the special cases of active ridge regression and kernel ridge regression, R X is shown tobe the equal to the statistical dimension sd  [3] and the effective dimension d  [28] of the problemrespectively, which directly implies query complexity guarantees for the two special cases.
Revised: In the special cases of active ridge regression and kernel ridge regression, R X is shown to be the equal to the statistical dimension sd λ [3] and the effective dimension d λ [28] of the problem respectively, which directly implies query complexity guarantees for the two special cases.

Original: Theseparameters are formally deﬁned in Section 3.
Revised: These parameters are formally deﬁned in Section 3.

Original: Optimal algorithms for active ridge regression.
Revised: Optimal algorithms for active ridge regression.

Original: The recent work of [6] studies the problem ofactive regression and following a long line of work (see [9]) provides a near optimal (up to constantfactors) algorithm for active regression querying labels of at most O ( d/ ✏ ) points.
Revised: The recent work of [6] studies the problem of active regression and following a long line of work (see [9]) provides a near optimal (up to constant factors) algorithm for active regression querying labels of at most O ( d/(cid:15) ) points.

Original: The algorithm of [6]can be used for active ridge regression too, where  6 = 0 , resulting in a query complexity guaranteeof O ( d/ ✏ ) .
Revised: The algorithm of [6] can be used for active ridge regression too, where λ (cid:54) = 0 , resulting in a query complexity guarantee of O ( d/(cid:15) ) .

Original: However, the dependence on d is not ideal in this setting and an instance-dependentparameter of the data known as statistical dimensions sd  is the right measure of the complexity ofthe problem.
Revised: However, the dependence on d is not ideal in this setting and an instance-dependent parameter of the data known as statistical dimensions sd λ is the right measure of the complexity of the problem.

Original: As a consequence of our general algorithm and guarantee, we obtain an algorithm foractive ridge regression with query complexity upper bounded by O ( sd  / ✏ ) , which we show is in factoptimal in the worst case.
Revised: As a consequence of our general algorithm and guarantee, we obtain an algorithm for active ridge regression with query complexity upper bounded by O ( sd λ /(cid:15) ) , which we show is in fact optimal in the worst case.

Original: As a result we resolve complexity of ridge regression in the active setting.
Revised: As a result we resolve complexity of ridge regression in the active setting.

Original: Improved guarantees for active kernel ridge regression.
Revised: Improved guarantees for active kernel ridge regression.

Original: In the context of kernel ridge regression,the work of [1] analyzed kernel leverage score sampling (and in general Nyström approximation basedmethods), showing that these sampling approaches query O ( d  log( N )) labels to ﬁnd a constantfactor approximation to the optimal loss.
Revised: In the context of kernel ridge regression, the work of [1] analyzed kernel leverage score sampling (and in general Nyström approximation based methods), showing that these sampling approaches query O ( d λ log( N )) labels to ﬁnd a constant factor approximation to the optimal loss.

Original: In contrast, our algorithm when applied in the kernel ridgeregression setting achieves a query complexity guarantee of O ( d  / ✏ ) bearing no dependence on thenumber of datapoints, N , which is often large in practice.
Revised: In contrast, our algorithm when applied in the kernel ridge regression setting achieves a query complexity guarantee of O ( d λ /(cid:15) ) bearing no dependence on the number of datapoints, N , which is often large in practice.

Original: Techniques.
Revised: Techniques.

Original: Our algorithm is based on a novel variant of the Randomized BSS spectral sparsiﬁcation algorithm of [16, 4], that returns a small weighted subset of the dataset, queries its labels,and subsequently carries out weighted least-squares regression on this dataset.
Revised: Our algorithm is based on a novel variant of the Randomized BSS spectral sparsiﬁcation algorithm of [16, 4], that returns a small weighted subset of the dataset, queries its labels, and subsequently carries out weighted least-squares regression on this dataset.

Original: The key technicalcontribution of this work is to construct a sampling algorithm which is guaranteed to query the labelsof only a limited number of unlabeled points.
Revised: The key technical contribution of this work is to construct a sampling algorithm which is guaranteed to query the labels of only a limited number of unlabeled points.

Original: We discuss the algorithm and its guarantees in moredetail in Section 4.
Revised: We discuss the algorithm and its guarantees in more detail in Section 4.

Original: The rest of the paper is organized as follows.
Revised: The rest of the paper is organized as follows.

Original: We cover related work in Section 1.1 and introducerelevant preliminaries in Section 2.
Revised: We cover related work in Section 1.1 and introduce relevant preliminaries in Section 2.

Original: We motivate and present the new parameter R X in Section 3,and present our algorithm for SSAR and a proof sketch in Section 4.
Revised: We motivate and present the new parameter R X in Section 3, and present our algorithm for SSAR and a proof sketch in Section 4.

Original: In Section 5 we introduce thedistributional SSAR problem, and show a matching instance-dependent lower bound for the problem.
Revised: In Section 5 we introduce the distributional SSAR problem, and show a matching instance-dependent lower bound for the problem.

Original: 1.1 Related Work
Revised: 1.1 Related Work

Original: Several prior works use active learning to augment semi-supervised learning based approaches[12, 29, 10, 22, 23].
Revised: Several prior works use active learning to augment semi-supervised learning based approaches [12, 29, 10, 22, 23].

Original: The key in several of these works is to use active learning methods to decidewhich labels to sample, and then use semi-supervised learning to ﬁnally learn the model.
Revised: The key in several of these works is to use active learning methods to decide which labels to sample, and then use semi-supervised learning to ﬁnally learn the model.

Original: In the context of active learning for linear regression, [9, 27] analyze the leverage score samplingalgorithm showing a sample complexity upper bound of O ( d log( d ) / ✏ ) .
Revised: In the context of active learning for linear regression, [9, 27] analyze the leverage score sampling algorithm showing a sample complexity upper bound of O ( d log( d ) / (cid:15) ) .

Original: Subsequently for the case of✏  d + 1 , [7] shows that volume sampling achieves an O ( d ) sample complexity.
Revised: Subsequently for the case of (cid:15) ≥ d + 1 , [7] shows that volume sampling achieves an O ( d ) sample complexity.

Original: Later, [8] showsthat rescaled volume sampling matches the sample complexity of O ( d log( d ) / ✏ ) achieved by leveragescore sampling.
Revised: Later, [8] shows that rescaled volume sampling matches the sample complexity of O ( d log( d ) / (cid:15) ) achieved by leverage score sampling.

Original: Finally, [6] shows that a spectral sparsiﬁcation based approach using the Randomized
Revised: Finally, [6] shows that a spectral sparsiﬁcation based approach using the Randomized

Original: BSS algorithm [16] achieves the optimal sample complexity of O ( d / ✏ ) .
Revised: BSS algorithm [16] achieves the optimal sample complexity of O ( d / (cid:15) ) .

Original: In the context of kernel ridge regression, the works of [11, 28, 19] study the problem of reducing theruntime and number of kernel evaluations to approximately solve the problem.
Revised: In the context of kernel ridge regression, the works of [11, 28, 19] study the problem of reducing the runtime and number of kernel evaluations to approximately solve the problem.

Original: These results showthat the number of kernel evaluations can be made to scale only linearly with the effective dimensionof the problem, d  ( K ) up to log factors.
Revised: These results show that the number of kernel evaluations can be made to scale only linearly with the effective dimension of the problem, d λ ( K ) up to log factors.

Original: Recently, [1] prove a statistical guarantee showing thatany optimal Nystrom approximation based approach samples at most d  ( K ) log( N ) labels to ﬁnd aconstant factor approximation to the kernel regression loss.
Revised: Recently, [1] prove a statistical guarantee showing that any optimal Nystrom approximation based approach samples at most d λ ( K ) log( n ) labels to ﬁnd a constant factor approximation to the kernel regression loss, where n is the dimension of the kernel.

Original: A closely related problem to active learning is the coreset (and weak coreset) problem [2, 18], wherethe objective is to ﬁnd a low-memory data structure that can be used to approximately reconstructthe loss function which naïvely requires storing all the training examples to compute.
Revised: A closely related problem to active learning is the coreset (and weak coreset) problem [2, 18], where the objective is to ﬁnd a low-memory data structure that can be used to approximately reconstruct the loss function which naïvely requires storing all the training examples to compute.

Original: [13] study theweak coreset problem for ridge regression and propose an algorithm that returns a weak coreset of
Revised: [13] study the weak coreset problem for ridge regression and propose an algorithm that returns a weak coreset of

Original: O ( sd  / ✏ ) points which can approximately recover the optimal loss.
Revised: O ( sd λ /(cid:15) ) points which can approximately recover the optimal loss.

Original: A drawback of their algorithmis that the labels of all the points in the dataset are required, which may come at a very high costin practice.
Revised: A drawback of their algorithm is that the labels of all the points in the dataset are required, which may come at a very high cost in practice.

Original: As we discuss in Section 4, our algorithm also returns a weighted subset of O ( sd  / ✏ )points in R d and a set of d additional weights that can be used to reconstruct a (1 + ✏ ) -approximationfor the original ridge regression instance.
Revised: As we discuss in Section 4, our algorithm also returns a weighted subset of O ( sd λ /(cid:15) ) points in R d and a set of d additional weights that can be used to reconstruct a (1 + (cid:15) ) -approximation for the original ridge regression instance.

Original: In terms of memory requirement, this matches the result of[13]; however it sufﬁces for our algorithm to query at most O ( sd  / ✏ ) labels .
Revised: In terms of memory requirement, this matches the result of [13]; however it sufﬁces for our algorithm to query at most O ( sd λ /(cid:15) ) labels .

Original:  Preliminaries
Revised:  Preliminaries

Original: In this section, we formally deﬁne Semi-supervised Active Regression (SSAR) under squared-loss inthe linear setting.
Revised: In this section, we formally deﬁne Semi-supervised Active Regression (SSAR) under squared-loss in the linear setting.

Original: The learner is provided datasets X un 2 R n un ⇥ d (comprised of n un points in R d ) and 
Revised: The learner is provided datasets X un ∈ R n un × d (comprised of n un points in R d ) and Setting Upper bounds [Table]

Original:  X lab 2 R n lab⇥ d .
Revised: [Table] X [Table] ∈ R n lab

Revised: [Table]

Original:  The labels of the points are respectivelydenoted Y un 2 R n un and Y lab 2 R n lab anddeﬁne,
Revised: d . The labels of the points are respectively Y un ∈ R n un and Y lab ∈ R n lab . Deﬁne,

Original: [Equation]
Revised: X = (cid:20) X un X lab (cid:21) , Y = (cid:20) Y un Y lab (cid:21) (4)

Original: In the agnostic setting, the labels of the points are not assumed to be generated from an unknownunderlying linear function and can be arbitrary.
Revised: In the agnostic setting [Table] the labels of the points are not assumed to be generated from an unknown underlying linear function and can be arbitrary.

Original: We study the problem in the overconstrained settingwith n un + n lab  d .
Revised: We study the problem in the overconstrained setting with n un + n lab ≥ d .

Original: Furthermore, in the active setting, it is typically the case that n un  d .
Revised: Furthermore, in the active setting, it is typically the case that n un (cid:29) d .

Original: Whenclear from the context, we use X , X lab and X un to represent the datasets in set notation.
Revised: When clear from the context, we use X , X lab and X un to represent the datasets in set notation.

Original: We assume that the learner is a priori provided Y lab and hence knows the labels of all points in X lab ,but the labels of points in X un are not known.
Revised: We assume that the learner is a priori provided Y lab and hence knows the labels of all points in X lab , but the labels of points in X un are not known.

Original: However, the learner can probe an oracle to returnthe label of any queried point in X un and incurs a unit cost for each label returned.
Revised: However, the learner can probe an oracle to return the label of any queried point in X un and incurs a unit cost for each label returned.

Original: The learner’sobjective is to return a linear function [Equation] d which approximately minimizes the squared ` 2 loss,
Revised: The learner’s objective is to return a linear function (cid:98) β ∈ R d which approximately minimizes the squared (cid:96) 2 loss,

Original: [Equation]
Revised: [Equation]

Original: where,
Revised: where,

Original: [Equation]
Revised: [Equation]

Original: , arg
Revised: (cid:44) arg

Original: [Equation]
Revised: [Equation]

Original: SSAR has the following two important problems as subcases.
Revised: SSAR has the following two important problems as subcases.

Original: Active ridge regression: In the active ridge regression problem, the learner is provided an unlabeleddataset X un 2 R n ⇥ d of n points.
Revised: Active ridge regression: In the active ridge regression problem, the learner is provided an unlabeled dataset X un ∈ R n × d of n points.

Original: The learner has access to an oracle which can be actively queried tolabel points in X un .
Revised: The learner has access to an oracle which can be actively queried to label points in X un .

Original: Representing the labels by Y un 2 R n , the objective of the learner is to minimize
Revised: Representing the labels by Y un ∈ R n , the objective of the learner is to minimize

Original: L  (  ) , k X un   Y un k 22{+(cid:107) 
+} +  k  k 22 .
Revised: L λ ( β ) (cid:44) (cid:107) X un β − Y un {+(cid:107) 
+} + λ (cid:107) β (cid:107) 22 .

Original: Deﬁning X
Revised: Deﬁning X

Original: = 
Revised: = (cid:20) X un √ λI (cid:21) [Equation]

Original: X un p  I  2 R ( n + d ) ⇥ d and Y =
Revised: ( n [Equation] ) × d and Y =

Original: Y un 2 R n + d ,observe that k X   Y k 

Revised: (cid:20) [Equation] (cid:21) ∈ R n + d , observe that (cid:107) Xβ − Y (cid:107)

Original: = k X un   Y un k 

Revised: = (cid:107) X un β − Y un (cid:107)

Original: +  k  k 22 .
Revised: + λ (cid:107) β (cid:107) 22 .

Original: Thus, active ridge regression is a special case138 of the SSAR objective in eq.
Revised: Thus, active ridge regression is a special case of the SSAR objective in eq.

Original: Active kernel ridge regression: In the kernel ridge regression problem, the learner operates in areproducing kernel Hilbert space H and is provided an unlabeled dataset X of n points with (implicit)feature representations  ( x ) for x 2 X .
Revised: Active kernel ridge regression: In the kernel ridge regression problem, the learner operates in a reproducing kernel Hilbert space H and is provided an unlabeled dataset X of n points with (implicit) feature representations Φ( x ) for x ∈ X .

Original: The objective is to learn a linear function F ( x ) , in the featurerepresentations of points in X that minimizes the squared ` 2 norm with regularization.
Revised: The objective is to learn a linear function F ( x ) , in the feature representations of points in X that minimizes the squared (cid:96) 2 norm with regularization.

Original: Namely,
Revised: Namely,

Revised: (cid:80) ni

Original: P ni =1 ( F ( x i )  Y i ) {+
+}
Revised: =1 ( F ( x i ) − Y i ) {+
+}

Original: +  k F k 2 H .
Revised: + λ (cid:107) F (cid:107) 2 H .

Original: By the Representer Theorem [14], it turns out the optimal regressionfunction F ⇤ ( · ) can be expressed as 
Revised: By the Representer Theorem [14], it turns out the optimal regression function F ∗ ( · ) can be expressed as (cid:80) ni

Original: P ni
Revised: =1 β

Original: [Equation]

Original:  i  ( x i , · ) where  ( · , · ) is the kernel function.
Revised: i κ ( x i , · ) where κ ( · , · ) is the kernel function.

Original: The kernelridge regression objective can be expressed in terms of the kernel matrix K , [  ( x i , x j )] ni,j =1 [-
-]
Revised: The kernel ridge regression objective can be expressed in terms of the kernel matrix K (cid:44) [ κ ( x i , x j )] ni,j =1 [-
-]∈

Original: R n ⇥ n .
Revised: R n × n .

Original: In other words, denoting the labels by Y un 2 R n and k  k 2 K =  T K  , the learner’s objectiveis to minimize over  2 R n ,
Revised: In other words, denoting the labels by Y un ∈ R n and (cid:107) β (cid:107) 2 K = β T Kβ , the learner’s objective is to minimize over β ∈ R n ,

Original: [Equation]
Revised: [Equation]

Original: (6) while minimizing the number of label queries of points in X un .
Revised: while minimizing the number of label queries of points in X un .

Original: The objective can be represented as,
Revised: The objective can be represented as,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where Z = p K is deﬁned as the unique solution to K
Revised: where Z = √ K is deﬁned as the unique solution to K

Original: =
Revised: =

Original: Z T Z .
Revised: Z T Z .

Original: Yet again, it is a special case of thelinear regression objective in eq.
Revised: Yet again, it is a special case of the linear regression objective in eq.

Original: (5) with X un = K , X lab = p K and Y lab = 0 .
Revised: (5) with X un = K , X lab = √ K and Y lab = 0 .

Original:  Motivating the reduced rank R X
Revised:  Motivating the reduced rank R X

Original: A natural algorithm for SSAR is to use the popular approach of leverage score sampling [9, 13].
Revised: A natural algorithm for SSAR is to use the popular approach of leverage score sampling [9, 13].

Original: This approach uses the entire dataset X (both labeled and unlabeled) to construct a diagonal weightmatrix W S : here S represents the support of the diagonal and E [ | S | ] , m =
Revised: This approach uses the entire dataset X (both labeled and unlabeled) to construct a diagonal weight matrix W S : here S represents the support of the diagonal and E [ | S | ] (cid:44) m =

Original: O ( d log( d ) / ✏ ) .
Revised: O ( d log( d ) / (cid:15) ) .

Original: Thelearner subsequently solves the problem: 
Revised: The learner subsequently solves the problem: (cid:98)

Revised: β = arg min β (cid:107) W S ( Xβ − Y )

Original: [Equation]{+(cid:107) 
+}
Revised: {+(cid:107) 
+}

Original: X   Y ) k 22 which is a weightedlinear regression problem on | S | examples.
Revised: which is a weighted linear regression problem on | S | examples.

Original: The works of [9, 13] show that resulting linear functionproduces a (1 + ✏ ) multiplicative approximation to the optimal squared-loss:
Revised: The works of [9, 13] show that resulting linear function produces a (1 + (cid:15) ) multiplicative approximation to the optimal squared-loss:

Original: [Equation]
Revised: [Equation]

Original: with constant probability.
Revised: with constant probability.

Original: The weight matrix W S returned by leverage score sampling is constructedas follows:
Revised: The weight matrix W S returned by leverage score sampling is constructed as follows:

Original: Letting U ⌃ V T denote the singular value decomposition of X , the algorithm iteratesover all the points in X and includes a point x in S with probability p ( x ) , min  1 , md k U ( x
Revised: Letting U Σ V T denote the singular value decomposition of X , the algorithm iterates over all the points in X and includes a point x in S with probability p ( x ) (cid:44) min (cid:8) 1 , md (cid:107)

Original: [Equation]
Revised: U ( x ) (cid:107) 22 (cid:9) proportional to its leverage score (cid:107) U

Original: k 22proportional to its leverage score k U ( x ) k 22 , where U ( x ) is the row in U corresponding to the pointx 2 X .
Revised: ( x ) (cid:107) 22 , where U ( x ) is the row in U corresponding to the point x ∈ X .

Original: The corresponding diagonal entry of W S is set as 1 p p ( x ) .
Revised: The corresponding diagonal entry of W S is set as 1 √ p ( x ) .

Original: Since the points in X lab arelabeled, the number of times the algorithm queries the oracle to label a point is equal to | S \ X un | .
Revised: Since the points in X lab are labeled, the number of times the algorithm queries the oracle to label a point is equal to | S ∩ X un | .

Original: Bydeﬁnition of the sampling probabilities, this is proportional to the sum of the leverage scores acrossthe points in X un :
Revised: By deﬁnition of the sampling probabilities, this is proportional to the sum of the leverage scores across the points in X un :

Original: [Equation]
Revised: [Equation]

Original: In Lemma 5, we prove that the term P [Equation] k U 
Revised: In Lemma 5, we prove that the term (cid:80) x ∈ X un (cid:107) U ( x ) (cid:107) 22 equals the reduced rank, R X deﬁned as

Original: [Equation] 22 equals the reduced rank, R X deﬁned as Tr ⇣
Revised: Tr (cid:16)(cid:0)

Original: [Equation]
Revised: [Equation]

Original: ⌘ .

Original: Combining with eq.
Revised: Combining with eq.

Original: (9) results in the equation,
Revised: (9) results in the equation,

Original: [Equation]
Revised: [Equation]

Original: [Equation] In particular, by Markov’s inequality, with large enough constant probability, leverage score samplingmakes at most O ⇣ [Equation] ⌘ label queries (of points in X 1 ).
Revised: In particular, by Markov’s inequality, with large enough constant probability, leverage score sampling makes at most O (cid:16) R X log( d ) (cid:15) (cid:17) label queries (of points in X 1 ).

Original: Combining with the guarantee onthe quality of the solution returned by leverage score sampling in eq.
Revised: Combining with the guarantee on the quality of the solution returned by leverage score sampling in eq.

Original: (8) results in the followingguarantee.
Revised: (8) results in the following guarantee.

Original: Theorem 1.
Revised: Theorem 1.

Original: For 0 < ✏ < 1 , the ✏ -query complexity of leverage score sampling for SSAR is upperbounded by O ( R X · log( d ) ✏ ) .
Revised: For 0 < (cid:15) < 1 , the (cid:15) -query complexity of leverage score sampling for SSAR is upper bounded by O ( R X · log( d ) (cid:15) ) .

Original: In order to further motivate R X we next turn to the ridge regression problem: Here, it turns out that
Revised: In order to further motivate R X we next turn to the ridge regression problem: Here, it turns out that

Original: R X equals the statistical dimension of X un , deﬁned as:
Revised: R X equals the statistical dimension of X un , deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where  i ( X un ) is the i th largest singular value of X un .
Revised: where σ i ( X un ) is the i th largest singular value of X un .

Original: This can be seen by plugging in X lab as p  Iin eq.
Revised: This can be seen by plugging in X lab as √ λI in eq.

Original: (3) and expanding X un using its singular value decomposition as U 1 ⌃ 1 V 1 .
Revised: (3) and expanding X un using its singular value decomposition as U 1 Σ 1 V 1 .

Original: In comparison, in kernel ridge regression, R X evaluates to the effective dimension , deﬁned as:
Revised: In comparison, in kernel ridge regression, R X evaluates to the effective dimension , deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where the  i ’s are the eigenvalues of K .
Revised: where the σ i ’s are the eigenvalues of K .

Original: This yet again follows by plugging the choice of X un and
Revised: This yet again follows by plugging the choice of X un and

Original: X lab in the kernel ridge regression setting from eq.
Revised: X lab in the kernel ridge regression setting from eq.

Original: The common theme in the prior sampling based approaches [1] for kernel ridge regression, as wellas the bound on leverage score sampling established for SSAR (Theorem 1) is that they face alogarithmic dependence in the dimension parameter d or N (in the case of kernel ridge regression)which are often large in practice.
Revised: The common theme in the prior sampling based approaches [1] for kernel ridge regression, as well as the bound on leverage score sampling established for SSAR (Theorem 1) is that they face a logarithmic dependence in the dimension parameter d or N (in the case of kernel ridge regression) which are often large in practice.

Original: Indeed, we show that this logarithmic dependence is unavoidable inthe worst case for leverage score sampling.
Revised: Indeed, we show that this logarithmic dependence is unavoidable in the worst case for leverage score sampling.

Original: Theorem 2.
Revised: Theorem 2.

Original: There exists an instance of SSAR such that the ✏ -query complexity of leverage scoresampling for any constant ✏ is  14 R X log( d ) .
Revised: There exists an instance of SSAR such that the (cid:15) -query complexity of leverage score sampling for any constant (cid:15) is ≥ 14 R X log( d ) .

Original: In this paper, we subsume previous results and show an algorithm with an ✏ -query complexity of
Revised: In this paper, we subsume previous results and show an algorithm with an (cid:15) -query complexity of

Original: O  R X ✏  for SSAR.
Revised: O (cid:0) R X (cid:15) (cid:1) for SSAR.

Original: This shows that it is possible to derive statistical guarantees for active ridge /kernel ridge regression that respectively depend only on the statistical dimension / effective dimension.
Revised: This shows that it is possible to derive statistical guarantees for active ridge / kernel ridge regression that respectively depend only on the statistical dimension / effective dimension.

Original:  Algorithm
Revised:  Algorithm

Original: In this section, we design a polynomial time algorithm
Revised: In this section, we design a polynomial time algorithm

Original: (Algorithm 1) for SSAR and prove that the✏ -query complexity of the algorithm is O ( R X ✏ ) .
Revised: (Algorithm 1) for SSAR and prove that the (cid:15) -query complexity of the algorithm is O ( R X (cid:15) ) .

Original: The core construction of the algorithm is based on an iterative mechanism known as an ✏ -wellbalanced sampling procedure , which decides which points to query the labels of points.
Revised: The core construction of the algorithm is based on an iterative mechanism known as an (cid:15) -well balanced sampling procedure , which decides which points to query the labels of points.

Original: At anintuitive level, an ✏ -well balanced sampling procedure guarantees that the points sampled so far arewell representative of the overall dataset (in a weighted sense), while at the same time guaranteeingthat a certain weighted condition number, related to the error incurred by least squares regression issmall.
Revised: At an intuitive level, an (cid:15) -well balanced sampling procedure guarantees that the points sampled so far are well representative of the overall dataset (in a weighted sense), while at the same time guaranteeing that a certain weighted condition number, related to the error incurred by least squares regression is small.

Original: Our main algorithm, Algorithm 1 follows the template of [6] which ( i ) repeatedly queries the labelsof points in an adaptive manner using an ✏ -well balanced sampling procedure Alg , and ( ii ) solves aweighted linear regression problem on the set of points sampled in step ( i ) .
Revised: Our main algorithm, Algorithm 1 follows the template of [6] which ( i ) repeatedly queries the labels of points in an adaptive manner using an (cid:15) -well balanced sampling procedure Alg , and ( ii ) solves a weighted linear regression problem on the set of points sampled in step ( i ) .

Original: Algorithm 
Revised: Algorithm 1: ASURA (Active semi-SUpervised Regression Algorithm)

Original: Input: X 2 R ( n un + n lab ) ⇥ d ; accuracy parameter ✏ ; any well-balanced sampling procedure Alg (cf.
Revised: Input: X ∈ R ( n un + n lab ) × d ; accuracy parameter (cid:15) ; any well-balanced sampling procedure Alg (cf.

Original: Deﬁnition 1 or Algorithm 2); 1 Run Alg  to return  of 
Revised: Deﬁnition 1 or Algorithm 2); 1 Run Alg ( X, (cid:15) ) to return a subset of points { x 1 , · · · , x m } and weights { w 1 , · · · , w m } .

Original: } and weights is determined internally by Alg 
Revised: ; // m is determined internally by Alg 2 Query the labels

Revised: y i of x 1 , · · · , x m for x i ∈ X un .

Original: ; // the label query cost of the algorithm
Revised: ; // the label query cost of the algorithm

Original: Output:

Original: [Equation]
Revised: [Equation]

Original: We ﬁrst begin by deﬁning the notion of a well-balanced sampling procedure.
Revised: We ﬁrst begin by deﬁning the notion of a well-balanced sampling procedure.

Original: Deﬁnition 1 (well-balanced sampling procedure) .
Revised: Deﬁnition 1 (well-balanced sampling procedure) .

Original: [6, Def 2.1]
Revised: [6, Def 2.1]

Original: A well-balanced sampling procedure is a randomized algorithm Alg
Revised: A well-balanced sampling procedure is a randomized algorithm Alg

Original: ( X, ✏ ) that outputs a set of points { x 1 , · · · , x m } and weights{ w 1 , · · · , w m } as follows: in each iteration i = 1 , · · · , m , the algorithm chooses a distribution D i over points in X to sample x i ⇠ D i .
Revised: ( X, (cid:15) ) that outputs a set of points { x 1 , · · · , x m } and weights { w 1 , · · · , w m } as follows: in each iteration i = 1 , · · · , m , the algorithm chooses a distribution D i over points in X to sample x i ∼ D i .

Original: Let D be the uniform distribution over all points in X .
Revised: Let D be the uniform distribution over all points in X .

Original: Then,
Revised: Then,

Original: Alg is said to be an ✏ -well-balanced sampling procedure if it satisﬁes the following two propertieswith probability  3 / 4 ,
Revised: Alg is said to be an (cid:15) -well-balanced sampling procedure if it satisﬁes the following two properties with probability ≥ 3 / 4 ,

Original: The matrix Z = diag ( { p w i } mi =1 ) U
Revised: The matrix Z = diag ( {√ w i } mi =1 ) U

Original: [ x i : x m ] 2 R m ⇥ d , where U ⌃ V T is the SVD of X and
Revised: [ x i : x m ] ∈ R m × d , where U Σ V T is the SVD of X and

Original: U [ x i : x m ] are the rows of U corresponding to the points { x 1 , · · · , x m } , satisﬁes
Revised: U [ x i : x m ] are the rows of U corresponding to the points { x 1 , · · · , x m } , satisﬁes:

Original: [Equation]
Revised: [Equation]

Original: Equivalently, For every  2 R d , 
Revised: Equivalently, For every β ∈ R d , (cid:80) mi

Revised: =1 w i (cid:104) β,

Original: [Equation]
Revised: [Equation]

Original: For each i 2 [ m ] , deﬁne ↵ i
Revised: For each i ∈ [ m ] , deﬁne α i

Original: = D i ( x i ) D ( x i )
Revised: = D i ( x i ) D ( x i )

Original: w i .
Revised: w i .

Original: Then, Alg must satisfy P [Equation]
Revised: Then, Alg must satisfy (cid:80) mi =1 α i

Original: 1 ↵ i = O (1) and↵ i K D i
Revised: = O (1) and α i K D i

Original: = O ( ✏ ) , where K D i is the re-weighted condition number:
Revised: = O ( (cid:15) ) , where K D i is the re-weighted condition number:

Original: [Equation]
Revised: [Equation]

Original: In [6] it is shown that any sampling procedure in step ( i ) of Algorithm 1 which is well-balanced(Deﬁnition 1) guarantees that the resulting solution is approximately optimal.
Revised: In [6] it is shown that any sampling procedure in step ( i ) of Algorithm 1 which is well-balanced (Deﬁnition 1) guarantees that the resulting solution is approximately optimal.

Original: Theorem 3.
Revised: Theorem 3.

Original: [6, Theorem 2.3]
Revised: [6, Theorem 2.3]

Original: For any 0 < ✏ < 1 , instantiate Algorithm 1 with any ✏ -well balancedsampling procedure Alg ( X, ✏ ) .
Revised: For any 0 < (cid:15) < 1 , instantiate Algorithm 1 with any (cid:15) -well balanced sampling procedure Alg ( X, (cid:15) ) .

Original: Denoting b  as the output of Algorithm 1, with constant probability,
Revised: Denoting (cid:98) β as the output of Algorithm 1, with constant probability,

Original: [Equation]
Revised: [Equation]

Original:  Theorem 3 shows that to solve the SSAR problem, it sufﬁces to design a well-balanced samplingprocedure which queries a minimal number of labels of datapoints in the unlabeled set, X un .
Revised: [Equation] Theorem 3 shows that to solve the SSAR problem, it sufﬁces to design a well-balanced sampling procedure which queries a minimal number of labels of datapoints in the unlabeled set, X un .

Original: Wenow present Algorithm 2 and show that it indeed satisﬁes both of these criteria.
Revised: We now present Algorithm 2 and show that it indeed satisﬁes both of these criteria.

Original: The algorithm isparameterized by  , chosen to be [-, p ✏ /C 
-] for a sufﬁciently large constant C 0 > 0 .
Revised: The algorithm is parameterized by γ , chosen to be [-, p ✏ /C 
-](cid:44) √ (cid:15)/C 0 for a sufﬁciently large constant C 0 > 0 .

Original: Theorem 4.
Revised: Theorem 4.

Original: Algorithm 2 is an ✏ -well-balanced sampling procedure, where 0 < ✏ < 1 .
Revised: Algorithm 2 is an (cid:15) -well-balanced sampling procedure, where 0 < (cid:15) < 1 .

Original: Furthermore,
Revised: Furthermore,

Original: Algorithm 2 samples the labels of at most O {+(cid:0) R X (cid:15) (cid:1) points in X un . Algorithm 2: (cid:15) -well balanced sampling procedure for SSAR Input: X ∈ R n × d .; 1 Initialization: j = 0 ; γ = √ (cid:15)/C 0 ; u 0 = 2 d/γ ; l 0 = − 2 d/γ ; A 0 = 0 . ; 2 while ( u j − l j ) + (cid:80) j − 1 i =
+}
Revised: Algorithm 2 samples the labels of at most O {+(cid:0) R X (cid:15) (cid:1) points in X un . Algorithm 2: (cid:15) -well balanced sampling procedure for SSAR Input: X ∈ R n × d .; 1 Initialization: j = 0 ; γ = √ (cid:15)/C 0 ; u 0 = 2 d/γ ; l 0 = − 2 d/γ ; A 0 = 0 . ; 2 while ( u j − l j ) + (cid:80) j − 1 i =
+}

Original: R X ✏  points in X un .{+Φ Id i < 8 d/γ do Deﬁne Φ Id j = Tr (cid:0) ( u j I − A j ) − 
 + (+}
Revised: {+Φ Id i < 8 d/γ do Deﬁne Φ Id j = Tr (cid:0) ( u j I − A j ) − 
 + (+}

Revised: [Equation]

Original: p
Revised: j

Original: [Equation]
Revised: [Equation]

Original: // Define the sampled point x as x j and p x as p j 
Revised: l j I

Original: [Equation]
Revised: [Equation]

Original: 
Revised: − 1 (cid:1) . Set coefﬁcient α j = γ / Φ Id j ; Sample point from the multinomial distribution assigns probability

Original: [Equation]
Revised: [Equation]

Original:  x  j[Equation]
Revised: // Define the sampled point x as x j and p x as p j

Original:  , set the coefﬁcient ↵ j = ↵ 0 j mid and the weight w j = w 0 j mid .;
Revised: 10 end 11 Assign and for each j , set the coefﬁcient α j = α (cid:48) j mid and the weight w j = w (cid:48) j mid .;

Original: Output: [Equation]
Revised: Output: { x 1 , · · · , x m } ; { w 1 , · · · , w m } .;

Original: Algorithm 2 is a variant of the Randomized BSS algorithm of [16] for spectral sparsiﬁcation (eq.
Revised: Algorithm 2 is a variant of the Randomized BSS algorithm of [16] for spectral sparsiﬁcation (eq.

Original: (13))with an updated stopping criterion, which we discuss in Remark 1.
Revised: (13)) with an updated stopping criterion, which we discuss in Remark 1.

Original: In each iteration, the algorithmmaintains a matrix A which up to a scaling factor is the same as the matrix Z T Z in Deﬁnition 1,and an upper and lower threshold u and l which are updated in an iterative manner.
Revised: In each iteration, the algorithm maintains a matrix A which up to a scaling factor is the same as the matrix Z T Z in Deﬁnition 1, and an upper and lower threshold u and l which are updated in an iterative manner.

Original: The updaterule is guaranteed to maintain the invariant lI  A  uI in every iteration, which is ensured bythe adaptive sampling distribution in eq.
Revised: The update rule is guaranteed to maintain the invariant lI (cid:22) A (cid:22) uI in every iteration, which is ensured by the adaptive sampling distribution in eq.

Original: Over iterations, the algorithm shows that the upperand lower thresholds u and l can be brought closer to each other which sandwich Z T Z (equal to Aup to scaling) until eq.
Revised: Over iterations, the algorithm shows that the upper and lower thresholds u and l can be brought closer to each other which sandwich Z T Z (equal to A up to scaling) until eq.

Original: (13) is ﬁnally satisﬁed.
Revised: (13) is ﬁnally satisﬁed.

Original: It turns out that the desirable quality for is that uand l progressively become larger (which dictates the scaling factor relating A and Z T Z ), while atthe same time gap between the two, u  l , remains small.
Revised: It turns out that the desirable quality for is that u and l progressively become larger (which dictates the scaling factor relating A and Z T Z ), while at the same time gap between the two, u − l , remains small.

Original: The proof of correctness of Algorithm 2requires two ingredients:
Revised: The proof of correctness of Algorithmrequires two ingredients:

Original: Algorithm 2 is indeed an ✏ -well balanced sampling procedure.
Revised: Algorithm 2 is indeed an (cid:15) -well balanced sampling procedure.

Original: (Section 4.1), and 
Revised: (Section 4.1), and 2.

Original: The number of label queries made by Algorithm 2 is bounded by O ( R X / ✏ ) (Section 4.2)
Revised: The number of label queries made by Algorithm 2 is bounded by O ( R X /(cid:15) ) (Section 4.2).

Original: Remark 1.
Revised: Remark 1.

Original: Notice the important difference in the stopping criterion of Algorithm 2 compared tothat of the Randomized BSS Algorithm of [16] which is just the condition u j  l j < 8 d /  .
Revised: Notice the important difference in the stopping criterion of Algorithm 2 compared to that of the Randomized BSS Algorithm of [16] which is just the condition u j − l j < 8 d / γ .

Original: The moreaggressive stopping criterion can be used to show that Algorithm 2 terminates within 2 d /  2 iterationsof the while loop in Step 2 almost surely, while the Randomized BSS algorithm [16] terminates within243 2 d /  2 iterations only with constant probability.
Revised: The more aggressive stopping criterion can be used to show that Algorithm 2 terminates within 2 d / γ 2 iterations of the while loop in Step 2 almost surely, while the Randomized BSS algorithm [16] terminates within 2 d / γ 2 iterations only with constant probability.

Original: This behavior is critical in bounding the number oflabel queries made by Algorithm 2.
Revised: This behavior is critical in bounding the number of label queries made by Algorithm 2.

Original: 4.1 Algorithm 2 is an ✏ -well-balanced sampling procedure
Revised: 4.1 Algorithm 2 is an (cid:15) -well-balanced sampling procedure

Original: We ﬁrst address the question of showing that Algorithm 2 is an ✏ -well balanced sampling procedure.
Revised: We ﬁrst address the question of showing that Algorithm 2 is an (cid:15) -well balanced sampling procedure.

Original: We provide a brief sketch which addresses the ﬁrst condition, eq.
Revised: We provide a brief sketch which addresses the ﬁrst condition, eq.

Original: First observe that the stoppingcriterion of the algorithm is the ﬁrst time j when P [Equation] P
Revised: First observe that the stopping criterion of the algorithm is the ﬁrst time j when (cid:80) j − 1 i =

Original: [Equation]
Revised: [Equation]

Original:  .

Original: Note thateach term in the summation,
Revised: Note that each term in the summation,

Original: [Equation]
Revised: [Equation]

Original:  by the AM-GMinequality.
Revised: inequality.

Original: Therefore, if the number of terms in the summation is at least 8 d  ⇥ 14  = 2 d  2 the stoppingcriterion must certainly be violated.
Revised: Therefore, if the number of terms in the summation is at least 8 dγ × 14 γ = 2 dγ 2 the stopping criterion must certainly be violated.

Original: This results in a bound on the number of iterations before thewhile loop is terminated in Algorithm 2, deﬁned as m .
Revised: This results in a bound on the number of iterations before the while loop is terminated in Algorithm 2, deﬁned as m .

Original: Lemma 1.
Revised: Lemma 1.

Original: Almost surely, m  2 d /  2 .
Revised: Almost surely, m ≤ 2 d / γ 2 .

Original: While m is upper bounded by 2 d /  2 , the key result of this section is that the number of iterations ofby the algorithm also satisﬁes m  cd /  2 with constant probability, for a sufﬁciently small constant c .
Revised: While m is upper bounded by 2 d / γ 2 , the key result of this section is that the number of iterations of by the algorithm also satisﬁes m ≥ cd / γ 2 with constant probability, for a sufﬁciently small constant c .

Original: The implication of the algorithm not terminating too soon is that in the upper and lower boundariesmaintained in the algorithm are updated over a large number of iterations and therefore the eigenvaluesof the matrix A are very tightly sandwiched, proving the eigenvalue condition in eq.
Revised: The implication of the algorithm not terminating too soon is that in the upper and lower boundaries maintained in the algorithm are updated over a large number of iterations and therefore the eigenvalues of the matrix A are very tightly sandwiched, proving the eigenvalue condition in eq.

Original: We ﬂeshout the details below.
Revised: We ﬂesh out the details below.

Original: Note that in each iteration, the upper boundary u j increments by  1  2  1  Id j .
Revised: Note that in each iteration, the upper boundary u j increments by [Equation]

Original: As we show in the
Revised: As we show in the

Original: Appendix A.3 (Lemma 7),  Id j is   / 2 almost surely and therefore u j increments by at least aconstant  1  2  ⇥ 2   2 in each iteration of Algorithm 2.
Revised: Appendix A.3 (Lemma 7), Φ Id j is ≥ γ/ 2 almost surely and therefore u j increments by at least a constant γ 1 − 2 γ × 2 γ ≥ 2 in each iteration of Algorithm 2.

Original: Since the algorithm does not terminatetoo soon and m  cd /  2 with constant probability, in the ﬁnal iteration, u m is also  cd /  2 ⇥ 2 withconstant probability.
Revised: Since the algorithm does not terminate too soon and m ≥ cd / γ 2 with constant probability, in the ﬁnal iteration, u m is also ≥ cd / γ 2 × 2 with constant probability.

Original: More precisely, we have the following result.
Revised: More precisely, we have the following result.

Original: Lemma 2.
Revised: Lemma 2.

Original: For  < 1 / 4 and any 0  p < 1 , with probability at least 1  p , u m  p 2 d / 8  2 .
Revised: For γ < 1 / 4 and any 0 ≤ p < 1 , with probability at least 1 − p , u m ≥ p 2 d / 8 γ 2 .

Original: We next relate the matrix A maintained in the algorithm and the matrix Z T Z in Deﬁnition 1.
Revised: We next relate the matrix A maintained in the algorithm and the matrix Z T Z in Deﬁnition 1.

Original: When
Revised: When

Original: Algorithm 2 terminates, the matrix A m stored is equal to P m  1 j [-=
  -] Id j 1 p j U ( x j )
Revised: Algorithm 2 terminates, the matrix A m stored is equal to (cid:80) m − 1 j [-=
  -]=0 γ Φ Id j 1 p j U ( x j )

Original: ( U ( x j )) T .
Revised: ( U ( x j )) T .

Original: From thedeﬁnition of mid and w j in Algorithm 2, it is a quick calculation to show that Z T Z = 1 mid A =2 u m + l m A m .
Revised: From the deﬁnition of mid and w j in Algorithm 2, it is a quick calculation to show that Z T Z = 1 mid A =u m + l m A m .

Original: Furthermore, since the invariant of Algorithm 2 guarantees that l m I 
Revised: Furthermore, since the invariant of Algorithm 2 guarantees that l m I (cid:22)

Original: A m  u m I , wehave that 2 l m u m + l m  Z T Z  2 u m u m + l m .
Revised: A m (cid:22) u m I , we have that [Equation] + l m .

Original: Thus to prove eq.
Revised: Thus to prove eq.

Original: (13) it sufﬁces to show that u m  l m u m + l m is small.
Revised: (13) it sufﬁces to show that u m − l m u m + l m is small.

Original: By the stopping criterion of the algorithm, we are guaranteed that the gap, u m  l m ⇡ u m  1  l m  1  272 8 d /  ; formally we show that u m  l m  9 d /  in Appendix A.4 (Lemma 9), implying that u m and l mare close to each other.
Revised: By the stopping criterion of the algorithm, we are guaranteed that the gap, u m − l m ≈ u m − 1 − l m − 1 ≤ 8 d / γ ; formally we show that u m − l m ≤ 9 d / γ in Appendix A.4 (Lemma 9), implying that u m and l m are close to each other.

Original: Moreover from Lemma 2, with constant probability, l m ⇡ u m = ⌦ ( d /  2 ) .
Revised: Moreover from Lemma 2, with constant probability, l m ≈ u m = Ω ( d / γ 2 ) .

Original: Thus, the ratio u m  l m u m + l m =
Revised: Thus, the ratio u m − l m u m + l m =

Original: O (  ) and satisﬁes eq.
Revised: O ( γ ) and satisﬁes eq.

Original: (13) for sufﬁciently small  .
Revised: (13) for sufﬁciently small γ .

Original: Lemma 3.
Revised: Lemma 3.

Original: With probability at least 3 / 4 , (1  O (  ))
Revised: With probability at least 3 / 4 , (1 − O ( γ ))

Original: I  A  (1 + O (  ))
Revised: I (cid:22) A (cid:22) (1 + O ( γ ))

Original: I .
Revised: I .

Original: We defer the proof of the fact that Algorithm 2 satisﬁes the second condition in Deﬁnition 1 to
Revised: We defer the proof of the fact that Algorithm 2 satisﬁes the second condition in Deﬁnition 1 to

Original: Appendix A (Lemma 12) since it largely follows from the analysis in [6].4.2 Bounding the number of label queries made by Algorithm[-
-]
Revised: Appendix A (Lemma 12) since it largely follows from the analysis in [6]. 4.2 Bounding the number of label queries made by Algorithm[-
-]

Original: Finally, we upper bound the number of times Algorithm 1 queries the labels of points in the unlabeledset, X un .
Revised: Finally, we upper bound the number of times Algorithm 1 queries the labels of points in the unlabeled set, X un .

Original: We denote this quantity by lq ( X un ) .
Revised: We denote this quantity by lq ( X un ) .

Original: First observe that the number of label queries, lq ( X un ) , is at most the number of iterations in whichthe well-balanced sampling procedure Alg in Line 1 of Algorithm 1 samples a point in X un .
Revised: First observe that the number of label queries, lq ( X un ) , is at most the number of iterations in which the well-balanced sampling procedure Alg in Line 1 of Algorithm 1 samples a point in X un .

Original: Theformer is smaller if a point is sampled more than once by Alg .
Revised: The former is smaller if a point is sampled more than once by Alg .

Original: Indeed,
Revised: Indeed,

Original: [Equation]
Revised: [Equation]

Original: where m is the total number of iterations the algorithm runs for, and I ( j ) x is an indicator capturingwhich is 1 if x is sampled in Step 5 of Algorithm 2 in iteration j and 0 otherwise.
Revised: where m is the total number of iterations the algorithm runs for, and I ( j ) x is an indicator capturing which is 1 if x is sampled in Step 5 of Algorithm 2 in iteration j and 0 otherwise.

Original: Note that
Revised: Note that

Original: E
Revised: E

Original: [ I ( j ) x ] = E [ p ( j ) x ] is the probability of sampling the point x as deﬁned in eq.
Revised: [ I ( j ) x ] = E [ p ( j ) x ] is the probability of sampling the point x as deﬁned in eq.

Original: Recall that in
Revised: Recall that in

Original: Lemma 1 we show that m is upper bounded by 2 d  2 almost surely.
Revised: Lemma 1 we show that m is upper bounded by 2 dγ 2 almost surely.

Original: By linearity of expectation,
Revised: By linearity of expectation,

Original: [Equation]
Revised: [Equation]

Original: Remark 2.
Revised: Remark 2.

Original: Note that the bound on the expected number of label queries in eq.
Revised: Note that the bound on the expected number of label queries in eq.

Original: (17) is the sum ofnon-negative random variables ( I ( j ) x ) which are correlated with the number of terms in the summation( m ).
Revised: (17) is the sum of non-negative random variables ( I ( j ) x ) which are correlated with the number of terms in the summation ( m ).

Original: The almost sure upper bound on m enables this correlation to be decoupled, which is noteasily possible with a constant or even high probability upper bound on m , as guaranteed by the
Revised: The almost sure upper bound on m enables this correlation to be decoupled, which is not easily possible with a constant or even high probability upper bound on m , as guaranteed by the

Original: Randomized BSS algorithm [16] (discussed in Remark 1).
Revised: Randomized BSS algorithm [16] (discussed in Remark 1).

Original: Before studying P [Equation] P
Revised: Before studying (cid:80)

Original: [Equation]

Original: p

Original: [Equation]
Revised: [Equation]

Original: i , we ﬁrst introduce some relevant notation.
Revised: , we ﬁrst introduce some relevant notation.

Original: Deﬁnition 2.
Revised: Deﬁnition 2.

Original: For a PSD matrix M , deﬁne the potential function  Mj =
Revised: For a PSD matrix M , deﬁne the potential function Φ Mj =

Original: Tr ( M ( u j I  A j )  1 +
Revised: Tr ( M ( u j I − A j ) − 1 +

Original: M ( A j  l j I )  1 )) .
Revised: M ( A j − l j I ) − 1 )) .

Original: u j , l j and A j are as deﬁned in Algorithm 2.
Revised: u j , l j and A j are as deﬁned in Algorithm 2.

Original: Simplifying from deﬁnition of p ( j ) x results in the following lemma.
Revised: Simplifying from deﬁnition of p ( j ) x results in the following lemma.

Original: Lemma 4.
Revised: Lemma 4.

Original: In iteration j = 0 , · · · , m  1 , 
Revised: In iteration j = 0 , · · · , m − 1 , (cid:80) x ∈ X un p ( j ) x

Original: [Equation]
Revised: = Φ Dj Φ Id j , where D = (cid:80) x ∈ X un U ( x )( U ( x )) T .

Original: As we show in Appendix A (Lemma 7), the term in the denominator,  Id j , is at least  Id j   / 2almost surely.
Revised: As we show in Appendix A (Lemma 7), the term in the denominator, Φ Id j , is at least Φ Id j ≥ γ/almost surely.

Original: By plugging in Lemma 4 into eq.
Revised: By plugging in Lemma 4 into eq.

Original: (18), the expected number of labels of points X unqueried by Algorithm 2 is upper bounded by,
Revised: (18), the expected number of labels of points X un queried by Algorithm 2 is upper bounded by,

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Over the course of the time, one expects the gap between u j and l j to grow apart (from the updaterule on line 8 of Algorithm 2), while A j remains tightly sandwiched in between the two bounds.
Revised: Over the course of the time, one expects the gap between u j and l j to grow apart (from the update rule on line 8 of Algorithm 2), while A j remains tightly sandwiched in between the two bounds.

Original: Itturns out that a consequence of this fact, the expectation of the potential, E ⇥  Dj ⇤ is a decreasingfunction in j .
Revised: It turns out that a consequence of this fact, the expectation of the potential, E (cid:2) Φ Dj (cid:3) is a decreasing function in j .

Original: We formally prove this result in Appendix A.3 (Lemma 8).
Revised: We formally prove this result in Appendix A.3 (Lemma 8).

Original: From eq.
Revised: From eq.

Original: (19), this resultsin the following upper bound on the number of label queries,
Revised: (19), this results in the following upper bound on the number of label queries,

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: where the last equation, E [  D 0 ] = 
Revised: where the last equation, E [Φ D 0 ] = E [ Tr

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: 0 I )  1 )] =  Tr ( D ) d is a shortcalculation from its deﬁnition using the fact that u 0 = {+− l 
+}
Revised: is a short calculation from its deﬁnition using the fact that u 0 = {+− l 
+}

Original:  l 0 = 2 d  and A 0 = 0 .
Revised: = 2 dγ and A 0 = 0 .

Original: The ﬁnal lemmarelates Tr ( D ) to the reduced rank R X .
Revised: The ﬁnal lemma relates Tr ( D ) to the reduced rank R X .

Original: Lemma 5 (Relating D to R X ) .
Revised: Lemma 5 (Relating D to R X ) .

Original: With D = P x 2 X un U ( x )( U ( x )) T , Tr ( D ) =
Revised: With D = (cid:80) x ∈ X un U ( x )( U ( x )) T , Tr ( D ) =

Original: R X .
Revised: R X .

Original: Putting together Lemma 5 with eq.
Revised: Putting together Lemma 5 with eq.

Original: (20), and noting the deﬁnition  = p ✏ /C 0 results in the ﬁnalbound on the number of label queries made by Algorithm 2. 
Revised: (20), and noting the deﬁnition γ = √ (cid:15)/C 0 results in the ﬁnal bound on the number of label queries made by Algorithm 2. [Equation]

Original: [Equation]
Revised: [Equation]

Original: Remark 3.
Revised: Remark 3.

Original: In the case of ridge regression, Algorithm 1 can be used to construct a weak-coresetfor  ⇤ .
Revised: In the case of ridge regression, Algorithm 1 can be used to construct a weak-coreset for β ∗ .

Original: The advantage of the algorithm over [13] is that the weak coreset can be constructedwithout having oracle access to the set of all labels.
Revised: The advantage of the algorithm over [13] is that the weak coreset can be constructed without having oracle access to the set of all labels.

Original: In particular, our algorithm returns a coresetwhich uses O ( d sd  / ✏ ) bits of memory: up to log-factors, the weight matrix W S requires d ✏ bits ofmemory to store and the sd  / ✏ points queried in X un require d sd  ✏ bits of memory to store.
Revised: In particular, our algorithm returns a coreset which uses O ( d sd λ /(cid:15) ) bits of memory: up to log-factors, the weight matrix W S requires d(cid:15) bits of memory to store and the sd λ /(cid:15) points queried in X un require d sd λ (cid:15) bits of memory to store.

Original: Solving:arg min  2 R d k W S X   W S
Revised: Solving: arg min β ∈ R d (cid:107) W S

Original:  Y k 22 returns a (1 + ✏ ) approximate optimizer of the loss.
Revised: Xβ − W S Y (cid:107) 22 returns a (1 + (cid:15) ) approximate optimizer of the loss.

Original: The memory318 requirement of our algorithm matches that proposed in [13], while only querying a few labels.
Revised: The memory requirement of our algorithm matches that proposed in [13], while only querying a few labels.

Original:  Instance-dependent lower bound for distributional SSAR
Revised:  Instance-dependent lower bound for distributional SSAR

Original: In this section we introduce a distributional or inductive [6] formulation of semi-supervised activeregression as a special case of the agnostic formulation discussed earlier.
Revised: In this section we introduce a distributional or inductive [6] formulation of semi-supervised active regression as a special case of the agnostic formulation discussed earlier.

Original: Deﬁnition 3 (Distributional SSAR) .
Revised: Deﬁnition 3 (Distributional SSAR) .

Original: The setup here is identical to the agnostic setting, but with thelabels revealed to the learner corrupted by independent stochastic noise.
Revised: The setup here is identical to the agnostic setting, but with the labels revealed to the learner corrupted by independent stochastic noise.

Original: In particular, at trainingtime, each time the label of x 2 X is observed (either in the labeled dataset, or by explicitly querying)a noisy version of the ground-truth label, f ( x ) , is revealed.
Revised: In particular, at training time, each time the label of x ∈ X is observed (either in the labeled dataset, or by explicitly querying) a noisy version of the ground-truth label, f ( x ) , is revealed.

Original: Namely, y = f ( x ) + Z is observed,where the training time noise Z ⇠ N (0 ,  2 x ) 1 .
Revised: Namely, y = f ( x ) + Z is observed, where the training time noise Z ∼ N (0 , σ 2 x ) 2 .

Original: The objective is to minimize the generalization error,
Revised: The objective is to minimize the generalization error,

Original: [Equation]
Revised: [Equation]

Original: (22) where for each x 2 X , Z 0 x ⇠ N (0 ,  2 x ) is sampled independent of the noise at training time.
Revised: where for each x ∈ X , Z (cid:48) x ∼ N (0 , σ 2 x ) is sampled independent of the noise at training time.

Original: Note that the distributional SSAR problem is a special case of the agnostic formulation.
Revised: Note that the distributional SSAR problem is a special case of the agnostic formulation.

Original: This justcorresponds to having inﬁnitely many copies of the labeled and unlabeled datasets, where the label ineach copy is the ground truth corrupted by a different realization of the stochastic noise.
Revised: This just corresponds to having inﬁnitely many copies of the labeled and unlabeled datasets, where the label in each copy is the ground truth corrupted by a different realization of the stochastic noise.

Original: Therefore, itmay be easily veriﬁed that the guarantee of ASURA (Theorem 3) carries over here, which we avoidrestating in the interest of space.
Revised: Therefore, it may be easily veriﬁed that the guarantee of ASURA (Theorem 3) carries over here. Corollary 1.

Revised: The ASURA algorithm for the distributional SSAR can be implemented efﬁciently (polynomial in d , the support size of X ) for the distributional SSAR problem, requiring to compute

Revised: SVDs over a single copy of the matrix X .

Revised: Furthermore, the guarantee of ASURA still carries over: the algorithm queries the labels of at most O

Revised: ( R X /(cid:15) ) points, and guarantees to achieve a (1 + (cid:15) ) approximation to the best achievable generalization loss (eq.

Original: It turns out that in the distributional setting, one may also come upwith an instance-dependent sample complexity lower bound, where the input design X is ﬁxed, butthe adversary has the power to control the ground truth labels f ( · ) as well as the noise variances  2 xin designing the instance.
Revised: It turns out that in the distributional setting, one may also come up with an instance-dependent sample complexity lower bound, where the input design X is ﬁxed, but the adversary has the power to control the ground truth labels f ( · ) as well as the noise variances σ 2 x in designing the instance.

Original: Theorem 5.
Revised: Theorem 5.

Original: For any 0 < ✏ < 1 , d and any  > 0 .
Revised: For any 0 < (cid:15) < 1 , d and any λ > 0 .

Original: Suppose X satisﬁes the condition, min [Equation]
Revised: Suppose X satisﬁes the condition, σ min (cid:0) X

Original: ✏ 1  ✏ .
Revised: ( X T X ) − 1 X T lab (cid:1) ≥ (cid:15) 1 − (cid:15) .

Original: In the inductive setting, for each X and learner there exists an instance of SSAR where if b  returned by the learner satisﬁes E 
Revised: In the inductive setting, for each X and learner there exists an instance of SSAR where if (cid:98) β returned by the learner satisﬁes E (cid:104) (cid:107) X (cid:98)

Original: [Equation]{+β − Y (cid:107) 
+}
Revised: {+β − Y (cid:107) 
+}

Original: [Equation]
Revised: (cid:105) ≤ (1 + (cid:15) ) min β ∈ R d E (cid:2) (cid:107)

Original: ⇤ must query the labels of at least ⌦ ( R X ✏ ) points.
Revised: Xβ − Y (cid:107) 22 (cid:3) must query the labels of at least Ω( R X (cid:15) ) points.

Original: The proof of this lower bound is quite complicated because of both the adaptive nature of the problem,as well the requirement for an instance-dependent bound.
Revised: The proof of this lower bound is quite complicated because of both the adaptive nature of the problem, as well the requirement for an instance-dependent bound.

Original: At a high level, we use an hypothesistesting based construction (Assouad’s lemma) with a careful choice of hypothesis space.
Revised: At a high level, we use an hypothesis testing based construction (Assouad’s lemma) with a careful choice of hypothesis space.

Original: The mainchallenge is choosing the underlying instance in a way to recover the reduced rank as the measure ofcomplexity, which has a very delicate dependence on (the spectrum of) X lab and X un .
Revised: The main challenge is choosing the underlying instance in a way to recover the reduced rank as the measure of complexity, which has a very delicate dependence on (the spectrum of) X lab and X un .

Original: In the interestof space, we defer the details of th proof to Appendix A.5.
Revised: In the interest of space, we defer the details of th proof to Appendix A.5.

Original: This result also shows that there is noneed to have any sort of commutation structure between the labelled and unlabelled data matrices
Revised: This result also shows that there is no need to have any sort of commutation structure between the labelled and unlabelled data matrices

Original: X lab and X un for the reduced rank to remain as the right instance dependent lower bound.
Revised: X lab and X un for the reduced rank to remain as the right instance dependent lower bound.

Original:  Conclusion
Revised:  Conclusion

Original: We introduce the semi-supervised active learning problem and prove a query complexity upper boundof R X / ✏ for linear regression.
Revised: We introduce the semi-supervised active learning problem and prove a query complexity upper bound of R X / (cid:15) for linear regression.

Original: In the special case of active ridge regression, this implies a samplecomplexity upper bound of O ( sd  / ✏ ) labels which we prove is optimal.
Revised: In the special case of active ridge regression, this implies a sample complexity upper bound of O ( sd λ / (cid:15) ) labels which we prove is optimal.

Original: The problem also generalizesactive kernel ridge regression, where we show a query complexity upper bound of O ( d  / ✏ ) improvingthe results of [1].
Revised: The problem also generalizes active kernel ridge regression, where we show a query complexity upper bound of O ( d λ / (cid:15) ) improving the results of [1].

Original: It is an open question to extend these results to other loss functions and to designnear-linear time algorithms for the problem.  
Revised: It is an open question to extend these results to other loss functions and to design near-linear time algorithms for the problem.  Acknolwedgements

Revised: NR was partially supported by NSF Grants IIS-1901252, and CCF-1909499.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] We describe some future directions in Section 6 which we do not address (c) Did you discuss any potential negative societal impacts of your work?[No]
Revised: [Yes] We describe some future directions in Section 6 which we do not address (c) Did you discuss any potential negative societal impacts of your work?

Original:  (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] All proofs are deferred to the supplementary material 
Revised: [Yes] All proofs are deferred to the supplementary material 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [N/A] (b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
Revised: [N/A] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [N/A] (c) Did you report error bars (e g., with respect to the random seed after running experi- ments multiple times)?
Revised: [N/A] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [N/A] (d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?
Revised: [N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [N/A] 
Revised: [N/A] 4.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [N/A] (b) Did you mention the license of the assets?
Revised: [N/A] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?[N/A]
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original:  (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?[N/A]
Revised: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: (e ) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects.. (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A
Revised: [N/A]

Original: [Table]
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [Table]
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [Table]
Revised: [N/A]