Original: R ETHINKING DEEP ACTIVE LEARNING : U SING UNLABELED DATA AT MODEL TRAINING
Revised: R ETHINKING DEEP ACTIVE LEARNING : U SING UNLABELED DATA AT MODEL TRAINING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Active learning typically focuses on training a model on few labeled examples alone, while unlabeled ones are only used for acquisition.
Revised: Active learning typically focuses on training a model on few labeled examples alone, while unlabeled ones are only used for acquisition.

Original: In this work we depart from this setting by using both labeled and unlabeled data during model training across active learning cycles.
Revised: In this work we depart from this setting by using both labeled and unlabeled data during model training across active learning cycles.

Original: We do so by using unsupervised feature learning at the beginning of the active learning pipeline and semi-supervised learning at every active learning cycle, on all available data.
Revised: We do so by using unsupervised feature learning at the beginning of the active learning pipeline and semi-supervised learning at every active learning cycle, on all available data.

Original: The former has not been investigated before in active learning, while the study of latter in the context of deep learning is scarce and recent ﬁndings are not conclusive with respect to its beneﬁt.
Revised: The former has not been investigated before in active learning, while the study of latter in the context of deep learning is scarce and recent ﬁndings are not conclusive with respect to its beneﬁt.

Original: Our idea is orthogonal to acquisition strategies by using more data, much like ensemble methods use more models.
Revised: Our idea is orthogonal to acquisition strategies by using more data, much like ensemble methods use more models.

Original: By systematically evaluating on a number of popular acquisition strategies and datasets, we ﬁnd that the use of unlabeled data during model training brings a spectacular accuracy improvement in image classiﬁcation, compared to the differences between acquisition strategies.
Revised: By systematically evaluating on a number of popular acquisition strategies and datasets, we ﬁnd that the use of unlabeled data during model training brings a spectacular accuracy improvement in image classiﬁcation, compared to the differences between acquisition strategies.

Original: We thus explore smaller label budgets, even one label per class.
Revised: We thus explore smaller label budgets, even one label per class.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Active learning (Settles, 2009) is an important pillar of machine learning but it has not been explored much in the context of deep learning until recently (Gal et al., 2017;
Revised: Active learning (Settles, 2009) is an important pillar of machine learning but it has not been explored much in the context of deep learning until recently (Gal et al., 2017;

Original: Beluch et al., 2018; Wang et al., 2017; Geifman & El-Yaniv, 2017; Sener & Savarese, 2018).
Revised: Beluch et al., 2018; Wang et al., 2017; Geifman & El-Yaniv, 2017; Sener & Savarese, 2018).

Original: The standard active learning scenario focuses on training a model on few labeled examples alone, while unlabeled data are only used for acquisition, i.e ., performing inference and selecting a subset for annotation.
Revised: The standard active learning scenario focuses on training a model on few labeled examples alone, while unlabeled data are only used for acquisition, i.e ., performing inference and selecting a subset for annotation.

Original: This is the opposite of what would normally work well when learning a deep model from scratch, i.e ., training on a lot of data with some loss function that may need labels or not.
Revised: This is the opposite of what would normally work well when learning a deep model from scratch, i.e ., training on a lot of data with some loss function that may need labels or not.

Original: At the same time, evidence is being accumulated that, when training powerful deep models, the difference in performance between acquisition strategies is small (Gissin & Shalev-Shwartz, 2018;
Revised: At the same time, evidence is being accumulated that, when training powerful deep models, the difference in performance between acquisition strategies is small (Gissin & Shalev-Shwartz, 2018;

Original: Chitta et al., 2019; Beluch et al., 2018).
Revised: Chitta et al., 2019; Beluch et al., 2018).

Original: In this work, focusing on image classiﬁcation, we revisit active deep learning with the seminal idea of using all data, whether labeled or not, during model training at each active learning cycle.
Revised: In this work, focusing on image classiﬁcation, we revisit active deep learning with the seminal idea of using all data, whether labeled or not, during model training at each active learning cycle.

Original: This departs from the standard scenario in that unlabeled data are now directly contributing to the cost function being minimized and to subsequent parameter updates, rather than just being used to perform inference for acquisition, whereby parameters are ﬁxed.
Revised: This departs from the standard scenario in that unlabeled data are now directly contributing to the cost function being minimized and to subsequent parameter updates, rather than just being used to perform inference for acquisition, whereby parameters are ﬁxed.

Original: We implement our idea using two principles: unsupervised feature learning and semi-supervised learning .
Revised: We implement our idea using two principles: unsupervised feature learning and semi-supervised learning .

Original: While both are well recognized in deep learning in general, we argue that their value has been unexplored or underestimated in the context of deep active learning.
Revised: While both are well recognized in deep learning in general, we argue that their value has been unexplored or underestimated in the context of deep active learning.

Original: Unsupervised feature learning or self-supervised learning is a very active area of research in deep learning, often taking the form of pre-training on artiﬁcial tasks with no human supervision for representation learning, followed by supervised ﬁne-tuning on different target tasks like classiﬁcation or object detection (Doersch et al., 2015; Wang & Gupta, 2015; Gidaris et al., 2018; Caron et al., 2018).
Revised: Unsupervised feature learning or self-supervised learning is a very active area of research in deep learning, often taking the form of pre-training on artiﬁcial tasks with no human supervision for representation learning, followed by supervised ﬁne-tuning on different target tasks like classiﬁcation or object detection (Doersch et al., 2015; Wang & Gupta, 2015; Gidaris et al., 2018; Caron et al., 2018).

Original: To our knowledge, all deep active learning research so far considers training deep models from scratch.
Revised: To our knowledge, all deep active learning research so far considers training deep models from scratch.

Original: In this work, we perform unsupervised feature learning on all data once at the beginning of the active learning pipeline and use the resulting parameters to initialize the model at each active learning cycle.
Revised: In this work, we perform unsupervised feature learning on all data once at the beginning of the active learning pipeline and use the resulting parameters to initialize the model at each active learning cycle.

Original: Relying on Caron et al.
Revised: Relying on Caron et al.

Original: , we show that such unsupervised pre-training improves accuracy in many cases at little additional cost.
Revised: , we show that such unsupervised pre-training improves accuracy in many cases at little additional cost.

Original: Semi-supervised learning (Chapelle et al., 2006) and active learning can be seen as two facets of the same problem: the former focuses on most certain model predictions on unlabeled examples, while the latter on least certain ones.
Revised: Semi-supervised learning (Chapelle et al., 2006) and active learning can be seen as two facets of the same problem: the former focuses on most certain model predictions on unlabeled examples, while the latter on least certain ones.

Original: Combined approaches appeared quite early (McCallum & Nigam, 1998; Zhu et al., 2003).
Revised: Combined approaches appeared quite early (McCallum & Nigam, 1998; Zhu et al., 2003).

Original: In the context of deep learning however, such combinations are scarce (Wang et al., 2017) and have even been found harmful in cases (Ducoffe & Precioso, 2018).
Revised: In the context of deep learning however, such combinations are scarce (Wang et al., 2017) and have even been found harmful in cases (Ducoffe & Precioso, 2018).

Original: It has also been argued that the two individual approaches have similar performance, while active learning has lower cost (Gal et al., 2017).
Revised: It has also been argued that the two individual approaches have similar performance, while active learning has lower cost (Gal et al., 2017).

Original: In the meantime, research on deep semi-supervised learning is very active, bringing signiﬁcant progress (Tarvainen & Valpola, 2017; Laine & Aila, 2017; Iscen et al., 2019;
Revised: In the meantime, research on deep semi-supervised learning is very active, bringing signiﬁcant progress (Tarvainen & Valpola, 2017; Laine & Aila, 2017; Iscen et al., 2019;

Original: Verma et al., 2019).
Revised: Verma et al., 2019).

Original: In this work, we use semi-supervised learning on all data at every active learning cycle, replacing supervised learning on labeled examples alone.
Revised: In this work, we use semi-supervised learning on all data at every active learning cycle, replacing supervised learning on labeled examples alone.

Original: Relying on Iscen et al.
Revised: Relying on Iscen et al.

Original: (2019), and contrary to previous ﬁndings Wang et al.
Revised: (2019), and contrary to previous ﬁndings Wang et al.

Original: ; Gal et al.
Revised: ; Gal et al.

Original: , we show that this consistently brings a dramatic accuracy improvement.
Revised: , we show that this consistently brings a dramatic accuracy improvement.

Original: Since Iscen et al.
Revised: Since Iscen et al.

Original: (2019) uses label propagation (Zhou et al., 2003a) to explore the manifold structure of the feature space, an important question is whether it is the manifold similarity or the use of unlabeled data during model training that actually helps.
Revised: (2019) uses label propagation (Zhou et al., 2003a) to explore the manifold structure of the feature space, an important question is whether it is the manifold similarity or the use of unlabeled data during model training that actually helps.

Original: We address this question by introducing a new acquisition strategy that is based on label propagation.
Revised: We address this question by introducing a new acquisition strategy that is based on label propagation.

Original: In summary, we make the following contributions:
Revised: In summary, we make the following contributions:

Original: • We systematically benchmark a number of existing acquisition strategies, as well as a new one, on a number of datasets, evaluating the beneﬁt of unsupervised pre-training and semi-supervised learning in all cases.
Revised: • We systematically benchmark a number of existing acquisition strategies, as well as a new one, on a number of datasets, evaluating the beneﬁt of unsupervised pre-training and semi-supervised learning in all cases.

Original: • Contrary to previous ﬁndings, we show that using unlabeled data during model training can yield a dramatic gain compared to differences between acquisition strategies.
Revised: • Contrary to previous ﬁndings, we show that using unlabeled data during model training can yield a dramatic gain compared to differences between acquisition strategies.

Original: • Armed with this ﬁnding, we explore a smaller budget (fewer labeled examples) than prior work, and we ﬁnd that the random baseline may actually outperform all other acquisition strategies
Revised: • Armed with this ﬁnding, we explore a smaller budget (fewer labeled examples) than prior work, and we ﬁnd that the random baseline may actually outperform all other acquisition strategies

Original: by a large margin in cases.
Revised: by a large margin in cases.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: We focus on deep active and semi-supervised learning as well as their combination.
Revised: We focus on deep active and semi-supervised learning as well as their combination.

Original: Active learning.
Revised: Active learning.

Original: Geometric methods like core sets (Geifman & El-Yaniv, 2017; Sener & Savarese, 2018) select examples based on distances in the feature space.
Revised: Geometric methods like core sets (Geifman & El-Yaniv, 2017; Sener & Savarese, 2018) select examples based on distances in the feature space.

Original: The goal is to select a subset of examples that best approximate the whole unlabeled set.
Revised: The goal is to select a subset of examples that best approximate the whole unlabeled set.

Original: We introduce a similar approach where
Revised: We introduce a similar approach where

Original: Euclidean distances are replaced by manifold ranking.
Revised: Euclidean distances are replaced by manifold ranking.

Original: There are methods inspired by adversarial learning .
Revised: There are methods inspired by adversarial learning .

Original: For instance, a binary classiﬁer can be trained to discriminate whether an example belongs to the labeled or unlabeled set (Gissin & Shalev-Shwartz, 2018; Sinha et al., 2019).
Revised: For instance, a binary classiﬁer can be trained to discriminate whether an example belongs to the labeled or unlabeled set (Gissin & Shalev-Shwartz, 2018; Sinha et al., 2019).

Original: Adversarial examples have been used, being matched to the nearest unlabeled example (Mayer & Timofte, 2018) or added to the labeled pool (Ducoffe & Precioso, 2018).
Revised: Adversarial examples have been used, being matched to the nearest unlabeled example (Mayer & Timofte, 2018) or added to the labeled pool (Ducoffe & Precioso, 2018).

Original: It has been observed however that deep networks can perform similarly regardless of the acquisition function (Gissin & Shalev-Shwartz, 2018;
Revised: It has been observed however that deep networks can perform similarly regardless of the acquisition function (Gissin & Shalev-Shwartz, 2018;

Original: Chitta et al., 2019), which we further investigate here.
Revised: Chitta et al., 2019), which we further investigate here.

Original: Ensemble and Bayesian methods (Gal et al., 2017; Beluch et al., 2018;
Revised: Ensemble and Bayesian methods (Gal et al., 2017; Beluch et al., 2018;

Original: Chitta et al., 2019) target representing model uncertainty, which than can be used by different acquisition functions.
Revised: Chitta et al., 2019) target representing model uncertainty, which than can be used by different acquisition functions.

Original: This idea is orthogonal to acquisition strategies.
Revised: This idea is orthogonal to acquisition strategies.

Original: In fact, Beluch et al.
Revised: In fact, Beluch et al.

Original: ; Chitta et al.
Revised: ; Chitta et al.

Original: (2019) show that the gain of ensemble models is more pronounced than the gain of any acquisition strategy.
Revised: (2019) show that the gain of ensemble models is more pronounced than the gain of any acquisition strategy.

Original: Of course, ensemble and Bayesian methods are more expensive than single models.
Revised: Of course, ensemble and Bayesian methods are more expensive than single models.

Original: Approximations include for instance a single model producing different outputs by dropout (Gal et al., 2017).
Revised: Approximations include for instance a single model producing different outputs by dropout (Gal et al., 2017).

Original: Our idea of using all data during model training is also orthogonal to acquisition strategies.
Revised: Our idea of using all data during model training is also orthogonal to acquisition strategies.

Original: It is also more expensive than using labeled data alone, but the gain is spectacular in this case.
Revised: It is also more expensive than using labeled data alone, but the gain is spectacular in this case.

Original: This allows the use of much smaller label budget for the same accuracy, which is the essence of active learning.
Revised: This allows the use of much smaller label budget for the same accuracy, which is the essence of active learning.

Original: Semi-supervised active learning has a long history (McCallum & Nigam, 1998; Muslea et al., 2002;
Revised: Semi-supervised active learning has a long history (McCallum & Nigam, 1998; Muslea et al., 2002;

Original: Zhu et al., 2003;
Revised: Zhu et al., 2003;

Original: Zhou et al., 2004; Long et al., 2008).
Revised: Zhou et al., 2004; Long et al., 2008).

Original: A recent deep learning approach acquires the least certain unlabeled examples for labeling and at the same time assigns predicted pseudo-labels to most certain examples (Wang et al., 2017).
Revised: A recent deep learning approach acquires the least certain unlabeled examples for labeling and at the same time assigns predicted pseudo-labels to most certain examples (Wang et al., 2017).

Original: This does not always help (Ducoffe & Precioso, 2018).
Revised: This does not always help (Ducoffe & Precioso, 2018).

Original: In some cases, semi-supervised algorithms are incorporated as part of an active learning evaluation (Li et al., 2019; Sener & Savarese, 2018).
Revised: In some cases, semi-supervised algorithms are incorporated as part of an active learning evaluation (Li et al., 2019; Sener & Savarese, 2018).

Original: A comparative study suggests that semi-supervised learning does not signiﬁcantly improve over active learning, despite its additional cost due to training on more data (Gal et al., 2017).
Revised: A comparative study suggests that semi-supervised learning does not signiﬁcantly improve over active learning, despite its additional cost due to training on more data (Gal et al., 2017).

Original: We show that this is clearly not the case, using a state of the art semi-supervised method (Iscen et al., 2019) that is an inductive version of label propagation (Zhou et al., 2003a).
Revised: We show that this is clearly not the case, using a state of the art semi-supervised method (Iscen et al., 2019) that is an inductive version of label propagation (Zhou et al., 2003a).

Original: This is related to Zhu & Ghahramani (2002); Zhu et al.
Revised: This is related to Zhu & Ghahramani (2002); Zhu et al.

Original: Long et al.
Revised: Long et al.

Original: (2008), which however are limited to transductive learning.
Revised: (2008), which however are limited to transductive learning.

Original: Unsupervised feature learning.
Revised: Unsupervised feature learning.

Original: A number of unsupervised feature learning approaches pair matching images to learn the representation using a siamese architecture.
Revised: A number of unsupervised feature learning approaches pair matching images to learn the representation using a siamese architecture.

Original: These pairs can come as fragments of the same image (Doersch et al., 2015; Noroozi & Favaro, 2016) or as a result of tracking in video (Wang & Gupta, 2015).
Revised: These pairs can come as fragments of the same image (Doersch et al., 2015; Noroozi & Favaro, 2016) or as a result of tracking in video (Wang & Gupta, 2015).

Original: Alternatively, the network is trained on an artiﬁcial task like image rotation prediction (Gidaris et al., 2018) or even matching images to a noisy target (Bojanowski &
Revised: Alternatively, the network is trained on an artiﬁcial task like image rotation prediction (Gidaris et al., 2018) or even matching images to a noisy target (Bojanowski &

Original: Joulin, 2017).
Revised: Joulin, 2017).

Original: The latter is conceptually related to deep clustering (Caron et al., 2018), the approach we use in this work, where the network learns targets resulting from unsupervised clustering.
Revised: The latter is conceptually related to deep clustering (Caron et al., 2018), the approach we use in this work, where the network learns targets resulting from unsupervised clustering.

Original: It is interesting that in the context of semi-supervised learning, unsupervised pre-training has been recently investigated by Rebufﬁ et al.
Revised: It is interesting that in the context of semi-supervised learning, unsupervised pre-training has been recently investigated by Rebufﬁ et al.

Original: (2019), with results are consistent with ours.
Revised: (2019), with results are consistent with ours.

Original: However, the use of unsupervised pre-training in deep active learning remains unexplored.
Revised: However, the use of unsupervised pre-training in deep active learning remains unexplored.

Original:  P ROBLEM FORMULATION AND BACKGROUND
Revised:  P ROBLEM FORMULATION AND BACKGROUND

Original: Problem.
Revised: Problem.

Original: We are given a set X :=
Revised: We are given a set X :=

Original: { x i } i ∈I ⊂
Revised: { x i } i ∈I ⊂

Original: X of n examples where I :=
Revised: X of n examples where I :=

Original: [ n ] := { 1 , . .
Revised: [ n ] := { 1 , . .

Original: . , n } and, initially, a collection y 0 :
Revised: . , n } and, initially, a collection y 0 :

Original: = ( y i ) i
Revised: = ( y i ) i

Original: ∈
Revised: ∈

Original: L 0 of b labels y i ∈ C for i ∈ L 0 , where C :=
Revised: L 0 of b labels y i ∈ C for i ∈ L 0 , where C :=

Original: [ c ] is a set of c classes and L 

Revised: [ c ] is a set of c classes and L

Original: ⊂
Revised: ⊂

Original: I a set of indices with | L 0 | =
Revised: I a set of indices with | L 0 | =

Original: b (cid:28) n .
Revised: b (cid:28) n .

Original: The goal of active learning (AL) (Settles, 2009) is to train a classiﬁer in cycles , where in cycle j = 0 , 1 , . . .
Revised: The goal of active learning (AL) (Settles, 2009) is to train a classiﬁer in cycles , where in cycle j = 0 , 1 , . . .

Original: we use a collection y j of labels for training, and then we acquire (or sample ) a new batch S j of indices with | S j | =
Revised: we use a collection y j of labels for training, and then we acquire (or sample ) a new batch S j of indices with | S j | =

Original: b to label the corresponding examples for the next cycle j + 1 .
Revised: b to label the corresponding examples for the next cycle j + 1 .

Original: Let L j :=
Revised: Let L j :=

Original: L j − 1 ∪ S j − 1 ⊂ I be the set of indices of labeled examples in cycle j ≥ 1 and U j :=
Revised: L j − 1 ∪ S j − 1 ⊂ I be the set of indices of labeled examples in cycle j ≥ 1 and U j :=

Original: I \
Revised: I \

Original: L j the indices of the unlabeled examples for j ≥ 0 .
Revised: L j the indices of the unlabeled examples for j ≥ 0 .

Original: Then y j :=
Revised: Then y j :=

Original: ( y i ) i ∈
Revised: ( y i ) i ∈

Original: L j are the labels in cycle j and S j ⊂ U j is selected from the unlabeled examples.
Revised: L j are the labels in cycle j and S j ⊂ U j is selected from the unlabeled examples.

Original: To keep notation simple, we will refer to a single cycle in the following, dropping subscripts j .
Revised: To keep notation simple, we will refer to a single cycle in the following, dropping subscripts j .

Original: Classiﬁer learning.
Revised: Classiﬁer learning.

Original: The classiﬁer f θ : X → R c with parameters θ , maps new examples to a vector of probabilities per class.
Revised: The classiﬁer f θ : X → R c with parameters θ , maps new examples to a vector of probabilities per class.

Original: Given x ∈ X , its prediction is the class of maximum probability
Revised: Given x ∈ X , its prediction is the class of maximum probability

Original: [Equation]
Revised: [Equation]

Original: where p k is the k -th element of vector p :
Revised: where p k is the k -th element of vector p :

Original: = f θ ( x ) .
Revised: = f θ ( x ) .

Original: As a by-product of learning parameters θ , we have access to an embedding function φ θ : X → R d , mapping an example x ∈ X to a feature vector φ θ ( x ) .
Revised: As a by-product of learning parameters θ , we have access to an embedding function φ θ : X → R d , mapping an example x ∈ X to a feature vector φ θ ( x ) .

Original: For instance, f θ may be a linear classiﬁer on top of features obtained by φ θ .
Revised: For instance, f θ may be a linear classiﬁer on top of features obtained by φ θ .

Original: In a typical AL scenario, given a set of indices L of labeled examples and labels y , the parameters θ of the classiﬁer are learned by minimizing the cost function
Revised: In a typical AL scenario, given a set of indices L of labeled examples and labels y , the parameters θ of the classiﬁer are learned by minimizing the cost function

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: on labeled examples
Revised: on labeled examples

Original: x i for i ∈ L , where cross-entropy (cid:96) ( p , y ) :
Revised: x i for i ∈ L , where cross-entropy (cid:96) ( p , y ) :

Original: = − log p y for p ∈ R c + , y ∈ C .
Revised: = − log p y for p ∈ R c + , y ∈ C .

Original: Acquisition.
Revised: Acquisition.

Original: Given the set of indices U of unlabeled examples and the parameters θ resulting from training, one typically acquires a new batch by initializing S ← ∅ and then greedily updating by
Revised: Given the set of indices U of unlabeled examples and the parameters θ resulting from training, one typically acquires a new batch by initializing S ← ∅ and then greedily updating by

Original: [Equation]
Revised: [Equation]

Original: until | S | ≥ b .
Revised: until | S | ≥ b .

Original: Here a is an acquisition (or sampling ) function, each time selecting one example from
Revised: Here a is an acquisition (or sampling ) function, each time selecting one example from

Original: U \ S .
Revised: U \ S .

Original: For each i ∈ S , the corresponding example x i is then given as query to an oracle (often a human expert), who returns a label y i to be used in the next cycle.
Revised: For each i ∈ S , the corresponding example x i is then given as query to an oracle (often a human expert), who returns a label y i to be used in the next cycle.

Original: Geometry.
Revised: Geometry.

Original: Given parameters θ , a simple acquisition strategy is to use the geometry of examples in the feature space F θ :=
Revised: Given parameters θ , a simple acquisition strategy is to use the geometry of examples in the feature space F θ :=

Original: φ θ ( X ) , without considering the classiﬁer.
Revised: φ θ ( X ) , without considering the classiﬁer.

Original: Each example x i is represented by the feature vector φ θ ( x i ) for i ∈
Revised: Each example x i is represented by the feature vector φ θ ( x i ) for i ∈

Original: I .
Revised: I .

Original: One particular example is the function (Geifman & El-Yaniv, 2017; Sener & Savarese, 2018)
Revised: One particular example is the function (Geifman & El-Yaniv, 2017; Sener & Savarese, 2018)

Original: [Equation]
Revised: [Equation]

Original: each time selecting the unlabeled example in U that is the most distant to its nearest labeled or previously acquired example in L .
Revised: each time selecting the unlabeled example in U that is the most distant to its nearest labeled or previously acquired example in L .

Original: Such geometric approaches are inherently related to clustering .
Revised: Such geometric approaches are inherently related to clustering .

Original: For instance, k -means++ (Arthur & Vassilvitskii, 2007) is a probabilistic version of (4).
Revised: For instance, k -means++ (Arthur & Vassilvitskii, 2007) is a probabilistic version of (4).

Original: Uncertainty.
Revised: Uncertainty.

Original: A common acquisition strategy that considers the classiﬁer is some measure of uncertainty in its prediction.
Revised: A common acquisition strategy that considers the classiﬁer is some measure of uncertainty in its prediction.

Original: Given a vector of probabilities p , one such measure is the entropy
Revised: Given a vector of probabilities p , one such measure is the entropy

Original: [Equation]
Revised: [Equation]

Original: taking values in [0 , log c ] .
Revised: taking values in [0 , log c ] .

Original: Given parameters θ , each example x i is represented by the vector of probabilities f θ ( x i ) for i ∈
Revised: Given parameters θ , each example x i is represented by the vector of probabilities f θ ( x i ) for i ∈

Original: I .
Revised: I .

Original: Then, acquisition is deﬁned by
Revised: Then, acquisition is deﬁned by

Original: [Equation]
Revised: [Equation]

Original: effectively selecting the b most uncertain unlabeled examples for labeling.
Revised: effectively selecting the b most uncertain unlabeled examples for labeling.

Original: Pseudo-labels.
Revised: Pseudo-labels.

Original: It is possible to use more data than the labeled examples while learning.
Revised: It is possible to use more data than the labeled examples while learning.

Original: In Wang et al.
Revised: In Wang et al.

Original: (2017) for example, given indices L , U of labeled and unlabeled examples respectively and parameters θ , one represents example x i by p i :
Revised: (2017) for example, given indices L , U of labeled and unlabeled examples respectively and parameters θ , one represents example x i by p i :

Original: = f θ ( x i ) , selects the most certain unlabeled examples
Revised: = f θ ( x i ) , selects the most certain unlabeled examples

Original: [Equation]
Revised: [Equation]

Original: and assigns pseudo-label ˆ y i :
Revised: and assigns pseudo-label ˆ y i :

Original: =
Revised: =

Original: π ( p i ) by (1) for i ∈
Revised: π ( p i ) by (1) for i ∈

Original: ˆ L .
Revised: ˆ L .

Original: The same cost function J deﬁned by (2) can now be used by augmenting L to L ∪ ˆ L and y to ( y , ˆ y ) , where ˆ y :=
Revised: The same cost function J deﬁned by (2) can now be used by augmenting L to L ∪ ˆ L and y to ( y , ˆ y ) , where ˆ y :=

Original: (ˆ y i )
Revised: (ˆ y i )

Original: i ∈
Revised: i ∈

Original: ˆ L .
Revised: ˆ L .

Original: This augmentation occurs once per cycle in Wang et al.
Revised: This augmentation occurs once per cycle in Wang et al.

Original: This is an example of active semi-supervised learning.
Revised: This is an example of active semi-supervised learning.

Original: Transductive label propagation (Zhou et al., 2003a) refers to graph-based, semi-supervised learning.
Revised: Transductive label propagation (Zhou et al., 2003a) refers to graph-based, semi-supervised learning.

Original: A nearest neighbor graph of the dataset X is used, represented by a symmetric non-negative n × n adjacency matrix W with zero diagonal.
Revised: A nearest neighbor graph of the dataset X is used, represented by a symmetric non-negative n × n adjacency matrix W with zero diagonal.

Original: This matrix is symmetrically normalized as W :=
Revised: This matrix is symmetrically normalized as W :=

Original: D − 1 / 2 WD − 1 / 2 , where D :=
Revised: D − 1 / 2 WD − 1 / 2 , where D :=

Original: diag( W 1 ) is the degree matrix and 1 is the all-ones vector.
Revised: diag( W 1 ) is the degree matrix and 1 is the all-ones vector.

Original: The given labels y :
Revised: The given labels y :

Original: = ( y i )
Revised: = ( y i )

Original: i ∈ L are represented by a n × c zero-one matrix Y :
Revised: i ∈ L are represented by a n × c zero-one matrix Y :

Original: = χ ( L, y ) where row i is a c -vector that is a one-hot encoding of label
Revised: = χ ( L, y ) where row i is a c -vector that is a one-hot encoding of label

Original: y i if example x i is labeled and zero otherwise,
Revised: y i if example x i is labeled and zero otherwise,

Original: [Equation]
Revised: [Equation]

Original: otherwise
Revised: otherwise

Original: [Equation]
Revised: [Equation]

Original: 8) for i ∈
Revised: 8) for i ∈

Original: I and k ∈ C .
Revised: I and k ∈ C .

Original: Zhou et al.
Revised: Zhou et al.

Original: (2003a) deﬁne the n × c matrix P :=
Revised: (2003a) deﬁne the n × c matrix P :=

Original: η [ h ( Y )] 1 , where
Revised: η [ h ( Y )] 1 , where

Original: [Equation]
Revised: [Equation]

Original: I is the n × n identity matrix, and α ∈ [0 , 1) is a parameter.
Revised: I is the n × n identity matrix, and α ∈ [0 , 1) is a parameter.

Original: The i -th row p i of P represents a vector of class probabilities of unlabeled example x i , and a prediction can be made by π ( p i ) (1) for i ∈ U .
Revised: The i -th row p i of P represents a vector of class probabilities of unlabeled example x i , and a prediction can be made by π ( p i ) (1) for i ∈ U .

Original: This method is transductive because it cannot make predictions on previously unseen data without access to the original data X .
Revised: This method is transductive because it cannot make predictions on previously unseen data without access to the original data X .

Original: Inductive label propagation.
Revised: Inductive label propagation.

Original: Although the previous methods do not apply to unseen data by themselves, the predictions made on X can again be used as pseudo-labels to train a classiﬁer.
Revised: Although the previous methods do not apply to unseen data by themselves, the predictions made on X can again be used as pseudo-labels to train a classiﬁer.

Original: This is done in Iscen et al.
Revised: This is done in Iscen et al.

Original: (2019), applied to semi-supervised learning.
Revised: (2019), applied to semi-supervised learning.

Original: Like Wang et al.
Revised: Like Wang et al.

Original: (2017), a pseudo-label is generated for unlabeled example x i as ˆ y i :
Revised: (2017), a pseudo-label is generated for unlabeled example x i as ˆ y i :

Original: = π ( p i ) by (1), only now p i is the i -th row of the result P of label propagation according to (9) rather than the classiﬁer output f θ ( x i ) .
Revised: = π ( p i ) by (1), only now p i is the i -th row of the result P of label propagation according to (9) rather than the classiﬁer output f θ ( x i ) .

Original: Unlike Wang et al.
Revised: Unlike Wang et al.

Original: (2017), all unlabeled examples are pseudo-labeled and an additional cost term J w ( X, U, ˆ y ; θ ) :
Revised: (2017), all unlabeled examples are pseudo-labeled and an additional cost term J w ( X, U, ˆ y ; θ ) :

Original: = (cid:80) i ∈ U w i (cid:96) ( f θ ( x i ) , ˆ y i ) applies to those examples, where ˆ y :=
Revised: = (cid:80) i ∈ U w i (cid:96) ( f θ ( x i ) , ˆ y i ) applies to those examples, where ˆ y :=

Original: (ˆ y i ) i ∈ U and w i :=
Revised: (ˆ y i ) i ∈ U and w i :=

Original: β ( p i ) is a weight reﬂecting the certainty in the prediction of ˆ y i :
Revised: β ( p i ) is a weight reﬂecting the certainty in the prediction of ˆ y i :

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Unlike Wang et al.
Revised: Unlike Wang et al.

Original: (2017), the graph and the pseudo-labels are updated once per epoch during learning in Iscen et al.
Revised: (2017), the graph and the pseudo-labels are updated once per epoch during learning in Iscen et al.

Original: (2019), where there are no cycles.
Revised: (2019), where there are no cycles.

Original: Unsupervised feature learning.
Revised: Unsupervised feature learning.

Original: Finally, it is possible to train an embedding function in an unsupervised fashion.
Revised: Finally, it is possible to train an embedding function in an unsupervised fashion.

Original: A simple method that does not make any assumption on the nature or structure of the data is Caron et al.
Revised: A simple method that does not make any assumption on the nature or structure of the data is Caron et al.

Original: Simply put, starting by randomly initialized parameters θ , the data φ θ ( X ) are clustered by k -means, each example is assigned to the nearest centroid, clusters and assignments are treated as classes C and pseudo-labels ˆ y respectively, and learning takes place according to
Revised: Simply put, starting by randomly initialized parameters θ , the data φ θ ( X ) are clustered by k -means, each example is assigned to the nearest centroid, clusters and assignments are treated as classes C and pseudo-labels ˆ y respectively, and learning takes place according to

Original: J ( X, I , ˆ y , θ ) (2).
Revised: J ( X, I , ˆ y , θ ) (2).

Original: By updating the parameters θ , φ θ ( X ) is updated too.
Revised: By updating the parameters θ , φ θ ( X ) is updated too.

Original: The method therefore alternates between clustering/pseudo-labeling and feature learning, typically once per epoch.
Revised: The method therefore alternates between clustering/pseudo-labeling and feature learning, typically once per epoch.

Original:  T RAINING THE MODEL ON UNLABELED DATA
Revised:  T RAINING THE MODEL ON UNLABELED DATA

Original: We argue that acquiring examples for labeling is not making the best use of unlabeled data: unlabeled data should be used during model training, appearing in the cost function that is being minimized.
Revised: We argue that acquiring examples for labeling is not making the best use of unlabeled data: unlabeled data should be used during model training, appearing in the cost function that is being minimized.

Original: We choose two ways of doing so: unsupervised feature learning and semi-supervised learning.
Revised: We choose two ways of doing so: unsupervised feature learning and semi-supervised learning.

Original: As outlined in Algorithm 1, we follow the standard active learning setup, adding unsupervised pretraining at the beginning and replacing supervised learning on L by semi-supervised learning on
Revised: As outlined in Algorithm 1, we follow the standard active learning setup, adding unsupervised pretraining at the beginning and replacing supervised learning on L by semi-supervised learning on

Original: L ∪ U at each cycle.
Revised: L ∪ U at each cycle.

Original: The individual components are discussed in more detail below.
Revised: The individual components are discussed in more detail below.

Original: Algorithm 1: Semi-supervised active learning
Revised: Algorithm 1: Semi-supervised active learning

Original: Data: data X , indices of labeled examples L , labels y , batch size b 1 U ← I \ L (cid:46) indices of unlabeled examples 2 θ 0 ← PRE ( X ) (cid:46) unsupervised
Revised: Data: data X , indices of labeled examples L , labels y , batch size b 1 U ← I \ L (cid:46) indices of unlabeled examples 2 θ 0 ← PRE ( X ) (cid:46) unsupervised

Original: pre-training 3 for j ∈ { 0 , . . .
Revised: pre-training 3 for j ∈ { 0 , . . .

Original: } do (cid:46) active learning cycles θ ← SUP ( X, L, y ; θ 0 )
Revised: } do (cid:46) active learning cycles θ ← SUP ( X, L, y ; θ 0 )

Original: (cid:46) supervised learning on L only for e ∈ { 1 , . . .
Revised: (cid:46) supervised learning on L only for e ∈ { 1 , . . .

Original: } do (cid:46) epochs (ˆ y , w ) ← LP (
Revised: } do (cid:46) epochs (ˆ y , w ) ← LP (

Original: L y
Revised: L y

Original: ) (cid:46) pseudo-labels ˆ y and labels w θ ← SEMI ( X, L ∪ U, ( y , ˆ y ) , w ; θ ) all
Revised: ) (cid:46) pseudo-labels ˆ y and labels w θ ← SEMI ( X, L ∪ U, ( y , ˆ y ) , w ; θ ) all

Original: S ← ∅ while | S | b do
Revised: S ← ∅ while | S | b do

Original: S ← S ∪ a
Revised: S ← S ∪ a

Original: y ← y
Revised: y ← y

Original: L ← L
Revised: L ← L

Original: S
Revised: S

Original: U ← U \ S
Revised: U ← U \ S

Original: Unsupervised pre-training ( PRE ) takes place at the beginning of the algorithm.
Revised: Unsupervised pre-training ( PRE ) takes place at the beginning of the algorithm.

Original: We follow Caron et al.
Revised: We follow Caron et al.

Original: (2018), randomly initializing θ and then alternating between clustering the features φ θ ( X ) by k -means and learning on cluster assignment pseudo-labels ˆ y of X according to J ( X, I , ˆ y , θ ) (2).
Revised: (2018), randomly initializing θ and then alternating between clustering the features φ θ ( X ) by k -means and learning on cluster assignment pseudo-labels ˆ y of X according to J ( X, I , ˆ y , θ ) (2).

Original: The result is a set of parameters θ 0 used to initialize the classiﬁer at every cycle.
Revised: The result is a set of parameters θ 0 used to initialize the classiﬁer at every cycle.

Original: Learning per cycle follows inductive label propagation (Iscen et al., 2019).
Revised: Learning per cycle follows inductive label propagation (Iscen et al., 2019).

Original: This consists of supervised learning followed by alternating label propagation and semi-supervised learning on all examples
Revised: This consists of supervised learning followed by alternating label propagation and semi-supervised learning on all examples

Original: L ∪ U at every epoch.
Revised: L ∪ U at every epoch.

Original: The supervised learning ( SUP ) is performed on the labeled examples L only using labels y , according to J ( X, L, y , θ ) (2), where the parameters θ are initialized by θ 0 .
Revised: The supervised learning ( SUP ) is performed on the labeled examples L only using labels y , according to J ( X, L, y , θ ) (2), where the parameters θ are initialized by θ 0 .

Original: Label propagation ( LP ) involves a reciprocal k -nearest neighbor graph on features φ θ ( X ) (Iscen et al., 2019).
Revised: Label propagation ( LP ) involves a reciprocal k -nearest neighbor graph on features φ θ ( X ) (Iscen et al., 2019).

Original: As in Zhou et al.
Revised: As in Zhou et al.

Original: (2003a), the resulting afﬁnity matrix W is normalized as W :=
Revised: (2003a), the resulting afﬁnity matrix W is normalized as W :=

Original: D − 1 / 2 WD − 1 / 2 .
Revised: D − 1 / 2 WD − 1 / 2 .

Original: Label propagation is then performed according to P =
Revised: Label propagation is then performed according to P =

Original: η [ h ( Y )] (9), by solving the corresponding linear system using the conjugate gradient (CG) method (Iscen et al., 2019).
Revised: η [ h ( Y )] (9), by solving the corresponding linear system using the conjugate gradient (CG) method (Iscen et al., 2019).

Original: The label matrix Y :
Revised: The label matrix Y :

Original: = χ ( L, y )
Revised: = χ ( L, y )

Original: (8) is deﬁned on the true labeled examples L that remain ﬁxed over epochs but grow over cycles.
Revised: (8) is deﬁned on the true labeled examples L that remain ﬁxed over epochs but grow over cycles.

Original: With p i being the i -th row of P , a pseudo-label ˆ y i =
Revised: With p i being the i -th row of P , a pseudo-label ˆ y i =

Original: π ( p i ) (1) and a weight w i =
Revised: π ( p i ) (1) and a weight w i =

Original: β ( p i ) (10) are deﬁned for every i ∈ U
Revised: β ( p i ) (10) are deﬁned for every i ∈ U

Original: (Iscen et al., 2019).
Revised: (Iscen et al., 2019).

Original: Semi-supervised learning ( SEMI ) takes place on all examples L ∪ U
Revised: Semi-supervised learning ( SEMI ) takes place on all examples L ∪ U

Original: =
Revised: =

Original: I , where examples in L have true labels y and examples in U pseudo-labels ˆ y :=
Revised: I , where examples in L have true labels y and examples in U pseudo-labels ˆ y :=

Original: (ˆ y i ) i ∈ U .
Revised: (ˆ y i ) i ∈ U .

Original: Different than Iscen et al.
Revised: Different than Iscen et al.

Original: (2019), we minimize the standard cost function J ( X, L ∪ U, ( y , ˆ y ) , θ ) (2), but we do take weights w :=
Revised: (2019), we minimize the standard cost function J ( X, L ∪ U, ( y , ˆ y ) , θ ) (2), but we do take weights w :=

Original: ( w i ) i ∈ U into account in mini-batch sampling, (cid:96) 1 -normalized as η [ w ] .
Revised: ( w i ) i ∈ U into account in mini-batch sampling, (cid:96) 1 -normalized as η [ w ] .

Original: In particular, part of each mini-batch is drawn uniformly at random from L , while the other part is drawn with replacement from the discrete distribution η [ w ] on U : an example may be drawn more than once per epoch or never.
Revised: In particular, part of each mini-batch is drawn uniformly at random from L , while the other part is drawn with replacement from the discrete distribution η [ w ] on U : an example may be drawn more than once per epoch or never.

Original: [Table]
Revised: [Table]

Original: Discussion.
Revised: Discussion.

Original: The above probabilistic weighting decouples the size of the epoch from n and indeed we experiment with epochs smaller than n , accelerating learning compared to Iscen et al.
Revised: The above probabilistic weighting decouples the size of the epoch from n and indeed we experiment with epochs smaller than n , accelerating learning compared to Iscen et al.

Original: It is similar to importance sampling , which is typically based on loss values (Katharopoulos &
Revised: It is similar to importance sampling , which is typically based on loss values (Katharopoulos &

Original: Fleuret, 2017; Cheng et al., 2018) or predicted class probabilities (Yang et al., 2015).
Revised: Fleuret, 2017; Cheng et al., 2018) or predicted class probabilities (Yang et al., 2015).

Original: Acceleration is important as training on all examples is more expensive than just the labeled ones, and is repeated at every cycle.
Revised: Acceleration is important as training on all examples is more expensive than just the labeled ones, and is repeated at every cycle.

Original: On the contrary, unsupervised pre-trained only occurs once at the beginning.
Revised: On the contrary, unsupervised pre-trained only occurs once at the beginning.

Original: The particular choice of components is not important: any unsupervised representation learning could replace Caron et al.
Revised: The particular choice of components is not important: any unsupervised representation learning could replace Caron et al.

Original: (2018) in line 2 and any semi-supervised learning could replace Iscen et al.
Revised: (2018) in line 2 and any semi-supervised learning could replace Iscen et al.

Original: (2019) in lines 4-7 of Algorithm 1.
Revised: (2019) in lines 4-7 of Algorithm 1.

Original: We keep the pipeline as simple as possible, facilitating comparisons with more effective choices in the future.
Revised: We keep the pipeline as simple as possible, facilitating comparisons with more effective choices in the future.

Original:  I NVESTIGATING MANIFOLD SIMILARITY IN THE ACQUISITION FUNCTION
Revised:  I NVESTIGATING MANIFOLD SIMILARITY IN THE ACQUISITION FUNCTION

Original: Label propagation (Zhou et al., 2003a; Iscen et al., 2019) is based on the manifold structure of the feature space, as captured by the normalized afﬁnity matrix W .
Revised: Label propagation (Zhou et al., 2003a; Iscen et al., 2019) is based on the manifold structure of the feature space, as captured by the normalized afﬁnity matrix W .

Original: Rather than just using this information for propagating labels to unlabeled examples, can we use it in the acquisition function as well?
Revised: Rather than just using this information for propagating labels to unlabeled examples, can we use it in the acquisition function as well?

Original: This is important in interpreting the effect of semi-supervised learning in Algorithm 1: is any gain due to the use of manifold similarity, or to training the model on more data?
Revised: This is important in interpreting the effect of semi-supervised learning in Algorithm 1: is any gain due to the use of manifold similarity, or to training the model on more data?

Original: Joint label propagation (jLP), introduced here, is an attempt to answer these questions.
Revised: Joint label propagation (jLP), introduced here, is an attempt to answer these questions.

Original: It is an acquisition function similar in nature to the geometric approach (4), with Euclidean distance replaced by manifold similarity.
Revised: It is an acquisition function similar in nature to the geometric approach (4), with Euclidean distance replaced by manifold similarity.

Original: In particular, the n -vector Y 1 c , the row-wise sum of Y =
Revised: In particular, the n -vector Y 1 c , the row-wise sum of Y =

Original: χ ( L, y )
Revised: χ ( L, y )

Original: (8), can be expressed as Y 1 c = δ ( L ) ∈ R n , where δ (
Revised: (8), can be expressed as Y 1 c = δ ( L ) ∈ R n , where δ (

Original: [Equation]
Revised: [Equation]

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: otherwise (11) for i ∈
Revised: otherwise (11) for i ∈

Original: I .
Revised: I .

Original: Hence, in the terminology of manifold ranking (Zhou et al., 2003b), vector Y 1 c represents a set of queries , one for each example x i for i ∈ L , and the i -th element of the n -vector h ( Y ) 1 c in (9) expresses the manifold similarity of x i to the queries for i ∈
Revised: Hence, in the terminology of manifold ranking (Zhou et al., 2003b), vector Y 1 c represents a set of queries , one for each example x i for i ∈ L , and the i -th element of the n -vector h ( Y ) 1 c in (9) expresses the manifold similarity of x i to the queries for i ∈

Original: I .
Revised: I .

Original: Similar to (4), we acquire the example in U that is the least similar to examples in L that are labeled or previously acquired:
Revised: Similar to (4), we acquire the example in U that is the least similar to examples in L that are labeled or previously acquired:

Original: [Equation]
Revised: [Equation]

Original: This strategy is only geometric and bears similarities to discriminative active learning (Gissin &
Revised: This strategy is only geometric and bears similarities to discriminative active learning (Gissin &

Original: Shalev-Shwartz, 2018), which learns a binary classiﬁer to discriminate labeled from unlabeled examples and acquires examples of least conﬁdence in the “labeled” class.
Revised: Shalev-Shwartz, 2018), which learns a binary classiﬁer to discriminate labeled from unlabeled examples and acquires examples of least conﬁdence in the “labeled” class.

Original:  E XPERIMENTS 6. E XPERIMENTAL SETUP
Revised:  E XPERIMENTS 6. E XPERIMENTAL SETUP

Original: Datasets.
Revised: Datasets.

Original: We conduct experiments on four datasets that are most often used in deep active learning:
Revised: We conduct experiments on four datasets that are most often used in deep active learning:

Original: MNIST (LeCun et al., 1998), SVHN (Netzer et al., 2011), CIFAR-10 and CIFAR-100 (Krizhevsky, 2009).
Revised: MNIST (LeCun et al., 1998), SVHN (Netzer et al., 2011), CIFAR-10 and CIFAR-100 (Krizhevsky, 2009).

Original: Table 1 presents statistics of the datasets.
Revised: Table 1 presents statistics of the datasets.

Original: Following Tarvainen & Valpola (2017); Iscen et al.
Revised: Following Tarvainen & Valpola (2017); Iscen et al.

Original: (2019), we augment input images by 4 × 4 random translations and random horizontal ﬂips.
Revised: (2019), we augment input images by 4 × 4 random translations and random horizontal ﬂips.

Original: Networks and training.
Revised: Networks and training.

Original: For all experiments we use a 13-layer convolutional network used previously in Laine & Aila (2016).
Revised: For all experiments we use a 13-layer convolutional network used previously in Laine & Aila (2016).

Original: We train the model from scratch at each active learning cycle, using
Revised: We train the model from scratch at each active learning cycle, using

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: SGD with momentum of 0.9 for 200 epochs.
Revised: SGD with momentum of 0.9 for 200 epochs.

Original: An initial learning rate of 0 . 2 is decayed by cosine annealing (Loshchilov & Hutter, 2017), scheduled to reach zero at 210 epochs.
Revised: An initial learning rate of 0 . 2 is decayed by cosine annealing (Loshchilov & Hutter, 2017), scheduled to reach zero at 210 epochs.

Original: The mini-batch size is 32 for standard training and 128 when SEMI is used, except for MNIST where the size of the mini-batch 10 and 64 with SEMI .
Revised: The mini-batch size is 32 for standard training and 128 when SEMI is used, except for MNIST where the size of the mini-batch 10 and 64 with SEMI .

Original: All other parameters follow Tarvainen & Valpola (2017).
Revised: All other parameters follow Tarvainen & Valpola (2017).

Original: Unsupervised pre-training.
Revised: Unsupervised pre-training.

Original: We use k -means as the clustering algorithm and follow the settings of Caron et al.
Revised: We use k -means as the clustering algorithm and follow the settings of Caron et al.

Original: The model is trained for 250 epochs on the respective datasets.
Revised: The model is trained for 250 epochs on the respective datasets.

Original: Semi-supervised learning.
Revised: Semi-supervised learning.

Original: Following Iscen et al.
Revised: Following Iscen et al.

Original: (2019), we construct a reciprocal k -nearest neighbor graph on features φ θ ( X ) , with k = 50 neighbors and similarity function s ( u , v ) :=
Revised: (2019), we construct a reciprocal k -nearest neighbor graph on features φ θ ( X ) , with k = 50 neighbors and similarity function s ( u , v ) :=

Original: [ˆ u (cid:62) ˆ v ] 3+ for u , v ∈ R d , where ˆ u is the (cid:96) 2 -normalized counterpart of u , while α = 0 .
Revised: [ˆ u (cid:62) ˆ v ] 3+ for u , v ∈ R d , where ˆ u is the (cid:96) 2 -normalized counterpart of u , while α = 0 .

Original: 99 in (9).
Revised: 99 in (9).

Original: We follow
Revised: We follow

Original: Iscen et al.
Revised: Iscen et al.

Original: (2019) in splitting mini-batches into two parts: 50 examples (10 for MNIST) are labeled and the remaining pseudo-labeled.
Revised: (2019) in splitting mini-batches into two parts: 50 examples (10 for MNIST) are labeled and the remaining pseudo-labeled.

Original: For the latter, we draw examples using normalized weights as a discrete distribution.
Revised: For the latter, we draw examples using normalized weights as a discrete distribution.

Original: The epoch ends when 12 | U | pseudo-labels have been drawn, that is the epoch is 50% compared to Iscen et al.
Revised: The epoch ends when 12 | U | pseudo-labels have been drawn, that is the epoch is 50% compared to Iscen et al.

Original: Given that | L | (cid:28) | U | in most cases, the labeled examples are typically repeated more than once.
Revised: Given that | L | (cid:28) | U | in most cases, the labeled examples are typically repeated more than once.

Original: Acquisition strategies.
Revised: Acquisition strategies.

Original: We evaluate our new acquisition strategy jLP along with the following baselines: (a) Random ;
Revised: We evaluate our new acquisition strategy jLP along with the following baselines: (a) Random ;

Original: (b) Uncertainty based on entropy (5); (c) CEAL (Wang et al., 2017), combining entropy with pseudo-labels (7); (d) the greedy version of CoreSet (4) (Sener & Savarese, 2018;
Revised: (b) Uncertainty based on entropy (5); (c) CEAL (Wang et al., 2017), combining entropy with pseudo-labels (7); (d) the greedy version of CoreSet (4) (Sener & Savarese, 2018;

Original: Geifman & El-Yaniv, 2017).
Revised: Geifman & El-Yaniv, 2017).

Original: Baselines.
Revised: Baselines.

Original: For all acquisition strategies, we show results of the complete Algorithm 1 as well as the the standard baseline , that is without pre-training and only fully supervised on labeled examples
Revised: For all acquisition strategies, we show results of the complete Algorithm 1 as well as the the standard baseline , that is without pre-training and only fully supervised on labeled examples

Original: L , and unsupervised pre-training
Revised: L , and unsupervised pre-training

Original: PRE ) alone without semi-supervised.
Revised: PRE ) alone without semi-supervised.

Original: In some cases we show semi-supervised ( SEMI ) alone.
Revised: In some cases, we show semi-supervised ( SEMI ) alone.

Original: For instance, in the scenario of 100 labels per class, the effect of pre-training is small, especially in the presence of semi-supervised.
Revised: For instance, in the scenario of 100 labels per class, the effect of pre-training is small, especially in the presence of semi-supervised.

Original: CEAL (Wang et al., 2017) is a baseline with its own pseudo-labels, so we do not combine it with semi-supervised.
Revised: CEAL (Wang et al., 2017) is a baseline with its own pseudo-labels, so we do not combine it with semi-supervised.

Original: The length of the epoch is ﬁxed for Algorithm 1 and increases with each cycle for the baselines.
Revised: The length of the epoch is ﬁxed for Algorithm 1 and increases with each cycle.

Original: Label budget and cycles.
Revised: Label budget and cycles.

Original: We consider three different scenarios, as shown in Table 1.
Revised: We consider three different scenarios, as shown in Table 1.

Original: In the ﬁrst we use an initial balanced label set L 0 of 10 labels per class, translating into a total of 100 for CIFARand SVHN and 1000 for CIFAR-100.
Revised: In the ﬁrst, we use an initial balanced label set L 0 of 10 labels per class, translating into a total of 100 for CIFARand SVHN and 1000 for CIFAR-100.

Original: We use the same values as label budget b for all cycles.
Revised: We use the same values as label budget b for all cycles.

Original: In the second, we use initially 100 labels per class in CIFAR-10 with b = 1000 per cycle; this is not interesting for CIFAR-100 as it results in complete labeling of the training set after 4 cycles.
Revised: In the second, we use initially 100 labels per class in CIFAR-10 with b = 1000 per cycle; this is not interesting for CIFAR-100 as it results in complete labeling of the training set after 4 cycles.

Original: Finally, we investigate the use of one label per class both as the initial set and the label budget, on MNIST, translating to 10 labels per cycle.
Revised: Finally, we investigate the use of one label per class both as the initial set and the label budget, on MNIST, translating to 10 labels per cycle.

Original: All experiments are carried out for 5 cycles and repeated 5 times using different initial label sets L 0 .
Revised: All experiments are carried out for 5 cycles and repeated 5 times using different initial label sets L 0 .

Original: We report average accuracy and standard deviation .
Revised: We report average accuracy and standard deviation .

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: 6. S TANDARD BASELINE RESULTS
Revised: 6. S TANDARD BASELINE RESULTS

Original: We ﬁrst evaluate acquisition functions without using any unlabeled data.
Revised: We ﬁrst evaluate acquisition functions without using any unlabeled data.

Original: Figure 1 presents results on
Revised: Figure 1 presents results on

Original: SVHN, CIFAR-10 and CIFAR-100.
Revised: SVHN, CIFAR-10 and CIFAR-100.

Original: The differences between acquisition functions are not signiﬁcant, except when compared to Random.
Revised: The differences between acquisition functions are not signiﬁcant, except when compared to Random.

Original: On SVHN, Random appears to be considerably better than the other acquisition functions and worse on CIFAR-10 with b = 1000 .
Revised: On SVHN, Random appears to be considerably better than the other acquisition functions and worse on CIFAR-10 with b = 1000 .

Original: All the other acquisition functions give near identical results; in particular, there is no clear winner in the case of 10 labels per class on CIFAR-10 and CIFAR-100 (Figure 1(b) and (d), respectively).
Revised: All the other acquisition functions give near identical results; in particular, there is no clear winner in the case of 10 labels per class on CIFAR-10 and CIFAR-100 (Figure 1(b) and (d), respectively).

Original: This conﬁrms similar observations made in Gissin & Shalev-Shwartz (2018) and Chitta et al.
Revised: This conﬁrms similar observations made in Gissin & Shalev-Shwartz (2018) and Chitta et al.

Original: Our jLP is no exception, giving similar results to the other acquisition functions.
Revised: Our jLP is no exception, giving similar results to the other acquisition functions.

Original: We study this phenomenon in Appendix B.
Revised: We study this phenomenon in Appendix B.

Original: In summary, we ﬁnd that while the ranks of examples according to different strategies may be uncorrelated, the resulting predictions of label propagation mostly agree.
Revised: In summary, we ﬁnd that while the ranks of examples according to different strategies may be uncorrelated, the resulting predictions of label propagation mostly agree.

Original: Even in cases of disagreement, the corresponding examples have small weights, hence their contribution to the cost function is small.
Revised: Even in cases of disagreement, the corresponding examples have small weights, hence their contribution to the cost function is small.

Original: Since those predictions are used as pseudo-labels in Iscen et al.
Revised: Since those predictions are used as pseudo-labels in Iscen et al.

Original: (2019), this can explain why the performance of the learned model is also similar in the presence of semi-supervised learning.
Revised: (2019), this can explain why the performance of the learned model is also similar in the presence of semi-supervised learning.

Original: 6. T HE EFFECT OF UNSUPERVISED PRE - TRAINING
Revised: 6. T HE EFFECT OF UNSUPERVISED PRE - TRAINING

Original: As shown in Figure 2, pre
Revised: As shown in Figure 2, pre

Original: -training can be beneﬁcial.
Revised: -training can be beneﬁcial.

Original: PRE by itself brings substantial gain on SVHN and CIFAR-10 with b = 100 , up to 6%, while the improvements on CIFAR-100 are moderate.
Revised: PRE by itself brings substantial gain on SVHN and CIFAR-10 with b = 100 , up to 6%, while the improvements on CIFAR-100 are moderate.

Original: In addition, numerical results in Table 2 for our acquisition strategy jLP show that PRE is beneﬁcial with or without SEMI in most cases.
Revised: In addition, numerical results in Table 2 for our acquisition strategy jLP show that PRE is beneﬁcial with or without SEMI in most cases.

Original: Pre-training provides a relatively easy and cost-effective improvement.
Revised: Pre-training provides a relatively easy and cost-effective improvement.

Original: It is performed only once at the beginning of the active learning process.
Revised: It is performed only once at the beginning of the active learning process.

Original: While Caron et al.
Revised: While Caron et al.

Original: (2018) was originally tested on large datasets like ImageNet or YFCC100M, we show that it can be beneﬁcial even on smaller datasets like CIFAR-10 or SVHN.
Revised: (2018) was originally tested on large datasets like ImageNet or YFCC100M, we show that it can be beneﬁcial even on smaller datasets like CIFAR-10 or SVHN.

Original: 6. T HE EFFECT OF SEMI - SUPERVISED LEARNING
Revised: 6. T HE EFFECT OF SEMI - SUPERVISED LEARNING

Original: Figure 4 shows results on different datasets and acquisition strategies like Figure 2, but including both PRE and PRE + SEMI .
Revised: Figure 4 shows results on different datasets and acquisition strategies like Figure 2, but including both PRE and PRE + SEMI .

Original: For the purpose of reproducibility, numeric results, including average and standard deviation measurements, are given in Appendix A for all cycles and datasets.
Revised: For the purpose of reproducibility, numeric results, including average and standard deviation measurements, are given in Appendix A for all cycles and datasets.

Original: The combination PRE + SEMI yields a further signiﬁcant improvement over PRE and the standard baseline, on all acquisition functions and datasets.
Revised: The combination PRE + SEMI yields a further signiﬁcant improvement over PRE and the standard baseline, on all acquisition functions and datasets.

Original: For instance, on CIFAR-10 with a budget of 100, the most noticeable improvement comes from Random, where the improvement of PRE + SEMI is around 15% over the standard baseline at all cycles.
Revised: For instance, on CIFAR-10 with a budget of 100, the most noticeable improvement comes from Random, where the improvement of PRE + SEMI is around 15% over the standard baseline at all cycles.

Original: The improvement is around 10% in most other cases, which is by far greater than any potential difference between the acquisition methods.
Revised: The improvement is around 10% in most other cases, which is by far greater than any potential difference between the acquisition methods.

Original: Also,
Revised: Also,

Original: [Figure]

Original: PRE

Original: [Figure]
Revised: [Figure]

Original:          cycle a v e r a g eacc u
Revised: SEMI         cycle a v e r a g eacc u

Original: r ac y         cycle        cycle
Revised: r ac y         cycle        cycle

Original: [Figure]
Revised: [Figure]

Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: noticeably, in the case of SVHN, Random with PRE + SEMI reaches nearly the fully supervised accuracy after just 2 cycles (300 labeled examples in total).
Revised: noticeably, in the case of SVHN, Random with PRE + SEMI reaches nearly the fully supervised accuracy after just 2 cycles (300 labeled examples in total).

Original: The gain from semi-supervised learning is dramatic in the few-labels regime of CIFAR-10 with b = 100 .
Revised: The gain from semi-supervised learning is dramatic in the few-labels regime of CIFAR-10 with b = 100 .

Original: A single cycle with PRE + SEMI achieves the accuracy of 4 cycles of the standard baseline in this case, which translates to a signiﬁcant reduction of cost for human annotation.
Revised: A single cycle with PRE + SEMI achieves the accuracy of 4 cycles of the standard baseline in this case, which translates to a signiﬁcant reduction of cost for human annotation.

Original: In Table 2 we present the effect of all four combinations: with/without PRE and with/without SEMI .
Revised: In Table 2 we present the effect of all four combinations: with/without PRE and with/without SEMI .

Original: We focus on our jLP acquisition strategy, which has similar performance as all other strategies and uses manifold similarity just like SEMI .
Revised: We focus on our jLP acquisition strategy, which has similar performance as all other strategies and uses manifold similarity just like SEMI .

Original: In most cases, PRE improves over SEMI alone by around 2%.
Revised: In most cases, PRE improves over SEMI alone by around 2%.

Original: The use of PRE appears to be particularly beneﬁcial in the ﬁrst cycles, while its impact decreases as the model performance improves.
Revised: The use of PRE appears to be particularly beneﬁcial in the ﬁrst cycles, while its impact decreases as the model performance improves.

Original: It is worth noting that CEAL, which makes use of pseudo-labels, has a low performance.
Revised: It is worth noting that CEAL, which makes use of pseudo-labels, has a low performance.

Original: This has been observed before (Ducoffe & Precioso, 2018) and can be attributed to the fact that it is using the same set of pseudo-labels in every epoch.
Revised: This has been observed before (Ducoffe & Precioso, 2018) and can be attributed to the fact that it is using the same set of pseudo-labels in every epoch.

Original: By contrast, pseudo-labels are updated in every epoch in our case.
Revised: By contrast, pseudo-labels are updated in every epoch in our case.

Original: 6. L ABEL PROPAGATION WITH ONE LABEL PER CLASS
Revised: 6. L ABEL PROPAGATION WITH ONE LABEL PER CLASS

Original: Since PRE and SEMI have a signiﬁcant gain in classiﬁcation accuracy, it is reasonable to attempt even fewer labeled examples than in previous work on active learning.
Revised: Since PRE and SEMI have a signiﬁcant gain in classiﬁcation accuracy, it is reasonable to attempt even fewer labeled examples than in previous work on active learning.

Original: We investigate the extreme case of one label per class using MNIST as a benchmark, that is, label budget at each cycle is equal to the number of classes.
Revised: We investigate the extreme case of one label per class using MNIST as a benchmark, that is, label budget at each cycle is equal to the number of classes.

Original: Figure 3 shows results on all acquisition strategies with and without SEMI .
Revised: Figure 3 shows results on all acquisition strategies with and without SEMI .

Original: As in the previous experiments, there is no consistent winner among the selection strategies alone, and accuracy remain below 80% after 5 cycles (50 labels in total) without SEMI .
Revised: As in the previous experiments, there is no consistent winner among the selection strategies alone, and accuracy remain below 80% after 5 cycles (50 labels in total) without SEMI .

Original: By contrast, Random with SEMI arrives at 90.89% accuracy after two cycles (20 labeled examples), which is 40% better than without SEMI .
Revised: By contrast, Random with SEMI arrives at 90.89% accuracy after two cycles (20 labeled examples), which is 40% better than without SEMI .

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: In this work, we have shown the beneﬁt of using both labeled and unlabeled data during model training in deep active learning for image classiﬁcation.
Revised: In this work, we have shown the beneﬁt of using both labeled and unlabeled data during model training in deep active learning for image classiﬁcation.

Original: This leads to a more accurate model while requiring less labeled data, which is in itself one of the main objectives of active learning.
Revised: This leads to a more accurate model while requiring less labeled data, which is in itself one of the main objectives of active learning.

Original: We have used two particular choices for unsupervised feature learning and semi-supervised learning as components in our pipeline.
Revised: We have used two particular choices for unsupervised feature learning and semi-supervised learning as components in our pipeline.

Original: There are several state of the art methods that could be used for the same purpose, for instance Tarvainen & Valpola (2017); Verma et al.
Revised: There are several state of the art methods that could be used for the same purpose, for instance Tarvainen & Valpola (2017); Verma et al.

Original: ; Berthelot et al.
Revised: ; Berthelot et al.

Original: ; Rebufﬁ et al.
Revised: ; Rebufﬁ et al.

Original: (2019) for semi-supervised learning.
Revised: (2019) for semi-supervised learning.

Original: Our pipeline is as simple as possible, facilitating comparisons with more effective choices, which can only strengthen our results.
Revised: Our pipeline is as simple as possible, facilitating comparisons with more effective choices, which can only strengthen our results.

Original: While the improvement coming from recent acquisition strategies is marginal in many scenarios, an active learning approach that uses unlabeled data for training and not just acquisition appears to be a very good option for deep network models.
Revised: While the improvement coming from recent acquisition strategies is marginal in many scenarios, an active learning approach that uses unlabeled data for training and not just acquisition appears to be a very good option for deep network models.

Original: Our ﬁndings can have an impact on how deep active learning is evaluated in the future.
Revised: Our ﬁndings can have an impact on how deep active learning is evaluated in the future.

Original: For instance, the relative performance of the random baseline to all other acquisition strategies depends strongly on the label budget, the cycle and the presence of pre-training and semi-supervised learning
Revised: For instance, the relative performance of the random baseline to all other acquisition strategies depends strongly on the label budget, the cycle and the presence of pre-training and semi-supervised learning

Original: A A LL DETAILED RESULTS
Revised: A A LL DETAILED RESULTS

Original: In order to facilitate reproducibility,we present in this section  all results detailed  in Table 3 and Table 4.
Revised: In order to facilitate reproducibility, in this section we present all the detailed results in Tableand Table 4.

Original: We describe results obtained with the ﬁve methods  Random, Uncertainty, CEAL, CoreSet and jLP presented before.
Revised: We describe results obtained with the ﬁve methods presented before, namely Random, Uncertainty, CEAL, CoreSet and jLP.

Original: We evaluate them on CIFAR-10 with 10 and 100 labels per class (budget b = 100 and b = 1000 respectively), CIFAR-100 with b = 1000 in Table 3.
Revised: We evaluate them on CIFAR-10 with 10 and 100 labels per class (budget b = 100 and b = 1000 respectively), CIFAR-100 with b = 1000 in Table 3.

Original: Wealso present accuracies obtained on MNIST with only 1 label per class ( b = 10 ) and SVHN with b = 100 in Table 4.
Revised: We present results obtained on MNIST with only 1 label per class ( b = 10 ) and SVHN with b = 100 in Table 4.

Original: B S TUDYING THE AGREEMENT OF ACQUISITION STRATEGIES
Revised: B S TUDYING THE AGREEMENT OF ACQUISITION STRATEGIES

Original: It has been observed that most acquisition strategies do not provide a signiﬁcant improvement over standard uncertainty when using deep neural networks; for instance, all strategies perform similarly on
Revised: It has been observed that most acquisition strategies do not provide a signiﬁcant improvement over standard uncertainty when using deep neural networks; for instance, all strategies perform similarly on

Original: CIFAR-10 and CIFAR-100 according to Gissin & Shalev-Shwartz (2018) and Chitta et al.
Revised: CIFAR-10 and CIFAR-100 according to Gissin & Shalev-Shwartz (2018) and Chitta et al.

Original: To better understand the differences, the ranks of examples acquired by different strategies are compared pairwise by Gissin & Shalev-Shwartz (2018).
Revised: To better understand the differences, the ranks of examples acquired by different strategies are compared pairwise by Gissin & Shalev-Shwartz (2018).

Original: We make a step further in this direction, using label propagation as a tool.
Revised: We make a step further in this direction, using label propagation as a tool.

Original: B. M EASURING AGREEMENT
Revised: B. M EASURING AGREEMENT

Original: After the classiﬁer is trained at any cycle using any reference acquisition function a , we apply two different acquisition functions, say a (1) and a (2) , followed by labeling of acquired examples and label propagation, obtaining two different sets of predicted pseudo-labels ˆ y (1) and ˆ y (2) and weights w (1) and w (2) on the unlabeled examples U .
Revised: After the classiﬁer is trained at any cycle using any reference acquisition function a , we apply two different acquisition functions, say a (1) and a (2) , followed by labeling of acquired examples and label propagation, obtaining two different sets of predicted pseudo-labels ˆ y (1) and ˆ y (2) and weights w (1) and w (2) on the unlabeled examples U .

Original: We deﬁne the weighted accuracy
Revised: We deﬁne the weighted accuracy

Original: [Equation]
Revised: [Equation]

Original: (13) for z , z (cid:48) ∈ R | U | , where δ is the Kronecker delta function.
Revised: (13) for z , z (cid:48) ∈ R | U | , where δ is the Kronecker delta function.

Original: Using the average weights w := 12 ( w (1) + w (2) ) , we then measure the weighted accuracy A U, w ( y (1) , y (2) ) , expressing the agreement of the two strategies, as well as the weighted accuracy A U, w ( y ( k ) , t ) of a ( k ) relative to the true labels t on
Revised: Using the average weights w := 12 ( w (1) + w (2) ) , we then measure the weighted accuracy A U, w ( y (1) , y (2) ) , expressing the agreement of the two strategies, as well as the weighted accuracy A U, w ( y ( k ) , t ) of a ( k ) relative to the true labels t on

Original: U for k = 1 , 2 .
Revised: U for k = 1 , 2 .

Original: More measurements include weighted accuracies relative to true labels on subsets of
Revised: More measurements include weighted accuracies relative to true labels on subsets of

Original: U where the two strategies agree or disagree.
Revised: U where the two strategies agree or disagree.

Original: This way, assuming knowledge of the true labels on the entire set X , we evaluate the quality of pseudo-labels used in semi-supervised learning in each cycle, casting label propagation as an efﬁcient surrogate of the learning process.
Revised: This way, assuming knowledge of the true labels on the entire set X , we evaluate the quality of pseudo-labels used in semi-supervised learning in each cycle, casting label propagation as an efﬁcient surrogate of the learning process.

Original: B. R ESULTS
Revised: B. R ESULTS

Original: We show results on CIFAR-10 with b = 1000 in this study.
Revised: We show results on CIFAR-10 with b = 1000 in this study.

Original: Following the experiments of Gissin & Shalev-Shwartz (2018), we ﬁrst investigate the correlation of the ranks of unlabeled examples obtained by two acquisition functions.
Revised: Following the experiments of Gissin & Shalev-Shwartz (2018), we ﬁrst investigate the correlation of the ranks of unlabeled examples obtained by two acquisition functions.

Original: As shown in Figure 5(a), Uncertainty and jLP are not as heavily correlated compared to, for example, CoreSet and Uncertainty in Figure 5(b).
Revised: As shown in Figure 5(a), Uncertainty and jLP are not as heavily correlated compared to, for example, CoreSet and Uncertainty in Figure 5(b).

Original: The correlation between jLP and CoreSet is also quite low as shown in Figure 5(c).
Revised: The correlation between jLP and CoreSet is also quite low as shown in Figure 5(c).

Original: It may of course be possible that two strategies with uncorrelated ranks still yield models of similar accuracy.
Revised: It may of course be possible that two strategies with uncorrelated ranks still yield models of similar accuracy.

Original: To investigate this, we measure agreement as described above.
Revised: To investigate this, we measure agreement as described above.

Original: Results are shown in Table 5.
Revised: Results are shown in Table 5.

Original: Uncertainty is used as a reference strategy, i.e .
Revised: Uncertainty is used as a reference strategy, i.e .

Original: we train the model for a number of cycles using
Revised: we train the model for a number of cycles using

Original: Uncertainty and then measure agreement and disagreement of another strategy to Uncertainty.
Revised: Uncertainty and then measure agreement and disagreement of another strategy to Uncertainty.

Original: After cycle 1, any two methods agree on around 80% of the pseudo-labels, while the remaining 20% have on average smaller weights compared to when the methods agree.
Revised: After cycle 1, any two methods agree on around 80% of the pseudo-labels, while the remaining 20% have on average smaller weights compared to when the methods agree.

Original: We reach the same conclusions from a similar experiment where we actually train the model rather than performing label propagation.
Revised: We reach the same conclusions from a similar experiment where we actually train the model rather than perform label propagation.

Original: Hence, although examples are ranked differently by different strategies, their effect on prediction, either by training or label propagation, is small.
Revised: Hence, although examples are ranked differently by different strategies, their effect on prediction, either by training or label propagation, is small.

Original: [Table]
Revised: [Table]

Original: Uncertainty j L P     · 10    · 

Revised: Uncertainty j L P     · 10    ·

Original: Uncertainty
Revised: Uncertainty

Original: C o r e S e t
Revised: C o r e S e t

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: Following Algorithm 1, we show the effect of unsupervised pre-training ( PRE ) and semi-supervised learning ( SEMI ) compared to the standard baseline

Original: C YCLE
Revised: C YCLE

Revised: M EASURE

Original: [Table]
Revised: [Table]