LADA: Look-Ahead Data Acquisition via Augmentation for Deep Active Learning
Authors: Yoon-Yeong Kim
Kyungwoo Song
JoonHo Jang
Il-chul Moon
Venue: NeurIPS 2021 Poster (2021)
Keywords: active learning
data augmentation
look-ahead acquisition
entropy
mixup

Abstract: Active learning effectively collects data instances for training deep learning models when the labeled dataset is limited and the annotation cost is high. Data augmentation is another effective technique to enlarge the limited amount of labeled instances. The scarcity of labeled dataset leads us to consider the integration of data augmentation and active learning. One possible approach is a pipelined combination, which selects informative instances via the acquisition function and generates virtual instances from the selected instances via augmentation. However, this pipelined approach would not guarantee the informativeness of the virtual instances. This paper proposes Look-Ahead Data Acquisition via augmentation, or LADA framework, that looks ahead the effect of data augmentation in the process of acquisition. LADA jointly considers both 1) unlabeled data instance to be selected and 2) virtual data instance to be generated by data augmentation, to construct the acquisition function. Moreover, to generate maximally informative virtual instances, LADA optimizes the data augmentation policy to maximize the predictive acquisition score, resulting in the proposal of InfoSTN and InfoMixup. The experimental results of LADA show a significant improvement over the recent augmentation and acquisition baselines that were independently applied.

LADA: Look-Ahead Data Acquisition via Augmentation for Deep Active Learning
Yoon-Yeong Kim!
Kyungwoo Song² Joonho Jang' II-Chul Moon 1.3
Korea Advanced Institute of Science and Technology (KAIST) *University of Seoul SSummary.A1 yoonyeong kim@kaist. ac. kyungwoo.. song@uos. ac..kr..
adkto8093@kaist. .c.kr icmoon@kaist. ac..kr
Abstract
Active learning effectively collects data instances for training deep learning models when the labeled dataset is limited and the annotation cost is high. Data aug- mentation is another effective technique to enlarge the limited amount of labeled instances. The scarcity of labeled dataset leads us to consider the integration of data augmentation and active learning. One possible approach is a pipelined combination, which selects informative instances via the acquisition function and generates virtual instances from the selected instances via augmentation. However, this pipelined approach would not guarantee the informativeness of the virtual instances. This paper proposes Look-Ahead Data Acquisition via augmentation, or LADA framework, that looks ahead the effect of data augmentation in the pro- cess of acquisition. LADA jointly considers both 1) unlabeled data instance be selected and 2) virtual data instance to be generated by data augmentation, to construct the acquisition function. Moreover, to generate maximally informative virtual instances, LADA optimizes the data augmentation policy to maximize the predictive acquisition score, resulting in the proposal InfoSTN and InfoMixup. The experimental results of LADA show a significant improvement over the recent augmentation and acquisition baselines that were independently applied.
Introduction
Large-scale datasets in the big data era have opened the blooming of data science, but the data labeling requires significant efforts from human annotators, oracle. Therefore, an adaptive sampling by an acquisition function, i.e. active learning, has been developed to select the most informative data instances in learning the decision boundary [+3] This selection is difficult because it is influenced by the learner and the dataset at the same time. Hence, the understanding of the relation between the two has become the components of active learning, which queries the next training example by the informativeness, assessed by acquisition function.
Besides active learning, data augmentation is another data source for learning models that provides virtual data instances generated from the training dataset (4) Conventional data augmentation has been simple transformation of labeled data instances, e.g., flipping, rotating, etc [5]. Recently, the data augmentation has expanded become deep neural model generating virtual instances, such as Generative Adversarial Networks (GAN) [6] or Variational Autoencoder (VAE) [7]. Spatial Transform Networks (STN) X also generate spatially transformed instances for learning classifier. Since the conventional and the deep neural model-based augmentations perform the Vicinal Risk Minimization (VRM) 9 they preserve labels of virtual instances and limit the feasible vicinity. To overcome the limited vicinity of VRM, Mixup [10] and its variants have been proposed by interpolating multiple data instances. The pair of interpolated features and labels, or the Mixup instances, become virtual instances to enlarge the support of the data distribution.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Given the scarce labeled dataset, it is natural to consider combin- ing active learning and data augmentation. One possible way is a pipelined approach, which selects data instances by an acquisition function and generates virtual instances from the selected instances by an augmentation model afterward [T]. However, the acquisition function does not consider the potential gain from the augmenta- tion in the assessment of the informativeness. Hence, without any feedback or integration effort at the acquisition level, the virtual instances generated by data augmentation would not guarantee the informativeness. Figure Ta illustrates the pipelined combination, where the averaged entropy value of the virtual instances is 1.61.
1.99
1.61
2.33
Pipelined combination of Max Entropy and Mixup
2.12
This paper proposes the Look-Ahead Data Acquisition via augmen- tation, or LADA framework. LADA looks ahead the effect of data augmentation in advance of the actual acquisition process, by se- lecting data instances according to the acquisition score of both unlabeled real instances and their augmented virtual instances, at the same time. The acquisition algorithm of LADA enables us to train the classifier with the instances that are informative 1) when labeled by oracle and 2) when augmented via data augmentation. Furthermore, the data augmentation policy in LADA is trained to maximize the acquisition score of the virtual instances. Figure [bb illustrates the different behavior of LADA with Max Entropy and Mixup, where the averaged entropy value of the virtual instances is 2.12, which is higher than the pipelined combination in Figure Ial
1.73
2.03
(b) LADA with Max Entropy and Mixup
Figure I: Illustration of dif- ferent behaviors of the acqui- sition process in active learn- ing; with selected instances (+), virtual instances (x), en- tropy values of selected in- stances (black number), and averaged entropy values of vir- tual instances (blue number).
Here are our contributions. First, propose generalized frame- work, named LADA, that looks ahead the acquisition score of the virtual data instances to be augmented, in advance of the acquisition. Second, we train the data augmentation policy to maximize the acquisition score, hence generate informative virtual instances. Particularly, propose two data augmentation methods, InfoSTN and InfoMixup, which are trained by the feedback of acquisition scores. Third, we instantiate the proposed framework with various combinations of cqquisition-augmentiie suggest the concept of look-ahead, without acknowledging the value of augmentation 12 3 13]
of known methods. There have been some prior works that We claim our novelty for look-ahead in conjunction with the augmentation of virtual instances. Moreover, look-ahead is a necessary concept in any active learning scheme because the active learning requires an active seeking on high-value data instances which will impact the classifier if they are used in the inference, so this assessment on the impact becomes the look-ahead in such active learning concept.
2 Preliminaries
2.1 Problem Formulation
This paper trains a classifier network parameterized by 0, i..., fo. with dataset 2 while our scenario is differentiated by assuming L Ly U Ly, and | >> 1.2... Here, 2y is a set of unlabeled data instances, and 21 is a set of labeled data instances. Given these notations, a data augmentation function, faug (x:; 7): 2 I (.2", transforms data, x: E 2, into modified data, 3 E T (.2); where 7 is a parameter describing the policy of transformation, and 7 (12) is the transformed set of 2". On the other hand, data acquisition function, facy (2; fo): Lu + E. calculates score of each data instance, x E Lú, based on the current classifier, fa- faog provides the criteria of selection strategy in the learning procedure of fo with the instance, x E Lt.. We categorize the acquisition functions and the augmentation functions by placing the name of the algorithm as superscript.
)
2.2 Data Augmentation
In the conventional data augmentations, T in fang (z; 7) indicates the predefined degree of rotating, flipping, cropping, etc. is manually chosen to describe the vicinity of each data instance.
Another approach of modeling r is utilizing the feedback from the current classifier network, fo- Spatial Transformer Network (STN) transforms data using grid sampler generated through

---

localisation network parameterized by T, or f. 8 The augmentation policy, in STN is trained by the cross-entropy (CE) loss of the transformed data with the ground- -otutht label, y, resulting in r' = argmin, C (Jang (x;
In a recent work, Mixup-based data augmentations generate virtual data instance in between pair of data instances to overcome the limited vicinity. In Mixup, becomes the mixing policy of two data instances, z; and j., faug fadizup (rg Ty;T) = Ao; + (1 - A) Beta(T, r), where the labels are also mixed by the proportion À [TO]. Further from mixing input features, Manifold Mixup mixes the hidden features from the multiple middle layers of neural networks learn a smoother decision boundary [14]. Whereas T is fixed value without learning process so far, AdaMixup learns t by adopting a discriminator, pade, as 5" arggax, [log P(ysuda (falg
Mizzup (I 25,,
1) + log P(ygad (z;) = 0) + log P(pada (2z) = 0)]
2.3 Active Learning
We focus the pool-based active learning with uncertainty score [3. Given this scope of active learning, the data acquisition function measures the utility score of the unlabeled data instances, facy (z; fe). The traditional acquisition functions measure the predic-
ie. zt argmaxze
tive entropy, I Saeq' (Iz; fo) H[ fo] [16), where Hi -
clay; So) logg P(ù
clav; fe) and ; = fo(av); or the variation ratio, ( fo) - marg mar,P(uj zy fe) [17. Bayesian approaches for active learning are also proposed as S0CLDD (wv; Jo) = H bor Jol Ep(e)Dera...) [z so]] [8]
Additional modules are also applied to measure the acquisition score. Variational Adversarial Active Learning (VAAL) introduces a discriminator, sp VAAL to estimate the probability of av belonging to 2y. as (YAA"(zu) Vor.pVAAA (AA)) P(Zy E 2y; v VAAL) [19]. Learning Loss for Active Learning (LL4AL) adopts a simple neural network called a loss prediction module, or fLPAS [20]. fLPM is trained to predict the loss of each data, and the instance with the highest predictive loss is selected, as flei (xu: fLPM) = fLPMS (f% (r )kk E K). Here, fg (zv) represents the kth hidden representation of ny, and K represents the set of hidden layers of the classifier, fo-
2.4 Active Learning with Data Augmentation
There have been prior researches on leveraging data augmentation for active learning. Bayesian Generative Active Deep Learning (BGADL) combines acquisition and augmentation in a pipelined approach [T] BGADL selects data instances via facg. and BGADL augments the selected instances via faugs which is VAE-ACGAN. However, BGADL limits the vicinity to preserve the label validity. Also, a large number of labeled instances are demanded to train the generative model, VAE-ACGAN, of BGADL every acquisition round. More importantly, BGADL does not consider the potential gain from data augmentation in the process of acquisition.
In comparison with BGADL, Consistency-basee augmentation in the acquisition process, by replacing the uncertainty with augmentation-brat inconsistency, resulting in (C; (x: fe) = D DÝ x fo). P(Ý ( fo)l (2T). Here, D denotes the L2 norm 22 KL divergence 23 that represents the inconsistency of the predictions from the transformation of the data instance x into ã. The algorithm in [2I] selects a data instance that has the highest variance of class-wise predictions when it is transformed over random set of data augmentations. However, the augmentation in (21] is not learaalee, i.e., not optimized to enhance the informativeness of the augmented instance. Also, the augmentation is restricted to label- peeeeeine transformations, such as random cropping or horizontal flipping, to measure the dissimilarity in predictions when the input is perturbed with the perceptual content of the instance being preserved.
Active Learning (CAL) algorithms consider data
3 Methodology
This paper differentiates itself from the previous acquisition-augiee augmentation integration by presenting the learnable augmentations in conjunction with the potential acquisition scores the virtual instances. Therefore, we start by formulating such a learnable framework in Section 3.1 Afterward, we propose an integrated function for acquisition and augmentation as the implementation of the framework in Section 3.2 and Section 3.3 Particularly, we propose adaptive versions of augmentation, i.c.,
3

---

InfoSTN and InfoMixup, whose policies are learned by the feedback of data acquisition score, i.c.
Max Entropy. It should be noted that InfoMixup, as well as InfoSTN, can adopt various types of acquisition functions, other than Max Entropy, for feedback to train, see Appendix and Section 4
3.1 Look-Ahead Data Acquisition via Augmentation
Since we look ahead the acquisition score of the augmented data instances, it is natural to inte- grate the functionalities of acquisitions and aug- mentations. This paper proposes a framework of Look-A Data Acquisition via augmentation, or the LADA framework, see Figure 2 The goal of LADA is to enhance the informativeness of both 1) real-world data instance, which is unla- beled at current, but will be labeled by the oracle in the future; and 2) virtual data instance, which will be generated from the unlabeled data instances that are selected. This goal is achieved by looking ahead of the virtual examples' acquisition scores before actual selection.
Euts Augmortztion fant
Unideded Xy
Mal Genertor
Leen len (461))
Chasrie
Optinal Augrentation
Daex Acq hition
Paliy e
C5 V Kyy (22 (2)
Nevedd wiit enight lond
Esdivard
Figure 2: Overview of LADA framework
Specifically, LADA trains the augmentation policy, T, of faug (z;; r) maximize the acquisition score of the transformed data instance of zu before the oracle annotations. Eq. speecifiss the learning objectives of the augmentation policy via the feedback from acquisition.
T argmax facy (aag (zu: fo)
(1)
With the optimal 7" corresponding zv. LADA calculates the acquisition score of zy for selection. by considering the utility of both zy and their augmented instances, faug (au; (IU;T") T"), as Eq. 2 In Eq. 2
Y weights the relative importance of the acquisition from the virtual instance.
zt- argmax[faag (zU: fo) + 7facc (zu; T"); fo)l E.%
To begin with, we introduce an integrated single function to substitute the composition of functions fe) for generality. finteg is a general formalism for LADA that 1) constructs part of the acquisition function that looks ahead the informativeness of the virtual data instances, as Eq. [2] Also, finteg 2) becomes the objective function for training the augmentation policy to maximize the informativeness of the virtual instances, as Eq. [
as finteg = faeq fang (xv) facg(( ( (zu; T);
If we choose the simplest form of LADA, Senteg can be simple composition of well-known acquisition functions and augmentation functions where the policy of augmentation is fixed. However, this does not generate maximally informative virtual data instances. Hence, we propose the integration where the policy of data augmentation is trained to maximize the acquisition score, within a single function. We name the fixed integration case as LADA fxxed and compare it with LADA in Section
3.2 Integrated Augmentation and Acquisition: InfoSTN
This section introduces LADA that adopts 1) STN for faug. i.e., f 4STN, and 2) Max Entropy for faeq. i.e., EEnt instantiate case of finteg as finteg InfoSTN resulting in the proposal of InfoSTN.
3.2.1 Data Augmentation Policy Learning
STN is learnable neural network inserted into the classifier network. fe. which spatially manipulates the data instance, x [8]. STN consists of three parts (see Appendix E): 1) The localization network, fr, i.c.,& neural network parameterized by т, regresses the transformation parameters, v. 2) The grid generator function, fr, generates a grid, g. from a regular grid, G, using the transformation parameters, 3) Finally, the sampler function, fs, is applied to the input data instance, x. with the generated grid, 9. to get a transformed instance, f. The overall process sums up to JSTN (x; r) = fs(x.g) g) = fs(2,, (r: = fs fT: f- (x)). Hence, the parameters of the localization network, or r, correspond to the augmentation policy of STN.
For ZV E Ly, we propose the adaptive version STN, or InfoSTN, which is trained via the feedback from the acquisition function of active learning. InfoSTN learns its policy, т, with the objective

---

Algorithm LADA with Max Entropy and Manifold Mixup
Input: Labeled dataset ZD. Classifier fe 1: for j = 0. 1,2,.... .....do do
> active learning iteration
2:
Get 27 which is randomly shuffled Zu
3: Randomly chose the layer index & of the fo
4:
Φ" argming [200 ((rr)t)( L (Ih (zx), h hx (x)I)
learning policy
5:
T TT- (xs),, , (x.)) 1) for (I,, æx) (2v, 2n)
6:
Construct I IAAADA as Eq. and select and query the dataset, U's Update the labeled dataset, 27,41 27, U 2s
7:
8:
for & = 0, 1, do
ttriniin fø
9:
Get virtual dataset, Lss. from Ms using the optimal augmentation policy, T,
10:
Update with the loss, L,, Eq. 14
11: end for
end for
function of Fintegs described as below:
IInfoSTN
Jinteg
JnfoSTN I limteg
(zu: T, fo) Sacig (Jang (mv; T): fo) (r (+v))) (3)
Then, the augmentation policy, т. of InfoSTN LADA is optimized maximize the informativeness of the transformations of the unlabeled data instances, i.c., fintog -InfoSTN as below:
argmax
Σ EVE
Hillssla,
LV
It should be noted that is originally designed to minimize the cross-entropy loss of the labeled data instances, the training process of the classifier network, fg. This optimization is different direction from the optimization of Eq. 4 The original optimization the classifier, but Eq. 4 has an optimization component to explore the augmentation space. Hence, at the beginning of each acquisition iteration, save the current parameters, of the localization network. Then, we load the saved parameters to insert to fo when learning the classifier fo-
is dedicated to exploiting
3.2.2 Acquisition Function by Learned Policy and Model Training
With the optimal policy, T", construct the acquisition function, fLADA, that looks ahead the informativeness of 1) the unlabeled instances and 2) the transformed instance by InfoSTN with the optimal policy, T"
S (LUDA(CU; fa) = -Hip so] +
(5)
For the active learning with the allowed budget acquisition as b, we acquire the top-b instances, ie., 2s, among the subsets, 2'3 C. Ly with 1.2% b. by the acquisition function, fL Zs = argmax Exer; faeq
ILADD( (x;; fa). After acquiring and labeling 2s. we load the saved parameters, т. current STN and insert the STN the classifier network, fe. for training with the labeled data instances and the augmented instances in an end-to-end fashion, see Appendix E
3.3 Integrated Augmentation and Acquisition: InfoMixup
STN and our proposed variant, InfOTN, are label-preserye data augmentations, which limit the vicinity the transformed instances. Hence, overcome the limitation of the vicinity, this section introduces LADA that adopts 1) Mixup for Jaug- i.e. MMzuup and 2) Max Enimoy for Sacg. i.e. Ent. TInfoMizup resulting in the proposal of InfoMixup. Here, we
to instantiate another case finteg as finteg adopt ManifoldMixup to learn smoother decision boundary at multiple levels of representations.
3.3.1 Data Augmentation Policy Learning
InfoMixup is an adaptive version Mixup to train the data augmentation via the feedback from an acquisition function. InfoMixup learns its mixing policy, Ti, corresponding to the i-th pair,
5

---

(0), E Ev x LV, with the objective function of pinfoddrup as Eq. F'nteg
Jinteg
IInfoMimup (x: T4; So) Socg ( (Sagguu (1z d4s;;
(6)
Then, the optimal mixing policy, of of InfoMixup is found by the optimization as below:
argmax In InfoMizup (It; 14 Ta So).
intng
Different from the learning process of T in InfoSTN, we need a policy generator network, Tg. to perform an amortized inference the Beta distribution of InfoMixup. While we provide the details in Section 4.1 and Figure 3, we formulate this inference process as Eq. 8 and Eq.
(*,,,) = 10 (ms), (xa) - 1 (x))
(8)
= (T (x))) = NNo N ((DI)
(9)
To train the parameters, Φ, of the policy genera- tor network, Tgi the paired latent features, hº (x;) and (x{). are mixed-up with N number of X; Beta(T;, Ti) to produce R.n (Ia 2/3 Te) as below:
Rocand pult with wight apod
Samp Ing
Clrafter Netock
Seneratr Natwort
Wall 12001 XIM60
h mis (I), als Ti) 17 (x,) + +1- (1-133) (a)) n (1.
64
(10)
By processing h mix for the rest layers of the classifier network, the predictive class probability of the mixed features is obtained as 07²' fb:K (h (h (2,, Tz)). Then, the policy generator network, Tg. is trained to minimize loss function of Eq. [ which is the negative value of the predictive entropy, so that the policy generates high entropy valued, or informative, virtual instances. The gradient of this loss function is calculated by averaging the N entropy values of the replicated mixed features.
Figure 3: Training process of the policy gener- ator network, Tg. in LADA with Max Entropy and Manifold Mixup
N Σ n=1
De La (1h (xs),,
= 0Φ N
H W (z,, (1))
(II)
In the backpropagation, we have process of sampling As from the Beta distribution parameterized by T.. To enable the backpropagation signals pass by, adopt the reparameterization technique of the optimal mass transport (OMT) gradient estimator, which utilizes the implicit differentiation [24] 25], see Appendix C Finally, the optimal augmentation policy, Tÿ. of InfoMixup for the i-th pair of unlabeled data instances, (xi, xt)), is found as below:
Φ argmin T. (11 (xx), 1 (x4)), Tf = (1 (xx) (x0)).
(12)
3.3.2 Acquisition Function by Learned Policy and Model Training
With the optimal policy, r", construct the acquisition function, fang FLADA, which aggregates the acquisition scores of 1) I, 2) 2/, and 3) their mixed feature maps, km (a, =(17?) as below, with the predicted labels, ÿ:
N
fe (LAAA((2A, = H0 fol H[y [a]: fe]
Y N
Hig t
kon (I,, 2/3 T, dk:K]. (13)
Assuming that we start the 5th iteration of active learning with an already acquired labeled dataset VV. we acquire the set of & pairs of instances, i.c. Ls. among the subsets, 2' C Ly x Lú
with 1.4 2. as Ls argmax 2 Zy E(syatt) FI.AAA 2)); fo). After querying the label of Ms to oracle, we construct a virtual dataset, 2 M. using InfoMixup with the optimal
We denote the forward path from the th layer to the jth layer of the classifier network as f::

---

mixing policy, r". as L' = (e (x (1-x)1 (z()),
where X ~ Beta( . 7.'.
Here, 7" is dynamically inferred by the neural network of n per each pair (see Appendix B.2
Up this phase, our training dataset becomes 27-1 27 U7222 Uig and MM- Our proposed algorithm. described Algorithm[]] [ utilizes 2' for this active learning iteration only, with various A;8 sampled at each training epoch. The classifier network's s parameter, 0, is learned via the gradient of the cross-entropy loss as Eq. [14] where denotes the ground-truth label annotated from the oracle for the first term; and the mixed label according the mixing policy for the second term.
L5 =
Σ NETT
CE(fo(Z)),)) +
Σ T.E.T
(14)
12%
1.2M
4 Experiments
4.1 Baselines and Datasets
We specify the instantiated augmentation-aaquuiiiin by its subscript, c.g., LADA which adopts Max Entropy as data acquisition and Mixup as data augmentation. Similarly, we experiment with the various acquisition functions, e.g., VarMix, VAALMix, LL4ALMix, see Appendix D Also, we experiment with the STN as augmentation policy, e.g., EMSTN, see Appendix E
We compare our models to 1) Random, 2) BALD [18] 3) Coreset 126]_ 4) BADGE 227. 5) Max
Entropy [6], 6) Variation Ratio [1],, 7) VAAL [9 and 8) LL4AL 20 We also include some
data augmented active learning: 1) BGADL II. 2) CAL (21) 3) Manifold Mixup [14 and 4) AdaMixup [15]. Here, Mixup variants are applied in a pipelined approach. BGADL is also a pipelined approach for the combination, without learning mechanism in the augmentation from the feedback of acquisition. CAL does not infer the augmentation policy, either. CAL originally aims at semi-supervised learning, acquisition by CAL. We also include ablated baselines to see the effect of learning т, introducing the fixed case, LADA fixed
turn CAL into supervised setting and we apply Mixup after the
We conduct experiments on four benchmark datasets: FashionMNIST (Fashion) 28 SVHN (9).
CIFAR-10, and CIFAR- 100 30 Since we assume the scarcity of labeled dataset, we construct
random but balanced initial dataset with 20 instances for Fashion, SVHN, CIFAR-10, and 1,000
instances for CIFAR-100; and we acquire 1=10 instances for Fashion, SVHN CIFAR-10, and /=100
instances for CIFAR-100 at each iteration, following the prior work for 100 iterations. To use the same amount of oracle queries for all models, we selected top- & pairs when adopting Mixup as data augmentation in the LADA framework. We set 7=1 for all experiments in Eq. [5] except for EntSTN with CIFAR- 10 as 00.33 We normalize the images with the channel mean and standard deviation over all the datasets. For CIFAR-10 and CIFAR-100 datasets, we apply a standard augmentation, such as random crop and random horizontal flip. We adopt the ResNet- 18 31 as fo, and we utilize Adam optimizer (22 with learning rate of 1e-03. The policy generator network, Hs is much smaller network. Appendix A provides details of our experimental settings.
[8] We repeat the acquisition
4.2 Quantitative Performance Evaluations
Table 1 shows the average test accuracy of five replications. and the accuracy of each replication represents the best accuracy over the acquisition iterations. We separate the performances by the instantiated acquisition functions. The group of baselines does not have any learning mechanism on the acquisition metric. When we examine the general performance gain caused by applying LADA, across datasets, we find the best performers as LADAE in Fashion (4 2.52); LAD in SVHN (Á 5.30) and CIFAR-10 (A 4. 4.22); and LADALLALLMI in CIFAR- -IFAR-100- (4 3.56). In terms of the data augmentation, the Mixup-based augmentation outperforms the STN augmentation. In all combinations of baselines and datasets, LADA variations show the best performance in all cases, except the CIFAR-100 100 with similar performances across CAL, LADA and LADA VarMix- From the ablation study of LADA fixed. the learning of the augmentation policy, т, is meaningful because in 19 out of 20 comparison cases of LADA and LADA fixed LADA outperforms LADA fixed
Figure shows the test accuracy of the LADA frameworks and the data augmented active learning methods over the acquisition iterations on the CIFAR-10 datasct. The result is averaged over the
7

---

Table 1: Comparison of the averaged test accuracy, the run-time of single iteration of acquisition (Time), and the number of parameters (Param.). The best performance in each category is indicated in boldface. The run-time is calculated the ratio the Random acq The number of
parameters is only reported for the auxiliary network, and - indicates that no auxiliary network is adopted in the corresponding method. The result is replicated by five-fold.
Method
Fashion
SVHN
CIFAR-10
CIFAR-100
Time
Param.
Random
00.97.10.55
1..31
50.15- 1.37
43.5110.33
BALD
80.79+00.38
74.49=3.39
54.33-1.23
46.29+0.50
1.36
Coreset
83.96+0.55
76.89-0.50
51.45-0.82
43.90++0.76
1.54
Baselines
BADGE
03.06
1.84
51.83
44.13:10.64
1.31
BGADL
80.47+0077
69.60=1.62
45.98-0.73
39.33+00.8
4.69
13M
CAL
78.10++0.70
75.17+2.00
53.74-0.89
47.38+0060
1.82
Max Entropy
81.66:1111
72.35=1.21
31.45-2.12
43.14:1.558
1.01
w.ManifoldMiixup
82.03+00663
72.15=11088
51.77=1.76
45.96+0069
1.03
Ent w.AdaMixup
81.29110.47
72.46.11001
21.86-2.32
46.23110.68
1.03
Entropy-based
Lisnd
83.62:10.43
1..9
52.77-2.54
46.23:10.75
1.06
LaD
83.68:10.52
1.06
53.45=1.67
46.92:10.61
77K
LADA LADAEaSTN
01.8::00.26
73.03=1142
3..20=1.73
45.68+1.73
1.02
LADAEaSTN
82.07-10.56
1..8
1.59
4.
1.20
5K
VarRatio
80.98+0.58
73.89+1.08
55.88-0.74
46. 16++0.59
1.01
LAAAA Varkis VarMii
82.84+00.64
74.61
17-033
46.54++0.40
1.06
VarRatio-basee
LADAVarMi
83.29 10.27
75.24 0..77
1.29 56.26
47.18
1.33
77K
LADA VARSTN
83.32.00.75
74.70
54.22 0.91
46.07
1.02
5K
LADAVaSTN
0.55
0.53
46.42+0040
1,20
5K
VAAL
83.53:10.22
12.17=1.85
51.05=1.27
44.49=0.770
3.55
301K
LADA VALMIX
83.77+0.844
75.77=0.97
53.17=1.13
05.98=0.41
3.56
301K
VAAL-based
LADAVAALME
0.408=+0441
77.47=0.84
55.27=1.30
46.04+1.09
3.60
378K
LADA Rxed
VAALSTN
83.32++0.77
72.86+1.599
51.3330.13
44.27+0.26
3.56
306K
LADAVAIISTN
83.56:10.53
1..53=1165
53.78-2.24
45.06++1.29
3.57
306K
LL4AL
83.3111.34
74.14
53.01
2.90
43.58 L0.42
LADA
84 54..53
74.92.11.08
55.39 1.49
46. 10.56
124K
LL4AL-bused
LADA
85.01100.54
1.682:11.4
55.73-1.35
47.14:0.81
1.85
201K
LADA ALAALLS
03.6910.288
74.63=1.82
45.01-10990 53.28-0.67
1.63
129K
LADALLAALSTN
83.16.10.22
74.74.1.17
0.22
45.94.0061
1.68
129K
Test Accuracy
Test Accuracy
LADK LADA
NantoidHoup
Test
8G4GL
LADA
LADARIS
LADAwes
LADAxsii
Acquisition Iteration
Acquisition Iteration
Acquisition Iteration
(a) LADAEaMis
(b) LADAFMSSN
Figure 4: Test accuracy over the acquisition iterations CIFAR-10 dataset
Figure 5: Ablation study of LADA on CIFAR-10 dataset
five-fold repeated trials, and the shaded area describes the standard deviations. We also provide the figure of the test accuracy on Fashion, SVHN, and CIFAR- 100 datasets Appendix B. Notably, BGADL performs the worst in all datasets, because of the inadequate training of the generative models with the small number of data instances in our active learning setting. The degradation in test accuracies of BGADL becomes apparent as the dataset becomes complex. CAL performs comparably with LADA except the Fashion dataset.
Additionally, we compare the integrated framework, a.k.a. LADA, to the pipelined approaches. In Table [ Max Entropy is the simplest model without an augmentation part. Then, Ent w.Manifold Mixup adds the Manifold Mixup augmentation, but it does not have learning process on the mixing policy. Finally, Ent w.AdaMixup has a learning process on the mixing policy, but the learning is separated from the acquisition. These pipelined approaches show lower performances than the integration cases LADA. This ablation study also shown in Figure Mixup-based augmentations in Figure 5a and STN-base augmentations in Figure 5b respectively. The figures confirm the effects

---

of 1) considering the gain of augmentation in the acquisition functions, as well as 2) learning the augmentation policy with feedback from the aequisition.
Next, since LADA trains the augmentation policy, we also compare LADA with Augoeet [3 To show the effectiveness of the look-ahead concept, we trained AutoAugment and then applied it in
1) pipe-lined approach (a.k.a. Ent w.AA) and 2) LADA approach (a.k.a. LADA fixed As shown in Table 2 the LADA fixed shows better performance than the pipelined approach case, indicating that looking ahead the informativeness of the virtual instances yields good performance. However, should be noted that training Augmmntm requires the labeled dataset, which is not available in the active learning setting. Therefore, the learned AutoAugment could have been cheating under the assumption of joint acquisition and augmentation. Second, the pre- pre-trained Augmnt does not select the augmentation maximize the acquisition score of the unlabeled instances, so the acquisition with AutoAugment would not be optimal considering the missing contribution of labeling hard instances. Some hard instances may become very informative after the acquisition because of the augmented virtual instances, but the learned Autonugment cannot anticipate this opportunity of information gain because it pre-trained and static.
Table 2: Test accuracy of pipe-lined method and LADA with learned AutoAugment
Method
Fashion
SVHN
CIFAR-10
CIFAR-100
Ent w.AA
84.77:11112
75.40 77.4442.97
53.44 55.2241.86
46.28
LADA fixed
85.09:0.19
48.31
EntAA
Also, we extend LADA by applying it to semi-supervised learning, since comi-supervised learning algorithms also rely on the augmentation. For this experiment, we adopt the II-model [4] for sepi-super model. As shown T the combination of IT-model with LADA shows better performance than the combination with Entropy acquisition.
Table 3: Test accuracy of semi-supervised learning with LADA on CIFAR-10 dataset
of labeled samples
Methods
250
500
1000
2000
4000
I1-model Entropy
45.47
56.40
66.09
75.46
81.61
II-model + LADA EntMix
45.47
59.51
68.30
77.77
82.86
11-model + LADA AAADEEntMis Mit
45.47
58.94
68.92
78.54
82.97
4.3 Qualitative Analysis on Acquired Data Instances
Besides the quantitative comparison, we provide reasoning the behavior of LADA. Therefore, we select LADA to contrast to the pipelined approach. We investigate on 1) selecting the informative data instances by acquisition, 2) validating the optimal r" in the augmentation learned from the policy generator network no. and 3) examining the coverage of the explored space.
To check the informativeness of data instances, Figure 6 shows the different process of acquiring instances between Max Entropy and LADA Max Entropy selects a data instance with the highest predictive entropy value. Compared to Max Entropy, LADAARmMii selects a pair of two data instances with the highest value of the aggregated predictive entropy, i.e., the summation of the predictive entropy from two data instances and one InfoMixup instance, as Eq. 13 By mixing two unlabeled data instances with the optimal mixing policy r", the virtual data instance, generated along the interpolated space, results in higher entropy value than the selected instance by Max Entropy.
To confirm the validity of the optimal T", we compare three cases of 1) the inferred 7 ALADAEatMis 2) the fixed т (LADA fixed fixed Mix:: and 3) the pipelined model's 7 (Ent w.Manifold Mixup). Figure 7a shows the entropy of the virtual data instances over the acquisition process. The optimal 7 inferred in LADA gutMix produces the highest entropy over the acquisition process. The differentiation becomes significant after some acquisition iterations, which comes from the requirement of training the classifier. Figure shows the entropy distribution of virtual instances, with the median value of each interval as x-axis. This also shows that the optimal 7 has the highest density beyond the interval of the median 2.2.

---

2.45(
2.82(+
73
2.22
2.22
0:49
1.85
1.77
1.77
(a) Max Entropy at early
(b) MADAE at early
(c) Max Entropy at late
(d) LADA Mix at late
plot of acquired instance (*) and augmented instance (x), with entropy values. The numbers written in black indicate the predictive entropy of unlabeled data instances that were selected from the unlabeled pool. The numbers written in blue indicate the maximum (*average) value of predictive entropy of the virtual data instances that were generated from InfoMixup. The acquisition iterations of early and late are and 76, respectively.
Figure 6: iSNE 35
Acqunttonn Beration
Eneropy (b) Histogram the virtual dataset with entropy values
(a) Entropy value of the virtual dataset
(a) LADA
(b) Ent w.AdaMixup
Figure 7: Entropy values of the virtual data fixed generated from LADAButMix LADA and Ent w.Manifold Mixup
Figure 8: (SNE plot of acquired data instances (*) and generated virtual data instances (x). The labels are categorized by colors.
Eiteiss
To examine the coverage of the explored latent space, Figure illustrates the latent space of the acquired data instances and the augmented data instances. Ent w.AdaMixup learns the policy t to avoid the manifold intrusion, its learned 7 limits sample of À to be placed near either one of the paired instances. Therefore, Ent w.AdaMixup ends up exploring the space near the acquired instances. In contrast, the generated virtual instances by LADA EatMix show further exploration, because the optimal r" is guided by the entropy maximization and seeks along with the space that has not been explored by the model yet. The latent space makes the linear interpolation of LADABotMix to be curved by the manifold, but it keeps the interpolation line of the curved manifold.
5 Conclusions
In the real world, gathering a large-scale labeled dataset difficult, and learning a deep neural network requires effective utilization of the limited resources. This limitation motivates the integration of data augmentation and active learning. In this paper, we propose generalized framework for such integration, named LADA, which adaptively selects the informative data instances by looking ahead the acquisition score of both 1) unlabeled data instances and 2) virtual data instances to be generated by data augmentation, in advance of the acquisition process. To enhance the effect of the data augmentation, LADA learns the augmentation policy to maximize the acquisition score of the virtual instance, well. Through quantitative and qualitative analysis with various instantiations, LADA is confirmed to select and augment informative data instances.
Limitations and Ethical Discussion
The proposed work is limited the image classification task, other tasks, e.g., object detection and semantic segmentation, need to be studied in the future. LADA can be applied to these tasks with simple extension in augmentations, and such extension will be the main topic of the study. However, LADA still maintains its structure by differentiating the implemented augmentation policies by tasks. On the societal impact, privacy issue concerned when selecting and labeling dataset. Also, we need to check the robustness of LADA to prevent the failure modes or sensitivities to architectural choices.
7 Acknowledgement
This work was supported by the Technology development Program (S3125937) funded by the Ministry of SMEs and Startups (MSS, Korea).
10

---

References
[1] David Cohn, Zoubin Ghahramani, and Michael I Jordan. Active learning with statistical models, Journal of artificial intelligence research, 4:129- 145, 1996.
[2] Simon Tong. Active learning: theory and applications, volume Stanford University USA, 2001.
[3] Burr Settles. Active learning literature survey. Technical report, University of Wisconsin- Madison Department of Computer Sciences, 2009.
[4] Luis Perez and Jason Wang. The effectiveness of data augmentation in image classification using deep learning. arXiv preprint 1712.0/071120441. 2017.
[5] Esben Jannik Bjerrum. Smiles enumeration as data augmentation for neural network modeling of molecules. arXiv preprint arXiv: (tf::::70300777 2017.
[6] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Proceedings of the 27th International Conference on Neural Information Processing Systems- Volume 2, NIPS' 14, page 2672-2680, Cambridge, MA, USA, 2014. MIT Press.
[7] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In Yoshua Bengio and Yann LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL
http:://axxivv.roo abs///////// hwwgg//b///1..e.c 11/11312.11111 6114
[8] Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. Spatial transformer networks. In Advances in neural information processing systems, pages 2017-2025, 2015.
[9] Olivier Chapelle, Jason Weston, Léon Bottou, and Vladimir Vapnik. Vicinal risk minimization. In Advances in neural information processing systems, pages 416-422, 2001.
[10] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. International Conference on Learning Representations, 2018.
[11] Toan Tran, Thanh- Do. lan D. Reid, and Gustavo Carneiro. Bayesian generative active deep learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICMI. 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 0295-6304 PMLR, 2019, URL /ttp::/tt//// htw.:/ig..m......s..eee.ee.eseee
mlr prw.speesseee v97 tran htm1
[12] Nicholas Roy and Andrew McCallum. Toward optimal active learning through monte carlo estimation of error reduction. ICMI,, Williamstown, 2:441 448, 2001.
[13] Ksenia Konyushkova, Raphael Sznitman, and Pascal Fua. Learning active learning from data. arXiv preprint 1rXiv:1703.000 073660336,,.... 2017.
[14] Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez- Paz, and Yoshua Bengio. Manifold mixup: Better representations by interpolating hidden states. In International Conference on Machine Learning, pages 6438-6447, PMLR, 2019.
[15] Hongyu Guo, Yongyi Mao, and Richong Zhang. Mixup as locally linear out-of-manifold regularization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1722, 2019.
[16] Claude E. Shannon. A mathematical theory of communication. Bell Syst. Tech. J., 27(3): 379-423, 1948.
[17] L.C. Freeman. Elementary applied statistics: for students in behavioral science. Wiley, 1965. URL https::/bookk. http:://bbbbbggooo.....ooooo////
co. www.kkkkbobkksi..e/A//AAAAA.....
[18] Neil Houlsby, Ferenc Huszar, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classification and preference learning. CoRR, abs/1112.55455 2011.
2011.
11

---

[19] Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell. Variational adversarial active learning. In Proceedings of the IEEE International Conference on Computer Vision, pages 5972-5981,
2019.
[20] Donggeun Yoo and In So Kweon. Learning loss for active learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 93-102, 2019.
[21] Mingfei Gao, Zizhao Zhang, Guo Yu, Sercan Ö Ank, Larry S Davis, and Tomas Pfister. Consistency-baaee semi-supervised active learning: Towards minimizing labeling cost. In European Conference on Computer Vision, pages 510-526. Springer, 2020.
[22] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach semi-supervised learning. Advances in Neural Information Processing Systems, 32, 2019.
[23] Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V Le. Unsupervised data augmentation for consisteney training. arXiv preprint arXiv: 2848, 2019.
[24] Martin Jankowiak and Fritz Obermeyer. Pathwise derivatives beyond the reparameterization trick. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 2240-2249, PMLR, 2018. URL httphtttpp//rrcceedinng mlr. www.pprsssvv//ooe ankowiak html
[25] Martin Jankowiak and Theofanis Karaletsos. Pathwise derivatives for multivariate distributions. In Kamalika Chaudhuri and Masashi Sugiyama, editors. The 22nd International Conference on Artificial Intelligence and Statistics, AISTATS 2019, 16-18 April 2019, Naha, Okinawa, Japan, volume 89 of Proceedings of Machine Learning Research, pages 333-342. PMLR, 2019, URL htt.a/preess///////oooaaaaaa. http:////aanwwiaaakk11. html
http://procceedingg mlr. wtw.eadwinnn......ee////////////
[26] Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. International Conference on Learning Representations, 2018
[27] Jordan T Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. In ICLR, 2020.
[28] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv: I 1778.077777 2017.
[29] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[30] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[31] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[32] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http:// http//rxivv.... www.oo/gg/bbbboeooooeooo abtp//2/./1112...... 1980
[33] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoe V Le. Autoaugment: Learning augmentation strategies from data. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 113-123, 2019.
[34] Laine Samuli and Aila Timo. Temporal ensembling for semi-supervised learning. In Interna- tional Conference on Learning Representations (ICLR), volume 4, page 6, 2017.
[35] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, ((Nov):2579-2600) 2008
12


Reviews:
[{"review_id": "x1mK4cDdPxk", "reviewer": "Reviewer_pwAg", "summary": "The paper proposes to combine data augmentation with active learning to improve the performance of Active Learning approaches. Instead of a pipelined approach, where the augmentation is applied on top of the data selected by the acquisition function, the paper aims to involve data augmentation in the data acquisition step too. Data augmentation policy is optimized to increase the acquisition function score, and the resulting augmented data is then used in the search process.\n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros - \nThe paper does a nice job in motivating the problem, and also the introduction of approach is clear and concise.\n\nCons - \n\n1. I don't think the paper considers good baselines. If the paper is learning augmentations, then the $LADA^{fixed}$ baseline should be done over stronger augmentations, such as Auto-augment [1] where the augmentation policies have been learnt from before. \n\n2. The baselines for active learning are also not strong. The authors should include results with Core-Set baseline[2] which performs better than VAAL, entropy based and VarRatio-based approaches. \n\n3. The experimental setup should also include results for more iterations of active learning, probably till 10k samples selected for labelling for CIFAR-10 as done in [2] and [3].\n\n4. Section 3.3.1 which is one of the important sections of the paper is not explained properly. Its hard to understand how manifold mixup augmentation is used for LADA.\n\nReferences - \n[1] AutoAugment: Learning Augmentation Policies from Data. Cubuk et al.  \n[2] Active Learning for Convolutional Neural Networks: A Core-Set Approach. Sener et al.  \n[3] Learning Loss for Active Learning. Yoo et al.  ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors have addressed it.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "FvyI8bLAIGc", "reviewer": "Reviewer_qwu7", "summary": "The paper presents a technique for selecting unlabeled instances in an active learning loop while also generating highly informative synthetic data for augmentation. The selection of unlabeled query instances is guided by a predictive acquisition score computed using the augmented data.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Main Comments\n-------------\n\n1. Lines 48-55 -- The concept introduced here as 'look-ahead' in the context of active learning is actually not new. Similar idea (the expected error reduction framework) was introduced in [1] and has also been discussed in [2, 3] among other works. The proposed technique in the current paper seems to extend that idea to the virtual instances. [1] should be cited in the present work.\n\n2. The proposed technique combines and extends several existing works to end up with a substantially complex system that is (1) mostly restricted to standard image domain and (2) as can be understood from Table 1, has very small practical benefit over baselines.\n\n3. Figure 4: It is not clear why BGADL is shown in the figure when it has already been stated (in lines 285-286) that BGADL has worst performance. As a result, this figure presents a biased plot.\n\n4. Lines 182-183 -- An explanation is required for why the cross-entropy loss related to labeled data was not included/combined in Equation 4 when computing optimal \\tau?\n\n5. Lines 234-235 -- What exactly does the paper mean by 'After querying the label of X_S to oracle...'? X_S contains a pair of unlabeled instances. Do we get the labels of all unique unlabeled instances in X_S? Or is the query to oracle for a 'pair' of instances (not sure what that means)?\n\n6. Line 207 -- \"(x_i, {x'}_i) \\in X_U\" should be \"(x_i, {x'}_i) \\in X_U \\times \\X_U\". Needs to be consistent in all places.\n\n7. Line 249, Table 1 -- The definition of the baseline 'Random' is unclear. Does it mean that for Fashion, SVHN, CIFAR-10 1000 random unlabeled instances were selected (10000 in case of CIFAR-100) to be labeled by the oracle and then the algorithm was trained on them?\n\n8. Figure 4: It would be good to plot the *fraction* of data labeled on the x-axis (including or instead of iteration).\n\n\nMinor\n-----\n\nLine 26 -- 'Beside of active learning...' -> 'Besides active learning...'\n\nLine 86 -- 'Recently, Mixup-based...' -> 'In a recent work, ....'\n\nLine 246-247 -- 'we experiment the various' -> 'we experiment with the various'\n\nLine 248 -- 'we experiment the STN' -> 'we experiment with the STN'\n\n\n\nReferences\n----------\n\n[1] N. Roy and A. McCallum. Toward optimal active learning through sampling estimation of error reduction. In Proceedings of the International Conference on Machine Learning (ICML), pages 441–448. Morgan Kaufmann, 2001.\n\n[2] Burr Settles. Active learning literature survey. Technical report, University of Wisconsin-Madison Department of Computer Sciences, 2009.\n\n[3] Ksenia Konyushkova, Raphael Sznitman, Pascal Fua, Learning Active Learning from Data, NeurIPS 2017\n\nPost Rebuttal:\n\nI would like to thank the authors for their detailed response. \n\nI'm increasing my score by one point to marginally above acceptance. IMO, the contribution is somewhat low on novelty.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Section 6 is very short. The paper could consider adding more details.", "review_text": "", "extra_scores": {"ethical_concerns": "Privacy issues in acquiring labeled image data.", "needs_ethics_review": "No", "ethics_review_area": ["Discrimination / Bias / Fairness Concerns", "Privacy and Security (e.g., consent)"], "time_spent_reviewing": "2-3 hours"}, "extra_text": {}}, {"review_id": "BpY6eMYwX4", "reviewer": "Reviewer_hTM1", "summary": "Summary \n\nThe paper’s main goal is to incorporate data augmentation schemes into an active learning framework to further reduce the need for labeled data. An approach is proposed that ensures that the augmentations generated are also informative from the learning perspective. Experimental results are provided that show that the proposed approach is better than a simple baseline where the augmentations are generated for candidate instances without considering their informativeness.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality \n\nThe idea of looking ahead and considering the informativeness of the augmentations for the data acquisition process is interesting and useful. The main effect of the joint approach is that selected instances and associated augmentation are on average more informative. As the authors point out, a similar idea has been considered in [19]. The key difference being that the augmentation scheme is learned instead of being fixed as in [19].\n\nSignificance \n\nImproving the data efficiency for the active learning process is an important problem. Optimizing the data augmentation scheme during the data acquisition process seems effective in reducing the need for labeled data. \n\nQuality \n\nThe paper is well motivated and easy to read. Empirical comparisons show that the method does improve over BGADL and CAL baselines. Qualitative results suggest that the approach chooses different instances compared to MaxEntropy scheme and considers the space of augmentations as well when choosing them.\n\nOther questions/Comments\nInfoMixup seems to perform better than InfoSTL, is the only reason that STL can only generate labels preserving in the augmentations? Is this also the only reason for improvement over CAL or is it the richer kinds of augmentations enabled by manifold mixup? Isolating the gains will be helpful.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations section is short which mainly says that the approach is limited to classification tasks. Any discussion on failure modes or sensitivities to architectural choices, etc will be useful. It is not clear what the authors mean by \"privacy issue is relevant\". This needs a bit more clarification.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "0YOCTLBU2T", "reviewer": "Reviewer_9dkm", "summary": "This paper proposes an interesting alternative approach to active learning. The main idea is an interesting combination of active learning and data augmentation by selecting instances that are informative with and without augmentation. Moreover, the proposed approach allows to optime the data augmentation policy instead of using a fixed policy by maximizing the acquisition score. Experiments shows that the proposed approach improves previous approaches, and it can be combined with different acquisition approaches. \n\nThe main contributions of the paper are the following:\n\n• An interesting way to combine both active learning and data augmentation instead of a straightforward pipeline.\n\n• The idea of using the large unlabeled dataset to optimize the augmentation policy allows to learn better data augmentation needed for further acquisition steps.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Originality\n\nAs it is mentioned in the previous section, the paper proposes an interesting idea of selecting samples that are informative with and without augmentation. Moreover, the augmentation policy is learnt using all unlabeled pool of instances instead of using the labelled pool; hence maximizing the predictive acquisition score. The augmentation consists of introducing different data augmentations: (a) a Spatial Transformer Network (STN) that transforms an instance using a grid sampler, (b) Mixup, (c) Manifold Mixup and (d) AdaMixup. These transformations are interesting to modify the instance. \n\nQuestion 1: Instead of learning the augmentation policies that creates new virtual instances, it will be interesting to learn policies on traditional augmentation techniques like cropping, flipping, rotation and the transformation strength. Moreover, there are differentiable techniques and fast algorithms to train such policies. Therefore, this comparison will be interesting to improve the quality and significance of the paper and allows the comparison with active learning paper that selects instances that are consistent to the augmentations.\n \n# Quality\n\nThe paper shows many experiments comparing the proposed approach to other methods; showing that the proposed approach is comparable to and better to the state of the art. However, the presentation of these results could be improved since several augmentation techniques are mentioned, but few of them are only shown in the entropy-based methods and not shown in other acquisition functions. \n\nQuestion 2: Is the proposed approach only able to learn Mixup augmentation policies for all the acquisition functions, but not the other augmentations? If so, why is it only shown these cases. The paper should clarify this approach in order to validate the usefulness of the idea.\n\nQuestion 3: Section B.2 of the supplemental material shows the \\tau is similar across acquisition iterations; therefore, it shows that learning once will be enough during the time or setting it by cross-validation. How could it affect the value of tau to the proposed approach? And also its benefits? A correlation between tau and the performance will be interesting to show. How are the values of the other augmentation techniques?\n\n\n# Significance\n\nThe paper proposes an interesting and different approach to combine data augmentation and active learning, and benefit the community to join the efforts like semi-supervised learning with the consistency-learning. However, as mentioned before, the paper should answer the previous questions to highly impact the community and validate the proposed approach. \n\n# Clarity\n\nThe clarity of the paper is the main weakness, because there is a lot of information in the paper like acquisition functions, augmentations, procedures, gradients, etc that difficult the understanding of the paper. Presenting a main idea like Mixup augmentation together with the acquisition score and active learning and then a section with all possible variants, or a general and understandable methodology and notation will be improve the quality of the paper.\n\nTherefore, the paper shows an interesting idea, but needs to answer previous questions before being accepted in order to improve the significance and quality of the paper.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "no limitations", "review_text": "", "extra_scores": {"ethical_concerns": "no ethical concerns", "needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The manuscript proposes to combine data augmentation with active learning. Instead of simply applying augmentation on top of the data selected by the acquisition function, the manuscript aims to also involve data augmentation in the data acquisition step. Data augmentation policy is optimized to increase the acquisition function score, and the resulting augmented data is then used in the search process.

Reviewers agreed that the idea of looking ahead and considering the informativeness of the augmentations for the data acquisition process is interesting and useful. As the authors point out, a similar idea has been considered; the key difference being that the augmentation scheme is learned instead of being fixed. Reviewer qwu7 also pointed out other related works on 'look-ahead' in the context of active learning. The authors emphasised that the novelty of the work lies in the look-ahead in conjunction with the augmentation of virtual instances. During rebuttal, Reviewer pwAg guided authors to produce stronger baselines, and synergetic merge of active learning and semi-supervised learning.

Decision: Accept (Poster)