Carpe Diem, Seize the Samples Uncertain "at the Moment" for Adaptive Batch Selection
Authors: Hwanjun Song
Minseok Kim
Sundong Kim
Jae-Gil Lee
Venue:  (2020)
Keywords: batch selection
uncertain sample
acceleration
convergence

Abstract: The performance of deep neural networks is significantly affected by how well mini-batches are constructed. In this paper, we propose a novel adaptive batch selection algorithm called Recency Bias that exploits the uncertain samples predicted inconsistently in recent iterations. The historical label predictions of each sample are used to evaluate its predictive uncertainty within a sliding window. By taking advantage of this design, Recency Bias not only accelerates the training step but also achieves a more accurate network. We demonstrate the superiority of Recency Bias by extensive evaluation on two independent tasks. Compared with existing batch selection methods, the results showed that Recency Bias reduced the test error by up to 20.5% in a fixed wall-clock training time. At the same time, it improved the training time by up to 59.3% to reach the same test error.

Under review as a conference paper at ICLR 2020
CARPE DIEM, SEIZE THE SAMPLES UNCERTAIN "AT THE MOMENT" FOR ADAPTIVE BATCH SELECTION
Anonymous authors Paper under double-blind review
ABSTRACT
The performance of deep neural networks is significantly affected by how well mini-batches are constructed. In this paper, we propose novel adaptive batch selection algorithm called Recency Bias that exploits the uncertain samples predicted inconsistently in recent iterations. The historical label predictions of each sample are used to evaluate its predictive uncertainty within a sliding window. By taking advantage of this design, Recency Bias not only accelerates the training step but also achieves a more accurate network. We demonstrate the superiority of Recency Bias by extensive evaluation on two independent tasks. Compared with existing batch selection methods, the results showed that Recency Bias reduced the test error by up 20.5% in fixed wall-clock training time. At the same time, it improved the training time by up to 59.3% to reach the same test error.
INTRODUCTION
Stochastic gradient descent (SGD) for randomly selected mini-batch samples is commonly used to train deep neural networks (DNNs). However, many recent studies have pointed out that the perfor- mance of DNNs is heavily dependent on how well the mini-batch samples are selected (Shrivastaval et al., 2016; Chang et al. 2017: Katharopoulos & Fleuret 2018). In earlier approaches, a sam- ple's s difficulty is employed
identify proper mini- mati-batch samples, and these approaches achieve a more accurate and robust network (Han et al.) 2018) or expedite the training convergence of SGD (Loshchilov & Hutter 2016). However, the two opposing difficulty- based strategies, i.e., prefer- ring easy samples (Kumar et al. 2010; Han et al.) 2018) versus hard samples (Loshchilov & Hutter) 2016: et al.) 2016), work well in different situations. Thus, for practical reasons cover more diverse situations, recent approaches begin to exploit sample's uncertainty that indicates the consistency of previous predictions (Chang et al. 2017 Song et al., 2019).
An important question here is how to evaluate the sample's uncertainty based on its historical predictions during the training process. Intuitively, because series of historical predictions can be seen as a series of data indexed in chronological order, the uncertainty can be measured based on two forms of handling time-series observations: (i) a growing window (Figure aa) that consistently increases the size of window to use all available observations and (ii) a sliding window (Figure [[b) that maintains a window of fixed size on the most recent observations by deleting outdated ones. While the state-of-the fetthhe-rt art algorithm, Active Bias (Chang et al. 2017), adopts the growing window, we propose to use the sliding window in this paper.
Historical observations
Historical observations
Growing
Sliding
All obervatios
Outdated observations Recent observations
(a) Growing Window.
(b) Sliding Window.
Figure 1: Two forms of handling the time-series observations.
In more detail, Active Bias recognizes uncertain samples based on the inconsistency of the predictions in the entire history of past SGD iterations. Then, it emphasizes such uncertain samples by choosing them with high probability for the next mini-batch. However, according to experiments presented

---

Under review as a conference paper at ICLR 2020
Images (Horse)
Inconsistent Predictions
Consistent Predictions
Sample Uncottainty
Method
History
Outdated
Rocent (too casy)
High
Horse
Deer
Horse
Deer
Deer
Horse
Deer
Horse
Horse
Horse
Low
Active Rias
Outdlted
Recent (too hand)
High+
Deer Horse
Horse
Deer
Horse
Deer Horse
Deer
Deer
Dear
Recency Blas
Low
Previous Training Iterations
Figure 2:
The difference in sample uncertainty estimated by Active Bias and Recency Bias.
in Section 5.2 such uncertain samples slowed down the convergence speed of training, though they ultimately reduced the generalization error. This weakness is attributed to the inherent limitation of the growing window, where older observations could be too outdated (Torgo) 2011). l). In other words, the outdated predictions no longer represent network's current behavior. As illustrated in Figure 2 when the label predictions of two samples were inconsistent for long time, Active Bias invariably regards them as highly uncertain, although their recent label predictions become consistent along with the network's 's training progress. This characteristic evidently entails the risk of emphasizing uninformative samples that are too easy or too hard at the current moment, thereby slowing down the convergence speed of training.
Therefore, we propose a simple but effective batch selection method, called Recency Bias, that takes advantage of the sliding window to evaluate the uncertainty in fresher observations. As opposed to Active Bias, Recency Bias excludes the outdated predictions by managing a sliding window of a fixed size and picks up the samples predicted inconsistently within the sliding window. Thus, shown in Figure [2] the two samples uninformative at the moment are no longer selected by Recency Bias simply because their recent predictions consistent. Consequently, since informative samples effectively selected throughout the training process, this strategy not only accelerates the training speed but also leads to a more accurate network.
To validate the superiority of Recency Bias, two popular convolutional neural networks (CNNs) were trained for two independent tasks: image classification and fine tuning. We compared Recency Bias with not only random batch selection (baseline) but also two state-of-ther of-thee--he-rt batch selection strategies. Compared with three batch selection strategies, Recency Bias provided a relative reduction of test error by 1.81 1.81%-20.5%% in fixed wall-clock training time. At the same time, it significantly reduced the execution time by 24. 44.6%%59..3. 3%% to reach the same test error.
2 RELATED WORK
Let D = ((I,,, willl < i S. N} be the entire training dataset composed of a sample x; with its true label yı, where N the total number of training samples. Then, a straightforward strategy to construct a mini-batch M = {( 34)[1 < i < b} is to select b samples uniformly at random (i.e., PDx,(D) = 1/N) from the training dataset D.
Because not all samples have equal impact on training, many research efforts have been devoted to develop advanced sampling schemes. [Bengio et al. (2009) first took easy samples and then gradually increased the difficulty of samples using heuristic rules. Kumar et al. (2010) determined the easiness of the samples using their prediction errors. Recently, [Tsvetoov Recentlyy[Tveetkov et al. al. (2016) used Bayesian optimization to learn optimal curriculum for training dense, distributed word representations. Sachan & Xing (2016) emphasized that the right curriculum must introduce a small number of the samples dissimilar lar those previously seen. Fan et al. (201)) proposed neural data filter based on reinforcement learning to select training samples adaptively. However, it is common for deep learning to emphasize hard samples because of the plethora of easy ones (Katharopoulos & Fleuret 2018).
Loshchilov & Hutter (2016) proposed a difficultybased sampling scheme, called Online Batch, that uses the rank of the loss computed from previous epochs. Online Batch sorts the previously computed losses of samples in descending order and exponentially decays the sampling probability of sample according to its rank r. Then, the r-th ranked sample æ<(r is selected with the probability dropping by factor of exp (log(s<)/N) where se is the selection pressure parameter that affects the probability gap between the most and the least important samples. When normalized to sum to 1.0, the probability (D sc) is defined by Eq. (1). It has been reported that Online Batch
2

---

Under review as a conference paper at ICLR 2020
accelerates the convergence of training but deteriorates the generalization error because of the overfitting to hard training samples (Loshchilov & Hutter 2016).
exexp (log(s.)/N)
P(x(r)(D:se) Σn 1/ exp (log(s))/N)²
(1)
Most close to our work, Chang et al. (2017) devised an umcertainty-based sampling scheme, called Active Bias, that chooses uncertain samples with high probability for the next batch. Active Bias maintains the history H I that stores all hy:: (z4) before the current iteration t (i.e., growing window), where h (z4) the softmax probability of a given sample 2; for its true label y/s Then, it measures the uncertainty of the sample I; by computing the variance over all ((
H and draws the next mani-batch samples based on the normalized probability P(x, (D, -4-1;cc in Eq. (2). where is the smoothness constant to prevent the low variance samples from never being selected again. As mentioned earlier Section |
Active Bias slows down the training process because the oldest part in the history Hi no longer represents the current behavior of the network.
sid(H) )) +E Σラ (sid(H))
var var(h(ullzs) var var(h( xx:))
P(z 14 :c)
std(H) =
176--
(2)
For the completeness of the survey, we include the recent studies on submodular batch selection. Joseph et al. (2019) and Wang et al. (2019) designed their own submodular objectives that cover diverse aspects, such sample redundancy and sample representativennsss, for more effective batch selection. Differently from their work, we explore the issuc of truly uncertain samples in an orthogonal perspective. Our uncertainty measure be easily injected into their submodular optimization framework as a measure of sample informativeness.
In Section 5 we will confirm that Recency Bias outperforms Online Batch and Active Bias, which regarded as two state-of-the-an adaptive batch selection methods for deep learning.
Recency Bias COMPONENTS
3.1 CRITERION AN UNCERTAIN SAMPLE
The main challenge of Recency Bias is to identify the samples whose recent label predictions are highly inconsistent, which are neither too easy nor too hard the moment. Thus, we adopt the predictive uncertainty (Song et al. (019)) in 3.finition 1987) to measure the inconsistency of recent label predictions. Here, the sample with high predictive uncertainty is regarded as uncertain and selected with high prohability for the next mini-batch.
uses the information entropy (Chandler)
Definition 3.1. (Predictive Uncertainty) Let ja (2 0e) be the predicted label of sample r; at time and Hz, (q) = {ú., It..... .... It,1 be the label history of the sample r; that stores the predicted labels at the previous g times, where t a neural network. The label history Hz, (ạ) corresponds to the sliding window of size q to compute the uncertainty of the sample 2;. Next, p(us/z;;; q) is formulated such that it provides the probability of the label y E 21,2,... k} estimated as the label of the sample based on Ha, (g) in Eq. (3), where [+] is the Iverson bracke
plyl 9) = Es. H., (1) 13 = yi]
(3)
(g)
Then, to quantify the uncertainty of the sample vs. the predictive uncertainty F F(a;; q) is defined using the empirical entropy as Eq. (4). Because the uncertainty is bounded, we add the standardization term d to normalize the value [0, 1]. For k classes, is the maximum entropy when ()p(()z;)) = 1/k.
F(x;g) (1/8) Σ p(ila q) ((jl(())
(4)
5 = log (1/k)
'The Iverson bracket [p) returns if is true; otherwise.
3

---

Under review as a conference paper at ICLR 2020
3.2 SAMPLING PROBABILITY FOR MINI-BATCH CONSTRUCTION
To construct next mini-bare batch samples, we assign the sampling probability according to the predictive uncertainty in Definition 3.1 Motivated by Loshchilov & Hutter (2016), the sampling probability of a given sample is exponentially decayed with its predictive uncertainty F(;;; q). In detail, we adopt the quantization method (Chen & Wornell, (00T) and use the quantization index to decay the sampling probability. The index is obtained by the simple quantizer Q in Eq. (5), where A is
the quantization step size. Compared with the rank-based index (Loshchilov & Hutter 2016), the 1996..
quantization index is known to well reflect the difference in actual values (Widrow et al.
((F(z:))
[[1 F(z,;; 4)) A1. 0 S F(254) S
(5)
In Eq. (5), we set A to be 1/N such that the index is bounded to N (the total number of samples). Then, the sampling probability P s.) is defined as in Eq. (6). The higher the predictive uncertainty. the smaller the quantization index. Therefore, higher sampling probability is assigned for uncertain samples in Eq. (6).
P(x.(D:s.)= = e/exp log(ss)/N) (wxog)
(6)
Σラ 1/exp (log(sa)/N) ((F(2,1))
Meanwhile, it is known that using only some part of training data exacerbates the overfitting problem at late stage of training (Loshchilov & Hutter] 2016 Zhou & Bilmes 2018). Thus, to alleviate the problem, we include more training samples the training progresses by exponentially decaying the selection pressure Se as in Eq. ( At each epoch e from 6g to Gend, the selection pressure Se exponentially decreases from Sea to 1. Because this technique gradually reduces the sampling probability gap between the most and the least uncertain samples, more diverse samples are selected for the next mini-batch later epoch. When the selection pressure 8, becomes the mini-batch samples are randomly chosen from the entire dataset.
Sc. Sea (exp( log
eo)) 6-E0
Recency Bias ALGORITHM
Algorithm Recency Bias Algorithm
INPUT: D: data, epochs, b: batch size, q: window size, Beg: initial selection pressure, y: warm-up
OUTPUT: 0;: model parameter
1: t 1++1, 1;
2: 0. <- Initialize the model parameter;
3: for i = 1 to epochs do
4: /* Sampling Probability Derivation */
5: if i then
6: 8e f- D-Decay_Sllctinnng Selection Pressure(Seo- a); ft Decaying 8. by Eq.
7:
for m to N do 1e Updating the index and the sampling probability in batch */ [_dict(zm) = Q(F(zmiq))
8:
/ By Eq.
9:
p. table Compute Prob(y_dick se): By Eq. 6
10: A Network Training */
11:
for j = to N/b do / Mini-batch +/
12:
if S then j* Warm-up */
13:
.., (Ir. vo)} Randomly select next mini-batch samples;
14:
else /* Adaptive batch selection */
15:
{(21, (1)1,,1),... ,I)...., (Is, )n)} Select next mini-batch samples based on p table; losses, labels + Inference_Stepp (itrrone__ttep(xx.............
16:
(.)). 7* Forward */
17:
041 + SGD BiD_Step(losses, 0.): 1" Backward */
18:
Update Label History(labels); /s By Definition 3.1
19:
1 <<- 1+1:;
20:
return 0,:
Algorithm describes the overall procedure of Recency Bias. The algorithm requires a warm-up period of epochs because the quantization index for each sample is not confirmed yet. During the warm-up period, which should be at least q epochs (y 2 q) to obtain the label history of size
4

---

Under review as a conference paper at ICLR 2020
q, randomly selected mini-batch samples are used for the network update (Lines 12-13). After the warm-up period, the algorithm decays the selection pressure &e and updates not only the quantization index but also the sampling probability in batch at the beginning of each epoch (Lines 4-9). Subsequently, the uncertain samples are selected for the next mini-batch according to the updated sampling probability (Line 14-15), and then the label history is updated along with the network update (Lines 16-19).
Overall, the key technical novelty of Recency Bias to incorporate the notion of sliding win- dow (Line 8) rather than a growing window into adaptive batch selection, thereby improving both training speed and generalization error.
The main "additional" cost of Recency Bias is the derivation of the sampling probability for each sample (Lines 4-9). Because only simple mathematical operations are needed per sample, its time complexity is linear to the number of samples (i.e., O(N)), which is negligible compared with that of the forward and backward steps of complex network (Lines 16-17). Therefore, we contend that Recency Bias does not add the complexity of underlying optimization algorithm.
Time Complexity:
EVALUATION
We empirically show the improvement of Recency Bias over not only Random Batch (baseline) but also Online Batch (Loshchilov & Hutter 2016) and Active Bias (Chang al.,, 2017), which are two state- of-the-art adaptive batch selections. In particular, we elaborate on the effect of the sliding window approach (Recency Bias) compared with the growing window approach (Active Bias). Random Batch selects next mini-batch samples uniformly at random from the entire dataset. Online Batch selects hard samples based on the rank of the loss computed from previous epochs. Active Bias selects uncertain samples with high variance of true label probabilities in the growing window. All the algorithms were implemented using TensorFlow 1.8.0 and executed using a single NVIDIA Titan Volta GPU. For reproducibility, we provide the source code at httpt:t///// tiu............ com anor zed
Image classification and fine-tuning tasks were performed to validate the superiority of Recency Bias. Because fine-tuning is used to quickly adapt to a new dataset, it is suitable to reap the benefit of fast training speed. In support of reliable evaluation, we repeated every task thrice and reported the average and standard error of the best test errors. The best test error in given time has been widely used for the studies on fast and accurate training (Katharopoulos & Fleuret 2018 Loshchilov & Hutter 2016).
5.1 ANALYSIS ON SELECTED MINI-BATCH SAMPLES
For in-depth analysis on selected samples, we plot the loss distribution of mini-batch samples selected from CIFAR- by four different strategies Figure (i) The distribution Online Batch is the most skewed toward high loss by the design principle of selecting hard samples. (ii) Active Bias emphasizes moderately hard samples at an early training stage in considering that its loss distribution lies between those of Random Batch and Online Batch. However, owing the outdated predictions caused by the growing window, the proportion of easy samples with low loss increases at a late training stage. These easy samples, which are misclassified as uncertain that stage, tend to make the convergence of training slow down. (iii) In contrast to Active Bias, by virtue of the sliding window, the distribution of Recency Bias lies between those of Random Batch and Online Batch regardless of the training stage. Consequently, Recency Bias continues to highlight the moderately hard samples, which are likely to be informative, during the training process.
0.20
0.15
Random Batch
Online Batch
Denaity
Density
Active Blas
(Crowingwindon
0.05
Recency Bias
(Siliding window)
-20
-15
-10
-5
0
-20
-15
-10
-5
0
Loss (Log-scale)
Loss (Log-scale)
(a) Early Stage (30%).
(b) Late Stage (70%).
Figure 3: The loss distribution of mini-batch samples selected by four batch selection strategics: (a) and (b) show the loss distribution the 30% and 70% of total training epochs, respectively.
5

---

Under review as a conference paper at ICLR 2020
5.2 TASK I: IMAGE CLASSIFICATION
Experiment Setting: We trained DenseNet (L=40, k=12) and ResNet (L=50) with momentum
optimizer and an SGD optimizer on three benchmark datasets: MNIST (10 classes R classification of handwritten digits (LeCun 1998), and CIFAR-10 (10 classes f
and CIFAR-100 classes)',
classification of a subset of 80 million categorical images (Krizhessky et al. 2014). Specifically, we used data augmentation, batch normalization, momentum of 0.9, and a batch size of 128. As for the algorithm parameters, we fixed the window size q 10 and the initial selection pressure Sea = 100 which were the best values found by the grid search (see Appendix A for details). The warm-up epoch y was set to be 15. To reduce the performance variance caused by randomly initialized model parameters, all parameters were shared by all algorithms during the warm-up period. Regarding the training schedule, we trained the network for 40, 000 iterations and used an initial learning rate of 0.1, which was divided by 10 at 50% and 75% of the total number of training iterations.
Results: shows the convergence curves of training loss and test error for four batch selection strategies using DenseNet and a momentum optimizer. In order to highlight the improvement of Recency Bias over the baseline (Random Batch), their lines are dark colored. The best test errors in
Figures 4b) Fd) and ff) are summarized on the left side of Table []]
In general, Recency Bias achieved the most accurate network while accelerating the training process on all datasets. The training loss of Recency Bias converged faster (Figures Ka). (), and
(()
without the increase in the generalization error, thereby achieving the lower test error (Figures (b.
[d), and
contrast, the test error of Online Batch was not the best even if its training loss converged the fastest among all strategies. As the training difficulty increased from CIFAR-10 to CIFAR- 100, the test error of Online Batch became even worse than that of Random Batch. That is, emphasizing hard samples accelerated the training step but made the network overfit to hard samples. Meanwhile, Active Bias was prone to make the network better generalized on test data. In CIFAR-10, despite its highest training loss, the test error of Active Bias was better than that of Random Batch. However, Active Bias slowed down the training process because of the limitation of growing windows, as discussed in Section [..] We note that, although both Recency Bias and Active Bias exploited uncertain samples, only Recency Bias based on sliding windows succeeded to not only speed up the training process but also reduce the generalization error.
The results of the best test error for ResNet or an SGD optimizer are summarized in Tables Land 2 (see Appendix C for more details). Regardless of a neural network and an optimizer, Recency Bias achieved the lowest test error except in MNIST with an SGD optimizer. The improvement of Recency Bias over the others was higher with an SGD optimizer than with a momentum optimizer.
Table 1: The best test errors (%) of four batch selection strategies using DenseNet.
Optimizer
Momentum in Figure 4
SGD in Figure 9 X Appendix C.I
Method
MNIST
CIFAR-10
CIFAR-100
MNIST
CIFAR-10
CIFAR-100
Random Batch
0.527 + 0.03
7.33 + 0.09
28.0 + 0.16
1.23 : 0.03
14.9 + 0.09
40.2 :: 0.06
Online Batch
0.514 0.01
7.00 : 0.10
28,4 0.25
0.765 = 0.02
13.5 0.02
40.7 = 0.12
Active Bias
0.616 + 0.03
7.07 + 0.04
27.9 + 0.11
0.679 + 0.02
14.2 4 0.25
42.9 + 0.05
Recency Bias
0.490 0.02
6.60 0.02
27.1 0.19
0.986 0.06
13.2 0.11
38.7 0.11
Table 2: The best test errors (%) of four batch selection strategies using ResNet.
Optimizer
Momentum Figure 10 Appendix C.2 MNIST
SGD Figure I| (Appendix C.3 MNIST
Method
CIFAR-TO
CIFAR-I00
CIFAR-10
100
Random Batch
0.636 1 0.04
10.2 + 0.12
33.2 + 0.07
1.16 + 0.03
12.7 : 0.09
40.1 = 0.16
Online Batch
0.666 1 0.05
10.1 1 0.05
33.4 : 0.01
0.890
+ 0.03
12.2 0.08
40.7 = 0.09
Active Bios
0.613 + 0.04
10.6 0.08
34.2 + 0.07
0.804
0.01
13.5
0.07
45.6 0.07
Recency Bias
0.607 1 0.01
9.79 1 0.04
32.4 + 0.04
0.972 + 0.03
11.6 1 0.09
38,9 = 0.14
httts://yamm.eecu.comm... https:/ /www//wwwwo cs. toronto................ wwtrontt.eduu-krrr c war.rrraaaaa... html
"Online Batch also used the same decaying selection pressure value.

---

Under review as a conference paper at ICLR 2020
Random Batch
Online Batch
A ctive Bias
Recency Bias
1,3E-01
3.6%
2.5E-02
Test Eror
Traiiin
1.2%
5.0E-03
1.0E-03
0.4%
2125
4250
6375
8500
2125
4250
6375
8500
Time
Time
(a) MNIST Training Loss.
(b) MNIST Test Error.
8.6E-01
26.0%
1.4E-01
Test Enoo
Training
13.0%
2.4E-02
4.0E-03
6.5%
2500
5000
7500
10000
2500
5000
7500
10000
Time (s)
Time (s)
(c) CIFAR- Training Loss.
(d) CIFAR- Test Error.
2.4E++00
54.0%
Test Eror
Thiiing
6.0E-01
1.5E-01
27.0%
2500
5000
7500
10000
2500
5000
7500
10000
Time (s)
Time (s)
(e) CIFAR- 100 Training Loss.
(1) CIFAR-100 Test Error.
Figure 4: Convergence curves of four batch selection strategies using DenseNet with momentum.
5.3 TASK II: FINE-TUNING
Experiment Setting: We prepared (I=:1221 k=32) previously trained on ImageNet (Deng
et al. 2009) and then fine-tuned the network on two benchmark datasets: MIT-67 (67 classes classification of indoor scenes (Quattoni Torralba 2009). and Food-100 (100 classes classifi cation of popular foods in Japan (Kawano & Yanai 2014). After replacing the last classification layer, the network was trained end-to-end for 50 epochs with a batch size 32 and a constant learning rate 2 x 10-4, Data augmentation was not applied here. The other configurations were the same as those in Section 5.2
Results on Test Error: Figure shows the convergence curves of training loss and test error for the fine-tuning task on MIT-67 and Food-100, Overall, all convergence curves showed similar trends those of the classification task in Figure Only Recency Bias converged faster than Random Batch in both training loss and test error. Online Batch converged the fastest in training loss, but its error was rather higher than Random Batch owing the overfitting, Active Bias converged the
http://eeb.iit.cbbooorrabbb///// html
htto://fooiaa..nn//aaaeeeeelll
7

---

Under review as a conference paper at ICLR 2020
Random Batch
Online Batch
A ctive Bias
Recency Bias
1.9E+00
39.0%
6.3E-01
Tost Ertrr
35.0%
Training
2.1E-01
31.0%
Time Reduction: 24 6%
7.0E-02
27.0%
1500
3000
4500
6000
1500
3000
4500
6000
Time
Time
(a) MIT-67 Training Loss.
(b) MIT-67 Test Error.
1.6E+00
44.0%
8.0E-01
Error
40.0%
Traning
4.0E-01
36.0%
Time Rediction: 26.1%
2.0E-01
2000
4000
8000
2000
4000
8000
Time (s)
Time
(c) Food- 100 Training Loss.
(d) Food- 100 Test Error.
Figure 5: Convergence curves for fine-tuning on two benchmark datasets.
Table 3: Recency Bias's 's reduction in training time over other batch selection strategies.
Method
MIT-67
FOOD-100
Random Batch
(5, 218
3, 936) 218 x 100 24.6%
77, 263
365)/7, 263
100
26.1%
Online Batch
(6, 079
3, 823) /, 079 X 100 37.1%
(8, 333
3, 685)/8, 333
100
55.8%
Active Bias
(5, 738
3, 032)/5 738
100 47.2%
(7, 933
227)
933
100
59.33%
slowest in both training loss and test error. Quantitatively, compared with Random Batch, Recency Bias reduced the test error 2.88% and 1.81% in MIT-67 and Food- 100, respectively.
Results on Training Time: Moreover, to assess the performance gain in training time, we computed the reduction in the training time taken reach the same error. For example, in 5)), the best test еrror of 28.8% achieved in 5, 218 seconds by Random Batch could be achieved only in 933 seconds by Recency Bias: thus, Recency Bias improved the training time by 24.6%. Table 3 summarizes the reduction in the training time of Recency Bias over three other batch selection strategies. Notably, Recency Bias improved the training time by 24.6%-4..2% and 26.1 26.1%-59.39 in fine-tuning MIT-67 and F00D-100 datasets, respectively.
5.4 ABLATION STUDY ON SELECTION PRESSURE
For an ablation study the selection pressure, we trained (L=440, k=12) on benchmark
datasets using Recency Bias with four different decaying strategies; Se 10 - 10, $a 100 + 100, and Se 100 + 1. The first two strategies used different initial selection pressures without decaying, but the remaining strategies exponentially decayed their initial selection pressures 1. We used a momentum optimizer and the other experimental configurations were the same as those in Section 5.2
8e :
10
Figure 6 shows the convergence curves of Recency Bias using the different decaying strategies along with that of Random Batch. Generally, the two strategies without decaying (i.e., 10 - 10, $. 100 + 100) showed much faster convergence speed in training loss compared with those with decaying (i.e., Se 10 1, Se 100 + 1). However, mentioned earlier in Section 3.2 the two strategies without decaying exacerbated the overfitting problem because they only used the training samples classified highly uncertain. Accordingly, their test errors rather higher than
8

---

Under review as a conference paper at ICLR 2020
Rocency 1s, 10)
Roceney (y::
100
100)
Roceny (n:
Rocancy (s.:
8.6E-01
26.0%
1,4E-01
Test Eroo
Teaining
13.0%
2.4E-02
4.0E-03
6.5%
2500
5000
7500
10000
2500
5000
7500
10000
Time (s)
Time (s)
(a) CIFAR-I0 Training Loss.
(b) CIFAR-10 Test Error.
2,4E+000
54.0%
Thaning LOSS)
Test Enor
6.0E-01
1.5E-01
27.0%
0
2500
5000
7500
10000
2500
5000
7500
10000
Time (s)
Time (s)
(c) CIFAR- 100 Training Loss.
(d) CIFAR- 100 Test Error.
Figure 6: Ablation study on the effect the selection pressure.
that of Random Batch in CIFAR-100 dataset. On the other hand, the two strategies with decaying converged faster than Random Batch in both training loss and test error in all datasets because they exploited more diverse training samples by exponentially decaying the selection pressure. Thus, these observations empirically prove that decaying the selection pressure is an effective way to alleviate the overfitting problem at the later stage of training.
CONCLUSION
In this paper, we presented novel adaptive batch selection algorithm called Recency Bias that emphasizes predictively uncertain samples for accelerating the training of neural networks. Toward this goal, the predictive uncertainty each sample is evaluated using its recent label predictions managed by sliding window of fixed size. Then, uncertain samples at the moment high probability for the next mini-batch. We conducted extensive experiments on both classification and fine-tuning tasks. The results showed that Recency Bias is effective in reducing the training time as well as the best test eтror. It was worthwhile to note that using all historical observations to estimate the uncertainty has the side effect of slowing down the training process. Overall, merger of uncertain samples and sliding windows greatly improves the power of adaptive batch selection.
selected with
REFERENCES
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In ICML, pp. 41-48, 2009.
David Chandler. Introduction to modern statistical mechanics. Oxford University Press, 1987.
Haw-Shan Sow-Shhuan Chang, Erik Learned Miller, and Andrew McCallum. Active Bias: Training more accurate neural networks by emphasizing high variance samples. In NeurIPS, pp. 1002-1012. 2017.
Brian Chen and Gregory W Wornell. Quantization index modulation: A class of provably good methods for digital watermarking and information embedding. IEEE Trans. on Information Theory, (7(3):143-11443 2001.

---

Under review as a conference paper at ICLR 2020
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR. pp. 248-255, 2009.
Yang Fan, Fei Tian, Tao Qin, and Tie- Liu. Neural data filter for bootstrapping stochastic gradient descent. In ICLR, 2017.
Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In NeurlPS, pp. 8527-8537 2018.
KJ Joseph, Krishnakant Singh, Vineeth Balasubramanian, et al. Submodular batch selection for training deep neural networks. In IJCAI, pp. 2677-3683,2011 2019.
Angelos Katharopoulos and François Fleuret. Not all samples created equal: Deep learning with importance sampling. In ICML, pp. 2525-2534, 2018.
Y. Kawano and K. Yanai. Food image recognition with deep convolutional features. In UbiComp. 2014.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. ImageNet classification with deep convolu- tional neural networks. In NeurlPS, pp. 1097-1105, 2012.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. CIFAR-10 and CIFAR-100 datasets, 2014. https: www. cs. www.stoooooo...oooooooooooo
html
M Pawan Kumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable models. In NeurlPS, pp. 1189-1197, 2010.
Yann LeCun. The MNIST database of handwritten digits, 1998. http::/yann. lecun. com/
Ilya Loshchilov and Frank Hutter. Online batch selection for faster training of neural networks. In ICLR, 2016.
Ariadna Quattoni and Antonio Torralba. Recognizing indoor scenes. In CVPR, pp. 413-420, 2009.
Mrinmaya Sachan and Eric Xing. Easy questions first? A case study on curriculum learning for question answering. In ACI. pp. 453-463, 2016.
Abhinay Shrivastava, Abhinav Gupta, and Ross Girshick. Training region- object detectors with online hard example mining. In CVPR, pp. 761-769, 2016.
Hwanjun Song, Minseok Kim, and Jae-Gil Lee. SELFIE: Refurbishing unclean samples for robust deep learning. In ICML, pp. 5907-5915, 2019.
Luis Torgo. Data mining with R: learning with case studies. Chapman and Hal//CRC, 2011.
Yulia Tsvetkov, Manaal Faruqui, Wang I Ling, Brian MacWhinney, and Chris Dyer. Learning the curriculum with bayesian optimization for task-specific word representation learning. In ACL, pp. 130-139, 2016.
Shengjie Wang, Wenruo Bai, Chandrashekhar Lavania, and Jeff Bilmes. Fixing mini-batch sequences with hierarchical robust partitioning. In AISTATS, pp. 3352-3361, 2019.
Bernard Widrow, Istvan Kollar, and Ming ShnngChang Liu. Statistical theory of quantization. IEEE Transactions on instrumentation and measurement, 45(2):353-361 1996.
Tianyi Zhou and Jeff Bilmes. Minimax curriculum learning: Machine teaching with desirable difficulties and scheduled diversity. In ICLR, 2018.
10

---

Under review as a conference paper at ICLR 2020
A HYPERPARAMETER SELECTION
Recency Bias receives the two hyperparameters: (i) the initial selection pressure Seo that determines the sampling probability gap between the most and the least uncertain samples and (ii) the window size that determines how many recent label predictions are involved in predicting the uncertainty. To decide the best hyperparameters. we trained ResNet (L=50) on CIFAR- 10 and CIFAR- 100 with momentum optimizer. For hyperparameters selection, the two hyperparameters were chosen in grid (1, 10, 100, 1000) and y E (5, 10, 15).
8no
10.4%
33.5%
Window Size
Best Enor
10.1%
33.0%
q=5
Q=10
9.8%
32.5%
9 15
9.5%
32.0%
100
1000
100
1000
Initial Selection Pressure (S.,)
Initial Selection Pressure (S...)
(a) CIFAR- 10.
(b) CIFAR-IO0.
Figure 7: Grid search on CIFAR-I0 and CIFAR-100 datasets using ResNet.
Figure 7 shows the test errors of Recency Bias obtained by the grid search on the two datasets. Regarding the initial selection pressure Sea. the lowest test error was typically achieved when the Seo value 100. As for the window size q, the test error almost always the lowest when the value was 10. Similar trends were observed for the other combinations of a neural network and an optimizer. Therefore, in all experiments,
Seo be 100 and q to be 10.
B EXPERIMENT USING TINY-IMAGENET DATASET
For larger-scale experiment, repeated the image classification task on Tiny- Ioaye-e (200 classes), subset of ImageNet (Krizhevsky et al.) 2012), with 100, 000 training and 10, 000 validation images, Because no test set exists, we used the validation set as the test data. For Tiny-ImageNet dataset, we trained the network for 80, 000 iterations and used an initial learning rate of 0.1, which was divided by 10 50% and 75% of the total number of training iterations. The remaining experimental configurations were the same as those in Section
Random Batch
Online Batch
ctive Bias
Recency Bias
3.9E+00
77.0%
Test Envr
68.0%
Triiiin
39.0%
1.3E+00
50.0%
5000
10000
15000 20000 25000
5000
10000
15000 20000 25000
Time (s)
Time (s)
(a) ImageNet Training Loss
(b) Tiny-ImageNel Test Error.
Figure 8: Convergence curves of four batch selection strategies using DenseNet with momentum.
Table 4: The best test enrors (%) of four batch selection strategies using DenseNet.
Method
Random Batch
Online Batch
Active Bias
Recency Bias
ImageNet
51.6 0.26
52.5 0.19
52.24 0.52
51.0 = 0.34
11

---

Under review as a conference paper at ICLR 2020
Figure 8 shows the convergence curves of training loss and test enror using four batch selection strategics on Tiny-ImageNet, where the best test errors are detailed in Table Again, only Recency Bias converged faster than Random Batch in both training loss and test emror On the other hand, although Online Batch showed the fastest convergence in training loss, its test error was worse than that of Random Batch because of the overfitting to hard training samples. Similarly, the test error of Active Bias was also worse than that of Random Batch because of the side effect of slowing down the convergence speed of training. In summary, Recency Bias achieved the error relatively lower by 1.16% than Random Batch, 2.86% than Online Batch, and 2. 2.30% than Active Bias.
C GENERALIZATION OF Recency Bias
C.I CONVERGENCE CURVES USING DENSENET WITH SGD
FigureJshowe the convergence curves training loss and test error for four batch selection strategies using DenseNet and an SGD optimizer, which corresponds to the right side of Table |
Random Batch
Online Batch
A ctive Bias
Recency Bias
2.7E-01
4.8%
9.0E-02
Test Envor
2.4%
Thiiinn
3.0E-02
1.2%
1.0E-02
0.6%
2000
4000
6000
8000
2000
4000
6000
8000
Time (s)
Time (s)
(a) MNIST Training Loss.
(b) MNIST Test Error.
1.0E+000
48.0%
5.2E-01
Test Ehor
24.0%
2.6E-01
1.3E-01
12.0%
2500
5000
7500
10000
2500
5000
7500
10000
Time (s)
Time (s)
(c) CIFAR-10 10 Training Loss.
(d) CIFAR- 10 Test Error.
3.2E+00
70.0%
Test Test Enor
1.6E++00
8.0E-01
35.0%
2500
5000
7500
10000
2500
5000
7500
10000
Time (s)
Time (s)
(c) CIFAR- 100 Training Loss.
(f) CIFAR- 100 Test Error.
Figure 9: Convergence curves of four batch selection strategies using DenseNet with SGD.
12

---

Under review as a conference paper at ICLR 2020
C.2 CONVERGENCE CURVES USING RESNET WITH MOMENTUM
Figure TO shows the convergence curves of training loss and test error for four batch selection strategies using ResNet and momentum optimizer, which corresponds the left side of Table
Random Batch
Online Batch
Active Blas
Recency Bias
1.5E-01
4.5%
1.9E-02
Test Enor
Training
1.5%
2.4E-03
3.0E-04
0.5%
2300
4600
6900
2300
4600
6900
Time (s)
Time
(a) MNIST Training Loss.
(b) MNIST Test Error.
1.7E+00
36.0%
2.5E-01
Test Elnor
18.0%
3.5E-02
5.0E-03
9.0%
2900
5800
8700
2900
5800
8700
Time (s)
Time (s)
(c) CIFAR-10 Training Loss.
(d) CIFAR- Test Error.
4.3E+00
64.0%
7.2E-01
Test Enor
Taining
1.2F-01
2.08-02
32.0%
2900
5800
8700
2900
5800
8700
Time (s)
Time (s)
(c) CIFAR- 100 Training Loss.
(f) CIFAR-100
100 Test Error.
Figure 10: Convergence curves of four batch selection strategies using ResNet with momentum.
13

---

Under review as a conference paper at ICLR 2020
C.3 CONVERGENCE CURVES USING RESNET WITH SGD
Figure 11 shows the convergence curves of training loss and test error for four batch selection strategies using ResNet and SGD optimizer, which corresponds to the right side of Table 2
Random Batch
Online Batch
Active Blas
Recency Bias
4.3E-01
6.3%
7.2E-02
Test Enor
Training
2.1%
1,2E-02
2.0E-03
0.7%
2300
4600
6900
2300
4600
6900
Time (s)
Time
(a) MNIST Training Loss.
(b) MNIST Test Error.
1,4E+00
44.0%
4.5E-01
Test Elnor
22.0%
1.5E-01
5.0E-02
11.0%
2900
5800
8700
2900
5800
8700
Time (s)
Time (s)
(c) CIFAR-10 Training Loss.
(d) CIFAR- Test Error.
3.6E+000
76.0%
Test Enor
Taining
1.2E+400
4.08-01
38.0%
2900
5800
8700
2900
5800
8700
Time (s)
Time (s)
(c) CIFAR- 100 Training Loss.
(f) CIFAR-100 100 Test Error.
Figure Convergence curves of four batch selection strategies using ResNet with SGD.
14


Reviews:
[{"review_id": "r1lfkyNrtS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper explores a well motivated but very heuristic idea for selecting the next samples to train on for training deep learning models. This method relies on looking at the uncertainty of predictions of in the recent history of statements and preferring those instances that have a predictive uncertainty over the recent predictions.  This allows the training method to train on instances that are neither too hard nor too easy and focus on reducing the uncertainty whenever it has the greatest potential gain to do so.\n\nThere are two extra components that make this method work: \n- Windowing: only looking at the recent history of the instances which has two effects: firstly, the current state of the model is explored which gives a more recent assessment relative to the current state of the model. Secondly, it makes the algorithm faster by reducing the overhead of analyzing the prediction history of samples.\n- Annealing the selection bias: as the training goes on the selection becomes more random and less biased.\n\nThis approach is evaluated in on three simple data-sets: MNIST, CIFAR-10 and CIFAR-100. Although this is a very limited subset of models, the results are consistent and statistically significant, although their effect is not really huge.\n\nThe paper gives very little theoretical justification or analysis of the results but gives only the presented empirical evidence which seems to support the hypothesis on the efficacy of the approach.\n\nAnother drawback of the approach is that it introduces new hyperparameters: those governing the annealing schedule for the selection bias.\n\nSince the approach seems efficient in a relatively constrained setup, it can be reasonably expected that it might be helpful in more general situations, therefore. On the other hand, since it is only evaluated on three very similar tasks, it limits the conclusiveness of the results.\n\nThat's why I would for weak accept. In the presence of more empirical (or even theoretical) evidence, I would vote for strong accept.\n\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "BJeM3ENTYB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an interesting heuristic of batch construction from samples. Instead of the usual random sampling, the authors to sample based on some measures of the ``uncertainty”. To be specific, the uncertainty is measured as a normalized entropy estimated from a window of historical predictions. \n\nI like the idea of designing more sophisticated ways to encourage more exploration over the samples that the model is not good at. The thought is similar as active learning. It is interesting to see how similar thought can be used to improve the performance of the algorithm in the general batch gradient descent setting.\n\nOn the other hand I am not quite convinced the proposed way is truly better. The main concern is the experiments do not quite show the state-of-the-art result at all. It is not even close on MNIST, CIFAR-10 and CIFAR-100. Also those datasets are relatively small one. Can authors add results on larger datasets such as tiny image net?\n\nBesides this main concern I also have some worries about the design of the algorithm. I listed them below:\n1. The vanilla stochastic gradient descent can be roughly justified since the expectation of the stochastic gradient is the true gradient of the loss. Now with the proposed heuristic will this still be true?\n2. Is there any guarantee the algorithm can converge? It is not clear to me as the optimization proceeds the ``uncertainty” may oscillate. Is there any condition when the convergence is guaranteed?\n3. As the number of classes grows the estimation of the entropy itself is a tough problem. Is there any way to mitigate this issue other than increase the window size?\n\nAnother minor comment:\nCould the authors add more explanation on equation (4)? For example, $\\delta$ is related to the maximum entropy led by a uniform distribution, and the summation term in (4) is related to the empirical entropy.\n \n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "B1xaxyLZ9r", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes Recency Bias, an adaptive mini batch selection method for training deep neural networks. To select informative minibatches for training, the proposed method maintains a fixed size sliding window of past model predictions for each data sample. At a given iteration, samples which have highly inconsistent predictions within the sliding window are added to the minibatch. The main contribution of this paper is the introduction of sliding window to remember past model predictions, as an improvement over the SOTA approach: Active Bias, which maintains a growing window of model predictions. Empirical studies are performed to show the superiority of Recency Bias over two SOTA  approaches. Results are shown on the task of (1) image classification from scratch and (2) image classification by fine-tuning pretrained networks. \n\n+ves:\n+ The idea of using a sliding window over a growing window in active batch selection is  interesting.\n+ Overall, the paper is well written. In particular, the Related Work section has a nice flow and puts the proposed method into context. Despite the method having limited novelty (sliding window instead of a growing window), the method has been well motivated by pointing out the limitations in SOTA methods. \n+ The results section is well structured. It's nice to see hyperparameter tuning results; and loss convergence graphs in various learning settings for each dataset. \n\nConcerns:\n- The key concern about the paper is the lack of rigorous experimentation to study the usefulness of the proposed method. Despite the paper stating that there have been earlier work (Joseph et al, 2019 and Wang et al, 2019) that attempt mini-batch selection, the paper does not compare with them. This is limiting. Further, since the proposed method is not specific to the domain of images, evaluating it on tasks other than image classification, such as text classification for instance, would have helped validate its applicability across domains. \n\n- Considering the limited results, a deeper analysis of the proposed method would have been nice. The idea of a sliding window over a growing window is a generic one, and there have been many efforts to theoretically analyze active learning over the last two decades. How does the proposed method fit in there? (For e.g., how does the expected model variance change in this setting?) Some form of theoretical/analytical reasoning behind the effectiveness of recency bias (which is missing) would provide greater insights to the community and facilitate further research in this direction. \n\n- The claim of 20.5% reduction in test error mentioned in the abstract has not been clearly addressed and pointed out in the results section of the paper. \n\n- On the same note, the results are not conclusively in favor of the proposed method, and only is marginally better than the competitors. Why does online batch perform consistently than the proposed method? There is no discussion of these \tinferences from the results.\n\n- The results would have been more complete if results were shown in a setting where just recency bias is used without the use of the selection pressure parameter. In other words, an ablation study on the effect of the selection pressure parameter would have been very useful.\n\n- How important is the warm-up phase to the proposed method? Considering the paper states that this is required to get good estimates of the quantization index of the samples, some ablation studies on reducing/increasing the warm-up phase and showing the results would have been useful to understand this.\n\n- Fig 4: Why are there sharp dips periodically in all the graphs? What do these correspond to?\n\n- The intuition behind the method is described well, however, the proposed method would have been really solidified if it were analysed in the context of a simple machine learning problem (such as logistic regression). As an example, verifying if the chosen minibatch samples are actually close to the decision boundary of a model (even if the model is very simple) would have helped analyze the proposed method well.  \n\nMinor comments: \n* It would have been nice to see the relation between the effect of using recency bias and the difficulty of the task/dataset. \n* In the 2nd line in Introduction, it should be \"deep networks\" instead of \"deep networks netowrks\". \n* Since both tasks in the experiments are about image classification, it would be a little misleading to present them as \"image classification\" and \"finetuning\". A more informative way of titling them would be \"image classification from scratch\" and \"image classification by finetuning\".\n* In Section 3.1, in the LHS of equation 3, it would be appropriate to use P(y_i/x_i; q) instead of P(y/x_i; q) since the former term was used in the paragraph. \n\n=====POST-REBUTTAL COMMENTS========\nI thank the authors for the response and the efforts in the updated draft. Some of my queries were clarified. However, unfortunately, I still think more needs to be done to explain the consistency of the results and to study the generalizability of this work across datasets.  I retain my original decision for these reasons.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a new mini-batch selection method for training deep NNs. Rather than random sampling, selection is based on a sliding window of past model predictions for each sample and uncertainty about those samples. Results are presented on MNIST and CIFAR.

The reviewers agreed that this is an interesting idea which was clearly presented, but had concerns about the strength of the experimental results, which showed only a modest benefit on relatively simple datasets. In the rebuttal period, the authors added an ablation study and additional results on Tiny-ImageNet. However, the results on the new dataset seem very marginal, and R1 did not feel that all of their concerns were addressed. I’m inclined to agree that more work is required to prove the generalizability of this approach before it’s suitable for acceptance.

Decision: Reject