DASGrad: Double Adaptive Stochastic Gradient
Authors: Kin Gutierrez
Cristian Challu
Jin Li
Artur Dubrawski
Venue:  (2020)
Keywords: stochastic convex optimization
adaptivity
online learning
transfer learning

Abstract: Adaptive moment methods have been remarkably successful for optimization under the presence of high dimensional or sparse gradients, in parallel to this, adaptive sampling probabilities for SGD have allowed optimizers to improve convergence rates by prioritizing examples to learn efficiently. Numerous applications in the past have implicitly combined adaptive moment methods with adaptive probabilities yet the theoretical guarantees of such procedures have not been explored. We formalize double adaptive stochastic gradient methods DASGrad as an optimization technique and analyze its convergence improvements in a stochastic convex optimization setting, we provide empirical validation of our findings with convex and non convex objectives. We observe that the benefits of the method increase with the model complexity and variability of the gradients, and we explore the resulting utility in extensions to transfer learning. 

Under review as a conference paper at ICLR 2020
DASGRAD: DOUBLE ADAPTIVE STOCHASTIC GRADIENT
Anonymous authors Paper under double-blind review
ABSTRACT
Adaptive moment methods have been remarkably successful for optimization un- der the presence of high dimensional or sparse gradients, in parallel to this, adap- tive sampling probabilities for SGD have allowed optimizers to improve conver- gence rates by prioritizing examples to learn efficiently. Numerous applications the past have implicitly combined adaptive moment methods with adaptive prob- abilities yet the theoretical guarantees of such procedures have not been explored. We formalize double adaptive stochastic gradient methods DASGRAD timization technique and analyze its convergence improvements in a stochastic convex optimization setting, we provide empirical validation of our findings with convex and non convex objectives. We observe that the benefits of the method in- crease with the model complexity and variability of the gradients, and explore the resulting utility in extensions to transfer learning.
op-
INTRODUCTION AND MOTIVATION
Stochastic gradient descent (SGD) is widely used optimization method, and currently through backpropagation this algorithm has propelled the success of many deep learning applications. The Deep Learning community has particularly adopted variants of adaptive moment methods for SGD that specialize in high-dimensional features and non convex objectives, examples include ADA-
GRAD, ADADELTA, RMSPRPP ADAM and AMSGRAD (Duchi et al. (2011): Zeiler (2012);
Tieleman & Hinton| 2012); ; Kingma & Ba (2014); Reddi et al. (2018)). All these adaptive moment
methods relied on the efficient use the information of the geometry of the problem to improve the
rate of convergence.
In parallel to the previous ideas, adaptive probabilities methods for SGD, traditionally focusing on convex objectives, have shown advantages over its uniform sampling baselines, by allowing more efficient use of the gradient information (Zhu 2018); Shen et al. (2016); Bottou et al. (2016) Csiba
et al. 2015); Stich et al. (2017)).
Adaptive probabilities were introduced to the Deep Learning community by Hinton the diseriminative fine tuning procedure. The method was further explored in range of applic applications like object detection, reinforcement learning and curriculum learning (Shrivastava et al. (2016);
(007) through
Schaul et al. (2015); [Bengio al. (2009)). In these examples the combination of adaptive moments and adaptive probabilities methods was implicit and its analysis as a pure optimization technique was still an open question.
In this paper we analyze the asymptotic convergence properties of combining adaptive probabilities and adaptive moments. To our knowledge such family has not yet been introduced as an optimization procedure. We will refer to this family of optimization algorithms as Double Adaptive Stochastic Gradient DASGRAD.
We show that the improvements of DASGRAD depend on its flexibility to control the variance of the adaptive moments methods. We prove the theoretical guarantees and improvements in a convex setting and validate these observations empirically in convex and deep learning objectives. Finally we demonstrate the generalization properties of the algorithm with a simple extension to importance weight transfer learning.

---

Under review as a conference paper at ICLR 2020
2 ADAPTIVE GRADIENT METHODS
Notation. In order to facilitate the proofs and reading process we introduce some simplified notation that will be common the analyzed algorithms. Let a, R and M E S then the multiplication of vector a by the inverse of M will be M -'a a M. Let ya be the element-wise square root of vector a, a² the element-wise square, a/b the lement-wise division, and max b) the element- wise max of vector a and vector b. Finally for any natural n the set [I.....n) n denoted as [n].
Let T = ((x,,y:)) be a training set; let f θ x x + R be differentiable function that represents the empirical risk of an agent over T for the parameters 0 E O, with θ S I convex feasible set parameters; let St the set of positive definite matrices in Rdxd, for given matrix
M E - and parameter 0' E θ; let le. r be the projection operator over the feasible domain 0')1l-
defined by Ie (0)) = arg mingee ||
For the iterative stochastic optimization algorithm A. let in be sampled index at step drawn from the training set indices [n]. with & ~ pt and p. E A = {p E R" -; > Σ.P: = 1}. We
denote the evaluated risk f(0, v:) = f(()), the complete gradient vf(0;) ) = _ V (02) and
the stochastic gradient f.,, (6,), analogous full descent direction m. = 1/4 Σ. m. and stochastic descent direction m.,.
Stochastic Optimization Framework. To analyze the convergence of the stochastic optimization algorithm A we use the convex optimization setting where we assume that the objective function is convex with bounded gradients, that is |/ (0)]ll> I G for all E [n]] 0 E O, and finally the parameter space has bounded diameter, that is | 0o I for 0, 0' E.
For our purposes, the algorithm A at time / chooses distribution over the training set p. AT. obtains a training example i, ~ Pi and its importance weights w; = (1/n)/Pi, /p.,, then updates its parameters 0, θ using the available data at time t and the importance weights ú; to unbias the direction of the gradients. After the update, the algorithm incurs in a loss from an unknown function f(0.). To assess the performance of the algorithm after T steps measures the difference of the loss at time & and the loss for optimal fixed parameter, along the possible trajectories induced by the chosen probabilities.
the expected regret, which
= = Σ
T E, f(,))) (i.
The goal is to design an algorithm A that has sub linear expected regret R(A)r oTT), which in turn implies that the algorithm will converge on average to the optimal parameter.
Algorithm 1: Double Adaptive Methods
Input: 0, E O, step size {ax > o functions ( D.
for t= = to T' do
Choose p. A and sample is p.
Calculate gt, V S., (0:) and w. = (1 (1/n)/Pu
mi; di (gi ........ 9i,) and V. (g:1 :........99
0.41 0, ani;
mi.
O.41 II 1/2 (0c+1)
Algorithm 1 constitutes the general family of double adaptive gradient methods. This algo- rithm comprehends the classical stochastic gradient descent, adaptive moment methods family Zeiler (2012): Tieleman & Hinton (2012); Kingma & Bal (2014); Reddi al. (2018). and sec- ond order methods Duchi et al. (201 varying the averaging functions of the past gradients with Φ, O ddd and approximating the Hessian matrix with the functions vA O 5td
'The projection operator enables the algorithm to deal with constrained optimization over compact convex domains that are equivalent common regularization techniques like ridge and LASSO.
2

---

Under review as a conference paper at ICLR 2020
Adaptive Probabilities Methods. The classic stochastic gradient descent algorithm is recovered with the following step size, sampling probabilities and functions:
=a/Vt Pt, = 1/n for all t E [T], E [n]] . gt.) = 9i, th (g.............................. =I
(SGD)
Adaptive probabilities methods can be obtained simply by allowing the algorithm choose a dif- ferent probability Pr at any time t:
O4 a/ P. E for all / E [T] 01 9t.) = 9t. velg............................ gt.) =
(AP-SGD)
Significant improvements in the convergence rate of the algorithm can be obtained by cleverly choos- ing and computing such probabilities that in turn enables the algorithm manner Stich et al. (017)). Fixed importance sampling is the case when p. = for all E [T].
data in more efficient
Adaptive Moments Methods. Duchi al. propelled interest and research adaptive algorithms. In their work they noticed that SGD lacked good convergence behavior in sparse settings, and pro- posed family of algorithms that allowed the methods dynamically incorporate information about the geometry of the data Duchi et al. (201I). Following huge gains obtained with ADAGRAD, the deep learning community proposed variants based on exponential moving average functions for the like ADADELTA, RMSPROP, ADAM and most recently AMSGRAD (Zeiler (0122) Tieleman & Hinton (2022); Kingma & Ba (014)); Reddi et al. (2018))
The first algorithm ADAGRAD is obtained by the following proximal functions:
0t = 1/VE Pi. = 1/n for all t E [T], i E [n]
de (g: -9.)) = 9:,
(ADAGRAD)
v 9 diag diag(2) (2, 11
The ADAM/AMSGRAI algorithm is obtained by setting:
at = 1/VE Pa, = 1/n Vt [T] [n]
0el ... 9.. (t))
(ADAM/AMSGRAD)
Vi2 (1 32)2 0/ g.- 0,, maxx Ú, U..)
w (g gt. = diag (Üs.)
Fortunately very simple and computationally efficient way implement ADAM is given by recursion. RMSPROP is the particular case of ADAM when 3 = 0 and without maximum operator for the second moments vector, while ADAM is recovered without the maximum operator.
Double Adaptive Methods. The key idea behind both the adaptive probabilities methods and adap- tive moment methods is the efficient use of the information available in the training data improve the convergence of the algorithms. In the case of adaptive moment methods the diagonal approx- imations of the Hessian matrix use the information about the geometry of the problem while the adaptive sampling methods, the probabilities pi., prioritize the examples with the highest impact on the learning progress. As these improvements rely on complementary sources information we can combine them into a general framework described by the double adaptive methods in Algorithm]] [
To analyze the theoretical improvement guarantees of the double adaptive methods we first we ex- tend the adaptive moments convergence guarantees to the stochastic case with uniform sampling and then we compare them to the convergence guarantees using optimal probabilities.
3

---

Under review as a conference paper at ICLR 2020
3 CONVERGENCE ANALYSIS
3.1 CONVERGENCE OF ADAPTIVE MOMENTS METHODS
We first provide regret bound of ADAM for weakly convex objectives with uniform probabilities adapting the arguments Reddi et al. (2018) and Kingma & Ba (2014). Then we extend these results to the adaptive probabilities case.
Theorem 1. Let (0:) be the sequence obtained with ADAM, then the regret bound is:
T
R(ADAM)
$
Σ
1
En
| - 0-1 11V
1/4
((2//(((0)))) (0 - -1
20 (1 - 31,)
t=1
04
En
11/4 1/4 mi, 113
2(1 $1.)
t=1
>
a,
1/4 m. 11-1112+
B1,
|/V.1/((0:
-11
t=1 2(1 31.)
20 B1,)
Corollary 1.1. Following the sequence {0,]== of ADAM with step size oz a./Vt, averaging parameters $1 = 31,, $1, I 31 for all l. E [T], = 31/ /B2 < and uniform probabilities Pi, .1/n. If we assume that & has bounded diameter D. fi, (0)lloo I for all E [2]] and
0 E O, then the expected regret bound is:
D2
R(ADAM) S 20(11 - B1) En
|10,
1/21/11 12]
0v 1 + log(T)
Σ 119 1:7.A |I
+
2(1 31)2 V(1 - B2) -2) h=1
aGd
D
1112
+
20(1 - 31)3 /(1 - -3)) -)
2a(1 - B1)
3.2 CONVERGENCE OF DOUBLE ADAPTIVE METHODS
Theorem 2. Ler (0 be DASGRAD sequence, for a trajectory pt E A the regret bound is:
T
R(DASGRAD) <
1
E Ppn
|| 1/4 (0. - -0") 11V
(0.41 - -0")1²]
20
-1,))
1=1
+
E Epic
W
2
11
1/4 m 112
2(1
31.)
1=1
T
+
1=1 2(1 - 31.)
3431,
m
2 +
20x - B1.)
B1,
11/1/((0 -0: 0²)
Corollary 2.1. Following the sequence {0. of DASGRAD, step size at a/Vt, averaging
parameters B1 31,, Br. S 31 for all i E (T), 7 31/ < 1 and the optimal adaptive proba-
bilities Pi, x || milll- If we assume that O has bounded diameter and for all [T] and 0 - then the expected regret bound is:
S G
D2VT
R(DASGRAD) S 2x(1 - 31)
Eppre
|10
1/411p
ir
+
a I + log(T)
d Σ
11 1:T.h ||
T Σ
Var,
(IV. mi.l)
2(1 )
B2)(1 -7)
h=1
t=1
aGd
D2
Σ
Vis -100//111
+
2a(1 - B1)3 /(1 (1) -7)
2xx - B1)
t=1
4

---

Under review as a conference paper at ICLR 2020
3.3 CONVERGENCE COMPARISON
Adaptive moment methods can improve classical gradient descent by integrating the geometry of the problem with a diagonal approximation of the Hessian may achieve exponentially smaller bound for the expected regret with respect to the dimensions of the input data d, when dealing with sparse features or small gradients in general. As shown by Duchi et al. (2011) for the adaptive moment methods in the sparse setting, the potential component and error component of the expected regret of Corollary 1.1 each will satisfy:
Epir
11 11 = E. PI.T
Σ
1/2 Ir
Vd and EllggT
< VeT
which turn translates a much better expected regret bound than O(VdT) for classic SGD on weakly convex objectives and sparse inputs.
In parallel to the previous improvements, the adaptive probabilities methods can further speed the convergence by allowing the algorithm to evaluate the relative importance of each data point to maximize the expected learning progress, and minimize the variance of the stochastic gradient at each step. Given the optimal adaptive probabilities the error component of the expected regret in Corollary 2.1 satisfies:
T || - Σ Var, (IV2 mi,1 mi,1)
111 1 I.A.A
g 1:T.A
R=1
h=1
This shows how the optimal adaptive sampling probabilities on a convex setting can further improve the convergence rate by allowing to flexibly control the variance of the gradients.
4 DASGRAD IMPLEMENTATION
To obtain the optimal probabilities it is necessary to compute the norm of the gradient for each training sample at each step. Given the deep learning libraries today, this calculation renders op- timal adaptive probabilities methods impractical for real applications the exposition of the theoretical results we provide empirical evidence of the convergence improve- ments, to do it we use an approximation of the optimal algorithm from the double adaptive methods, following Algorithm
Still, for completeness of
Algorithm 2: DASGRAD approximation
Input: E O, functions (Φr, v ) frequency J to T do if mod J = 0 then
Compute p. E A setting p., x V|V,, 1/4 mm.ll +
Sample i ~ P: using the segment tree
Calculate gt. V f., (0:) and w.
(1/n)/Pu. /p..
m, 31m--1 (1 (1.)g. and
8201-1 -3))
ē: mar(0:-1, v.) and V. = diag(E.)
θct1 -athi, V. 1 m,
Oc+1 II 10///(((1+))) (0ct1) 11
"Current dynamical computational graph libraries compute the average gradient batches by de- fault and available workarounds are still slow. See http://ghhhaacoooyymmeeeeyyss//s and
5

---

Under review as a conference paper at ICLR 2020
5 EMPIRICAL RESULTS
In this section we provide empirical evidence of the convergence rates on classification problems using logistic regression and deep neural networks, using ADAM, AMSGRAD, and DASGRAD.
Logistic Regression: For the convex setting we solve two classification problems with L2 regular- ization. For the non sparse feature experiment we use the MNIST digit dataset, which is composed of 60, 000 images of 288 x 28 hand written digits. For the sparse feature experiment we use the IMDB movie rating dataset which is composed of 25, 000 highly polar movie reviews and the sentiment label for the review Maas et al. 12011)
Neural Networks: For the non convex setting we perform one experiment, we use the CIFARIO dataset, which is composed of 60, 000 colour images of 32 x 32 pixels labeled in 10 classes. For this multiclass classification problem we use convolutional neural network following the SMALL- CIFARNET architecture, consisting of two convolution filters combined with max pooling and local response normalization, followed by two fully connected layers of rectified linear units Krizhevsky
ct al. 2012))
100
Train Accuracy Difference
2.2
DASGrad
Mean Train Accuracy
20
DASGrad AMSGrad
AMSGrad
DASGrad ADAM
10
Adam
90
Mean
1.8
DASGrad
AMSGrad
1.6
Adam
-20
75
500
1000
1500
0
500
1000
1500
500
1000
1500
Iterations
Iterations
Rteration
100
Train Accuracy Difference
DASGrad
Mean Train Accuracy
DASGrad AMSGrad
AMSGrad
95
20
DASGrad ADAM
Train
0.6
Adam
10
0.5
85
DASGrad
0-
AMSGrad
-10-
Adam
75
-20
0
500
1000
1500
500
1000
1500
500
1000
1500
Iterations
Iterations
Reeaaionn
100
Train Accuracy Difference
Mean Train Loss
0.8
AMSGrad
DASGrad
Mean Train Accuracy
95
20
0.6
Adam
90
0.4
DASGrad
0.2
80
AMSGrad
-20-
DASGrad AMSSGad
Adam
DASGrad ADAM
0.0
75
5000
7500 10000 12500 15000 Rterations
5000
7500
10000
12500 15000
5000
7500
10000 12500 15000 Iterations
Rerations
Figure 1: Trajectories in convex and deep learning settings. First row 100 logistic regressions on MNIST, second row 100 logistic regressions on IMDB, third row 10 convolutional neural networks on CIFARIO. We show the mean over the trajectories for training loss (left), training accuracy (center), and the accuracy improvement of DASGRAD with respect to AMSGRAD and ADAM with 95% confidence intervals (right).
Foor both experiments, use a batch of size 32, with probability update every 10 steps, and the step size a₁ = Vf. We set 81 = 0.9, 32 = 0.99, and choose a through a grid search. For the MNIST dataset, for all three optimizers, the optimal learning rates are a = 0.01. For the IMDB dataset, we find the optimal learning 0.006 for AMSGRAD, and a = 0.02 for DASGRAD. *For the experiment we use a batch size 32, with a prohability update every 300 steps, and step size of ae a/ Vf. We set 81 0.9, 32 = 0.99, and choose through grid search, for which the optimal learning rate for all optimizers is 0.001.
rates to be a 0.005 for ADAM,

---

Under review as a conference paper at ICLR 2020
From the comparison in Figure I observe that in all cases the DASGRAD optimization algorithm outperforms its adaptive moment counterparts represented by ADAM and AMSGRAD, as predicted by the theoretical analysis. The improvement is more significant for the IMDB dataset than it is for the MNIST dataset. From Figure we can see that DASGRAD continues to outperform ADAM and A AMSGRAD in the deep learning setting. These results reinforce the previous statement that the benefits from DASGrad increase with the complexity of the data and the models.
DISCUSSION
6.1 CONVERGENCE IMPROVEMENTS AND VARIANCE GRADIENTS
To further explore the relationship between variance and the improvements to the convergence rate of the DASGRAD algorithm, we implemented an online centroid learning experiment. Because of the linear relationship between the features and the gradients, variance. For this experiment, the empirical risk and gradients will given by (,)
are able to explicitly control their
Σ 0 x,1? and (f(0,x,) - x4,-
As we can see from Figure the greater the variance of the gradients, the greater the benefit that one can obtain from an adaptive probabilities method such DASGRAD in convex objectives, since those probabilities will prioritize the data points with the most learning potential.
VariX) 0.0 0.03
VarúX 0.06
15.0
4p5GD
12.5
Regret
10.0
25
0.0
Iterations
Var(X = 0.15
Var(X) =
15.0
ap5GD
SpS
12.5
Regret
10.0
25
0.
Iterations
Figure 2: Trajectories of 100 random seeds, for the online centroid learning problem with different variance for the features. Enhanced improvements of adaptive methods with higher variance of the gradients.
6.2 FLEXIBLE CONTROL OF VARIANCE
Recent insights on the generalization properties of minibatch SGD for non convex objectives suggest that higher variance gradients tend to converge to flatter regions of the loss surfaces (Keskar et al. (2 Applications like curriculum learning that shape the learning procedure, by gradually mak- ing the task more difficult through importance sampling, may allow to maintain higher variance of the gradients for longer steps, this combined with the previous intuitions offers explanation of the mathematical basis of its success.
While curriculum learning is contrary to the optimal probabilities of the double adaptive methods for convex settings, the underlying principle of flexible control of the variance of the gradients operates the mechanism behind both procedures. This observations strengthens the argument that improving our understanding of the implicit optimization techniques in this algorithms can also improve our understanding that so far has relied mostly on intuitive explanations of their success.
7

---

Under review as a conference paper at ICLR 2020
6.3 IMPORTANCE WEIGHT TRANSFER LEARNING
When the training T and test T' set do not share the same distribution, we may face a distribution mismatch problem. The DASGRAD algorithm is compatible with the re-weighting correction technique Elkan (200T): [Bowyer et al. (201T) as we can set the importance weights wy for any trajectory of distributions Pt. to unbias the gradients for the test distribution instead of the training.
R(DASGRAD)
Σ Ep [f((0.) mingl [f:(]]]
Epn [w,,)((0,) ming (s((0))
To test generalization properties of the DASGRAD algorithm empirically, we unbalanced MNIST training data set by reducing ninety percent the observations from the and digit. We set the importance weights to wi, = (|L: m /p.,, where [L] is the count of the label L associated with index i in test over m, the number of test samples. As in Figure 3 using DASGRAD with the correct importance weights has the desired generalization properties when facing a domain shift.
100
DASGrad
Mean Test Accuracy
Mean Test Loss
AMSGrad
90
Adam
80
70
DASGrad
AMSGrad
60
Adam
50
2000
500
1000
1500
Iterations
Iterations
Figure 3: Trajectories 20 random seeds for 2,000 iterations in convex optimization settings. Mul- ticlass logistic regression on unbalanced MNIST dataset. We show the mean over 20 trajectories of training loss (left), training accuracy (center), and the improvement in accuracy of DASGrad with respect AMSGRAD and ADAM with 95% confidence interval (right)
7 CONCLUSION
Capability of learning from data efficiently is a prerequisite for practical success of complex learning models across various problem settings and application contexts. We have shown how double adap- tive stochastic gradient descent methods enable efficient learning in generalizable manner, while ensuring convergence improvement. We observed that DASGRAD algorithm outperforms currently prevalent variants of adaptive moment algorithms such as ADAM and AMSGRAD overall, in the context of the number of iterations required to achieve comparable performance, under the thcoret- ical convergence guarantees in stochastic convex optimization setting. With empirical validation in convex and non convex settings, we have shown that the advantages of DASGrad become more prominent with the increasing complexity of data and models, and with more variance in the gradi- ents. We have also broadened our results to demonstrate generalization properties of our approach and its extensions transfer learning, well intuitive connections other learning scenarios.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th Anmual International Conference on Machine Learning. ICML '09, pp. 41-48, New York, NY, USA, 2009. ACM. ISBN 978- 978-1-60558-516-11 doi: 10.1145/1553374 1553380. URL http: acm. org htt.://........1111133333.......
Leon Bottou, Frank E. Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning, 2016. URL htttttt//aaaiiii http://..rgg////////////// 04838 cite Mixiv:1666004833
Kevin W. Bowyer, Nitesh V. Chawla, Lawrence O. Hall, and W. Philip Kegelmeyer. SMOTE: synthetic minority over-sampling technique. CoRR, abs/I 100..813,, 2011.
Dominik Csiba, Zheng Qu, and Peter Richtrik. Stochastic dual coordinate ascent with adaptive prob- abilities. In Proceedings of The 32nd International Conference on Machine Learning, ICML' 15, 2015.
John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res., 12:2121 2159, July 2011. ISSN 1532-4435. URL http://dl- acm.
http:/ioon..mmmmmm1111111 2021066
Charles Elkan. The foundations of cost-sensitive learning. In Proceedings of the 17th International Joint Conference on Artificial Intelligence Volume 2, IJCAT'O1, pp. 973-978, San Francisco, CA, USA, 2001. Morgan Kaufmann Publishers Inc. ISBN 1-55860-81245 978-1-558-60812 URL http::ttp/// //1l awa.laacm.orgg. tia ion, c 6421 94. 642224
Geoffrey E. Hinton. To recognize shapes, first learn to generate images. Progress in brain research, 165:535-47, 2007.
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Pe- ter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. CoRR, abs//609,.4886, 2016.
Diederik P. Kingma and Jimmy Ba. ADAM: A method for stochastic optimization. CoRR, abs/ (bb//141,,6900, 2014. URL BRRp:tt:://arxt wttp://..ggaabb///2// 6980
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convo- lutional neural networks. In Proceedings of the 25th International Conference on Neural Infor- mation Processing Systems- - Volume 1, NIPS' 12, pp. 1097-1105, USA, 2012. Curran Associates 2999257
Inc. URL http::/t: amm. www..orww....................... ci tat ion wtt.fffmm=22999
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies Volume 1, HLT '11, pp. 150. Stroudsburg, PA, USA, 2011. Association for Computational Linguistics. ISBN 978-1-932432-87-9 URL http: //dl acm. www..o/gggiitttnnnccoommmmmom cfm? -d00mmm20000222222222222 2002491
Sashank J. Reddi, Satyen Kale, and Sanjiv Kumar. On the convergence of ADAM and beyond. In International Conference on Learning Representations, 2018. URL
Tom Schaul, John Quan, loannis Antonoglou, and David Silver. Prioritized experience replay.
CoRR, abs/151 1.05952, 2015.
org/ abs//5l1.............. 05952
Zebang Shen, Hui Qian, Tengfei Zhou, and Tongzhou Mu. Adaptive variance reducing for stochas- tic gradient descent. In Proceedings of the 32nd International Joint Conference on Artificial Intelligence, LJCAI' 2016.
Abhinav Shrivastava, Abhinav Gupta, and Ross B. Girshick. Training region- object detectors with online hard example mining. CoRR, abs 1 604.444 03540)
2016. PRLaatp::/ /tttp::////ii....// org/

---

Under review as a conference paper at ICLR 2020
Sebastian U. Stich, Anant Raj, and Martin Jaggi. Safe adaptive importance sampling. CoRR, abs/1711.02637, 2017. URL BRLhhtt::/// arxia. org/ http/bbs//1111 02637
T. Tieleman and G. Hinton. Lecture :.S-RMPPrpp: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, 2012.
Matthew D. Zeiler. ADADELTA: an adaptive learning rate method. CoRR, abs/1212.5701, 2012. URL (0t.::/p.//aar.....o//////22.2..
R. Zhu. Gradient-based Sampling: An Adaptive Importance Sampling for Least-squares. ArXiv e-prints, March 2018.
10

---

Under review as a conference paper at ICLR 2020
A APPENDIX
A.1 PROOF OF THRORFM 2
The proof of Theorem assumes convex differentiable objective function f. bounded diameter for the parameters, and bounded norm of the gradients for any trajectory of probabilities p: E AT
Proof. We build an upper bound of the expected regret using the convexity of the loss:
f())) f(0") I (ge. 0, 0*) E, K gi:. 0, 0*)]
While using DASGRAD the update the parameter will given by the stochastic update dependent on the training example i, and the current parameter 0ş:
O
IT e v. (0,+) (0++1) e v./a ra (0, 04 V., 1/2 m,.) arg min || V//1(0, a, V,-1/2
1/2 m.:)
oee
To bound the expected regret of the algorithm, we use the fact that:
δ.t1 - 0" = =(,, 0*) aew; V..
[V 1/4 (0.41 1-
1V 1//111(8 (0, 107--0% 01 20, (B1, m + (1 B1, )gk, 0. - 0-) + 0f w
1/4 (6: 1'11 wa, { m,,, 0: 0" ) + 0 w/
1/4
m; 1|?
= =1
V-114 1/4 m:,! ||2
We identify the first three components the potential, the immediate cost, now with extra terms associated to the moving average, and the error.
Lemma 1. For any M S and convex feasible set θ C Rd with the projection operator Ile.M let 01 = IT. (21) and u2 = -TT (22) then:
| (u1 u2) - | (21 - x2)
Taking the expectation at time t, and using the extended norm reduction property of the projections from Lemma I
we obtain the following inequality:
Ep, (0 0)1 0,
- Ep.
(1, 100--0111" 0,
22aws, (B1, mi.,, + (1 - B1,)9i. O, 00)(0)
Ep.
+ o
(w") 11V 1/4 m.. 11 [o.
Since wi is such that the interior product will be unbiased, then:
Ep.
|| 1/4
1/11//000011 (0.-1
- 1 0,
s Ep.
[1
1//1/1((0, (0, 00) 0.
20r (i, m:-1 + (1 - 31, )gt, -2-0% 0*)
+ a w 1/4 m; 11 0,
Finally rearranging the terms, summing until time T and taking expectations:
R(DASGRAD)
Σ
21 31,) Epi
[
1/4//4(00
0-)12-
- 14
(0++
-0)1100111
T
+
04
E
Epi.t
W VI 1/4 m, 112) (1)
2(1
B1.)
t=1
T B1,
T
V.
1/4
m
+
B1.
|| (0, - -
2(1 1=1
B1.)
20,
B1r)
1=1
Last line is Cauchy-Schwarz and Young's inequality applied to the inner product of the extra terms associated with the moving average in the immediate cost.
11

---

Under review as a conference paper at ICLR 2020
A.2 PROOF OF COROLLARY 2.1
Proof. The proof of Corollary is in the line of the improvements provided by Reddi et al. to the convergence proof of Kingma & Ba for ADAM, we adapt the arguments the stochastic case. We assess separately each component of the expected regret from Equation 1
Lemma 2 addresses the potential, Lemma the error, and Lemma and Lemma the moving average terms. The proof of Corollary 2. is consequence of all the previous Lemmas using the optimal probabilities while Corollary is the case with uniform probabilities.
Following the sequence {0: - of DASGRAD, with step size a, a/ Vt, averaging parameters 31 = 31, and 31, Σ 31 for all E [7] and y = -1/v < 1. and bounded diameter D for @ and [i (0) I G for all E [T) and 0 E e.
Lemma 2. From Equation 7 the potential component will satisfy:
T
D2VT E Ep.r
Σ 20 (1 B1,) E Pi:e
[V 1/4 (0, -0')1 -11V -0=10011
-0011]]
[16%//
1
2a(11 B1)
Proof
One can decompose the potential in the following manner:
E PI:
|| (0.
-00)11 1 11V/1//100 (0+11-00)))) 00)11
20, (1
-B1,)
Ep,
111 - 00)11 2ar(1 - -31 Epior |115/1/4
1111//(01++ (0r+1 0 01
201 (1
31)
1
||V:1/4000 (00-11111
+
E Pp:
E Ep
2(1 31)
at
ax-1
1=2
Epi
|102//1
11
2a (1 - B1)
T
+
2(1 - B1)
1
Σ
ag
E
Epic
|
① D
2
112]
a4-1
1
Epia-n
E
|111//1 o [
1=2
D2
1
<
Epp
[||0;,
112]+ +
Epp [1] 11]
2(1 31)
01
at
a4-1
t=2
2ar - 31) Pi:T
D2
E
||01//11
= 2a(1 - B1)
D2
Epiir
E
[108%/411 ||11/1122
The first inequality comes from rearranging and the definition of 31,, the second inequality comes from the bounded diameter assumption applied to each entry 0 and using the Hadamard's d's product to represent the original matrix multiplication, the third inequality comes from the defini- tion of b. max( Us. Vs,), ). the last equality comes from the property of the telescopic sequence. This completes the proof of Lemma 2
'The third inequality is of particular importance since Reddi et al. showed that it is one of the main flaws in the convergence analysis of ADAM and RMSPROP, and provided a simple fix to the adaptive moment methods that guarantees the non increasing property needed to achieve the telescopic sequence upper bound.
12

---

Under review as a conference paper at ICLR 2020
Lemma 3.
From Equation
7 the error component, once evaluated in the optimal probabilities pr
will satisfy:
Epe W m. 1| 0. = En [ m. 1? 0: Varn (112) m..l)
Proof
Creating lower bound with Cauchy-Schwarz and showing that it is achievable with the optimal probabilities pr, IIV, 1/4 m.,ll-
Lemma 4. The first component of the extra terms associated with the moving average in Equation
7 Twill satisfy:
Σ
0331,
V1 1/4
m1-1l1
aGd
20 B1.)
2a(1 31)3 V11 32)(1 -
Proof
Following very similar arguments as those from Lemma 6 we can get:
T
3r,31
1/4
1112 <
Σ
3 g
20 (1 (1.)
2(1 - 3,)2 (132) (1 - 7) 1=1
t=1
aGd
T 1111
2(1 (11-
2a - 31 - -1)) -)
This completes the proof of Lemma
Lemma 5. finish the second component of the extra terms associated with the moving average in Equation 7 will satisfy:
Σ t=1
31 |IV|//(0, 0-]
D2
110///110//11
Vt.l
20 (1 -3,)
2a(1
-B1)
1=1
Proof
31. 20 (1 31.)
V. (0 0-) <
Σ
31.
100///
1/4
3 DDOl
20r
(1
31,)
[=1
2
Vt
B1.
110 111
2a 1=1
-
31.)
D
T
VisT 1 -100,/1111 112
$
2a
(1)
(=1
This completes the proof of Lemma
13

---

Under review as a conference paper at ICLR 2020
Lemma 6. From Equation 7 the evvor component. once evaluated in the optimal probabilities pa. and the total error will satisfy that:
αx
1/4 mi; 1|2 0:
Ep:
2(1 (1,)
2(1 81)2 /
a' log
B2) 7)
Σ
11
1.T.B
Σ
T
Varn
(11V21 mi:ll
h=1
Proof
For Lemma we follow Kingma & Ba, for every element time of the enror component:
at 2(1-31,)
En
1/4 m.,!
m,,11?]
- 2(1-31,)
at
En
1/4
m.
|12 81 (1), 9t. 112
at
= 2(1-31.)
En
Ure
1===)1(1) (1) gs. 112
En
2(1-B1,)
Vūr
19
En
(1(t),
(t),,
Ui,
11
81
2(1 - 31)
UJ+
119. 113
$
2(1 - B1) En
04
89-1
198 9.,
8
11::~11
<
E,,
2(1 - 31 32)
3/-
19:
a
En
7 -
1g:
ll1
2(1-8,) /1 32)
The first inequality follows the definition of the auxiliary vectors ê. max( Vz), the second inequality comes from the non negativity of 31 (1))). The third and fourth inequality comes from the decreasing property 31 Br, and B1, B1, and the property the geometric sequence. The fifth inequality comes from the non negativity of 32 and l|g,, 1|2, the last equality uses the definition of the step size. Using induction one can show that:
T
T
[ r=t
T (1r.111
Or
Bn
1/4 m;,1 12
<
E,
2(1
31,)
2(1
3.)2
1/1
(1
32)
VF
(2)
14

---

Under review as a conference paper at ICLR 2020
Continuing the proof of Lemma 6 let k
from Equation 2 we have that:
2(1-B,)² )a (1-
Σ
En
1/4 | mi,
1|2
t Bn
9
111
2(1 (1,)
b
T
Σ 3
T
k
11
VT
1=1
9 ||
Σ
vt
r=t
T
T
9
|11
1
Σ )tT
Vi
119t.111
(1
VF
k
g fichh
Vi
h=1
d
T
T 1
gichhl
[ t
hml
1=1
2
A 1og(1og(1
n
gicke
The first equality comes from a change of indexes, the second inequality is an upper bound for the arithmetic sequence that begins at the third inequality is upper bound for the geometric sequence, the fourth inequality comes is an application of Cauchy-Schwart inequality, finally the fifth inequality is an upper bound for the arithmetic sequence.
T
T
Epp
Wh.
1/4 mi:
0,
En
1/4 mi. 112
0.
Varn (111) 1/4 mi. /mml///mmll) Il)
av1 log
d
T
111
|1
Σ
Varn (11 mi: Il)
2(1 31) /2
31
h=1
With the optimal probabilities pi., we complete the proof of Lemma
15

---

Under review as a conference paper at ICLR 2020
Finally can combine the results from Lemma to
S and obtain the following bound for the
expected regret of the general double adaptive algorithms:
Corollary 2.1
R(DASGRAD) S DVVT
20(1 - 31) Epir |101/11/2
+
av 1 + log(T)
d Σ
11g 1:TTA ||
T Σ
Var,
(IV= mi:ll)
2(1 ) (1 B2) -7)
h=1
1=1
aGd
D
Vis - -1 00 1//
+
-1 20(1 - /1 (1--1/111)))) -7)
20 -1)
t=1
16


Reviews:
[{"review_id": "SyxiU5SatH", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThis paper explores the combination of adaptive moment methods and adaptive data sampling procedures for stochastic optimization. The paper presents standard analysis in the stochastic convex setting which raised some question for me (presented below). The empirical results are promising but in my opinion can be further improved through minor changes to experimental procedure.\n\n\nOverall:\n\n1) There are a few instances of undefined notation. For example, $\\beta_1(t)$ in the definition of Adam/AMSGrad or $V_t$ (the latter I assume is the average over all datapoints?).\n\n2) Theorem 1 is stated very informally with some notation left unexplained (e.g. V_t above or E_n[...]). Further, it is not clear to me whether the authors are proving results in the online convex optimization setting or the stochastic convex optimization setting (though, there is some equivalence between them). Finally, I am concerned by the presentation of Theorem 1 and Corollary 1.1 as it has been shown by Reddi et al. that Adam does not have bounded regret in general without algorithmic changes. In particular, bimodal gradient noise distributions can be designed such that the average Regret is unbounded (see Reddi et al. for details). While these issues can be addressed through modifications to the standard Adam algorithm I am unable to find a suitable discussion within the paper.\n\n3) Theorem 2 introduces DASGrad which has not yet been clearly defined (Algorithm 1 depends on averaging functions of the past gradients for flexibility --- which averaging functions are being used here?). Once again, the notation Var_n appears whose definition is not clear to me. Additionally, could you provide a citation for the optimal adaptive probabilities and clearly state under which conditions these adaptive probabilities are optimal?\n\nThe crux of Theorem 2 depends on the informal statements present in Section 3.3, in which it is claimed that the variance of the second order moments are sufficiently large to dominate the sum of gradient histories. I am having a hard time seeing how this would be true generally and I believe that further theoretical analysis is necessary to justify this discussion.\n\n4) The introduction of Algorithm 2 felt like a bait-and-switch. This algorithm seems intuitively sensible and uses the same check-pointing ideas that have been introduced in recent variance reduction methods (e.g. [1]). However, this is a different algorithm which is not analyzed in this paper. \n\nAdditionally, the fourth line of Algorithm 2 says to compute p_t lying in the simplex --- I believe this is a typo? Further, could you explain the introduction of epsilon in line 4. This seems unnecessary given the propto symbol but I am worried I am missing something.\n\n5) The empirical results introduce an additional deviation from the proposed algorithm by using batch sizes of 32. Could the authors please explain how the adaptive sampling probabilities are used in this setting? (My apologies if this is obvious, I am not overly familiar with such methods).\n\nOverall I felt that the empirical results were fairly promising but some questions remained for me. The toy experiments in section 6.1 were illustrative but seem to only consider adaptive probabilities without the adaptive moments?\n\nSection 6.3 was also an interesting experimental setup which is well suited to DASGrad but I felt that the baseline comparison was unfair. DASGrad was provided with the ground-truth balance statistics in the importance weights while the baselines were trained without any correction. I would recommend that the authors explore combinations of enabling balance correction in the baseline methods (possibly through reweighting the loss according to the balance statistics) or requiring DASGrad to compute the optimal probabilities through the adaptive moments as suggested throughout the rest of the paper.\n\nMinor:\n\n- Pg 3, typo \"we first we extend\"\n\n\nReferences\n\n[1] Accelerating Stochastic Gradient Descent using Predictive Variance Reduction, Rie Johnson and Tong Zhang\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "Hye3nf-ptr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the doubly adaptive stochastic gradient method (DASGrad) is introduced via augmenting adaptive moment methods with adaptive (as opposed to uniform) probabilities for data sampling. The convergence of the proposed method is analyzed in terms of regret bound and is compared to similar results for ADAM. The method is validated with experiments on both convex and non-convex objectives as well as in applications to transfer learning.\n\nThe paper is well written and provides novel, to the best of my knowledge, theoretical results on the convergence of adaptive moment methods when augmented with non-uniform data sampling, which is quite interesting. However, my main concern is the proposed implementation of non-uniform sampling. In particular, the adaptive probability of every training sample is set proportional to its gradient and, in theory, this has to be updated at every iteration. This essentially means that the full batch gradient is available at every iteration and, since the full gradient is available, wouldn’t it be better to use full batch gradient descent instead?\n\nThe proofs in the appendix repeat the known results in detail, while the definitions and proofs of the concepts related to the main contribution of this paper are rather short. In particular:\n1. The expectations are not formally defined. The expectation with respect to p_{1:T} must be with respect to the join of all distributions from every iteration (?); sometimes there is conditioning on parameters and sometimes it disappears (proof of Lemma 1). This should be clarified.\n2. It would also be helpful to see more detailed proof of Lemma 3 since it is not obvious and is the key ingredient of the presented results. \n3. As a sanity check, I would expect that the results of Th. 1 and Th. 2 (or Corollary 1.1 and Corollary 2.1) should coincide when p is set to the uniform distribution. Is this correct and, if yes, can one show this?\n\nLess important questions:\n1. In Algorithm 2, there is a parameter J which controls the frequency of computation of optimal probabilities. How is this parameter set and what are the tradeoffs? After the parameters are updated, does it actually help to use such probabilities from the previous step when the parameters are in fact different?\n2. Why parameter epsilon is introduced in Algorithm 2? How is it set?\n3. Gradient aggregation methods are mostly criticized for their linear runtime in the number of examples n (see, e.g., Bottou et. al. 2016), while adaptive moment methods are widely used in practice since their runtime doesn’t depend on n. Since the proposed method is also linear in n, it would be interesting to see how it compares to aggregation methods in practice.\n4. Conceptually, what is the difference of the results in Theorem 1 and Corollary 1.1 from the previous results in Kingma & Ba (2014) and Reddi et. al. (2018)?\n5. In Lemma 4, there is unnecessary \\alpha_t in the denominator. Does it change the final result?\n6. To improve the runtime of the proposed algorithm, can one develop any approximation of optimal probabilities that wouldn’t require full data pass?\n\nMinor:\n1. p. 3, “Duchi et. al.” is this a reference?\n2. functions phi and psi are called proximal; are they related to proximal operators?\n3. typos: p. 3, “we first we”; p. 5, “an may”\n4. p. 2, it is AMSGrad, not ADAM; betas aren’t defined\n5. Inconsistent numbering: Theorems 1 and 2, but Corollaries 1.1 and 1.2\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "HklOgsSRFr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In the paper, the authors propose a general form that covers most stochastic gradient methods so far, e.g. stochastic gradient descent, Adam,  or adaptive probabilities methods. Then, they also provide a convergence analysis of convex problems.  In the experiments, they compared the proposed DASGRAD method with Adam, AMSGrad or SGD. Experimental results show that the proposed method converges faster than compared methods.  Following are my concerns:\n\n1) It is important to cite properly. There are some related works missing in the paper. e.g.,  [1][2][3] for adaptive methods, and [4] adaptive sampling methods.\n2) In corollary 1.1, the rightmost term is in the order of \\sum_t=1^T \\sqrt{t}, which should be O(T\\sqrt(T)). If we divide both sides by T, the right side of the inequality is still upper bounded by \\sqrt{T}, it is not convergent. Otherwise, please correct me and show the convergence rate of the proposed method.\n3) The same question in Corollary 2.1 about the convergence result. \n4) Some variables are not defined clearly in the paper, e.g., \\bar g and  \\beta_{1_{t}}  in corollary 1.1.\n5) Algorithm 2 requires to compute p every J iterations. How about the time complexity of sampling? Please also show the convergence regarding time in the experiments.  \n\n[1] Zhou, Dongruo, et al. \"On the convergence of adaptive gradient methods for nonconvex optimization.\" arXiv preprint arXiv:1808.05671 (2018).\n[2] Chen, Xiangyi, et al. \"On the convergence of a class of adam-type algorithms for non-convex optimization.\" arXiv preprint arXiv:1808.02941 (2018).\n[3]Zaheer, Manzil, et al. \"Adaptive methods for nonconvex optimization.\" Advances in Neural Information Processing Systems. 2018.\n [4]Csiba, Dominik, Zheng Qu, and Peter Richtárik. \"Stochastic dual coordinate ascent with adaptive probabilities.\" International Conference on Machine Learning. 2015.\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers were confused by several elements of the paper, as mentioned in their reviews and, despite the authors' rebuttal, still have several areas of concerns.

I encourage you to read the reviews carefully and address the reviewers' concerns for a future submission.
Decision: Reject