Stochastic AUC Maximization with Deep Neural Networks

1 INTRODUCTION . Deep learning has been witnessed with tremendous success for various tasks , including computer vision ( Krizhevsky et al. , 2012 ; Simonyan & Zisserman , 2014 ; He et al. , 2016 ; Ren et al. , 2015 ) , speech recognition ( Hinton et al. , 2012 ; Mohamed et al. , 2012 ; Graves , 2013 ) , natural language processing ( Bahdanau et al. , 2014 ; Sutskever et al. , 2014 ; Devlin et al. , 2018 ) , etc . From an optimization perspective , all of them are solving an empirical risk minimization problem in which the objective function is a surrogate loss of the prediction error made by a deep neural network in comparison with the ground-truth label . For example , for image classification task , the objective function is often chosen as the cross entropy between the probability distribution calculated by forward propagation of a convolutional neural network and the vector encoding true label information ( Krizhevsky et al. , 2012 ; Simonyan & Zisserman , 2014 ; He et al. , 2016 ) , where the cross entropy is a surrogate loss of the misclassification rate . However , when the data is imbalanced , this formulation is not reasonable since the data coming from minor class have little effect in this case and the model is almost determined by the data from the majority class . To address this issue , AUC maximization has been proposed as a new learning paradigm ( Zhao et al. , 2011 ) . Statistically , AUC ( short for Area Under the ROC curve ) is defined as the probability that the prediction score of a positive example is higher than that of a negative example ( Hanley & McNeil , 1982 ; 1983 ) . Compared with misclassification rate and its corresponding surrogate loss , AUC is more suitable for imbalanced data setting ( Elkan , 2001 ) . Several online or stochastic algorithms for AUC maximization have been developed based on a convex surrogate loss ( Zhao et al. , 2011 ; Gao et al. , 2013 ; Ying et al. , 2016 ; Liu et al. , 2018 ; Natole et al. , 2018 ) . However , all of these works only consider learning a linear predictive model . This naturally motivates the following question : How to design stochastic algorithms with provable guarantees to solve the AUC maximization problem with a deep neural network as the predictive model ? In this paper , we make some efforts to answer this question . We design two algorithms with stateof-the-art complexities for this problem . Based on a surrogated loss of AUC and inspired by the min-max reformulation in ( Ying et al. , 2016 ) , we cast the problem into a non-convex concave minmax stochastic optimization problem , where it is nonconvex in the primal variable and concave in the dual variable . This allows us to leverage the inexact proximal point algorithmic framework proposed in ( Rafique et al. , 2018 ) to solve stochastic AUC maximization with a deep neural network . However , their algorithms are limited for stochastic AUC maximization with a deep neural network due to three reasons . First , their algorithms are general and do not utilize the underlying favorable property of the the objective function induced by an overparameterized deep neural network , which prevents them from designing better algorithms with faster convergence . Second , these algorithms use a polynomially decaying step size scheme instead of the widely used geometrically decaying step size scheme in deep neural network training . Third , the algorithm in ( Rafique et al. , 2018 ) with the best attainable complexity only applies to the finite-sum setting , which needs to go through all data at the end of each stage and are not applicable to the pure stochastic setting . To address these limitations , we propose to leverage the Polyak-Łojasiewicz ( PL ) condition of the objective function for AUC maximization with a deep neural network . The PL condition ( or its equivalent condition ) has been proved for a class of linear and non-linear neural networks ( Hardt & Ma , 2016 ; Charles & Papailiopoulos , 2017 ; Zhou & Liang , 2017 ) . It is the key to recent developments that prove that ( stochastic ) gradient descent can find a global minimum for an overparameterized deep neural network ( Allen-Zhu et al. , 2018 ; Du et al. , 2018b ) . It is also observed in practice for learning deep neural networks ( Li & Yuan , 2017 ; Kleinberg et al. , 2018 ) . From an optimization perspective , the PL condition has been considered extensively for designing faster optimization algorithms in the literature ( Karimi et al. , 2016 ; Reddi et al. , 2016 ; Lei et al. , 2017 ) . However , there still remains a big gap between existing algorithms that focus on solving a minimization problem and the considered min-max problem of AUC maximization . It is a non-trivial task to leverage the PL condition of a non-convex minimization objective for developing faster primal-dual stochastic algorithms to solve its equivalent non-convex concave min-max problem . The main theoretical contributions in this paper are to solve this issue . Our contributions are : • We propose a stochastic algorithm named Proximal Primal-Dual Stochastic Gradient ( PPD-SG ) for solving a min-max formulation of AUC maximization under the PL condition of the surrogated AUC objective with a deep neural network . We establish a convergence rate in the order of O ( 1/ ) , which is faster than that achieved by simply applying the result in ( Rafique et al. , 2018 ) to the considered problem under the PL condition , i.e. , O ( 1/ 3 ) and O ( n/ ) with n being the size of training set . • In addition , we propose an AdaGrad-style primal-dual algorithm named Proximal Primal-Dual Adagrad ( PPD-Adagrad ) , and show that it enjoys better adaptive complexity when the growth of cumulative stochastic gradient is slow . This is the first time an adaptive convergence of a stochastic AdaGrad-style algorithm is established for solving non-convex concave min-max problems . • We evaluate the proposed algorithms on several large-scale benchmark datasets . The experimental results show that our algorithms have superior performance than other baselines . To the best of our knowledge , this is the first work incorporating PL condition into stochastic AUC maximization with a deep neural network as the predictive model , and more generally into solving a non-convex concave min-max problem . Our results achieve the state-of-the-art iteration complexity for non-convex concave min-max problems . 2 RELATED WORK . Stochastic AUC Maximization . Stochastic AUC maximization in the classical online setting is challenging due to its pairwise nature . There are several studies trying to update the model each time based on a new sampled/received training data . Instead of storing all examples in the memory , Zhao et al . ( 2011 ) employ reservoir sampling technique to maintain representative samples in a buffer , based on which their algorithms update the model . To get optimal regret bound , their buffer size needs to be O ( √ n ) , where n is the number of received training examples . Gao et al . ( 2013 ) design a new algorithm which is not buffer-based . Instead , their algorithm needs to maintain the first-order and second-order statistics of the received data to compute the stochastic gradient , which is prohibitive for high dimensional data . Based on a novel saddle-point reformulation of a surrogate loss of AUC proposed by ( Ying et al. , 2016 ) , there are several studies ( Ying et al. , 2016 ; Liu et al. , 2018 ; Natole et al. , 2018 ) trying to design stochastic primal-dual algorithms . Ying et al . ( 2016 ) employ the classical primal-dual stochastic gradient ( Nemirovski et al. , 2009 ) and obtain Õ ( 1/ √ t ) convergence rate . Natole et al . ( 2018 ) add a strongly convex regularizer , invoke composite mirror descent ( Duchi et al. , 2010 ) and achieve Õ ( 1/t ) convergence rate . Liu et al . ( 2018 ) leverage the structure of the formulation , design a multi-stage algorithm and achieve Õ ( 1/t ) convergence rate without strong convexity assumptions . However , all of them only consider learning a linear model , which results in a convex objective function . Non-Convex Min-max Optimization . Stochastic optimization of non-convex min-max problems have received increasing interests recently ( Rafique et al. , 2018 ; Lin et al. , 2018 ; Sanjabi et al. , 2018 ; Lu et al. , 2019 ; Jin et al. , 2019 ) . When the objective function is weakly convex in the primal variable and is concave in the dual variable , Rafique et al . ( 2018 ) design a proximal guided algorithm in spirit of the inexact proximal point method ( Rockafellar , 1976 ) , which solves a sequence of convexconcave subproblems constructed by adding a quadratic proximal term in the primal variable with a periodically updated reference point . Due to the potential non-smoothness of objective function , they show the convergence to a nearly-stationary point for the equivalent minimization problem . In the same vein as ( Rafique et al. , 2018 ) , Lu et al . ( 2019 ) design an algorithm by adopting the block alternating minimization/maximization strategy and show the convergence in terms of the proximal gradient . When the objective is weakly convex and weakly concave , Lin et al . ( 2018 ) propose a proximal algorithm which solves a strongly monotone variational inequality in each epoch and establish its convergence to stationary point . Sanjabi et al . ( 2018 ) consider non-convex non-concave min-max games where the inner maximization problem satisfies a PL condition , based on which they design a multi-step deterministic gradient descent ascent with convergence to a stationary point . It is notable that our work is different in that ( i ) we explore the PL condition for the outer minimization problem instead of the inner maximization problem ; ( ii ) we focus on designing stochastic algorithms instead of deterministic algorithms . Leveraging PL Condition for Minimization . PL condition is first introduced by Polyak ( Polyak , 1963 ) , which shows that gradient descent is able to enjoy linear convergence to a global minimum under this condition . Karimi et al . ( 2016 ) show that stochastic gradient descent , randomized coordinate descent , greedy coordinate descent are able to converge to a global minimum with faster rates under the PL condition . If the objective function has a finite-sum structure and satisfies PL condition , there are several non-convex SVRG-style algorithms ( Reddi et al. , 2016 ; Lei et al. , 2017 ; Nguyen et al. , 2017 ; Zhou et al. , 2018 ; Li & Li , 2018 ; Wang et al. , 2018 ) , which are guaranteed to converge to a global minimum with a linear convergence rate . However , the stochastic algorithms in these works are developed for a minimization problem , and hence is not applicable to the min-max formulation for stochastic AUC maximization . To the best of our knowledge , Liu et al . ( 2018 ) is the only work that leverages an equivalent condition to the PL condition ( namely quadratic growth condition ) to develop a stochastic primal-dual algorithm for AUC maximization with a fast rate . However , as mentioned before their algorithm and analysis rely on the convexity of the objective function , which does not hold for AUC maximization with a deep neural network . Finally , we notice that PL condition is the key to many recent works in deep learning for showing there is no spurious local minima or for showing global convergence of gradient descent and stochastic gradient descent methods ( Hardt & Ma , 2016 ; Li & Yuan , 2017 ; Arora et al. , 2018 ; Allen-Zhu et al. , 2018 ; Du et al. , 2018b ; a ; Li & Liang , 2018 ; Allen-Zhu et al. , 2018 ; Zou et al. , 2018 ; Zou & Gu , 2019 ) . Using the square loss , it has also been proved that the PL condition holds globally or locally for deep linear residual network ( Hardt & Ma , 2016 ) , deep linear network , one hidden layer neural network with Leaky ReLU activation ( Charles & Papailiopoulos , 2017 ; Zhou & Liang , 2017 ) . Several studies ( Li & Yuan , 2017 ; Arora et al. , 2018 ; Allen-Zhu et al. , 2018 ; Du et al. , 2018b ; Li & Liang , 2018 ) consider the trajectory of ( stochastic ) gradient descent on learning neural networks , and their analysis imply the PL condition in a certain form . For example , Du et al . ( 2018b ) show that when the width of a two layer neural network is sufficiently large , a global optimum would lie in the ball centered at the initial solution , in which PL condition holds . Allen-Zhu et al . ( 2018 ) extends this insight further to overparameterized deep neural networks with ReLU activation , and show that the PL condition holds for a global minimum around a random initial solution .

Summary: The authors propose stochastic algorithms for AUC maximization using a deep neural network. Under the assumption that the underlying function satisfies the PL condition, they prove convergence rates of the proposed algorithms. The key insight is to use the equivalence between AUC maximization and some min-max function. Experiments results show the proposed algorithms works better than some baselines.