New Perspective on the Global Convergence of Finite-Sum Optimization

1 INTRODUCTION . In recent years , deep neural networks ( DNNs ) have shown a great success in many machine learning tasks . However , training these neural networks is challenging since the loss surface of network architecture is generally non-convex , or even non-smooth . Thus , there have been a long-standing question on how optimization algorithms may converge to a global minimum . Many previous work have investigated Gradient Descent algorithm and its stochastic version for over-parameterized setting ( Arora et al. , 2018 ; Soudry et al. , 2018 ; Allen-Zhu et al. , 2019 ; Du et al. , 2019a ; Zou & Gu , 2019 ) . Although these works have shown promising convergence results under certain assumptions , there is still a lack of new efficient methods that can guarantee global convergence for machine learning optimization . In this paper , we address this problem using a different perspective . Instead of analyzing the traditional finite-sum formulation , we adopt a new composite formulation that exactly depicts the structure of machine learning where a data set is used to learn a common classifier . Representation . Let { ( x ( i ) , y ( i ) ) } n i=1 be a given training set with x ( i ) ∈ Rm , y ( i ) ∈ Rc , we investigate the following novel representation for deep learning tasks : min w∈Rd { F ( w ) = 1 n n∑ i=1 φi ( h ( w ; i ) ) } , ( 1 ) where h ( · ; i ) : Rd → Rc , i ∈ [ n ] = { 1 , . . . , n } , is the classifier for each input data x ( i ) ; and φi : Rc → R , i ∈ [ n ] , is the loss function corresponding to each output data y ( i ) . Our composite formulation ( 1 ) is a special case of the finite-sum problem minw∈Rd { F ( w ) = 1n ∑n i=1 f ( w ; i ) } where each individual function f ( · ; i ) is a composition of the loss function φi and the classifier h ( · ; i ) . This problem covers various important applications in machine learning , including logistic regression and neural networks . The most common approach for the finite-sum problem is using first-order methods such as ( stochastic ) gradient algorithms and making assumptions on the component functions f ( · ; i ) . As an alternative , we further investigate the structure of the loss function φi and narrow our assumption on the classifier h ( · ; i ) . For the purpose of this work , we first consider convex and Lipschitz-smooth loss functions while the classifiers can be non-convex . Using this representation , we propose a new framework followed by two algorithms that guarantee global convergence for the minimization problem . Algorithmic Framework . Representation ( 1 ) admits a new perspective . Our key insight is to ( A ) define z ( t ) i = h ( w ( t ) ; i ) , where t is an iteration count of the outer loop in our algorithmic framework . Next ( B ) , we want to approximate the change z ( t+1 ) i − z ( t ) i in terms of a step size times the gradient ∇φi ( z ( t ) i ) = ( ∂φi ( z ) /∂za ) a∈ [ c ] ∣∣ z=z ( t ) i , and ( C ) we approximate the change h ( w ( t+1 ) ; i ) − h ( w ( t ) ; i ) in terms of the first order derivative H ( t ) i = ( ∂ha ( w ; i ) /∂wb ) a∈ [ c ] , b∈ [ d ] ∣∣ w=w ( t ) . Finally , we combine ( A ) , ( B ) , and ( C ) to equate the approximations of z ( t+1 ) i − z ( t ) i and h ( w ( t+1 ) ; i ) − h ( w ( t ) ; i ) . This leads to a recurrence on w ( t ) of the form w ( t+1 ) = w ( t ) − η ( t ) v ( t ) , where η ( t ) is a step size and which involves computing v ( t ) by solving a convex quadratic subproblem , see the details in Section 4 . We explain two methods for approximating a solution for the derived subproblem . We show how to approximate the subproblem by transforming it into a strongly convex problem by adding a regularizer which can be solved in closed form . And we show how to use Gradient Descent ( GD ) on the subproblem to find an approximation v ( t ) of its solution . Convergence Analysis . Our analysis introduces non-standard bounded style assumptions . Intuitively , we assume that our convex and quadratic subproblem has a bounded solution . This allows us to prove a total complexity of Õ ( 1ε3 ) to find an ε- ( global ) solution that satisfies F ( ŵ ) − F∗ ≤ ε , where F∗ is the global minimizer of F . Our analysis applies to a wide range of applications in machine learning : Our results hold for squared loss and softmax cross-entropy loss and applicable for a range of activation functions in DNN as we only assume that the h ( · ; i ) are twice continuously differentiable and their Hessian matrices ( second order derivatives ) as well as their gradients ( first order derivatives ) are bounded . Contributions and Outline . Our contributions in this paper can be summarized as follows . • We propose a new representation ( 1 ) for analyzing the machine learning minimization problem . Our formulation utilizes the structure of machine learning tasks where a training data set of inputs and outputs is used to learn a common classifier . Related work in Section 2 shows how ( 1 ) is different from the classical finite-sum problem . • Based on the new representation we propose a novel algorithm framework . The algorithmic framework approximates a solution to a subproblem for which we show two distinct approaches . • For general DNNs and based on bounded style assumptions , we prove a total complexity of Õ ( 1ε3 ) to find an ε- ( global ) solution that satisfies F ( ŵ ) −F∗ ≤ ε , where F∗ is the global minimizer of F . We emphasize that our focus is on developing a new theoretical foundation and that a translation to a practical implementation with empirical results is for future work . Our theoretical foundation motivates further study , implementation , and optimization of the new algorithmic framework and further investigation of its non-standard bounded style assumptions . This new direction broadens our understanding of why and under what circumstances training of a DNN converges to a global minimum . The rest of this paper is organized as follows . Section 2 discusses related work . Section 3 describes our setting and deep learning representation . Section 4 explains our key insight and derives our Framework 1 . Section 5 presents our algorithms and their global convergence . All technical proofs are deferred to the Appendix . 2 RELATED WORK . Formulation for Machine Learning Problems . The finite-sum problem is one of the most important and fundamental problems in machine learning . Analyzing this model is the most popular approach in the machine learning literature and it has been studied intensively throughout the years ( Bottou et al. , 2018 ; Reddi et al. , 2016 ; Duchi et al. , 2011b ) . Our new formulation ( 1 ) is a special case of the finite-sum problem , however , it is much more complicated than the previous model since it involves the data index i both inside the classifiers h ( · ; i ) and the loss functions φi . For a comparison , previous works only consider a common loss function l ( ŷ , y ) for the predicted value ŷ and output data y ( Zou et al. , 2018 ; Soudry et al. , 2018 ) . Our modified version of loss function φi is a natural setting for machine learning . We note that when h ( w ; i ) is the output produced by a model , our goal is to match this output with the corresponding target y ( i ) . For that reason , the loss function for each output has a dependence on the output data y ( i ) , and is denoted by φi . This fact reflects the natural setting of machine learning where the outputs are designed to fit different targets , and the optimization process depends on both outer function φi and inner functions h ( · ; i ) . This complication may potentially bring a challenge to theoretical analysis . However , with separate loss functions , we believe this model will help to exploit better the structure of machine learning problems and gain more insights on the neural network architecture . Other related composite optimization models are also investigated thoroughly in ( Lewis & Wright , 2016 ; Zhang & Xiao , 2019 ; Tran-Dinh et al. , 2020 ) . Our model is different from these works as it does not have a common function wrapping outside the finite-sum term , as in ( Lewis & Wright , 2016 ) . Note that a broad class of variance reduction algorithms ( e.g . SAG ( Le Roux et al. , 2012 ) , SAGA ( Defazio et al. , 2014 ) , SVRG ( Johnson & Zhang , 2013 ) , SARAH ( Nguyen et al. , 2017 ) ) is designed specifically for the finite-sum formulation and is known to have certain benefits over Gradient Descent . In addition , the multilevel composite problem considered in ( Zhang & Xiao , 2021 ) also covers empirical risk minimization problem . However our formulation does not match their work since our inner function h ( w ; i ) is not an independent expectation over some data distribution , but a specific function that depends on the current data . Global Convergence for Neural Networks . A recent popular line of research is studying the dynamics of optimization methods on some specific neural network architectures . There are some early works that show the global convergence of Gradient Descent ( GD ) for simple linear network and two-layer network ( Brutzkus et al. , 2018 ; Soudry et al. , 2018 ; Arora et al. , 2019 ; Du et al. , 2019b ) . Some further works extend these results to deep learning architectures ( Allen-Zhu et al. , 2019 ; Du et al. , 2019a ; Zou & Gu , 2019 ) . These theoretical guarantees are generally proved for the case when the last output layer is fixed , which is not standard in practice . A recent work ( Nguyen & Mondelli , 2020 ) prove the global convergence for GD when all layers are trained with some initial conditions . However , these results are for neural networks without bias neurons and it is unclear how these analyses can be extended to handle the bias terms of deep networks with different activations . Our novel framework and algorithms do not exclude learning bias layers as in ( Nguyen & Mondelli , 2020 ) . Using a different algorithm , Brutzkus et al . ( 2018 ) investigate Stochastic Gradient Descent ( SGD ) for two-layer networks in a restricted linearly separable data setting . This line of research continues with the works from Allen-Zhu et al . ( 2019 ) ; Zou et al . ( 2018 ) and later with Zou & Gu ( 2019 ) . They justify the global convergence of SGD for deep neural networks for some probability depending on the number of input data and the initialization process . Over-Paramaterized Settings and other Assumptions for Machine Learning . Most of the modern learning architectures are over-parameterized , which means that the number of parameters are very large and often far more than the number of input data . Some recent works prove the global convergence of Gradient Descent when the number of neurons are extensively large , e.g . ( Zou & Gu , 2019 ) requires Ω ( n8 ) neurons for every hidden layer , and ( Nguyen & Mondelli , 2020 ) improves this number to Ω ( n3 ) . If the initial point satisfies some special conditions , then they can show a better dependence of Ω ( n ) . In Allen-Zhu et al . ( 2019 ) , the authors initialize the weights using a random Gaussian distribution where the variance depends on the dimension of the problem . In non-convex setting , they prove the convergence of SGD using the assumption that the dimension depends inversely on the tolerance . We will discuss how these over-paramaterized settings might be a necessary condition to develop our theory . Other standard assumptions for machine learning include the bounded gradient assumption ( Nemirovski et al. , 2009 ; Shalev-Shwartz et al. , 2007 ; Reddi et al. , 2016 ; Tran et al. , 2021 ) . It is also common to assume all the iterations of an algorithm stays in a bounded domain ( Duchi et al. , 2011a ; Levy et al. , 2018 ; Gürbüzbalaban et al. , 2019 ; Reddi et al. , 2018 ; Vaswani et al. , 2021 ) . Since we are analyzing a new composite formulation , it is understandable that our assumptions may also not be standard . However , we believe that there is a strong connection between our assumptions and the traditional setting of machine learning . We will discuss this point more clearly in Section 4 .

Summary: The paper provides a new gradient-based algorithm. The algorithm is based on the observation that a loss function for a single sample can be written as composition of two functions (the logits and the actual loss function). It computes the direction by means of solving a quadratic MSE problem. They provide a convergent analysis of the algorithm (there is a version where the quadratic problem is solved explicitly through a closed form expression and a version where gradient descent is applied to solve the problem approximately). There are no computational experiments.   The main contributions are in the algorithms themselves and the accompanying analyses. It's unclear how novel are the proof techniques since everything resembles second order algorithms. The authors also claim the actual reformulation to be a novel contribution however such a formulation is straightforward and used in many contexts (some of my lecture slides from years ago show the formulation used by authors as a possible formulation for the overall loss function). Despite of this, the design of the algorithm should get credit.