Stochastic Gradient Descent with Biased but Consistent Gradient Estimators

1 INTRODUCTION . Consider the standard setting of supervised learning . There exists a joint probability distribution P ( x , y ) of data x and associated label y and the task is to train a predictive model , parameterized by w , that minimizes the expected loss ` between the prediction and the ground truth y . Let us organize the random variables as ξ = ( x , y ) and use the notation ` ( w ; ξ ) for the loss . If ξi = ( xi , yi ) , i = 1 , . . . , n , are iid training examples drawn from P , then the objective function is either one of the following well-known forms : expected risk f ( w ) = E [ ` ( w ; ξ ) ] ; empirical risk f ( w ) = 1 n n∑ i=1 ` ( w ; ξi ) . ( 1 ) Stochastic gradient descent ( SGD ) , which dates back to the seminal work of Robbins & Monro ( 1951 ) , has become the de-facto optimization method for solving these problems in machine learning . In SGD , the model parameter is updated until convergence with the rule1 wk+1 = wk − γkgk , k = 1 , 2 , . . . , ( 2 ) where γk is a step size and gk is an unbiased estimator of the gradient∇f ( wk ) . Compared with the full gradient ( as is used in deterministic gradient descent ) , an unbiased estimator involves only one or a few training examples ξi and is usually much more efficient to compute . 1.1 LIMITATION OF UNBIASED GRADIENT AND REMEDY : CONSISTENT GRADIENT . This scenario , however , does not cover all learning settings . A representative example that leads to costly computation of the unbiased gradient estimator ∇ ` ( w , ξi ) is graph nodes . Informally speaking , a graph node ξi needs to aggregate information from its neighbors . If information is aggregated 1For introductory purpose we omit the projection operator for constrained problems . All analysis in this work covers projection . across neighborhoods , ξi must request information from its neighbors recursively , which results in inquiring a large portion of the graph . In this case , the sample loss ` for ξi involves not only ξi , but also all training examples within its multihop neighborhood . The worst case scenario is that computing ∇ ` ( w , ξi ) costs O ( n ) ( e.g. , for a complete graph or small-world graph ) , as opposed to O ( 1 ) in the usual learning setting because only the single example ξi is involved . In a recent work , Chen et al . ( 2018 ) proposed a consistent gradient estimator as an economic alternative to an unbiased one for training graph convolutional neural networks , offering substantial evidence of empirical success . A summary of the derivation is presented in Section 2 . The subject of this paper is to provide a thorough analysis of the convergence behavior of SGD when gk in ( 2 ) is a consistent estimator of ∇f ( wk ) . We show that using this estimator results in the same convergence behavior as does using unbiased ones . Definition 1 . An estimator gN of h , where N denotes the sample size , is consistent if gN converges to h in probability : plimN→∞ g N = h. That is , for any > 0 , limN→∞ Pr ( ‖gN − h‖ > ) = 0 . 1.2 DISTINCTIONS BETWEEN UNBIASEDNESS AND CONSISTENCY . It is important to note that unbiased and consistent estimators are not subsuming concepts ( one does not imply the other ) , even in the limit . This distinction renders the departure of our convergence results , in the form of probabilistic bounds on the error , from the usual SGD results that bound instead the expectation of the error . In what follows , we present examples to illustrate the distinctions between unbiasedness and consistency . To this end , we introduce asymptotic unbiasedness , which captures the idea that the bias of an estimator may vanish in the limit . Definition 2 . An estimator gN of h , where N denotes the sample size , is asymptotically unbiased if E [ gN ] → h. An estimator can be ( asymptotically ) unbiased but inconsistent . Consider estimating the mean h = µ of the normal distribution N ( µ , σ2 ) by using N independent samples X1 , . . . , XN . The estimator gN = X1 ( i.e. , always useX1 regardless of the sample sizeN ) is clearly unbiased because E [ X1 ] = µ ; but it is inconsistent because the distribution of X1 does not concentrate around µ . Moreover , the estimator is trivially asymptotically unbiased . An estimator can be consistent but biased . Consider estimating the variance h = σ2 of the normal distribution N ( µ , σ2 ) by using N independent samples X1 , . . . , XN . The estimator gN =∑N i=1 ( Xi − X ) 2/N , where X = ∑N i=1Xi/N , has mean σ 2 ( N − 1 ) /N and variance 2σ4 ( N − 1 ) /N2 . Hence , it is consistent owing to a straightforward invocation of the Chebyshev inequality , by noting that the mean approaches σ2 and the variance approaches zero . However , the estimator admits a nonzero bias σ2/N for any finite N . An estimator can be consistent but biased even asymptotically . In the preceding example , the bias σ2/N approaches zero and hence the estimator is asymptotically unbiased . Other examples exist for the estimator to be biased even asymptotically . Consider estimating the quantity h = 0 with an estimator gN that takes the value 0 with probability ( N − 1 ) /N and the value N with probability 1/N . Then , the probability that gN departs from zero approaches zero and hence it is consistent . However , E [ gN ] = 1 and thus the bias does not vanish as N increases . 1.3 CONTRIBUTIONS OF THIS WORK . To the best of our knowledge , this is the first work that studies the convergence behavior of SGD with consistent gradient estimators , which result from a real-world graph learning scenario that will be elaborated in the next section . With the emergence of graph deep learning models ( Bruna et al. , 2014 ; Defferrard et al. , 2016 ; Li et al. , 2016 ; Kipf & Welling , 2017 ; Hamilton et al. , 2017 ; Gilmer et al. , 2017 ; Velic̆ković et al. , 2018 ) , the scalability bottleneck caused by the expensive computation of the sample gradient becomes a pressing challenge for training ( as well as inference ) with large graphs . We believe that this work underpins the theoretical foundation of the efficient training of a series of graph neural networks . The theory reassures practitioners of doubts on the convergence of their optimization solvers . Encouragingly , consistent estimators result in a similar convergence behavior as do unbiased ones . The results obtained here , including the proof strategy , offer convenience for further in-depth analysis under the same problem setting . This work opens the opportunity of improving the analysis , in a manner similar to the proliferation of SGD work , from the angles of relaxing assumptions , refining convergence rates , and designing acceleration techniques . We again emphasize that unbiasedness and consistency are two separate concepts ; neither subsumes the other . One may trace that we intend to write the error bounds for consistent gradient estimators in a manner similar to the expectation bounds in standard SGD results . Such a resemblance ( e.g. , in convergence rates ) consolidates the foundation of stochastic optimization built so far . 2 MOTIVATING APPLICATION : REPRESENTATION LEARNING OF GRAPH NODES . For a motivating application , consider the graph convolutional network model , GCN ( Kipf & Welling , 2017 ) , that learns embedding representations of graph nodes . The l-th layer of the network is compactly written as H ( l+1 ) = σ ( ÂH ( l ) W ( l ) ) , ( 3 ) where Â is a normalization of the graph adjacency matrix , W ( l ) is a parameter matrix , and σ is a nonlinear activation function . The matrix H ( l ) contains for each row the embedding of a graph node input to the l-th layer , and similarly for the output matrix H ( l+1 ) . With L layers , the network transforms an initial feature input matrix H ( 0 ) to the output embedding matrix H ( L ) . For a node v , the embedding H ( L ) ( v , : ) may be fed into a classifier for prediction . Clearly , in order to compute the gradient of the loss for v , one needs the corresponding row ofH ( L ) , the rows ofH ( L−1 ) corresponding to the neighbors of v , and further recursive neighbors across each layer , all the way down to H ( 0 ) . The computational cost of the unbiased gradient estimator is rather high . In the worst case , all rows of H ( 0 ) are involved . To resolve the inefficiency , Chen et al . ( 2018 ) proposed an alternative gradient estimator that is biased but consistent . The simple and effective idea is to sample a constant number of nodes in each layer to restrict the size of the multihop neighborhood . For notational clarity , the approach may be easier to explain for a network with a single layer ; theoretical results for more layers straightforwardly follow that of Theorem 1 below , through induction . The approach generalizes the setting from a finite graph to an infinite graph , such that the matrix expression ( 3 ) becomes an integral transform . In particular , the input feature vector H ( 0 ) ( u , : ) for a node u is generalized to a feature function X ( u ) , and the output embedding vector H ( 1 ) ( v , : ) for a node v is generalized to an embedding function Z ( v ) , where the random variables u and v in two sides of the layer reside in different probability spaces , with probability measures P ( u ) and P ( v ) , respectively . Furthermore , the matrix Â is generalized into a bivariate kernel Â ( v , u ) and the loss ` is written as a function of the output Z ( v ) . Then , ( 1 ) and ( 3 ) become f = Ev∼P ( v ) [ ` ( Z ( v ) ) ] with Z ( v ) = σ ( ∫ Â ( v , u ) X ( u ) W dP ( u ) ) . ( 4 ) Such a functional generalization facilitates sampling on all network layers for defining a gradient estimator . In particular , defining B ( v ) = ∫ Â ( v , u ) X ( u ) dP ( u ) , simple calculation reveals that the gradient with respect to the parameter matrix W is G : = ∇f = ∫ q ( B ( v ) ) dP ( v ) , where q ( B ) = BT∇h ( BW ) and h = ` ◦ σ . Then , one may use t iid samples of u in the input and s iid samples of v in the output to define an estimator of G : Gst : = 1 s s∑ i=1 q ( Bt ( vi ) ) , vi ∼ P ( v ) , with Bt ( v ) : = 1 t t∑ j=1 Â ( v , uj ) X ( uj ) , uj ∼ P ( u ) . The gradient estimator Gst so defined is consistent ; see a proof in the supplementary material . Theorem 1 . If q is continuous and f is finite , then plims , t→∞Gst = G .

Summary: The paper studies stochastic optimization with consistent (may not be unbiased) estimators. This problem is well-motivated through the example of learning graph representations where consistent estimators are easier to obtain than unbiased one. Under the assumption that the estimate converges to the consistent gradient exponentially fast w.r.t. the sample size, the authors give convergence rates for convex, strongly-convex and non-convex optimization. The authors validate their theory through synthetic experiments.