RSO: A Gradient Free Sampling Based Approach For Training Deep Neural Networks

1 INTRODUCTION . Deep neural networks solve a variety of problems using multiple layers to progressively extract higher level features from the raw input . The commonly adopted method to train deep neural networks is backpropagation ( Rumelhart et al . ( 1985 ) ) and it has been around for the past 35 years . Backpropagation assumes that the function is differentiable and leverages the partial derivative w.r.t the weight wi for minimizing the function f ( x , w ) as follows , wi+1 = wi − η∇f ( x , w ) ∆wi , where η is the learning rate . Also , the method is efficient as it makes a single functional estimate to update all the weights of the network . As in , the partial derivative for some weight wj , where j 6= i would change once wi is updated , still this change is not factored into the weight update rule for wj . Moreover , it may not even be optimal for all weights to move in the same direction as obtained from the gradients in the previous layer . Although deep neural networks are non-convex ( and the weight update rule measures approximate gradients ) , this update rule works surprisingly well in practice . To explain the above observation , recent literature ( Du et al . ( 2019 ) ; Li & Liang ( 2018 ) ) argues that because the network is over-parametrized , the initial set of weights are very close to the final solution and even a little bit of nudging using gradient descent around the initialization point leads to a very good solution . We take this argument to another extreme - instead of using gradient based optimizers - which provide strong direction and magnitude signals for updating the weights ; we explore the region around the initialization point by sampling weight changes to minimize the objective function . Formally , our weight update rule is wi+1 = { wi , f ( x , wi ) < = f ( x , wi + ∆wi ) wi + ∆wi , f ( x , wi ) > f ( x , wi + ∆wi ) , where ∆wi is the weight change hypothesis . Here , we explicitly test the region around the initial set of weights by computing the function and update a weight if it minimizes the loss , see Fig . 1 . Surprisingly , our experiments demonstrate that the above update rule requires fewer weight updates compared to backpropagation to find good minimizers for deep neural networks , strongly suggesting that just exploring regions around randomly initialized networks is sufficient , even without explicit gradient computation . We evaluate this weight update scheme ( called RSO ; random search optimization ) on classification datasets like MNIST and CIFAR-10 with deep convolutional neural networks ( 6-10 layers ) and obtain competitive accuracy numbers . For example , RSO obtains 99.1 % accuracy on MNIST and 81.8 % accuracy on CIFAR-10 using just the random search optimization algorithm . We do not use any other optimizers for optimizing the final classification layer . Although RSO is computationally expensive ( because it requires updates which are linear in the number of network parameters ) , our hope is that as we develop better intuition about structural properties of deep neural networks , we will be able to accelerate RSO ( using Hebbian principles , Gabor filters , depth-wise convolutions ) . If the number of trainable parameters are reduced drastically ( Frankle et al . ( 2020 ) ) , search based methods could be a viable alternative to back-propagation . Furthermore , since architectural innovations which have happened over the past decade use backpropagation by default , a different optimization algorithm could potentially lead to a different class of architectures , because minimizers of an objective function via different greedy optimizers could potentially be different . 2 RELATED WORK . Multiple optimization techniques have been proposed for training deep neural networks . When gradient based methods were believed to get stuck in local minima with random initialization , layer wise training was popular for optimizing deep neural networks ( Hinton et al . ( 2006 ) ; Bengio et al . ( 2007 ) ) using contrastive methods ( Hinton ( 2002 ) ) . In a similar spirit , recent work , Greedy InfoMax by Löwe et al . ( 2019 ) maximizes mutual information between adjacent layers instead of training a network end to end . Taylor et al . ( 2016 ) finds the weights of each layer independently by solving a sequence of optimization problems which can be solved globally in closed form . Weight perturbation ( Werfel et al . ( 2004 ) ) based methods have been used for approximate gradients estimation in situations where gradient estimation is expensive . However , these training methods do not generalize to deep neural networks which have more than 2-3 layers and its not shown that the performance increases as we make the network deeper . Hence , back-propagation with SGD or other gradient based optimizers ( Duchi et al . ( 2011 ) ; Sutskever et al . ( 2013 ) ; Kingma & Ba ( 2014 ) ) are commonly used for optimizing deep neural networks . Recently , multiple works have proposed that because these networks are heavily over-parametrized , the initial set of random filters is already close to the final solution and gradient based optimizers only nudge the parameters to obtain the final solution ( Du et al . ( 2019 ) ; Li & Liang ( 2018 ) ) . For example , only training batch-norm parameters and keeping the random filters fixed can obtain very good results with heavily parametrized very deep neural networks ( > 800 layers ) as shown in Frankle et al . ( 2020 ) . It was also shown that networks can be trained by just masking out some weights without modifying the original set of weights by Ramanujan et al . ( 2020 ) - although one can argue that masking is a very powerful operator and can be used to represent an exponential number of output spaces . The network pruning literature covers more on optimizing subsets of an over-parametrized randomly initialized neural network ( Frankle & Carbin ( 2019 ) ; Li et al . ( 2017 ) ) . Our method , RSO , is also based on the hypothesis that the initial set of weights is close to the final solution . Here we show that gradient based optimizers may not even be necessary for training deep networks and when starting from randomly initialized weights , even search based algorithms can be a feasible option . Recently , search based algorithms have gained traction in the deep learning community . Since the design space of network architectures is huge , search based techniques are used to explore the placement of different neural modules to find better design spaces which lead to better accuracy ( Zoph & Le ( 2016 ) ; Liu et al . ( 2018 ) ) . This is done at a block level and each network is still trained with gradient descent . Similar to NAS based methods , weight agnostic neural networks ( WANN ) ( Gaier & Ha ( 2019 ) ) also searches for architectures , but uses a fixed set of weight values [ −2 , −1 , −0.5 , +0.5 , +1 , +2 ] . WANN operates at a much finer granularity as compared to NAS based methods while searching for connections between neurons and does not use gradient descent for optimization . Algorithms like Deep Neuroevolution by Such et al . ( 2017 ) and evolution strategies ( ES ) by Salimans et al . ( 2017 ) are search based optimization algorithms which have been used for training neural networks for reinforcement learning . ES is comprehensively reviewed by Beyer & Schwefel ( 2004 ) . Both Deep Neuroevolution and Salimans et al . ( 2017 ) create multiple replicas ( children ) of an initial neural network by adding small perturbations to all the weight parameters and then update the parameters by either selecting the best candidate or by performing a weighted average based on the reward . Both the methods update all the parameters of the network in each update . The problem with changing all the weights is that updating all the parameters of the network at once leads to random directions which are unlikely to contain a direction which will minimize the objective function and slows down learning ( results shown in Section 4.5 ) . Also , these methods were only trained on networks with 2-3 hidden layers , which is fairly shallow when compared to modern deep architectures . 3 APPROACH . Consider a deep neural network with D layers , where the weights of a layer d with nd neurons is represented byWd = { w1 , w2 .. , wid , .. wnd } . For an input activationAd−1 = { a1 , a2 , ... and−1 } , Wd generates an activation Ad = { a1 , a2 , ... and } , see Fig 2 . Each weight tensor wid ∈ Wd generates an activation ai ∈ Ad , where ai can be a scalar or a tensor depending on whether the layer is fully connected , convolutional , recurrent , batch-norm etc . The objective of the training process is to find the best set of weights W , which minimize a loss function F ( X , L ; W ) given some input data X and labels L. To this end , we initialize the weights of the network with a Gaussian distribution N ( 0 , √ 2/|wid | ) , like He et al . ( 2015 ) . The input data is also normalized to have zero mean and unit standard deviation . Once the weights of all layers are initialized , we compute the standard deviation σd of all elements in the weight tensor Wd . In the weight update step for a weight wj ∈ wid , ∆wj is sampled from ∼ N ( 0 , σd ) . We call this ∆Wj which is zero for all weights of the network but for wj , where j ∈ id . For a randomly sampled mini-batch ( x , l ) ∈ X , L , we compute the loss F ( x , l ; W ) for W , W + ∆Wj and W −∆Wj . If adding or subtracting ∆Wj reduces F , W is updated , otherwise the original weight is retained . This process is repeated for all the weights in the network , i.e. , to update all the weights of the network once , F needs to be computed three times the number of weight parameters in the network , 3|W | . We first update the weights of the layer closest to the labels and then sequentially move closer to the input . This is typically faster than optimizing the other way , but both methods lead to good results . This algorithm is described in Algorithm 1 . In Algorithm 1 , in line 12 , we sample change in weights from a Gaussian distribution whose standard deviation is the same as the standard deviation of the layer . This is to ensure that the change in weights is within a small range . The Gaussian sampling can also be replaced with other distributions like uniform sampling from ( −2σd , 2σd ) or just sampling values from a template like ( −σd , 0 , σd ) and these would also be effective in practice . The opposite direction of a randomly sampled weight is also tested because often it leads to a better hypothesis when one direction does not decrease the loss . However , in quite a few cases ( close to 10 % as per our experiments ) , not changing the weight at all is better . Note that there is no concept of learning rate in this algorithm . We also do not normalize the loss if the batch size increases or decreases as the weight update step is independent of the magnitude of the loss . There is widespread belief in the literature that randomly initialized deep neural networks are already close to the final solution ( Du et al . ( 2019 ) ; Li & Liang ( 2018 ) ) . Hence , we use this prior and explore regions using bounded step sizes ( N ( 0 , σd ) ) in a single dimension . We chose to update one weight at a time instead of sampling all the weights of the network as this would require sampling an exponential number of samples to estimate their joint distribution . RSO will be significantly faster even if prior knowledge about the distribution of the weights of individual neurons is used .

Summary: The paper proposes an RSO (random search optimization) method for training deep neural networks. This method is gradient-free and based on the Markov Chain Monte Carlo search. In particular, it adds a perturbation to weight in a deep neural network and tests if it reduces the loss on a mini-batch: the weight is updated if this reduces the loss, and retained otherwise.