Local Critic Training of Deep Neural Networks

1 INTRODUCTION . In recent days , deep learning has been remarkably advanced and successfully applied in numerous fields ( LeCun et al. , 2015 ) . A key mechanism behind the success of deep neural networks is that they are capable of extracting useful information progressively through their layered structures . It is an increasing trend that more and more complex deep neural network structures are developed in order to solve challenging real-world problems , e.g. , He et al . ( 2016b ) . Training of deep neural networks is based on backpropagation in most cases , which basically works in a sequential and synchronous manner . During the feedforward pass , the input data is processed through the hidden layers to produce the network output ; during the feedback pass , the error gradient is propagated back through the layers to update each layer ’ s weight parameters . Therefore , training of each layer has dependency on all the other layers , which causes the issue of locking ( Jaderberg et al. , 2017 ) . This is undesirable in some cases , e.g. , a system consisting of several interacting models , a model distributed across multiple computing nodes , etc . There have been attempts to remove the locking constraint . In Carreira-Perpinan & Wang ( 2014 ) , the method of auxiliary coordinates ( MAC ) is proposed . It replaces the original loss minimization problem with an equality-constrained optimization problem by introducing an auxiliary variable for each data and each hidden unit . Then , solving the problem is formulated as iteratively solving several sub-problems independently . A similar approach using the alternating direction method of multipliers ( ADMM ) is proposed in Taylor et al . ( 2016 ) . It also employs an equality-constrained optimization but with different auxiliary variables , so that resulting sub-problems have closed form solutions . However , these methods are not scalable to deep learning architectures such as convolutional neural networks ( CNNs ) . The method proposed in Jaderberg et al . ( 2017 ) , called decoupled neural interface ( DNI ) , directly synthesizes estimated error gradients , called synthetic gradients , using an additional small neural network for training a layer ’ s weight parameters . As long as the synthetic gradients are close to the actual backpropagated gradients , each layer does not need to wait until the error at the output layer is backpropagated through the preceding layers , which allows independent training of each layer . However , this method suffers from performance degradation when compared to regular backpropagation ( Czarnecki et al. , 2017a ) . The idea of having additional modules supporting the layers of the main model is also adopted in Czarnecki et al . ( 2017a ) , where the additional modules are trained to approximate the main model ’ s outputs instead of error gradients . Due to this , however , the method does not resolve the issue of update locking , and in fact , the work does not intend to design a non-sequential learning algorithm . In this paper , we propose a novel approach for non-sequential learning , called local critic training . The key idea is that additional modules besides the main neural network model are employed , which we call local critics , in order to indirectly deliver error gradients to the main model for training without backpropagation . In other words , a local critic located at a certain layer group is trained in such a way that the derivative of its output serves as the error gradient for training of the corresponding layers ’ weight parameters . Thus , the error gradient does not need to be backpropagated , and the feedforward operations and gradient-descent learning can be performed independently . Through extensive experiments , we examine the influences of the network structure , update frequency , and total number of local critics , which provide not only insight into operation characteristics but also guidelines for performance optimization of the proposed method . In addition to the capability of implementing training without locking , the proposed approach can be exploited for additional important applications . First , we show that applying the proposed method automatically performs structural optimization of neural networks for a given problem , which has been a challenging issue in the machine learning field . Second , a progressive inference algorithm using the network trained with the proposed method is presented , which can adaptively reduce the computational complexity during the inference process ( i.e. , test phase ) depending on the given data . Third , the network trained by the proposed method naturally enables ensemble inference that can improve the classification performance . 2 PROPOSED APPROACH . 2.1 LOCAL CRITIC TRAINING . The basic idea of the proposed approach is to introduce additional local networks , which we call local critics , besides the main network model , so that they eventually provide estimates of the output of the main network . Each local critic network can serve a group of layers of the main model by being attached to the last layer of the group . The proposed architecture is illustrated in Figure 1 , where fi is the ith layer group ( containing one or more layers ) , hi is the output of fi , and hN is the final output of the main model having N layer groups : hi = fi ( hi−1 ) ( 1 ) ci is the local critic network for fi , which is expected to approximate hN based on hi , i.e. , ci ( hi ) ≈ hN ( 2 ) Then , this can be used to approximate the loss function of the main network , LN = l ( hN , y ) , which is used to train fi , by Li = l ( ci ( hi ) , y ) ( 3 ) for i = 1 , ... , N − 1 , i.e. , Li ≈ LN ( 4 ) where y is the training target and l is the loss function such as cross-entropy or mean-squared error . Then , the error gradient for training fi is obtained by differentiating Li with respect to hi , i.e. , δi = ∂Li ∂hi ( 5 ) which can be used to train the weight parameters of fi , denoted by θi , via a gradient-descent rule : θi ← θi − η δi ∂hi ∂θi ( 6 ) where η is a learning rate . Note that the final layer group hN does not require a local critic network and can be trained using the regular backproagation because the final output of the main network is directly available . Therefore , the update of fi does not need to wait until its output hi propagates till the end of the main network and the error gradient is backpropagated ; it can be performed when the operations from ( 2 ) to ( 5 ) are done . For ci , we usually use a simple model so that the operations through ci are simpler than those through fi+1 till fN . While the dependency of fi on fj ( j > i ) during training is resolved in this way , there still exists the dependency of ci on fj ( j > i ) , because training ci requires its ideal target , i.e. , hN , which is available from fN only after the feedforward pass is complete . In order to resolve this problem , we use an indirect , cascaded approach , where ci is trained so that its training loss targets the training loss for ci+11 : Lci = l ( Li , Li+1 ) ( 7 ) In other words , training of ci can be performed once the loss for ci+1 is available . Figure 1 compares the proposed architecture with the existing DNI approach that also employs local networks besides the main network to resolve the issue of locking ( Jaderberg et al. , 2017 ) . In DNI , the local network ci directly estimates the error gradient , i.e. , ci ( hi ) ≈ ∂LN ∂hi ( 8 ) so that each layer group of the main model can be updated without waiting for the forward and backward propagations in the subsequent layers . And , to update ci , the error gradient for fi+1 estimated by ci+1 is backpropagated through fi+1 and is used as the ( estimated ) target for ci . Therefore , all the necessary computations in the forward and backward passes can be locally confined . The performance of the two methods will be compared in Section 3 . 2.2 STRUCTURAL OPTIMIZATION . In many cases , determining an appropriate structure of neural networks for a given problem is not straightforward . This is usually done through trial-and-error , which is extremely time-consuming . There have been studies to automate the structural optimization process ( Cortes et al. , 2017 ; Feng & Darrell , 2015 ; Kwok & Yeung , 1997 ; Reed , 1993 ) , but this issue still remains very challenging . In deep learning , the problem of structural optimization is even more critical . Large-sized networks may easily show overfitting . Even if large networks may produce high accuracy , they take significantly large amounts of memory and computation , which is undesirable especially for resourceconstrained cases such as embedded and mobile systems . Therefore , it is highly desirable to find an optimal network structure that is sufficiently small while the performance is kept reasonably good . During local critic training , each local critic network is trained to estimate the output of the main network eventually . Therefore , once the training of the proposed architecture finishes , we obtain different networks that are supposed to have similar input-output mappings but have different structures and possibly different accuracy , i.e. , multiple sub-models and one main model ( see Figure 2b ) . Here , a sub-model is composed of the layers on the path from the input to a certain hidden layer 1We found that this is more effective than directly forcing ci to approximate ci+1 using Lci = l ( ci ( hi ) , ci+1 ( hi+1 ) ) . Algorithm 1 : Progressive inference Input : data x , threshold t Model : sub-model ci , main-model f Initialize : classification = 0. for i = 1 to N − 1 do if max softmax ( ci ( x ) ) > t then classification = argmax softmax ( ci ( x ) ) break end if end for if classification == 0 then # if all sub-models are not confident classification = argmax softmax ( f ( x ) ) end if and its local critic network . Among the sub-models , we can choose one as a structure-optimized network by considering the trade-off relationship between the complexity and performance . It is worth mentioning that our structural optimization approach can be performed instantly after training of the model , whereas many existing methods for structural optimization require iterative search processes , e.g. , Zoph & Le ( 2017 ) . 2.3 PROGRESSIVE INFERENCE . We propose another simple but effective way to utilize the sub-models obtained by the proposed approach for computational efficiency , which we call progressive inference . Although small submodels ( e.g. , sub-model 1 ) tend to show low accuracy , they would still perform well for some data . For such data , we do not need to perform the full feedforward pass but can take the classification decision by the sub-models . Thus , the basic idea of the progressive inference is to finish inference ( i.e. , classification ) with a small sub-model if its confidence on the classification result is high enough , instead of completing the full feedforward pass with the main model , which can reduce the computational complexity . Here , the softmax outputs for all classes are compared and the maximum probability is used as the confidence level . If it is higher than a threshold , we take the decision by the sub-model ; otherwise , the feedforward pass continues . The proposed progressive inference method is summarized in Algorithm 12 .

Summary: This paper describes a method of training neural networks without update locking. The idea is a small modification on top of Czarnecki et al. Critic training, where instead of using final loss as a critic target, one bootstraps from critics on other layers. In particular, if only one module is present, these two approaches are actually identical. To be more precise, the only difference between these two methods is that (7) in Critic training would change to l(L_i, L_N). As a consequence, method becomes forward unlocked too. It is worth noting, that in the appendix of Czarnecki et al. it is shown that this particular method (critic training) under simple conditions actually "degenerates" to deep supervision (which is forward unlocked too). Consequently unlocking property as such is not a big contribution of the proposed method. Rest of the paper includes following elements: