Training Hard-Threshold Networks with Combinatorial Search in a Discrete Target Propagation Setting

Learning deep neural networks with hard-threshold activation has recently become an important problem due to the proliferation of resource-constrained computing devices . In order to circumvent the inability to train with backpropagation in the present of hard-threshold activations , Friesen & Domingos ( 2018 ) introduced a discrete target propagation framework for training hard-threshold networks in a layer-by-layer fashion . Rather than using a gradient-based target heuristic , we explore the use of search methods for solving the target setting problem . Building on both traditional combinatorial optimization algorithms and gradient-based techniques , we develop a novel search algorithm Guided Random Local Search ( GRLS ) . We demonstrate the effectiveness of our algorithm in training small networks on several datasets and evaluate our target-setting algorithm compared to simpler search methods and gradient-based techniques . Our results indicate that combinatorial optimization is a viable method for training hard-threshold networks that may have the potential to eventually surpass gradient-based methods in many settings . 1 INTRODUCTION . Interest in network quantization has rapidly increased in recent years ( Shan et al. , 2018 ; Hubara et al. , 2016 ; Friesen & Domingos , 2018 ) . Network quantization reduces the complexity of a neural network by lowering the precision requirements of the activations and/or weights ( Wu et al. , 2018 ; Choi et al. , 2016 ) . As more network-based models are being deployed to mobile devices , computational power reduction and efficiency increase become critical issues . In order to train a deep neural network with full precision parameters efficiently , we depend upon powerful computational devices such as high-performance CPUs and GPUs . These devices use a lot of energy and have abundant memory , which mobile devices do not have . Learning networks with hard-threshold activations allows binary or low-precision inference and training which reduces computation time and energy demands . This seemingly innocuous change leads to a surprising challenge—networks with hard-threshold activations can not be trained using the method of backpropagation since gradient descent requires a differentiable activation function . Unfortunately , the derivative of a hard-threshold function is 0 almost everywhere . The standard remedy is to use a gradient estimator known as the straight-through estimator ( STE ) , first proposed by Hinton et al . ( 2012 ) , and later analyzed by Bengio et al . ( 2013 ) . Friesen & Domingos ( 2018 ) designed a novel discrete target propagation framework based on the idea of separating the network into a series of perception problems each with artificially set targets . They gave a simple gradient-based target setting heuristic using the sign of the next layer ’ s loss gradient . Instead , we propose the use of search methods for target setting . The application of combinatorial optimization methods to training neural networks remains largely unexplored . Discrete target propagation frameworks offer an opportunity to apply these methods in deep learning . We initiate a feasibility study on search methods for learning hard-threshold networks . This may open the door to significantly improving performance by leveraging state-of-the-art combinatorial optimization algorithms . Building on the convex-combinatorial optimization framework of Friesen & Domingos ( 2018 ) , we present a search algorithm for target setting . We also develop a novel gradient-search hybrid target setting algorithm . We demonstrate the effectiveness of our method in training small networks on several data sets , with performance approaching the state of the art achieved by Friesen & Domingos ( 2018 ) for hard-threshold networks . We evaluate all target-setting algorithms mentioned in this paper , indicating significant room for improvements . 1.1 RELATED WORK . Using the conventional backpropagation algorithm to train a neural network via gradient descent , the gradient of the loss is computed with respect to the output of each layer . This is propagated recursively from the output layer to the input layer using the chain rule . In our case , the activation function at each hidden layer would involve a non-linearity , meaning that the partial derivative will be zero , hence ruling out the use of backpropagation . In the case that we can not use gradient descent , we use a gradient estimator . The straight-through estimator ( STE ) ( Hinton et al. , 2012 ; Bengio et al. , 2013 ) backpropagates through the hard threshold function as if it were the identity function . When considering a single layer of neurons , the STE has the right sign , but once it is used through more hidden layers , this is not guaranteed . Our work builds on the idea of target propagation , first proposed by LeCun et al . ( 1989 ) . The main idea is to compute targets rather than gradients , at each layer . Since then many different methods of target propagation have been proposed ( Bengio & Frasconi , 1995 ; Courbariaux et al. , 2015 ) . In particular , Bengio ( 2014 ) proposed a method called “ vanilla target propagation ” to avoid the chain of derivatives through the network by considering an “ approximate inverse ” . Later , Hubara et al . ( 2016 ) improved this method , claiming that the imperfection of the inverse yields severe optimization problems . This led to their difference target propagation , which is a linearly corrected formula of Bengio ( 2014 ) . They showed that this method of target propagation performs comparable to backpropagation methods . Most recently , Friesen & Domingos ( 2018 ) show that learning a deep hard-threshold network reduces to finding a feasible setting of its targets and then optimizing its weights in a layer-local fashion . Peng et al . ( 2018 ) introduce a new technique which they call SPIGOT : structured projection of intermediate gradients optimization technique . SPIGOT is a backpropagation technique for neural networks with hard-threshold predictions . It defines gradient-like quantities associated with intermediate non-differentiable operations , allowing backpropagation before and after them . 2 FEASIBLE TARGET PROPAGATION . We begin by describing feasible target propagation , the method introduced by Friesen & Domingos ( 2018 ) for training deep hard-threshold networks upon which our work is based . Consider a dataset D = ( X , T ) composed of an n ×m real-valued matrix X of vector-valued inputs { x ( j ) } mj=1 and a corresponding { +1 , −1 } -valued vector T of targets { t ( j ) } mj=1 . The aim is to learn an ` -layer neural network with hard-threshold activations f ( x ; W ) = g ( W ` g ( W ` −1 · · · g ( W2g ( W1x ) ) ) ) , with input vector x ∈ Rn , weight matrices W = { Wi | Wi ∈ Rni×ni−1 , 1 6 i 6 ` } , and activation function g ( x ) = sign ( x ) , where n0 = n and sign ( x ) = 1 if x > 0 and −1 otherwise . For simplicity , the bias terms are incorporated into the weight matrices . Let Zi = Wig ( Wi−1 · · · g ( W1X ) ) and Hi = g ( Zi ) denote the full-batch pre-activation and postactivation matrices , and zi = ( zi1 , . . . , zini ) and hi = ( hi1 , . . . , hini ) denote the corresponding per-instance vectors . The learning goal is to find weights W of f that minimize the training loss L ( H ` , T ) = ∑m j=1 L ( h ( j ) ` , t ( j ) ) for some per-instance loss L ( h ` , t ) . Because the derivative of the sign function is zero almost everywhere , backpropagation can not be used to train the network f when ` > 1 . Instead , Friesen & Domingos ( 2018 ) considered the decomposition f ` ◦ f ` −1 ◦ · · · ◦ f1 of f into perceptrons fi ( hi ) = hi+1 = g ( Wihi ) . Given a matrix Ti of targets and a loss function Li for each perceptron fi ( Hi ) , we can use the perceptron algorithm , or a gradient descent method , to set Wi such that fi produces these targets ( if the dataset ( Hi , Ti ) is linearly separable ) or minimizes the loss Li ( Zi , Ti ) . The problem then becomes setting the targets T = { T1 , . . . , T ` } in such a way as to minimize the output loss L ( H ` , T ) . Towards that end , Friesen & Domingos ( 2018 ) proposed a recursive approach that involves setting the targets Ti of fi in order to minimize the next layer ’ s loss Li+1 ( Zi+1 , Ti+1 ) . More specifically , the method proceeds as follows . For each minibatch ( Xb , Tb ) from datasetD , initialize T ` = Tb and T1 , . . . , T ` −1 as the activation of the corresponding layer in f ( Xb ; W ) and , starting with i = ` , for all 1 6 i 6 ` , 1. assign the targets T̂i−1 for the next layer based on the current weights Wi and loss Li ( fi ( Ti−1 ) , Ti ) , 2. update Wi with respect to the loss Li ( f ( Ti−1 ) , Ti ) , and 3. set Ti−1 ← T̂i−1 . 2.1 RELATIONSHIP TO GRADIENT ESTIMATION . In their work , Friesen & Domingos ( 2018 ) used the heuristic tij = sign ( − ∂ ∂hij Li+1 ( Zi+1 , Ti+1 ) ) ( 1 ) for setting the jth component tij of the target vector ti at layer i , and the layer loss Li ( zij , tij ) = ( tanh ( −tijzij ) + 1 ) ∣∣∣∣∂Li+1∂hij ∣∣∣∣ , ( 2 ) where tanh ( −tz ) + 1 is the soft hinge loss , so called because it is a smooth approximation of the saturated hinge loss max ( 0 , 1−max ( tz , −1 ) ) commonly used for training classifier networks . This combination of target heuristic and layer loss function allows information to be transmitted from the output loss back through the network to every layer , enabling effective learning as empirically demonstrated by Friesen & Domingos ( 2018 ) . They briefly commented on the relationship between this form of feasible target propagation and the straight-through estimator , but we believe it is worth further consideration here . The gradient of the weighted loss Li given in ( 2 ) with respect to the pre-activation zij is ∂Li ( zij , tij ) ∂zij = − sign ( −∂Li+1 ∂hij ) sech2 ( − sign ( −∂Li+1 ∂hij ) zij ) ∣∣∣∣∂Li+1∂hij ∣∣∣∣ = sech2 ( zij ) ∂Li+1 ∂hij since sech is an even function . This implies that the method of Friesen & Domingos ( 2018 ) is equivalent to a sech2 gradient estimator , that is , the straight-through estimator with the identity function replaced with sech2 . This is significant for two reasons . First , it suggests a simpler , independent justification for the performance improvements obtained by their method . Gradient estimation can be viewed as a way of propagating errors from the output layer by pretending the network has a different activation function . In particular , the saturated straight-through estimator corresponds to using a truncated identity function network as the error propagation proxy , while the sech2 gradient estimator corresponds to instead using a tanh network . When the activation function of the proxy network approximates the hard-threshold activation ( in our case , the sign function ) of the actual network ( see Figure 1 ) , we may expect the network to train effectively . From this perspective , the contributions to the state of the art by Friesen & Domingos ( 2018 ) reduce to finding an approximation to the sign function , the hyperbolic tangent , with more favorable properties for training ; specifically , the derivative of tanh is nonzero outside the range [ −1 , 1 ] , unlike the derivative of the truncated identity function . Although this justification and , moreover , the relationship between the straight-through estimator and their method were noted by Friesen & Domingos ( 2018 ) , recognizing that this is the sole contributor to the obtained performance improvements clarifies the role of their method . It also leads into our second point , which is that this line of reasoning motivates investigating alternative implementations of the discrete target propagation framework , which may not bear any connection to gradient estimation .

Summary: This paper addresses the problem of training neural networks with the sign activation function. A recent method for training such non-differentiable networks is target propagation: starting at the last layer, a target (-1 or +1) is assigned to each neuron in the layer; then, for each neuron in the layer, a separate optimization problem is solved, where the weights into that neuron are updated to achieve the target value. This procedure is iterated until convergence, as is typical for regular networks. Within the target propagation algorithm, the target assignment problem asks: how do we assign the targets at layer i, given fixed targets and weights at layer i+1? The FTPROP algorithm solves this problem by simply using sign of the corresponding gradient. Alternatively, this paper attempts to assign targets by solving a combinatorial problem. The authors propose a stochastic local search method which leverages gradient information for initialization and improvement steps, but is essentially combinatorial. Experimentally, the proposed algorithm, GRLS, is sometimes competitive with the original FTPROP, and is substantially better than the pure gradient approximation method that uses the straight-through estimator.