I NITIALIZED E QUILIBRIUM P ROPAGATION FOR B ACKPROP -F REE T RAINING
A BSTRACT
Deep neural networks are almost universally trained with reverse -mode automatic differentiation (a k a backpropagation).
Biological networks, on the other hand, appear to lack any mechanism for sending gradients back to their input neurons and thus cannot be learning in this way.
In response to this, Scellier & Bengio (2017) proposed Equilibrium Propagation - a method for gradient-based training of neural networks which uses only local learning rules and, crucially, does not rely on neurons having a mechanism for back-propagating an error gradient.
Equilibrium propagation, however, has a major practical limitation: inference involves doing an iterative optimization of neural activations to ﬁnd a ﬁxed-point, and the number of steps required to closely approximate this ﬁxed point scales poorly with the depth of the network.
In response to this problem, we propose Initialized Equilibrium Propagation , which trains a feedforward network to initialize the iterative inference procedure for Equilibrium propagation.
This feed-forward network learns to approximate the state of the ﬁxed-point using a local learning rule.
After training, we can simply use this initializing network for inference, resulting in a learned feedforward network.
Our experiments show that this network appears to work as well or better than the original version of Equilibrium propagation while requiring fewer steps to converge.
This shows how we might go about training deep networks without using backpropagation.
I NTRODUCTION
Deep neural networks are almost always trained with gradient descent, and gradients are almost always computed with backpropagation.
For those interested in understanding the working of the brain in the context of machine learning, it is therefore distressing that biological neurons appear not to send signals backwards.
Biological neurons communicate by sending a sequence of pulses to downstream neurons along a one-way signaling pathway called an “axon”.
If neurons were doing backpropagation, one would expect a secondary signalling pathway wherein gradient signals travel backwards along axons.
This appears not to exist, so it seems that biological neurons cannot be doing backpropagation.
Moreover, backpropagation may not be the ideal learning algorithm for efﬁcient implementation in hardware, because it involves buffering activations for each layer until an error gradient returns.
This requirement becomes especially onerous when we wish to backpropagate through many steps of time, or through many layers of depth.
For these reasons, researchers are looking into other means of neural credit assignment - mechanisms for generating useful learning signals without doing backpropagation.
Recently, Scellier & Bengio (2017) proposed a novel algorithm called Equilibrium Propagation , which enables the computation of parameter gradients in a deep neural network without backpropagation.
Equilibrium Propagation deﬁnes a neural network as a dynamical system, whose dynamics follow the negative-gradient of an energy function.
The “prediction” of this network is the ﬁxedpoint of the dynamics - the point at which the system settles to a local minimum energy given the input, and ceases to change.
Because of this inference scheme, Equilibrium Propagation is impractically slow for large networks - the network has to iteratively converge to a ﬁxed point at every training iteration.
In this work, we take inspiration from Hinton et al.
and distill knowledge from a slow, energy based equilibrating network into a fast feedforward network by training the feedforward network to predict the ﬁxed-points of the equilibrating network with a local loss.
At the end of training, we can then discard the equilibrating network and simply use our feedforward network for testtime inference.
We thus have a way to train a feedforward network without backpropagation.
The resulting architecture loosely resembles a Conditional Generative Adversarial Network (Mirza &
Osindero, 2014), where the feedforward network produces a network state which is evaluated by the energy-based equilibrating network.
To aid the reader, this paper contains a glossary of symbols in Appendix A.
M ETHODS 2. B ACKGROUND : E QUILIBRIUM P ROPAGATION
Equilibrium Propagation (Scellier & Bengio, 2017) is a method for training an Energy-Based Model
Hopﬁeld (1984) for classiﬁcation.
The network performs inference by iteratively converging to a ﬁxed-point, conditioned on the input data, and taking the state of the output neurons at the ﬁxed point to be the output of the network.
The network’s dynamics are deﬁned by an energy function over neuron states s and parameters θ = ( w, b ) :
[Equation]
Where I is the set of input neuron indices, S is the set of non-input neuron indices; s ∈ R |S| is the vector of neuron states; where α j ⊂ {I ∪ S} is the set of neurons connected to neuron j ; x denotes the input vector; and ρ is some nonlinearity; w is a weight matrix with a symmetric constraint: w ij = w ji 1 , and entries only deﬁned for { w ij : i ∈ α j } The state-dynamics for non-input neurons, derived from Equation 1, are:
[Equation]
The network is trained using a two-phase procedure, with a negative and then a positive phase.
In the negative phase, the network is allowed to settle to an energy minimum s − :=
arg min s E θ ( s, x ) conditioned on a minibatch of input data x .
In the positive phase, a target y is introduced, and the energy function is augmented to “perturb” the ﬁxed-point of the state towards the target with a “clamping factor” β : E βθ ( s, x, y ) =
E θ ( s, x ) + βC
( s O , y ) , where β is a small scalar and C ( s O , y ) is a loss deﬁned between the output neurons in the network and the target y
(we use C ( s O , y )
= (cid:107)
s O − y (cid:107) 22 ).
The network is allowed to settle to the perturbed state s + :=
arg min s ( E β ( s, x, y )) .
Finally, the parameters of the network are learned based on a contrastive loss between the negativephase and positive-phase energy, which can be shown to be proportional to the gradient of the output loss in the limit of β → 0 :
[Equation]
Where η is some learning rate; O ⊆ S is the subset of output neurons.
This results in a local learning rule, where parameter changes only depend on the activities of the pre- and post-synaptic neurons:
[Equation]
Intuitively, the algorithm works by adjusting θ to pull arg min s E θ ( s, x ) closer to arg min s E βθ ( s, x, y ) so that the network will gradually learn to naturally minimize the output loss.
Because inference involves an iterative settling process, it is an undesirably slow process in Equilibrium propagation.
In their experiments, Scellier & Bengio (2017) indicate that the number of settling steps required scales super-linearly with the number of layers.
This points to an obvious need for a fast inference scheme.
2. A DDING AN I NITIALIZATION N ETWORK
We propose training a feedforward network f φ ( x ) → s f ∈ R |S| to predict the ﬁxed-point of the equilibrating network.
This allows the feedforward network to achieve two things: First, it initializes the state of the equilibrating network, so that the settling process starts in the right regime.
Second, the feedforward network can be used to perform inference at test-time, since it learns to approximate the minimal-energy state of the equilibrating network, which corresponds to the prediction.
f φ ( x ) is deﬁned as follows:
[Equation]
Where α fj =
( i : ( i ∈ α j ) ∧ ( i < j )) is the set of feedforward connections to neuron j (which is a subset of α j - the full set of connections to neuron j from the equilibrium network from Equation 1); φ = ( ω, c ) is the set of parameters of the feedforward network.
This feedforward network produces the initial state of the negative phase of equilibrium propagation network, given the input data - i.e., instead of starting at a zero-state, the equilibrium-propagation network is initialized in a state s f :
= f φ ( x ) .
We train the parameters φ to approximate the minimal energy state s − of the equilibrating network 2 .
In other words, we seek:
[Equation]
The derivative of the forward parameters of the i ’th neuron, φ i =
( ω α
i ,i , c i ) , can be expanded as: close to
, and so lead better gradient
+
[Equation]
∂φ
[Equation]
= local ∂ L i ∂s fi ∂s fi ∂φ i + distant (cid:88)
j>i ∂
L j ∂s fj ∂s fj ∂s fi ∂s fi ∂φ i (9)
The distant term is problematic, because computing ∂s fj ∂s fi would require backpropagation, and the entire purpose of this exercise is to train a neural network without backpropagation.
However, we ﬁnd that only optimizing the local term ∂ L i ∂φ i does not noticeably harm performance.
In Section 2.
we go into more detail on why it appears to be sufﬁcient to minimize local losses.
Over the course of training, parameters φ will learn until our feedforward network is a good predictor of the minimal-energy state of the equilibrating network.
This feedforward network can then be used to do inference: we simply take the state of the output neurons to be our prediction of the target data.
The full training procedure is outlined in Algorithm 1.
At the end of training, inference can be done either by taking the output activations from the forward pass of the inference network f φ (Algorithm 2), or by initializing with a forward pass and then iteratively minimizing the energy (Algorithm 3).
Experiments in Section 3 indicate that the forward pass performs just as well as the full energy minimization.
Learning Rate Network Architecture α , Number of negative-phase steps T − ,
Number positive-phase steps T + 2: φ ← InitializeFeedforwardParameters ( α ) 3: θ InitializeEquilibriumParameters (
Phase 8: s ← s − (cid:15) ∂E
θ ( s,x m ) ∂s 9: s − ← s 10: for t ∈ 1 ..
T + do # Pos.
Phase 11: s ← s − (cid:15)
∂E
βθ
( s,x m ,y m ) ∂s 12: s + ← s 13: θ ← θ − ηβ (cid:16)
∂E
θ ( s + ,x ) ∂θ − ∂E
θ ( s − ,x ) ∂θ (cid:17) 14: φ i ← φ
i − η ∂ L i ( s fi ,s − i )
∂φ i ∀
i 15: Return: φ, θ # Parameters
1: Input: Input Data x , Inference Parameters φ 2: s ← f φ ( x ) 3: return ( s i : i ∈ O )
# Output unit states
1: Input: Input Data x , Initialization Parameters φ , Equilibriating Parameters θ ,
Number of Negative Steps T − 2: s ← f φ ( x ) 3: for t ∈ 1 ..
T − do # Neg.
Phase 4: s ← s − (cid:15) ∂E
θ ( s,x m ) ∂s 5: return ( s i : i ∈ O )
# Output unit states
2. I NCLUDING THE FORWARD STATES IN THE ENERGY FUNCTION
The ﬁxed point s − of the equilibrating network is a nonlinear function of x , whose value is computed by iterative bottom-up and top-down inference using all of the parameters θ .
The initial state s f , by contrast, is generated in a single forward pass, meaning that the function relating s fj to its direct inputs s fα j ∈ R | α j | is constrained to the form of Equation 6.
Because of this, the computation resulting in s − may be more ﬂexible than that of the forward pass, so it is possible for the equilibrating network to create targets that are not achievable by the neurons in the feedforward network.
This is similar to the notion of an “amortization gap” in variational inference, and we discuss this connection more in Section 4.2.
Neurons in the feedforward network simply learn a linear mapping from the previous layer’s activations to the targets provided by the equilibrating network.
In order to encourage the equilibrating network to stay in the regime that is reachable by the forward network, we can add a loss encouraging the ﬁxed points to stay in the regime of the forward pass.
[Equation]
Where λ is a hyperparameter which brings the ﬁxed-points of the equilibrating network closer to the states of the forward pass, and encourages the network to optimize the energy landscape in the region reachable by the forward network.
Of course this may reduce the effective capacity of the equilibrating network, but if our goal is only to train the feedforward network, this does not matter.
This trick has a secondary beneﬁt: It allows faster convergence in the negative phase by pulling the minimum of E λθ ( s, x ) closer to the feedforward prediction, so we can learn with fewer convergence steps.
It can however, cause instabilities when set too high.
We investigate the effect of different values of λ with experiments in Appendix D.
2. W HY THE LOCAL LOSS IS SUFFICIENT : G RADIENT A LIGNMENT
In Equation 9 we decompose the loss-gradient of parameters φ into a local and a global component.
Empirically (see Figures 1, 3),
we ﬁnd that using the local loss and simply ignoring the global loss led to equally good convergence.
To understand why this is the case, let use consider a problem where we learn the mapping from an input x to a set of neuron-wise targets: s ∗ .
Assume these targets are generated by some (unknown) set of ideal parameters φ ∗ , so that s ∗ =
f φ ∗ ( x ) .
To illustrate, we consider a two layer network with φ = ( w 1 , w 2 ) and φ ∗ =
( w ∗ 1 , w ∗ 2 ) :
[Equation]
It may come as a surprise that when φ is in the neighbourhood of the ideal parameters φ ∗ , the cosine similarity between the local and distant gradients: S (cid:16) ∂ L 1 ∂w 1 , [Equation] 1 (cid:17) is almost always positive, i.e. the local and distant gradients tend to be aligned.
This is a pleasant surprise because it means the local loss will tend to guide us in the right direction.
The reason becomes apparent when we deﬁne ∆ w := w − w ∗ , and write out the expression for the gradient in the limit of ∆ w → 0 (see Appendix
B for derivation)
[Equation]
When the term w 2 (cid:12) ρ (cid:48) ( s 1 · w 2 ) 2 · w T 2 is proportional to an identity matrix, we can see that ∂ L 1 ∂wand G 1 are perfectly aligned.
This will be the case when w 2 is orthogonal and layer 2 has a linear activation.
However, even for randomly sampled parameters and a nonlinear activation, w 2 (cid:12) ρ (cid:48) ( s 1 · w 2 ) 2 · w T 2 tends to have a strong diagonal component and the terms thus tend to be positively aligned.
Figure 1 demonstrates that this gradient-alignment tends to increase as then network trains to approximate a set of targets (i.e. as φ → φ ∗ ).
Note that the alignment of the local loss-gradient with the global loss-gradient is at least as high as with the distant loss-gradient, because ∇ φ L global = ∇ φ L local + ∇ φ L distant and S ( ∇ φ L distant , ∇ φ L local ) ≤
S ( ∇ φ L distant + ∇ φ L local , ∇ φ L local )
∀∇
φ L local , ∇ φ L distant .
This explains the empirical observation in Figures 1 and 3 that optimizing the local, as opposed to the global, loss for the feedforward network does not appear to slow down convergence: Later layers do not have to “wait” for earlier layers to converge before they themselves converge - earlier layers [Figure] optimize the loss of later layers right from the beginning of training.
As shown in Figure 1, it may in fact speed up convergence since each layer’s optimizer is solving a simpler problem ( with changing input representations for layers > 1 ).
When local targets s − are provided by the equilibrating network, it is not in general true that there exists some φ ∗ such that s − = s ∗ .
In our experiments, we observed that this did not prevent the forward network from learning to classify just as well as the equilibrating network.
However, this may not hold for more complex datasets.
As mentioned in Section 2.3, this could be resolved in future work with a scheme for annealing λ up to inﬁnity while maintaining stable training.
E XPERIMENTS
We base our experiments off of those of Scellier & Bengio (2017): We use the hard sigmoid ρ ( x ) = max (0 , min (1 , x )) as our nonlinearity.
We clip the state of s i to the range (0, 1) because, since ρ (cid:48)
( x ) =
: x < 0 ∨ x > 1 , if the system in Equation 2 were run in continuous time, it should never reach states outside this range.
Borrowing a trick from Scellier & Bengio (2017) to avoid instability problems arising from incomplete negative-phase convergence, we randomly sample β ∼
U ( {− β base , + β base } ) , where β base is a small positive number, for each minibatch and use this for both the positive phase and for multiplying the learning rate in Equation 3 (for simplicity, this is not shown in Algorithm 1).
Unlike Scellier & Bengio (2017), we do not use the trick of caching and reusing converged states for each data point between epochs.
In order to avoid “dead gradient” zones, we modify the activation function of our feedforward network (described in Equation 6) to ρ mod ( x ) =
ρ ( x ) + 0 .
01 x , where the 0 .
01 “leak” is added to prevent the feed-forward neurons from getting stuck due to zero-gradients in the saturated regions.
We use λ = 0 .
1 as the regularizing parameter from Equation 10 , having scanned for values in Appendix D.
3. MNIST
We verify that the our algorithm works on the MNIST dataset.
The learning curves can be seen in
Figure 2.
We ﬁnd, somewhat surprisingly, that the forward pass of our network performs almost indistinguishably from the performance of the negative-phase of Equilibrium Propagation.
This encouraging result shows that this approach for training a feedforward network without backprop does indeed work.
We also see from from the top-two panels of Figure 2 that our approach can stabilize Equilibrium-Prop learning when we run the network for fewer steps than are needed for
, but gradients still point in the correct direction because the learning rate is scaled by − 1 /β .
This trick avoids an instability when, due to incomplete negative-phase convergence, the network continues approaching the true minimum of E ( s, x ) in the positive phase, and thus on every iteration contues to push down the energy of this “true” negative minimum
[Figure]
[Figure]
i [Figure] full convergence. By initializing the negative phase in a close-to-optimal regime, the network is able to learn when the number of steps is so low that plain Equilibrium Propagation cannot converge.
Moreover we note that as the number of steps is enough for convergence, there is not much advantage to using more negative-phase iterations - the longer negative phase does not improve our error.
In Figure 3 we demonstrate that using only local losses to update the feedforward network comes with no apparent disadvantage.
In line with our results from Section 2.4, we see that local loss gradients become aligned with the loss gradients from higher layers, explaining why it appears to be sufﬁcient to only use the local gradients.
R ELATED W ORK
The most closely related work to ours is by Bengio et al.
There, the authors examine the idea of initializing an iterative settling process with a forward pass.
They propose using the parameters of the Equilibriating network to do a forward pass, and describe the conditions under which this provides a good approximation of the energy-minimizing state.
Their conclusion is that this criterion is met when consecutive layers of the energy-based model form a good autoencoder.
Their model differs from ours in that the parameters of the forward model are tied to the parameters of the energy-based model.
The effects of this assumption are unclear, and the authors do not demonstrate a training algorithm using this idea.
Our work was loosely inspired by Hinton et al.
(2015), who proposed “distilling” the knowledge of a large neural network or ensemble into a smaller network which is designed to run efﬁciently at inference time.
In this work, we distill knowledge from a slow, equilibrating network in to a fast feedforward network.
4. R ELATION TO A DVERSARIAL L EARNING
Several authors (Kim & Bengio, 2016), (Finn et al., 2016), (Zhai et al., 2016) have pointed out the connection between Energy Based Models and Generative Adversarial Networks (GANs).
In these works, a feedforward generator network proposes synthetic samples to be evaluated by an energybased discriminator , which learns to push down the energy of real samples and push up the energy of synthetic ones.
In these models, both the generator/sample proposer and the discriminator/energybased-model are deep feedforward networks trained with backpropagation.
In our approach, we have a similar scenario.
The inference network f φ can be thought of as a conditional generator which produces a network state s f given a randomly sampled input datum x : s f = f φ ( x ) .
Parameters φ are trained to approximate the minimal-energy states of the energy function: min φ (cid:107) f φ ( x ) − arg min s E θ ( s, x ) (cid:107) .
However, in our model, the Energy-Based network
E θ
( s, x ) does not directly evaluate the energy of the generated data s f , but of the minimizing state s − = arg min s E
θ ( s, x ) which is produced by performing T − energy-minimization steps on s f (see Algorithm 1).
Like a discriminator, the energy-based model parameters θ learn based on a contrastive loss which pushes up the energy of the “synthetic” network state s − while pushing down the energy of the “real” state s + .
4. R ELATION TO A MORTIZED V ARIATIONAL I NFERENCE
In variational inference, we aim to estimate a posterior distribution p ( z | x ) over a latent variable
z given data x , using an approximate posterior q ( z ) .
Algorithms such as Expectation Maximization (Dempster et al., 1977) iteratively update a set of posterior parameters µ per-data point, so that z n ∼ q
( z | µ n ) .
In amortized inference , we instead learn a global set of parameters φ which can map a sample x n to a posterior estimate z n ∼ q φ ( z | x n ) .
Dayan et al.
(1995) proposed using a “recognition’ network” as this amortized predictor, and Kingma & Welling (2013) showed that you can train this recognition network efﬁciently using the reparameterization trick.
However, this comes at the expense of an “amortization gap” (Cremer et al., 2018) - where the posterior estimate suffers due to the sharing of posterior estimation parameters across data samples.
Several recent works (Marino et al., 2018), (Li et al., 2017), (Kim et al., 2018), have proposed various versions of a “teacher-student” framework, in which an amortized network q θ ( z | x ) provides an initial guess for the posterior, which is then reﬁned by a slow, non-amortized network which reﬁnes q ( z ) in several steps into a better posterior estimate.
The “student” then learns to reﬁne its posterior estimate using the ﬁnal result of the iterative inference.
In the context of training Deep Boltzmann Machines,
Salakhutdinov & Larochelle (2010) trained a feedforward network with backpropagation to initialize variational parameters which are then optimized to estimate the posterior over latent variables.
Initialized Equilibrium Propagation is a zero-temperature analog of amortized variational inference.
In the zero-temperature limit, the mean-ﬁeld updates of variational inference reduce to coordinate ascent on the variational parameters.
The function of the amortized student network q φ ( z | x ) is then analogous to the function of our initializing network f φ ( x ) , and the negative phase corresponds to the iterative optimization of varational parameters from the starting point provided by f φ ( x ) .
4. R ELATION TO OTHER WORK IN L OCAL C REDIT -A SSIGNMENT
Another interesting approach to shortening the inference phase in Equilibrium propagation was proposed by Kohan et al.
The authors propose a model that is almost a feedforward network, except that the output layer projects back to the input layer.
The negative phase consists of making several feedforward passes through the network, reprojecting the output back to the input with each pass.
Although the resulting inference model is not a feedforward network, the authors claim that this approach allows them to dramatically shorten convergence time of the negative phase.
There is also a notable similarity between Initialized Equilibrium Propagation and Method of Auxiliary Coordinates (Carreira-Perpinan & Wang, 2014).
In that paper, the authors propose a scheme for optimizing a layered feedforward network which consists of alternating optimization of the neural activations (which can be parallelized across samples) and parameters (which can be parallelized across layers).
In order to ensure that the layer activations z k remain close to what a feedforward network can compute, the objective includes a layerwise cost µ 2 (cid:107)
z k − f k ( z k − 1 ) (cid:107) 2 , where z k is layer k ’s activation, f k is layer k ’s function, and µ is a the strength of the layerwise cost (as they anneal µ → ∞ this cost becomes a constraint).
This is identical in form and function to our λ (cid:80) j ∈S ( s f − s − ) 2 term in Equation 10.
However, they differ from our method in that their neurons backpropagate their gradients back to input neurons (albeit only across one layer).
Taylor et al.
(2016) do something similar with using the Alternating Direction Method of Multipliers (ADMM), where Lagrange multipliers enforce the “layer matching” constraints exactly.
Both methods, unlike Equilibrium Prop, are full-batch methods.
More broadly, other approaches to backprop-free credit assignment have been tried.
DifferenceTarget propagation (Lee et al., 2015) proposes a mechanism to send back targets to each layer, such that locally optimizing targets also optimizes the true objective.
Feedback-Alignment (Lillicrap et al., 2014) shows that, surprisingly, it is possible to train while using random weights for the backwards pass in backpropagation, because the forward pass parameters tends to “align” to the backwards-pass parameters so that the pseudogradients tend to be within 90 ◦ of the true gradients.
A similar phenomenon was observed in Equilibrium Propagation by Scellier et al.
(2018), who showed that when one removed the constraint of symmetric weight in Equilibrium propagation, the weights would evolve towards symmetry through training.
Finally, Jaderberg et al.
(2016) used a very different approach - rather than create local targets, each layer predicts its own “pseudogradient”.
The gradient prediction parameters are then trained either by the true gradients (which no longer need to arrive before a parameter update takes place) or by backpropagated versions of pseudogradients from higher layers.
D ISCUSSION
In this paper we describe how to use a recurrent, energy-based model to provide layerwise targets with which to train a feedforward network without backpropagation.
This work helps us understand how the brain might be training fast inference networks.
In this view, neurons in the inference network learn to predict local targets, which correspond to the minimal energy states, which are found by the iterative settling of a separate, recurrently connected equilibrating network .
More immediately perhaps, this could lead towards efﬁcient analog neural network designs in hardware.
As pointed out by Scellier & Bengio (2017), it is much easier to design an analog circuit to minimize some (possibly unknown) energy function than it is to design a feedforward circuit and a parallel backwards circuit which exactly computes its gradients.
However it is very undesirable for the function of a network to depend on peculiarities of a particular piece of analog hardware, because then the network cannot be easily replicated.
We could imagine using a hybrid circuit to train a digital, copy-able feedforward network, which is updated by gradients computed in the analog hardware.
Without the constraint of having to backpropagate through the feedforward network, designs could be simpliﬁed, for example to do away with the need for differentiable activation functions or to use feedforward architectures which would otherwise suffer from vanishing/exploding gradient effects.
A G LOSSARY
Here we we have a reference of symbols used in the paper, in (Greek, Latin) alphabetical order.
α i ⊂ { j : j ∈ S , j (cid:54) = i }
The set of neurons in the Equilibrating Network that connect to neuron i
α fi =
j : j ∈ α
i , j < i } : The set of neurons in the Feedforward Network connected to neuron i
β ∈ R :
The perturbation factor, which modulates how much the augmented energy E βθ is affected by the output loss.
η ∈ R + : The learning rate.
θ : The set of parameters (all w ij ’s and b j ’s, in the Equilibrating network) ρ : a neuron nonlinearity.
In all experiments it is ρ ( x ) = max 0 , min 1 , x )) φ = ( ω, c ) :
The set of parameters (all ω ij ’s and c j ’s), in the feedforward network φ j : The set of parameters (all ω · j ’s and c j ’) belonging to a neuron j ( ω, c ) : The (weights, biases) of the feedforward network.
Collectively called φ C ( s O , y
) ∈ R : The output loss function, deﬁned on the states of the output units.
E θ ( s, x ) ∈ R :
The energy function of the Equilibrating network (Equation 1) produces a scalar energy given a set of states s and input x E
[Equation]
∈ R : The augmented energy function of the Equilibrating network, when it has been perturbed by a factor β by target data y f φ ( x ) (cid:55)→ s f : The initialization function: A feedforward network which initializes the state of the Equilibrating network.
I : The set of indices of input neurons.
L : The total loss in the Feedforward network’s prediction.
Deﬁned in EquationL
i : The local loss on the i (cid:48)
th neuron in the feedforward network.
Deﬁned in EquationO : The set of indices output neurons, a subset of S S :
The set of inidices non-input neurons.
s : The set of neuron states.
s :
= { s i : i ∈ S} ∈ R dim( S ) s − :=
arg min s E ( s, x ) ∈ R dim( S ) :
The minimizing state of the Energy function.
s + :=
arg min s E βθ ( s, x, y ) ∈ R dim( S )
The minimizing state of the augmented energy function.
s f :
= f φ ( x ) z ∈ R dim( S ) :
The state output by the feedforward network.
s O ∈ R dim( O ) : the states of the output units T − , T + : Hyperparameters for Equilibrium Prop deﬁning the number of steps of convergence of the negative/positive phase.
w, b : the parameters of the Equilibrating network (collectively called θ ) x ∈ R dim I : The input data
y ∈ R dim O : The target data
B G RADIENT A LIGNMENT
Here to derive the result in Equation 12.
First
[Equation]
we restate Equation 11 substituting w ∗ =
w + ∆ w :
[Equation]
Where:
[Equation]
Now we will compute the gradient of each of the local losses with respect to ∆ w 1 , in the limit where ∆ w 1 is small.
[Equation]
1st order Taylor Expansion about
[Equation]
1st order Taylor Expansion about ∆ w
[Equation]
=
[Equation]
= 0 )
[Equation]
Why do we observe gradient alignment even at random initialization? Let us start with the same 2-layer network deﬁned in Appendix B
[Equation]
G A and G B tend to be aligned because the terms ρ ( xw 1 ) and ρ ( ρ ( xw 1 ) w 2 ) (cid:12) ρ (cid:48) ( ρ ( xw 1 ) w 2 ) w Ttend to be aligned.
Suppose ρ is a piecewise saturating nonlinearity (as we have in this paper) with ρ ( x ) =
[ a if ( x < a ); x if ( x ∈ [ a, b ]); b otherwise ]
{+Then we can deﬁne a weight matrix w (cid:48) 2 by ﬁltering rows of w 2 to only include weights projecting to non-saturated neurons: w (cid:48) 
+}
= [ w ( i ) 2 ∀
i : ρ (cid:48) ( ρ ( xw 1 ) w ( i ) 2 ) (cid:54) = 0] Where w ( i ) 2 denotes the i (cid:48)
th row of w 2 .
Then our second term can be rewritten as: ρ ( ρ ( xw 1 ) w 2 ) (cid:12) ρ (cid:48)
( ρ ( xw
[Equation]
w
[Equation]
2 w
[Equation]
T 2 .
Given a random matrix w (cid:48) 2 , the matrix w (cid:48) 2 w (cid:48) T 2 will tend to have a strong diagonal component, causing this term to be aligned with ρ ( xw 1 )
D E FFECT OF λ PARAMETER
In Equation 10 we introduce a new parameter λ which encourages the state of the equilibrating network to state close to that of the forward pass.
Here we perform a sweep of parameter λ to evaluate its effect.
[Figure]
different number of steps of negative-phase convergence.
We see in Figure 4 that when the number of steps of negative-phase convergence is small, introducing λ can allow for more stable training.
This makes sense - if the minimizing state of the equilibating network is “pulled” towards the state at the forward pass, it should take fewer steps of iteration to reach this state when initialized at the state of the forward pass.
However, we also see that training fails when λ is too high.
We believe this is because the simple iterative settling scheme (Euler integration) used in this paper, as well as the original Equilibrium Prop by Scellier & Bengio (2017), can become unstable when optimizing a loss surface with sharp, steep, minima (as are induced with large λ ).
This could be addressed in later work by either using an adaptive λ term or an adaptive
Euler-integration step-size.