The brain processes information through many layers of neurons. This deep architecture is representationally powerful 1, 2, 3, 4 , but it complicates learning by making it hard to identify the responsible neurons when a mistake is made 1, 5 . In machine learning, the backpropagation algorithm 1 assigns blame to a neuron by computing exactly how it contributed to an error. To do this, it multiplies error signals by matrices consisting of all the synaptic weights on the neuron's axon and farther downstream. This operation requires a precisely choreographed transport of synaptic weight information, which is thought to be impossible in the brain 1, 6, 7, 8, 9, 10, 11, 12, 13, 14 . Here we present a surprisingly simple algorithm for deep learning, which assigns blame by multiplying error signals by random synaptic weights. We show that a network can learn to extract useful information from signals sent through these random feedback connections. In essence, the network learns to learn. We demonstrate that this new mechanism performs as quickly and accurately as backpropagation on a variety of problems and describe the principles which underlie its function. Our demonstration provides a plausible basis for how a neuron can be adapted using error signals generated at distal locations in the brain, and thus dispels long-held assumptions about the algorithmic constraints on learning in neural circuits. Networks in the brain compute via many layers of interconnected neurons 15,16 . To work properly neurons must adjust their synapses so that the network's outputs are appropriate for its tasks. A longstanding mystery is how upstream synapses (e.g. the synapse between α and β in Fig. 1a) are adjusted on the basis of downstream errors (e.g. e in Fig. 1a) . In artificial intelligence this problem is solved by an algorithm called backpropagation of error 1 . Backprop works well in real-world applications 17,18,19 , and networks trained with it can account for cell response properties in some areas of cortex 20,21 . But it is biologically implausible because it requires that neurons send each other precise information about large numbers of synaptic weights -i.e. it needs weight transport 1, 6, 7, 8, 12, 14,22 (Fig. 1a, b) . Specifically, backprop multiplies error signals e by the matrix W T , the transpose of the forward synaptic connections, W (Fig. 1b) . This implies that feedback is computed using knowledge of all the synaptic weights W in the forward path. For this reason, current theories of biological learning have turned to simpler schemes such as reinforcement learning 23 , and "shallow" mechanisms which use errors to adjust only the final layer of a network 4,11 . But reinforcement learning, which delivers the same reward signal to each neuron, is slow and scales poorly with network size 5, 13, 24 . And shallow mechanisms waste the representational power of deep networks 3,4,25 . Here we describe a new deep-learning algorithm that is as fast and accurate as backprop, but much simpler, avoiding all transport of synaptic weight information. This makes it a mechanism the brain could easily exploit. It is based on three insights: (i) The feedback weights need not be exactly W T . In fact, any matrix B will suffice, so long as on average, where e is the error in the network's output (Fig. 1a) . Geometrically, this means the teaching signal sent by the matrix, Be, lies within 90 • of the signal used by backprop, W T e, i.e. B pushes the network in roughly the same direction as backprop would. (ii) Even if the network doesn't have this property initially, it can acquire it through learning. The obvious option is to adjust B to make equation (1) true, but (iii) another possibility is to do the same by adjusting W . We will show this can be done very simply, even with a fixed, random B (Fig. 1c ). We first demonstrate that this mechanism works for a variety of tasks, and then explain why it works. For clarity we consider a three-layer network of linear neurons (see Random feedback weights can deliver useful teaching signals to preceding layers of a neural network. a, The backprop learning algorithm is powerful, but requires biologically implausible transport of individual synaptic weight information. For backprop, neurons must know each other's synaptic weights, e.g. the three coloured synapses on the feedback cell at bottom must have weights equal to those of the corresponding coloured synapses on three cells in the forward path. On a computer it is simple to use the synaptic weights in both forward and backward computations, but synapses in the brain communicate information unidirectionally. b, Implemented in a computer, backprop computes how to change hidden unit activities by multiplying the error vector, e = y * − y, by the transpose matrix of the forward weights, i.e., ∆h BP = W T e. c, Our feedback alignment method uses the counterintuitive observation that learning is still effective if W T is replaced by a matrix of fixed random weights, B, so that ∆h FA = Be. d, Four algorithms learn to mimic a linear function: 'shallow' learning (light gray), reinforcement learning (dark gray), backprop (black), and feedback alignment (green). NSE is normalized squared error. e, Angle between the hidden-unit update vector prescribed by feedback alignment and that prescribed by backprop, i.e., ∆h FA ∆h BP . Error bars are two standard deviations for a sliding window of 10 examples. 3 Methods). The network's output is y = W h, where h is the hidden-unit activity vector, given by h = W 0 x, where x is the input to the network. W 0 is the matrix of synaptic weights from x to h and W is the weights from h to y. The network learns to approximate a linear function, T (for "target"). Its goal is to reduce the squared error, or loss, T e, where the error e = y * − y = T x − y. We trained the network using four algorithms (Fig. 1d) . If only the output weights, W , are adjusted, as in shallow methods, then the loss hardly decreases. If a fast variant of reinforcement learning is used to adjust the hidden weights, W 0 , then there is some progress but it is slow. In contrast, backprop sends the loss rapidly towards zero. It adjusts the hidden-unit weights according to the gradient of the loss, T . Thus, backprop adjusts the hidden units according to the vector: ∆h BP = W T e. Here and throughout ∆h denotes the update sent to the hidden layer, rather than the change in the hidden units. Our new algorithm adjusts W in the same way as backprop (∆W ∝ (∂L/∂W ) = −eh for ∆h it uses a much simpler formula, which needs no information about W or any other synapses but instead sends e through a fixed random matrix B, This algorithm, which we call feedback alignment drives down the loss as quickly as backprop does (Fig. 1d) . The network learns how to learn -it gradually discovers how to use B, which then allows effective modification of the hidden units. At first, the updates to the hidden layer are not helpful, but they quickly improve by an implicit feedback process that alters W so that e T W Be > 0. To reveal this, we plot the angle between the hidden-unit updates prescribed by feedback alignment and backprop, ∆h FA ∆h BP ( Fig. 1e ; see Methods). Initially the angles average about 90 • . But they soon shrink, as the algorithm begins to take steps that are closer to those of backprop. This alignment of the ∆h's implies that B has begun to act like W T . And because B is fixed, the alignment is driven by changes in the forward weights W . In this way, random feedback weights come to transmit useful teaching signals to neurons deep in the network. Feedback alignment learning also solves nonlinear, real-world problems. We ran it on a benchmark classification problem (Fig. 2a) , learning to recognize handwritten digits 17 (see Methods). On this task, backprop brings the mean error on the test set to 2.4% (average of n=20 runs). Feedback alignment learns as quickly as backprop, reaches 2.1% mean error (n=20), and develops similar feature detectors (Supplemen- tary Fig. S1 ). As measured by the angle, ∆h FA ∆h BP , feedback alignment quickly learns to use the random weights to transmit useful error information to the hidden units (Fig. 2b) . Even when we randomly remove 50% of the elements of the W and B matrices, so that neurons in h and y have a 25% chance of reciprocal connection, feedback alignment still matches backprop (2.4% mean error; n=20). Some tasks are better performed by networks with more than one hidden layer, but for that we need a learning algorithm that can exploit the extra power of a deeper network 2,3,4 (Fig. 2c) . Backprop assigns blame to a neuron by taking into account all of its downstream synapses. Thus, the update for the first hidden layer in a four-layer network (Fig. 2c) is ∆h (Fig. 2c) . On a non-linear function-fitting task (see Methods), both backprop (t-test, n = 20, p = 3 × 10 −12 ; Fig. 2d ) and feedback alignment (t-test, n = 20, p = 9 × 10 −13 ; Fig. 2d ) deliver better performance with a four-layer network than with a three-layer network. Thus, the algorithm builds useful feature detectors in the deepest layers of a network by relaying errors via random connections. This flexibility makes feedback alignment more plausible for the brain: learning proceeds even when error vectors are indiscriminately broadcast via random feedback weights to multiple layers of cells. Why does feedback alignment work? Its mechanism arises from certain novel network 6 dynamics. To explain them, we consider a minimal network with just one linear neuron in each layer (Fig. 3a , see Methods). We visualize (Fig. 3b) how the network's two weights, W 0 and W , evolve when the feedback weight B is set to 1. The flow field shows that the system moves along parabolic paths. From most starting points the network weights travel to the hyperbola at upper right (Fig. 3b) . This hyperbola is a set of stable equilibria solutions where W > 0 and therefore e T W Be > 0 for all e -that is, equation (1) is satisfied, which means W has evolved so that the feedback matrix B is delivering useful teaching signals. In higher dimensions we can identify conditions under which feedback alignment is guaranteed to reduce errors to zero (Supplementary Proof 1) . Importantly, the proof holds for cases where the error can reach zero only if B transmits useful information to the hidden neurons. The proof also demonstrates that high-dimensional analogues of the pattern of parabolic paths seen in the minimal network (Fig. 3a, b) , also hold for networks with large numbers of units. Indeed, the proof hinges on the fact that feedback alignment yields the relation where C is a constant, i.e. the left-hand side is a quadratic function of W 0 . Feedback alignment updates do not converge with backprop (Fig. 1e, Fig. 2b ), superficially suggesting that they are merely suboptimal approximations of ∆h BP . Further analysis shows this view is too simplistic. Our proof says that weights W 0 and W evolve to equilibrium manifolds, but simulations (Fig. 4) and analytic results (Supplementary Proof 2) hint at something more specific: that when the weights begin near 0, feedback alignment encourages W to act like a local pseudoinverse of B around the error manifold. This fact is important because if B were exactly W + (the MoorePenrose pseudoinverse of W ), then the network would be performing Gauss-Newton optimization (Supplementary Proof 3). We call this update rule for the hidden units pseudobackprop and denote it by ∆h PBP = W + e. Experiments with the linear network show that the angle, ∆h FA ∆h PBP quickly becomes smaller than ∆h FA ∆h BP (Fig. 4b, c ; see Methods). In other words feedback alignment, despite its simplicity, displays elements of second-order learning. In the 1980's, new artificial network learning algorithms promised to provide insight into brain function 1 . But the most powerful class of algorithms use error signals tailored to each neuron and seemed impossible to implement in the brain because they required weight transport 6, 7 . More-plausible algorithms have been devised 9, 10, 13, 14,22, 26, 27, 28, 29, 30 , but these either fall far short of backprop's speed or call for a lot of additional processing 10, 11, 12, 13, 22 . In marked contrast, the mechanism developed here is much simpler than backprop, but still matches its speed and accuracy. Feedback alignment dispels the central assumption of previous neuron-specific algorithms -that error information 8 must be precisely tailored for each neuron. Our work shows that it is far easier than previously thought to send neuron-specific teaching signals through a deep network: all you need is random feedback connections. Thus, the principles underlying feedback alignment learning are compatible with many brain circuits in which reciprocal feedback connections exist, such as occur within, and between regions of the neocortex 15,16 . This makes it an attractive basis for understanding various forms of learning in deep networks, including the integration of sensory information and motor adaptation processes. Finally, feedback alignment may offer new opportunities to integrate neuroscience with recent advances in machine learning which have highlighted the power of deep architectures 2, 3, 18,19,25 . We trained feedforward networks on three tasks. In all cases the goal was to minimize the square of the error, L = (1/2)e T e, where e = y * − y is the difference between the desired and actual output. Task 23,24 . We chose the learning rate η for each algorithm via manual search 31 in order to optimize learning speed. Task (2): A 784-1000-10 network with standard sigmoidal hidden and output units (i.e., σ(x) = 1/(1+exp(−x))) was trained to classify images of handwritten digits, 0-9. Each unit had an adjustable input bias. Standard 1-hot representation was used to code desired output. The network was trained with 60,000 images from the standard MNIST dataset 17 , and performance was measured as the percentage of errors made on a held aside test set of 10,000 images. Both algorithms used a learning rate of, η = 10 −3 , and weight decay, α = 10 −6 . Parameter updates were the same as those used in the linear case, but with ∆h FA = (Be) • σ , where • is element-wise multiplication and σ is the derivative of the output activations. A 30-20-10 and 30-20-10-10 network were trained to approximate the output of a 30-20-10-10 target network. All three networks had tanh(·) hidden units, linear output units, and an adjustable input bias for each unit. Input/output training pairs were produced via, y * = W 2 tanh(W 1 tanh(W 0 x + b 0 ) + b 1 ) + b 2 , with x ∼ N (µ = 0, Σ = I). The angle between two vectors, e.g. a b, was computed as: θ = cos −1 (||a T b||/(||a|| · ||b||)).
Addressing the weight transport problem (the weight symmetry constraint) was also done for the supervised case using feedback alignment REF : even if the feedback weights do not exactly match the feedforward weights, the latter learn to align to the former and "back-propagation" (with the wrong feedback weights) still works.
Random feedback weights support learning in deep neural networks
ArXiv
ArXiv
Computer Science Biology