Latent Equilibrium: A unified learning theory for arbitrarily fast computation with arbitrarily slow neurons

1 Introduction . Physical systems composed of large collections of simple , but intricately connected elements can exhibit powerful collective computational properties . A prime example are animals ’ nervous systems , and most prominently the human brain . Its computational prowess has motivated a large , crossdisciplinary and ongoing endeavor to emulate aspects of its structure and dynamics in artificial substrates , with the aim of being ultimately able to replicate its function . The speed of information processing in such a system depends on the response time of its components ; for neurons , for example , it can be the integration time scale determined by their membrane time constant . * Senior authors . §Code is available at https : //github.com/unibe-cns/le_NeurIPS_code . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . If we consider hierarchically organized neural networks composed of such elements , each layer in the hierarchy causes a response lag with respect to a changing stimulus . This lag introduces two related critical issues . For one , the inference speed in these systems decreases with their depth . In turn , this induces timing mismatches between instructive signals and neural activity , which disrupts learning . For example , recent proposals for bio-plausible implementations of error backpropagation ( BP ) [ 1–4 ] in the brain all require some form of relaxation , both for inference and during learning [ 5–11 ] . Notably , this also affects some purely algorithmic methods involving auxiliary variables [ 12 ] . To deal with this inherent property of physical dynamical systems , two approaches have been suggested : either phased plasticity that is active only following a certain relaxation period , or long stimulus presentation times with small learning rates . Both of these solutions entail significant drawbacks : the former is challenging to implement in asynchronous , distributed systems such as cortical networks or neuromorphic hardware , while the latter results , by construction , in slow learning . This has prompted the critique that any algorithm requiring such a settling process is too slow to describe complex brain function , particularly when involving real-time responses [ 13 ] . To the best of our knowledge this fundamental problem affects all modern models of approximate BP in biological substrates [ 5–11 ] . To overcome these issues , we propose a novel framework for fast computation and learning in physical substrates with slow components . As we show below , this framework jointly addresses multiple aspects of neuronal computation , including neuron morphology , membrane dynamics , synaptic plasticity and network structure . In particular , it provides a biologically plausible approximation of BP in deep cortical networks with continuous-time , leaky neuronal dynamics and local , continuous plasticity . Moreover , our model is easy to implement in both software and hardware and is well-suited for distributed , asynchronous systems . In our framework , inference can be arbitrarily fast ( up to finite simulation resolution or finite communication speed across physical distances ) despite a finite response time of individual system components ; downstream responses to input changes thus become effectively instantaneous . Conversely , responses to instructive top-down input that generate local error signals are also near-instantaneous , thus effectively removing the need for any relaxation phase . This allows truly phase-free learning from signals that change on much faster time scales than the response speed of individual network components . Similarly to other approaches [ 5 , 6 , 9 , 14 , 15 ] , we derive neuron and synapse dynamics from a joint energy function . However , our energy function is designed to effectively disentangle these dynamics , thus removing the disruptive co-dependencies that otherwise arise during relaxation . This is achieved by introducing a simple , but crucial new ingredient : neuronal outputs that try to guess their future state based on their current information , a property we describe as “ prospective ” ( which should not be confused with the “ predictive ” in predictive coding , as we also discuss below ) . Thereby , our framework also constructs an intimate relationship between such “ slow ” neuronal networks and artificial neural network ( ANN ) 1 , thus enabling the application of various auxiliary methods from deep learning . 2 The problems of slow components . To illustrate the issue with relaxation , we consider two neurons arranged in a chain ( Fig . 1a ) . Biological neuronal membrane potentials u are conventionally modeled as leaky integrators of their input I : Cmu̇ = −glu + I , where the membrane capacitance Cm and leak conductance gl determine the membrane time constant τm : = Cm/gl and thereby its response speed . These dynamics attenuate and delay input , resulting in significant differences between neuronal output rates and those expected in an instantaneous system ( Fig . 1b ) . This mismatch increases with every additional layer : a feedforward network with n layers has an effective relaxation time constant of approximately nτm . Besides slow inference , this delayed response leads to critical issues during learning from downstream instructive signals . Consider the typical scenario where such a target signal is present in the output layer and plasticity is continuously active ( and not phased according to some complicated schedule ) . If the system fulfills its task correctly , the target signal corresponds to the output of the relaxed system and no learning should take place . However , due to delays in neuronal responses , the output signal differs from the target during relaxation , which causes plasticity to adapt synaptic weights in an effort to better match the target . As a result , the system “ overshoots ” during early relaxation and has to 1To differentiate between biologically plausible , leaky neurons and abstract neurons with instantaneous response , we respectively use the terms “ neuronal ” and “ neural ” . undo these synaptic changes through further learning in the late relaxation phase ( Fig . 1c ) . Also , since output errors need inputs to propagate forward through the entire network before propagating backwards themselves , the input layer only observes correct errors after about 2nτm . We refer to this issue as the “ relaxation problem ” . In the following , we present a solution for this problem , combining prospective neuron dynamics with continuously active , local synaptic plasticity . 3 Fast computations in slow substrates . Inspired by the canonical coordinates from classical mechanics , we describe the state of a neuronal network by its position in phase space ( u , u̇ ) , with the generalized position u and the generalized momentum u̇ . The relation between these two components describes the physics of the system . To obtain the leaky integrator dynamics that characterize biological neurons , we first define the abstract network state ŭm as ŭm : = u+ τmu̇ . ( 1 ) We next define an energy function E that characterizes this state : E ( ŭm ) : = 1 2 ‖ŭm −Wϕ ( ŭm ) − b‖2 + βL ( ŭm ) . ( 2 ) Here , W and b represent the weight matrix and bias vector , ϕ is the neuronal activation function and the loss L is scaled by a constant β ; we use bold font for matrices , vectors , and vector-valued functions . Intuitively speaking , E measures the difference between “ what neurons guess that they will be doing in the future ” ( ŭm ) and “ what their biases and presynaptic afferents believe they should be doing ” ( b+Wϕ ( ŭm ) ) . Furthermore , for a subset of neurons , it adds a loss L that provides an external measure of the error for the states of these neurons , determined by instructive signals such as labels , reconstruction errors or rewards . This formulation of the energy function is very generic and can , by appropriate choice of parameters , apply to different network topologies , including multilayer perceptrons , convolutional architectures and recurrent networks . Note that this energy can be written as a neuron-wise sum over mismatch energies plus , for factorizing loss functions , a local loss term Ei ( ŭ m i , ri , pre ) = 1 2 ( ŭ m i −Wiri , pre − bi ) 2 + βLi ( ŭmi ) , where ri , pre : = ϕ ( ŭ m i , pre ) is the presynaptic input vector for the ith neuron in the network . We can now derive neuronal dynamics as extrema of the energy function from Eqn . 2 : ∇ŭmE = 0 =⇒ τmu̇ = −u+Wϕ ( ŭm ) + b+ e , ( 3 ) with presynaptic bottom-up input Wϕ ( ŭm ) and top-down error signals e = ϕ′ ( ŭm ) W T [ ŭm −Wϕ ( ŭm ) − b ] ( 4 ) for hidden neurons and e = −β∇ŭmL for neurons which directly contribute to the loss . Plugging Eqn . 3 into Eqn . 4 , it is easy to see that in hierarchical networks these errors can be expressed recursively over layers ` , e ` = ϕ′ ( ŭm ) W T ` +1e ` +1 , thus instantiating a variant of BP . Eqn . 3 can be interpreted as the dynamics of a structured pyramidal neuron receiving presynaptic , bottom-up input via its basal dendrites and top-down input via its apical tree ( Fig . 1d ) . We provide a more detailed description of our model ’ s biophysical implementation in Section 5 . We note that our approach is in contrast to previous work that introduced neuron dynamics via gradient descent on an energy function , such as [ 5 , 6 , 16 ] , whereas we require a stationary energy function with respect to ŭm . Indeed , this difference is crucial for solving the relaxation problem , as discussed below . Since for a given input our network moves , by construction , within a constant-energy manifold , we refer to our model as Latent Equilibrium ( LE ) . We can now revisit our choice of ŭm from a functional point of view . Instead of the classical output rate ϕ ( u ) , our neurons fire with ϕ ( ŭm ) , which depends on both u and u̇ ( Eqn . 1 ) . As neuron membranes are low-pass filters ( Eqn . 3 ) , ŭm can be viewed as a prospective version of u : when firing , the neuron uses its currently available information to forecast the state of its membrane potential after relaxation . The prospective nature of ŭm also holds in a strict mathematical sense : the breve operator ˘m : = ( 1 + τmd/dt ) is the exact inverse of an exponential low-pass filter ( see SI ) . While neuronal membranes continue to relax slowly towards their steady states , neuronal outputs use membrane momenta to compute a correct instantaneous reaction to their inputs , even in the case of jumps ( Fig . 1e ) . Thus , information can propagate instantaneously throughout the network , similarly to an ANN , counterintuitively even when membrane dynamics are never in equilibrium . The activity of the output layer hence reflects arbitrarily fast changes in the input – even on time scales smaller than the neuronal time constant τm – rather than responding with a significant time lag and attenuation as in previous , gradient-based models . The idea of incorporating derivatives into the input-output function of a system has a long history in control theory [ 17 ] and also represents a known , though often neglected feature of ( single ) biological neurons [ 18 , 19 ] . A related , but different form of neuronal prospectivity has also been considered in other models of bio-plausible BP derived from a stationary action [ 20 , 21 ] . At the level of neuronal populations with additive Gaussian noise , there exists a long tradition of studying faster-than-τm responses , both with [ 22 ] and without [ 23 ] recurrent connectivity . Similar observations also hold for single neurons in the presence of noise [ 24 , 25 ] . Building on these insights and integrating them into a unified theory of neuronal dynamics and learning , our model proposes a specific form of prospective coding that can also be learned by local adaptation mechanisms , as we discuss in Section 6 . We should also stress the difference between the terms “ prospective ” and “ predictive ” . Predictive coding , or more generally , predictive processing , is a theory of brain function which proposes that brains maintain internal models of their environment which they update and learn by trying to predict sensory input at the same moment in time . Originally based on a specific Bayesian model and error communication scheme [ 14 ] , the notion of predictive coding can be generalized to layers in a hierarchical model predicting the activities of subsequent layers . This principle is instantiated in our networks as well , as will become clear in the following sections . In contrast , prospective coding in our framework refers to the ability of neurons to “ look forward in time ” using the current state of the membrane potential in phase space ( position and velocity ) , as described above.2 Prospective and predictive coding are thus complementary : our specific prospective mechanism provides a cure for relaxation in predictive coding networks , thus significantly speeding up both learning and inference , as we describe below . Note that , even for functionally feedforward networks , our resulting network structure is recurrent , with backward coupling induced by error inputs to the apical tree . As a non-linear recurrent network , it can not settle instantaneously into the correct state ; rather , in numerical simulations , it jumps quickly towards an estimated stationary activity state and reaches equilibrium within several such jumps ( of infinitesimal duration ) . In practice , saturating activation functions can help avoid pathological behavior under strong coupling . Moreover , we can introduce a very short exponential low-pass filter τ s on top-down signals , slightly larger than the temporal resolution of the simulation . Thus , in physical systems operating in continuous time , τ s can effectively become infinitesimal as well and does not affect the speed of information propagation through the network . In particular , as we discuss below , the perpetual consistency between input and output allows our model to continuously learn to reduce the loss , obviating the need for network relaxation phases and the associated global control of precisely timed plasticity mechanisms .

Summary: This paper asks an important question, proposes novel ideas, and presents compelling contributions. The problem being addressed is fundamental and yet has received not much attention,  with no existing good solution. The problem is that physical and biological elements in the brain are slow. Computations propagating through a neural network hierarchy with such elements will introduce time lag.  The mismatch between teaching signals and the current state of the system,  when the input is constantly changing, presents a problem for learning.  The two existing solutions involve (1) separating inference and learning into the "relaxation" phase for the computation to complete and the "learning" phase for weight update, or (2) use a very slow weight update rate. Both of these are suboptimal.  They proposed a prospective energy function in which the membrane potential of a neuron is trained to map onto a latent equilibrium space where during the relaxation state, the instantaneous outputs of the neurons are the prospective target membrane potentials of the neurons with those recurrent computations. This will allow weight update to be done concurrently with relaxation and solving a problem of achieving arbitrary fast computation with arbitrarily slow neurons.  The resulting model can be interpreted as a biologically plausible approximation of error backpropagation in deep cortical networks with continuous-time leaky neuronal dynamics and continuous active local plasticity. They showed this method work in MINST and CIFAR100, and in fact, the method works better than BP during the early epochs of training.