Out of equilibrium learning dynamics in physicalallosteric resistor networks
Menachem Stern∗
Department of Physics and Astronomy
University of Pennsylvania

Sam Dillavou
Department of Physics and Astronomy
University of Pennsylvania

Marc Z. Miskin
Department of Electrical and Systems Engineering
University of Pennsylvania

Douglas J. Durian
Department of Physics and Astronomy
University of Pennsylvania

Andrea J. Liu
Department of Physics and Astronomy
University of Pennsylvania

Abstract
Physical networks can learn desirable functions using local learning rules in space
and time. Real learning systems, like natural neural networks, can learn out of
equilibrium, on timescales comparable to their physical relaxation. Here we study
coupled learning, a framework that supports learning in equilibrium in diverse
physical systems. Relaxing the equilibrium assumption, we study experimentallyand theoretically how physical resistor networks learn allosteric functions far fromequilibrium. We show how fast learning produces oscillatory dynamics beyond a
critical threshold, and that learning succeeds well beyond that threshold. Thesefindings show how coupled learning may train systems much faster than assumed
before, suggesting their applicability to slowly relaxing physical systems.

1

Introduction

Recent years have seen the explosion of learning based methods in the design and manufacturing ofphysical networks with desired properties [1, 2, 3, 4, 5], most of which use computational machinelearning. More recently, new learning methods emerged, where the physical network itself is trained,
evolving its parameters in order to perform desired tasks. Such physical learning machines have
either minimized the energy of their desired response (directed aging) [6, 7, 8], or used contrastive
learning [9, 10, 11, 12, 13] to minimize cost functions of interest. As an example, a network ofvariable resistors whose resistances are tuned by minimizing the cost function has “physical" degrees
of freedom (e.g. currents on edges) and “learning" degrees of freedom (e.g. resistances of edges).
The physical degrees of freedom adjust rapidly so that the system is always fully equilibrated withrespect to the physical degrees of freedom at each step of the learning process.
Algorithms that rely on equilibration of physical degrees of freedom during learning have not seen
wide use in machine learning [14] because the equilibration slows their performance. Recent work
has relaxed that requirement slightly, showing that continual update of the learning parameters (asthe system equilibrates) is possible as long as the learning rate is low enough [15, 16, 17].
Certain natural neural networks, however, do not require that the learning rate is slow compared toequilibration of physical degrees of freedom. They reach a steady spiking state on a similar timescale∗nachis@sas.upenn.edu

Fourth Workshop on Machine Learning and the Physical Sciences (NeurIPS 2021).

with synaptic plasticity processes [18, 19, 20]. If the environment supplying the training signals
changes frequently, and the learning system must quickly adapt to the changing environment, it issensible for learning to occur on similar timescales to that of the system response (equilibration of
the physical degrees of freedom). This may also be the case in the humoral immune response, where
antibodies are formed rapidly after infection, even before the onset of symptoms [21, 22].
Here we show that learning is highly effective even when equilibration rates of the physical andlearning degrees of freedom are comparable. We find that for simple allosteric learning tasks [23](where target nodes produce desired outputs when inputs are applied at source nodes) in linear resistor
networks, learning can be made faster than the physical dynamics. This has the effect of speeding up
network training in real time by orders of magnitude. We show experimental and numerical evidence
for a critical learning rate beyond which learning dynamics become oscillatory, yet still successful.

2

Out of equilibrium coupled learning

In most contrastive learning approaches, physical degrees of freedom xa first equilibrate, and then theequilibrium states (denoted by stars) x∗a are used to modify learning degrees of freedom wj . Coupled∗C
learning [12] compares two equilibrium states (free state x∗Fa and clamped state xa ) to derive local
learning rules (approximating the gradient of the cost function). Suppose that a network trains on
an allosteric task, in which when a set of constraints are applied to source nodes XS , the network
responds by having desired values at (typically physically distant) target nodes XT . Such allostericeffects are not generic in physical networks, but are common in biology, for example in proteins(mechanical allostery) and vasculature (flow allostery) [23]. A free state x∗F
a is defined by applying
source constraints XS and letting the network equilibrate, reading off the target values x∗F
T . Then, a∗F∗F
supervisor clamps the target nodes, nudging them toward the desired output xC=x+η[XT −xT ],
T
T
with η  1. The coupled learning rule is given byẇj = αη −1∂∗C{E ∗F (XS , x∗F(XS , xC
T )−E
T )},
∂wj

(1)

where E ∗F and E ∗C are the energies (or dissipated power) of the equilibrium free and clamped states,and α a learning rate. The contrastive function C ∗ ≡ η −1 [E ∗C − E ∗F ] was shown to approximate
2
the mean squared error cost function C ∗ ∼ (x∗F
T − XT ) in the limit η → 0 [12].
The coupled learning rule above assumes both the free and clamped states are in equilibrium (orsteady state). Suppose instead that the network learns at the same time as it attempts to equilibrate,
following its physical dynamics, e.g. overdamped dynamics:ẋF = −γ∂x E F;ẇ = −α∂w C,(2)

α/γ sets the ratio between learning and physical rates (α/γ → 0 is the quasi-static limit of coupledlearning). Note that computing Eq. 2 is linear in network size, so the computational efficiency oflearning is comparable to back-propagation on feed-forward neural networks.
We study the special case of linear electric resistor networks. Here, the physical degrees of freedom arevoltages at nodes va , and the learning degrees of freedom are edge conductances (inverse resistances)gj = Rj−1 . The voltage drop over each edge is the difference in voltage between the two nodes
connected by that edge ∆vj ≡ ∆jaP
va , with ∆ja the incidence matrix. The power dissipation is
given by a sum over edges P = 12 j gj ∆vj2 . The network steady state is given by voltages va
minimizing the power dissipation P, subject to constraints. In ideal networks without capacitance,
the network reaches its steady state instantaneously. Once capacitance is relevant, it decreases thephysical relaxation rate γ, and we can simulate the system, e.g. by postulating overdamped dynamicsv̇aF,C = − γX∆Taj gj ∆ja vaF,C(3)

jġj =αη−1{[∆vjF ]2 − [∆vjC ]2 },
2

A

B

C

4V

D

0V

E

10 22

2200 F
2.2 F

5V

10 00

2V
InputsDesired Outputs

10 -2-2
αexp
exp/γexp
exp

Figure 1: Out of equilibrium learning in a physical network. (A) A single edge (in both free andclamped networks) with parallel 2.2µF capacitors highlighted with arrows. (B) A single edge with aparallel 2200µF capacitors. (C) Network structure with inputs and outputs for the allosteric task usedin (D)-(E). Typical error traces for an allosteric task as a function of training steps (D) and relaxation
times (E). Colors indicate ratio of learning to relaxation rates.
A

B

D

C

Figure 2: Out of equilibrium learning in simulated resistor networks. (A) Disordered resistor network(64 nodes, 5, sources, 3, targets). Full (empty) circles denote source (target) nodes, dot sizes showing
the magnitudes of the source voltages VS (target voltages VT ). Edge thickness indicates power
through each resistor. (B) Error during training at different ratios of learning to physical rates α/γ.
When α & γ, more training steps are required to reduce the error (dotted line - α/γ → 0). (C) Higher
learning rate results in more rapid learning in real time. (D) The error in trained networks does notchange up to α ≈ γ. Even beyond α ≈ γ, learning greatly reduces errors far from equilibrium.
Physical experiments.– To study if physical and learning dynamics can be coupled in the way suggested by Eqs. 2-3, we perform experimental and numerical studies. In recent work, we demonstrated
a variable electric resistor learning network [24] based on coupled learning [12]. The experiment
operated in the quasi-static regime, where learning was far slower than the network response times.
To relax the quasi-static condition, we slow down the physical relaxation within such a network using
capacitors in parallel to each variable resistor, as shown in Fig. 1AB. The charging of capacitors
increases the time to reach steady current in the network. During the experiment a pause τ /2 prior
to each learning step allows the system to partially equilibrate before updating its resistance value.
The learning rate is αexp = τ −1 and the physical relaxation rate is γexp ≈ (R0 c)−1 , where c is thecapacitance parallel to each resistor (2.2, 22, 220, or 2200 µF ). We estimate the relaxation rate using
resistance R0 ≡ 10KΩ (each resistor varies between 781Ω − 100KΩ).
The system is trained for a 2-target, 2-source allosteric task (Fig. 1C) as capacitance and τ are varied.
We measure the instantaneous mean-squared error, i.e. the squared difference between the free state
outputs and the desired outputs, normalized by its initial value. When physical relaxation is slow
compared to the learning rate (αexp /γexp  1) the system learns the task and the error hovers at
the noise floor (Fig. 1DE). In this regime α/γ does not affect the training time when measured intraining steps (Fig. 1D) but does when measured in time (Fig. 1E). As the learning rate is increased,
the system’s behavior changes markedly, at first taking more training steps to learn, and then entering
into a regime with perpetual oscillations (αexp /γexp & 1), as seen best by the blue curve in Fig. 1.
Numerical experiments.– In simulations, we train a disordered resistor network (64 nodes, 143 edges,
code available [25]) for an allosteric task involving 5 source nodes with voltages sampled fromVS ∼ N (0, 1), and 3 target nodes with desired voltages VT ∼ 0.2N (0, 1) (Fig. 2A).P
During training
using Eq. 3 (η = 10−3 ), we measure the instantaneous mean squared error C ≡ T (VT − vTF )2(normalized at its initial value). In Fig. 2B, error values are shown for different ratios α/γ as afunction of training steps. At slow learning α/γ → 0 we recover the quasi-static case of coupledlearning (white dotted line). The learning rate α can be increased substantially to α ≈ γ with little
3

A

B

C

Figure 3: Training times, i.e. time for normalized error to reach a threshold C̃ = 10−3 in (A)experiments and (B) simulations. A transition from near equilibrium dynamics to strongly out of
equilibrium dynamics occurs at αc ∼ γ. For α < γ, training time improves linearly√ with α. Whenα  γ networks are trained far from equilibrium, with training times improving as α. (C)p Startingat the critical learning rate αc ≈ γ, error exhibits oscillations whose frequency grows as α/γ.

effect on learning dynamics. Further increasing α, the error oscillates at short times, yet the network
still learns successfully. Notably, simulations recover the experimental phenomenology despite
multiple differences in how these systems learn, suggesting that Eq. 2 captures essential aspects ofphysical learning out of equilibrium.
In physical systems, where γ is fixed by the network physics, slow learning rates entail long training
times. In Fig. 2C we set the time scale by γ = 1, finding that increasing the learning rate, even far
beyond the physical rate, significantly reduces training times. In Fig. 2D, we plot the error for trained
networks of size 64, with 10 sources and 3 targets, averaged over 50 tasks. We see that the error ofthese networks after training is largely unaffected for α < γ. At higher α, errors are larger but still
small; networks can be trained effectively for allosteric tasks far beyond equilibrium. This transition
to strong out of equilibrium dynamics is marked by the onset of error oscillations (Fig. 2BC).
Out of equilibrium effects also manifest through a change in scaling of the training time. In Fig. 3AB,
we plot the time taken the networks to reach a normalized error threshold C̃(T ) = 10−3 . For quasistatic training α  γ we see a linear improvement in the training time T ∼ (α/γ)−1 . Transitioning1
to out of equilibrium learning, training becomes faster, but with slower scaling (α/γ)− 2 .
Origin of learning oscillations.– How do networks learn even far from equilibrium? What gives riseto learning oscillations at α & γ? We can gain insight on these questions by taking a second time
derivative of the physical degrees of freedom in Eq. 2:

γ −2 δ ẍF + γ −1 Hδ ẋF +α 2 F
D δx = 0,γ(4)where we have defined the difference between the observed and desired states δxF ≡ xF − X, andthe physical Hessian and cross derivatives H ≡ ∂x2 E F , D ≡ ∂x ∂w E F . These dynamics can bepartitioned to an overdamped term for physical relaxation and an oscillatory term which scales withthe learning rate. As H, D do not depend on α, γ, the oscillatory term becomes important only whenlearning is fast enough. The solutions to this equation become oscillatory when the discriminant
H 2 − 4 αγ D2 becomes negative at αc ∼ γ. Beyond this critical learning rate the oscillation frequencypincreases, scaling as α/γ. This frequency scaling then dictates the training time, which can be
estimated by the first pass of the oscillatory dynamics through zero. These predictions are verified inFig. 3C for 64 networks with 10 sources and 3 targets. The oscillatory learning dynamics converge tosolutions δx → 0 as long as the physical derivatives H, D vary slowly relative to the physical andlearning degrees of freedom x, w, i.e. if the energy landscape is smooth enough.
Eq. 4 also explains the observed power laws in the training time. At low learning rates error
convergence is dominated by the slowest overdamped decay mode of H, scaling as τ ∼ γ/α. Far
from equilibrium however,pwhere α  γ, convergence is dominated by the inverse oscillation
frequency, scaling as τ ∼ γ/α.
43Concluding remarks

Going beyond quasi-static contrastive learning, we observe that physical networks can learn out-ofequilibrium. This idea may be important in biological learning, where plasticity time scales of someneural circuits are similar to that of reaching steady state [18, 20]. In machine learning, equilibrating
recurrent networks constitutes a computational bottleneck [14]. As full equilibration is not required,the computational load of training recurrent networks may be significantly reduced.
Our results show that trained physical networks exhibit overdamped learning dynamics until a criticallearning rate similar to the physical rate αc ∼ γ. Increasing the learning rate to αc can speed up
learning by orders of magnitude compared to quasi-static training. Beyond the critical threshold, we
describe the formation of under-damped oscillations in the learning dynamics, resulting from thelearning parameters varying too quickly and overshooting desired solutions. In complex non-convex
landscapes underlying some learning problems, one would expect training far from equilibrium tofail as the network rapidly jumps between energy basins due to the changing learning parameters. We
show that at least in the linear flow networks we trained for allosteric tasks, in which the learning
landscape is convex, training is possible and successful far from equilibrium.

Broader Impact
Local physically plausible learning rules based on contrastive learning have emerged as possible
candidates for constructing physical “learning machines", able to autonomously adapt to external
influence in order to gain desired functionality. However, such rules typically rely on equilibration of
the physical state before any learning update, so such systems are prone to very slow training. We
break the assumption of quasi-static learning in physical systems, showing that they can be trained
at comparable rates to their physical relaxation, similar to certain biological neural circuits. This
property is crucial for training of slow physical systems, as well as using such contrastive learning
methods as machine learning algorithms for recurrent networks.

Acknowledgments and Disclosure of Funding
This research was supported by the U.S. Department of Energy, Office of Basic Energy Sciences,
Division of Materials Sciences and Engineering award DE-SC0020963 (M.S.), the Army Research
Office award W911NF-21-1-0076 and air force office of scientific research award (FA9550-21-10313) (M.Z.M.), and the National Science Foundation via the UPenn MRSEC/DMR-1720530 (S.D.
and D.J.D.), and DMR-2005749 (A.J.L.).

References[1] Kyungdoc Kim, Seokho Kang, Jiho Yoo, Youngchun Kwon, Youngmin Nam, Dongseon Lee,
Inkoo Kim, Youn-Suk Choi, Yongsik Jung, Sangmo Kim, et al. Deep-learning-based inverse
design model for intelligent discovery of organic molecules. npj Computational Materials,
4(1):1–7, 2018.
[2] Miguel A Bessa, Piotr Glowacki, and Michael Houlder. Bayesian machine learning in metamaterial design: Fragile becomes supercompressible. Advanced Materials, 31(48):1904845,
2019.
[3] Juhwan Noh, Jaehoon Kim, Helge S Stein, Benjamin Sanchez-Lengeling, John M Gregoire,
Alan Aspuru-Guzik, and Yousung Jung. Inverse design of solid-state materials via a continuous
representation. Matter, 1(5):1370–1384, 2019.
[4] Wei Ma, Feng Cheng, Yihao Xu, Qinlong Wen, and Yongmin Liu. Probabilistic representationand inverse design of metamaterials based on a deep generative model with semi-supervised
learning strategy. Advanced Materials, 31(35):1901111, 2019.
[5] Paul Z Hanakata, Ekin D Cubuk, David K Campbell, and Harold S Park. Forward and inverse
design of kirigami via supervised autoencoder. Physical Review Research, 2(4):042006, 2020.
5[6] Nidhi Pashine, Daniel Hexner, Andrea J Liu, and Sidney R Nagel. Directed aging, memory, andnature’s greed. Science Advances, 5(12):eaax4215, 2019.
[7] Daniel Hexner, Nidhi Pashine, Andrea J Liu, and Sidney R Nagel. Effect of directed aging on
nonlinear elasticity and memory formation in a material. Physical Review Research, 2(4):043231,
2020.
[8] Menachem Stern, Chukwunonso Arinze, Leron Perez, Stephanie E. Palmer, and Arvind Murugan. Supervised learning through physical changes in a mechanical system. Proceedings of the
National Academy of Sciences, 117(26):14843–14850, 2020.
[9] Javier R Movellan. Contrastive hebbian learning in the continuous hopfield model. In Connectionist models, pages 10–17. Elsevier, 1991.
[10] Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between
energy-based models and backpropagation. Frontiers in computational neuroscience, 11:24,
2017.
[11] Jack Kendall, Ross Pantone, Kalpana Manickavasagam, Yoshua Bengio, and Benjamin Scellier.
Training end-to-end analog neural networks with equilibrium propagation. arXiv preprint
arXiv:2006.01981, 2020.
[12] Menachem Stern, Daniel Hexner, Jason W Rocks, and Andrea J Liu. Supervised learningin physical networks: From machine learning to learning machines. Physical Review X,
11(2):021045, 2021.
[13] Victor Lopez-Pastor and Florian Marquardt. Self-learning machines based on hamiltonian echo
backpropagation. arXiv preprint arXiv:2103.04992, 2021.
[14] Sergey Bartunov, Adam Santoro, Blake Richards, Luke Marris, Geoffrey E Hinton, and Timothy
Lillicrap. Assessing the scalability of biologically-motivated deep learning algorithms andarchitectures. In Advances in Neural Information Processing Systems, pages 9368–9378, 2018.
[15] Maxence Ernoult, Julie Grollier, Damien Querlioz, Yoshua Bengio, and Benjamin Scellier.
Updates of equilibrium prop match gradients of backprop through time in an rnn with static
input. arXiv preprint arXiv:1905.13633, 2019.
[16] Maxence Ernoult, Julie Grollier, Damien Querlioz, Yoshua Bengio, and Benjamin Scellier.
Equilibrium propagation with continual weight updates. arXiv preprint arXiv:2005.04168,
2020.
[17] Benjamin Scellier. A deep learning theory for neural networks grounded in physics. arXiv
preprint arXiv:2103.09985, 2021.
[18] Robert S Zucker and Wade G Regehr. Short-term synaptic plasticity. Annual review ofphysiology, 64(1):355–405, 2002.
[19] Ryota Kanai and Frans AJ Verstraten. Perceptual manifestations of fast neural plasticity: Motion
priming, rapid motion aftereffect and perceptual sensitization. Vision research, 45(25-26):3109–
3116, 2005.
[20] Yaniv Sagi, Ido Tavor, Shir Hofstetter, Shimrit Tzur-Moryosef, Tamar Blumenfeld-Katzir, andYaniv Assaf. Learning in the fast lane: new insights into neuroplasticity. Neuron, 73(6):1195–
1203, 2012.
[21] World Health Organization et al. WHO guidelines on hepatitis B and C testing. World HealthOrganization, 2017.
[22] Jangu E Banatvala and David WG Brown. Rubella. The Lancet, 363(9415):1127–1137, 2004.
[23] Jason W Rocks. Allosteric Functionality In Mechanical And Flow Networks. PhD thesis,
University of Pennsylvania, 2019.
[24] Sam Dillavou, Menachem Stern, Andrea J Liu, and Douglas J Durian. Demonstration ofdecentralized, physics-driven learning. arXiv preprint arXiv:2108.00275, 2021.
6

[25] Menachem Stern. Source code: Out of equilibrium coupled learning. https://github.com/
NachiStern/OutOfEq-Coupled-Learning, 2021.

Paper checklist
Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions
and scope? Yes
Have you read the ethics review guidelines and ensured that your paper conforms to them? Yes
Did you discuss any potential negative societal impacts of your work? We discuss speeding up
learning in physical networks. We do not see a specific way in which this work can havenegative impact beyond other physically plausible learning algorithms.
Did you describe the limitations of your work? Yes. We study out of equilibrium dynamics inlinear resistor networks learning simple allosteric tasks. We argue on how training may fail in
more complicated settings.
Did you state the full set of assumptions of all theoretical results? Yes
Did you include complete proofs of all theoretical results? No. Our work is not concerned withmathematical proofs, just physical experiments and computer simulations.
Did you include the code, data, and instructions needed to reproduce the main experimental results?
Yes
Did you specify all the training details? Yes
Did you report error bars? Yes
Did you include the amount of compute and the type of resources used? Minor computation
resources used on internal cluster. Experimental resources include one electric resistor systemwith a power consumption significantly lower than a desktop computer.

7