Title: Curl Descent : Non-Gradient Learning Dynamics with Sign-Diverse Plasticity
Venue: NeurIPS 2025
Accepted: yes
Keywords: learning dynamics

non-gradient

learning rules

Hebbian

plasticity

anti-Hebbian

Excitatory-inhibitory

biologically plausible

biological networks

theoretical neuroscience

teacher-student

non-reciprocal
Abstract: Gradient-based algorithms are a cornerstone of artificial neural network training, yet it remains unclear whether biological neural networks use similar gradient-based strategies during learning. Experiments often discover a diversity of synaptic plasticity rules, but whether these amount to an approximation to gradient descent is unclear. Here we investigate a previously overlooked possibility: that learning dynamics may include fundamentally non-gradient "curl"-like components while still being able to effectively optimize a loss function. Curl terms naturally emerge in networks with excitatory-inhibitory connectivity or Hebbian/anti-Hebbian plasticity, resulting in learning dynamics that cannot be framed as gradient descent on any objective. To investigate the impact of these curl terms, we analyze feedforward networks within an analytically tractable student-teacher framework, systematically introducing non-gradient dynamics through rule-flipped neurons. Small curl terms preserve the stability of the original solution manifold, resulting in learning dynamics similar to gradient descent. Beyond a critical value, strong curl terms destabilize the solution manifold. Depending on the network architecture, this loss of stability can lead to chaotic learning dynamics that destroy performance. In other cases, the curl terms can counterintuitively speed up learning compared to gradient descent by allowing the weight dynamics to escape saddles by temporarily ascending the loss. Our results identify specific architectures capable of supporting robust learning via diverse learning rules, providing an important counterpoint to normative theories of gradient-based learning in neural networks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors demonstrate that learning is still possible in neural networks in which the sign of synaptic plasticity for some neurons is flipped. They develop a formal theory that characterizes the curl dynamics induced by such sign flips, and characterize the effects of sign flips on the fixed point structure of learning dynamics. Lastly, the authors show that under some conditions the presence of sign flips can, counterintuitively, be beneficial for optimization.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is very clearly written, and quite a bit of effort is put into building the reader’s intuitions, which was much appreciated. It is also original, as far as I can tell. I think that this paper is very interesting, primarily because of the formal theory that the authors are able to develop to mathematically analyze learning dynamics in sign-flipped networks that learn via gradient-based signals—in my mind, this contribution alone is probably sufficient to warrant acceptance with some revisions, but other parts of the paper could be significantly improved. I think that the value of Section 2 is less clear, and that the authors could significantly improve the paper by clarifying a few points regarding that section (discussed below). I am also generally unconvinced by the results that curl descent can improve performance relative to gradient descent in more complicated networks or on more complicated tasks.\n\nQUESTIONS:\nMajor concerns:\n\n1.\tIn section 2, it is unclear to me whether the curl descent analysis provided is useful, because unlike in the gradient descent framework in section 3, fixed points of the network are not well-characterized. This is relevant, because the Hebbian synaptic plasticity rule selected in Eq. 1 typically produces explosive weight dynamics if M_ij is positive, and vanishing dynamics if M_ij is negative. After all, if the plasticity rule is not stable or results in vanishing synaptic magnitudes, what is the value of this analysis?\n\n2.\tFor Figure 5, the authors have provided a demonstration that flipping the sign of plasticity for one readout weight improves performance significantly in a student teacher formulation. However, this is really not enough to be able to make much of any claim about the effects of curl descent on learning dynamics under other conditions. For instance, what happens as the number of flipped signs increases? What happens if signs are flipped in the hidden layer instead? What happens if the network is trained on an even moderately more complex task (e.g. MNIST), or if more complicated network architectures are used (e.g. convolutional)? Because this claim is counterintuitive (and potentially very cool), more empirical evidence is required to support the claim; while I accept the intuition that curl descent could help networks avoid local minima or slow points, making such a method function adequately at scale may require additional tricks (e.g. taking an annealing-style/learning rate scheduled approach to curl descent).\n\n3.\tWhile the authors seem to assume that sign flips in plasticity exclusively induce non-gradient curl terms in weight dynamics, this is not the only possibility, and alternatives should be discussed. I see four alternative possibilities. First, and least interesting, the functional form of ‘optimal’ synaptic plasticity within a normative framework can change quite radically based on the biophysics of a neuron and the ‘experimental’ conditions under which plasticity is characterized, meaning that cell-type specific plasticity could simply be a due to evolutionary corrections for differences in the biophysics of different neuron subtypes. Second, the sign of synaptic plasticity has been shown to vary considerably based on the position of a synapse relative to the soma in pyramidal neurons (Froemke et al., 2010; Sjostrom & Hausser, 2006); several other theoretical frameworks have argued that this switch in the sign of plasticity is actually fully consistent with approximate gradient descent learning (Richards & Lillicrap, 2019). Third, neuromodulators have been show to induce flips in the sign of synaptic plasticity, and many studies have attributed this effect to a natural consequence of reinforcement learning (Frémaux & Gerstner, 2016). And lastly, cell-type specific variations in plasticity could be due to the optimization of local auxiliary objectives that effectively place constraints on the optimization landscape for other non-local optimization processes (e.g. inhibitory plasticity could simply be serving the purposes of local homeostatic normalization of network activity). Recent studies have shown that network stabilization can emerge as a consequence of a wide diversity of different plasticity updates (e.g. Confavreux et al., 2025).\n\nMinor concerns:\n\n1.\tA recent paper (Shoji et al., 2024) has argued that a broad family of plasticity rules (even those with apparent curl) can be viewed as natural gradient descent if one chooses an appropriate timescale over which to average plasticity updates. While at face value this argument appears inconsistent with the conclusions of this paper, I actually think that they are fully consistent—either the plasticity covered in the current paper violates the constraints required for formal argument in the natural gradient study, or it may indeed be viewed as a form of natural gradient descent if averaged over an appropriate timescale. Regardless, the paper should be discussed because of their similar setup and differing conclusions; it would fit nicely into the ‘Natural gradients’ section.\n\n2.\tWhy can the first term in Eq. 2 be viewed as the negative gradient of an energy function? It is not immediately clear to me why the expression written can be viewed as an energy function (or what assumptions are being made to make it an energy function).\n\n3.\tIs it possible to generalize the mathematical analysis here to perturbations more general than sign flips? It seems to me like this analysis ought to be able to be generalized to a much broader family of diagonal perturbation matrices without much change to the underlying math.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces curl dynamics as an extension to standard gradient descent optimization, drawing inspiration from biological neural networks. Using a toy model, the authors offer clear intuition behind the proposed approach. They then analyze the stability of curl descent within a teacher-student framework using linear transformation networks, supporting their theoretical findings with numerical experiments.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\nThe paper offers a solid theoretical foundation for curl descent, employing analytical techniques based on a toy model and random matrix theory to provide clear rationale and insight.\n\nWeakness:\nWhile the work serves as a strong proof of concept, it falls short by not demonstrating the practical effectiveness of curl descent in real-world deep learning scenarios with more realistic datasets.\n\nQUESTIONS:\nI noticed that the test error heat map (Fig. 4a) appears to differ significantly from the theoretical prediction (Fig. 2, bottom). In contrast, the max error heat map seems to align more closely with the theoretical results based on random matrix theory. I may be misunderstanding something, but the discrepancy between the numerical results and theoretical predictions does not seem to be fully addressed in the paper. Could the authors clarify this point?\n\nAdditionally, what is the origin of the U-shaped pattern observed in the test error heat map (Fig. 4a)? This U-shape appears even more pronounced when using the nonlinear tanh activation function, as shown in Supplementary Material (page 14, Fig. 3a). Is there an underlying theoretical explanation for this behavior?\n\nFor reference, the importance of curl dynamics in neural networks has been previously highlighted—for example, in Yan et al., Nonequilibrium landscape theory of neural networks, PNAS 110(45): E4185–E4194 (https://doi.org/10.1073/pnas.1310692110).\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is a normative study on synaptic plasticity in neural networks. Normative approaches have mostly focused on learning rules that minimize a given cost function (i.e. that follow a gradient). Conversely, the plasticity rules that are not derived from a cost function are typically considered from a purely empirical or numerical point of view. Here, the authors provide an analytical perspective on the latter group of rules. They first prove that “sign diversity” (in the plasticity rule: co-active Hebbian + anti-Hebbian plasticity; or in the neuron types: excitatory and inhibitory neurons) generate learning dynamics that do not follow any gradients.\nThe trick to nevertheless provide a normative treatment of such systems is to use a teacher-student framework in linear feedforward networks with a proportion of synapses doing gradient ascent while others do gradient descent. The learning dynamics can then be broken down into a classic gradient-descent component and additional “curl” terms that affect the learning dynamics depending on parameters such as the network architecture and the proportion of flipped synapses. The authors show that curl terms can change which solution from the gradient-descent dynamics is reached, sometimes speeding up learning.\n\nSTRENGTHS AND WEAKNESSES:\nOverall, I found this paper clear, well-explained and nicely presented. The model chosen is simple, at the limit of being simplistic, but just complex enough to obtain interesting conclusions about the problem at hand. As such I find it a very elegant and important work in the current scientific scene around synaptic plasticity, although it could be attempting to formulate more precise predictions towards the end.\n\nQUESTIONS:\n1/ Each plasticity rule/cell type could be understood to optimize its own gradient/objective. Even though the whole system doesn’t optimize a single objective, could it still be understood as some competition across neurons for different cost functions? This would somewhat nuance your claim challenging the prevailing view of gradients for synaptic plasticity.\n\n2/ Over-focus on one specific instantiation of Hebbian and anti-Hebbian learning In section 2. What about other rules? For instance variations of hebbian/anti hebbian. I don’t think you need to add anything in main, but maybe point to a supplementary paragraph that suggests that the findings are not overly rule-specific. Right now, this makes your results appear to have a narrower scope than they surely have.\n\n3/ Your motivation for adding different plasticity rules in the same network are cell types, which are typically defined by their connectivity patterns to other cell types and their unique cellular properties. Your model is thus making a rather strong assumption of only changing the rule. I understand an analytical treatment of such networks with more complex initial connectivity is out of scope, but maybe you could provide some numerics? Another way of asking the same question is: What if the identity of the flipped synapses was not random: can you show further/more robust benefits of curl terms in such networks? You could take inspiration from/recover classical synapse types connectivities? (local PV/SOM interneurons for instance)\n\n4/ To make this work more impactful and strengthen the claim that curl terms are more than an “approximation error” to gradient descent and deserve proposer attention in the future, I think that figure 5 needs to be expanded. I understand this is out of scope/hard to do analytically, but some additional simulations would be welcome. E.g. recurrent networks/a more “real-world” task than the teacher student framework.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Curl Descent, a theoretical framework for analyzing and leveraging non-gradient learning dynamics induced by biologically plausible, sign diverse plasticity rules. By decomposing learning updates via the Helmholtz decomposition into gradient-conforming and curl components, the authors formalize how heterogeneous synaptic plasticity; specifically the coexistence of Hebbian and anti-Hebbian rules: generates structured, controllable non-gradient dynamics. Using random matrix theory and explicit Jacobian spectral analysis, the paper quantitatively characterizes how the magnitude of these curl terms governs both the stability of learning trajectories and the geometry of the solution manifold. The results reveal that expansive architectures, where hidden layers have greater dimensionality than inputs, can inherently buffer against curl-induced instability, while networks operating near critical curl magnitudes exhibit rich, nonlinear learning dynamics.\nThe paper demonstrates that curl components are not universally detrimental to learning. Through an analytically tractable two-neuron system and larger simulations, the authors show that moderate curl magnitudes can facilitate beneficial dynamics, such as detours through unstable regions of parameter space that ultimately accelerate convergence or improve escape from saddle points. This phenomenon is particularly evident in non-linear networks, where introducing controlled curl terms reduces the number of training epochs required to reach low error levels, outperforming strict gradient descent under certain conditions.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\nThis work presents a well constructed and theoreticaly rigorous framework for understanding how biologically motivated, sign diverse plasticity rules induce curl components within learning dynamics. Through a principled application of Helmholtz decomposition, the authors distinguish between gradient conforming and non-gradient components of weight updates, providing a clear mathematical basis for analyzing non-gradient learning dynamics. The use of random matrix theory to characterize the stability properties of these dynamics, particularly through explicit derivations of Jacobian spectra and phase boundaries, reflects a commendable level of analytical depth. These results give concrete, testable predictions about how curl magnitude, network architecture, and plasticity sign diversity interact to govern learning behavior.\nBy linking curl dynamics to well-established biological principle; such as Dale’s law and empirical findings on heterogeneous long-term potentiation and depression—the authors successfully ground their mathematical results in experimentally observable phenomena. The inclusion of toy models, such as the analytically tractable two-neuron system, effectively conveys key intuitions regarding how curl components can introduce detours in parameter space that initially destabilize but ultimately accelerate convergence. These insights challenge prevailing assumptions in machine learning that gradient-following updates are strictly necessary for efficient learning and open promising avenues for exploring biologically inspired, non-gradient learning rules in artificial systems.\n\nWeakness:\nWhile the theoretical framework is compelling, certain technical claims, notably those regarding the uniqueness of the gradient term within the Helmholtz decomposition and the determinant-based stability conditions for larger networks, are only partially elaborated or deferred to supplementary material. Although the supplemental derivations appear sound upon spot-checking, presenting these results more completely within the main text, or providing an accessible computational (jupyter) notebook, would greatly improve transparency and facilitate verification by others.\nWhile the simulation results qualitatively support the theoretical predictions, the empirical scope remains narrow, focusing primarily on small-scale synthetic tasks without extending to real-world benchmarks or diverse architectural settings. The absence of classification tasks, recurrent models, or larger datasets makes it difficult to assess how robust or generalizable the observed benefits of curl dynamics truly are.Lack of an accompanying, open source code release and incomplete reporting of experimental details;such as the implementatino details. Addressing these aspects would significantly improve the paper’s reproducibility and ensure that its insights are more easily transferable to practical machine learning and neuroscience research contexts.\n\nQUESTIONS:\na. How does the interplay between the network’s expansion ratio (c = M/N) and the magnitude of the curl component quantitatively affect the phase boundaries for stable versus unstable learning dynamics in the random matrix regime, and can this relationship be generalized beyond the specific student-teacher model analyzed?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}