Learning From the Past with Cascading Eligibility Traces
Authors: Tokiniaina Raharison Ralambomihanta
Ivan Anokhin
Roman Pogodin
Samira Ebrahimi Kahou
Jonathan Cornford
Blake Aaron Richards
Venue: ICLR 2026 Poster (2026)
Keywords: biological credit assignment
eligibility traces
synaptic plasticity
computational neuroscience
Primary area: applications to neuroscience & cognitive science
TL;DR: A cascade of synaptic eligibility traces can account for delays in biological credit assignment

Abstract: Animals often receive information about errors and rewards after significant delays. In some cases these delays are fixed aspects of neural processing or sensory feedback, for example, there is typically a delay of tens to hundreds of milliseconds between motor actions and visual feedback. The standard approach to handling delays in models of synaptic plasticity is to use eligibility traces. However, standard eligibility traces that decay exponentially mix together any events that happen during the delay, presenting a problem for any credit assignment signal that occurs with a significant delay. Here, we show that eligibility traces formed by a state-space model, inspired by a cascade of biochemical reactions, can provide a temporally precise memory for handling credit assignment at arbitrary delays. We demonstrate that these cascading eligibility traces (CETs) work for credit assignment at behavioral time-scales, ranging from seconds to minutes. As well, we can use CETs to handle extremely slow retrograde signals, as have been found in retrograde axonal signaling. These results demonstrate that CETs can provide an excellent basis for modeling synaptic plasticity.

Published as a conference paper at ICLR 2026
LEARNING FROM THE PAST WITH CASCADING ELIGI- BILITY TRACES
Tokiniaina Raharison Ralambomihanta" 1.2 Ivan Anokhin 1, Roman Pogodin " 1,44, Samira Ebrahimi Kahout Jonathan Cornford 188 Blake A. Richards 1.4..7799.10
Mila Quebec Artificial Intelligence Institute DDpartmentt of Bioengineering. McGill University
' Universitté de Montréal
DDepartment of Neurology & Neurosurgery, McGill University
Montreal Ncurological Institute, McGill University
University of Calgary
CIFAR AI Chair
Schooo of Computer Science, University of Leeds
"chooo of Computer Science, McGill University
"CIFAR Learning in Machines and Brains Program
Equal contribution
Co-senior authors
(toky.raharison-raa ambomi
i hanta, ivan. anakain roman. pogodin, ebrahims, cornforj, Mlake.richare @mm quebec
ABSTRACT
Animals often receive information about errors and rewards after significant delays. In some cases these delays are fixed aspects of neural processing or sensory feed- back, for example, there is typically a delay of tens to hundreds of milliseconds between motor actions and visual feedback. The standard approach to handling delays in models of synaptic plasticity is to use eligibility traces. However, stan- dard eligibility traces that decay exponentially mix together any events that happen during the delay, presenting problem for any credit assignment signal that oc- curs with a significant delay. Here, we show that eligibility traces formed by a state-space model, inspired by a cascade of biochemical reactions, can provide a temporally precise memory for handling credit assignment at arbitrary delays. We demonstrate that these cascading eligibility traces (CETs) work for credit signment at behavioral time-scales, ranging from seconds to minutes. As well, we can use CETs to handle extremely slow retrograde signals, as have been found retrograde axonal signaling. These results demonstrate that CETs can provide an excellent basis for modeling synaptic plasticity.
INTRODUCTION
Learning requires a mechanism for assigning credit for errors and successes to past neural activity (Gerstner al., 2018). In biological learning, the signals necessary for credit assignment in neural circuits arrive after temporal delay, for instance via latency in sensory feedback following motor actions (Omrani et al., 2016; Scott, 2016). The source of delays can be generally categorized as either external or internal. External delays occur due to latency in environmental reward signals, for example receiving food or other external reinforcers, and as such can variable. In contrast, internal delays arise from delays in the neural processing mediating the credit assignment calculation itself. Depending on the neural mechanisms involved, they are approximately fixed and can range from hundreds of milliseconds to several minutes (Fitzsimonds & Poo, 1998). Synaptic plasticity rules that model the credit assignment calculation, therefore, need to account for such fixed internal delays.
The traditional solution to internal delays in synaptic plasticity rules is to exponentially decaying synaptic eligibility traces (ETs) (Gerstner et al., 2018; Shouval & Kirkwood, 2025), which are decaying records of synaptic activity. However, some experimentally observed synaptic plasticity

---

Published as a conference paper at ICLR 2026
rules are tuned to fixed non-zero delays (Suvrathan et al., 2016; Shindou et al., 2019), and therefore do not fit with an exponential decay of credit eligibility. When the neural activity and the corresponding credit assignment signal are separated by few intervening events, such delays will have minimal impact on learning. However, in general, ongoing neural activity will override past activity relevant to the current reward or error signal. Therefore, traditional eligibility traces are not well-suited for the temporal scale of credit assignment delays in biological learning when the delay is non-zero and fixed.
To solve this problem we present generalization of traditional eligibility traces. Inspired by synaptic biochemical cascades (Zhang et al., 2021), we model eligibility traces as state-space models that incorporate a cascade of synaptic memory traces. These cascading eligibility traces (CETs) provide a delayed and concentrated temporal window of maximal credit assignment. This model fits the experimentally observed unimodal delays, and is also consistent with biological mechanisms of synaptic plasticity (Friedrich et al., 2011; Fusi et al., 2005; Zhang et al., 2021).
We present series of results that demonstrate the utility of CETs for credit assignment with biologically realistic delays. Specifically, we show that we can engage in both supervised and reinforcement learning in multi-layer networks under two distinct delay scenarios. First, we examine learning situations where the delays are consistent across layers of the network, as would be the case for various models of biological credit assignment in which learning signal is broadcast across layers (e.g. direct feedback alignment (Nøkland, 2016) and other local learning rules (Frémaux & Gerstner, 2016; Ororbia, 2023)). Second, we show that CETs work when delays stacked through the network, such that late layers receive credit signals sooner than early layers. We provide evidence that this approach works in both scenarios under variety of biologically relevant delays, ranging from hundreds of milliseconds to minutes. Notably, the fact that CETs work when delays are stacked across layers and last for minutes shows that CETs could be applicable to credit assignment signals carried by retrograde axonal signals or neuropeptides, making this approach relevant for a number of biologically plausible credit assignment models (Liu et al., 2022; Fan & Mysore, 2024).
Altogether, our results indicate that CETs are a promising approach for handling delayed credit assignment signals in models of biological learning. More broadly, this provides a general framework for reasoning about synaptic memory in real neural networks.
RELATED WORK
Our work is related to and builds upon several strands of research on synaptic plasticity and biological credit assignment.
Eligibility traces (ETs) have long been a dominant framework for modeling how synaptic plasticity mechanisms may bridge temporal gaps between neural activity and feedback (Gerstner et al., 2018; Shouval & Kirkwood, 2025). Related theoretical extensions include ETs to approximate backpropa- gation through time (BPTT; Bellec et al. (2020)). Experimental evidence for ETs is well-established, with multiple studies reporting how synaptic changes can be induced by reward signals arriving seconds to minutes after neural activity (Brzosko et al., 2015; He et al., 2015; Bittner al., 2017; Suvrathan, 2019).
In some experimental results, the timing of maximum synaptic change tuned to specific delays (c.g. 120ms in the cerebellum (Suvrathan et al., 2016) and 2s in the striatum (Shindou et al., 2019)). This is in contrast to traditional ETs, and indicates plasticity rule that encodes temporal structure in addition to the presence of past activity. One approach to model these findings is to combine two independent ET's for potentiation and depression to produce composite ET that peaks a required time delay (He et al., 2015; Huertas et al., 2016). This approach is conceptually similar to our CET model with states. However, as we discuss below, this approach restricted to producing ETs with broad integration window, making it suitable for short delays only. As we show in our work, CETs with a larger number of states overcome this issue.
An example of extreme delays in plasticity signals is retrograde axonal signaling: i.e. "back- ward"' propagation of chemical signal through the axon and synapses (Maday et al., 2014; Alger, 2002; Fitzsimonds & Poo. 1998). These signals play a role in artivity-depended synaptic plasticity at the level of individual synapses (Regehr et al., 2009), and have been suggested to coordinate plasticity across several neurons (Fitzsimonds et al., 1997; Hui-zhong al., 2000; Du & Poo, 2004).
2

---

Published as a conference paper at ICLR 2026
However, retrograde signals have generally been discarded component of credit assignment (e.g- Lillicrap et al. (2016)) because retrograde axonal signaling is extremely slow (on average 1.3 p.31pm/s; Cui et al. (2007)). meaning that any error signal delivered via retrograde signaling would arrive minutes after the relevant neural activity. Nevertheless, there's been recent interest in this approach (Fan & Mysore, 2024). Here, we study to what extent delays the order of retrograde process timescales could be compensated with CETs.
In parallel to neuroscience, the deep learning community has also worked on the the problem of delayed feedback from the perspective of decoupling the forward and backward passes for efficiency (Jaderberg al.. 2017; Malinowski et al., 2020). More generally, there an extensive body of related work modeling how neural circuits may estimate and communicate credit in a biological plausible manner (Lillicrap et al., 2020). This includes credit computations with dendrites and bursts (Greedy et al., 2022; Payeur et al., 2021; Sacramento et al., 2017), and neuropeptides (Liu et al., 2022).
3 CASCADING ELIGIBILITY TRACES (CETs)
Synaptic plasticity for learning always requires some memory for presynaptic activity in the network. Consider a network containing neuron with activity 2 f(x, w), where w are the synaptic weights and are the presynaptic inputs to the neuron time t. To minimize a loss L, synaptic changes in w can follow the negative gradient over the loss L,
e [x w) i w
aT(a)) zt
f' ((xTw)x x = -6, (x, w)xx.. x
This gradient-bused formulation of plasticity Eq. (1) covers various forms of biological learning. For example, Hebbian learning can be recovered by using the loss L(z,) = = and layer-wise learning rules can be defined similarly. And, of course, backpropagation follows the same chain rule logic for the loss defined over several layers of neurons.
Importantly, these updates require that the credit assignment signal 6 is paired with the appropriate presynaptic inputs, X,. Hence, in the presence of any delays in computation of the credit assignment signal learning system would face temporal mismatch problem: if it takes T seconds to calculate and propagate the credit assignment, then at time the error signal & received by a neuron would have to be matched to older presynaptic activity memory x,-T (see Fig. IA, top row). If learning is done in phases this need not be problematic. But, if learning happens online, as is likely the case in real brains, neural activity would correspond to the current time point, x; only, so the previous presynaptic activity information, X, would have to be somehow stored by the synapses.
Eligibilitt traces (ET) represent the classic solution this problem: they add memory component to the synapse that keeps track of recent activation for a single presynaptic neuron. Here, we will pick one weight w' and the corresponding input =t; and diseuss ET M such that changes in w are proportional to -Ō, A.T (as in Eq. (1); dropping the index from h for convenience).
Denoting the Hebbian-like term h f'x w) xt (in the sense of it being a product of pre- and postsynaptic factors, o and (xx w) correspondingly),
/ - ha ds,
ET h
where 7 > 0 a discount factor. The main advantage of ETs is that they 're easy to implement:
A
ET
= -y A + he-
ET's effectively convolve the presynaptic activity h, with an exponential kernel g(t), i.e.
h = (g = h)()) = S. g(t 3) hg
and use this means of weighting past activity for combining it with credit assignment signals. One of the appeals of ETs as a solution to delayed credit assignment signals is that they do not require extensive memory, and are therefore biologically plausible approach for learning.
3

---

Published as a conference paper at ICLR 2026
A
B 1.00
C 1.00
cascade
ET
activity (h)
signal (6)
state k
CET(2)
0.75-
trace value
0.75
CET(6)
ET
0.50
0.50
CET(10)
0.25
0.25
CET
0.00+
0.00
t-T
time t+2T
L+T
1-T
T+T
t+2T
time
Figure 1: A. Learning with eligibility traces: neural activity h followed by a delayed error signal The standard eligibility trace (ET) is an exponentially decaying trace of / that can be matched the error signal at time I. The eascading ET (CET) reflects like regular ET, but peaks at the required time t. B. Time evolution each state of a 6-state CET with a delay and unit input Comparison of standard ET and CETs with 2/6/10 10 states representing delay T for a unit input at T.
T. C.
Notably, the classic form of ET assigns the maximal trace values the most recent time- tome-points- in Eq. (3). That is appropriate for situations in which there are few intervening presynaptic events between times when the credit assignment signal arrives, and T. when the presynaptic activity occurred (as in Fig. 1A). But, if he changes frequently relative to the delay in the credit assignment signal then gradients calculated with classic ETs, i.e. 6, h can be a poor approximation of the true gradient, & h4-r-
Ideally, when we consider the long delays faced by biological learning agents we would have ETs satisfying two conditions. First, the maximal value of the synaptic trace should occur delay of 8 1 T, rather than / Eq. (3). Second, it is better if we can use a more precise weighting of the past, i.e. if the ET values g(l are as small as possible for anything other than -
Classic ET's do not provide these characteristics. Combined L.TP LTD eligibility traces (He et al., 2015; Huertas al., 2016) satisfy the first condition, but not the second. They effectively take a difference of two standard ETs in Eq. (2) to convolve past activity with g(t) = exp((7r-) exp(->ut) (as in Eq. (3)). While this peaks with a delay T = (log YD log //p/(/D 7p), keeps a large weight for more recent points. Additionally, we show Section A.3 that the limiting sequence of 7D; 7D for minimizing the second moment around T constrained maximum at T converges to our proposed method.
As flexible solution to both of these prob- lems, we propose using ETs constructed from simple state-space model:
1.00-
ET CET(2)
trace value
0.75
CET(6)
0.50
CET(10)
ht -a hi! + he, ...
0.25
(shifted by T)
it a h h k
(4)
0.00
0
2
6
time
içET aa hi
CET
Figure 2: Representation of the input signal h (gray) with a = delay (dashed orange line) using a standard eligibility trace (ET) and cascading ETs (CETs) of different orders. Only CETs of higher order reflect the time evolution of the input (i.e. match the orange line).
Eq. (4) can be used as model of a cascade of biochemical reactions. This could involve, for example, a cascade of phosphorylation pro- cesses or enzymatic reactions (Zhang et al., 2021). Implementing CET(n) online requires n states per synapse each with a constant update cost so the computational costs scale linearly in time and memory.
This form of CET gives us the following formulation (see Section A for a derivation):
(n 1)! I ((--s) -t (t--s) h2 ds,
CET h
(5)
4

---

Published as a conference paper at ICLR 2026
which for 1=1 convolves the presynaptic activity with kernel g(t) t---le- e- that peaks at t= T (Fig. IC). For classical ETs (which correspond to a single-state model of n = 1) we either set the decay to be + (for supervised learning) or we conduct grid search this hyperparameter (for reinforcement learning).
The dynamical system in Eq. (4) is defined by two parameters: the number of states and the decay term a. Increasing n while keeping the peak-time fixed leads to a narrower kernel (see Fig. 1C for a visualization of different kernels g()), but having even two states (instead of one for standard ETs) can account for delayed signals. However, to accurately represent delayed signals, more states are typically needed (see Fig. 2 for time domain response and Section A.2 for Laplace domain analysis).
4 EXPERIMENTS
We illustrate the influence of CETs on learning with delays in two scenarios': (1) learning with delays on behaviorally relevant timescales (e.g. on the order of seconds) in Section 4.1; (2) credit propagation through very slow chemical signals (e.g. retrograde axonal signaling (Fitzsimonds et al., 1997)) Section 4.2. In our simulations we assume that each input lasts for 200 ms, which is roughly one saccade or one theta cycle in the brain (Young & Stark, 1963). Thus, a single time-step in the simulation treated as a 200 ms, so a delay of T = 1 would mean that the signal arrives time-steps after the input is initially presented to the network. Put another way, with a simulated delay of T
there are 4 image presentations that occur after the initial image presentation and
before the ō signal for that image arrives.
In (I), we assume that the error signal ō is propagated to all layers simultaneously since credit signals propagated via action potentials could be transmitted to the entire network in parallel. As well, we calculate the error signal explicitly, but we note that this calculation could easily be substituted with any of the available mechanisms for biologically plausible error calculation (e.g. 3-factor Hebbian learning rules Frémaux & Gerstner (2016)).
In (2), we consider a much longer delay T of 2 minutes, which corresponds to roughly the amount of time it takes for chemical signals to travel backwards along the axon. At the speed of 1.31 jum/s (Cui 160 pm, corresponding to the typical < 200 μam distance in the cortex (Song et al., 2005; Cui et al., 2007). As well, in-line with propagation of retrograde signal, we assume that the delays stack up over layers. Thus, the last layer has no delay, the penultimate layer has delay of T = 2 minutes, the next layer has delay of T = 4 minutes, and so on. Thus, each preceding layer's delay is increased by T.
al., 2007), this covers roughly-
We use two types of tasks: supervised image recognition on MNIST (LeCun, 1998) and CIFAR- (Krizhevsky et al., 2014), and reinforcement learning on state-based environments (namely CartPole and LunarLander as well as on more complex visual environment (namely MinAttar/Spacelnaader (Young & Tian, 2019), which use raw pixel observations as input). We use a 3-layer MLP (input + 512 + 512 + 10) for MNIST and a small CNN with 3 convolutional layers (input - 32 - 64 + 128) and two linear layers (512 - 10) for CIFAR-10. 10. For RL, we use a 3-layer MLP with hidden dimension of 256 that we train with the Actor-Critic method, and report results over seeds. To simplify training in the delayed setup, only the Actor is trained with a delayed error signal, while the Critic is updated via standard backpropagation. The Actor is trained using an online implementation of the A-return via RL. eligibility traces (Sutton & Barto, 2018). Other experimental details (hyperparameters, compute resources) can be found in Section The PyTorch (Paszke et al., 2019) implementation and experiments are provided in the Supplementary Material.
4.1 LEARNING WITH DELAYS ON BEHAVIORALLY RELEVANT TIMESCALES
On MNIST, we observed that classical ETs (corresponding to CET with one state) maintain strong performance up to delays of two seconds, i.e. up 10 image presentations before arrives (Fig. left). This shows that classical ETs can remain effective for relatively simple tasks and short delays. However, their performance breaks down at longer delays of 2 4 s. At these longer delays we can see that increasing the number of states in the CETs improves performance, and can keep the accuracy level high at up to 10x s delay (50 image presentations). Past this point, we found that only
"oode is available bctts://giie com avecpleeelleeeeee Mezipleziin T/CKT
5

---

Published as a conference paper at ICLR 2026
MNIST test accuracy
CIFAR-10 test accuracy
0.98
0.98
0.98
0.98
0.95
0.89
0.80
0.71
0.67
0.62
0.58
0.47
CET order
0.80
0.75
0.98
0.99
0.96
0.86
0.38
0.95
0.81
0.81
0.80
0.74
0.68
0.57
10
0.98
0.99
0.98
0.81
0.82
0.81
0.77
0.68
0.60
backprog: 0:99
backprop: 0.81
0.2
0.6
2
10
0.2
0.6
4
10
delay (s)
delay (s)
Figure 3: Accuracy for MNIST and CIFAR-10 datasets across varying numbers of CET states and delays on behaviorally relevant timescales, A single state (top row) corresponds to standard ETs.
a perfect eligibility trace (i.e. an infinite number of states corresponding to a Dirac delta memory) would preserve performance.
The results with CIFAR-10 were even more pronounced (Fig. 3, right). Classical ET performance deteriorates at any delay tested and rapidly decreases, showing that more complex visual tasks are less robust to imprecise time resolution. A key observation is the gradient in performance, with accuracy generally improving with the number of states in the CETs and decreasing with delays. This trend reflects how the CET impulse response becomes increasingly concentrated around the target delay as number of states increases, which provides finer temporal resolution. The same trends also holds on more challenging dataset: see Section F in the Appendix for TinylmageNet performance.
CartPole-v1 mean episodic return
LunarLander-v2 mean episodic return
Spacelnvaders-v0 mean episodic return
500
159
~V
494
488
486
480
365
244
264
256
268
173
189
148
CV-
128
100
86
CET order
485
482
437
492
486
406
69
265
267
258
211
182
184
6
149
142
101
66
190
484
166
494
480
168
258
266
236
236
199
192
0
140
130
102
10
495
500
433
483
268
267
154
108
backkpoo:: 493
backprop: 264
backprop: 266
4
8
32
64
128
2
8
32
64
128
1.2
8
delay (s)
delay (s)
delay (s)
Figure 4: Mean episodic return for different RL environments across varying numbers of CET states and delays on behaviorally relevant timescales. A single state (top row) corresponds standard ETs.
We observe the same trend for the RL tasks (see Fig. 4): shorter delays and a higher number of CET states result in better performance. Note that CartPole and LunarLander are simple RL tasks and re- main solvable even with long delays and fewer states in the CETs. In contrast, MinAtaa/Spaacelnvderr (Fig. 4, right) is more complex, maaagebbaed environment where performance begins to degrade more quickly when delay introduced. In fact, CET performance mance of perfect memory, even at the shortest delay we tested. We therefore hypothesize that precise credit assignment, without mixing nearby time points, especially important for complex, non-i.i.d. tasks. Altogether,
best only half the perfor-
results demonstrate that at behaviorally relevant time delays higher-order CETs can greatly enhance performance beyond that achieved by classical ETs, particularly at long delays. However, they cannot fully compensate for delays in highly complex, non-i.i.d. tasks.
To better understand the reasons for the performance we examined how well the weight updates were aligned to the true gradient. We measured the cosine similarity, a b/[[|a]] [[b]], between the vector of weight updates given by our CETs, a. and the true gradient as calculated by backpropagation, b. In Fig. 5, we plot cosine similarity for all CET models for delays of 1, 2, and 4 s on CIFAR-10

---

Published as a conference paper at ICLR 2026
delay= 1.0s
delay 2.0s
delay 4.0s
mean cosine
similarit
0.7
ET CET(2) CET(6)
0.6
0.5
CET(10)
0.4
20
40
0
20
40
0
20
40
epochs
epochs
epochs
Figure 5: Average cosine similarity over all layers between true gradients and gradients computed with either ETs or CET's for the CIFAR-10 10 dataset.
(as performance differences were noticeable across these delays in Fig. 3). We observed that at all times during training, and at shorter delays, an increase in the number of states in the CETs lead to better alignment with the true gradient (Fig. 5, left and center). However, the delay increases, the alignment drops even for higher-order CET's (e.g. with 10 states; Fig. 5, right). When we broke this down by layer, we observed similar patterns (Section D).
Finally, we studied if CETs can handle variable and unknown delays. First, we compared the performance of CETs and ETs in situations where reward delay followed uni-modal distributions with differing variance. We found that CETs always outperformed ETs (Section L in the Appendix), though the benefit decreased as the variance increased and the delay distribution became closer to uniform. This is expected since the kernel shape should me matched to the delay distribution. Indeed, the standard ET can approximate the uniform delay well enough so the advantage of CETs is reduced. Second, for unknown delays, found that (and hence the position of the CET's peak) can be learned using weight perturbation, outperforming ETs (see Section L.4 in the Appendix).
4.2 COMPUTATION WITH EXTREMELY LONG DELAYS FOR RETROGRADE AXONAL SIGNALING
We next investigated the possibility of using CETs to model situations with very long, and accumulat- ing, delays. Here, the goal was to consider delays introduced by chemical signals (e.g. retrograde axonal signaling) which could in principle be used for credit assignment (Fan & Mysore, 2024), but would take minutes to propagate from synapses back to cell bodies. We assume that we do not have solve the weight transport problem of backpropagation (Grossberg, 1987), since retrograde signals could easily have access to synaptic weight values (Fan & Mysore, 2024). (See Discussion.) However, the approach with CETs here could also easily be used in conjunction with other solutions to weight transport, including feedback alignment (Lillicrap et al., 2016) or feedback learning mechanisms (Akrout et al., 2019). Additionally, we assume that the calculation of the signals has access to the post-synaptic activation derivative f'(x w) at the appropriate delay, which implies another memory mechanism at the soma, rather than the synapse. This could be modeled with CETs as well, but we leave that for future work.
t+3T
t+2T
t+T
An additional consideration that we took into account here is that if credit signals were propagating backwards via retroaxonal biochemical transmission, then error signals would take progres- sively more time as they travel across a number of synaptic steps (i.e. network depth). Therefore, in feedforward network if we assume that a single layer takes T time propagate the error signal backwards, then a layer m synaptic steps back will receive the error signal at time t (m 1yT (Fig. 6).
+iti backward pass
last layer
Finally, to handle very long delays on visual tasks (MNIST, 10F,R-10), we make an additional assumption: the CET' are modulated by an additional "salience signal" that zeros out the input to the CETs unless the loss is very large. This mechanism reduces the number of inputs being stored in CETs, in line with work on reducing the energetic costs of plasticity (van Rossum & Pache, 2024). This assumption is also consistent with theories of predictive processing and surprise-drin driven learning, in which events with high prediction error drive plasticity (Friston, 2005;
Figure 6: Backprop using retro- grade axonal signaling results in delay accumulation: in the last layer, the forward and backward signal
computed simultane- ously at time I. Each consecutive layer of the backwards pass takes T more seconds (time taken by retrograde signaling).
7

---

Published as a conference paper at ICLR 2026
Itti & Baldi, 2009; Dabney al., 2020). We use 1.25 % of points with the largest losses in batch (keeping their positions in the batch, such that position index encodes time; see Section B.2 for details). To handle long retrograde signal delays in RL without sparsification, we simplify the setting by assuming each time step lasts 300 ms instead of 200 ms as in other our experiments. This corresponds to delay of 400 frames for the second layer and 800 frames for the first layer.
When training networks on visual tasks with large stacking delays across layers, we observed that the performance increased with increasing CET order (Fig. 7). Moreover, networks with different CET orders trained at markedly different rates, with higher orders learning faster (Fig. 7) This was more pronounced for the deeper convolution network trained on CIFAR- 10 (Fig. 7B) than the shallow MLP trained on MNIST (Fig. 7A). Given that the last layer was trained without delay in these experiments, these results must be due to the impact of delays on learning in the intermediate representations.
To understand the performance differences for different orders of CET we analyzed gradient alignment during training. Because networks learned at different speeds, which affects the dynamics of gradient alignment (Section D), we used test accuracy as the independent variable, rather than training iteration. On both MNIST and CIFAR-I0 we observed that gradient alignment degrades for the earlier layers, as is expected for the increasing delays (Fig. 7C,D). Across all layers, increasing CET order was associated with an increase in gradient alignment, in line with task performance. However, higher orders were unable to fully recover alignment with the gradient.
MNIST
CIFAR-10
fc1
fc2
1.00
0.85
accuracy
eimilerit
0.85
0.98
0.75
0.80
0.65
0.90
0.58
1500 0.885
2
0.45
0.65
CET 6
0.35
backprop
0.55
0.80
0.25
500
1000
1500
0.86
0.88 0.90 0.92 0.94 test accuracy conv3
ols olss 0.90 0.92 0.94 test accuracy fc1
iter
(x1000)
iter (x1000)
conv1
conv2
cosine similarity
0.6
0.5
0.4
0.3
0.2
0.1
0.2
0.3
0.4
0.5
0.6
0.2
0.3
0.4 0.5 0.6 0.2
0.3
0.4
0.5
0.6
0.2
0.3
0.4
0.5
0.6
test accuracy
Figure 7: A. Test accuracy on MNIST as function of number of CET states for the retrograde experiments. B. Same as A, but on CIFAR-10. 10. C. Cosine similarity between the true gradient and the ET/CET approximation across different test accuracies (see A). Each plot shows an individual layer of an MLP during training on MNIST. D. Same as C, but each plot shows an individual layer of CNN during training on CIFAR-10.
For the RL tasks, we observed similar trends. Increasing the number of CET states leads to improved performance on both CartPole and LunarLander (Fig. 8A-B). As before, the increase in the number of CET states also led to increased alignment with the true gradient (Fig. 8C-D), although mostly in the second layer, which helps to explain the improved performance.
Altogether, our results demonstrate that when delays in credit signals are very long (on the order of minutes), and stacked (summing for each synaptic step), CETs can be used to store memory for previous activity in order to accurately estimate gradients and learn. As such, CETs would, in principle, permit credit assignment in situations where errors are propagated backwards via very slow chemical retrograde signals (Fitzsimonds et al., 1997). However, there is a depth limit beyond which the delay would be too large to accurately approximate the gradient.
5 DISCUSSION
For organisms to learn, their brains must have mechanisms for handling delays between learning signals and past neural activity. Here we presented cascading eligibility traces (CETs), generalization
8

---

Published as a conference paper at ICLR 2026
A
CartPole-v1
B
LunarLander-v2
ET
400
200
return
retur
CET(2)
CET(5)
200
0
CET(10)
backprop
0
-200
2e6
4e6
2e6
4e6
samples
samples
cosine similarity 6
1c1
CartPole-v1
fc2
fc1 LunarLander-v2
fc2
cosine similarity
0.8
0.8-
0.6
0.6
0.4
200
400
200 40b return
-200
200
-200
200
return
return
return
Figure 8: A. Episodic return on CartPole-vl during training in the retrograde experiments. Solid lines: mean (3 seeds); shaded area: min/max values; dashed line: mean final backprop performance. B. Same as A but for L.ndera. C. Mean cosine similarity (over cosine similarity values assigned binned return values) between the true gradient and the ET/CET approximation w.r.t. return values in A.. D. Same as C, but for LunarLander-v2.
of classical eligibility traces and an abstract model of interacting biochemical processes within cells, as a candidate mechanism for bridging such delays. We showed that CETs enable learning over long delays on standard image datasets and RL settings, and found that by increasing the number of states in the eligibility trace cascade (with state being equivalent to classical ETs) learning performance be maintained with delays the order of seconds. We explored how CETs contribute to the ongoing discussion around biologically plausible implementations of backpropagation (Fan & Mysore, 2024; Liu al., 2022). Here we tested the hypothesis that synaptic CETs and learning from salient examples enable slow cytoskeletal retroaxonal signals to carry gradient information recursively over layers - an idea popular over two decades ago but since discarded (Harris, 2008). Again, we found CETs with a larger number of states improved performance. Though accumulating delays with network depth was still problematic for learning. CETs demonstrate that learning over timescales relevant for slow chemical signaling is feasible. Our experiments with CETs validate retroaxonal signals a potential solution to the credit assignment problem.
Our work was limited to experiments with feedforward architectures. While extending it to more biologically relevant scenarios, such as RNNs or spiking networks, is beyond the scope of this work, preliminary experiments with recurrent reservoir networks (Appendix E) and leaky intergrate-and-firg neurons (Appendix J) show promising results.
Classical ETs are exponentially decaying "memories" of synaptic activation that thought to be implemented by activation of biochemical processes, such as CaMKII activation or other protein kinases (such PKA, PKC, ERK, MAPK), which are typically triggered by the activation of G- protein coupled receptors (Gerstner et al., 2018). While the complexity of such interacting pathways has been recognized, there has been very little work exploring interactions between such biochemical processes for learning (though see (Friedrich et al., 2011: I; Huertas et al., 2016)). In this context, we are building off work exploring how complex interactions between kinase cascades mediates plasticity (Zhang et al., 2021). More generally, CETs provide normative explanation for the complexity of cascade interactions the context of learning the improved performance with higher order CETs could explain why cells use biochemical cascades rather than a single biochemical signal.
One of the well-known biological implausibilities of backpropagation that it requires that weights in the forward pass be reused in the backward pass. In the context of biology. this algorithmic requirement known as the weight transport problem (Grossberg, 1987). Retroaxonal signals provide a potential solution to this problem because they pass back through the very same synapses used in the forward pass. As such, they could, in principle, carry information about the synaptic weights, thereby solving the weight transport problem (Fan & Mysore, 2024). However, the challenge with retroaxonal signals is that they are very slow, taking minutes to pass from the synapse to the cell body (Fitzsimonds & Poo, 1998). As we showed here, CETs provide potential mechanism for

---

Published as a conference paper at ICLR 2026
making learning at such delays feasible. Therefore, they open up the possibility of using retroaxonal signals for credit assignment. However, our results also showed that you can only stack such long delays over few synaptic steps before learning deteriorates significantly, which would suggest that if retroaxonal signals are used for learning in the brain they would only be used for learning at relatively shallow "depths". Indeed this is consistent with experimental findings: for example Hui-zhong et al. (2000); Fitzsimonds et al. (1997) only found retroaxonal potentiation and depression over one "layer" in cultured neurons (i.e. one synaptic step). Although was originally suggested that the lack of further propagation may be due to the size of the plasticity change (Fitzsimonds et al., 1997), our results provide evidence that recursive propagation delays are problematic, even with CETs, and would require additional mechanisms such as direct reward signaling (Nøkland, 2016).
For the retrograde experiments, we also relied on salience signal to gate which presynaptic events were stored in the CET's. Conceptually, this mechanism reflects broader class of theories in neuroscience and machine learning in which a subset of events-typically those associated with large prediction ertors-drive plasticity. Three-factor learning rules, for instance, can encode surprise, effectively shutting down learning when no unexpected outcome occurs (Gerstner et al., 2018). Similarly, normative accounts link large prediction errors high information gain (Friston, 2005; Itti & Baldi, 2009). Our use of top-k losses operationalizes this principle by ensuring that only high-salience presynaptic events are retained in the CETs. While do not commit to specific biological mechanism, this abstraction is consistent with proposals in which global neuromodulators signal behavioral relevance (Dabney et al., 2020) or dendritic compartments represent mismatches between predictions and inputs (Sacramento et al., 2017; Aizenbud et al., 2025).
In summary, our work on CETs provides an extension to the classical ET approach for handling delays between activity and feedback error signals or rewards. We have demonstrated that cascades of biochemical processes could be used by cells store more temporally precise memories of past cell activity. These memories could then be combined with delayed error signals to estimate loss gradients. Therefore, our work provides another potential means of understanding how the brain can learn complicated tasks in biologically plausible manner.
ACKNOWLEDGMENTS AND DISCLOSURE OF FUNDING
This work was supported by: NSERC (Discovery Grant: RGPIN-2020-051005: Discovery Ac- celerator Supplement: RGPAS-2020-0003:; Arthur B. McDonald Fellowship: 566355-2022);
CIFAR (Canada AI Chair; Learning in Machine and Brains Fellowship); AccelNet (Interna- tional network for brain-inspired computation, NSF 2019976); Canada Excellence Research Chairs (CERC) Program. This research enabled in part by support provided by (Calcul Québec) (https: http:///.aaccuuubbeeeeeeeeeeeeee ca/en/) and the Digital Research Alliance of Canada (https: : htt//allaannceecc lanceccan ca cene... The authors acknowledge the material support of NVIDIA in the form of computational resources. The research was enabled in part by computational resources provided by Mila Quebee Artificial Intelligence Institute.
IA thanks Stephen Chung for his many helpful discussions and valuable comments on the reinforce- ment learning components of the project.
10

---

Published as a conference paper at ICLR 2026
Reproducibility Statement. Code is available at: https://github. con/avecpleziz CET, with instructions for running the main experiments. Appendix C describes details for all hyperparameter configurations, model architectures, and the training pipeline.
REFERENCES
Ido Aizenbud, Nicholas Audette, Ryszard Auksztulewicz, Krzysztof Basiński, André M Bastos, Michael Berry, Andres Canales-Johnson, Hannah Choi, Claudia Clopath, Uri Cohen, et al. Neu- ral mechanisms of predictive processing: a collaborative community experiment through the openscope program. arXiv preprint arXiv:2504.090 2025.
Mohamed Akrout, Collin Wilson, Peter Humphreys, Timothy Lillicrap, and Douglas B Tweed. Deep learning without weight transport. Advances in neural information processing systems, 32, 2019.
Bradley E Alger. Retrograde signaling in the regulation of synaptic transmission: focus on endo- cannabinoids. Progress in neurobiology. 6844):24,, 2002.
Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj, Robert Legenstein, and Wolfgang Maass. A solution to the learning dilemma for recurrent networks of spiking neurons. Nature communications, 11(1):3625, 2020.
Katie C Bittner, Aaron D Milstein, Christine Grienberger, Sandro Romani, and Jeffrey C Magee. Behavioral time scale synaptic plasticity underlies cal place fields. Science, 35336355) 2017.
Zuzanna Brzosko. Wolfram Schultz, and Ole Paulsen. Retroactive modulation of spike timing- dependent plasticity by dopamine. elife, 4:c09685, 2015.
Bianxiao Cui, Chengbiao Wu, Liang Chen, Alfredo Ramirez, Elaine L Bearer, Wei-Ping Li, William C Mobley, and Steven Chu. One time, live tracking of ngf axonal transport using quantum dots. Proceedings of the National Academy of Sciences, 104(34):13666 1661-13661,6166 2007.
Will Dabney, Zeb Kurth-Nelson, Naoshige Uchida, Clara Kwon Starkweather, Demis Hassabis, Rémi Munos, and Matthew Botvinick. A distributional code for value in dopamine-based reinforcement learning. Nature, (77((772))6711667) 2020.
Jiu-lin Du and Mu-ming Poo. Rapid bdnf-induced retrograde synaptic modification in developing retinotectal system. Nature, 429(6994) 2004.
Xinhao Fan and Shreesh P Mysore. Contribute to balance, wire in accordance: Emergence of backpropagation from a simple, bio-plausible neuroplasticity rule. bioRxiv, pp. 2024-05, 2024.
Reiko Maki Fitzsimonds and Mu-ming Poo. Retrograde signaling in the development and modification of synapses. Physiological reviews, 78(1): 143-)10,44770, 1998.
Reiko Maki Fitzsimonds, Hong-jun Song, and Mu-ming Poo. Propagation of activity-dependent synaptic depression in simple neural networks. Nature, 388(6641):4394445 1997.
Nicolas Frémaux and Wulfram Gerstner. Neuromodulated ppike-timing-dependee plasticity, and theory of three-factor learning rules. Frontiers in neural circuits, 9:85, 2016.
Johannes Friedrich, Robert Urbanczik, and Walter Senn. Spatio-temporal credit assignment in neuronal population learning. PLoS computational biology, 7())x 2011.
Karl Friston. A theory of cortical responses. Philosophical transactions of the Royal Society B: Biological sciences,
2005.
Stefano Fusi, Patrick Drew, and Larry F Abbott. Cascade models of synaptically stored memories. Neuron, 45(4):599-61) 2005.
Wulfram Gerstner, Marco Lehmann, Vasiliki Liakoni, Dane Corneil, and Johanni Brea. Eligibility traces and plasticity on behavioral time scales: experimental support of neohebbian three-factor learning rules. Frontiers in neural circuits, 12:53, 2018.
11

---

Published as a conference paper at ICLR 2026
Will Greedy, Heng Wei Zhu, Joseph Pemberton, Jack Mellor, and Rui Ponte Costa. Single-phasee deep learning in cortico-cortictt networks. Advances neural information processing systems, 35: 14213-2422) 2022.
Stephen Grossberg. Competitive learning: From interactive activation to adaptive resonance. Cogni- tive science, 11(1):23-66 63, 1987.
Kenneth D Harris. Stability of the fittest: organizing learning through retroaxonal signals. Trends in neurosciences, 31(3):130-13 2008
Kaiwen He, Marco Huertas, Su Z Hong, XiaoXiu Tie, Johannes W Hell, Harel Shouval, and Alfredo Kirkwood. Distinct eligibility traces for ltp and cortical synapses. Neuron, 88(3):528- 2015.
Gregor M Hoerzer, Robert Legenstein, and Wolfgang Maass. Emergence of complex computational structures from chaotic neural networks through swwar-mooulatted hebbian learning. Cerebral cortex, 24(3):677-690, 2014.
Marco A Huertas, Sarah E Schwettmann, and Harel Z Shouval. The role of multiple neuromodulatorrs in reinforcement learning that is based on competition between eligibility traces. Frontiers in synaptic neuroscience, 8:37, 2016.
W Tao Hui-zhong, Li Zhang, Guo-qiang Bi, and Mu-ming Poo. Selective presynaptic propagation of long-term potentiation in defined neural networks. Journal of neuroscience, 20(9):3233-3244 2000.
Laurent Itti and Pierre Baldi. Bayesian surprise attracts human attention. Vision research, 49(10): 1295-1306, 2009.
Max Jaderberg, Wojciech Marian Czarnecki, Simon Osindero, Oriol Vinyals, Alex Graves, David Silver, and Koray Kavukcuoglu. Decoupled neural interfaces using synthetic gradients. In International conference on machine learning, pp. 1627-1635. PMLR, 2017.
Alex Krizhevsky, Vinod Nair, Geoffrey Hinton, et al. The cifar- dataset. online: http:://ww.. cs. toronto. edu/kriz/cifar. huml, 55(5):2, 2014.
Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdb/nmistf/ 1998.
Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman. Random synaptic feedback weights support eиror backpropagation for deep learning. Nature communications, 7(1): 13276, 2016.
Timothy P Lillicrap, Adam Santoro, Luke Marris, Colin J Akerman, and Geoffrey Hinton. Backprop- agation and the brain. Nature Reviews Neuroscience, 21 21(6):33--346 2020.
Yuhan Helena Liu, Stephen Smith, Stefan Mihalas, Eric Shea-Brown, and Uygar Sümbül. Biologically- backpropagation through arbitrary timespans via local neuromodulators. Advances in Neural Information Processing Systems, 15:17528-17542. 2022.
Sandra Maday, Alison Twelvetrees, Armen J Moughamian, and Erika LF Holzbaur, Axonal transport: cargo-specific mechanisms of motility and regulation. Neuron, 84(2):292-309, 2014.
Mateusz Malinowski, Grzegorz Swirszez, Joao Carreira, and Viorica Patraucean. Sideways: Depth- parallel training of video models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1184-1 1843, 2020.
Arild Nøkland. Direct feedback alignment provides learning in deep neural networks. Advances in neural information processing systems, 29, 2016.
Mohsen Omrani, Chantelle Murnaghan, Andrew Pruszynski, and Stephen H Scott. Distributed task-specific processing of somatosensory feedback for voluntary motor control. Elife, 1:e11111, 2016.
Alexander G Ororbia. Brain-inan ieain-inspired machine intelligence: survey of meurobiologically--lassii credit assignment. arXiv preprint arXiv:23/2.092257 2023.
12

---

Published as a conference paper at ICLR 2026
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, segh-peffrraace deep learning library, 2019. URL https::tpp..omo........tt wwtp///wwvvvv/////////////////aa org /abs///////// 91 2 01 01703.
Alexandre Payeur, Jordan Guerguiev, Friedemamn Zenke, Blake A Richards, and Richard Naud. Burst- dependent synaptic plasticity can coordinate learning in hierarchical circuits. Nature neuroscience, 2407):101-1019, 2021.
Wade G Regehr, Megan Carey, and Aaron R Best, Activity-dependen regulation of synapses by retrograde messengers. Neuron, 63(2):154-177 2009.
J Sacramento, RP Costa, Y Bengio, and W Senn. Dendritic error backpropagation in deep cortical 2017.
microcircuits, arxiv. arXiv preprini
Stephen H Scott. A functional taxonomy of bottom-up sensory feedback processing for motor actions. Trends in neurosciences, 39(8):512-526, 2016
Tomomi Shindou, Mayumi Shindou, Sakurako Watanabe, and Jeffery Wickens. A silent eligibility trace enables dopamine-depender synaptic plasticity for reinforcement learning in the mouse striatum. European Journal of Neuroscience, 49(5):726-736, 2019.
Harel Z. Shouval and Alfredo Kirkwood. Eligibility traces as a synaptic substrate for learning. Current Opinion in Neurobiology, 91:102978, 2025.
Sen Song, Per Jesper Sjöström, Markus Reigl, Sacha Nelson, and Dmitri B Chklovskii. Highly nonrandom features of synaptic connectivity in local cortical circuits. PLoS biology, 3(3):e68, 2005.
David Sussillo and Larry Abbott, Generating coherent patterns activity from chaotic neural networks. Neuron, 63(4):544 557, 2009.
Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction, chappe 12, pp. 287-319, MIT Press, 2 edition, 2018. wwe.////tetbtoo//Baaaeaee
wwwrew..dw....mm cmu. watp:eeduccouuues
pdf.
Apara Suvrathan. Beyond stdp-toward, diverse and functionally relevant plasticity rules. Current opinion in neurobiology, 54:12-15 19, 2019.
Aparn Suvrathan, Hannah L Payne, and Jennifer L Raymond. Timing rules for synaptic plasticity matched to behavioral function. Neuron, 92(5):959- 2016.
Mark CW van Rossum and Aaron Pache. Competitive plasticity to reduce the energetic costs of learning. PLOS Computational Biology, 20(10):e101253, 2024.
Kenny Young and Tian Tian. Minatar: An atari-inspired testbed for thorough and reprodueible reinforcement learning experiments. arXiv preprint arXiv: 037906 2019.
Laurence R Young and Lawrence Stark. Variable feedback experiments testing sampled data model for eye tracking movements. IEEE Transactions on Human Factors in Electronics, (1):38-51, 1963.
Yili Zhang, Paul D Smolen, Leonard Cleary, and John H Byrne. Quantitative description of the interactions among kinase cascades underlying long-term plasticity of aplysia sensory neurons. Scientific reports, 11(1::1491, 2021.
13

---

Published as a conference paper at ICLR 2026
A CASCADING ELIGIBILITY TRACES DERIVATIONS
A. UPDATE DERIVATION
The ET we presented in Eq. (4) has the form
x(t) = A A b(t),
which are solved by
x(t) = exp( A1) x(0) + / exp(AAt 8)) b(s)ds
(6)
While the matrix exponent (At(At) is hard to compute in general, in our case,
0
It
(7)
i
-a 0
0
OFT
CET
therefore A I, + for nilpotent N (i.c. N 0), hence
exp((a + N) 0) = exp(o
i N'C
1
0
1
12
= exp(all
t
2T
(n-2
t 1
(n-1))
(n-2))
Therefore, for x(0) = 0 and b(t) being non-zero only for the first coordinate, the last coordinate x implements
(t) [' expla (t s)) (t (5-s)"a s)n=1 bu(s) ds. (n 1)!
Moreover, if b(t) is step-wise function taking on a new value every At points, a single step of the integration between and At can be computed (exactly) using Eq. (6)
x(t + At) x0) +
exp(A(At s)) ds b(t)
exp(AAA) + exp(A 8) ds b(()
As A is non-singular, we can integrate this solution further to obtain
x(t + At) exp(AAt) x(t) [exp At) I b(t). -]t)-.
(8)
A.2 LAPLACE DOMAIN ANALYSIS
To clarify the effect of increasing we include brief Laplace domain analysis. We can show that the Laplace transform converges pointwise to the Laplace transform of the perfect delay. Therefore, the outputs u hn converge to u(t - T)) for fiitt-bandwidtth low-frequency signals since the system acts as a low-pass filter, which explains why increasing n sharpens the kernel and yields better results. Proposition 1. Let
Zn = 1
(t) e at/T
the uut/T dt.
Then the Laplace transform L (s))((s) converges pointwise =T co.
14

---

Published as a conference paper at ICLR 2026
Proof. We first compute the normalization constant:
Zn = //000 ene-nt/T T(n 1)
(n/T) n+1*"
The Laplace transform of h, is
((hn()))(s) = 1, /// e -n/TTTTe dt.
Evaluating the integral gives
C(hn (n/T) m+1 T(n+1) I(n 11 (s n/T n+1
n/T m+1
sT
-(++1)
1+
8 + n/TT
m
Taking the limit,
-(++1)
sT +
lim (8) = lim
e =T ST
700
n
which is the Laplace transform of the ideal delay 6(t T).
A.3 CONNECTION TO COMBINED LTP & LTD TRACES
Proposition 2. The L1 normalized LTP-LTD kernel over [0, oo) is given by
ab h(() b b- (e and we wish to find a, concentrate the mass around When viewing h(()) as a probability density function want minimize f(a,b) En [[t T)31
bt)
f(a, b) = + + -
a²
subject reaching a maximum or equivalently with the constraint
In(a) In(b)
T, a a b
= 0 = g(a,b)
a b.
Then the infimum over {(a,b) g(a, b) = 0, a b,a 0,b 0} achieved only in the limit a, 1/T, with such a sequence converging pointwise and in L, to + t/T
Proof. Solving the constraint g(a, b) for (a, b) in terms of single real parameter x yields the smooth parametrization
TET>
a(x)
b(x)
TTx
el'2
valid for x + 0, with the continuous extension a0) = b(0) 1/T.
Substituting this parametrization into the objective gives
= f(((x),,(())) (1 (L-eT T222 (1 (1- (1-eTwye TT 2T (-Txe + 2Tn - -1)2 2T
x2
2
x22
We can directly check that f(x) is an even function, hence x 0 is an extreme point.
computational verification shows that the extended function is strictly convex and therefore attains its unique minimum at æ = O.
follows that the objective minimized by taking sequence (anz bn) with
1
an, bn - T On + bn.
15

---

Published as a conference paper at ICLR 2026
It remains show that such sequence converges to yoz te te-
Let f(A. t) e By the mean value theorem, there exists (an, bn)) such that
0x s(A, = -to ctt at bb
b
te 6 e-b
a
Then for any sequence (an, b,) with (a,, bnl + 0 and On 1/T, the corresponding kernels h
converge pointwise the desired limit. Since 0 S hn S Ce C((+/(+)) for all sufficiently large n. dominated convergence ensures L1 convergence.
B
IMPLEMENTATION DETAILS
B.1 ALTERNATIVE EXPRESSION FOR STEPWISE INPUTS
For experiments on visual tasks, we consider that batch of inputs corresponds to time-series where the batch dimension corresponds to the time dimension. In this case, output of the state-space model in Eq. (4) can be obtained via a discrete convolution, which can be efficiently computed as matrix multiplication.
Starting from Eq. (8), we can denote M = exp(AA)) and K [exp(A At) IJ A-1, such that
x(t + Ar) = Mx + Kb((),
and therefore
k
x(kAt) M³ x(0)
Σ M Kb(iAtt).
CET If we assume the initial state was zero, x(0) = 0, the SSM outputs I At (dropping the SSM superscript for convenience)
will be computed as
20 JAt
9o
0
bo bat
90
GGb[t]
(9)
: bb Ar
2C A1
9k-1
_ol
where 9j = (M'K)no
Alternatively. to get closed form expression for g. may rewrite the stepwise constant input as the convolution the appropriate impulse train with the rectangular function. Using for the Dirac delta and 0 for the Heavyside function, we have
b(t) = ajô(t - t.), rect(t) 0(t) - -0(t 1)
6(1) = Σ
arrect(L - 1.) = (b (e)t)(1)
1=1
x(1) = * y b (rect g)
= (rect gt tz).
We can therefore compute the exact continuous time output with a discrete convolution using
i rect g-
16

---

Published as a conference paper at ICLR 2026
g(0) O(1)kl"
h(F)rect(l- T))
g
(10)
k
-"
max((,,-11)
- u+
12 11 7(1 + 1, n max t 1))
where y is the incomplete Gamma function.
B.2 SPARSIFICATION
In matrix form, sparsification with indices in T (f1,..., then corresponds a matrix multipli- cation with the diagonal matrix ST diag (17),
[t| S
When using the Hebbian-like term h[4 = [1( w) x[4 as inputs to the SSM, the gradient computation when both inputs and gradients are sparsified. respectively with T, T will be
b L(x w) S o GSyh[(] i w
(11)
S @ ST GS)
Since the sparsifying matrices are indicator functions, this is equivalent to indexing 6,, G, and h h[t] at the appropriate positions defined by and T'. When simulating sparsity, we obtain the original input presentation indices T as the salient image indices and compute the arrival time gradient indices as T' T (m 1)T. The gradient over the batch is then computed by summing over the time-or equivalently, batch dimension.
C EXPERIMENTAL DETAILS
Visual experiments. For visual experiments, consider the batch dimension to be the time dimension, and we compute the delayed signals over the batch dimension using a matrix convolution described in Section B.2. The experiments in Section 4.1 use batch size of 128 samples, while the experiments in Section 4.2 use a batch size of 1280 samples, where only the samples with the top 1.25% of training losses are used. Networks were trained using the cross-entropy loss and the AdamW optimizer with 31 = 0.9, 32 0.999. The learning rate was scaled with a linear warm-up over 10% and 20% of the training steps for Section 4.1 and Section 4.2, respectively, followed by cosine annealing to 10% of the initial learning rate. For the experiments in Section 4.1, the maximal learning rate was selected from logarithmic grid of points spanning 10-3 10-7, and the weight decay was chosen from the set 10.1, 0.01, 0.001, 0.03. For the CIFAR-10 experiments in Section 4.2, the maximal learning rate was selected from {5 x 10 2.5 x 10-5 5 1 x 10 7.5 x 10-6, x 10 10-6,7 2.5 x 10
10 10-6), and the weight decay was fixed to 0. Hyper-parameter for the MNIST experiments in Section 4.2, were searched the same way as for Section 4.1. Hyper-parametes presented in Tables to 4 were independently selected using 90%/10% split of the standard training set, and the models were retrained using the standard training set and tested on the standard test set. All experiments in Section 4.1 well the MNIST experiments Section 4.2 were run for 20000 training steps. The CIFAR-10 experiments in Section 4.2 were run for 1562500 steps. in large part due to lower learning rates. Data augmentation using random horizontal flips was applied only the CIFAR- 10 experiments.
17

---

Published as a conference paper at ICLR 2026
Table 1: Experiment configurations for CIFAR-
Table 2: Experiment configurations for MNIST
10 experiments at behavioural timescales.
experiments at behavioural timescales.
CET order
delay
Ir
weight decay
CET order
delay
weight decay
0.2
le-3
le-1
1
0.2
le-3
le-1
le-3
le-1
2
0.2
le-3
le-3
0.2
le-3
le-1
6
0.2
le-3
le-2
10
0,2
le-3
lc-1
10
0.2
le-3
le-2
0.6
lc-3
le-3
1
0.6
le-3
lc-3
le-1
2
0.6
1e-3
le-3
le-1
6
0.6
1e-3
10
0.6
le-3
le-1
10
0.6
1c-3
1e-3
1,0
le-4
0
1
1.0
1e-3
1.0
le-3
le-2
2
1.0
le-3
le-3
le-1
6
1.0
le-3
10
1.0
le-3
le-1
1.0
le-3
le-3
2.0
lc-4
le-3
1
2.0
1c-3
1c-3
2.0
lc-4
le-3
2
2.0
lc-3
lc-2
2.0
lc-3
le-3
6
2.0
1c-3
10
le-3
2.0
Ie-3
le-2
4.0
le-4
le-3
1
4.0
le-3
4.0
le-4
0
2
4.0
le-3
4.0
le-4
le-3
6
4.0
le-3
10
le-4
le-2
4.0
le-3
le-3
10.0
le-4
0
1 10.0
le-4
10.0
le-4
le-2
2
10.0
1c-4
le-2
10.0
le-4
1e-3
6
10.0
1c-3
0
10 10.0
lc-4
1e-3
10.0
1c-3
1e-3
Table Experiment configurations for CIFAR-
Table 4: Configurations for MNIST experiments at retrograde timescales.
10 experiments at retrograde timescales.
CET order
weight decay
CET order
Ir
weight decay
le-5
le-1
le-4
le-3
2
le-5
lc-1
lc-4
le-1
6
5e-5
Ic-1
6
lc-4
1c-3
10
5e-5
Ic-1
10
le-4
0.0
In Actor-Critic, we train the Critic using the standard A-return, while the Actor is trained using RL eligibility traces (see Algorithm 1). The term V log (a(a, (s) in Algorithm 1 refers either to the true gradient obtained via backpropagation or to its ET/CET approximations, computed as the product of the top-level gradient signal and the ET/CET output. The CET update is computed using Eq. 8.
RL.
The learning rate was selected from the set 2.5e-4, 5e-4, 9e-4, 1e-4 based on performance for all experiments. For all classic ET runs, the ET discount factor, B, was chosen from 0.5, 0.7, 0.9, 0.99. Additionally, used two normalization schemes for CET outputs: area and peak normalizations, In area normalization. the CET output is scaled so that the response to unit input integrates to one across all future states. In peak normalization, the CET output is scaled such that the maximum response to unit input is one. For Fig.8, this normalization hyperparameter was also searched.
For the Critic, we used either the same architecture as the Actor, a three-layer ML.P with hidden dimension 256, or convolutional neural network (CNN) for MinAtaarSpaaeennvddess of three convolutional layers (kernel size 3, zero-padding 1) followed by two fully connected layers. The ReLU actívation function was used in all experiments. For CartPole and LunarLander, controlled the simulated time elapsed between the environment receiving an action from the agent and producing the corresponding next state and reward. This time was set to 200 ms for behavioral timescale experiments (Section 4. and 300 ms for retrograde signaling (Section 4.2), based on the time modeling assumptions described in the referenced sections.
consisting
also
18

---

Published as a conference paper at ICLR 2026
Algorithm 1 Actor learning via RL eligibility traces.
Initialize actor parameters 0, RL eligibility trace vector 0, gradient accumulator VoL learning rate and trace decay X.
2: Sample initial state Su from the environment
3: for E O,..... L do
4: Select action at TTO- 8t)
5: Take action observe 86+11
6: Update RL eligibility trace vector: e = Aye + Vg VologTo(e To(a st)
7:
Compute TD error: ?i = Ti > V(s))
8:
Accumulate gradient: VoL. = Vol. + mhe n/
9:
if mod
then
10: Update actor: = + VVL
Reset gradient: VaL 0
12: end if
end for
Table Hyperparameters used in RL experiments.
Parameter
Value
Common
Optimizer
Adam
Adam beta
(0.9, 0.999)
Adam epsilon
le-5
Weight decay
0
Policy entropy regularization coefficient
0.01
Maximum gradient norm for clipping
0.5
Learning rate
Tuned
Discount rate Y
0.99
CartPole-v1
Total number of samples
5_000_000
Number of environments
4
Number of steps to accumulate a policy gradient
128
Lambda for general advantage estimation
0.95
Anneal Ir
True
CET normalization
Peak
LunaLander-v22
Total number of samples
5_000_000
Number of environments
Number of steps accumulate policy gradient
128
Lambda for general advantage estimation
0.95
Anneal Ir CET normalization
False
Area or Tuned
MinAtaarSppaeenvadessss
Total number of samples
10 000
Number of environments
32
Number of steps accumulate a policy gradient
32
Lambda for general advantage estimation
Random Uniform(0.1, 0.99)
Anneal
False
CET normalization
Peak
To better preserve gradient alignment with ET/CET in the first layer, ensured positive inputs by doubling the input dimensionality and representing each original dimension with separate positive and negative components.
19

---

Published as a conference paper at ICLR 2026
Table 6: Optimal learning rate and ET discounting factor configurations, B, for experiments at behavioral timescales.
CartPole-vl
LunarLander--2
CET order delay
Ir
3
CET order delay
backprop
0,00090
backprop
0.00050
0.000500 0.5
0,000000
0.99
MinnAturSpaacciavaeess-t
0.00025
0.00050
CET order
delay
Ir
0.00025
0.00050
0.00025 10
0.00050 10
backprop
0.00090
0.00050 0.9
0.00090
0.99
1.2
0.00025
0.9
0.00025
0.00090
0.00022 8
0.001050
5
1.2
0,00050
00050 10
0,00090 10
10
1.2
0,00090
0.00050
0.99
0,00090
0.99
0.00090
0.99
0.000225 5
0,00050
0,00050 2
0.00025
0.00050
5
2
0,00050
0.00050 10
10
0.00050
10
2
0,00050
0.00010 0.99
0.99
0.99
0.00225 32
0,00090 32
0,00050 2
32
0.00025
32
U.D0U10
5
0.00090
0.00025 32
0.00050 32
0,00050 8
10
32
0.00010
10
0.001025 32
0.00050 10
0.99
2
0.00025 64
0.00090 64
0.00n50 2
0.00090
0.00090
64
0.00090
64
0,00090
8
0,00025
10
64
0.00050
10
64
0.00050
0,00050
128
0.000500 0.5
0.00050 128
2
128
0.000100
0.000000 2
128
8
128
0.00025
128
0.00090
128
0.00025
10 128
0.00090
Table Optimal learning rate, CET normalization, and ET discounting factor configurations, B, for experiments at retrograde timescales.
CartPole-vl
LunarLander-v2
CET order
Ir
normalization
8
CET order
Ir
normalization
B
I
0.00010
0.9
0.00025
0.5
2
0.00010
pcak
0.00050
area
5
0.00010
peak
0.00025
0.00025
pcak
10
0.00050
peak
The remaining hyperparameters used in the experiments are summarized in Table 5, and tuned hyperparameters are reported Tables 6 and 7. Note that for MinAtarrSpaceenvaderr, we use randomly sampled À value, we found this improves performance in this environment. A separate is sampled independently for every learned parameter, which is feasible due to our RL eligibility traces implementation of Actor learning.
Compute. All experiments were done on RTX 8000 and A100 GPUs. Each MNIST run takes between 3 and 10 minutes on an RTX 8000 GPU, while each CIFAR-10 run takes approximately 30 minutes for Section 4.1 and up to 24 hours for Section 4.2 RTX 8000. Each RL run takes approximately 1-2 hours to complete.
D SUPPLEMENTAL RESULTS ON GRADIENT ALIGNMENT
To complement the analyses in the main text, provide additional results on gradient alignment across all layers for different tasks. First, we show in Fig. 9 that the separation observed Fig. generally holds across different layers. A similar trend is observed for RL tasks in Fig. 11, although the separation much noisier and sometimes does not hold for the first layer. We hypothesize that the training dynamics of ET and CET can differ significantly, guiding parameters distinct regions
20

---

Published as a conference paper at ICLR 2026
in the loss landscape. In these regions, gradient alignment might occasionally be higher for ET, yet overall performance remains lower.
We can also see in Fig. 10 that the separation between the different CETs is still evident when plotting the similarity against training steps for MNIST. However, Fig. shows that this relationship is muddied for CIFAR-IO, which justifies plotting against accuracy.
Additionally. we note that no experiments were conducted with MinAtarrSpacclnvaders at the retrograde timescale. As shown in the rightmost heatmap of Fig. 4, CET does not scale well to longer timescales on Spacelnvaders, exhibiting only modest performance with an 8-second delay and consequently failing 120-second delay the retrograde timescale (not shown).
C1
C2
C3
W1
W2
Delay 1.0s
0.8
0.6
0.4
0.2
Epochs
Epochs
Epochs
Epochs
Epochs
Delay 2.05
0.8
0.6
0.4
0.2
Epochs
Epochs
Epochs
Epochs
Epochs
Delay 4.0s
0.8
0.6
0.4
0.2
25
50
25
50
0
25
50
0
25
50
25 Epochs
50
Epochs
Epochs
Epochs
Epochs
ET
CET(2)
CET(6)
CET(10)
Figure 9: Cosine similarity for different layers of CNN between between true gradients and ETs or CETs approximated gradients for all considered environments during training on CIFAR-10. C1-3: convolutional layers 1-3; W1-2: MLP layers 1-2.
21

---

Published as a conference paper at ICLR 2026
first layer
second layer
1.0
Cosine similarity
0.8
ET
CET(2)
CET(6)
0.6
CET(10)
50
100
150
200
50
100
150
200
Training steps (x100)
Training steps (x100)
Figure 10: Cosine similarity between the true gradient and the ET/CET approximation in a retrograde setting. Each plot shows an individual layer of MLP during training on MNIST
CartPole-vl 128s delay
LunarLander-v2 128s delay
Spacelnvaders-vo 8s delay
Layer 3 cosine similarity
0.95
0.9
0.8 N
0.90
0.8
0.7
0.6
0.85
0.6
0.80
0.4
0.90
Layer 2 cosine similarity
0.9
0.8
0.8
0.7
0.6
W
0.6
0.4
0.75
0.5
Layer cosine similarity
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
Samples 1e6
Samples
le6
Samples
le6
CET(2)
CET(S)
CET(10)
Figure 11: Average cosine similarity between the gradients and those approximated by ETs CETs during training, computed for each layer of a 3-layer MLP across all considered environments. The delay is the maximum Setavioral-timsseee value reported in the main text.
22

---

Published as a conference paper at ICLR 2026
conv1
conv2
conv3
fc1
Cosine similarity
0.4
0.2
200
400
200
400
200
400
200
400
Epochs
Epochs
Epochs
Epochs
ET
CET(2)
CET(6)
CET(10)
Figure 12:
Cosine similarity between the true gradient and the ET/CET approximation in a retrograde setting. Each plot shows an individual layer of CNN during training CIFAR-10
E SUPPLEMENTAL RESULTS WITH RESERVOIR (RECURRENT) NEURAL NETWORK
To demonstrate the compatibility of CETs with learning in recurrent settings, consider partially observable variant of LunarLander-v2 which one velocity component masked (POMDP). this setting, memory improves performance, motivating recurrent augmentation.
We employ reservoir network, a recurrent architecture with fixed, non-trainable recurrent matrix (commonly used in motor learning; see, e.g., Sussillo & Abbott (2009); Hoerzer et al. (2014)). Concretely, we augment the masked POMDP observation with a 256-dimensional reservoir state whose dynamics driven by the LunarLander-v2 observation. Apart from this augmentation, the downstream MLP head identical to the feedforward architecture used the fully observable MDP. Both the feedforward and reservoir-augmentte variants are trained with CET(5) under fixed delay of seconds.
CETs effectively solved the POMDP task (average return 200) with the reservoir augmentation, whereas a non-recurrent MLP baseline failed entirely (average return & 0), see Table 8. For completeness, we also evaluated Exponential Traces (ET) on the same reservoir architecture with decay factors 0.5 and 0.9; both configurations failed to learn (average return ~ 0).
Table 8: Partially observable LunarLander-v2 with masked velocity and 2s. delay. Reservoir has 256 units with fixed recurrent weights.
Architecture / Rule
Avg. Return
MLP (no recurrence) CET(5) Reservoir (256) + ET
0
2 0 200
Reservoir (256) + CET(5)
In POMDP with delayed credit assignment, CETs leverage recurrent state to overcome temporal mismatch, while ET fails under identical conditions.
F TINYIMAGENET
As a step towards scalability, we report results on TINY IMAGENET (Table 9) under the same training setup Section 4.1, using ResNer-20 with strided convolutions replaced with average pooling. We compare Exponential Traces (ET), Cascading Eligibility Traces with 10 states, and standard backpropagation. We evaluate performance with fixed delay of second after 50K training steps.
23

---

Published as a conference paper at ICLR 2026
Table 9: TINY IMAGENET performance with 1 delay after 50K training steps.
Method
Top-1 Acc.
Top-5 Acc.
0.0973
0.2658
CET (10)
0.3431
0.6136
Backprop 0.4060
0.6791
These results reinforce our hypothesis that more complex visual tasks are more sensitive to temporal mismatch, and that CETs provide substantial gains when the system is well matched to the delay. To check whether ET could be improved by mitigating gradient alignment issues via smaller learning rate, we also tested ET with reduced learning rate; for equal training time (50K steps), but the performance decreased relative to the table above.
G APPROXIMATING ACTIVATION FUNCTION DERIVATIVES ON RETROGRADE TIMESCALE
In Subsection 4.2, for retrograde experiments to test the CET model ability with fewer compounding factors, we assumed a simplified setup: perfect derivatives activation functions are given to us by oracle, though they delayed in the same way activations are. In this supplementary section, we remove this assumption.
LunarLander-v2: CET new
LunarLander-v2; CET reused
200
200
Return
100
Return
CET(2)
CET(5)
0
-100
CET(10)
-100
0
4
Samples
1e6
Samples 1e6
Figure 13: Preliminary results on approximating activation function derivatives on a retrograde independent additional CET model to approximate the
2-minute delay setup. Left: We derivative of the activation function. Right: We use the same CET model that we use approximate activations also approximate the derivatives. In both cases, use a CET output threshold (CET output threshold) to obtain the final approximation, motivated by the usage of ReLU activation function.
To account for delayed activation function derivatives, we perform preliminary experiments with two setups: (1) we use separate CET model to approximate the derivatives during the backward pass; (2) we reuse the same CET model that we use to approximate activations each layer. Given that we ReLU activation function, to obtain the final derivative approximation apply a threshold on the CET output: CET output > threshold, Results are shown in Fig. 13.
We used learning rate of 5e-4 and the threshold value was set to in all experiments in this section. This choice of the threshold is motivated by the fact that in all LunarLandes experiments the CET output is scaled such that the maximum response to unit input one. We view this reasonable heuristic, although more work is needed to derive an optimal threshold value, and the results should be considered preliminary.
24

---

Published as a conference paper at ICLR 2026
H VARYING THE ALPHA PARAMETER FOR A FIXED DELAY
"=1 which peaks at t T. In this section, assume the delay is T, but vary / by varying T", making it peak the wrong T", We experiment with a 32-second delay, T', on the behavioral timescale in LunarLander, varying the CET peak, T", from seconds 56 seconds. Fig. 14 shows the results.
In Section 3, assume that the parameter is chosen
LunarLander-42 mean episodic return
200
137
150
CET states
100
0
10
.100
75
58
193
228
223
233
247
191
143
50
8.0 20.0
25.6
28.8
30.4
31.2
33.6
35.2
38.4
44.0
56.0
CET peak (T')
Figure 14: Mean episodic return for the LunarLander environment with delay of 32 seconds across different numbers of CET states and a parameters that peak at shown timesteps. All results are averaged over seeds. Note when T' is equal to 32 seconds it recovers kernel shape assumed in the rest of the paper.
While the results are noisy, we can see that T' can vary by a few seconds without hurting performance; is symmetric it supports overestimating the delay more than underestimating it, which is consistent with the shape of the kernel. Moreover, performance degrades more rapidly for narrower kernels (e.g., (ET() as expected.
Similar to the experiments in the main text, the CET output is scaled such that the maximum response unit input one, which achieved at time 7'. We used a fixed learning of 9e-4. That and different seeding explains the difference between these results and the results in the main paper.
I NON I.I.D. VISUAL DATA
To study the effect of non i.i.d. visual streams performance, we conduct additional experiments on synthetic videos derived from CIFAR-I0 setup matching experiments in Section 4.1. We generate temporally coherent sequences by interpolating intermediate frames and the label probabilities using a Hahn kernel. Across interpolation factors, the proposed approach maintains stable performance on these non-i.i.d. inputs. Intuitively, increasing temporal correlation shifts the input spectrum toward lower frequencies, where CETs are particularly effective because they act as low-pass temporal filters whose fidelity improves with
J SUPPLEMENTAL RESULTS WITH LEAKY INTEGRATE-AND-FIE NEURONS
As additional step towards biologically plausible learning, evaluate feedforward network of leaky integrate-and-ff (LIF) neurons on the standard LunarLander-v2, see Table 10. We compare ET and CET with 5 states across multiple feedback delays. LIF neurons implemented with
25

---

Published as a conference paper at ICLR 2026
Accuracy Heatmap
0.68
0.56
0.57
0.59
0.59
Numbe States
0.64
0.60
0.60
0.00
0.68
0.69
Interpolation Factor
Figure 15: Accuracy for CIFAR-10 dataset across varying numbers of CET states and interpolation factors with delay of 2s. Interpolation factor of corresponds i..d. setting, while factors 2,4, and 8 introduce 1.3, and 7 interpolated frames between real images respectively during training. Performance is measured on non-interpolated frames only.
Interpolation Taclor=1
Interpolation factor=2
Interpolation factor=4
Interpolation f
similanty
0.8
ET CET(2)
CET(6)
Epochs
Epochs
Epochs
Epochs
Figure 16: Average cosine similarity between the true gradient and the gradients produced by CETs on CIFAR-10, evaluated across varying numbers of CET states and interpolation factors with fixed delay of ss.
snnTorch, using membrane decay B = 0.9 and fast-sigmoid surrogate gradient for the spike nonlinearity.
Table 10: Lunarl.ander-v2 v2 average return with LIF neurons across delays (in seconds).
Method / Delay
2
8
16
64
ET
274
226
-133
- 1136
-99
CET(5)
270
246
127
122
64
The results mirror the trends in the main paper: increasing delay degrades performance for both ET and CET, but CET becomes significantly better than ET longer delays (e.g., T' > 16s).
All architectural components other than the spiking layer remain identical to the feedforward MLP used in the main text.
K STANDARD DEVIATION FOR REINFORCEMENT LEARNING RESULTS
We report results with standard deviations for RL tasks reported the main text at the behavioral timescale in Fig. 17 to show reliability.
26

---

Published as a conference paper at ICLR 2026
Cardiblev1
Lamaruandewz
209
450
86
250
48
140
50
400
32
staake
482
40%
265
231
102
OFT states
149
142
301
66
25
166
43
8
1
27
1
24
14
1
14
2 2
12
4
100
490
484
496
494
490
4.8
258
266
256
236
100
192
3400
133
302
62
2.5
26
26
250
15
1
2.
1.3
19
80
500
273
186
137
108
200
254
254
254
298
298
13
18
13
27
64
128
2
32
64
128
1.2
delay
delay (6)
delay 1s1
Figure 17: Same as Fig. 4, but with standard deviation over three seeds. "inf"" refers to backpropaga- tion baseline.
L SUPPLEMENTAL RESULTS ON VARIABLE DELAY
We report result variable delay in three settings long delays, moderate delays, and the RL.
Across settings, CETs consistently outperform ETs for unimodal, peaky
distributions. As expected, when the delay variance approaches the full delay range (i.e., the truncated Gaussian approaches uniform), the performance gap between CET and ET diminishes.
L.1 VARIABLE DELAYS ON BEHAVIORAL TIME SCALE
For visual experiments, we study robustness to variable feedback delays by drawing per-trial delays from a Gaussian centered at the nominal mean (7 s or T 2), truncated to [0, 27]. We sweep standard deviations Tvar E {0, T
mirroring the CIFAR-10 setup in Section 4.1.
CETs consistently outperform ETs under variable delays; increasing CET degree improves accuracy even when Tvar T.. Importantly, CET performance does not collapse below ET under high variability.
Table 11: Accuracy standard deviation for T 1 s with delays in [0, 2T).
Method
00.0
Tyar 00.5
-0.50
Tyar -1.00
ET
0.68
0.68
0.68
0.68
CET(2)
0.71
0.70
0.70
0.68
CET(6)
0.73
0.73
0.72
0.70
CET(10)
0.74
0.73
0.72
0.69
Table 12: Accuracy vs. standard deviation for T = 2 with delays in [0, 27).
# States
Tea 0.00
Tvai 0..0
=1.00
Tva -2.00
ET
0.62
0.63
0.64
0.64
CET(2)
0,66
0.66
0.65
0.63
CET(6)
0.71
0.70
0.69
0.65
CET(10)
0.72
0.72
0.70
0.65
L.2 VARIABLE DELAYS ON MINUTES-SCALE
We also evaluate CIFAR- 10 with delays sampled from a truncated Gaussian on [0, 2T] at minutes- scale and show that CETs remain viable under longer delays with sparse activity. In these experiments, we consider a variable delay applied uniformly to the entire network and the sparsification method in Section B.2 used in Section 4.2 with random 1% sparsity.
27

---

Published as a conference paper at ICLR 2026
Table 13: CIFAR- 10 accuracy with mean delay and range [0, 27).
Method
Taa 0.00
Tva 15.00
=
Tva 60.00
ET
0.70
0.70
0.71
0.69
CET(2)
0.73
0.72
0.72
0.70
CET(6)
0.74
0.74
0.72
0.69
CET(10)
0.74
0.74
0.72
0.68
Table 14: CIFAR-10 accuracy with mean delay T = 120 and range [0, 2T'.
Method
Va 00.00
Tva 30000
Tva -60.0
Tvar 110.000
ET
0.67
0.68
0.68
0.65
CET(2)
0.70
0.69
0.68
0.66
CET(6)
0.73
0.72
0.70
0.66
CET(10) 0.73
0.73
0.70
0.65
L.3 VARIABLE DELAY ON RL TASKS
We report variable feedback delay for trancated variances in Table 15.
under mean delay of T 32s and different
Table 15: Mean episodic return on LunarLander-vv under mean delay of T 32s, across different delay variances.
Method / Var
var 2.0
var 8.0
var = 16
var
32
ET
-42
-66
-152 - 146
CET(5)
198
55
81
-109
L.4 VARIABLE DELAYS: UNKNOWN FIXED DELAY (LEARNED)
We next consider fixed but unknown delays that must be learned. The CET kernel is initialized to peak at 200 ms (ET initialized with 200 ms mean exponential distribution). We tune single delay parameter a via a simple weight- right-perturbation (finiteddifference) update:
L+
Aa
26
where 8 A/ 1), and I. (resp. T._) the loss obtained when computing the network update assuming time constant a++5 (resp. a 0). This approximates gradient descent on T.-.) in expectation via a first-order Taylor expansion.
We can see that eligibility trace performance degrades if the decay parameter is poorly initialized, more so higher-order CETs; thus time-constant tuning is shared challenge. Nevertheless, when is learned online, CETs outperform ETs.
Table 16: Accuracy vs. true (unknown) delay T when initialized at 200 ms.
# States
T=1s
T-28
ET
0.67
0.55
CET(2)
0.72
0.61
CET(6)
0.74
0.65
CET(10)
0.74
0.67
28


Reviews:
[{"review_id": "mMDCiNlZFU", "reviewer": "Reviewer_SqB5", "summary": "The paper explores state space models as a model for cascading eligibility traces (CETs) to tackle the credit assignment problem of exponentially decaying eligibility traces.\nTuning the decay parameter of their CET lets them decide the time delay at which the CET peaks and assigns maximal credit.", "questions": "Does alpha need to be hand-tuned, or could it be learned if one thinks of a neural network-based application?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Strong performance with long delays, in some cases even comparable to backpropagation\n\n- CETs always outperform ETs\n\n - Strong performance on very long delays if input to CETs is zeroed for timesteps with low loss\n\n - Extensive experiments\n\n- Source code provided for reproducibility", "weaknesses": "Since I am not very familiar with the related work literature, I cannot assess the novelty of the work", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "N2JbNRJU0v", "reviewer": "Reviewer_kyjJ", "summary": "The paper proposes a new mechanisms for supporting temporal credit assignment in biological networks when reinforcement or error signals arrive with significant delays.The authors propose what they term as “cascading eligibility traces” (CETs), i.e., eligibility traces defined in terms of a state-space model inspired by cascading biochemical reactions. Motivated by the observation that biological systems often process error or reward signals with fixed but significant delays, the authors propose CETs to obtain temporally specific memory of past network activity, that can be matched to delayed credit signals  at behavioral time-scales ranging from seconds to minutes.\n\nThey demonstrate the proposed mechanism in a supervised experimental setting on visual classification tasks employing the MNIST/CIFAR-10 datasets (and the TinyImageNet) with delays up to seconds, where all network layers receive the same delayed error signal, on reinforcement learing tasks (namely CartPole, LunarLander, and MinAtar/SpaceInvaders), and in a retrograde signalling setting on reinforcement learning experiments, where delays stack across layers up to minutes, mimicking very slow biochemical backpropagation.\n\nThey show that increasing CET order systematically improves performance and gradient alignment when compared to standard eligibility traces (whose performance deteriorates for delayed larger than 4 seconds), but does not fully close the gap to true backpropagation for long-delay settings, where the gradient alignment stops to improve.   The authors further demonstrate results against classical eligibility traces, and provide in the supplement additional analyses that study the robustness to variable values and unknown delays, provide ablation studies on cascade order, study gradient alignment across all network layers, and application to recurrent and spiking (leaky integrate and fire) networks.\nOverall the paper provides a well-motivated, biologically grounded extension of eligibility traces, however I find that the strongest biological claim depends on extra assumptions (see below).", "questions": "## Questions\n\n- See weaknesses\n- In the retrograde experiments, how is the postsynaptic derivative at $t-T$ stored and retrieved?\n- How would you select the optimal CET order and decay parameters?\n- In Fig. 5 the authors show that gradient alignment saturates even for high CET orders at long delays. What do you think is the limiting factor and how would you overcome this?\n- have you tried the framework on visual experiments with non-i.i.d. input streams?\n\n\n## Comments\n\n\n- In the MNIST/CIFAR experiments, the authors state that in the CIFAR dataset the performance of the classical eligibility traces deteriorates at any increasing delay value, while for the MNIST there is some robustness wrt to delays. They conclude that this indicates that “more complex visual tasks are less robust to imprecise time resolution” (lines 256-257). However I am not sure whether one can immediately state this as conclusion. The fact is that the two experiments use widely different architectures (MLP vs CNN)  \n> We use a 3-layer MLP (input→512 →512 →10) for MNIST and a small CNN with 3 convolutional layers (input →32 →64 →128) and two linear layers (512 →10) for CIFAR-10.\n\nFor the MNIST dataset the 512-512 architecture is probably on the generous side of the required size spectrum, and it can probably reach very low error even if the updates are a bit noisy or slightly wrong. The CIFAR model has moderate size for the dataset, and thus needs accurate gradients to get decent accuracy. Corrupting the gradients with temporal smearing results in performance that deteriorates right away with increasing delays in the CIFAR model, but this is not directly related to the fact that the task is complex, but rather because (probably) the model is at its expressivity threshold. While I understand that more complex tasks need larger networks, I would say that to disambiguate whether the drop in performance with increasing delays for the simple eligibility traces is directly related to the complexity of the task or the expressivity of the network for the given task, one should perform the same experiment for increasingly more/less expressive architectures and observe how results change. **I am not asking authors to perform these experiments**, I just mention this as a comment for their argument in lines 256-257. Also I don’t have good intuition on how the difference in the architecture MLP vs CNN could influence the results, but would be grateful if the authors could comment on this.\n\n- The idea of the cascading eligibility traces conceptually reminds me of the much earlier and much more primitive idea of temporal filterbank employed in [1] that shape the effective learning window. I think it would be interesting if the authors could connect their  more conceptually advanced idea to this earlier work.\n\n\n## Minor\n\n\n- the authors start several sentences the text with the phase “As well,…” which reads a bit ackward and from my side is the first time I encounter this phrase at the start of a sentence.\n- In Figure 3 in vertical axis I would propose to make the axis increasing as you go up in the figure (pot the origin at the  lower left)\n\n---\n## References\n\n[1] Porr, Bernd, and Florentin Wörgötter. \"Learning with “relevance”: using a third factor to stabilize Hebbian learning.\" Neural computation 19.10 (2007): 2694-2719.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Clearly formulated and biologically motivated problem of wanting to model a mechanism for temporal credit assignment that is both temporally fine tuned, but also provides temporal extent to be able to account for delayed error or reinforcement signals with delays up to seconds.\n- The authors provide systematic experiments both in supervised and reinforcement learning settings.\n- To understand the limits in performance for long delays they study how the weight updates align with the true gradients obtained by backpropagation.", "weaknesses": "- There is no comparison with the eligibility trace formalisms that consider different traces for potentiation and depression (He et al., 2015; Huertas et al., 2016)\n- The authors assume that in the retrograde setting  the delayed error has access to the current synaptic weights and to $f’(x^\\top w)$ at the time of the original activation. This does some heavy lifting for the biological plausibility of the proposed mechanism for delayed learning since it requires that post-synaptic derivatives at the right delayed time, in order for CETs to match them to the right pre-synaptic trace.\n- As I understand it, the experiments are a slightly fine-tuned to make the proposed mechanism look good. For example, in the visual experiment the authors consider the time dimension to be the batch dimension and then convolve over it with the CET kernel. This makes the temporal credit-assignment problem much easier than in a genuine online  non-i.i.d. input stream. Later they need salience-based sparsification (top $1.25$% losses) to make the long-delay setting train at all. This sparsification strongly biases what is remembered and is itself a nontrivial extra mechanism, but it’s introduced as an implementation detail rather than a core assumption.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LcFWVmfjf6", "reviewer": "Reviewer_7gVv", "summary": "This paper introduces Cascading Eligibility Traces (CETs), a generalization of classical eligibility traces for handling delayed credit assignment in neural networks. The authors frame CETs as a state-space model inspired by biochemical cascades, which creates a temporally precise memory trace that peaks at a specific, non-zero delay. Their mechanism overcomes several of the limitations of the exponential decays in classic ETs. The paper provides extensive empirical evidence showing that CETs enable effective learning in both supervised and reinforcement learning settings with delays ranging from seconds to minutes, and connects these findings to the viability of slow retrograde axonal signaling for credit assignment in neuroscience.", "questions": "1. Could the authors expand on the biological plausibility of the salience signal mechanism in the main text?  A brief discussion on how a salience signal could be computed and propagated quickly enough to gate the slow memory formation process would be helpful for readers to understand the complete proposed system.\n\n2. The theoretical derivations could possibly benefit from including the inverse Laplace transform. The kernel g(t) of the convolution in Eq 5 is the inverse Laplace transform of the function G(s)=(s+alpha)^-n. This interpretation could provide a deeper understanding of why and how decodability improves with higher n. See [Masset, Tano et al Nature 2025] for an example of using the inverse laplace transform framework to understand what kind of information is decodable from a multi-timescale system.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well-written, and the theoretical derivations are clear and appear to be correct. The paper provides a solid and well-motivated extension of prior theory on ETs.\n\n1. The formulation of CETs as a state-space model is mathematically sound and provides a powerful generalization of a classic learning mechanism. Showing that a standard eligibility trace is equivalent to a CET with a single state (n=1) makes this work a direct and solid extension of prior work. The model's parameters (n and alpha) offer intuitive control over the precision and timing of the memory trace.\n\n2. They demonstrate the superiority of CETs over standard ETs across a diverse set of non-trivial supervised and RL tasks. As task complexity and feedback delay increase, the performance gap between higher-order CETs and standard ETs widens significantly. The included gradient alignment analysis provides a convincing explanation for these performance gains. I really appreciate this level of benchmarking, which is not common in basic computational neuro work.\n\n3. Modeling learning with minute-long, stacking delays is, to my knowledge, novel and impactful. It provides a plausible computational mechanism by which slow chemical signals, such as retrograde messengers, could effectively transmit credit information.", "weaknesses": "1. The retrograde experiments on visual tasks rely on a salience signal to sparsify memory storage, which may be a critical component for making learning tractable over minute-long delays. If my understanding is correct, this assumes the existence of an oracle-like signal that tells the synapse which events are important and thus which memories to form, which seems to be solving part of the most difficult part of the temporal credit assignment problem (i.e. what to remember) before the delayed error signal arrives.\n\n2. The results clearly show that more states are better. However, this benefit presumably comes at a cost. It would strengthen the paper to include a brief discussion on the computational complexity of the CET mechanism as a function of the number of states n.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vbhKK5mrRd", "reviewer": "Reviewer_bZyU", "summary": "This paper introduces the Cascade Eligibility Trace (CET), a novel mechanism for the temporal credit assignment problem in deep neural networks, motivated by biological plausibility. The core idea replaces the standard exponential decay of classic eligibility traces (ETs) with a cascade of $n$ coupled differential equations. The CET's primary function is to generate a memory kernel that is centered around the time of the error signal, rather than that of the sensory input. This allows learning systems to accurately assign credit for rewards or feedback signals that arrive significantly after the causal neural activity. This is achieved by effectively substituting the standard exponential kernel of ETs with a Gamma-like kernel which peaks at a given delay. The authors conduct extensive benchmarking against the classical ET across various tasks (both supervised and RL) and datasets (e.g. MNIST, LunarLander), systematically demonstrating the computational advantages of the CET paradigm. The work is well-written, with a clear and compelling narrative.", "questions": "1. How sensitive are results to the trade-off $T=(n-1)/\\alpha$? For fixed $T$, what is the optimal $n$ given compute/memory budgets? Please provide heatmaps of performance and cosine similarity over $(n,\\alpha)$ at one or two T values.\n2. In stacked delays, how would you maintain the postsynaptic derivative at the soma? Could an analogous CET run there, and do you expect additional degradation?\n3. What happens when the critic is also trained with delayed signals (e.g., delayed TD errors)? Does CET still yield stable learning?\n4. How does CET compare with *tuned* two-exponential kernels?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- ⁠The new formulation is clear and straightforward, addressing with clarity the weaknesses of previous models. Recasting eligibility as a Gamma-shaped temporal window with tunable peak and width is a principled generalization beyond two-exponential composites.\n- Theoretical derivations for both continuous-time and exact discrete-time update. The parameters $(n, \\alpha)$ have a nice interpretation as \"delay\" and \"precision\".\n- ⁠The benchmarking is extensive, and most of the results are quite solid, demonstrating  a clear advantage w.r.t. classic ET, with systematic sweeps over delays and CET orders. Author provide gradient-alignment analysis to connect performance with optimization geometry.\n- ⁠The application with accumulating delays is interesting from both a computational and a neuroscientific perspective.", "weaknesses": "- While the work strongly underlines the method’s biological plausibility, benchmarks focus on accuracy on classification and reinforcement learning tasks, but misses benchmarking against real biological data (e.g. neural predictivity metrics (e.g. Yamins et al., 2014; Yamins & DiCarlo, 2016). I’d like to see how a model trained with CET  on a image classification task (e.g. CIFAR10) compares to the same model trained with ET or backpropagation in their neural predictivity.\n- The paper argues that two-ET composites produce broad windows and thus underperform for long delays. However, no optimized two-exponential (or multi-exponential) baseline is reported in the main text under the same tuning budget as CETs. Author could for example include a two-ET case with grid-searched $(\\gamma_P,\\gamma_D)$.\n- CETs increase state size linearly in $n$, which results in per-step computational overhead. The paper should quantify the relevance of such overhead for example by comparing it with ET (e.g. FLOPs/memory). This might become relevant for example in the long-delay experiments that rely on sparsification.\n- In stacked-delay experiments, the $\\delta$-computation assumes access to $f’(⋅)$ at the appropriate delayed postsynaptic state (a soma-side memory). \n-  It seems to me that one of the key advantages of CET is built on the assumption that the variable delays are clustered. As shown in Section I in the Appendix, the benefit of CET w.r.t. ET performances decreased as the variance increased and the delay distribution became closer to uniform. The solution suggested (Perturbation-Based Learning) violates the principle of locality, which is foundational to biological functioning. This method requires a synapse to access, store, and compare the global loss from multiple non-local, trial-level executions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)