Title: Error Forcing in Recurrent Neural Networks
Venue: NeurIPS 2025
Accepted: yes
Keywords: recurrent neural networks

bio-plausible learning
Abstract: How should feedback influence recurrent neural network (RNN) learning? One way to address the known limitations of backpropagation through time is to directly adjust neural activities during the learning process. However, it remains unclear how to effectively use feedback to shape RNN dynamics. Here, we introduce error forcing (EF), where the network activity is guided orthogonally toward the zero-error manifold during learning. This method contrasts with alternatives like teaching forcing, which impose stronger constraints on neural activity and thus induce larger feedback influence on circuit dynamics. Furthermore, EF can be understood from a Bayesian perspective as a form of approximate dynamic inference.  Empirically, EF consistently outperforms other learning algorithms across several tasks and its benefits persist when additional biological constraints are taken into account. Overall, EF is a powerful temporal credit assignment mechanism and a promising candidate model for learning in biological systems.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTo enable learning long-range dependencies in RNNs, this paper proposed Error Forcing (EF) as an algorithm to provide explicit greedy error-based guidance of circuit dynamics along with temporal credit assignment. It builds on top of teacher forcing scheme wherein the updated RNN state is a linear interpolation between with the teacher guided state ($r^*_t$) and the natural RNN state ($r_t$), i.e., the updated RNN state $\\tilde{r}_t = (1-\\alpha) r_t + \\alpha r^*_t$. The RNN non-linear hidden state recursion uses the updated hidden state, i.e., $r_t = F_\\theta(r^*_{t-1}, x_t)$. Since the output $y = W_\\phi r_t$, for teacher forcing, $r^*_t = W^{-1}_\\phi y^*_t$, if the inverse does not exist, one can opt for pseudo inverse $W^{+}_\\phi$. However, the issue with teacher forcing is that it constraints the RNN state update and activity heavily towards the teacher guidance.  \n\nIn this proposed work, Error Forcing uses the target state $r^*_t = r_t + g(W^{+}_\\phi  (y^*_t - y_t) )$, this enables exploration of the RNN hidden state along with relaxed guidance from teacher. This work analyzes the proposed error forcing mechanism through the geometric and Bayesian lens. Various empirical experiments have been done to validate the efficacy of the proposed error forcing mechanism on diverse set of tasks. When combined with BPTT, EF-BPTT was able to outperform TF-BPTT and BPTT, for delayed XOR, Sine Wave Generation and evidence integration tasks. Similar observations were achieved when replacing BPTT with RFLO paradigm.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Since the teacher guidance is more relaxed compared to the teacher forcing update, Error Forcing (EF) allows the RNN to explore a better hidden state trajectory while still guided with the teacher update. \n- Error Forcing outperforms Teacher Forcing in both BPTT and RFLO optimization paradigms in the empirical experiments\n\nWeaknesses:\n- The proposed method has only been evaluated with toy experimental setups (such as delayed XOR, sine wave generation, evidence int.). It is unclear how the error forcing would perform on tasks with more complexity and higher dimension input / outputs.\n-  In the update equation, Error Forcing proposes to use the stop gradient operation to remove feedback loop back to the teacher guidance. This proposed update seems a bit ad-hoc and has not been explored in detail.\n\nQUESTIONS:\n- In Sec. 3.1, how were the $\\alpha$ values chosen for GTF-BPTT and EF-BPTT?\n- In Fig 3c, since EF-BPTT reduces to BPTT for $\\alpha=0$, why doesn’t the percentage of network learned the same for both these curves at $\\alpha=0$ ? \n- What is the significance of $\\alpha=1$ in the update Eq. (10)? This does not translate to the equivalent teacher forcing update due to the stop gradient operation. Similarly, shouldn’t teacher forcing and error forcing achieve similar performance for $\\alpha=1$ regime?\n- In evidence int. , BPTT seems to be performing similar to GTF and EF, it’s unclear what advantage these two methods bring over the traditional BPTT on this task. \n- In supplementary (C additional results), it seems the experiments without stopping gradients seem to be obtaining decent results. If so, is there any reason to keep the stop-gradient operation in the hidden state update?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce Error Forcing, a departure from Teacher Forcing (TF) that focuses on minimal intervention in including error-based learning, while also adding gradient stopping to deal with long-term temporal dependencies.  The authors provide geometric and Bayesian perspectives to explain the methodology, as well as consider more biologically realistic alternatives. They demonstrate the learning model on a variety of tasks, constrasting it against classical BPTT (a subcase of their method) and TF.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The motivation for Error forcing as minimal intervention is well-framed. \n- The geometric and Bayesian perspectives are complementary and both useful to provide intuition. \n- The authors demonstrate their inference methodology across a variety of tasks with varying difficulties. \n\nWeaknesses:\n- The argument that less forcing implies more exploration could be made stronger. Theoretical derivations, providing geometric intuition beyond full forcing or incorporating numerical experiments to corroborate would all strengthen the claim. \n- The work is fairly skim on the inclusion of other relevant works, especially on online learning from feedback errors.\n- The Bayesian perspective is insightful but would benefit from being more carefully detailed. I gather the EF forward pass emulates Kalman filtering (or potentially recursive least squares for static targets) and the variational EM comes from the alternation with the backward pass. These connections feel high-level, however, when in fact they are quite exact. I believe fleshing out these ideas even further and bringing in details from the Appendix could help the reader and strengthen the argument.\n\nQUESTIONS:\n- Could you explain further the distinction between training and testing alluded to at the end of Section 2.3? \n- Is there a mismatch at $\\alpha = 0$ in Fig 3c with EF-BPTT and BPTT not coinciding?\n- What is going on in early training in Fig. 3F, with EF-BPTT being a bit worst at first than BPTT? What explains this difference in behavior?\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a variation on teacher forcing for RNNs, which moves the internal state on each timestep toward a target state on the solution manifold (the set of states yielding zero error). The proposed Error Forcing modifies teacher forcing by picking the target state that minimizes the norm of the adjustment rather than the norm of the target state itself.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- Well motivated variation on the existing teacher forcing; retrospectively this is clearly the right way to do TF\n- Principled Bayesian motivation\n- Clear experimental advantage over TF and baselines\n\nWeaknesses\n\n- Experiments are in toy domains. Scaling up is especially important for this project because of the nature of the problem it aims to solve (vanishing gradients over long sequences).\n\nQUESTIONS:\nEF and TF both train the model for a different task than it will be tested on. That is, its weights are optimized for a setting where its state is nudged by an oracle on each timestep. How does this affect generalization to the test scenario where there is no nudging? Would the model benefit from being weaned off EF in the latter phases of training (by reducing alpha toward zero)?\n\nThe greedy MAP estimation of epsilon makes the Bayesian connection somewhat superficial; it’s just the classic idea that delta-rule learning can be interpreted as the mean update of a conjugate Gaussian prior. If you wanted to more seriously pursue the Bayesian connection you could do a proper (extended) KF and carry along uncertainty in r_t (probably using a diagonal covariance matrix for computation efficiency).\n\nIt’s unclear whether the metrics in the experiments are from training or testing — specifically the % networks learned in figs 3bchi and 4abd and MSE in 3e (whereas the caption for 3f says test MSE).\n\nA similar issue comes up in Transformers, specifically causal language modeling where teacher forcing entails that LLMs are typically trained only on one step ahead prediction (e.g., arxiv:2403.06963). This goes beyond the present paper’s scope but I wonder whether EF would be helpful there.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}