Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: Summary: this paper introduces a new variant of equilibrium propagation algorithm that continually updates the weights making it unnecessary to save steady states. The also mathematically prove the GDD property and show the effectiveness of 
 their algorithm (Continual-EP) on MNIST. They also show C-EP is conceptually closer to biological neurons than EP.

This paper tackles an important problem in bridging the gap between artificial neural networks and biological neurons. It is well-motivated and stands well in the literature as it improves its precedent algorithm (EP). The contributions are clear and well-supported by mathematical proofs. The experiments are accurately designed and results are convincing. I recommend accepting this paper as a plausible contribution to both fields. 
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: I think it is an intriguing paper, but unfortunately left me a bit confused. I have to admit is not a topic I'm really versed in, so it might be that this affected my evaluation of the work. But also, as a paper submitted to ICLR I would expect the paper to be self-contained and be able to provide all the details needed. 

I do appreciate the authors providing in the appendix the proofs of the other theorems even if they come form other works. 


The paper introduces C-EP, an extension of  a previously introduced algorithm EP, such that it becomes biologically plausible. In particular EP is local in space but not in time (you need the steady state of the recurrent state after the first stage at the end of the second stage to get your gradients). I think this is fair, and the need for biological plausibility is well motivated in the beginning of the work.  

My first issue is with the proof for the equivalence between EP and C-EP. This is done by taking the limit of eta goes to 0. I think I must be missing something. But the proof relies on eta being small enough such that \theta_i = \theta (i.e. theta does not change). Given this state evolves the same way as for EP, because we basically not changing theta. 
Yet the crux of my issue is exactly here. The proof relies on the fact that we don't change theta. So then when you converged on the second phase, isn't theta the same as theta_0? So you haven't actually learned anything!? Basically looking at the delta isn't this just misleading? 
Ok lets assume that on the last step you allow yourself to change eta to be non-zero. (I.e. we are just after the delta in theta, and what to show we can get the same delta in theta as EP which is how the proof is phrased). Then in that difference aren't you looking at s_{t+1} and s_t rather than s_{t+1} and s_0, which is what EP would do? In EP you have s^\beta_* - s_*. This is not what you get if you don't update theta and apply C-EP? 

I think there might be something I'm missing about the mathematical argument here. 

At a higher-level question, we talk about the transition function F as being a gradient vector field, i.e. there exist a phi such that F is d phi/d theta.  Why is this assumption biologically plausable ? Parametrizing gradient vector fields in general is far from trivial, and require very specific structure of the neural implementation of F to be true. Several works have looked at parametrizing gradient vector fields (https://arxiv.org/abs/1906.01563, https://arxiv.org/pdf/1608.05343.pdf) and the answer is that without parametrizing it by actually taking the gradient of a function there is not much of a choice. 
Incidentally, here we exploit that  F = sigma (Wx), with W symmetric. This is a paramtrization of a gradient vector field, i.e. of xU, where UU^T =W I think. But if you want to make F deep than it becomes non-trivial to restrict it to gradient vector field. Is the assumption that we never want to move away from vanilla RNNs? And W symmetric is also not biologically plausible. In C-EP you say is not needed to be symmetric, but that implicitly means there exist no phi and everything that follows breaks, no? 

I'm also confused by how one has access to d phi / ds and d phi / d theta. Again I feel like I'm missing information and the formalism is not introduced in a way that it is easy to parse. My understand is that you have an RNN that updates the state s. And the transfer function of this RNN is meant to be d phi / ds, which is trues if the recurrent weight is symmetric. Fine. But then why do we have access to d phi/ dtheta? Who is this function? Is the assumption that d s / dtheta is something we can compute in a biologically plausible way? Is this something that is obvious? 
 



review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: The paper is concerned with biologically plausible models of learning. It takes equilibrium propagation -- where updates depend on local spatial information, in the sense that the information is available at each neuron -- and modifies the algorithm so updates are also local in time, thus obtaining C-EP. The key insight is that the updates in EP can be written as a telescoping sum over time points, eq (5).

Making equilibrium propagation more biologically plausible is an interesting technical contribution. But, taking a step back, the setup is misguided. It is true that humans can solve classification problems. And various animals can be trained to do so as well. However, it should be obvious that animals learn to solve these problems by reinforcement learning -- they are literally given rewards like sugar water for correct answers. 

MNIST is an unusual dataset with a stark constrast between foreground and background that is far from biologically plausible. I know it has a long and important history in machine learning, but if you are interested in biologically plausible learning then it is simply the wrong dataset to start with from both an evolutionary and developmental perspective. It’s not the kind of problem evolution started with, nor is it the kind of problem human babies start with. 

Maybe C-EP can be repurposed into a component of some much larger, biologically plausible learning system that does a mixture of RL and unsupervised learning. Maybe not. The MNIST results provide no indication.

The authors have done a lot of solid work analysing BPTT, RBT, and C-EP. I suspect they are far more interested in understanding and designing efficient mechanisms for temporal credit assignment than they are in biological learning. That work can and should stand on its own feet.

review_title: Official Blind Review #3

Meta-review: Main content: paper introduces a new variant of equilibrium propagation algorithm that continually updates the weights making it unnecessary to save steady states. T

Summary of discussion:
reviewer 1: likes the idea but points out many issues with the proofs. 
reviewer 2: he really likes the novelty of paper, but review is not detailed, particularly discussing pros/cons. 
reviewer 3: likes the ideas but has questions on proofs, and also questions why MNIST is used as the evaluation tasks.
Recommendation: interesting idea but writing/proofs could be clarified better. Vote reject.


Decision: Reject