Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a biologically plausible learning method, called global error-vector broadcasting (GEVB). The authors also propose vectorized nonnegative networks (VNNs) on which the GEVB method operates. The key property of the GEVB method is that the gradient has the same sign as the ground truth gradient returned by back-propagation (BP) on VNNs. Experiments on MNIST and CIFAR are conducted to show that the proposed method can sometimes match the performance of BP and sometimes outperform competitors like direct feedback alignment.

main_review: Pros:

1, The proposed GEVB essentially propagates the error signal from the readout layer to all other layers in a direct manner. It seems to be more efficient and biologically more plausible compared to BP since it avoids the symmetric weights and the successive computation of gradients across layers. I think this is a novel and interesting contribution towards more biologically plausible learning methods.

2, The authors also show that in the limit of infinite network width, the proposed update in GEVB is proportional to the exact gradient at the initialization. This observation provides some potentially interesting connections to the recent study on infinite wide networks like NTK.

3, The empirical results show that the proposed learning method 1) does converge and 2) is comparable or even outperforms other alternatives. In particular, it successfully trains convolutional layers whereas other biologically plausible methods like direct feedback alignment (DFA) have some troubles.

Cons and Questions:

1, I agree that the proposed GEVB is more biologically plausible compared to BP. However, the introduction of this new type of architecture, i.e., VNNs, seems to be unsatisfying. In particular, do you have some evidence to support that VNNs are biologically more plausible than other common architectures?

2, Why do you need the random gating vectors per layer? If I understood correctly, your method and theory still work if those random gating vectors are removed. It is unclear to me what the motivation and the impact of this additional design choice are. More importantly, this random gating would inevitably bring some stochasticity to the final predictions. Therefore, it would be important to do a thorough ablation study on this component and show the standard deviations of the final performances corresponding to the aforementioned stochasticity.

3, Could you explain why the training and testing errors of GEVB are inconsistent for fully-connected and convolutional VNNs, i.e., better than or comparable to BP for fully-connected VNNs and significantly worse than BP for convolutional VNNs? Moreover, for convolutional VNNs, why does GEVB outperform DFA on CIFAR but perform worse on MNIST?

4, When you compute the gradient alignment angle, do you compute the angle per sample and then take the average, or do you compute the angle between gradients averaged over mini-batch? I am curious whether these two have significant differences since matching the signs of individual gradients is different from matching the signs of gradients averaged over mini-batch. From the perspective of optimization, we should care more about the latter.

5, In Fig. 2, could you comment on why the angle curve of the last readout (FC) layer seems to be diverging on fully connected VNNs? Also, why do the curves of the last readout (FC) layer first increase and then decrease rather than directly decrease on both fully-connected and convolutional VNNs?

6, The current method seems to be surprisingly slow, e.g., taking 10 days of GPU training even on CIFAR and MNIST. Could you comment on what are the exact factors that slow down the training?

## After Rebuttal

I'd like to thank the authors for the detailed response! It does address most of my concerns. 
I was expecting authors to comment in detail on if there is evidence in neural science to back up the existence of some form of the VNNs. 
However, I found the current response on this point less informative.
In addition, I agree with *Reviewer wVMT* that thorough experimental comparison on more datasets would significantly improve the paper.
In summary, I would like to keep my original rating.

limitations_and_societal_impact: The authors have discussed the computational cost as the potential negative societal impact, which I think is fair.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper proposes to circumvent the need of backpropagation in deep networks by introducing two architectural changes. First, the weights are restricted to be non-negative so the chain rule simplifies significantly. Second, each unit in a network is represented by a vector to match the dimensionality of the error signal in the top layer, further simplifying the chain rule. The resulting weight update approximates backpropagation similar to the sign symmetry method.      
main_review: The paper proposes a very novel and intriguing way to approximate backprop. The discussion on feedback alignment-type (FA) algorithms is missing two important papers, however. First, sign symmetry [1] is another FA-type algorithm that uses signs of the weights for the backward pass, and I think this algorithm works for a similar reason (although GEVB should be a tighter approximation due to non-negative weights; [26] in the paper seems relevant too, but [1] was published a few years earlier). Also, [2] shows how to introduce plasticity in the backward connections of FA to bring it on par with backprop, so it’s worth including into the discussion. Finally, non-FA-type methods [3-7] might be worth discussing (and comparing in experiments) too.
     
Quality: The submission is technically sound, and provides sufficient theoretical and experimental justification.
     
Clarity: The paper is well-written. One comment: line 236: “One possible reason for this gap is that weight sharing in convolutional networks breaks GEVB’s gradient sign match guarantee.” Does it break it though? Convolutions would add a sum over $i$ in Eq.14, resulting in $(\\sum_i \\delta_i G(h_i)) (\\sum_\\mu ...)$. GEVB should change $\\sum_i \\delta_i G(h_i)$  to $\\sum_i G(h_i)$, which doesn’t seem to break the approximation due to non-negativity of $G$. 
     
Significance: Overall, the paper is a significant result with future potential. The major limitation of the paper is weight sharing within vector-valued units. While the authors do provide a potential mechanism for that (through time unfolding), it implies a rather complicated training curriculum (especially for large K). However, this limitation is unconventional and might be solved with future methods/experimental evidence. 

[1] How Important is Weight Symmetry in Backpropagation?, Qianli Liao, Joel Z. Leibo, Tomaso Poggio

[2] Deep Learning without Weight Transport, Mohamed Akrout, Collin Wilson, Peter C. Humphreys, Timothy Lillicrap, Douglas Tweed

[3] Deep supervised learning using local errors, Hesham Mostafa, Vishwajith Ramesh, and Gert Cauwenberghs

[4] Training neural networks with local error signals, Arild Nøkland and Lars Hiller Eidnes

[5] Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks, Roman Pogodin and Peter Latham

[6] Local plasticity rules can learn deep representations using self-supervised contrastive predictions, Bernd Illing, Jean Ventura, Guillaume Bellec, Wulfram Gerstner

[7] Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing Its Gradient Estimator Bias, Axel Laborieux, Maxence Ernoult, Benjamin Scellier, Yoshua Bengio, Julie Grollier and Damien Querlioz

limitations_and_societal_impact: The authors have adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes (i) a learning rule called global error-vector broadcasting (GEVB), and (ii) a class of DNNs called
vectorized nonnegative networks (VNNs) in which this rule operates - to solve the credit assignment problem. While Backpropgation (BP) is remarkably effective at training deep neural networks, it is biologically implausible because of the weight transport problem. In this paper, the weight transport problem is circumvented by globally broadcasting the error signal to all hidden units in the network. However, in order to accommodate the vector error signal a new network model with vector units is proposed. 

Further, by deriving gradients using a BP like approach, the authors demonstrate gradient alignment with the GEVB learning rule. The performance of VNNs trained with GEVB is evaluated on MNIST and CIFAR-10 and compared with BP on VNNs and conventional networks.


main_review: __Significance__: Understanding credit assignment in biological networks is an important problem. Several approaches have been considered to circumvent the weight transport problem of BP, such as node perturbation and feedback alignment. In this work, an interesting alternative approach based on broadcasting an error signal is considered. However, the impact of this approach hinges on finding a biological plausible implementation of VNNs. 

__Originality__:  The main idea in this paper is the broadcasting of the error signal to all hidden units of the network. The broadcast error is combined in a Hebbian-like learning rule for each weight. This is in contrast to sending a unique projection of the error signal to each hidden unit as in direct feedback alignment. However, in order to accommodate a vector error signal, the units in the network are modified to have vector activities instead of scalars. This work also provides analyses that demonstrate gradient matching for the GEVB rule.

__Quality__: The experimental results show the impact of vectorization and nonnegativity of weights on the VNN model performance. In most cases the performance of VNNs trained with GEVB was quite close to BP. Gradient alignment analysis reveals that GEVB results in better alignment than DFA on average. While the model itself is interesting, the biological implementation of vectorization is not quite convincing.  Another important issue is that training large networks with this approach is computationally challenging. 

__Clarity__:  The paper is well presented and easy to follow. 
Minor: Caption for figure 2 has label (A) for both panels.

__Questions__:
* An important consideration is how the error signal is broadcast to all units in the network. Do we expect the influence of a neuromodulatory system to be exactly the same on all units in a large network? How would some stochasticity in the error signal received at each unit affect the performance?
* The activation function used here is of a very specific form. How could this be biologically implemented? 
* Non-negativity is important for gradient sign matching. And this corresponds well with the fact that a majority of cortical neurons are excitatory. However, inhibitory neurons do constitute a small but crucial neuronal class in the cortex. How well would VNNs with GEVB work without the sign constraint on the weights?
* The comparison of learned representations was done using networks trained just on CIFAR-10. This is an interesting analysis but is it conclusive enough to say GEVB in VNNs learns better representations? 
 

limitations_and_societal_impact: The limitations of the work have been discussed by the authors - performance discrepancies on more challenging tasks, biological plausibility, and the computational complexity of the forward pass in VNNs. 

There are no other potential negative societal impacts or ethical concerns.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper proposes to overcome an important biological implausibility of neural networks trained by backpropagation using two novelties: 1) to train neural networks using a new globally available error signal, 2) a new type of neural network with vector-valued units and nonnegative weights. Interestingly, the model shows that the proposed learning rules lead to update rules with the same sign as those obtained by gradient-descent. 

This work justifies the biological plausibility of the learning rule by relating it to the growing literature on three-factor Hebbian rules and justifies the choice of nonnegative weights to observed cortical properties, as well as the presence of ON/OFF cells. 

The performance of the proposed network is compared, on MNIST and CIFAR-10 datasets, against a very standard set network trained with BP, as well as more recent models trained using direct feedback alignment. The model outperforms competing DFA-trained models for fully connected and convolutional architectures but not for locally connected architectures. 
main_review: Below is addressed the clarity. 

This paper is clear and well-written, and the claims are clearly laid out. However, the notations are not easy to follow, in particular the not-standard choice of indexing of the network. It would have been useful to relate the notation with neural architecture presented in Figure 1, where it is very unclear how it relates to the rest of the notations used in the paper. It is also true for the "biological interpretation" (section 7) that does not easily relate to the architecture in Fig.1. 
The results in Table 2 are not easy to compare as the competing models (bold) results are presented in different "sub"-tables limiting the readability. The rest of the results, Fig.2 and Fig.3, are clear and well described in the caption and the text. 

Below is addressed the originality. 

This work makes an interesting effort at providing "local" learning rules, which is an important limitation of BP; as such, the learning rule is novel. The comparison to three-factor Hebbian rules is mentioned but is not clearly analyzed. The work of [24], mentioned in the paper, offers a generic framework and discusses possible biological structures in which it could occur, unlike in this work. The comparison with this work needs to be clearly stated as the model claims to be biologically sensible due to the presence of the aforementioned third factor. 
This work claims to be both theoretical and original; however, proposing an ad-hoc learning rule can hardly be qualified to be theoretical when its analysis lacks theorems, or propositions, or even lemmas. It is suggested to the author to either rewrite the paper more mathematically or to provide more experiments and biological realism. The next point relates to the lack of experiments and even mention of competing models that are potentially more biologically plausible than those presented in the paper. Indeed, pieces of work from [i] and [ii] could rely on standard unsupervised Hebbian/Hopfield models followed by an SVM (that can be easily trained with biologically plausible models) and reach competing performance on CIFAR-10. As mentioned in the limitation review, the lack of proper theoretical analysis, and biological implementation, of the algorithm does not allow for a weak experimental section. We invite the reader to strengthen at least two out of three of those contributions. Various other models have also been proposed [iii-v] that have been omitted to be cited, using local biologically plausible learning rules and using 3-factor Hebbian rules. 

Below is addressed the significance. 

This work is definitely a step in the right direction in building biologically plausible algorithms but falls short of making a significant contribution. As mentioned above, the paper needs substantial improvement in at least one of the following contributions, theoretical, experimental, or biological realism. 


[i] Krotov, Dmitry, and John J. Hopfield. "Unsupervised learning by competing hidden units." Proceedings of the National Academy of Sciences 116.16 (2019): 7723-7731.
[ii] Bahroun, Yanis, and Andrea Soltoggio. "Online representation learning with single and multi-layer Hebbian networks for image classification." International Conference on Artificial Neural Networks. Springer, Cham, 2017.

[iii] Pogodin, Roman, and Peter E. Latham. "Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks." NeurIPS 2020/
[iv] Golkar, Siavash, et al. "A biologically plausible neural network for local supervision in cortical microcircuits." NeurIPS 2020.
[v] Qin, Shanshan, Nayantara Mudur, and Cengiz Pehlevan. "Contrastive Similarity Matching for Supervised Learning." Neural Computation 33.5 (2021): 1300-1328.
limitations_and_societal_impact: Two other important limitations of deep networks have not been addressed. 

The first one is their lack of transparency, and this model remains "black-box," not offering a better understanding of how the learning is performed. The learning rule is compared to that of the gradient methods, which have shown to be hard to analyze and understand and do not address this problem. The claimed biological plausibility could have led to more understanding of the learning in the brain but fails to do so as it only vaguely addresses the topic. However, it appears to be a central point to the paper. 

The second one is the bias that has been observed in many deep neural networks trained with backpropagation. Although it is not expected that the model would solve this problem, it still needs to be mentioned. 

On a side note, the paper claims that a limitation of this work is the lack of experiments reported, i.e., only evaluated on MNIST and CIFAR-10. This paper is mostly experimental rather than theoretical; no theorem, proposition, or lemma is stated in the main text. As a result, more experiments are expected. This is thus not a limitation of the paper but a weakness of the submission.
review_title: null

Meta-review: The paper introduces a method to train neural networks in a way that is more biologically plausible than backpropagation. The key idea is to derive an update that depends on a global error signal. Work in this direction had already been identified for neural network with a scalar output, however the authors generalize the setup with vector outputs, but restricting the architectures to a very particular class of neural networks they introduce called VNN (effectively operating on vectors with vectorized weight sharing). They show the resulting update matches the sign of the gradient. Reviewers were in agreement the idea was interesting and elegant. 
Doubts remained regarding biological plausibility, with reviewers thinking the authors arguments in that direction were somewhat hand-wavy. Nevertheless, we hope this work can inspire further work in that field that could be even better substantiated.
There were some concerns regarding the extensiveness of the literature review that should be addressed for the final version.

Decision: Accept (Poster)