Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The work presents an interesting equivalency between a generative probabilistic mixture model and a winner-take-all Hebbian local learning rule-based learning. The approach seems promising, especially the adversarial robustness results compared to a multi-layer perceptron. 

**UPDATED SCORE**
I would like to thank the authors for their revisions and modification based on my feedback. I am changing my score from 5 --> 6 to reflect that. I would also like to mention that there have been works based on Hebbian/Non-Hebbian Learning that outperform backdrop-based  approaches in challenging scenarios such as continual supervised/reinforcement learning. There are other strategies for backprop networks such as implicit formulation (treating NN as a dynamical system) that improves adversarial robustness without adversarial training. Moreover, from a practical usability standpoint, even though SoftHebb is faster, the accuracy still trails the backprop-based networks. 


main_review: 
* The equivalency between mixture model and WTA has been proposed in a previous work (Moraitis et al. 2020) so the novelty on that end is quite limited.

* There are several other Hebbian and non-Hebbian learning rules that have been used in other works. Will any of them be adoptable in this framework?

* The comparisons are only made with 1 or 2-layer MLP and the accuracy differences are not significant. It is not clear what the advantage of softHebb is. The performance seems to be poor on the F-MNIST dataset. Discussion and experiments on learning with larger/more complex datasets need to be addressed.

* The adversarial robustness results seem promising compared to MLP, but a comparison is need with other adversarial learning/robustness techniques to assess the utility.

summary_of_the_review: The approach is promising, but is limited in terms of novelty, experiments and the comparison with existing approaches that makes it difficult to assess the true potential of this approach.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors develop a theoretical framework to incorporate winner-takes-all (WTA) connectivity with Hebbian-like plasticity which translates into a Bayesian generative model that can be used with generic artificial neural network (ANN) elements. This implementation, which they call SoftHebb, marginally outperforms a competing WTA alternative in digit classification on the MNIST. Furthermore, it also trains marginally faster than a standard backprop multilayer perceptron (MLP), though its performance saturates at a lower accuracy. Finally, the model appears to be substantially more robust than the standard MLP for both black-box and white-box attacks.
main_review: # UPDATED SCORE
* Score was updated to reflect the changes made by the authors during the rebuttal phase: 5-> 6

# Strengths
* Biologically-inspired Bayesian generative neural network that combines Hebbian learning with winner-takes-all connectivity
* Apparent improvement in robustness both to white-box adversarial attacks (PGD) and black-box attacks (Gaussian Noise)

# Weaknesses
* Model is evaluated only on MNIST and Fashion-MNIST datasets and is not clear whether it can be expanded to more challenging tasks
* Details on the adversarial attacks are missing
* Improvements other than those related to robustness are marginal at best and some controls are missing
* Detailed explanation on the model implementation is missing


# Detailed explanation of criticism
## MNIST and Fashion-MNIST only
The authors implement a SoftHebb model for the MNIST and Fashion-MNIST datasets. While these are interesting toy datasets, they are very simple to solve and solutions to them not always generalize to more challenging problems closer to real-world applications. In that sense the authors should do a better job at pointing a direction for future model development. For example, the authors claim that the number of true causes for the data, which in their single-layer implementation translates to number of neurons, can be chosen using common heuristics from cluster analysis. Can the authors indicate what this number would be for the CIFAR-10, TinyImageNet or even ImageNet datasets? How can this framework be adapted for a multilayer network? And can it be used in conjunction with convolutional layers which are a key component of neural networks for visual tasks? If not, wouldn’t it make more sense to demonstrate the use of this framework in a non-visual problem?

## Details on adversarial attacks are missing
Very interestingly, the SoftHebb model is very robust to both white-box and black-box attacks. While the robustness to Gaussian noise is undisputed (though it would be interesting to evaluate robustness to other types of random noise), the same cannot be said about the PGD adversarial attacks. When claiming improvements in robustness to gradient-based attacks, the burden of proving that the attacks have been properly performed is on the proponents of the defense. Unfortunately, in the current form of the paper, it is impossible to evaluate if the attack has been properly implemented. PGD has multiple hyperparameters, such as the number of the attack steps and the attack step size, that need to be optimized for different models separately. The authors make no reference to the choice of these hyperparameters and whether they tested different combinations to ensure that the attack was optimal. A bare minimum would be to show attack iteration curves for different attack step sizes for an intermediate perturbation strength. Also, it has been shown that the activation function greatly affects adversarial robustness (Xie et al 2021), does the standard MLP use the same activation function of the SoftHebb? It is possible that the improvements in robustness in SoftHebb are due to more trivial causes and do not depend necessarily on the proposed framework. 

## Marginal improvements
The performance gains of the SoftHebb over the hard WTA model are marginal at best as well as the gains after 1 epoch over the standard backprop model. In the paper, I think a few visualizations are missing. For example, the authors should include the backprop model on Figure 1A and the hard WTA model on Figure 1B,C,D and Figure 2. This way, it gives the impression that the authors are cherry picking the baselines to compare based on the benchmark. While the authors make it clear in the main text that the backprop model outperforms their model in 100 epochs, comparisons between the SoftHebb and the hard WTA model for the remaining benchmarks are missing. How much better is the SoftHebb model over the hard WTA in terms of robustness?

## Improving model explanation
Theory could be better introduced/contextualized for example by illustrating how the different terms relate to the components of the ANNs. Also, the model description should be expanded either in the main text or supplementary. Diagram of the connectivity, formula for the neurons activation function as implemented in the code, and an algorithmic description of the model training would greatly improve understanding. Also, details on computational times are missing. While the authors focus their analyses on the number of epochs and training examples, it is not clear whether the training times and inference times of the different models is the same or deviate considerably, as well as the memory requirements to train/run each model.

## Minor points
* The authors point the dependence on feedback by standard ANNs as a criticism. However, there is a high degree of feedback in the brain. I suggest that the authors soften this point.

summary_of_the_review: This study is definitely an interesting take on incorporating biologically-plausible components in an ANN design. However, the proposed model contains several limitations in terms of its expansibility to more challenging problems and some controls are missing to ensure the validity of the main claims, particularly those related to the robustness of the model. For these reasons, I cannot suggest accepting the paper in the current, but am willing to update the score if the authors address my concerns.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors pursue a formalization of WTA networks as optimization. In the paper, they show that WTA networks that compete via soft-max (approximating possibly plausible biological lateral inhibition) and are updated by Hebbian learning are implicitly minimizing the cross-entropy between inputs and layer activation.

main_review: Strengths:
- The formalization of (a type of) competitive Hebbian learning from an optimization perspective is interesting and useful.
- The proposed method does seem to perform well.
- The experimental investigation on the effects of noise and adversarial attacks in the proposed method vs standard gradient-based methods is very nice.

Weaknesses:
- Evaluation is a bit weak (more below).
- Evaluation is performed only on two simple datasets, MNIST and Fashion-MNIST.
- The models evaluated only have 1 or 2 layers, which are extremely simplistic and limited. It is also not clear whether and how well the method would scale when a deep architecture is used.

summary_of_the_review: The paper has both valid strengths and weaknesses. The paper could be a clear 'accept' for me if the authors would at least present results with more standard MNIST architectures, like conv+pool layers and/or 3-4+ layers. Additionally, it would be nice to see results of the method at least on CIFAR-10.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper wants to provide an optimization theory for WTA networks that it claims to be missing. More precisely they consider soft WTA networks that are implemented as ANNs.They apply their model to MNIST and Fashion-MNIST. In addition some robustness to adversarial attacks is demonstrated.
main_review: The paper mentions prior work in the same direction, such as (Nessler et al., 2009), but claims that a Bayesian theory for WTA learning has been missing. This claim is not correct.
The incomplete discussion of the state-of-the art makes it hard to identify the innovations of this paper.

Although the paper states that is aims at biological plausibility, its synaptic plasticity rule (8) is not local: the rule for adapting the weight from neuron i to neuron k requires knowledge of the current weights of all other synapses to the same postsynaptic neuron k (via the term u_k, see (4)).

A nice aspect of this paper is that it includes empirical studies of the noise robustness of the model. But it only compares the robustness of their model with that of MLPs, rather than with other unsupervised learning approaches.
..................
Some of these points have been improved in the update.


summary_of_the_review: The paper addresses an interesting topic. But it is not clear what its innovations are, and whether it can be implemented with a local learning rule.
review_title: null

Meta-review: The authors provide an analysis of soft-winner-take-all (WTA) networks with Hebbian local learning as a generative probabilistic mixture model. They then present experiments on comparably simple data sets, MNIST and F-MNIST. Results are compared to hard WTA networks and an MLP of the same size (single hidden layer) trained with backprop. Besides accuracy, the learning speed and adversarial robustness of the networks are compared.

This paper is borderline, and I was discussing it quite a bit with the reviewers.
The reviewers agree that the manuscript has some merits, but they also point to a number of weak points.

Besides the objective evaluation, I would like to comment on the review dynamics of this paper. The paper had initially rather low ratings. The authors commented extensively on the reviews, in several waves, and with suggestive text such as "All reviewer's points addressed" (as a comment title) or "Based on the Reviewer's earlier comment, the revised paper is now a clear 'accept'." to name just a few. I and the reviewers had the impression that the authors strongly urged the reviewers to increase their scores.

Due to the borderline ratings, I decided to read the paper carefully. My impression is in-line with the main criticisms of the reviewers, and summarized in the following:
On the positive side:
- The manuscript tackles an interesting problem. WTA architectures are biologically highly relevant structures and it is relevant to study learning in them.
- The authors provide a nice theoretical analysis.
- The observation that WTA architectures improve adversarial robustness is very interesting.
- Learning is local.
- The manuscript is well-written.

On the negative side:
- Theory: Similar analyses have been performed before. While there are differences, the main ideas are rather similar, in particular with respect to (Nessler et al., 2009). The authors argue that in contrast to their work, Nessler et al. 2009 deals with spiking neurons. But since the authors argue with biological plausibility, I would see that as an advantage of Nessler et al.
- Performance: The performance of their model is comparable to the standard hard WTA network, often showing only a very slight advantage. This raises the question why the soft WTA should be preferred over the hard WTA. The performance of the single-hidden-layer ANN is clearly better. This raises the question of the scalability of the approach.
- The analysis of adversarial robustness is interesting, but there is no comparison to other defense methods (e.g. adversarial training). The authors argued in their comments that it is not an adv. defense paper, so this comparison is out of scope. This reasoning is understandable, but since this is maybe the most interesting point of the paper, it would be a nice to have.
- Scalability: It is true that the learning is local, but the question is whether it scales to larger problems and deeper networks. After the first reviews, the authors added experiments on CIFAR-10 and a convolutional version of the model. However, the results were clearly below the state-of-the-art and the convolutional model is barely described (5 lines in the appendix).

Conclusion: The manuscript has some interesting points. Given the the strong competition within ICLR however, I cannot propose acceptance.
Decision: Reject