Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper discusses learning in biological networks, and proposes a model that combines spiking networks, local connections (convolutions without weight sharing) and reward-modulated spike-timing dependent plasticity. All three components are chosen to be more realistic compared to artificial neural networks. The model is tested on MNIST and on a classical conditioning task. 
main_review: ### Major comments 

**Strengths.** The paper aims at combining multiple biologically plausible features of deep networks: spikes, local connections and RL-based error signals. This is a reasonable direction, and it’s been tried by multiple papers as shown in Tab. 3. 

**Weaknesses.**

However, I struggle to see the novelty in the author’s approach: spikes and local connections alone have been tried many times (Tab.3 and also [1]). Training the output layer (rather than the whole network) with an RL-based rule is somewhat new, but I find this approach unreasonable for the following reasons:
1. The last layer is usually trained with SGD + cross-entropy to assess the quality of representations built by previous layers. So the performance of R-STDP in any case would be limited by the representations it gets from earlier layers, which are arguably more important for training networks. (This paper tries to do that too with SVM, however.) 
2. There’s no reason for this approach to scale beyond MNIST, as the hardest part of training is done by a simple STDP rule. Maybe some layer-wise R-STDP can be a valid approach (akin to [2]), or a backprop-like RL error [3].

As a side point, I couldn’t run the code in colab (with PyTorch 1.8 and Bindsnet installed). Running `image_classification_experiment.py` gives `record() got an unexpected keyword argument 'n_labels'`. Disabling recording makes it go away, but then there’s a shape mismatch. And if you make the running time 256 instead of 256*3 to fix it, the accuracy doesn't improve at all.
 
The main result of the paper -- MNIST accuracy (Tab. 3) -- is very weak. It’s pretty straightforward to achieve 95%+ test accuracy with spiking networks, local connections and unsupervised pre-training (using SGD for predictions) [1] (Tab. 2 there). Therefore, even ignoring the potential weakness of the R-STDP in the final layer and concentrating on the STDP + SVM result (87.5%), it is clear that the network does not learn useful representations. There are multiple potential reasons for that:
1. The STDP in the first layer is at fault, which would be a bit surprising given the clarity of filters in Fig.3A. As a sanity check, you can train an SVM on the hidden layer without any pre-training, and see if it improves the results. 
2. The decoding scheme is ill-fitted for SVM. I’d suggest using SGD with cross-entropy like in [1] and probably many papers in Tab. 3 of your paper. If you see a large improvement, then R-STDP needs some rethinking to properly make use of the pre-trained layer.
3. Local connections make it harder. Some works in Tab. 2 of [1] successfully use LC layers, however. I would test performance with the same architecture, but using convolutions. Another thing I noticed is really large filters -- 15x15 filters for a 28x28 image are not too far from a fully connected layer.   
4. When the winner-take-all in the LC layer makes a mistake by activating the wrong “digit” (and the filter weights do look like digits in Fig.3A), the readout layer can’t fix it. 

Finding the root of poor performance would improve the paper, but the overall approach (hidden layer STDP + last layer R-STDP) is still unlikely to scale to harder problems and deeper networks.

**Recommendation**

Due to limited novelty and unsatisfying results, I would recommend rejecting the paper. 

### Minor comments

> It is noteworthy that in many learning problems, we do not have direct access to the explicit label of the data. Consequently, we may need to abandon gradient-based methods, and utilize reinforcement and reward-modulated learning rules

Gradient-based doesn’t mean it uses labels. See VAEs, self-supervised learning, etc. that all use backprop. 

> The proposed network is … the first locally-connected SNN with a hidden layer

That’s not true. See [1] and references therein.

Various problems:
1. Eq. 3 has extra underscores. It has to be dg/dt.

1. P_ij^+- in Eqs. 7-8 only need one index, j for Eq. 7 and i for Eq. 8.

1. Eq. 12 is confusing. Where does the reward come from at each trial? Is one of the r_i taken from Eq. 11?

1. Explaining the network model in Sec. 4.2 with equations would greatly improve clarity.

[1] https://www.sciencedirect.com/science/article/pii/S0893608019301741

[2] https://www.frontiersin.org/articles/10.3389/fnins.2018.00608/full

[3] https://proceedings.neurips.cc/paper/2020/hash/1abb1e1ea5f481b589da52303b091cbb-Abstract.html

summary_of_the_review: The paper presents a somewhat novel combination of biologically plausible features of deep nets. However, the overall approach is too simple to scale to hard tasks (STDP in hiddden layers + R-STDP in the last layer), and it performs poorly even on MNIST. The paper should be rejected.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors proposed an SNN model, dubbed BioLCNet, for image classification. The proposed network consists of three layers: 1) input layer, accepting spike trains as input;  2) locally connected (LC) hidden layer; 3) decoding fully connected (FC) output layer. The LC layer is for feature extraction and is trained first using STDP in an unsupervised manner. The decoding output FC layer is trained second with R-STDP in a semi-supervised manner. The proposed SNN is validated with the MNIST dataset to show its effectiveness. 
main_review: **Strengths:**
The manuscript proposed a training scheme mixed between STDP and R-STDP and proposed reward functions for training the 3-layer SNN with R-STDP. I think it is unique. However, unique does not mean good or better. 

**Weaknesses:**
1. The authors proposed a dedicated 3-layer SNN network for image classification. The validation is only limited to MNIST but with worse performance than other wildly used SNN approaches. From my point of view, it is too limited and it is not convincing enough in the effectiveness of the proposed method.
2. The authors claimed, “The proposed network is the first to employ RL … for image recognition”. Even it is true, why the community should care? From my point of view, the motivation for utilizing RL for classification is not very clear, especially since the performance is worse than other commonly used SNNs. 
3. It is not clear to me why the authors mixed STDP and R-STDP for training the network? I think it would make sense to show the performance with pure STDP and pure R-STDP training. 
4. The last sentence of the first paragraph of the introduction, “Spiking neural networks also require fewer labeled data….”, does not make sense to me. 
     * Why SNNs require fewer labeled data?
     * Fewer labeled data and operations can also be compatible with regular GPU, right?
5. I appreciate the authors tried to bring the bio-plausible concepts to the SNN domain. However, not all bio-plausible concepts revealed by biology/neuroscience research can be directly leveraged and meaningful for the SNN research as SNN neuron models themselves are just bio-inspired and they are super simple compared to the neurons in the nervous system. I do not think combining a few bio-plausible components together without clear insights could be a meaningful research project.  

summary_of_the_review: The main concern on my end is the proposed approach lacks clear motivation why the developed SNN network and corresponding training approach are meaningful. As an SNN researcher, I did not really see the significant value. The validation is only with MNIST, and I think it is too limited. 

But, my rating is not firmed at the moment. Based on the authors' feedback, if I realize I indeed miss something important, I am flexible to change my recommendation.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents a spiking neural network (SNN) architecture for
image classification, designed to be more biologically plausible than
comparable existing architectures. In particular, it eschews
convolutional connectivity in favour of local connectivity, and uses
STDP (vanilla and reward-modulated) instead of gradient-based
learning. The paper describes in detail the proposed architecture
(providing introductory material on topics such as spiking artificial
neurons and STDP), and then compares its performance on an MNIST
benchmark to other SNN architectures. Finally, the paper presents the
performance of the architecture on a conditioning task, which is used
to highlight the capacity of the network to track a dynamically
changing reward.

main_review: ## Strengths
- The paper is adequately written, and the research is motivated and
  presented clearly.
- To the best of my knowledge, the proposed architecture is a novel
  combination of existing approaches, and the reported results
  convincingly show that it achieves reasonable performance on the
  benchmark task.
- The problem of devising flexible architectures that can perform
  local (Hebbian) learning is one of significant theoretical and
  practical interest, especially with an eye to potential neuromorphic
  applications.

## Weaknesses
- The paper seems rather incremental, in the sense that its main
  innovations seem to be tweaks or variations of existing
  approaches. The Introduction section ends with this sentence: *"The
  proposed network is the first to employ reinforcement learning with
  Poisson rate-coded inputs for image recognition and the first
  locally-connected SNN with a hidden layer."* Even if taken at face
  value, being the first paper to "employ reinforcement learning with
  poisson rate-coded inputs for image recognition" does not seem such
  an impressive primate, if it means that there already exist papers
  where reinforcement learning with poisson rate-coded inputs is
  applied to a different domain, or where rate-coded inputs for image
  recognition are used with a different learning scheme. Moreover, a
  cursory search brought up https://doi.org/10.3389/fncom.2021.543872,
  where reinforcement learning and Poisson rate-coded inputs are
  applied to an MNIST classification benchmark. I haven't searched
  more, but this should suffice to show that the claim may not even be
  true (I'd like for the authors to comment on this). As for being the
  first locally-connected SNN with a hidden layer (again assuming that
  the claim is true), I'm similarly unsure how significant an
  achievement that is, if there already exist convolutional and
  fully-connected spiking networks with hidden layers, or
  locally-connected networks without hidden layers.
- Some of the arguments of the paper seem oddly fixated on the
  importance of developing spiking networks that are biologically
  plausible. For instance, on page 1: *"although convolution has shown
  great potential in solving any translation-invariant task, its use
  of weight sharing is biologically problematic"* . I agree that
  biological plausibility is important if the goal of the study is to
  learn something about biological vision, but this doesn't seem to be
  the goal of the present paper, which is instead more aligned with
  the interests of the computer vision field. In other words, several
  passages in the paper present biological plausibility as something
  desirable per se, but this is not a self-evident truth. Relatedly, the paper seems very
  quick to dismiss or ignore alternative (and very successful) methods
  for training SNNs, such as surrogate gradient-based ones (see for
  instance https://doi.org/10.1109/MSP.2019.2931595 and
  https://doi.org/10.1038/s41467-020-17236-y), because of their lack
  of biological plausibility (see for instance on page 2: *"the
  majority of works in this area use derivations of the Hebbian
  learning rule where changes in connection weights depend on the
  activities of the pre and post-synaptic neurons"* - is there a
  reference for this statement?).

## Minor points
- The introduction contains a few surprising and unsubstantiated
  statements that are likely to elicit surprise and confusion in the
  reader. For instance, *"in many learning problems, we do not have
  direct access to the explicit label of the data. Consequently, we
  may need to abandon gradient-based methods, and utilize
  reinforcement and reward-modulated learning rules"*: this seems
  completely gratuitous, and it is not clear how going gradient-free
  is a necessary implication of not having access to labels, as the
  wealth of existing gradient-based unsupervised learning methods
  shows.
- When discussing biological plausibility, I do not understand the
  insistence on the primate brain specifically. What is special about
  the primate brain that is relevant to the arguments made in this
  paper, and that does not apply, say, more generally to mammal or
  vertebrate brains?
- On page 3, please clarify in what sense *"Spike-timing-dependent
  plasticity is a type of biological Hebbian learning rule that is
  also aligned with human intuition"*. Is this supposed to imply that
  other types of biological hebbian learning rules are not "aligned
  with human intuition"? What does "aligned with human intuition"
  mean?

## Typos/clarity
- It would help clarity if the final paragraph of the discussion
  started with something like "In the future...", to signpost that
  further developments are now being discussed.
- Throughout the whole paper, please use the appropriate latex
  citation commands - I believe that in most cases you will want to
  use \citep instead of \cite or \citet. In the current state of the
  paper, many sentences are made needlessly hard to parse because the
  name of the authors of some reference is just inserted into the
  sentence without any parenthesis.
- page 1: "although convolution has shown" → "although convolutional
  networks.."
- page 2: "online on-cheap" training → "on-chip"
- page 2: "this differs with artificial neurons" → "differs from"
- page 3: "adapt the learning rule with the excitatory to
  inhibitory..." → "adapt the learning rule to"
- page 4, above eq 5: I believe "post-synaptic" neuron should be
  "pre-synaptic".
- page 4, below eq 5: "pre-syanptic"
- page 4, below eq 10: "can be considered a special CASE of R-STDP"


summary_of_the_review: This paper presents a novel combination of existing approaches. It is
in my opinion incremental in nature, but reasonably consistent in
achieving its goals. In its current state I can't recommend it for
acceptance, but I may revise my score if the confusing biological
plausibility angle is softened and if better connections are made with
related work, especially gradient based methods which at the moment
seem unjustly penalized.


---
Update after discussion: I am updating my recommendation to "marginally above the acceptance threshold" to reflect the effort made by the authors to engage at least with some of my concerns. See my message downthread for more details.
review_title: null

Meta-review: This paper presents a locally connected spiking neural network model trained to do classification of MNIST using spike-timing-dependent plasticity (STDP) and reward-modulated STDP. The authors show that this model can learn to classify MNIST images (though not at a very high accuracy) and that it can engage in classical conditioning. The reviews were initially all in the reject range. The common theme in the reviews was concerns about the weak and limited nature of the results. After a good amount of author response and reviewer replies to the authors, one reviewer increased their score to a borderline accept, but the other reviewers did not change their scores, producing scores of 3,3, and 6. Given these scores, and the reviewers' remaining concerns, a reject decision was reached.
Decision: Reject