Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposed a hybrid neural network of SNN and ANN to solve the binding problem. This hybrid network on the one hand expands the representation space with additional temporal dimension, and on the other hand holds the computational power of ANNs. The effectiveness of the model is evaluated on several artificially generated datasets, showing its availability, though not concrete enough. The limitations of the current paper were well illustrated (in the appendix), however, the proposed model could be a significant step towards a broader work.
strengths_and_weaknesses: Strengths: The idea of combing SNN and ANN in binding problem is quite novel. The theory (although with idealized conditions) is built very solidly. The experiments show the advances of this combined model compared to pure ANN or SNN. 

Weakness: As mentioned above, the complexity of the implemented experiments are not significant enough. 

questions: The current model only deals with the binary images, which is quite compatible with the binary spike in the spike coding space (SCS), when this input extends to real-valued pixels, how would it influence the performance of this model, in particular, how would this influence the SCS (I guess you will then need an additional coding mechanism which will then violate the one-to-one correspondence described in line 107)?

Based on the paper, the SNNs provide an additional temporal domain by inducing the neuron dynamics, what is the function of the “spikes” in this model, i.e., what if we replace the spiking neuron models with real-valued dynamic neuron models, which still provide the temporal dimension but then remove the binary coding mechanism? 

limitations: please see above
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a neural network architecture for solving the visual binding problem. The proposed architecture is comprised of a single layer of spiking neurons and a denosing autoencoder. Importantly, the top-down feedback from the output of the autoencoder modulates the spike timing of the spiking neurons. The solution to the binding problem for every visual input is formed through the iterative top-down and bottom-up processing.

strengths_and_weaknesses: Strengths: I enjoyed reading this paper and very much liked the proposed idea. A combination of top-down modulation and spike timing is an intriguing idea as a solution to the binding problem. I believe that both neuroscience and AI communities would, in general, benefit from the publication of this paper as a proof of concept for the potential role of spike timing in binding different properties of an object.

Weakness: The paper could benefit a lot from some clarification in the method. Also, some of the figures need further explanation and editing. For example, it's not clear what the rows and columns are representing in the zoomed in views in figure 2c. Moreover, the details in some figures are not readable, e.g. in figure 3. I have a few concerns and comments that I elaborate below. 
questions: Major comments: 

1- It is not clear in the paper how the k-means clustering is applied to the pure-ANN model. For DASBE, as mentioned in the paper, the clustering is applied to the spike trains. Without the spiking output of the first layer, how is the clustering method implemented? Also, for sake of clarity, it would be helpful to include an example result of the pure-ANN-based clustering in the figures. Moreover, it seems that pure-ANN and folded-DAE are used interchangeably throughout the paper referring to the same type of model. I suggest that the authors use a single term to avoid confusion. 

2- In the formulation of DASBE, the output of the DAE decoder is used to module spike timing and it's called $\gamma$ in equation (4). In folded-DAE what seems to be the same variable is called X'. If they're referring to the same variable, and there is no difference between the two in the two models, I strongly suggest that the authors use the same notation in the two models to avoid any confusion. If there is any difference in how the output of the DAE decoder is used in the two models, it should be explained.

3- In section 4.2.4, the contrastive loss that is used for training the model is a *supervised* contrastive loss, as explained in the supplementary materials. Given that contrastive losses are mostly associated with self-supervised methods, it'd be more clear if the authors would clearly explain that in the case of this problem, a supervised contrastive loss is used to learn a disentangled representation of object shapes. Also, How does the results compare with the results of using only a reconstruction loss?

Minor comments: 

1- How are the number of clusters set in k-means? 

2- Figure 2b is too small and the text is not readable. It's not clear what the rows and columns and each square are showing in the zoomed-in views of figure 2c. 

3- In the last paragraph of page 6, pulse-coupled neural network (PCNN) is introduced, but it's referred to as pure-SNN in the rest of the paper. I suggest that the authors use only one term for referring to this model to avoid confusion.

4- In line 212, "It is interesting to note that the grouping information is continuous (Fig.2 d.g)" How are the plots in Figures 2 d-g related to what is stated in this sentence? 
limitations: Given that the authors are mostly relying on qualitative comparisons to present the performance of the model, it's important to also include cases where the model doesn't perform very well. Ideally, one would like to see some statistics on the number of cases in each dataset where the model doesn't outperform other approaches or even possibly performs worse. In the absence of such statistics, at least I need to see a comparison between successes and failures of the proposed model.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In the paper "Dance of SNN and ANN: Solving binding problem by combining spike timing and reconstructive attention" the authors propose a hybrid architecture consisting of a spiking neural networks and an attention map modulating denoising autoencoder to address the binding problem. They test their approach by encoding input images consisting of binary images and evaluate binding properties of the network. Notably, they observe gamma-like oscillations. Moreover, they prove four propositions pertaining to convergence of the network.
strengths_and_weaknesses: The paper describes an interesting idea to address the binding problem: the combination of a spiking neural network with a denoising autoencoder. To the best of my knowledge, this is a novel approach in the context of the binding problem.

However, the significance of the paper is lacking. The authors compare their approach to a pure-ANN and a pulse-coupled neural network (PCNN) as a pure-SNN, which do not perform well. However, in the PCNN the authors do not tune all hyperparameters (lines 206ff in the supporting information). Indeed, in the Discussion the authors note that their approach does not compete with a dedicated segmentation algorithm. Hence, significance to the machine learning community is clearly limited. On the other hand, with a denoising autoencoder as an ANN and missing biological ingredients such as STDP, relevance to the neuroscience community is limited too.

Regarding the proven propositions, the authors admit that their propositions do not guarantee convergence of their network in practice. Therefore, their propositions seem to be of limited value.

I could not find conceptual or technical mistakes in the main paper. I did not check the proofs in the supporting information.

The clarity of the main text is fine. The figure quality could be improved. The labels are often small and sometimes not legible (see Questions).

EDIT: The authors clarified some misunderstandings and I increased my score from 3 to 5.
questions: Line 50: The statement is too strong. Continuous does not necessarily imply representation of uncertainty.

What value was used for V_th?

Algorithm 1, line 2: misspelled Bernoulli.

Fig 2a: Why is the rate score approximately zero? When the units start firing, the rate should increase, shouldn't it?

Fig 2, panels a, b and h: The authors used an extremely tiny font size. It is not legible even when zooming in a bit.

Fig 3: This figure wastes a lot of whitespace and unnecessarily repeats the legend.

Table 1: It should be explained that the values are percentages. The main text describes values between 0 and 1.
limitations: The authors discuss potential societal impacts in the supporting information. The authors also discuss limitations with respect to missing biological components and competitiveness when compared to segregation methods.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this article, the authors propose to combine Artificial Neural Network (ANN) and Spiking Neural Network (SNN) to solve the binding problem. In their model the ANN is a Denoising Autoencoder (DAE) that has been trained to remove salt and pepper noise on images of single object. The SNN is a coincidence detector that is feeding the encoder of the DAE. In return, the decoder of the DAE is providing an attention map used by the SNN. This is defining a dynamical loop between SNN and ANN. The authors show that the combination of both (ANN and SNN) allows to solve the binding problem on 5 different datasets (Bars, Shapes, Corners, Multi-MNIST, MNIST+Shape).

strengths_and_weaknesses: The paper is clear and well written. To the best of my knowledge the type of model presented in this article is novel. Thus, this is a valuable contribution to the field. Another strength of the article lies in the method to evaluate the synchrony level (which is also a novelty)
 
The main weakness of this article is the lack of comparison with other related models. Even if the principle behind the presented model is novel, other models also claim to solve the binding problem (e.g. complex autoencoders in [1]...). 

Overall, I would tend to accept the article but it is borderline because there is no comparison with other algorithms solving a similar tasks (so it is hard to say if the proposed model is bringing a better binding ! )

[1] : Löwe, Sindy, et al. "Complex-Valued Autoencoders for Object Discovery." arXiv preprint arXiv:2204.02075 (2022).
questions: * The training procedure is never really detailed. I have understood that only the ANN was trained (as a DAE trained to remove salt and pepper noise in images of single objects), not the SNN. Is it correct ? If so could you explain if the ANN is trained separately ? Including a proper mathematical formulation of the loss function might improve the understanding of the training procedure

* In general, the value of the hyper-parameters are not detailed. Sometimes, I don't even know if they are parameters (updated through a learning process) or hyper-parameters (i.e. set by hand). For example, what about p in eq 3 (Is it learned ) ? I would suggest the authors to update their article to make this specific point clearer.

* In figure 3 (and in table 1), we observe that the performances are lower for Multi-MNIST (and MNIST + shape). This is not properly discussed in the article. 

* In section 5: [1] should be added in the reference (this is one of the most recent model that solve the binding problem)

* The proposed network should be compared to other state of the art networks. For exemple other complex auto encoder [1] are known to also solve the binding problem.

Minor:
Line 107 : N is not defined (I guess this is the size of the image, but it should be explicitly said)

Line 115: peroid —> period

Line 142 dynamcs —> dynamics

Line 209: On the contrast —> "In Contrast" or "On the contrary"  

Line 255 Equivarince —> Equivariance


[1] : Löwe, Sindy, et al. "Complex-Valued Autoencoders for Object Discovery." arXiv preprint arXiv:2204.02075 (2022).
limitations: The authors have properly discussed the limitations and the societal of their work in their article.
review_title: null

Meta-review: metareview: This paper presents a solution to the binding problem that uses a combination of spiking and reconstructive attention. The authors show how spike timing can be used to group sensory inputs into objects and then have attention alter the synchrony of firing to bind multiple objects. They demonstrate this on a variety of visual datasets including basic shapes and MNIST.

The reviews for this paper were borderline. The reviewers felt that the paper was novel, technically sound, and and well-written, but they were not completely convinced by the paper, in particular there were worries that the experiments did not provide adequate comparisons to other techniques to make the significance of this approach clear, nor did they test appropriately complex situations. Nonetheless, the authors were able to partially alleviate the reviewers concerns, and the final scores were 7,6,5,5. Given these scores, and the general agreement that the paper was technically sound and novel, an 'accept' decision was made.
confidence: Less certain
Decision: Accept