Venue: ICLR 2023 TinyPapers

Reviews:

final_score: rating: Great Start (GS): a submission which meets some of the reviewing criteria but has room for improvement
review_content: summary_of_contributions: Cell assemblies are units of neural computation that manifest themselves as repeating and temporally coordinated activity of neurons. In this paper, the authors propose using a graph neural network to embed spike data into a sequence of fixed-size vectors and clustering them based on their self-similarity across time. They validate their method on synthetic data and real neural recordings. 
strengths_and_weaknesses: # Strengths
1. This paper designs a method for cell assemblies detection based on graph neural network, which overcomes the problems of traditional methods such as template matching and non-negative matrix factorization.
2. This paper designs an unsupervised training goal, that is, to ensure the consistency of the same spike embedding in different time windows, and shows the effectiveness of their proposed method on two data sets.

# Weaknesses
1. It is not a novel idea to encode biological data into a graph and use graph neural network to mine the relationship on the graph. I think the author needs to further clarify the motivation.
2. As the author mentioned, this paper does not give a specific quantitative experimental measurement metric, and there is also a lack of comparative experiments with baseline methods.
3. Both datasets are relatively small in size, and it is doubtful whether the proposed method can perform the same on larger datasets.

suggested_changes: In general, I think this paper is a good start, and the author needs to comprehensively consider the points mentioned in the *Weaknesses* to make further improvements to the paper
review_title: null

final_score: rating: Clear, Correct, and Reproducible (CCR): a submission which meets the reviewing criteria
review_content: summary_of_contributions: This paper proposes a method of embedding spike data into clusters based on their self similarity across time. First a time-varying directed graph between neurons is constructed with edge weights based on how close one spike follows another. Then a GNN is applied at each time to (soft) assign each time window to a cluster. 
strengths_and_weaknesses: Context: I have little experience in this datatype, but significant experience in GNNs and clustering methods. Cool work! Thanks for the read. 

Strengths:

- The model specification was quite clear and reproducible.
- The losses seem reasonable.
- I think this is a cool idea, essentially this is a problem of clustering graph structures (with the same number of nodes). I haven’t seen this problem setup before. I’m unclear if a GNN is necessary here, but certainly an interesting idea to try.

Weaknesses:

- The results are qualitative up to this point.
- Its unclear from the exposition exactly why we want to cluster time periods of the neurons. “The model successfully disentangles repeating patterns in both the synthetic and real datasets”. This seems interesting, but it would be nice to add a sentence on why this is interesting / useful. I assume they correlate with the mouse doing something maybe?
- Why not include all of the neurons in the graph so that the graph size is fixed? (There may be some nodes with no connections but that is okay). This is one of the points I didn’t get.
suggested_changes: Suggested Changes:

- Add more context on why we want to cluster neuron activity patterns over time.
- I didn’t quite understand why things were updated at the beginning / end of each epoch and what effect this has. Perhaps this could be clarified in a sentence?

(Possibly) interesting ideas:

- You may want to add additional features (besides the neuron embeddings) to the input features of the GNN. Work on molecular graphs has found degree, and k-eigenvectors as useful additional features.
- Since the graph is directed, I would be surprised if GCN works very well as compared to more recent GNN methods. Ideas around the magnetic Laplacian could be of interest here.
- Obviously (as mentioned by the authors) more work in benchmarking this method would be interesting too.
review_title: null

final_score: rating: Great Start (GS): a submission which meets some of the reviewing criteria but has room for improvement
review_content: summary_of_contributions: This paper proposes a GNN-based unsupervised embedding method for the detection of cell assemblies in spike recordings.
strengths_and_weaknesses: I assess the submission with respect to each of the reviewing criteria below: 

- **Clarity**: The writing is generally clear. However, there is room to improve the description of the method, and the significance testing approach is not clearly motivated (see **Suggested Changes**). 

- **Correctness**: The experiments appear to be correctly performed, but are limited to a single dataset. Benchmarking the proposed algorithm against existing methods would make for a stronger paper. Admittedly, the authors explicitly state that they leave this for future work, but without such comparisons it is not clear whether the proposed method represents progress towards solving the motivating problem. See **Suggested Changes** for more details.

- **Reproducibility**: The experimental setup is described fairly clearly, but code is not made available. 

- **Follows basic requirements**: The paper fits within the length requirement, and is adequately anonymized. 
suggested_changes: **Major**:

1. Some of the design decisions should be more clearly motivated. For example, why is the time constant in the edge weighting set to $\tau = 0.04$? In what units is this measured? If the goal is to "emphasize the contribution of spikes that occur in close temporal proximity" (Lines 31-32), shouldn't this parameter be adjusted depending on the acquisition rate of the underlying recording (if it's measured in units of data samples) and on the relevant timescale over which activity in the area of interest evolves? 

2. I think it would be useful to more clearly motivate the GNN model. Some of the prior works cited (e.g., Williams et al.) build up their methods from clearly-defined statistical models. Though the present method is not set up in that way, it would be useful to motivate why graph encoding and GNN embedding is a sensible way to detect structure in spike data that would result from the existence of assemblies. In the submitted manuscript, the paper simply states the algorithm without motivating its design.

3. As mentioned above under **Correctness**, I find it hard to judge the usefulness of the method vis-à-vis the problem of assembly detection stated in the introduction without any explicit comparisons against previously-proposed methods. The experiment on place cell recordings in mice running on linear tracks is nice, but one can detect the sequential structure present there just from firing rates. In this case, also, the clustering shown in Figure 1 seems to artificially discretize the population into clusters that respond within a certain interval of the track. It would be useful to provide a direct comparison of the proposed method with at least one of the previously-proposed algorithms on a more complex dataset that contains more clearly discretized syllables, e.g. the zebra finch HVC data considered in Williams et al. and Mackevicius et al. 

4. The criterion for statistical significance of a pattern needs to be justified more rigorously. As described, it is as much an assessment of the robustness of model fitting as an assessment of characteristics of the pattern. At the very least, I would suggest that the authors benchmark this approach on toy synthetic data containing very clear clusters, to give a baseline measure of how variable the clustering is for noiseless data. Also, they should apply the method to null data containing no clusters, and show that this approach successfully fails to reject the hypothesis that there are no clusters present.

**Minor**:

1. In the top panel of Figure 2, the ordinate axis label is misspelled: "Instantion" -> "Instantiation."
review_title: null

Decision: Invite to archive