Paper: Unsupervised Detection of Recurrent Patterns in Neural Recordings with Constrained Filters (ICLR-2024)

Reviewer comment (Experiments):
The point of unsupervised detection of neural patterns is to find them even when ground truth isn’t known; however, the method wasn’t applied to such data. While such an application couldn’t be used to test the accuracy of the method, it would help to illustrate qualitatively what can be expected from it in a realistic scenario and might provide an example of downstream use of the results.

Author response:
That is correct. The point of unsupervised detection is to find patterns for which the ground truth is unknown. However, before we can trust any such method, it must be shown to produce results that make sense when checked against known ground truth. This is why we focused on cases with known ground truth.

Reviewer comment (Writing):
Line 74: Can “repeating” be defined more clearly? What kinds of variations aside from independent jitter are expected biologically, if any?

Author response:
A “repeating” (which perhaps should be changed to “recurring”) pattern is one that appears more than a couple of times in the data. The occurrences of a pattern are not expected to be exact copies of each other (that is with same relative interspike intervals and number of participating neurons, in which case their detection would be much easier). Rather, the occurrences of a sequence are expected to differ due to spike timing jitter, dropout (that is when some neurons, otherwise part of a sequence, fail to fire) and noise (the presence of spikes from other neurons that are not part of the sequence).

Reviewer comment (Reproducibility):
What are the spike rates of the background activity?

Author response:
We assumed an average rate of 0.04 Hz based on the dataset from Rubin et al. (2019), which the original authors obtained by the thresholding Ca2+ imaging data of area CA1 of the hippocampus.

Reviewer comment (Evaluation):
Figure 3E: Why does it appear that the network learns nothing for 150 epochs and then suddenly converges? This seems inconsistent with the choice of 100 steps in section 4.4, particularly the claim of faster convergence in lines 195-197.

Author response:
This is because to be able to start detecting the sequences, the main objective (variance term in the loss function (Fig 3D)) needs to reach a certain level. In other words, as the filter optimization progresses, the filter becomes better “tuned” to the sequence, producing higher peaks when convolved with the data. Initially none of these peaks reach the statistical significance threshold (hence no true positives). At around 150 epochs a few peaks exceed the threshold and finally, at about 200 epochs, all the peaks reach significance.  

This is because of the different datasets: the one used in Fig 3E had 452 neurons and those used in the grid of runs in Section 4.4 had only 76 and 152 neurons. For those datasets, we observed that 100 epochs was sufficient for the $Var(x^{(k)})$ term in the loss function to reach a plateau (as we say on lines 195-197).

Reviewer comment (Writing):
Line 204: Is there really a 2D convolution operation? In the neuron dimension maybe you have a non-padded convolution with kernel size equal to input size, but it’s not standard to call that 2D.

Author response:
It is indeed a 2D convolution (we use the `torch.nn.Conv2D`). For each timestep, this convolution operation outputs one scalar. If we used 1D convolution, then we would have to use $N$ 1D kernels producing a vector of size $N$ at each timestep.

Reviewer comment (Evaluation):
Appendix B.4 & B.5: The comparison with PP-Seq is hard to interpret because both the true and false positive rates of PP-Seq are higher. Can you change a threshold to match one of these measures and compare the other?

Author response:
In our algorithm, we set the significance threshold to four standard deviations (PP-Seq does not have a similar setting). Given that these figures were drawn based on a grid of simulations, scanning different thresholds would take a very long time. If the Reviewer insists, we will do so.

Reviewer comment (Theory):
The dropout probabilities range from 0.2 to 0.4, and it is unclear how to relate that to spike statistics (e.g., Poisson or otherwise). Can this be clarified?

Author response:
Dropout was applied to vary the effective signal-to-noise ratio and test the method’s tolerance to noise. That is, the higher the dropout, the lower the SNR and the more difficult it is to detect a sequence. For simplicity, we did not specifically enforce any particular distribution of interspike intervals. However, to make the comparisons as fair as possible, all our synthetic datasets were constructed from the (permuted row- and columnwise) real CA1 dataset, which roughly preserves the original firing rates. This is mentioned on L144-147.

Reviewer comment (Evaluation):
Figure B.9: The sorted spike sequences look tighter here than in Figure 3. Are they? Why? Does it matter?

Author response:
Indeed, the sequences in B.9 look tighter than in Fig. 3, and this is this is expected. As we explained above, the raster rearranged with the filter(s) is not guaranteed to look identical to the ground truth due to sorting based on optimized filters. Also, while the results are mostly consistent from one optimization run to another, variations of the shape of the recovered sequences are expected unless the same seed for weight initialization is used. Finally, B.9 shows results for the second formulation of the method (in Fig. 3 we used the first formulation), and thus the difference is normal.

Reviewer comment (Presentation):
Figure B.12: This looks qualitatively quite different than Figure 7 and perhaps more should be said about this in the main text.

Author response:
The reason for the visibly imperfect recovery of sequences in B.12 is the same as we have explained in our answer to the previous question. If one is looking for repeated patterns, what matters ultimately is the presence of multiple significant peaks, while visual inspection of the rearranged neural data should be used as additional evidence in support of the presence of the peaks. We add a note about this on L103-105 of the updated manuscript.

Reviewer comment (Evaluation):
The learned kernels seem to include all the neurons, whether or not they participate in the sequence. Is it desirable to ignore non-participating neurons? Figure B.16 seems to suggest one way to do this, i.e., by checking for a Gaussian-like kernel. Are there better ways?

Author response:
That is a great question. Indeed, ignoring non-participating neurons might (although we have not tried it) help produce visually cleaner patterns after rearranging the neurons with the optimized filters. One way to achieve this is to introduce $N$ learnable parameters constrained between 0 and 1, per each filter. These would act as per-neuron weights reflecting how actively the corresponding neuron participates in a sequence to which the filter is tuned (with 0 for no participation and 1 for maximum participation).

Reviewer comment (Theory):
I am concerned about priors that may be “baked in” to the method (perhaps inadvertently). At minimum these priors should be made more explicit.

Author response:
***I am concerned about priors that may be “baked in” to the method (perhaps inadvertently). At mimimum these priors should be made more explicit. In particular, I'm concerned that the model seems to find straighter' sequences than ¬are present in the data (Fig 7).***

The sequences in Fig. 7 (panels C and D) show the spike raster rearranged with one of the two optimized filters. The objective only maximizes the detection strength, but imposes no prior on how the rearranged sequences will look in the raster. The seeming straightness or the sequences is an artifact of this (imperfect) rearrangement, not the effect of priors. To illustrate this, compare Fig. B. 12 and Fig. 7 (in the main text). Even though the datasets were exactly the same, the sequences look somewhat different. Unless the same seed is used to initialize all the trainable parameters of the model, the look and shape of the recovered sequences will differ from run to run. Most importantly, the main purpose of rearranging the rows of the spike raster is to help the researcher to visually confirm the presence of a sequence detected by convolution peaks exceeding the significance threshold.

Reviewer comment (Experiments):
In particular, I'm concerned that the model seems to find “straighter” sequences than are present in the data (Fig 7). The ground truth experiments all use linear sequences, exacerbating this concern.

Author response:
***The ground truth experiments all use linear sequences, exacerbating this concern.***

We assume that Reviewer refers to a "linear sequence” as a pattern where neurons (or ensembles of neurons) A, B, C, and D are activated in a strict predictable order (e. g. A → B → C → D, thus following a linear path). In that case, a "non-linear sequence” would involve a more complex pattern (e. g. a branching sequence like A → B → C or D). In fact, our approach detects such “non-linear” sequences without a problem if they fit entirely within the width of the filter ($M$). We illustrate this in a Supplementary Note, which we upload as Supplementary Material.

Reviewer comment (Experiments):
It's not obvious that the method can generalize to sequences (such as PBEs, theta etc) that unfold over more than 1 continuous dimension.

Author response:
***It's not obvious that the method can generalize to sequences that unfold over more than 1 continuous dimension.***

Thank you for the chance to clarify this. The method works for any pattern if it fits entirely within the filter’s temporal width ($M$). To substantiate this, in the Supplementary Note (that we have uploaded as a separate document) we consider an artificial 2D case in which, place cells tile a 2D enclosure, and this enclosure is traversed by an animal along two trajectories resulting in two sequences that unfold over more than 1 continuous dimension. The place cell neighboring in 2D space often end up far apart in the 1D space of the raster. However, even though this “locality” is broken, the method still works fine.

Reviewer comment (Theory):
Suppose we had a set of place cells that tile a 2-D enclosure. Would this method work? I'm concerned that the filters will have to cover a 2-D surface with piecewise 1-D filters and this will fail really badly.

Author response:
***Suppose we had a set of place cells that tile a 2-D enclosure. Would this method work? I'm concerned that the filters will have to cover a 2-D surface with piecewise 1-D filters and this will fail really badly***

On the contrary, our model uses 2D, not 1D, filters (as we explain on L72). This allows the detection of any (sufficiently strong) pattern as long as it fits entirely within the temporal width of the filter ($M$). We illustrate a 2D case in the Supplementary Note uploaded with the revised manuscript.

Reviewer comment (Evaluation):
Take the situation in Fig. 7. Suppose the animal starts out on the linear track at a constant velocity, stops half way through, backtracks for 10 cm, then turns around and continues along its original trajectory to the end. What filters does this method find? What should it identify?

Author response:
***Take the situation in Fig. 7. Suppose the animal starts out on the linear track at a constant velocity, stops half way through, backtracks for 10 cm, then turns around and continues along its original trajectory to the end. What filters does this method find? What should it identify?***

Thank you for the interesting question. If we assume that the animal runs back and forth most of the time and only backtracks once or twice, then the model is unlikely to learn a separate filter to capture this rare backtracking event, but will only learn two filters – one corresponding to the animal running in one direction and the other to it running in the other direction. Conversely, if the animal backtracks often, then the model is likely to learn a separate filter tuned to capture this zig-zag pattern. Finally, if the animal backtracks all (or almost all) the time, then the model will learn two filters (again, one for the forward direction and the other for the return trip).