Paper: Learning Sequence Attractors in Recurrent Networks with Hidden Neurons (ICLR-2024)

Reviewer comment (Experiments):
The authors don't make considerable efforts to compare their work to previous ones relating to sequence encoding. There are many works on sequence learning, see for example (Lipton, 2015). Your work is a particular subclass of networks that can perform sequence-to-sequence processing (namely where you have a sequence with a single input and the start and zero inputs afterwards). Make comparison with other methods to measure the performance of your framework in terms of robustness to noise, memory capacity (number of recallable pattern sequences), etc.

Author response:
* **Comparison of encoding efficiency to other methods/experiments**  
    In the revised paper, we have compared our approach with networks of continuous neurons trained with backpropagation and modern Hopfield networks. Please see Appendix A and B in the revised paper.

Reviewer comment (Experiments):
The demonstration contribution of this framework could benefit from some additional experiments. Experiments for effect of noise level of the first pattern on the performance.

Author response:
* **Experiments for effect of noise level of the first patter on the performance.**  
    We observed a trade-off between the noise level that can be tolerated and the length of the stored sequence in our experiments. Additionally, we provide the code of our experiments in the supplementary material for further exploration.

Reviewer comment (Experiments):
The paper doesn't address how inputs to the network influence the output or performance. Experiments that track performance of the networks as a function of injected noise during sequence generation would better demonstrate the usefulness of this framework for neuroscience.

Author response:
* **Further, the paper doesn't address how inputs to the network influence the output or performance.**  
    We agree that the inputs are important. However, in this paper, we focus on spontaneous neural activities and sequence memory for which the inputs are absent. The exploration of inputs to the network is beyond the scope of this paper.

Reviewer comment (Evaluation):
Overall, the information in the figures is very low. For Figure 4 and 11 for example, it is unclear what is to be gained from seeing the convergence of the algorithm, if it is not compared to other algorithms to see which converges in fewer epochs.

Author response:
* **For Figure 4 and 11 for example it is unclear what is to be gained from seeing the convergence of the algorithm, if it is not compared to other algorithms to see which converges in less epochs for example.**  
    Figure 4 is to demonstrate that why the algorithm works despite that $\mathbf{P}$ is a random matrix and fixed during learning. Figure 11 is to demonstrate that both errors $\mu_i(t)$ and $\nu_j(t)$ would jointly reduce to zero, rather than conflicting with each other. Additionally, we provide comparison experiments in Figure 12 in Appendix A.

Reviewer comment (Experiments):
For Figure 7 and 8 it would be beneficial to see how increasing $T$ influences retrieval success as $M$ and $N$ are changed.

Author response:
* **For Figure 7 and 8 it would be beneficial to see how increasing  influences retrieval success as and are changed.**  
    Due to space limitation, we cannot provide all possible experiments suggested by the reviewer in the paper, as the number of combinations of the hyperparameters is enormous. Nevertheless, we provide the code of our experiments in the supplementary material for further exploration.

Reviewer comment (Writing):
The section on the robustness hyperparameter is unclear. Better describe how the margin of the margin perceptron is related to the robustness here.

Author response:
* **Better describe how the margin of the margin perceptron is related to the robustness here.**  
    In margin perceptron, the margin is disproportional to the magntitude of the weights, which is related to robustness, as can be been in eq (12) in Proposition 4. We improved the presentation in the revised paper.

Reviewer comment (Theory):
Relating to the derivations: Why does such a $U^\star$ exist? After Equation (19): "due to (1)". Should this be "due to (17)"?

Author response:
* **Why does such a** $\mathbf{U}^*$ **exist?**  
    The assumption that such a $\mathbf{U}^*$ exists is a condition that the algorithm converges in finite steps. This condition is also in convergence proof of the perceptron learning algorithm.  

* **After Equation (19): "due to (1)". Should this be "due to (17)"?**  
    We corrected this typo in the revised paper.

Reviewer comment (Writing):
There are a couple of unintroduced variables/notation. In Eq. (33) $q$ is appearing without introduction. Should this be $p$? Clarify step (32) to (33) in the derivation. In Eq. (39) $\Omega$ has not been introduced. Did you mean $O$?

Author response:
* **In Eq. (33) $q$ is appearing without introduction**  
    $q$ is an index which ranges from $1$ to $p$. It should be clear from the context.

* **Clarify step (32) to (33) in the derivation.**  
    We improved our writing of the derivation in the revised paper.

* **In Eq. (39) $\Omega$ has not been introduced.**  
    $\Omega$ (Big-Omega) is a standard notation in for analyzing complexity of algorithms. It is defined as the asymptotic lower bounds.

Reviewer comment (Theory):
The justification of the relevance of this work to V1 neurons is insufficient. The authors should justify why this particular activation function is a good model for V1 neural activity. Furthermore, the claim about unexplained V1 neural activity relies on the limit of the Heaviside activation function, for other activation functions hidden neurons would not be necessary.

Author response:
* **The justification of the relevance of this work to V1 neurons is insufficient.**  
    The reviewer made good points on our speculation on V1. After reconsideration, we found our speculation on V1 is too premature and therefore removed it in the revised paper.

Reviewer comment (Theory):
For V1 neurons in particular, but for all subnetworks in the brain really, inputs are a very important contribution to the dynamics.

Author response:
We agree with the reviewer that external inputs are crucial in neural information processing. However, it is beyond the scope of this paper. We have acknowledged this limitation in the Conclusion and Discussion section of the revised paper and plan to explore this important topic in the future work.

Reviewer comment (Theory):
What is the expressivity of the Heaviside function for a fixed $M$ and $N$ in terms of $T$?

Author response:
* **What is the expressivity of the Heaviside function for a fixed $M$ and $N$ in terms of $T$?**  
    In the revised paper, we derived the lower bound of the capacity of our network model. The maximal length of sequences that can be stored is at least $M$ (number of hidden neurons).  Please see the updated Proposition 1 in the revised paper.

Reviewer comment (Theory):
On page 8, just above Figure 4, when is $M$ large enough? When is the solution the pseudo-inverse and when the transpose of $\mathbf{P}$? How can high-dimensional probability theory explain this?

Author response:
* **On page 8, just above Figure 4, when is $M$
 large enough?**  
 Empirically, we found when $M$ is slightly larger than the sequence length $T$, the network can learn to store the sequence. Interestingly, we also found that larger $M$ leads to faster convergence in learning.

* **When is the solution the pseudo-inverse and when the transpose of $P$?**  
 Both works fine when $M$ is large enough as shown in Figure 4.

* **How can high-dimensional probability theory explain this?**  
The phenomenon that $\mathbf{x} = \text{sign}(\mathbf{V} \text{sign}(\mathbf{Px}))$ for $\mathbf{V}=\mathbf{P}^\top$ or $\mathbf{V}=\mathbf{P}^+$ only happens when $M$ is sufficiently large. We cannot prove it at the moment but offer our intuition and conjecture for the interested reader to explore further.

Reviewer comment (Novelty):
From page 1: “the Hopfield model and related works typically only consider static attractors” — No, there exist many works looking at non-static attractors. Some of these are cited at the end of the first sentence in section 2 on page 2. A few additional examples include: H. Gutfreund and M. Mezard. Processing of temporal sequences in neural networks. Phys. Rev. Lett., 61:235–238 1998. Arjun Karuvally, Terrence Sejnowski, and Hava T Siegelmann. General sequential episodic memory model, ICML 2023. Hamza Chaudhry, Jacob Zavatone-Veth, Dmitry Krotov, Cengiz Pehlevan, Long Sequence Hopfield Memory, NeurIPS 2023.

Author response:
**No, there exist many works looking at non-static attractors.**  
    What we meant is most previous works on attractor networks of Hopfield type considered only static attractors. Although there are also some other works on sequence attractors as the reviewer pointed out and we cited, the amount of them is smaller compared to those on static attractors. We have clarified this in the revised paper.

Reviewer comment (Experiments):
Previous work on encoding sequences in attractor networks have taken many different approaches. Additionally, there exist many methods for learning sequences in the machine learning literature. There is a lack of comparison with these prior methods.

Author response:
**There is a lack of comparison with these prior methods.**  
In the revised paper, we have compared our approach with networks of continuous neurons trained with backpropagation and modern Hopfield networks. Please see Appendix A and B in the revised paper.

Reviewer comment (Theory):
Where is the evidence that the examples shown in Figure 2 cannot be generated by a network without hidden neurons?

Author response:
**Where is the evidence that the examples shown in Figure 2 cannot be generated by a network without hidden neurons?**  
"In Figure 2, we show additional examples of
sequences that cannot be generated by the network. The sequences are synthetically constructed. We
then test if the perceptron learning algorithm can learn the sequences. Since the algorithm converges
if the linear separability condition is met [Minsky \& Papert, 1969], the divergence of the algorithm
implies that the sequences cannot be generated by the networks." This is stated in the text above Figure 2.

Reviewer comment (Novelty):
How is your approach conceptually different to hierarchical attractor network architectures? E.g., Dmitry Krotov, Hierarchical Associative Memory, arXiv:2107.06446; Kunihiko Fukushima, A hierarchical neural network model for associative memory, Biological Cybernetics volume 50, pages105–113 (1984).

Author response:
**How is your approach conceptually different to hierarchical attractor network architectures?**  
We do not claim our approach is conceptually different to hierarchical attractor network architectures such as [Chaudhry et al, NeurIPS 2023] for sequence learning. However, our model and algorithm are more biologically plausible, as stated in Appendix B in the revised paper.

Reviewer comment (Theory):
What is the memory capacity of this network? How does this depend on memory load?

Author response:
**What is the memory capacity of this network? How does this depend on memory load?**  
In the revised paper, we derived the lower bound of the capacity of our network model. The maximal length of sequences that can be stored is at least $M$ (number of hidden neurons).  Please see the updated Proposition 1 in the revised paper.

Reviewer comment (Evaluation):
In the second paragraph of the conclusion, there is some speculation that neurons in V1 with unknown function may be akin to the hidden neurons of this model. How would a neuroscientist test for this? How should the tuning properties be studied, i.e., what should be measured? Does there exist some structure(s) in the hidden neuron activity data from your own model which you would expect in the aforementioned V1 neurons?

Author response:
**In the second paragraph of the conclusion, there is some speculation that neurons in V1 with unknown function may be akin to the hidden neurons of this model.**  
The reviewer raised good questions for us to reflect on our speculation. After reconsideration, we found our speculation on V1 is too premature and therefore removed it in the revised paper.

Reviewer comment (Theory):
A discussion of the capacity of the proposed architecture is missing, and is pretty important to be able to meaningfully connect this approach with biology and apply it in machine learning. This, in my opinion, is the biggest weakness of the paper.

Author response:
In the revised paper, we derived the lower bound of the capacity of our network model. The maximal length of sequences that can be stored is at least $M$ (number of hidden neurons).  Please see the updated Proposition 1 in the revised paper.

Reviewer comment (Reproducibility):
The experiments section does not have sufficient details about hyper-parameters ($\eta, \tau$).

Author response:
We have tested different hyper-parameters and found the results are consistent. Meanwhile, we provide the code in the supplementary material for reproduction of our experiments.

Reviewer comment (Reproducibility):
How many times is each sequence presented to the model?

Author response:
In training, we ran our algorithm for maximal 500 epochs. Each epoch correspond to presenting the sequences to the model once. Empirically, we found the algorithm usually converges within 100 epochs.

Reviewer comment (Experiments):
What's the capacity of the model? How many sequences can it store?

Author response:
The maximal length of an arbitrary sequence that can be stored in the model is at least $M$ (number of hidden neurons). Please see the updated Proposition 1 in the revised paper. The model can store as many sequences as possible as long as the total length of the sequences is within the model capacity.

Reviewer comment (Writing):
The sign function and the Heaviside function seem to be identical the way it's been defined in the paper.

Author response:
Both functions are needed. The sign function is to simplify the math such that one does not have to explicitly convert the values of network states from $\{0,1\}$ to $\{-1,1\}$, as in the classical Hopfield networks. The Heaviside function is needed as $\mu_i(t)$ and $\nu_j(t)$ are errors which need to be non-negative.

Reviewer comment (Presentation):
The first sentence of Related work just lists a bunch of papers, which seems redundant, since these papers are explained later anyway.

Author response:
We modified the writing as you suggested in the revised paper.

Reviewer comment (Presentation):
Would have been useful to have Fig. 2 and Fig. 5 side by side for comparison. Merge the two figures perhaps?

Author response:
We considered this suggestion but found merging two figures would break the logic line of our presentation for the first-time readers.

Reviewer comment (Presentation):
Bar plots in Fig. 7 and 8 are hard to read. Having concrete values for each bar, and mentioning the specific values of $M$ and $T$ used would be very useful.

Author response:
We considered this suggestion but found adding specific values in the figure would make the figures visually messy.

Reviewer comment (Writing):
There are many more grammar and language issues throughout the paper. It is suggested to pass the text through a grammar check.

Author response:
We thank the reviewer for the writing suggestions and incorporated them in the revised paper. We will polish our paper further by a large language model later.