Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents a hierarchical chunking model (HCM), which learns chunks from non-i.i.d. sequential data. The full learning process elicits a trace of building hierarchy from the ground up, similar to humans' perception and mind process, and hence human-interpretable. The proposed model is effective, adaptive, and successful in considered applications spanning from symbolic sequences to visual temporal data, from artificial settings to real-world problems like learning hierarchies of brain activation from resting-state fMRI data.
strengths_and_weaknesses: This paper is well written, whose ideas are easy to follow. The goal, motivation, and scope is clear; theory and algorithm parts are solid; experiments are well designed and conducted, spanning a diverse range of topics; experimental results are significant, which are further interpreted and discussed thoroughly. There are two things I would suggest the authors considering improving.

How the work presented in this paper is positioned among other related work in the literature is not clear to me until the final discussion section. However, comparisons were only discussed in general terms, largely focusing on the methodological perspective. If any of the techniques introduced in this paper are derived, influenced, or inspired by prior work in the similar field, could the authors include comparisons regarding performances on the considered experiments in the paper? The only experimental comparison in this paper seems to be with RNN (and human performance), which is methodologically quite different from the proposed HCM. In fact, I think comparing to RNN and showing HCM's superiority (e.g., as in Figures 2 and 3) is relatively weak, as HCM is especially built for the purpose of studying those internal sequential structures. For instance, the similar way how HCM and generative model in Sec 3 are built, should naturally make HCM work well for that type of generative model. So, to me, experimentally (rather than algorithmically) comparing HCM to more related other algorithms (like the ones mentioned in the discussion section) will be more convincing.

Figures are generally well designed, however, their layouts and arrangements are not that clean to me. For example, in Figure 1, it took me a while to figure out where the different subfigures are, and it seems to me there are overlaps between subfigures.

questions: 1. Is it possible to have ambiguity in the process of parsing larger and larger chunks, e.g., what if there are multiple largest probabilities in the transition matrix (not necessarily an exact tie, but close)? Would one choice of breaking the (approximate) tie greatly change the later process? The whole procedure seems conceptually similar to hierarchical clustering, which merges small parts in a greedy fashion. Then, is the final result going to be sensitive to some "noisy" decisions made earlier? Here, a "noisy" decision may be a choice of breaking the tie as mentioned above. But, in addition to that, can the authors also explain if the final result tends to be sensitive to a small phase shift (e.g., start reading from the second item instead of the very beginning of a sequence)? Is it sensitive to noisy observations?

2. In Figure 2a, it seems 0 is treated differently from other symbols, why so and is this kind of treatment always needed for empty observation?

3. This paper presents several cases where the model is capable of discovering known patterns. While these are necessary to show the model's efficacy when tested against ground truth, has the model also exhibited its ability to discovery new patterns, e.g., in fMRI data?
limitations: Limitations and their connections to related and future work were adequately addressed.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This work proposes a model that learns hierarchical chunks from non-i.i.d data in an unsupervised fashion. The central part of the model uses a statistical test to assess the dependence between consecutive items in a sequence. The authors use a sequence learning task to show that the model learns more similarly to humans than RNNs, and predicts human reaction times more strongly. This leads them to conclude that the HCM is more cognitively plausible than an RNN. They also show the performance of the model on visual sequences and resting state fMRI data, and illustrate the ability of the model to do online learning.
strengths_and_weaknesses: Edit: I thank the authors for their extensive replies to the reviews. They have addressed several concerns, particularly with their proposed  changes to summarize related work in reply to both me and reviewer TVcV. For now I have increased my score to a Weak Accept (6). This may change with any continued discussion.

The experiments are varied and illustrate different possible uses of the model. In particular, the extension of the HCM to spatiotemporal data and to online learning are both intriguing areas of future research. I also appreciate the pursuit of cognitively plausible models, the proof of recoverability, and the discussion of the importance of hierarchical representation learning in human intelligence.  The paper and method are also explained clearly, though some details about experimental choices are missing (even when referring to the supplement).

I do find the discussion of how this model relates to other previous approaches quite lacking. This makes the originality and significance of the proposal less clear, as some related work is missing. They write that "other types of hierarchical representations such as clustering or relational hierarchies are out of the scope for this work." Some clustering algorithms are certainly related and within scope. In NLP models, tokenizer methods such as Byte-Pair Encoding or WordPiece can be thought of as clustering approaches that, like chunking, facilitate compression of sequential data. Furthermore, it would be useful for the authors to discuss the relationship between their definition of a hierarchical structured sequence and a probabilistic context free grammar. They look quite similar to me, and much theoretical work has been done to study the ability of various models to capture different types of artificial languages (e.g. Hewitt et al. 2020, RNNs can generate bounded hierarchical languages with optimal memory). This discussion may even facilitate the authors' comparison of the HCM to other models.

Given this and some of the lacking details, I am rating the contribution of this paper as fair. If the ethical concerns are addressed, I would rate this as a 5 (borderline accept). I do think the framing of this work to connect it to the cognitive science literature on chunking is useful, and I am willing to increase this score if my concerns and questions are addressed.
questions: * What are the memory demands of the model? A formal analysis would be useful, given the vague claim that the HCM can handle "semi-high dimensional" data. This is also of interest because one of the uses for chunking may be in memory compression.
* How did you choose the size and architecture of the RNN for comparison with the HCM?
* The text suggests that the online HCM can be subject to memory decay to adapt to changing statistics. Is a significant decay / deletion threshold actually required for any of the present experiments?
* Supplemental Section G gives a hyperparameter for the HCM reaction time analysis (the probability of within chunk RT is 1 - 4/eps). Why was this value chosen? What happens when other values are chosen?
* What are the implications of actual value of the statistical threshold? Is it possible that different contexts may lead learners to have more strict or relaxed thresholds?

Suggestions (from major to minor)
* Dedicate a separate section to Related Work.
* Figure 1 does not define the variable d as depth.
* In the Supplemental Material E.1.3, Equations 23 & 24 appear to be identical.
limitations: Edit: the authors have addressed these items on the checklist.

The authors did not complete items 5a-5c on the checklist. I will rate this paper a reject as the ethical concerns are not completely addressed.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors proposed and tested a Hierarchical Chunking Model. This model discovers a hidden hierarchical structure while having a stream of state transitions as inputs. By processing a linear stream of one-dimensional information as transitions between sets of different lengths of sequences, the model starts to build a hierarchical structure. Human participants also use this chucking strategy. The hierarchical structure inferred from changes in the reaction times is closely related to the model-generated structure. Last, the authors provide various examples in which sequential information processing can get benefit from this human-like chunking strategy.
strengths_and_weaknesses: Strengths. By constructing a hierarchical structure, the model can efficiently learn a novel sequence if it was generated from the same structure and afford generalization. This hierarchical chunking model doesn't require a prior model structure but can be generated by the data itself. Interestingly, without knowing the outer stimuli causing these activations, this model could capture the meaningful neural networks associated with different cognitive states.

Potential weaknesses. The model only learns the sequence determined by the deterministic transition. This may capture better some types of sequences (where a temporal order matters). However, this can be more flexible if the model can learn the hidden structure generating a chunk of sequence. For example, if a sequence is generated by a random walk, the current model would increase the number of independent chunks, while it could be better to be captured by a certain network structure describing the states of each of the communities and how they are connected. 

questions: Is there any threshold determining which level of correlation is good enough to have a new chunk? And how the memory decay parameter and the deletion threshold are determined? I may miss how they were determined. Are they predetermined as fixed values? Or, these can also be determined by another hierarchical variable which tracks to what extent the environment is volatile. In this way, the model may be able to reduce the learning time.

 

limitations: The current model may not be able to generalize the structure learned from a sequence if the stimuli changes while keeping the same structure. (e.g., 12221212 -> 34443434).
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors give a statistical algorithm for hierarchical chunking of data tokens.  They prove that when their algorithm's structure matches an underlying data-generating process, it can and will learn the correct structure.  They then demonstrate the interpretability of the learned chunkings, with application to cognitive science.  They perform simple experiments with visual, temporal, and visuotemporal chunking, and demonstrate that an application of their algorithm to fMRI data finds chunks corresponding to known areas of the brain.
strengths_and_weaknesses: Strengths:
* Originality and clarity of the statistical justification for chunking
* Strong experiment comparing the HCM to an RNN for predictive convergence

Weakness:
* Incorrect use of p-values
* Too much of how the HCM actually works is in the appendix: pseudo-code, halting criteria, etc.
questions: When the authors write about the Gestalt principle of grouping by proximity, is this meant to reflect proximity in topographic neural maps, egocentric space, or allocentric space?

What is the big difference between learning a semi-Markov HMM (Hidden Markov model) or hierarchical HMM with a likelihood exploiting the Size Principle (https://www.sciencedirect.com/science/article/abs/pii/S0001691809001553) and the actual HCM?

Do the independence tests used in the HCM have a relation to an optimization criterion, such as total correlation or similar?  Why use p<0.05, since we generally want stronger guarantees in an automated system?

What is the beta-test statistic mentioned on line 178-179?

What did the authors mean by an "overlapping representation" in terms of switching environments at line 187?  Representations are typically a part of some internal model, not a part of the environment itself.
limitations: The authors have mostly addressed my questions and concerns, but nonetheless, using a false-positive rate of 5% when you expect to form more than 20 chunks means you expect there to be at least one false-positive chunk.
review_title: null

Meta-review: metareview: The paper presents a hierarchical chunking model, which builds meaningful and interpretable representations of larger units from sequential data. Effectively "parsing" the data.  The key to this model is that it more accurately predicts human response times than baselines (e.g. RNN in the original paper and PARSER in response).  The approach is also generalized to show performance on visual-spatial and fMRI domains (part-hole and sequential).  Independence testing and memory decay are the two key tunable parameters. 

Primary concerns by reviewers were around clarifications (addressed) and the introduction of a stronger baseline (PARSER -- addressed).
confidence: Certain
Decision: Accept