Original: Learning Structure from the Ground up—Hierarchical Representation Learning by Chunking
Revised: Learning Structure from the Ground up—Hierarchical Representation Learning by Chunking

Original: Abstract
Revised: Abstract

Original: From learning to play the piano to speaking a new language, reusing and recombining previously acquired representations enables us to master complex skills andeasily adapt to new environments.
Revised: From learning to play the piano to speaking a new language, reusing and recombining previously acquired representations enables us to master complex skills and easily adapt to new environments.

Original: Inspired by the Gestalt principle of grouping byproximity and theories of chunking in cognitive science, we propose a hierarchicalchunking model (HCM).
Revised: Inspired by the Gestalt principle of grouping by proximity and theories of chunking in cognitive science, we propose a hierarchical chunking model (HCM).

Original: HCM learns representations from non-i.i.d.
Revised: HCM learns representations from non-i.i.d.

Original: sequentialdata from the ground up by first discovering the minimal atomic sequential units aschunks.
Revised: sequential data from the ground up by first discovering the minimal atomic sequential units as chunks.

Original: As learning progresses, a hierarchy of chunk representations is acquiredby chunking previously learned representations into more complex representationsguided by sequential dependence.
Revised: As learning progresses, a hierarchy of chunk representations is acquired by chunking previously learned representations into more complex representations guided by sequential dependence.

Original: We provide learning guarantees on an idealizedversion of HCM, and demonstrate that HCM learns meaningful and interpretablerepresentations in a human-like fashion.
Revised: We provide learning guarantees on an idealized version of HCM, and demonstrate that HCM learns meaningful and interpretable representations in a human-like fashion.

Original: Our model can be extended to learn visual,temporal, and visual-temporal chunks.
Revised: Our model can be extended to learn visual, temporal, and visual-temporal chunks.

Original: The interpretability of the learned chunkscan be used to assess transfer or interference when the environment changes.
Revised: The interpretability of the learned chunks can be used to assess transfer or interference when the environment changes.

Original: Finally, in an fMRI dataset, we demonstrate that HCM learns interpretable chunks offunctional coactivation regions and hierarchical modular and sub-modular structures confirmed by the neuroscientific literature.
Revised: Finally, in an fMRI dataset, we demonstrate that HCM learns interpretable chunks of functional coactivation regions and hierarchical modular and sub-modular structures supported by the neuroscientific literature.

Original: Taken together, our results showhow cognitive science in general and theories of chunking in particular can informnovel and more interpretable approaches to representation learning.
Revised: Taken together, our results show how cognitive science in general and theories of chunking in particular can inform novel and more interpretable approaches to representation learning.

Original:  Introduction
Revised:  Introduction

Original: Sequential data in our everyday life is often hierarchically structured.
Revised: Sequential data in our everyday life is often hierarchically structured.

Original: From streaming this sequentialsensory perceptual data, we can identify repeated patterns – and bootstrap these to recognize higherorder patterns.
Revised: From streaming this sequential sensory perceptual data, we can identify repeated patterns – and bootstrap these to recognize higher order patterns.

Original: In cognitive science, identifying repeated, invariant patterns from sequences in unitsis known as chunking .
Revised: In cognitive science, identifying repeated, invariant patterns from sequences in units is known as chunking .

Original: To get an intuition for chunking, try to read through the following sequenceof letters: “DFJKJKJKDFDFJKJKDFDF”.
Revised: To get an intuition for chunking, try to read through the following sequence of letters: “DFJKJKJKDFDFJKJKDFDF”.

Original: Upon reaching the end, if you were asked to repeatthe letters from memory, you might recall fragments of the sequence such as “DF” or “JK”.
Revised: Upon reaching the end, if you were asked to repeat the letters from memory, you might recall fragments of the sequence such as “DF” or “JK”.

Original: Byparsing the sequence of letters only once, you have already detected frequently occurring patternsand memorized them together as units, i.e. chunks .
Revised: By parsing the sequence of letters only once, you have already detected frequently occurring patterns and memorized them together as units, i.e. chunks .

Original: Chunking has been observed in a range of sensoryand behavioral modalities including language learning [1, 2], action organization [3, 4] and visualperception of structures [5–7].
Revised: Chunking has been observed in a range of sensory and behavioral modalities including language learning [1, 2], action organization [3, 4] and visual perception of structures [5–7].

Original: Chunking as a mechanism is a basis for humans to identify patternsas objects, assigning labels to them to facilitate memory compression [8, 9], sequence prediction[10, 11], communication [12, 13], and generalization[14].
Revised: Chunking as a mechanism is a basis for humans to identify patterns as objects, assigning labels to them to facilitate memory compression [8, 9], sequence prediction [10, 11], communication [12, 13], and generalization[14].

Original: Learning hierarchical representations ofthe world is a feature central to human intelligence.
Revised: Learning hierarchical representations of the world is a feature central to human intelligence.

Original: Despite recent success, deep learning models, on the other hand, do not represent explicit hierarchies.
Revised: Despite recent success, deep learning models, on the other hand, do not represent explicit hierarchies.

Original: Neural networks contain sub-symbolic, nested, non-linear structures whose prediction processesare hard to comprehend.
Revised: Neural networks contain sub-symbolic, nested, non-linear structures whose prediction processes are hard to comprehend.

Original: This lack of interpretability raises concerns over their fairness, privacy, robustness and trust-worthiness [15–18] and manifests itself as a key shortcoming of these models 
Revised: This lack of interpretability raises concerns over their fairness, privacy, robustness and trust-worthiness [15–18] and manifests itself as a key shortcoming of these models [19, 20].

Original: To address these shortcomings, researchers have urged to seek inspiration from cognitivescience to construct models that resemble the hierarchical and interpretable representations asobserved in human learners [21, 22].
Revised: To address these shortcomings, researchers have urged to seek inspiration from cognitive science to construct models that resemble the hierarchical and interpretable representations as observed in human learners [21, 22].

Original: We take a two-fold approach to this problem.
Revised: We take a two-fold approach to this problem.

Original: First, insteadof learning from iid data, we ask: what if the time series data that comprises streams of perceptioncomes from a hierarchical structure?
Revised: First, instead of learning from iid data, we ask: what if the time series data that comprises streams of perception comes from a hierarchical structure?

Original: Under this assumption, what could be an algorithm that learnsthe embedded hierarchical structure?
Revised: Under this assumption, what could be an algorithm that learns the embedded hierarchical structure?

Original: We take inspiration from models in cognitive science showingthat people perceive structures based on the Gestalt principle of grouping by proximity , and formulatea generic hierarchical pattern discovery algorithm that enables the rational discovery of structureswith embedded hierarchies.
Revised: We take inspiration from models in cognitive science showing that people perceive structures based on the Gestalt principle of grouping by proximity , and formulate a generic hierarchical pattern discovery algorithm that enables the rational discovery of structures with embedded hierarchies.

Original: We refer to this as the hierarchical chunking model (HCM).
Revised: We refer to this as the hierarchical chunking model (HCM).

Original: HCM starts out learning a minimal set of units sufficient to explain the sequence and graduallycombines these units into increasingly larger and more complex chunks, constructing interpretablehierarchical structures.
Revised: HCM starts out learning a minimal set of units sufficient to explain the sequence and gradually combines these units into increasingly larger and more complex chunks, constructing interpretable hierarchical structures.

Original: We derive learning guarantees on an idealized generative model and demonstrate convergence on sequential data coming from this generative model.
Revised: We derive learning guarantees on an idealized generative model and demonstrate convergence on sequential data coming from this generative model.

Original: Thereby, Gestalt principlesof grouping can be understood as a rational way of learning representations from sequences with aninherent hierarchical structure.
Revised: Thereby, Gestalt principles of grouping can be understood as a rational way of learning representations from sequences with an inherent hierarchical structure.

Original: We then show that HCM resembles human learning in qualitativeways (compared to a recurrent neural network): it resembles human chunk learning and reactiontime measures and flexibly transfers components learned from one task to another.
Revised: We then show that HCM resemble human chunking in qualitative ways compared to a recurrent neural network and flexibly transfers components learned from one task to another.

Original: We extended HCM to the visual-temporal domain capable of learning visual-temporal parts and wholes fromhigher dimensional sequential data.
Revised: We extended HCM to the visual-temporal domain capable of learning visual-temporal parts and wholes from higher dimensional sequential data.

Original: Taking it one step further, we deploy HCM to learn from highdimensional fMRI data, which exerts a hierarchical structure.
Revised: Taking it one step further, we deploy HCM to learn from high-dimensional fMRI data, which exerts a hierarchical structure.

Original: We demonstrate HCM’s interpretablefeature extraction ability to discover submodules of brain activations directly linkable to behaviorconfirmed by the neuroscientific literature and discoveries such as the correlation between brainchunk size with age and stimulus-specificbrain chunks.
Revised: We demonstrate HCM’s interpretable feature extraction ability to discover submodules of brain activations directly linkable to behavior supported by the neuroscientific literature and discoveries such as the correlation between brain chunk size with age and stimulus-specific chunks.

Original:  Hierarchical Chunking Model
Revised:  Hierarchical Chunking Model

Original: A chunk is a unit created by concatenating several atomic sequential units together.
Revised: We define a chunk as a unit created by concatenating several atomic sequential units together.

Original: Taking thetraining sequence shown in Figure 1a as an example, the sequence is made up of discrete, size-oneatomic units from an atomic alphabet A 0 : in this case A 0 = { 0 , 1 , 2 } .
Revised: Taking the training sequence shown in Figure 1a as an example, the sequence is made up of discrete, size-one atomic units from an atomic alphabet A 0 : in this case A 0 = { 0 , 1 , 2 } .

Original: A chunk c is made up of acombination of one or more atomic units in A 0 \{ 0 } .
Revised: A chunk c is made up of a combination of one or more atomic units in A 0 \{ 0 } .

Original: 0 denotes an empty observation in the sequence.
Revised: 0 denotes an empty observation in the sequence.

Original: Intuitively, if a sequence contains inherent hierarchical structure, then there are patterns which spanseveral sequential units sharing these internal structures, examples of such sequences are repeatedmelodies and sub-melodies in music.
Revised: Intuitively, if a sequence contains inherent hierarchical structure, then there are patterns which span several sequential units sharing these internal structures, examples of such sequences are repeated melodies and sub-melodies in music.

Original: If the pattern occurs in the sequence, observations betweensequential units within the pattern will be correlated.
Revised: If the pattern occurs in the sequence, observations between sequential units within the pattern will be correlated.

Original: In this case, chunking patterns within a sequenceas units simplifies perceptual processing in the sense that the sequence can be perceived one chunkafter another, instead of one sequential unit at a time.
Revised: In this case, chunking patterns within a sequence as units simplifies perceptual processing in the sense that the sequence can be perceived one chunk after another, instead of one sequential unit at a time.

Original: Furthermore, the acquired “primary” chunksserve as building blocks to discover larger chunks that are embedded within the hierarchy of thesequential structure.
Revised: Furthermore, the acquired “primary” chunks serve as building blocks to discover larger chunks that are embedded within the hierarchy of the sequential structure.

Original: Formally, HCM acquires a belief set B of chunks, and uses chunks from the belief set to parse thesequence.
Revised: Formally, HCM acquires a belief set B of chunks, and uses chunks from the belief set to parse the sequence.

Original: HCM assumes that a sequence is generated from samples of independently occurringchunks with probability of P B ( c ) evaluated on the belief set B .
Revised: HCM assumes that a sequence is generated from samples of independently occurring chunks with probability of P B ( c ) evaluated on the belief set B .

Original: The probability of observinga sequence of parsed chunks c 1 , c 2 , ..., c N can be denoted as P ( c 1 , c 2 , ..., c N ) = (cid:81) c i ∈ B P B ( c i ) .
Revised: The probability of observing a sequence of parsed chunks c 1 , c 2 , ..

Original: Chunks as perceiving units serve as independent factors that disentangle observations in the sequence.

Original: Starting at the beginning of the sequence, HCM ‘perceives’ sequential units one chunk at a time,in other words, the training sequence is parsed by HCM into chunks.
Revised: Starting at the beginning of the sequence, HCM ‘perceives’ sequential units one chunk at a time, in other words, the training sequence is parsed by HCM into chunks.

Original: At every parsing step, thelongest chunk in the belief set that is consistent with the upcoming sequence is chosen to explain theup-coming sequential observations.
Revised: At every parsing step, the longest chunk in the belief set that is consistent with the upcoming sequence is chosen to explain the up-coming sequential observations.

Original: The end of the previous chunk parse initiates the next parse.
Revised: The end of the previous chunk parse initiates the next parse.

Original: Observing a hierarchically structured sequence as illustrated in Figure 1a, HCM gradually builds upa hierarchy of chunks starting from an empty belief set.
Revised: Observing a hierarchically structured sequence as illustrated in Figure 1a, HCM gradually builds up a hierarchy of chunks starting from an empty belief set.

Original: It first identifies a set of atomic chunks toconstruct its initial belief set B .
Revised: It first identifies a set of atomic chunks to construct its initial belief set B .

Original: Initially, these will be chunks of length one, yielding one-by-oneprocessing of the primitive elements.
Revised: Initially, these will be chunks of length one, yielding one-by-one processing of the primitive elements.

Original: For one belief set B , HCM keeps track of the marginal parsing frequency M ( c i ) for each chunk c i in86 B , a vector with size | B | and the transition frequency T between chunk c i followed by chunk c j , asillustrated in Figure 1b.
Revised: For one belief set B , HCM keeps track of the marginal parsing frequency M ( c i ) for each chunk c i in B , a vector with size | B | and the transition frequency T between chunk c i followed by chunk c j , as illustrated in Figure 1b.

Original: Entries in M and T are used to test the hypothesis that consecutive chunkparses have a correlated consecutive occurrence within the sequence via a χ 2 -independence test.
Revised: Entries in M and T are used to test the hypothesis that consecutive chunk parses have a correlated consecutive occurrence within the sequence via a χ 2 -independence test.

Original: If two chunks c L and c R have a significant adjacency dependence based on their entries in M and T ,they are chunked together to become c L ⊕ c R , which augments the belief set B by one.
Revised: If two chunks c L and c R have a significant adjacency dependence based on their entries in M and T , they are chunked together to become c L ⊕ c R , which augments the belief set B by one.

Original: One exampleof chunk merging is shown in Figure 1c.
Revised: One example of chunk merging is shown in Figure 1c.

Original: Independence Test We use a χ 2 -test of independence to assess the correlation of consecutive occurrences of c L followed by c R in B .
Revised: Independence Test We use a χ 2 -test of independence to assess the correlation of consecutive occurrences of c L followed by c R in B .

Original: Let c L be an indicator variable that is 1 when chunk c l is parsed and 0 otherwise, similarly we formulate c R as another indicator variable of parsing the chunk c r .
Revised: Let c L be an indicator variable that is 1 when chunk c l is parsed and 0 otherwise, similarly we formulate c R as another indicator variable of parsing the chunk c r .

Original: We evaluate the χ 2 -value as a criterion to reject the null hypothesis that the consecutive observation of c l followed by c r is statistically independent:
Revised: We evaluate the χ 2 -value as a criterion to reject the null hypothesis that the consecutive observation of c l followed by c r is statistically independent:

Original: [Equation]
Revised: [Equation]

Original: p ( c L , c R ) and p ( c L ) p ( c R ) are evaluated from M an T .
Revised: p ( c L , c R ) and p ( c L ) p ( c R ) are evaluated from M an T .

Original: The degree of freedom is 1.
Revised: The degree of freedom is 1.

Original: A χ 2 -probabilityof less than 0.05 is the criterion to reject the null hypothesis (i.e. that c l and c r occur independently).
Revised: A χ 2 -probability of less than 0.05 is the criterion to reject the null hypothesis (i.e. that c l and c r occur independently).

Original: There are two versions of HCM.
Revised: There are two versions of HCM.

Original: The Rational Chunk Learning HCM learns chunks in an idealizedway which we use to study learning guarantees.
Revised: The Rational Chunk Learning HCM learns chunks in an idealized way which we use to study learning guarantees.

Original: The online version of HCM is an approximationto the rational HCM that can be adapted to different environments and processes sequences online.
Revised: The online version of HCM is an approximation to the rational HCM that can be adapted to different environments and processes sequences online.

Original: Pseudo-code for both algorithms can be found in the SI.
Revised: Pseudo-code for both algorithms can be found in the SI.

Original: Rational Chunk Learning: HCM as an Ideal Observer HCM is initiated with an empty beliefset and it first finds a minimally complete belief set after the first sequence parse.
Revised: Rational Chunk Learning: HCM as an Ideal Observer HCM is initiated with an empty belief set and it first finds a minimally complete belief set after the first sequence parse.

Original: In each iteration,the entire sequence is parsed to evaluate M and T , which are used to find consecutive chunk parsesin the existing belief set that violate the independence testing criterion.
Revised: In each iteration, the entire sequence is parsed to evaluate M and T , which are used to find consecutive chunk parses in the existing belief set that violate the independence testing criterion.

Original: From these dependent chunkpairs, the pair with the largest estimated joint probability is combined into a new chunk.
Revised: From these dependent chunk pairs, the pair with the largest estimated joint probability is combined into a new chunk.

Original: The newchunk enlarges the belief set by one.
Revised: The new chunk enlarges the belief set by one.

Original: The chunks in the new belief set are used to parse the sequencein the next iteration.
Revised: The chunks in the new belief set are used to parse the sequence in the next iteration.

Original: This process repeats until all of the chunks in the belief set pass an independencehalting criterion, which measures if all of the chunks in the belief set are currently independent, againassessed via a  2 -test (see SI).
Revised: This process repeats until all of the chunks in the belief set pass an independence halting criterion, which measures if all of the chunks in the belief set are currently independent, again assessed via a χ 2 -test (see SI).

Original: Online Chunk Learning The online chunk learning HCM approximates the ideal observer HCMby learning new chunks when the training sequence is processed online.
Revised: Online Chunk Learning The online chunk learning HCM approximates the ideal observer HCM by learning new chunks when the training sequence is processed online.

Original: To have a feature thatencourages adaptation to new environmental statistics, entries in M and T can be subject to memorydecay.
Revised: To have a feature that encourages adaptation to new environmental statistics, entries in M and T can be subject to memory decay.

Original: We use the ideal observer model to demonstrate learning guarantees, but use the online modelto learn representations in realistic and more complex set-ups.
Revised: We use the ideal observer model to demonstrate learning guarantees, but use the online model to learn representations in realistic and more complex set-ups.

Original: 2.1 HCM Learns Representations from the Ground Up
Revised: 2.1 HCM Learns Representations from the Ground Up

Original: As HCM learns from a sequence, it starts with no representation and gradually builds up interpretablerepresentations described by a chunk hierarchy graph ˆ G with the vertex set being the chunks andedges pointing from constituents to composites.
Revised: As HCM learns from a sequence, it starts with no representation and gradually builds up interpretable representations described by a chunk hierarchy graph ˆ G with the vertex set being the chunks and edges pointing from constituents to composites.

Original: Shown in Figure 1d is the gradual build-up of onesuch graph as the model learns from a training sequence coming from the generative hierarchy in
Revised: Shown in Figure 1d is the gradual build-up of one such graph as the model learns from a training sequence coming from the generative hierarchy in

Original: Figure 1a.
Revised: Figure 1a.

Original: At t = 10 , HCM learns only the atomic chunks, at t = 60 , HCM has already constructedtwo additional chunks; when t = 100 , two more additional chunks are constructed based on previousconstructions.
Revised: At t = 10 , HCM learns only the atomic chunks, at t = 60 , HCM has already constructed two additional chunks; when t = 100 , two more additional chunks are constructed based on previous constructions.

Original: HCM arrives at the final chunk hierarchy at t = 150 .
Revised: HCM arrives at the final chunk hierarchy at t = 150 .

Original:  Generating Sequences with a Hierarchical Structure
Revised:  Generating Sequences with a Hierarchical Structure

Original: We construct a generative model to study HCM’s behavior formally and empirically.
Revised: We construct a generative model to study HCM’s behavior formally and empirically.

Original: The generativemodel constructs random chunk hierarchies from which non-iid sequences are sampled.
Revised: The generative model constructs random chunk hierarchies from which non-iid sequences are sampled.

Original: Such graph
Revised: Such graph

Original: G d contains vertex set V A d and edge set E A d to describe the relation between chunks and theirconstituents.
Revised: G d contains vertex set V A d and edge set E A d to describe the relation between chunks and their constituents.

Original: One example is illustrated in Figure 1a.
Revised: One example is illustrated in Figure 1a.

Original: A d is the set of chunks used to construct thesequence.
Revised: A d is the set of chunks used to construct the sequence.

Original: The depth d specifies the number of chunks created in the generative process.
Revised: The depth d specifies the number of chunks created in the generative process.

Original: Starting with an initial set of atomic chunks A 0 , at the i-th iteration, two chunks c L , c
Revised: Starting with an initial set of atomic chunks A 0 , at the i-th iteration, two chunks c L , c

Original: R are randomlychosen from the current set of chunks A i and are concatenated into a new chunk c L ⊕ c R , augmenting128 A i by one to A i +1 .
Revised: R are randomly chosen from the current set of chunks A i and are concatenated into a new chunk c L ⊕ c R , augmenting A i by one to A i +1 .

Original: Meanwhile, an independent occurrence probability is assigned to each chunkunder the constraint that the probability of occurrence for every new chunk c i in the constructionprocess evaluated on the support set A i carries the largest probability mass.
Revised: Meanwhile, an independent occurrence probability is assigned to each chunk under the constraint that the probability of occurrence for every new chunk c i in the construction process evaluated on the support set A i carries the largest probability mass.

Original: Once a graph hierarchy is constructed, we construct non-iid observational sequences by consecutivelysampling chunks from the hierarchy with their corresponding probability, under the constraint that notwo chunks with a child chunk are sampled consecutively.
Revised: Once a graph hierarchy is constructed, we construct non-iid observational sequences by consecutively sampling chunks from the hierarchy with their corresponding probability, under the constraint that no two chunks with a child chunk are sampled consecutively.

Original: 3.1 Learning Guarantee
Revised: 3.1 Learning Guarantee

Original: Theorem : As the length of the sequence approaches infinity, HCM learns a hierarchical chunkinggraph ˆ G isomorphic to the generative hierarchical graph G .
Revised: Theorem : As the length of the sequence approaches infinity, HCM learns a hierarchical chunking graph ˆ G isomorphic to the generative hierarchical graph G .

Original: Proof Sketch : We approach this proof by induction.
Revised: Proof Sketch : We approach this proof by induction.

Original: Further details can be found in the SI.
Revised: Further details can be found in the SI.

Original: Base step:
Revised: Base step:

Original: The first step of the rational chunking algorithm is to find the minimally complete atomic set of chunksto form its initial belief set.
Revised: The first step of the rational chunking algorithm is to find the minimally complete atomic set of chunks to form its initial belief set.

Original: This procedure guarantees that ˆ G 0 = G 0 .
Revised: This procedure guarantees that ˆ G 0 = G 0 .

Original: Additionally, the probabilitymass of the learning model at step 0 and the generative model at step 0 is asymptotically the sameas the sequence length approaches infinity.
Revised: Additionally, the probability mass of the learning model at step 0 and the generative model at step 0 is asymptotically the same as the sequence length approaches infinity.

Original: Induction hypothesis: Assume that the learned beliefset B i at step i contains the same chunks as the alphabet set A i in the generative model, the chunkcombination pair with the biggest evaluated joint occurrence probability violating the independencetest is picked to be concatenated into a chunk to extend the belief set: this chunk is the same chunknode created by the hierarchical generative model.
Revised: Induction hypothesis: Assume that the learned belief set B i at step i contains the same chunks as the alphabet set A i in the generative model, the chunk combination pair with the biggest evaluated joint occurrence probability violating the independence test is picked to be concatenated into a chunk to extend the belief set: this chunk is the same chunk node created by the hierarchical generative model.

Original: End step: The chunk learning process stops oncethe independence criterion is no longer violated.
Revised: End step: The chunk learning process stops once the independence criterion is no longer violated.

Original: This is the case once the chunk learning algorithmhas learned a belief set B d = A d .
Revised: This is the case once the chunk learning algorithm has learned a belief set B d = A d .

Original: 3.2 Learning Convergence and Comparative Data-efficiency
Revised: 3.2 Learning Convergence and Comparative Data-efficiency

Original: To evaluate and show HCM’s learning performance, we trained HCM to learn hierarchies of chunksfrom sequences generated by the hierarchical generative model.
Revised: To evaluate and show HCM’s learning performance, we trained HCM to learn hierarchies of chunks from sequences generated by the hierarchical generative model.

Original: Shown in Figure 2 is HCM’s learningperformance as sequence length increases, averaged over 30 independently generated random graphswith the same depth d .
Revised: Shown in Figure 2 is HCM’s learning performance as sequence length increases, averaged over 30 independently generated random graphs with the same depth d .

Original: One example of such graphs is shown in Figure 2a.
Revised: One example of such graphs is shown in Figure 2a.

Original: Kullback-Leiblerdivergence was used to evaluate learning performance.
Revised: Kullback-Leibler divergence was used to evaluate learning performance.

Original: To this end, learned hierarchies by HCM wereused to generate sequences, which were then evaluated on the support set of the alphabet set in thegenerative model.
Revised: To this end, learned hierarchies by HCM were used to generate sequences, which were then evaluated on the support set of the alphabet set in the generative model.

Original: Figure 2b shows the KL-divergence between the learned and ground-truth distribution for increasingdepths d of the generative graphs.
Revised: Figure 2b shows the KL-divergence between the learned and ground-truth distribution for increasing depths d of the generative graphs.

Original: For each depth, the KL-divergence was evaluated on 30 randomgenerative models with sequence length increasing from 50 to 3000.
Revised: For each depth, the KL-divergence was evaluated on 30 random generative models with sequence length increasing from 50 to 3000.

Original: Overall, the KL-divergencedecreased as the training sequence length increased and converged with longer training sequences,showing a closer representation resemblance to the generative model.
Revised: Overall, the KL-divergence decreased as the training sequence length increased and converged with longer training sequences, showing a closer representation resemblance to the generative model.

Original: A similar training and learning evaluation was conducted on a 3-layer Recurrent Neural Network(RNN) with 40 hidden units for comparison.
Revised: A similar training and learning evaluation was conducted on a 3-layer Recurrent Neural Network (RNN) with 40 hidden units for comparison.

Original: As the length of the training sequence increased, the
Revised: As the length of the training sequence increased, the

Original: KL-divergence of RNN converged at a slower rate than HCM.
Revised: KL-divergence of RNN converged at a slower rate than HCM.

Original: This competitive advantage in dataefficiency became more pronounced with increasing depth of the generative hierarchy.
Revised: This competitive advantage in data efficiency became more pronounced with increasing depth of the generative hierarchy.

Original:  HCM Resembles Human Chunk Learning
Revised:  HCM Resembles Human Chunk Learning

Original: [Figure]

Original: Here we compare the chunk learning behavior of HCM to the learning characteristics of humans.
Revised: Here we compare the chunk learning behavior of HCM to the learning characteristics of humans.

Original: Tothat end, we used data collected from a sequence learning study by [23] with 47 participants underthe license CC-BY 4.0.
Revised: To that end, we used data collected from a sequence learning study by [23] with 47 participants under the license CC-BY 4.0.

Original: As shown in Figure 3a, the training sequence comprised chunks ABC and D,independently occurring with equal probability.
Revised: As shown in Figure 3a, the training sequence comprised chunks ABC and D, independently occurring with equal probability.

Original: The study assessed how participants built up chunkknowledge gradually.
Revised: The study assessed how participants built up chunk knowledge gradually.

Original: Participants’ reaction times reflected that, after enough training, they wereanticipating several upcoming sequence elements, suggesting that they have acquired longer chunks(Figure 3b, left) [23].
Revised: Participants’ reaction times reflected that, after enough training, they were anticipating several upcoming sequence elements, suggesting that they have acquired longer chunks (Figure 3b, left) [23].

Original: In a similar vein, we measured online chunk size increase of HCM using the same method asin [23] and, for comparison, RNNs (see SI for further comparisons to other algorithms).
Revised: In a similar vein, we measured online chunk size increase of HCM using the same method as in [23] and, for comparison, RNNs (see SI for further comparisons to other algorithms).

Original: HCM,similarly to humans, started learning longer chunks early in the sequence.
Revised: HCM, similarly to humans, started learning longer chunks early in the sequence.

Original: By contrast, RNN didnot start to chunk until after step 300, and when it started to learn chunks, the increase rate of thepredictive horizon was not as steep as HCM’s.
Revised: By contrast, RNN did not start to chunk until after step 300, and when it started to learn chunks, the increase rate of the predictive horizon was not as steep as HCM’s.

Original: Evaluating the average rate of chunk growth alsoshowed that HCM builds up chunks as learning progresses was more similar to participants’ thanthe RNN’s
Revised: Evaluating the average rate of chunk growth also showed that HCM builds up chunks as learning progresses was more similar to participants’ than the RNN’s

Original: (Figure 3c).
Revised: (Figure 3c).

Original: The negative log-probabilities of sequence elements generated by the HCMand RNN were both significantly related to human reaction times (that reflect the certainty of theirinternal predictions [24]).
Revised: The negative log-probabilities of sequence elements generated by the HCM and RNN were both significantly related to human reaction times (that reflect the certainty of their internal predictions [24]).

Original: Yet, the relationship was substantially stronger (Figure 3d) between HCM( β = 16 .
Revised: Yet, the relationship was substantially stronger (Figure 3d) between HCM ( β = 16 .

Original: 74 , p ≤ 0 . 001  ) and human participants compared to that of the RNN ( β = 9 .
Revised: 74 , p ≤ 0 . 001 , τ = 0 . 165 , BIC = 313586 . 5 ) and human participants compared to that of the RNN ( β = 9 .

Original: 24 , p ≤ 0 .
Revised: 24 , p ≤ 0 .

Original:  These results suggest that HCM resembles human chunk learning more strongly than RNNs and cantherefore be seen as the cognitively more plausible approach to hierarchical representation learning.
Revised: 001 , τ = 0 . 085 , BIC = 314236 . 4 ). These results suggest that HCM resembles human chunk learning more strongly than RNNs and can therefore be seen as the cognitively more plausible approach to hierarchical representation learning.

Original:  HCM Permits Transfer Between Environments
Revised:  HCM Permits Transfer Between Environments

Original: One characteristic of human learning is that previous learning experience facilitates and sometimesinterferes with acquiring a new skill [25].
Revised: One characteristic of human learning is that previous learning experience facilitates and sometimes interferes with acquiring a new skill [25].

Original: Having an interpretable representation can inform us aboutpositive or negative transfer a priori.
Revised: Having an interpretable representation can inform us about positive or negative transfer a priori.

Original: An HCM has learned a chunking graph in Figure 4a.
Revised: An HCM has learned a chunking graph in Figure 4a from an environment.

Original: When it switches to another environmentwith overlapping  representation, it can reuse the learned subgraphs of chunks marked in gray as in Figure 4b and learns faster than a naive HCM in Figure 4c.
Revised: When it switches to another environment with a generative model overlapping with its previously acquired representation, it can reuse the learned subgraphs of chunks marked in gray as in Figure 4b and learns faster than a naive HCM in Figure 4c.

Original: Vice versa, transfer can be detrimentalwhen there is no or little overlap between the learned chunking graph and the  new environment.
Revised: Vice versa, transfer can be detrimental when there is no or little overlap between the learned chunking graph and the generative model of the new environment.

Original: Forthe same HCM as in Figure 4a, transferring to an environment with a chunking graph in Figure 4dimplies learning the shaded chunks anew, in addition to running the risk of being misled by theprevious representations.
Revised: For the same HCM as in Figure 4a, transferring to an environment with a chunking graph in Figure 4d implies learning the shaded chunks anew, in addition to running the risk of being misled by the previous representations.

Original: As a result, the early performance of the pre-trained HCM suffers more froman interfering environment than a naive model in Figure 4e.
Revised: As a result, the early performance of the pre-trained HCM suffers more from an interfering environment than a naive model in Figure 4e.

Original: Interpretability of HCM’s representationsenables the assessment of facilitation or interference when the environment changes.
Revised: Interpretability of HCM’s representations enables the assessment of facilitation or interference when the environment changes.

Original:  Generalizing to Visual Temporal Chunks via the Principle of ProximalGrouping
Revised:  Generalizing to Visual Temporal Chunks via the Principle of Proximal Grouping

Original: Humans excel at finding structures in hierarchical visual objects and grouped movements.
Revised: Humans excel at finding structures in hierarchical visual objects and grouped movements.

Original: The Gestaltprinciple of grouping by proximity suggests that we tend to group objects that are close to one anotherinto a cohesive unit [26, 27].
Revised: The Gestalt principle of grouping by proximity suggests that we tend to group objects that are close to one another into a cohesive unit [26, 27].

Original: This principle has been suggested to play a key role in human perceptualgrouping [28], benefits working memory [29] and reduces visual complexity [30].
Revised: This principle has been suggested to play a key role in human perceptual grouping [28], benefits working memory [29] and reduces visual complexity [30].

Original: Indeed, in humansand other animals, learning of adjacent relationships prevails over non-adjacent ones [31].
Revised: Indeed, in humans and other animals, learning of adjacent relationships prevails over non-adjacent ones [31].

Original: Therefore,the adjacent dependency structure can be expanded to chunking in visual temporal domains [32].
Revised: Therefore, the adjacent dependency structure can be expanded to chunking in visual temporal domains [32].

Original: Toemulate this ability of chunking via proximal grouping, we extend HCM to learn visual temporalchunks.
Revised: To emulate this ability of chunking via proximal grouping, we extend HCM to learn visual temporal chunks.

Original: Visual temporal chunks subsume temporal length and varying visual slices in each temporal slice(Figure 5a).
Revised: Visual temporal chunks subsume temporal length and varying visual slices in each temporal slice (Figure 5a).

Original: One can imagine a visual temporal chunk as having a 3D shape — the first twodimensions are the visual part of the chunk, and the object’s length is the temporal part, made ofstacked visual-temporal pixels.
Revised: One can imagine a visual temporal chunk as having a 3D shape — the first two dimensions are the visual part of the chunk, and the object’s length is the temporal part made of stacked visual-temporal pixels.

Original: Within each temporal slice are the visual features identified bythe chunk.
Revised: Within each temporal slice are the visual features identified by the chunk.

Original: As the model iterates through data across its temporal slice, the chunk that attains thebiggest visual temporal volume explains part of the observational sequence.
Revised: As the model iterates through data across its temporal slice, the chunk that attains the biggest visual temporal volume explains part of the observational sequence.

Original: Multiple visual temporalchunks can occur simultaneously.
Revised: Multiple visual temporal chunks can occur simultaneously.

Original: Starting at the visual temporal time point marked by the previouschunk, chunks are identified and stored in M .
Revised: Starting at the visual temporal time point marked by the previous chunk, chunks are identified and stored in M .

Original: The transition matrix T is modified to account forthe temporal lag difference between adjacent chunk pairs within a proximity parameter and recordsthe frequency that one chunk transitions into another for each time lag.
Revised: The transition matrix T is modified to account for the temporal lag difference between adjacent chunk pairs within a proximity parameter and records the frequency that one chunk transitions into another for each time lag.

Original: Whenever a pair of adjacentchunks are identified, an independence hypothesis test evaluates whether the adjacent observation arecorrelated.
Revised: Whenever a pair of adjacent chunks are identified, an independence hypothesis test evaluates whether the adjacent observation are correlated.

Original: Chunks that violate the hypothesis test are combined to parse future sequences.
Revised: Chunks that violate the hypothesis test are combined to parse future sequences.

Original: Learning Part-Whole Relationship Between Visual Components We show HCM learned chunksin the visual domain from a sequence of independently sampled images.
Revised: Learning Part-Whole Relationship Between Visual Components We show HCM learned chunks in the visual domain from a sequence of independently sampled images.

Original: Figure 5b left shows ahierarchical generative model in the pixel-wise image domain.
Revised: Figure 5b left shows a hierarchical generative model in the pixel-wise image domain.

Original: A set of elementary visual units inthe lowest hierarchy level combines into intermediate and more complex visual units higher up inthe hierarchy.
Revised: A set of elementary visual units in the lowest hierarchy level combines into intermediate and more complex visual units higher up in the hierarchy.

Original: All of the constructed elements in the hierarchy occurred independently according toa probability drawn from a Dirichlet flat distribution.
Revised: All of the constructed elements in the hierarchy occurred independently according to a probability drawn from a Dirichlet flat distribution.

Original: Images in the hierarchy were independently sampled from the generative distribution to become the training sequence.
Revised: Images in the hierarchy were independently sampled from the generative distribution to become the training sequence.

Original: In Figure 5b, right weshow the chunk representations learned by HCM at different stages.
Revised: In Figure 5b, right we show the chunk representations learned by HCM at different stages.

Original: Initially, HCM acquires theindividual pixels as chunks to explain the observations.
Revised: Initially, HCM acquires the individual pixels as chunks to explain the observations.

Original: As HCM proceeds with learning, it discoversvisual correlations among the pixels and constructs increasingly complex visual parts.
Revised: As HCM proceeds with learning, it discovers visual correlations among the pixels and constructs increasingly complex visual parts.

Original: Learning Visual-Temporal Movement Hierarchies Instead of seeing one image after anothersampled from an independent, identically distributed distribution, real-world experiences containcorrelations in both the visual and temporal dimensions.
Revised: Learning Visual-Temporal Movement Hierarchies Instead of seeing one image after another sampled from an independent, identically distributed distribution, real-world experiences contain correlations in both the visual and temporal dimensions.

Original: From observing object movements acrossspace and time, the visual system learns structures from correlated visual and temporal observations,decomposes motion structure, and groups moving objects together as a whole [33].
Revised: From observing object movements across space and time, the visual system learns structures from correlated visual and temporal observations, decomposes motion structure, and groups moving objects together as a whole [33].

Original: To emulate thistype of environment, an animated GIF of a squid swimming in the sea (Figure 5c) was used as avisual-temporal sequence to train HCM.
Revised: To emulate this type of environment, an animated GIF of a squid swimming in the sea (Figure 5c) was used as a visual-temporal sequence to train HCM.

Original: As learning advances, HCM learns chunks spanning boththe visual and temporal domains.
Revised: As learning advances, HCM learns chunks spanning both the visual and temporal domains.

Original: There are visual-temporal chunks that mark the movements of atentacle and the rising-up motion of a bubble.
Revised: There are visual-temporal chunks that mark the movements of a tentacle and the rising-up motion of a bubble.

Original: Additionally, visual chunks resemble a part of thevisual’s eye and face.
Revised: Additionally, visual chunks resemble a part of the visual’s eye and face.

Original: The meaningful chunks in the visual-temporal domain suggest the groupingprinciple enables the plausible learning of movement sequences and aids the perception of objects aswholes and their corresponding parts.
Revised: The meaningful chunks in the visual-temporal domain suggest the grouping principle enables the plausible learning of movement sequences and aids the perception of objects as wholes and their corresponding parts.

Original:  Learning Hierarchies of Brain Activation from Resting-state fMRI data
Revised:  Learning Hierarchies of Brain Activation from Resting-state fMRI data

Original: HCM learns hierarchies from structured sequential data.
Revised: HCM learns hierarchies from structured sequential data.

Original: As brain activation has been suggested to behierarchically structured [34], we demonstrate HCM’s usefulness to learn structures in biologicalneural networks activating in response to complex stimuli by running HCM on a resting-state fMRIdata set.
Revised: As brain activation has been suggested to be hierarchically structured [34], we demonstrate HCM’s usefulness to learn structures in biological neural networks activating in response to complex stimuli by running HCM on a resting-state fMRI data set.

Original: We used a developmental data set provided by the nilearn package with BSD License [35] andoriginally collected by [36].
Revised: We used a developmental data set provided by the nilearn package with BSD License [35] and originally collected by [36] with its corresponding IRB approval.

Original: This data set contains the resting-state BOLD activity of 155 participantsranging from age 3 to 40, while watching the silent movie “Partly Cloudy”.
Revised: This data set contains the restingstate BOLD activity of 155 participants ranging from age 3 to 40, while watching the silent movie “Partly Cloudy”.

Original: BOLD signal wasextracted from functional brain regions defined by the MSDL Atlas [37], with confounds excludedand transformed into a rounded, normalized time series.
Revised: BOLD signal was extracted from functional brain regions defined by the MSDL Atlas [37], with confounds excluded and transformed into a rounded, normalized time series.

Original: HCM’s Chunks Reflect Structural, Functional and Anatomical Connectivity Figure 6 showsthree typical examples of learned chunks for a randomly-chosen participant.
Revised: HCM’s Chunks Reflect Structural, Functional and Anatomical Connectivity Figure 6 shows three typical examples of learned chunks for a randomly-chosen participant.

Original: The labels of functionalregions come from the MSDL atlas [37].
Revised: The labels of functional regions come from the MSDL atlas [37].

Original: The first example is the co-activation of D ACC and R A Ins.
Revised: The first example is the co-activation of D ACC and R A Ins.

Original: These two regions have been observed to co-activate in the presence of emotions, pain, and humor.
Revised: These two regions have been observed to co-activate in the presence of emotions, pain, and humor.

Original: They have been suggested to be a key hub of the salience network [38–41].
Revised: They have been suggested to be a key hub of the salience network [38–41].

Original: The second examplechunk contains the activation of R DLPFC and R Front Pole.
Revised: The second example chunk contains the activation of R DLPFC and R Front Pole.

Original: These regions belong to the visualattention network and are known to be anatomically connected [42–44].
Revised: These regions belong to the visual attention network and are known to be anatomically connected [42–44].

Original: A final example is the chunk of L Ins and Cing, which are also known to be anatomically and functionally connected [45].
Revised: A final example is the chunk of L Ins and Cing, which are also known to be anatomically and functionally connected [45].

Original: Thus,the chunks discovered by HCM correspond to empirically-verified patterns of functional activity.
Revised: Thus, the chunks discovered by HCM correspond to empirically-verified patterns of functional activity.

Original: HCM’s Chunks Recover Hierarchical Activation Patterns In fMRI data, hierarchies of chunkactivation constructed by HCM reflect networks of functional regions.
Revised: HCM’s Chunks Recover Hierarchical Activation Patterns In fMRI data, hierarchies of chunk activation constructed by HCM reflect networks of functional regions.

Original: On the top of the hierarchy,the largest chunk contained L DLPFC, D ACC, R A Ins, Cing, and L Ant IPS (Figure 6b).
Revised: On the top of the hierarchy, the largest chunk contained L DLPFC, D ACC, R A Ins, Cing, and L Ant IPS (Figure 6b).

Original: Thoseregions are known to co-activate during cognitive tasks that demand attention, working memory, andcontrol [41].
Revised: Those regions are known to co-activate during cognitive tasks that demand attention, working memory, and control [41].

Original: Chunks in the intermediate levels of the hierarchy reflect sub-networks of functionalconnectivity.
Revised: Chunks in the intermediate levels of the hierarchy reflect sub-networks of functional connectivity.

Original: Sub-chunks such as D ACC, R A Ins, L Ant IPS, and L DLPFC have been suggested toconjointly activate in cognitive effort-related activities [41].
Revised: Sub-chunks such as D ACC, R A Ins, L Ant IPS, and L DLPFC have been suggested to conjointly activate in cognitive effort-related activities [41].

Original: Atomic chunks in the hierarchy suchas D ACC and L Ant IPS activate individually sometimes without their parent chunks.
Revised: Atomic chunks in the hierarchy such as D ACC and L Ant IPS activate individually sometimes without their parent chunks.

Original: Indeed, theyhave distinct functional signatures for affect processing [46, 47], and visual attention control [48].
Revised: Indeed, they have distinct functional signatures for affect processing [46, 47], and visual attention control [48].

Original: Upon exposure to a time series in fMRI data, HCM constructs chunks from their constituents andarrives at a hierarchy of chunk relations, indicating nested network structures in the human brain.
Revised: Upon exposure to a time series in fMRI data, HCM constructs chunks from their constituents and arrives at a hierarchy of chunk relations, indicating nested network structures in the human brain.

Original: HCM’s Chunks can be Matched with Stimulus Onsets
Revised: HCM’s Chunks can be Matched with Stimulus Onsets

Original: The retrieved chunks by HCM can betagged with critical stimulus onsets.
Revised: The retrieved chunks by HCM can be tagged with critical stimulus onsets.

Original: We tagged 19 critical moments involving social and emotionalcontent in the movie.
Revised: We tagged 19 critical moments involving social and emotional content in the movie.

Original: Figure 6c shows one example chunk activation upon stimulus onsets.
Revised: Figure 6c shows one example chunk activation upon stimulus onsets.

Original: Frontal
Revised: Frontal

Original: DMN, right frontal pole, ventral anterior cingulate cortex, and right anterior insula activate togetheras a recurring unit after participants witness a scene with characters greeting and hugging each other.
Revised: DMN, right frontal pole, ventral anterior cingulate cortex, and right anterior insula activate together as a recurring unit after participants witness a scene with characters greeting and hugging each other.

Original: These regions have been suggested to be involved in social and cognitive processing [49].
Revised: These regions have been suggested to be involved in social and cognitive processing [49].

Original: Anotherexample is the activation of areas known to be involved in emotion and language processing: D ACCand Broca [50], during a scene containing social interactions.
Revised: Another example is the activation of areas known to be involved in emotion and language processing: D ACC and Broca [50], during a scene containing social interactions.

Original: In the meantime, the left and rightprefrontal cortex, involved in theory-of-mind [36], also lights up.
Revised: In the meantime, the left and right prefrontal cortex, involved in theory-of-mind [36], also lights up.

Original: We categorized the tagged moments into 3 groups of different emotional load: sadness, anger, andcompassion.
Revised: We categorized the tagged moments into 3 groups of different emotional load: sadness, anger, and compassion.

Original: We then looked at the activation probability of retrieved chunks within the 6 secondsafter watching those tagged scenes.
Revised: We then looked at the activation probability of retrieved chunks within the 6 seconds after watching those tagged scenes.

Original: Figure 6d shows a list of such chunks from one participant withtheir activation probability for each emotional category.
Revised: Figure 6d shows a list of such chunks from one participant with their activation probability for each emotional category.

Original: For example, the left and right insula, knownto be involved in affective processing [51], have a 0.4 activation probability after witnessing scenesof sadness or anger, but no activation after witnessing scenes of compassion.
Revised: For example, the left and right insula, known to be involved in affective processing [51], have a 0.4 activation probability after witnessing scenes of sadness or anger, but no activation after witnessing scenes of compassion.

Original: The same holds for R
Revised: The same holds for R

Original: DLPFC and R Par that have been documented to activate in response to emotional conflict [52].
Revised: DLPFC and R Par that have been documented to activate in response to emotional conflict [52].

Original: Onthe other hand, regions such as R TPJ and R pars opercularis that are involved in emotional reactionsand theory of mind processes [53], activate in response to a scene of compassion or anger, but not to a scene of sadness.
Revised: On the other hand, regions such as R TPJ and R pars opercularis that are involved in emotional reactions and theory of mind processes [53], activate in response to a scene of compassion or anger, but not to a scene of sadness.

Original: Thus, chunks of active brain regions can be related to complex stimuli, andregions activate selectively in response to one or more categories of emotional stimuli, but not others.
Revised: Thus, chunks of active brain regions can be related to complex stimuli, and regions activate selectively in response to one or more categories of emotional stimuli, but not others.

Original: HCM’s Average Chunk size Correlates with Participants’ Age HCM can also be used to performmeaningful analyses at the population level.
Revised: HCM’s Average Chunk size Correlates with Participants’ Age HCM can also be used to perform meaningful analyses at the population level.

Original: Specifically, we find that HCM’s returned average chunksize per participant correlates significantly with age (Figure 6e).
Revised: Specifically, we find that HCM’s returned average chunk size per participant correlates significantly with age (Figure 6e).

Original: The older the participants are, thelonger are the chunks found in their data ( r = 0 .
Revised: The older the participants are, the longer are the chunks found in their data ( r = 0 .

Original: 23 , p ≤ 0 . 001 ).
Revised: 23 , p ≤ 0 . 001 ).

Original: This discovery is in line withfindings in the original study, which showed an increase in modularization of ToM and pain circuitsacross development [36].
Revised: This discovery is in line with findings in the original study, which showed an increase in modularization of ToM and pain circuits across development [36].

Original: To summarize, we applied HCM to learn chunks from a developmental fMRI data set.
Revised: To summarize, we applied HCM to learn chunks from a developmental fMRI data set.

Original: HCM enabledthe discovery of spatially and temporally correlated activation chunks that are theoretically andempirically meaningful.
Revised: HCM enabled the discovery of spatially and temporally correlated activation chunks that are theoretically and empirically meaningful.

Original: The resulting chunks can be linked to complex stimuli and offer directlyinterpretable insights into the structure and function of brain activity.
Revised: The resulting chunks can be linked to complex stimuli and offer directly interpretable insights into the structure and function of brain activity.

Original:  Discussion
Revised:  Related Work

Original: HCM extends upon decades of previous cognitive science and psychology research on chunking.
Revised: HCM extends upon decades of previous cognitive science and psychology research on chunking.

Original: Incognitive science, process models such as PARSER and competitive chunking were demonstratedto generate qualitatively similar chunks as in human sequence learning [54, 55].
Revised: In cognitive science, process models such as PARSER and competitive chunking were demonstrated to generate qualitatively similar chunks as in human sequence learning [54, 55].

Original: HCM is a rationalalgorithm that learns the underlying chunks when the sequence is generated from a hierarchicalchunking graph.
Revised: HCM is a rational algorithm that learns the underlying chunks when the sequence is generated from a hierarchical chunking graph.

Original: Therefore, the chunking criterion is no longer a heuristic but a rational learning strategy that enables hierarchical structural discovery.
Revised: Therefore, the chunking criterion is no longer a heuristic but a rational learning strategy that enables hierarchical structural discovery.

Original: On top of inheriting the merits of its predecessors,
Revised: On top of inheriting the merits of its predecessors,

Original: HCM generalizes the chunk learning principle to higher dimensional sequential domains such asvisual-temporal sequential data.
Revised: HCM generalizes the chunk learning principle to higher dimensional sequential domains such as visual-temporal sequential data.

Original: Another related line of research is program  induction,where explicit representations are acquired bysearching for programmatic structures that best explain observational samples [22], and consolidatingthese offline with library learning [56].
Revised: HCM relates to several other lines of research. One is program induction. In program induction, explicit representations are acquired by searching for programmatic structures that best explain observational samples [22], and consolidating these offline with library learning [56].

Original: However,primitive programs need domain expert knowledgeto specify; the relations and composition rules must adapt to the task settings and sensitively influencethe quality of retrieved representations.
Revised: However, domain expert knowledge is needed to specify the primitive programs; the relations and composition rules must adapt to the task settings and sensitively influences the quality of retrieved representations.

Original: The one-dimensional HCM is similar to natural languageprocessing methods such as Byte-Pair Encoding [57] or WordPiece [58], which were developed tooptimize sequential data compression instead of a cognitively inspired background and works withone-dimensional sequences.

Original: Other approaches to structure learning include unsupervised parsing[59], which learns a stochastic and-or graph from sequential data.
Revised: Other approaches to structure learning include unsupervised parsing [57], which learns a stochastic and-or graph from sequential data.

Original: HCM is distinct in adapting itsrepresentation granularity to discover bigger chunks from data without pre-specifying the structure.
Revised: HCM is distinct in adapting its representation granularity to discover bigger chunks from data without pre-specifying the structure.

Original: Our work has its limitations.
Revised: Another category of models to learn from sequential data are traditional sequence learning models including Hidden Markov Models (HMM), n-gram models and their variants to capture multi-scale sequential structure such as Hidden semi-Markov model [58] and hierarchical HMM [59]. The parameters of these models proliferate exponentially as a function of chunk length, implying memory inefficiency. Additionally, these models demand a structure specification before fitting parameters to the data. They also lack the adaptive recombination and reuse of pre-existing components.

Revised: The same issue is with neural network approaches to extract chunks from sequences [60–62].

Revised: Apart from lacking in interpretability, these models do not leverage the concatenation process observed in humans or reusing previously learned representations to construct new representations.

Revised: The principle of iterative merging of chunks has been used in compression algorithms, such as tokenizing methods in NLP, which were developed to optimize sequential data compression.

Revised: Tokenizing methods such as Byte-Pair Encoding [63] iteratively merges the most frequent pairs of chunks to build a vocabulary of a text corpus.

Revised: This objective is easy to compute but gives rise to ambiguous parses of the text (e.g. [AB, C] /

Revised: [A, BC]).

Revised: To minimize parse ambiguity, WordPiece [64] merges chunks that increase the likelihood of the corpus the most.

Revised: However, the objective of WordPiece is expensive to compute.

Revised: HCM circumvents the problem of computing the global sequence likelihood by instead maximizing the local chunk continuation likelihood.

Revised: The computational efficiency of

Revised: HCM makes it a plausible cognitive model of chunking as well as a promising method for NLP.

Revised: Probabilistic context-free grammars (PCFGs) are related to HCM in that they use trees as a representational form of sequences.

Revised: The parse trees of PCFGs denote production rules, such as S → NP + VP.

Revised: These production rules define how abstract syntactic units (non-terminals), such as a noun phrase and a verb phrase, are instantiated into a concrete string of words (terminals) to compose a sentence, such as ‘we wrote the paper’.

Revised: In comparison, the generative tree of HCM denotes statistical relationships among concrete chunks, such as ‘we’-‘wrote’-’the paper’.

Revised: Extending HCM to represent abstraction is an exciting future direction, on which avenue the comparison to PCFGs will be instructive.

Revised:  Discussion

Original: At the moment, HCM learns representations from semi-high dimensionalsequential data.
Revised: Our work has its limitations. Currently, we fix the memory decay and the deletion threshold parameters to a priori plausible values. In future work, these parameters could be adapted online based on environment volatility. Another limitation is its scalability: at the moment, HCM learns representations from semi-high dimensional sequential data (i.e., currently between 1 to 625 dimensions).

Original: We are actively looking into generalizing this algorithm to higher dimensional datadomains by combining it with existing neural network approaches or computer vision algorithmssuch as coherent point drift [60] or normalized cuts [61] to allow for the learning of ambiguous andhigh dimensional chunk exemplars.
Revised: We are actively looking into generalizing this algorithm to higher dimensional data domains by combining it with existing neural network approaches or computer vision algorithms such as coherent point drift [65] or normalized cuts [66] to allow for the learning of ambiguous and high dimensional chunk exemplars.

Revised: It is also possible to combine HCM with the compressed representation, such as the hidden activity of an auto-encoder to process and learn the structure from downstream representation.

Original: Another limitation is that HCM learns one type of hierarchy ofcompound representations.
Revised: In this work, HCM learns one type of hierarchy of compound representations.

Original: However, we can show that HCM can be generalized to not only learnsimple chunks but also chunks in projected spaces and thereby generalize between two chunks thatcontain the same motif (for example, “12221212” and “34443434”; see SI for detailed results).
Revised: However, we can show that HCM can be generalized to not only learn simple chunks but also chunks in projected spaces and thereby generalize between two chunks that contain the same motif (for example, “12221212” and “34443434”; see SI for detailed results).

Original: In thefuture, it might be worthwhile to further combine our approach with others amongst a taxonomy ofrepresentational hierarchies.
Revised: In the future, it might be worthwhile to further combine our approach with others amongst a taxonomy of representational hierarchies.

Original: HCM also opens up other application directions.
Revised: HCM also opens up other application directions.

Original: One direction is integrating HCM with deep neuralnetwork approaches as an interface between human understanding and distributed computation.
Revised: One direction is integrating HCM with deep neural network approaches as an interface between human understanding and distributed computation.

Original: Learning hierarchies of coherent activations from intermediate hidden units has the potential to revealneural networks’ underlying computation structure.
Revised: Learning hierarchies of coherent activations from intermediate hidden units has the potential to reveal neural networks’ underlying computation structure.

Revised: Furthermore, it is also possible to equip HCM with additional top-down encoded representations, for example, by pre-training on other sequences or by adjusting the chunks by hand before the training starts.

Original: Another direction in neuroscience or behavioralresearch is to learn chunks of tagged animal movements that enable insights into the emergence ofbehavioral structure [62].
Revised: Another direction in neuroscience or behavioral research is to learn chunks of tagged animal movements that enable insights into the emergence of behavioral structure [67].

Original: Finally, finding patterns that form as a cognitive unit is a vital task forinfants to learn about the structures of the world and resembles the process of formulating a scientifictheory from observation [63].
Revised: Finally, finding patterns that form as a cognitive unit is a vital task for infants to learn about the structures of the world and resembles the process of formulating a scientific theory from observation [68].

Original: HCM can function as one means to come up with theories of theworld by observation, ready for experimental interventions, or active learning to delineate the causalstructure within [64].
Revised: HCM can function as one means to come up with theories of the world by observation, ready for experimental interventions, or active learning to delineate the causal structure within [69].  Conclusion

Revised: We have proposed a hierarchical chunking model (HCM) that learns chunks from non-iid sequential data with a hierarchical structure.

Revised: HCM starts out learning an atomic set of chunks to explain the sequence and gradually combines them into increasingly larger and more complex chunks.

Revised: The output of the model is a dynamical graph that is a trace of the evolving representation.

Revised: The resulting representations are easy to interpret, and flexibly reusable.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]
Revised: [N/A]

Original: As this work introduces an algorithmic approach, it is not clear to us with the potential negative societal impact it can induce.
Revised: As this work introduces an algorithmic approach, it is not clear to us with the potential negative societal impact it can induce.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] A proof sketch is described in the main paper, while the full proof can be found in the supplementary material.
Revised: [Yes] A proof sketch is described in the main paper, while the full proof can be found in the supplementary material.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] They will beincluded in the supplementary material and publicly available.
Revised: [Yes] They will be included in the supplementary material and publicly available.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] Details on the experiments will be included in the supplementary.
Revised: [Yes] Details on the experiments will be included in the supplementary.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [N/A] Results obtained run without special computing resources.
Revised: [N/A] Results obtained run without special computing resources.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [Yes] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [Yes] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [No] From our awareness, no personally identifiable information is present in any of the assets.
Revised: [No] From our awareness, no personally identifiable information is present in any of the assets.

Original: If you used crowdsourcing or conducted research with human participants...
Revised: If you used crowdsourcing or conducted research with human participants...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] We used existing data from experiments conducted in previous work.
Revised: [N/A] We used existing data from experiments conducted in previous work.

Original: This information can be found in the original articles.
Revised: This information can be found in the original articles.

Original: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: This question is not applicable to this project.
Revised: This question is not applicable to this project.

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A] Since no human experiment is conducted for this project and we used experimental data from other published and publicly available work, this question is not applicable to us here and can be found in the original articles.
Revised: [N/A] Since no human experiment is conducted for this project and we used experimental data from other published and publicly available work, this question is not applicable to us here and can be found in the original articles.