Dynamical Equations With Bottom-up Self-Organizing Properties Learn Accurate Dynamical Hierarchies Without Any Loss Function
Authors: Danilo Vasconcellos Vargas
Tham Yik Foong
Heng Zhang
Venue: Submitted to ICLR 2023 (2023)
Keywords: self-organization
dynamical systems
continual learning
dynamical hierarchy
adaptation

Abstract: Self-organization is ubiquitous in nature and mind. However, machine learning and theories of cognition still barely touch the subject. The hurdle is that general patterns are difficult to define in terms of dynamical equations and designing a system that could learn by reordering itself is still to be seen. Here, we propose a learning system, where patterns are defined within the realm of nonlinear dynamics with positive and negative feedback loops, allowing attractor-repeller pairs to emerge for each pattern observed. Experiments reveal that such a system can map temporal to spatial correlation, enabling hierarchical structures to be learned from sequential data. The results are accurate enough to surpass state-of-the-art unsupervised learning algorithms in seven out of eight experiments as well as two real-world problems. Interestingly, the dynamic nature of the system makes it inherently adaptive, giving rise to phenomena similar to phase transitions in chemistry/thermodynamics when the input structure changes. Thus, the work here sheds light on how self-organization can allow for pattern recognition and hints at how intelligent behavior might emerge from simple dynamic equations without an objective/loss function.

Under review as a conference paper at ICLR 2023
DYNAMICAL EQUATIONS WITH BOTTOM-UP SELF- ORGANIZING PROPERTIES LEARN ACCURATE DYNAMICAL HIERARCHIES WITHOUT ANY Loss FUNCTION
Anonymous authors Paper under double-blind review
ABSTRACT
Self-organization ubiquitous nature and mind. However, machine learning and theories of cognition still barely touch the subject. The hurdle is that gen- eral patterns are difficult to define in terms of dynamical equations and designing system that could learn by reordering itself is still to be seen. Here, pro- pose a learning system, where patterns are defined within the realm of nonlinear dynamics with positive and negative feedback loops, allowing attractor-repeller pairs to emerge for each pattern observed. Experiments reveal that such a sys- can map temporal spatial correlation, enabling hierarchical structures to be learned from sequential data. The results are accurate enough to surpass state- of-the-art unsupervised learning algorithms in seven out of eight experiments as well as two real-world problems. Interestingly, the dynamic nature of the system makes it inherently adaptive, giving rise to phenomena similar to phase transitions in chemistry/thermadyannie when the input structure changes. Thus, the work here sheds light on how self-organization can allow for pattern recognition and hints at how intelligent behavior might emerge from simple dynamic equations without any objective/loss function.
INTRODUCTION
self-organization is present in diverse scientific fields, from biology (Misteli, 2007 Deglincerti et al.
2016 [Saaii 2013) to neuroscience (Linsker 1988 Tognoli & Kelso 2014 Imam & L. Finlay 2020
Schoner & Kelso 1988), chemistry (Montalti et al. 2017; Lehn 2002a b) and physics (Haken 1975 Wickman & Korley 1998 Tersoff et al.) 1996; Haken 1977). It shows how order can arise intrinsically from system. It is of interactions that allows for the emergence of patterns and is responsible for complex behavior from simple interactions (Kauffman et al. 1993 Haken] [977). Albeit the ubiquitous presence of seff-organization in nature and in the brain, it is unknown how self-organization can lead to intelligence. For this reason, theories of intelligence rarely use the concept in their development. The free energy principle (Friston) 2010; (2009) and reinforcement learning paradigms (Sutton & Barto 2018 Mnnhh et al. 2015) Schrittwieser et al., 2020) define a top-down view of learning based on objectives that satisfied locally globally. However, from bottom-uu perspective, it is still barely understood how Hebbian learning (Hebb) 2005) Magee & Johnston 1997) and other neuron behaviors allow for top-down theories of intelligence to emerge. In fact, there is strong evidence the brain does not behave as computer but more self-organizing system (GRAY 1987, Eechhrrn al. 1988). In this paper, we show how the learning of patterns can be achieved by Hebbian and anti-Hebbian learning dynamics, linking between Hebbian learning and top-down thcories of intelligence (Hebb) 2005).
The recent success of machine learning, similar to the current theories of intelligence, is mostly given to optimization-brat based deep learning algorithms. While deep learning utilizes optimization and loss functions (objective functions) to leamn the model's parameters and improve in the task at hand, self-organizatiin existence in machine learning is mostly limited to Self-Organizing Map (SOM) variations (Kohonen, 1982 Chang et al. 2020: Reker al. 2014). Such SOMs only

---

Under review as a conference paper at ICLR 2023
employed in clustering and dimensional reduction tasks, they lack the ability find patterns in data required for further processing and acting on the environment.
Here, inspired by many successful modelings of neuron behaviors based on dynamical equations composed of attractor dynamics (Tognoli & Kelso 2014 Wills et al. 2003 Spalla et al. [022]] Ooi et al., 2018), show how system of dynamical equations can give rise to order and represent pat- terns. Our proposed system is arguably more biologically plausible, and it is also shown to more accurate and adaptive than state-of-the-art unsupervised algorithms. In fact, sets up a foundation for a new paradigm in machine learning solely based on self-organization from dynamical equa- tions, namely Self- Self--rgaad Organizing Dynamical Equations, which are inherently accurate and adaptive. We propose Hierarchical Temporal Spatial Feature Map (TSFMap), learning system implement- ing the Self-Organizing Dynamical Equations paradigm. It creates space which distances in reflect the temporal correlation between input variables. A simple clustering in this self self-organized space reveals that the representation learned is very accurate. Adaptation comes from the fact that the proposed system, Hierarchical TSFMap, couples its internal dynamics with the input, resulting in patterns encoded as emergent attractor-repellers at equilibrium. Consequently, alterations in the underlying structure of the problem result in different equilibrium with new ettractor-repellen trig- gering an inherent adaptation when the problem changes. Interestingly, structural changes in the environment cause in Hierarchical TSFMap a phenomenon very similar to phase transition observed in thermodynamics, and chemistry, among other areas (Fig.
Phase Transition of Hierarchical TSFMap
:.allaation
Equi curn
Eig are Baae
Second
20-
Dall, Reaave Divance
Reattn Drntnno
0.04
0.00
10-
0.02
6-
0.01
0
100000
2000C0
300000
430000
S00000
600000
100000
200000
300000
400000
500000
600000
Timeriip
Timestipp
Figure 1: Hierarchical TSFMap's phase transition. A phenomenon similar to phase transition takes place the proposed algorithm when the underlying structure of the problem changes. (a) Lines indicate the relative distance for all weight pairs. (b) The average rate of change for all weight pairs' distances (a). Random initialized weights start to form patterns with respect to the input and enter an equilibrium state. Once the problem's 's data structure is altered, Hierarchical TSFMap automatically adapts its weights. Subsequently, weights enter another equilibrium state.
In this paper, Hierarchical TSFMap is evaluated in one of the hardest types of patterns, e.g., recog- nition of dynamical and imbalanced hierarchical patterns present in sequential data. The problem of learning the hierarchical relationships from sequential input is a challenging unsolved one (Uddén et al. 2020). This becomes even harder when the problem structure is dynamic, e.g., variable cor- relations change over time. Since any information can be serialized, the pattern recognition over sequences is a general one that can be applied ubiquitously to any type of serialized data. Albeit the difficulty of the task, Hierarchical TSFMap provides, perhaps surprisingly, near-optimal solutions to more than half of the problems. Lastly, we have demonstrated that Hierarchical TSFMap can extract hicrarchical structures from sequential data generated from two real world networks: (1) Zachary's s karate club network and (2) Lusseau's bottlenose dolphin social network.
RELATED WORK
recent work (Vasconcellos Vargas & Asabuki 2021) demonstrated how a self-organizing system called SyncMap, can learn features from sequences using dynamical equations alone (e.g.. without any type of optimization). Here we go beyond this work on simple chunks to show how dynamical equations that self- self-organize compose paradigm and can be used to deal with challenging hierarchi- cal structures and imbalanced problems. fact, the experiments suggest that Hierarchical TSFMap can deal with dynamical variations of the problems with little difficulty.
2

---

Under review as a conference paper at ICLR 2023
Community detection in complex networks can also extract hierarchies (Clauset et al. 2007 Corominas-Murtte et al.) 2013). Although the input data, and therefore the problem, is different from the one seen here, sequence data and complex networks interchangeably convert to one other (e.g.. via an adjacency matrix from transition probabilities or a random-walk over a complex network). This reveals Hierarchical TSFMap's connection with complex networks. Having said that, the similarities stop here as both the objective and methodology differ. Complex networks' al- gorithms usually maximize a metric on the network to find communities (while here only dynamical equations are used). Given the nature of optimization problems, such models are inherently not able deal with any possible dynamies of the network.
A closely related body of work is that of learning an embedding that also preserves the variables' correlations. Word2vec, specifically, can create embeddings that preserve the relationships of neigh- boring variables based on their context (Mikolov et al.) 2013). However, we showed here that hi- erarchical structure does not seems to preserved in this embedding. To make matters worse, adaptation is tricky with deep neural networks (it is in direct conflict with techniques that make them learn well such as decreasing learning rate) and there is no inherent system that can adapt to changes in the environment.
HIERARCHICAL TEMPORAL SPATIAL FEATURE MAP
Here demonstrated the general workflow of Hierarchical TSFMap, which is composed of three steps: (1) input encoding. (2) the dynamics process, and (3) the hierarchical chunking phase. Refers to Fig. for an overview Hierarchical TSFMap's workflow.
Input Encoding, We first encoded the sequence data generated from the problem into a specific type of input before feeding into our model. Given a sequence of data X with T be the sequence length, all unique items in X represent different states and we denote the total number of unique states using n. We converted input sequence into a sequence of state S, = (S1, ... Sz). S, be a vector of state values in time I. We set 8; 81,1, 82,1, ,8n.t with E S, and total number of unique state n as the dimension of states. For 8, (0, 1)" Ei= &1,, 1, simulating the activation of neurons.
The input encoding is modeled number states:
exponentially decaying vector 22, sharing the same size as the
Bi,ta x -011((t-t)) - < m X tstep otherwise
Di.t
0,
in which la is the most recent state transition state 8;. State transitions happen every (slep step and variables with time of activation greater than m x tstep are set to O. Thus, only the last m states activated are remembered as F,,t O, and set m to 10. At each time step, Fi, will be fed into the model as a spike encoded input.
Dynamics. Hierarchical TSFMap represents patterns with a formation of attractor-repeller pairs for each identified one. The space made of attractor-repeller pairs defines a temporal to spatial mapping of variables' correlation, and is given the name a space. There no optimization or objective function, the dynamical system merely self orfaairgasszes to the input, following positive and negative feedback loops. Experiments suggest that the distance between patterns in the learned a space proportional to the strength of their temporal correlation.
To begin the dynamic process, all inputs Fz have set of corresponding weights wi,, initialized to a random position in a space Wi, E R$ at the beginning, with & be a hyperparameter that defines the dimension of the map, or the degrees of freedom that organize the weights.
Hierarchical TSFMap defines positive and negative feedback loops related to which state variables activate together (synchronous behavior). In each iteration f, state variables that activate or deac- tivate together are first included into PS or NS sets respectively. Here, PS or NS refers to: (I) activated and recently activated input set PS, and (2) non-recently activated input set N.S. Inputs with value greater than or equal to 0.1 are considered an element of PS₁ otherwise, inputs are a member of S,S Thus, the cardinality of both sets are greater than one, where PSS. > 1 and [NS,] > 1, the centroid of both sets computed as follows (otherwise no update is made in this iteration):
define PS, fi) > 0.1} and NS, {i] < 0.1). If and only if
3

---

Under review as a conference paper at ICLR 2023
Input Encoding
Dynamics
Hierarchical Chunking Phase
sequence Input [ABCACDEFDF...LKKKM)
one-hol vector encoding
15
45.00 4
0
44
spike encoded input
45.00
-1
Figure 2: Hierarchical TSFMap's workflow. (a) A sequence of variables is converted to spikes that decay exponentially. (b) Hierarchical TSFMap's weights initialized randomly with weight for each possible variable. Every time step the spike encoded input is presented to the algorithm which self-organe self-organizes to it. (c) The space stores the temporal relationship of variables spatially. To extract this hierarchical information into dendrograms, a simple hierarchical clustering is used.
ΣiEPs, Wi,t [PS|
ΣIENS, Wi,t INS]
CP:
where cpt and en2 are the centroids of PS; and NS, respectively. With ep; and en,, we determine the distance of all weights to cpN and en, as dey ||wa, - cptll and den ||Wa,r en.ll respec- tively, using Euclidean distance metric. Subsequently, state variables are updated (Fig. 3 by either attracting ept (activated states) or repelling from en, (inactive states).
M1 (a(ppe-wij wi) INs, (i)
(2((w; enu)
us(w; cpt)
01 = Ove, + [Ps) (1)
(3)
dep
d
d²
Wi,++1 Wi.t + 001i10+1
(4)
where 1e is the learning rate, 0 0.999 is the velocity decay and is the velocity. L (i) (or LNs, (i)) is the indicator function that maps elements the subset PS; (or PN,) to one, and all other elements to zero. The term 1 Lps, (i)
(i (cpe-wer
acts as attraction force between activated
dep
variables; On the other hand, 1 1s (i) Pa(en, den
acts on in-active variables as an Ma(m;
dip
attraction force between in-active variables and repulsion foree from activated variables. μ4 6, #2 3, and μ3 = 2 are the coefficients that control the strength of the attraction and repulsion forces, tuned from range of values that were tried they produced the best results. This dynamic law governed by the Hebbian and meti-Hebbian learning dynamics is arguably analogous to force- directed algorithm (Fruchterman & Reingold 1991), or even a gravity and anti-gravity force. We then specify the attraction force (FI) with 1ps, (i) indicating that F1 only appears among the activated state variables. For the inactive ones (1 (i)), we specify the attraction force between inactive variables (F2) and repulsion force from activated variables (F3). Weights (e.g., state variables) are finally updated by Eq. 4 Each update iteration ends by scaling all weights to a fixed size space. Furthermore, the velocity parameters create inertia to avoid the instability caused
4

---

Under review as a conference paper at ICLR 2023
by instantaneous weight update. At the end of the iteration, all values of the updated weights Witt1 normalized: 1fi,++1 = mex((w) to keep them in a relative space. Overall, the dynamical equation was found to work well in preserving the hierarchical structure of the corresponding input, despite its simplicity. The process above iterates until the final time step. Theoretically, final time step is not required to be defined, as this is an adaptive system.
cP
cn
= 00s,t (ps(())
Mn(cpt wi)
+I (i)
[2(ww; cni) den
Ma(w;; cpi)
dep
dg
F2
P3
Witt+1 = Wif + 000j,1+1
Figure 3: Hierarchical TSFMap's main dynamical equations and emerging behavior. The cir- cles with arrows represent the emergent attractor-repeller pairs (or just repellers); a consequence of the dynamical equations. Regarding the equations, Vi, is the velocity with 0 as its decaying factor at time while Wi,t the weight. The positive centroid (cpt Eiers, wt.r/|PS)) at- tracts recently activated weights (F1) and repels inactive weights (F3); while the negative centroid ene = EiENS, ( only repels inactive weights (F2). Distance all weights to cpt and en, are computed as dep = || cpell and den = ||wi, enell respectively using Euclidean distance metric. With as learning rate, w is subsequently updated with regard to
Hierarchical Chunking Phase. Methods like hierarchical clustering can produce a dendrogram. Yet, dendrogram does not promptly reveal which level of the hierarchy should be viewed as a collection of meaningful chunks. To solve that, we use Hierarchical Chunking Phase to extract the information of how variables are chunked together on each level of the hierarchy. With Hierarchical Chunking Phase, the proposed algorithm produces an L x N matrix, which refers to the output Y, including the predicted class label, with L being the total levels of hierarchy (see Appendix. for the implementation detail Hicrarchical Chunking Phase).
EXPERIMENTS AND RESULTS
We investigate the performances of Hierarchical TSFMap and the baselines (SyncMap, Word2vec, Modularity Maximization, and transition probability matrix) in two types of hierarchical problems: imbalanced hierarchies and dynamical hierarchies (hierarchies that change during experiments). Each problem is represented by graph preserving the hierarchical structures. Each sequence ob- served by the algorithms is derived from a random walk in the above-mentionel graph, with de- creasing transition probabilities when variables pertain to different chunks. Variable is placed into the sequence input whenever random- walker travel to one. Refer to Appendix. B for the implemen- tation of the baselines and Appendix. C for the details of graph-to-inpe ien-inuuu-u-ueceeeeeeeeeeeeeeeee generation.
4.1 IMBALANCED HIERARCHICAL STRUCTURE
Real-world events rarely share equal possibilities, suggesting that most of the real-world structures arguably imbalanced. We first introduce three environments quantify the performances of the models on imbalanced data: Imbalanced Hierarchy (IH), Hierarchy with Branches (HB), and Im- balanced with Extra Hierarchy (IEH). These are generated from three graphs indicating the desired imbalanced structures (Fig. 4. The environments create a distribution of variables where the occur- rence of some variables more frequent than others. detail, IH defines a sequence where single chunk contains much more variables than other chunks, while HB is hierarchical structure where a branch has a shallower hierarchy with fewer nodes. With more complexity, IEH has a branch with deeper hierarchical structure.
Results shown Fig. reveal that Hierarchical TSFMap surpasses all other algorithms in imbal- anced hierarchical problems. This suggests that self-organization alone can, perhaps surprisingly, represent such complex structures. Hicrarchical TSFMap's 's behavior also resembles the bottom-up behavior observed in natural self-organization processes (Simon 1991) (Fig. 5). Weights tend to
5

---

Under review as a conference paper at ICLR 2023
form chunks that belong to the lower level of hierarchies first. These chunks then proceed to cluster into bigger chunks that belong to the next level of the hierarchy. is important to note that weights manoeuvre and form chunks around the surface of k dimensional n-sphere. Such an n-sphere composition allows for negative centroids (cnz) mostly at the center of the n-sphere and for positive centroids (cpt), when correctly clustered, to be at the border. The result is uniform negative feedback (F2) away from the center and non-uniform positive feedback perpendicular to the center (F1). Notice that, since all weights are scaled back feedback F2 is canceled, bringing the system to equilibrium (only F1 and F3 move weights re- spectively close and far apart from each other on the border of the n-sphere; proportionally to their temporal correlation). Important to notice that F2 allows for a degree of freedom (e-g., they not fixed at the border of the n-sphere) for weights to move around while keeping them mostly stable in equilibrium. See Fig. [3 for the visualization of Hierarchical TSFMap's 's dynamic in Imbalanced Hicrarchical Structure problems.
fixed size space, the negative
HB
34-
37-
1909
2000
21000
IEH
Modiiaray Macimization
6.3620000
Worthbee
0.7800
0.900008
Syn:Map
0.718007
035-00)
0.64:0009
3.4
TP Matiis
020000033
0.49.0.12
Proposed: Hicrarchical TSPMap
0.76etus
1,tt/U
0.71:0.66
100000
Figure 4: Experiment setting and results of imbalanced hierarchical structure (HB). (a, b, c) The graphs of hierarchical structures (left) of the environments IH, HB and IEH used to generate input sequences. Leaf nodes represent variables, and sibling nodes with the same color coding belong to the same chunk linked by the nodes in red. A chunk can contain either child chunks variables. The plot (right) illustrated the Normalized Mutual Information (NMI) score progression over time. The solid lines and shade represent the mean and standard deviation of the NMI score over 30 instances. Results are smoothed by a ten-timestep moving average. (d) The NMI score Hierarchical TSFMap compared baselines in table view. The result indicates that Hierarchical TSFMap performs the best in all imbalanced hierarchical structure experiments.
Weights Inrializaiin
Lower-level organ (ctusters of deeper levels the hiararohy)
Higher-level organizetion (clusters of top levels the hikrarehy)
Figure 5: Analyzing Hierarchical TSFMap's Dynamics. Weights are initialized randomly three- dimensional space (k: 3). The weights progressively self-organize into six smaller chunks, these smaller chunks proceed to then merge into two bigger chunks respectively. This emerging behavior reveals the bottom-up self-organization properties of Hierarchical TSFMap, in which individual components gradually aggregate to form more complex systems iteratively.
To understand the reason for the accurate results from Hierarchical TSFMap when compared to other embedding- based learning systems such as Word2vec and SyncMap, we compared their learned embeddings/maps in Fig. 6. Specifically, the space learned by Hierarchical TSFMap shows it can

---

Under review as a conference paper at ICLR 2023
learn the temporal correlation between variables. In fact, the learned space respects both local and global temporal correlations. Word2vec is shown able to identify local chunks with substantial accuracy, but global relationships not preserved in its embedding. The rationale behind this lies in how Word2vec learns, e.g., by using local contextual information which less predictive of global contexts/relationshiis. SyncMap, However, there seems to be a scaling problem in how local chunks are clustered, that is, local chunks tend overlap with each other, making it difficult to accurately identify the lower level structure of a given hierarchy. Regarding the TP matrix, the precision of the transition probability's table affected strongly by the standard deviation of variables. This problem further increases cases with smaller chunks that have a smaller probability of activating, justifying the poor performance. This is also the case of Modularity Maximization which is also based on the transition probabilities. Moreover, researchers have already shown that the used modularity metric tends to overestimate cither the global context or local context chunk (Sun 2016).
the contrary, can identify high-level chunks precisely.
Sigma
Imbalanced Hierarchical Structure
Hierarchy
(HB) Environment
Word2vec
SyndMap
Figure Comparison of the Learned Representation of Hierarchical TSFMap, SyncMap, and Word2vec. Here we show (b) the 2D learned a space of Hierarchical TSFMap, (c) the learned map of SyncMap, and (d) the word embedding of Word2vee; together with the dendrogram formed corresponding to their pattern in (a) HB environment. Chunks in the lowest level of hierarchies are color-coded. As shown in (b), Hierarchical TSFMap can produce a pattern that matched the distri- bution of variables. (c) The word embedding learned by Word2vec can identify local chunks, yet failed to identify chunks beyond the lowest level hierarchies. (d) SyncMap successfully identi- fies chunks global scale; however, local chunks overlapped, which increases the difficulty to distinguish them.
4.2 DYNAMIC HIERARCHICAL STRUCTURE
Real-world problems are constantly changing. Yet, humans adapt to it almost effortlessly while understanding complex hierarchical relationships (Conway & Christiansen 2001 Werchan et al.) 2015 Collins & Frank 2013). To quantify the performance of algorithms under problems with hi- erarchies that change over time (dynamical hierarchies), five problems with different characteristics are defined (Fig. 7 In detail, DIH (Dynamic Imbalanced Hierarchy) starts with HB's 's imbalanced hierarchical structure and then merges two chunks into a new branch. DCH (Dynamic Chunk Hi- erarchy) splits two chunks into three chunks over time. EC2EH (Extra Chunk to Extra Hierarchy) shifts from shallow hierarchical structure with two levels levels. EH2EC (Extra Hierarchy to Extra Chunk) is the reverse of EC2EH, specifically designed to test what happens when hierarchical structures decrease in the number of levels. DCS (Dynamic Chunk Swap) swaps chunks of level two to form different structure. The distribution of variables in dynamic environments shifts over time (Fig. 9 halfway through the input sequence, when T 7/2 with total number of input 600000 (This scheme applies to all the following dynamic problems). Note that the number of variables remains consistent despite the changes. The dynamic problems aim evaluate how models adapt the latest changes in the environment.
deep hierarchical structure with four
7

---

Under review as a conference paper at ICLR 2023
Results show that Hierarchical TSFMap can adapt well in dynamic environments, achieving near- optimum solutions in 4 out of environments. The experiments here extend the results with im- balance hierarchies to demonstrate that the good performance is not only limited to static problems. Moreover, the rapid remapping of the weights when an instantaneous change occurred environ- ments is analogous the attractor dynamics of place cells, they switch between representations to respond to the changes in environments (Wills et al. 2005
DEH
DCH
11.8
EC2EH
EN2EC
DCs
DOH
DCH
[C3EH
Wellery Manricaton
875-200
Westbe
bxt.0:11
6x6.317
TP 34
653e104
612s6:1
(3te(33
Eripontt lltmrrttiaal MS1
Figure 7: Experiment setting and results of dynamic hierarchical structure. (a, b, c, d, e) The graphs of hierarchical structures (left) of the environments DIH, DCH, EC2EH, EH2EC, and DCS used to generate input sequences. The blue arrow implies that the distribution of variables changes through- single experiment. The plot (right) illustrated the NMI score progression over time. The proposed method, Hierarchical TSFMap, is shown to be the only one capable of quickly adapting to structural changes. (f) The table showed the NMI score of Hierarchical TSFMap compared to baselines. The proposed algorithm performs the best in nearly all dynamic hierarchical structure experiments.
In fact, when compared with other methods, Hierarchical TSFMap shows a great performance be- fore and after the change in structure (Fig. [7. Much of the great performance derives from phase transitions that happen naturally in Hierarchical TSFMap when the input structure changes (Fig. [T]. See Fig. [4 and Fig. 15 for the visualization of Hierarchical TSFMap's 's dynamic in Dynamic Hi- erarchical Structure problems. All the other methods face different but related problems related to adaptation. TP matrix and Modularity Maximization become imprecise when the underlying probabilities change throughout the test. Word2vec has learned weights that become, after the change, local minimum which is hard overcome and bias the learning toward a previously learned nearby region. SyncMap was not designed for hierarchies (reflected by the relatively poor performance even in static problems). Increasing the difficulty of hierarchical problems with dynamical structural changes only makes matters worse. Additionally, although initialized higher dimensional weight space, the rank SyncMap's weight matrix con- verged to given enough time, where p(W) with W being the weight matrix. This indicates that SyncMap's 's dynamic can restricted in mie-dimnsioonl space. The weight matrix of Hicrarchical TSFMap however, can retain its high dimensionality, where 1 S p(W) & (Fig.
based transition probabilities which
4.3 REAL WORLD SCENARIOS
In this section, we consider two network datasets with interpretable hierarchical structures: (1) Zachary's karate club network and (2) Lusseau's bottlenose dolphin social network. Despite being
8

---

Under review as a conference paper at ICLR 2023
well-establish benchmarks, the hierarchical information of both networks is seldom explored in depth. Therefore, we investigate the hierarchical structure extracted from Hierarchical TSFMap, utilizing the input sequence generated from the networks (refers to Appendix. for the experiment details).
The ground truths provided by Girvan & Newman (2002); Zachary (1977) and Lusseau et al. (2003)
are only available for one level of the hierarchy; Thus, the interpretation of the remaining hierarchi- cal structure relies on the visualization of the representation space. The table in Fig. 8 showed the NMI score of the most significant chunk predicted by the models compared to the ground truth in both tasks. The result showed that our models could configure their weights to match the ground truth in most instances, reflected by the relatively high NMI score. Furthermore, Fig. (c) demon- strated the weight space of Hierarchical TSFMap for the Karate club network. Two chunks are formed on the most significant level of the hierarchy, aligning with the ground truth where the members of the Karate club were eventually split into two factions. When looking deeper into the hierarchy, smaller groups of members and less social individuals are formed into their own chunks, showing signs hierarchical structure in the network. Lastly, Fig. [6 and Fig. 17 displayed the representation space of Hierarchical TSFMap, SyncMap, and Word2vee in both tasks.
NMI soore of Hiersrchical TSFMap compared baselines
Zachary's s karata club network
Baselines
Karate club network
Dolphin social network
Modularity Maximization
0.59g0.000
0.48201.7
WordZvee
0,71+0.18
0.03=0.00 02
SyncMap
0.7720.13
0.48g0.04
TP Matrix
0.29
0.2620.00
Proposed: Hierarchical TSFMap
0.866=0.06
0.78=00.17
Hiararchical structure predicted by Hierarchical TSFMap
0.70
20
10
6P)
Figure 8: Hierarchical TSFMap can extract the hierarchical structure of real-world networks. (a) The NMI score of Hierarchical TSFMap compare to baselines on Karate club and Dolphin social network. (b) The ground truth of the Karate club network provided by Zachary (1977). (c) The dendrogram shows the hicrarchical clustering of weights. where the red dashed lines are the cut- off. The weight space 5) describes how weights are chunked in three of the most significant hierarchies, visualized using multidimensional scaling.
5 CONCLUSION
We show here how dynamical equations alone are enough to create self-organizing systems capable of learning complex structures such as imbalanced and dynamical hierarchies. In fact, experiments have shown that these dynamical equations have two emerging properties that are typical of self- organization systems: (a) bottom-up organization and (b) presence of phase transition. Moreover, we propose Self- Pega-Organi Dynamical Equations as a paradigm for machine learning together with an algorithm that implements it (Hierarchical TSFMap). Results show that, perhaps surprisingly, Hierarchical TSFMap is both more accurate and more adaptive than state-of-the-art algorithms in seven out of eight tasks.
This work also has implications in many areas such as cognitive science and neuroscience, shed- ding light how self-organization circuits can be established as fundamental mechanism in the brain. Results here suggest that the learning of chunking and hierarchical structures can be done by self-organizing circuits with Hebbian and anti- anti-Hebbian plasticity. Thus, it reveals relationship of Hebbian theory with brain orf-nnaaaizttion and sets up the stage for novel cognitive theories to emerge, using self-organization a principle rather than byproduct.
9

---

Under review as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
We have made the experiments easily reproducible by providing the following: (1) The parameters
setting to reproduce results available in Appendix, B (2) The parameters setting and implemen-
tation details of our model are available in the main text and Appendix. [ (3) We have provided
extensive details regarding the setup of the environment (generating input sequence) in Appendix, C (4) Code to reproduce our environments from scratch. the proposed model, and the baselines will be submitted as supplementary material and made available through GitHub after acceptance. the result for Imbalanced Hierarchical Structure and Dynamic Hierarchical Structure experiments was obtained from at least independent experiments. Values different experimental groups are expressed the mean + s.td. < 0.05 was considered statistically significant. The p-values of each experiment are shown Table 1.
REFERENCES
Ziv Bar-Joseph, David K Gifford, and Tommi S Jaakkola. Fast optimal leaf ordering for hierarchical clustering. Bioinformatics, 17(suppl.1):S22 S29, 2001.
Li-Chiu Chang, Fi-John Chang, Shun-Nien Yang, Fong-He Tsai, Ting-Hua Chang, and Edwin Her- ricks. Self-organizing maps of typhoon tracks allow for flood forecasts up to two days in advance. Nature Communications, 11, 04 2020. doi:
Jianjun Cheng, Mingwei Leng, Longjie Li, Hanhai Zhou, and Xiaoyun Chen. Active semi- supervised community detection based must-link and cannot-link constraints. PloS one, (10)ae1 10088, 2014.
Aaron Clauset, M. E. J. Newman, and Cristopher Moore. Finding community structure in very large networks, Phys. Rev. E, 70:066111, Dec 2004. doi: 10. 10.1103/Phy.Re22.7006666 1..006611... URL
Sttps:///inke aps org. 2o1/00 1103/P10P0 203//hyseev....m.. 70.066111
Aaron Clauset, Cristopher Moore, and Mark E. J. Newman. Structural inference of hierarchies in networks. In Edoardo Airoldi, David M. Blei, Stephen E. Fienberg, Anna Goldenberg, Eric P. Xing, and Alice X. Zheng (eds.), Statistical Network Analysis: Models. Issues, and New Directions, pp. 1-13, Berlin, Heidelberg, 2007. Springer Berlin Heidelberg. ISBN 978-3-540- 73133-7.
Anne GE Collins and Michael J Frank. Cogmitive control over learning: creating, clustering, and generalizing task-set structure. Psychological review, 120(1):190, 2013.
Christopher M Conway and Morten H Christiansen. Sequential learning in non-human primates.
Trends cognitive sciences, 5(12):539-555 2001.
Bernat Corominas- Murtra, Joaquín Goñi, Ricard V. Solé, and Carlos Rodríguez-Caso On the origins of hierarchy in complex networks. Proceedings of the National Academy of Sciences, 110(33): 13316-13321, 2013. ISSN 0027-8424. doi: 10. 10.3003pps (0.10773/naas.10088222 1300832110.
Alessia Deglincerti, Gist F Croft, Lauren N Pietila, Magdalena Zernicka Goetz, Eric D Siggia, and Ali H Brivanlou. Self-organization of the in vitro attached human embryo. Nature, 533(7602): 251-254, 2016.
Reinhard Eckhorn, Roman Bauer, Wolfgang Jordan, Michael Brosch, Wolfgang Kruse, Matthias Munk, and HJ Reitboeck. Coherent oscillations: A mechanism of feature linking in the visual cortex? Biological cybernetics, 60(2): 1988.
Karl Friston. The free-energy principle: rough guide to the brain? Trends in cognitive sciences,
13(7):293-301 2009.
Karl Friston. The free-energy principle: unified brain theory? Nature reviews neuroscience, 1 (2):127-138, 2010.
Thomas MJ Fruchterman and Edward M Reingold. Graph drawing by force-directed placement.
Software: Practice and experience, 21(11):1129-111 164, 1991.
10

---

Under review as a conference paper at ICLR 2023
Michelle Girvan and Mark EJ Newman. Community structure in social and biological networks. Proceedings the national academy of sciences, 99(12):7821-7826 2002.
J. C. Gower and G. J. S. Ross. Minimum spanning trees and single linkage cluster analysis. Journal of the Royal Statistical Society: Series (Applied Statistics), 18(1):54-64, 1969. doi: https: //bb.org/10.230722444499
Ch M GRAY. Stimulus-specifie neuronal oscillations in the cat visual cortex: A cortical functional unit In Society of Neuroscience Abstracts, volume 13, pp. 4033, 1987.
H. Haken. Cooperative phenomena in systems far from thermal equilibrium and in nonphysical systems. Rev. Mod. Phys., 47:67-121, Jan 1975. doi: 10. 10.10000MModP
Herman Haken. Synergetics. Physics Bulletin, 28(9):412, 1977.
Donald Olding Hebb. The organization of behavior: neuropsychological theory. Psychology Press, 2005.
Nabil Imam and Barbara Finlay. Self-organization of cortical areas in the development and evolution of neocortex. Proceedings of the National Academy of Sciences, 117(46):29112-29222) 2020. ISSN 0027-8424. doi: 10. 10.1073/pnas
Stuart A Kauffman et al. The origins of order: Self-organizationn and selection in evolution. Oxford University Press, USA, 1993.
Teuvo Kohonen. Self-organized formation of topologically correct feature maps. Biological Cyhernetics, 1982. ISSN 1432-0770. doi: 10. 10.1007/BF000337288. URL httttts::://dd org/
10.1007 00.10007/BF0037286
Jean-Marie Lehn. Toward complex matter: Supramolccular chemistry and self-organization. Proceedings of the National Academy of Sciences, 9968):4763-4768, 2002a. ISSN 0027-8424.
doi: 10. 00007//pnas.0220655550
Jean-Marie Lehn. Toward self-organization and complex matter. Science, 295(5564):2400-4444 2002b. ISSN 0036-8075. doi: 10.1126//cience.1000 1071063.
Ralph Linsker. Self-organization in perceptual network. Computer, 21(3):105-117, 1988.
David Lusscau, Karsten Schneider, Oliver Boisseau, Patti Haase, Elisabeth Slooten, and Steve M Dawson. The bottlenose dolphin community of doubtful sound features large proportion of long-lasting associations. Behavioral Ecology and Sociobiology, 54(4):396-405, 2003..
Jeffrey C. Magee and Daniel Johnston. A synaptically controlled, associative signal for hebbian plasticity in hippocampal neurons. Science, 275(5297):29--21 1997. ISSN 0036-8075. doi: 10. 10.111266 2al.11//scienee2555559777
Tomas Mikolov, Kai Chen, G.s Corrado, and Jeffrey Dean. Efficient estimation of word representa- tions in vector space. Proceedings of Workshop at ICLR, 2013, 01 2013.
Tom Misteli. Beyond the sequence: Cellular organization of genome function. Cell, 128(4):787- 800, 2007. ISSN 0092-8660 8674. doi: https://ddo..or/10... 1011://..00661....22000000000000
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare Belle- mare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 51(7540):529-5333 2015.
M Montalti, G Zhang, D Genovese, Jaime Morales, M Kellermeier, and Juan Manuel García-Ruiz. Local ph oscillations witness autocatalytic orfaniaganiztiion of biomorphic nanostructures. Nature communications, 8(1):1-6, 2017.
Daniel Müllner. Modern hicrarchical, agglomerative clustering algorithms. arXiv: 109.2378. 2011.
ar Xiv preprint
11

---

Under review as a conference paper at ICLR 2023
Mark Newman and Michelle Girvan. Finding and evaluating community structure in networks. Physical review. E, Statistical, nonlinear, and soft matter physics, 69:026113, 03 2004. doi: (0.1103/PhyssRcEE.66.0066.
Rise Ooi, Chao-Han Huck Yang, Pin-Yu Chen, Victor Eguiluz, Narsis Kiani, Hector Zenil, David Gomez-Cabrero, and Jesper Tegnèr. Controllability, multiplexing, and transfer learning in net- works using evolutionary learning. arXiv preprint arXiv: 111118111..000 1.05592. 2018.
Daniel Reker, Tiago Rodrigues, Petra Schneider, and Gisbert Schneider. Identifying the macro- molecular targets of de novo-designed chemical entities through self-organizing map consensus. Proceedings of the National Academy of Sciences, 111(11):007-4407, 2014. ISSN 0027-8424.
doi: 10.1073/pnaa 1 1320001111.
Yoshiki Sasai. Cytosystems dynamics in self-organization of tissue architecture. Natrr,, 493(7432):
318-326, 2013.
Gregor Schoner and JA Kelso. Dynamic pattern generation in behavioral and neural systems. Science, 239(4847):1513-15222 1988.
Julian Schritwieser,, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588(7839):604 609, 2020.
Erich Schubert, Jörg Sander, Martin Ester, Hans Peter Kriegel, and Xiaowei Xu. Dbscan revisited, revisited: why and how you should (still) use dbscan. ACM Transactions on Database Systems (TODS), 42(3): 1-21, 2017.
Herbert A. Simon. The Architecture of Complexity, pp. 457-476. Springer US, Boston, MA, 1991. ISBN 978-1 11881--499-077888
Davide Spalla, Isabel Maria Comacccia, and Alessandro Treves. Continuous attractors for dynamic memories. Elife, 10:e69499, 2021.
Peng Gang Sun. Imbalance problem in community detection. Physica A: Statistical Mechanics and its Applications. 157:364-376, 2016.
Richard Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.
J Tersoff, Chr Teichert, and MG Lagally. Self-organization in growth of quantum dot superlattices. Physical Review Letters, 76( 10): 1996.
Emmanuelle Tognoli and A. Scott Kelso. The metastable brain. Neuron, 81(1):35-48, 2014. ISSN 0896-6273. doi: https://doi.org/100 htte//...116/...neeeee (016/).neuron.2033.. 201..
Julia Uddén, Mauricio de Jesus Dias Martins, Willem Zuidema, and W Tecumseh Fitch. Hierarchical structure in sequence processing: How to measure and determine its neural implementation. Topics in cognitive science, 12(3):91--924, 20200.
Danilo Vasconcellos Vargas and Toshitake Asabuki. Continual general chunking problem and syncmap. Proceedings of the AAAI Conference on Artificial Intelligence, 35(11):10006-10144 May 2021.
Denise M Werchan, Anne GE Collins, Michael J Frank, and Dima Amso. 8-month-old infants spontancously learn and generalize hicrarchical rules. Psychological science, 2606))805-815, 2015.
H Hollis Wickman and Julius N Korley. Colloid crystal self-organization and dynamics at the air/water interface. Nature, 393(668)):445-447, 1998.
Tom J. Wills, Colin Lever, Francesca Cacucci, Neil Burgess, and John O'Keefe. Attractor dynamics in the hippocampal representation of the local environment. Science, 308(5723):873-88) 876, 2005. doi: 10. 12/sccennce 108905.
Wayne W Zachary. An information flow model for conflict and fission in small groups Journal of anthropological research, 33(4):452-473 1977.
12

---

Under review as a conference paper at ICLR 2023
A HIERARCHICAL CHUNKING PHASE
The output reveals the information on how many levels of hierarchy can be distinctively identified and how variables of each level form chunks. We first perform linkage (Mülner 201T) on input w and return a distance matrix Z (or sometimes referred as a linkage matrix). Using Z. we com- pute the distance between each formation of the non-singleton cluster as 8(Z). We index 8(Z) with ascending order, sort it according to branch distance with descending order sort(ō (()) [d E Did, di-1] and return their corresponding index, where sorted_ôid other words, index sorted_0sd represents the n-th number of the formation of the non-singleton cluster. We then remove the index in sorted_ôid that is larger than its previous index. The length of sorted dnd defined as H is considered as the total number of levels in the hierarchy. Lastly, We iterate over sorted 6id and perform flat clustering from linkage matrix Z based on the criteria of number of cluster n_cluster = sorted _iid on the same flat level. Predicted chunks on each level are then combined to form the final output Y.
argsorl( In
Doing essentially prioritizes putting the chunks with the most distinctive distance feature together, and divisively decomposing them into smaller chunks while taking the overall distance feature of all chunks into account. This opposes merely performing hierarchical clustering as it does not infers the exact number of levels in the hierarchy and which levels are important enough to be highlighted (Bar-Joseph et al. 2001). Algorithm 1 displays the pseudo-code for this method. Though we feed Hierarchical TSFMap's weight as an input, other feature vectors are acceptable. Here we use linkage with single method (Gower & Ross 1969), yet other methods such complete, ward, average can be used.
Algorithm Hierarchical Clustering Phase
W + Retrieve Hierarchical T'FMapp' weights
Z t- Perform linkage on W and return distance matrix
8(Z) + Return all the branch distance between each formation of non-singleton cluster sorted_ōia
(rgsort(((Z) (Z) Sort branch distance with descending order where sort(ō(Z)) [d E Did: > d=-1] and return their corresponding index
marimumm id &- sorted. for id E sorled dia do id > maximum__dd then mozimum-id < id
else
Remove id from sorted Rid end if
end for set H with length of sorled_ria Initialize predicted label Y for h E { do
Number of clusters Detuster = n sorted 6ealh]- -1
Form clusters from linkage matrix 7 with the number of cluster cluster be the parameter Set y =
end for
Return predicted label
B BASELINES
We compare Hierarchical TSFMap to Word2Vec Mikolov et al. (2013), Modularity Maximization Newman & Girvan (2004). SyncMap Vasconcellos Vargas & Asabuki (202T) and directly apply hi- erarchical chunking phase on transition probability matrix of state on every experiment mentioned (Y) previously. NMI score, NMI(Ý Y) = H(Y)) is used as evaluation metric to compare pre- dicted chunk Y with the true label Y (provided by the environments) on each level of hierarchy, which produce the final score defined by "i NMI((.)) Where I(;; the mutual informa- L
tion between & and Y, H.)) is the entropy, Î. is the total number of hierarchy in the environments,
13

---

Under review as a conference paper at ICLR 2023
generated by the environments itself. Hierarchical TSFMap produce more than L number of hierarchies I. in its matrix output Y. In this case, only take the first L. rows for evaluation they represent the most distinctive hierarchies.
B.1 SYNCMAP
SyncMap and Hierarchical TSFMap belong under the same learning paradigm Self-Organizing Dy- namical Equations. In summary, SyncMap learns by creating dynamic map that performs chunking from sequence data. Here, we inherited the parameters setting from the previous work. Learning rate set to 1e Input time delay m is set 10. We the map dimension with Hierarchical TSFMap's 's setting. Distance between weights is calculated using the Euclidean metric. Note that we replaced DBSCAN (Schubert et al. (2017) with Hierarchical clustering in the clustering phase to remove the necessity perform DBSCAN different levels of hierarchies. SyncMap can identify well the global context of the variables. However, local context is usually difficult to extract due to the overlapping of local chunks. Moreover, little to adaptation occurred in responding to the structural changes in the environment (Fig. 12
standardize
B.2 WORD2VEC
We adopted a Skip-gram Word2vec to compare with our model. The modified Word2vec used dense deep neural network model that takes the shape of Variational Autoencoder. The latent di- mension is set to 3 and the output size is equal to the number of input sizes. The model is trained under 10 epochs with batch size of 64, with a learning rate of 1e was used to calculate the output probability of skip-gram. We then performed hierarchical chunking on the learned word embeddings to identify its hierarchical structure and chunks on each hierar- chy. Since word embedding can encode items with identical features closer in a vector space, we assumed might preserve the hierarchical relationships amongst variables to some extent. When inspecting Fig. II it is apparent that variables that share the same direct parent chunk are clustered together. Yet, the relationship of chunks beyond that is vague. This hints that Word2vec can learn the relationship of co-occurrence of local variables effectively, but can hardly preserve any global relationship amongst variables/chunks.
window of 100 steps
B.3 MODULARITY MAXIMIZATION
One of the community detection algorithms, Modularity Maximization is used here as baseline. Using modularity as measure, the modularity maximization approach used identify commu- nities from a network. It begins with each node in its community and joins the pair of communities with maximum modularity until all nodes form into a single community. We can then select the maxima of modularity to decide on how to split the network into communities. With multiple local maxima, we constructed hicrarchy structure of communitics Newman & Girvan (2004). Here, we wwweee--eeaaa--Moooe Modularity Maximization algorithm to incorporate multi- ple local modularity maxima Clauset et al. (200)). Since modularity maximization can only operate under the premise of graph, transformed the sequence of inputs into an adjacency/transition probability matrix (Fig. 10), which can then turn into weighted directed graph. Therefore, despite being deterministic algorithm, an imprecise description of the adjacency matrix can induce un- certainty in the result. However, an accurate mean is achievable with a large sequence under the asymptotic central limit theorem. Although the input data, and therefore the problem, is different from the one seen in this work, sequence data and complex networks can interchangeably convert to one another (e.g., via an adjacency matrix from transition probabilities or a random-waid over complex network). This reveals Hierarchical TSFMap's 's connection with complex networks. Having said that, the similarities stop here as both the objective and methodology differ.
used a modified
B.4 HIERARCHICAL CHUNKING ON TRANSITION PROBABILITY MATRIX
We recorded the occurrence of state to state from the input sequence and create the transition prob- ability matrix of the current state to the next state (Fig. 10). We then applied Hierarchical Chunking Phase directly on the matrix as we considered it as a feature. The probability of the state transi- tion does necessarily reveal the hierarchical relationship among variables. Using the problems
14

---

Under review as a conference paper at ICLR 2023
in the imbalanced hierarchical structure environment, for example, variables within bigger chunk have a smaller transition probability to other variables in the same chunk; Otherwise, if the chunk is smaller. This induce inaccurate cut-off the dendrogram produced by hierarchical clus- tering and extract incorrect information about chunks. That being said, using merely the transition probability of sate-to-state transition not interpret the hierarchical relationship of variables. In the dynamic environment, TP matrix records all state transitions throughout the time step; it does not account for the changes occurring throughout the time step. Moreover, the accuracy of this method is also sensitive to the precision of the TP matrix itself.
C INPUT GENERATION
In our experiments, we consider the extraction of hierarchical structure and presume the input se- quence comes in discrete form. Overall, any events that can be serialized as sequence can be processed by Hierarchical TSFMap. To reproduce the environments used in our experiment or to cre- new environment, one can utilize graph to generate sequence input. Such graph G (V, E) resembles the distribution of variables, with E V being a set of nodes and e E E being set of edges. The graphs reveal the number of variables, the composition of chunks, and their hierarchical structure. A leaf node by E V/ signifies a variable; while non-leaf node represents chunk. A chunk is composed of sub-chunks or variables.
Based on G. we create an all-to-all connection weighted directed graph T that describes the transi- tion probability from variable to variable, composed of merely variables, with the weight of edges be the transition probability. The transition probability between variables w can be defined as:
[d(vi, 2//
with d(w) of) be the path length between variables un of in G. We then normalized all out-going edges from variables:
Worof
We
(6)
Vii
that Ger.og E [0, 1]. To generate a sequence of input, we apply a walker on graph T to travel from variable variable, based on the probabilities Wer.ef associated with the current variable. Variable is placed into the sequence input whenever random-walkes travel to one.
In the imbalanced hierarchical structure and two real-world networks environments, the models receive 300,000 sequential input signals S, = (S1,S2.- S....... S, where = 300000. In the dynamic hierarchical structure environment, We doubled 600,000 to accommodate the time step needed for adaptation. The number of variables, chunks, and hierarchies varies across different problems.
D EXPERIMENTAL SETUP FOR REAL-WORLD. SCENARIOS
In this paper, we consider Zachary's karate club network and Lusseau's bottlenose dolphin social network the modelings of real-world scenarios. Since the dataset we used here is graphs, we generate the input sequence using the same method from Appendix. C For the experiments, ward linkage is used by all the baselines when performing the hierarchical chunking phase, except TP matrix, to increase the NMI score for all baselines.
The metric used to evaluate the performance of our model on this dataset can difficult. Consider the case of the karate club network: despite the ground truth provided by the literature is usually two communities formed according to the factions; Members can form a smaller community within the faction. This indicated that such networks contain hierarchical structure in them, which is useful as it provides more insight into the data but is often overlooked by literature. Hence, for evaluation: (1) compare the predicted chunk from the most significant hierarchy with the ground truth provided using NMI score, and (2) visualize the representation formed in weight space/word embedding for the remaining hierarchies. Note that, provided here the hierarchical structure extracted using
15

---

Under review as a conference paper at ICLR 2023
model reference, rather than ground truth, whereas the definition of it depends various perspectives.
D.1 ZACHARY'S KARATE CLUB NETWORK
The well-establish Zachary's karate club network data collected by Zachary (9777) represents the social interactions ties among the members of the club. Due to internal conflict, the club later split up into two factions that become the ground truth for the community detection/chustering of this dataset. This network contains 34 nodes and 78 edges, with nodes representing the members of the club and edges the presence of social interactions within or away from the karate club (Fig. 16p.
D.2 LUSSEAU'S S BOTTLENOSE DOLPHIN SOCIAL NETWORK
Another famous network, Lusseau's bottlenose dolphin social network, is used benchmark to verify the performance of our model. The network contains 62 nodes that represent each individual bottlenose dolphin and 159 edges that represent the interaction between dolphin pairs observed to co-occur more often than expected. Furthermore, the ground truth of this network can be partitioned
into two main groups (Lusseau et al., ) On the other hand. Cheng al. (2014) considers the ground truth with four groups. Atter all, we compare our prediction to the former ground truth using NMI score while taking the latter as reference when visualizing the representation formed in weight space (Fig. 17
16

---

Under review as a conference paper at ICLR 2023
120
10
15
15
Figure 9: Setting of dynamic hierarchical structure experiment. (a) Leaf node indicates vari- ables and node in red represents a chunk. Color coding is used to indicate chunks. Consequently, sibling nodes belong to the same chunk. A chunk contain either child chunks variables. The graph shows the distribution of how the sequence variables is generated. Transition probability between variables increases transition probability is higher when variables belong to the same chunk. In a dynamical setting, the distribution of variables started with the graph on left, changes halfway through the time step to the graph on right. In this example, two big chunks are split into three big chunks. (b) The dy- namic of weights Hierarchical TSFMap reconfigure adapt to the changes in the distribution of input sequence, in dire-dimenssonal weight space. Observation shows that the weights first form into two big chunks and enter an equílibrium state. Once the distribution of the sequence is altered, the weights reconfigure their position to split into three big chunks. (c) The dendrogram produced during the hicrarchical chunking phase, extracted from the weights in (b). The color coding reveals the hicrarchical structure produced matches against the latest distribution of variables in (a).
the number of edges connecting them decreases. Essentially, the
17

---

Under review as a conference paper at ICLR 2023
30
A42
10
100
DIH
DCH
0.3
020
-66
EH2EC
015
019
055
10
000
Figure 10: Heat map of transition probability matrix of the current state (Y-axis) to the next state (X-axis). (IH, HB, IEH) Transition probability matrix (TP matrix) from the imbalanced hierar- chical structure environments. Alternatively, The recorded transition of state represents an adjacency matrix, convertible an all-to-all connection weighted directed graph. The boundaries shown in the heat map usually hint at the presence of chunks. (DIH, DCH, EC2EH, EH2EC, DCS) TP matrix generated from dynamic hierarchical structure environments. It is important note that the matrix has encoded the transition from two kinds of distribution. Therefore, the TP matrix failed to capture the dynamic of changes in environments. Consequently, baselines that utilized the matrix are only as accurate as the probability distribution is.
18

---

Under review as a conference paper at ICLR 2023
IEH
DIH
DCH
EC2EH
EH2EC
DCS
Figure 11: Word embeddings of Word2vec. The word embeddings of Word2vec are learned in three-dimensional latent space and its hierarchical structure is produced by hierarchical clustering algorithm. This provides clear indication that Word2vee can learn the relationship of co-occurrence of local variables effectively; however, the embeddings reveal that hardly preserves the global relationship amongst variables.
19

---

Under review as a conference paper at ICLR 2023
Hierarchical TSFMap in DCS
-22
2
Weights initialized the with same seed
Formation low-level chunks
Formalion high-level chunks
Hierarchical TSFMap adapts changes input
SyncMap failed adapt to changes in input
SyncMap DCS
Figure 12: Comparison of adaptive capability between Hierarchical TSFMap and SyneMap in DCS. We observed that Hierarchical TSFMap is significantly better than SyncMap when adapting to the changes in the environment. To place both methods in fair comparison, we initialized weights in three-dimensions space with the same seed. Both methods managed to form hierarchies of chunks with respect to the input. Once the structural changes in the environment occurred, Hierarchical TSFMap adapt its weights and form new pattern accordingly; in contrast, SyncMap failed to adapt and remained in the same configuration prior to the changes in the environment. Additionally, although initialized in higher dimensional weight space (in this example, three-dimensional weight space), the rank of the weight matrix converged to given enough time. 1 = p(w). This indicates that SyncMap's s dynamic restricted one-dimensional space. The weight matrix of Hierarchical TSFMap however, can retain its high dimensionality, S p(w) < k.
20

---

Under review as a conference paper at ICLR 2023
IH
HB
IEH
0
3
4
-10
18.10
-10
-19
10
15.10
15
-10
-1
110
112
025
The dynamics of Hierarchical TSFMap Imbalanced Hierarchical Structure envi- ronments. The dynamics start with set of weights initialized in space, and progressively form into pattern that corresponds to the input. The patterns are extracted to produce dendrogram that describes the hierarchical structure of underlying data. Both the patterns and dendrogram portray somewhat distinctive hierarchical structures, even through manual inspection.
Figure 13:
21

---

Under review as a conference paper at ICLR 2023
DIH
DCH
EC2EH
MAISE
Figure 14: The dynamics of Hierarchical TSFMap in Dynamic Hierarchical Structure environ- ments. Notice that the a space shown the third and fifth rows are the patterns corresponding to the first and second data distribution respectively. The latest pattern formed is used for identifying the hierarchical structure of the latest changes data distribution. This proves that, given enough time, Hierarchical TSFMap can adapt to any distribution of data, regardless of the current state; for example, from either random initialization or pattern that are already established.
22

---

Under review as a conference paper at ICLR 2023
EH2EC
DCS
1 8 14-00
110
10
11
11
12
1
100
Figure 15: Refers to Fig.
14
23

---

Under review as a conference paper at ICLR 2023
Zachary's a karate club network
b
NMI score cierarch TSFMap compared to baselines
Baselines
Karate Club
Modularity Maximization
0.5920.00
Word2vee
0.71
SyncMap
0.77=0.13
TP Matrix
0.29#0.04
Proposed: Hierarchical TSFMap
0.86+0.06
Hierarchical TSFMap
1
20
SyndMap
00
1+1
194
20
WordZves
hw!
1+
30
9
8
205.00
250 ZOLAN ON
1..% 124 16
Figure 16: Extracting the hierarchical structure Zachary's 's karate club network. (a) The ground truth of the Karate club network provided by Zachary (1977). (b) The NMI score of Hicrarchical TSFMap compared to its baselines, comparing only predicted chunks from the most significant level of hierarchy the ground truth. (c) The weight space/word embedding of Hierarchical TSFMap, SyncMap, and Word2vec. The weight space (k 5) describes how weights are chunked in three of the most significant hierarchies, visualized using multidimensional scaling. The predicted chunks are labeled by color. The dendrogram shows the hierarchical clustering of weights, where the red dashed lines are the cut-off. It appears that the models shown here can configure their weights to match the ground truth in most instances, with Hierarchical TSFMap being the most accurate one reflected by the relatively high NMI score in (b). Interestingly, members can form a smaller community within the faction. Removal of hubs can further break down faction into a smaller community and exhibit the property hicrarchy. Hicrarchical TSFMap, when looking deeper into the hierarchy, smaller groups of members and less social individuals can be formed into their own chunks. Yet, a similar occurrence is not displayed in the weight space/word embedding of SyncMap/Wordd2ver Note that, individuals (i.e., node 9) that are weakly associated with both factions cannot be mapped in the middle the two factions. This is due the fact that edges not weighted and hubs with smaller connectivity have larger probability to attract them.
is also demonstrated in the weight space of
24

---

Under review as a conference paper at ICLR 2023
Lusseau's bottlenoee dolphin social network
b
NMI cirrarchicll TSFMap compared baselines
Baselines
Dolphin social network
Modalarity Maximizatinn
0.48:0.07
Word2vee
0.03a0.00
SyncMup
0.48g0.04
Matrix
Propaced: Heraarciial TSFMap
0.7850.17
Hierarchical TSFMap
142
SyncMap
Wordi2vec
143
102
609
050
033
Figure 17: Extracting the hierarchical structure Lusseau's bottlenose dolphin social network. (a) The ground truth of the dolphin social network provided by Lusseau al. (2003). (b) The NMI score of Hierarchical TSFMap compared to its baselines, comparing only predicted chunks from the most significant level of hierarchy to the ground truth. (c) The weight space/word embedding of Hicrarchical TSFMap, SyncMap, and Word2vec. The weight space (k 5) describes how weights are chunked in three of the most significant hierarchies, visualized using multidimensional scaling. The predicted chunks are labeled by color. The dendrogram shows the hierarchical clustering of weights, where the red dashed lines are the cut-off. We observed that Hierarchical TSFMap can form chunks on the most significant level of the hierarchy accurately when compare to the ground truth. More surprisingly, going deeper into similar to the ground truth provided by Cheng et al. (2014) with slight differences. On the other hand, SyncMap can form two chunks with lower accuracy, whereas some nodes are incorrectly placed. Lastly, Word2vee failed to learn meaningful chunks, hence the lower NMI score.
hierarchy, the weights can form into four groups,
25

---

Under review as a conference paper at ICLR 2023
Table 1: Statistical Results. We used two-sample t-test with a p-value larger than 0.05 to accept the null hypothesis that the means are equal. We calculate the two-tailed value. The results were obtained from least 30 independent experiments.
Environments IH HB
SyncMap
Hicrarchical TSFMap
Word2vec
Modularity Maximization
0.424
0.518
0.854
1.000
0.618
0.766
0,496
1.000
IEH
0.846
0.153
0.734
1.000
DIH
0.535
0,673
0.635
1.000
DCH
0.578
0.556
0,932
1.000
EC2EH
0.897
0.173
0.649
1.000
EH2EC
0.554
0.368
0.395
1.000
DCS
0.880
0.759
0.639
1.000
Karate club network
0.882
1.000
0.553
0,440
Dolphin social network
0.629
0.702
0.541
0.362
26


Reviews:
[{"review_id": "pdV5K6XK1YC", "reviewer": "Reviewer_7qEB", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The submission proposes a novel unsupervised learning method for dynamically identifying hierarchical structures called \"Hierarchical TSFMap\". The method is based on three parts, encoding, dynamics and a 'hierarchical chunking phase'. During the encoding, sequential data is encoded via temporally exponentially decaying kernels. In the dynamics, pattern forming occurs via the TSFMap.  ", "summary_of_the_review": "The submission suggests a novel method for unsupervised identification of hierarchical structures in sequential data. While there was extensive numerical evaluation, a theoretical analysis of TSFMap was missing and should be added. E.g. the map could be written as a PDE and a link to the extensive literature on pattern formation in PDEs could be made, existence and uniqueness of solutions should be discussed. Note that I am not an expert on unsupervised learning/pattern formation.", "strength_and_weaknesses": "Strength:\n* seems a novel unsupervised learning method\n\nWeakness:\n* Can the dynamics be written in the form of a differential equation? That might help with a more rigorous theoretical analysis of the dynamics of the TSFMap. Given that the link to pattern formation is stressed so much in the manuscript, an analytical analysis with the rich tool set of the field of Pattern Formation and Dynamics in Nonequilibrium Systems ( https://doi.org/10.1017/CBO9780511627200 ) would strengthen the submission.\n* Both abstract, introduction and conclusion should be more specific about the innovation here which (as far as I understand) is to cluster data with hierarchical structure. Some statement in the paper about the novel hierarchical clustering method seem overly generic and slightly more pompous than the actual result (e.g. \"This work also has implications in many areas such as cognitive science and neuroscience, shedding light on how self-organization circuits can be established as fundamental a mechanism in the brain.\", \"In this paper, we show how the learning of patterns can be achieved by Hebbian and anti-Hebbian learning dynamics, linking between Hebbian learning and top-down theories of intelligence\", \"Our proposed system is arguably more biologically plausible, and it is also shown to be more\naccurate and adaptive than state-of-the-art unsupervised algorithms. In fact, it sets up a foundation for a new paradigm in machine learning solely based on self-organization from dynamical equations, namely Self-Organizing Dynamical Equations, which are inherently accurate and adaptive.\"). Are more clear-cut and less pretentious presentation might help readers to appreciate the actual contribution of the paper.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "hysVRjQSmb", "reviewer": "Reviewer_fCck", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "- Overview\n\nThis work studies how to introduce dynamic equations for the complex networks under self-organization extended from previous work on [AAAI 2021]. \n\nThe authors introduce a method called Hierarchical Temporal Spatial Feature Map (TSFMap) that uses a weighted coefficient learning manner to encode input sequences. \n\nThe workflow further combines with the chunking phase to cluster data. A related dynamical equation has also been studied in the design of gene circuits [Ref 1].\n\n- Baseline\n\nThe authors only use word2vec (2013) to serve as one representation of learning, when several methods, such as GloVe, FastText, BERT, are actually much more widely adapted baselines in the representation learning community. \n\nThe authors studied simulated data, karate club network, and bottlenose dolphin social network data in the real-world and evaluated its mutual information scores. \n\n**References**\n\n1. Controllability, multiplexing, and transfer learning in networks using evolutionary learning, 2018 \n2. Glove: Global vectors for word representation, EMNLP 2014\n3. Enriching word vectors with subword information, TACL 2017\n4. FastText.zip: Compressing text classification models, 2016\n", "summary_of_the_review": "Overall, this work tackles a problem with the limited interest in the representation learning community with weak baselines. \n\nFurthermore, that could be a plus if the authors conduct some experiments on the information flow in some common biological, but it is unfortunately missing. \n\n\n### Post-rebuttal \n\nThanks author for the efforts on some baselines. However, the nlp baselines are still very weak. (e.g., Glove was proposed in 2014)\n\nWith both experience in the complex system and language modeling, I did not see a major benefit from the proposed method.\n\nI consider this paper with limited interests in a very narrow problem. This is a 4 score paper to me. \n\n", "strength_and_weaknesses": "Pros\n\n- The authors extend the dynamic equation in the complex network community for a community detection problem. \n\nCons\n\n- Overall, the current application is still limited to a narrow problem. \n\nThere are also some neuro-circuit and gene-circuit are much often serving as public benchmarks. \n\n- The framework is very incremental from the previous study [AAAI 2021]. \n\n- the baselines are weak and out of date. Some graph embedding and more recent word embedding methods are missing. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "_2KJizK6W-", "reviewer": "Reviewer_iaFP", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors have introduced a dynamical method to infer the hierarchical structure from temporal sequences. Inputs are encoded into temporal sequences of exponentially-decaying spikes that then determine the temporal evolution of weight variables associated with the inputs. Clustering is then performed to determine the hierarchical structure encoded in the weights.  The authors show that their method outperforms other hierarchical methods when used on imbalanced or dynamically changing hierarchies.  To the reviewer's knowledge, this work represents a novel method of dynamically learning and inferring hierarchical structure in a bottom-up manner from sequential data.", "summary_of_the_review": "The present work provides a novel method to dynamically learn and infer hierarchical structures from sequential data that appears to perform favorably compared to other methods.  However, the manuscript is hampered by a lack of clarity in several aspects.", "strength_and_weaknesses": "Strengths:\n\nThe authors provide a novel, yet simple dynamical method that infers the hierarchical structure of dynamical inputs well. The results appear to improve upon current methods and may provide insight into how biological neural networks perform similar analyses on incoming data streams.\n\nWeaknesses:\n\nThe manuscript would benefit from a clearer explanation of the method and from the example inputs that were used being made explicit (please see below for more details).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ACxMfjw3iwG", "reviewer": "Reviewer_S42c", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is a method for finding embedding for components of encoded dynamical data that captures “co-occurance” of state variables and then perform hierarchical clustering on those embeddings. This method is compared to other representations.", "summary_of_the_review": "Even if there are some interesting ideas in this paper, it is hard to get very excited about another clever exploration of complex network data.", "strength_and_weaknesses": "Strengths:\nThe paper proposes a method for exploring sequence data and extracts a graphical representation of possible interaction among features/variables, as the \n\nWeaknesses:\nThe use of ‘dynamical equations’ in the title could create a wrong impression. It seems to be about $w_i$ update that about dynamics in the sequence data.\nThe setting of the dynamic data generation is confusing to me: which part of it is real data-based and what is synthetic?\nIt is not clear what one would achieve with the final network/networks outputted by the algorithm.\n\n\nMinor issues:\nThe explanation of the term associated with ‘non-recently activated input set’ in equation (3), in text in the paragraph after equation (4), has some extra minus signs.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject