Intelligence at the Edge of Chaos

Authors: Shiyang Zhang, Aakash Patel, Syed A Rizvi, Nianchen Liu, Sizhuang He, Amin Karbasi, Emanuele Zappala, David van Dijk
Primary area: learning on time series and dynamical systems
Keywords: Large Language Models (LLMs), Elementary Cellular Automata (ECA), Emergent Intelligence, Complex Systems, Complexity Theory

Abstract:
We explore the emergence of intelligent behavior in artificial systems by investigating how the complexity of rule-based systems influences the capabilities of models trained to predict these rules. Our study focuses on elementary cellular automata (ECA), simple yet powerful one-dimensional systems that generate behaviors ranging from trivial to highly complex. By training distinct Large Language Models (LLMs) on different ECAs, we evaluated the relationship between the complexity of the rules' behavior and the intelligence exhibited by the LLMs, as reflected in their performance on downstream tasks. Our findings reveal that rules with higher complexity lead to models exhibiting greater intelligence, as demonstrated by their performance on reasoning and chess move prediction tasks. Both uniform and periodic systems, and often also highly chaotic systems, resulted in poorer downstream performance, highlighting a sweet spot of complexity conducive to intelligence. We conjecture that intelligence arises from the ability to predict complexity and that creating intelligence may require only exposure to complexity.

Decision: accept (poster)
Reviewer scores: 8, 6, 6, 8

Reviews and discussion:
<thread>
creation date:
1729793684516

thread_type:
official_review

<thread_object>
creator: reviewer_jmh7

<content>
summary:
This paper explores a potential connection between complexity and intelligence using data from elementary cellular automata (ECA) — a class of discrete dynamical systems that exhibit simple, complex, and chaotic dynamics. The authors find that transformers trained on complex and chaotic dynamics perform better on downstream tasks. They first train transformers on these dynamical systems, then adapt the pre-trained transformer to other tasks while freezing the transformer weights and only training the encoder/decoder layers.

strengths:
This paper proposes a novel approach to studying connections between complexity and intelligence. By pre-training transformers on complex patterns, it aligns with large language model (LLM) studies, which have demonstrated that high-quality training data leads to better performance on testing tasks. This study conducts thorough experiments across various tasks, examining the topic from different perspectives. It reveals significant performance differences in transformers trained on different cellular automata.

weaknesses:
The most critical issue in this paper is the method of generating training data. As mentioned in lines 146-152, you first generate a large spatiotemporal evolution, then extract only a small spatiotemporal window from it. This approach inadvertently introduces randomness into the evolution because information outside the spatial window is lost, yet its effects continue to diffuse into the observed space. The authors appear to overlook this randomness, claiming the system is deterministic. They then use a linear encoder/decoder to map states into hidden representations and predict the next state from these hidden states.

This mismatch may lead to several problems, potentially explaining some of the results observed in your paper:

1. Your deterministic model may not be strong enough to learn highly random patterns;
2. The attention pattern discovered in Figure 4 might be caused by this mismatch: For simple tasks, external unknown information may not diffuse into the space, so the cut windows contain truly deterministic dynamics, hence the transformer doesn't need to have attention on previous states. However, for complex patterns, information from states outside the window is necessary for prediction. Consequently, the transformers use previous states for better prediction, resulting in the observed attention patterns;
3. This could lead to a trivial, yet unfalsified explanation of your discovery: the reason transformers trained on complex patterns perform well on downstream tasks might be because they have more attention. And the reason they have more attention might be caused by the unknown information outside the spatial window.

To address this issue, you could predict only part of the next state. For example, given state indices 1-60 as input, you could predict only states 2-59, thereby mitigating the problem. I’m wondering if you still observing the results when removing such randomness.

Some other minor issues:

1. The models should not been named as large language model (LLM) — they are not large, and not natural language. I would suggest using transformer instead;
2. Around line 158, please define “binary vector”. Is it a vector of spatial pattern at given time?
3. In line 232, why there isn’t rule 110 in ARC Easy / Chess Move? It is Turning complete, may need more attention on it.
4. It will be good to compare to a randomly initialized transformer. Would complex ECAs beat random transformer?

questions:
1. How does this relate to Reservoir Computing (RC)? A randomly initialized RC can also perform well on nonlinear and chaotic tasks. How do you distinguish between the sources of good performance—complexity versus reservoir effects?
2. Beyond the trivial explanation mentioned above, what other reasons might explain why the transformer learns to use attention? Have you experimented with varying model sizes? It's possible that more powerful models might not require attention, while smaller models (or those trained with high L2 regularization) might need attention to allocate more computational resources.
3. Does this relate to the principle of computational equivalence?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730116114604

thread_type:
official_review

<thread_object>
creator: reviewer_8jty

<content>
summary:
The authors of this paper utilize a large language model (GPT-2) to demonstrate "Intelligence at the Edge of Chaos," showing that models achieve optimal performance in moderately complex, but not overly chaotic, systems. They train on tasks based on elementary cellular automata (ECA) and evaluate on various downstream tasks to explore the link between system complexity and emergent intelligence. The results align well with their hypothesis, though additional experiments could further strengthen these findings.

strengths:
The paper’s topic is innovative, as I believe the emergence of intelligence is fostered by increasingly large architectures capable of handling complex data. The authors devote significant attention to explaining the concept of the "Edge of Chaos," ensuring a thorough understanding of this idea. Their explanation of the results is also convincing, and the paper’s visual illustrations are highly effective.

weaknesses:
The experiments are somewhat limited. Adding more downstream tasks, such as nim games or toroidal chess, could enhance the findings. Additionally, testing the hypothesis on smaller models could provide further support, rather than relying solely on results from larger models.

questions:
1. Could you include more downstream tasks, like nim games or cylinder chess, to expand the evaluation?

2. Could you test on smaller models, such as ResNet-50, BERT, or Mamba-170m, to validate the results?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730370517688

thread_type:
official_review

<thread_object>
creator: reviewer_ycfn

<content>
summary:
The authors explore how the complexity of the rule of the data generative system influences the model capabilities on the downstream tasks. They use elementary cellular automata (ECA) as a rule-based dynamic system to control complexity of the training data trained GPT-2 derived transformer architecture and evaluated the trained model on reasoning/chess move prediction tasks. They found that higher but not too chaotic complex system leads to higher downstream task performance. Also, they found that the model trained with more complex system tends to obtain more complex solution rather than a simple rule, which the authors hypothesized as a reason for better downstream task performance.

strengths:
- I really enjoyed reading the paper. The authors explore the impact of rule complexity of the training data on the model's downstream task performance. I believe it is important and timely subject as there are already many literatures suggesting the importance of data property on acquired intelligence of the neural network models. 
- The paper is well motivated and the authors made good connection to the related previous works on complexity and computational capabilities.
- Particularly, I liked the ECA data model that the authors use, which provides a good control of complexity of the rule-based dynamical system. Also, the authors used multiple measures of complexity rather than single definition. 
- The paper is well-written, it was easy to follow and understand the key claims that the authors make.

weaknesses:
- While I am fairly convinced with the motivation and general results, but the presented quantitative result is rather weak in the results and figures they provide. Is the difference between accuracy across complexity, for example, in figure 2, shows fairly low r value.  
For wolfram's complexity class categorization, does the p-test shows significance? The absolute accuracy difference between the categories look fairly low. And the baseline performance without the ECA pretraining is not provided. 

- Did the authors run experiments with multiple seeds?

- The claimed 'emergent' intelligence is only evaluated on the downstream tasks after post-tuning, and in limited kinds. Also, the baseline performance without the pre-training on ECA is not given. 

- While authors found that the model trained with complex system tends to obtain more complex solution than the actual rule and made a interesting conjecture that this might be the reason for better downstream performance, no rigorous experiment to further investigate this point is provided. 

Further questions and comments are given in the next section.

questions:
- Could you comment on the differences between "complexity" and "diversity" in ECA model? Is any of the complexity measures you incorporate captures diversity of the data as well? Are complex data always more diverse and vice-versa? 

-  In figure 2 violin plot, it seems like it's not only mean values but also variances that are correlated to the complexity measure and downstream tasks which I found interesting.  Could the authors provide comment?

- Related to the weaknesses I wrote above, I am curious about baseline performance of the models before pre-trained on ECA data. 

- I see different tendency on Krylov complexity compare to the other on fig 3 i.e. high variance on both very low and very high complexity measure. Is there any reason that you could think of?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730611150771

thread_type:
official_review

<thread_object>
creator: reviewer_re2s

<content>
summary:
The paper explores whether LLMs can develop generalisable reasoning abilities by pretraining on structured but simple datasets, specifically sequences generated by cellular automata. The authors hypothesise that exposing models to complex patterns, even if the underlying system is deterministic and rule-based, could improve the models’ performance on reasoning tasks. By pretraining LLMs on cellular automata generated data, they test whether this training improves performance on abstract reasoning benchmarks, such as tasks from the Abstraction and Reasoning Corpus and chess move prediction (after finetuning).

The results show that models exposed to complex patterns, those generated by "edge-of-chaos" traces, perform better on these reasoning tasks compared to models pretrained on simpler or more chaotic traces. This suggests that pretraining on richer, more complex patterns helps LLMs develop broader generalisation abilities, even when the training data lacks semantic content.

strengths:
I like about this paper that it uses a relatively simple approach to create data with varying (and quantifyiable) degrees of complexity. The approach could help in exposing LLMs to large amounts of data, potentially leading to smaller requireemtns of corpora during pretraining. The "edge-of-chaos" idea has been investigated in neural networks before, but not as a result of pretraining with data from CAs.

The approach of pretraining on CAs and finetuning the outputs later seems like a great fit to learning representations / ICLR - an interesting way to derive good features for training models.

weaknesses:
While I like the idea and the approach of the paper, the choice of words could be improved and language be more precise. The choice of "Intelligence" in the title and throughout the paper is awkward, the work doesn’t necessarily show "intelligence" in the sense of true reasoning or understanding: it demonstrates however that structured complexity in data can lead to the development of generalisable features or representations, that is a better choice would be to describe the approach as learning useful features or representations for predictions or other complex tasks.

The language around the CA rules is also imprecise - all CA rules are similarly simple, but some of the rules lead to complex (or chaotic, or simple...) outputs / traces. It is the generated data that exhibits the complex patterns in case of the class iv CAs. 

As an ablation, it might also be interesting to explore if it is the *structured* complexity that leads to better features, or if complex patterns are sufficient. This could be achieved by using surrogate outputs from CAs (ie scramble the temporal order), to allow the distinction between the complex (spatial) pattern or the structured, sequential nature of the data.

questions:
In my view the paper is quite interesting and a good fit for ICLR, and would like see it accepted if some of its weaknesses are addressed. Some questions:

- (see above) What role does the temporal structure of CA sequences play in learning generalisable representations?
- Is the amount of data required from CA pretraining comparable to the amount of data required using natural langage (or similar complex data), to reach similar generalisation effects?
- what's the trade-off between complexity and volume of data
</content>
</thread_object>

</thread>
<thread>
creation date:
1732122132151

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
We sincerely thank all the reviewers for their insightful comments and are encouraged by their unanimous approval of our work.
\[rE2s\] and \[YCfn\] appreciated the innovative use of elementary cellular automata (ECA) as a framework for controlling and analyzing complexity in pretraining transformer models. \[8JtY\] and \[JMh7\] valued our focus on the "Edge of Chaos" concept and its connection to emergent intelligence, noting the novelty and potential of our approach.

Several reviewers raised important points that we have addressed:

* **Baseline Comparisons**: **\[rE2s\]**, **\[YCfn\]**, and **\[JMh7\]** requested comparisons with randomly initialized transformers. We have now included baseline performance results, demonstrating that models pretrained on complex ECA data outperform their untrained counterparts.  
* **Additional Experiments**: **\[rE2s\]** and **\[8JtY\]** suggested exploring the role of temporal structure and incorporating more downstream tasks. We conducted new experiments that highlight the critical role of temporal dynamics in learning generalizable representations and added a downstream task based on the Nim game, which reinforces our findings.  
* **Model Size and Capacity**: **\[8JtY\]** and **\[JMh7\]** recommended testing with different model sizes. We performed experiments with smaller transformer models to assess the impact of model capacity on performance, confirming that both data complexity and model size influence learning efficiency.  
* **Complexity Measures and Statistical Significance**: **\[YCfn\]** and **\[JMh7\]** questioned the statistical significance of our results and the measures of complexity used. We have clarified our analysis, provided additional statistical tests, and expanded our discussion on different complexity measures to address these concerns.  
* **Theoretical Connections**: **\[JMh7\]** inquired about the relation to Reservoir Computing and the principle of computational equivalence. We have added discussions to situate our work within these theoretical frameworks, emphasizing how our findings offer new insights into the computational capabilities of neural networks trained on complex data.

We once again thank the reviewers for their valuable feedback, which has significantly strengthened our paper. We are excited about the future implications of this work and look forward to further advancements in understanding the interplay between data complexity and emergent behavior in machine learning models.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733168807615

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
We appreciate all of the thoughtful feedback and constructive comments throughout this discussion. Below, we summarize the additional experiments conducted to address the reviewers’ questions:
- **Temporal Structure:** We examined the impact of temporal structure by shuffling the order of ECA states during training. The results confirmed that temporal structure is critical for learning effective representations (and not spatial complexity alone), as its removal led to significant drops in downstream performance. ([Shuffled vs non-shuffled](https://postimg.cc/vgjR8NVy))
- **Complexity vs Data Volume:** We analyzed the interplay between data complexity, data volume, and model size. Our findings showed that higher-complexity data requires more training tokens to converge, and larger models exhibit greater data efficiency. These results provide insights into the relationship between data complexity and model capacity, highlighting it as an important direction for future research. ([Token consumption](https://postimg.cc/DSDYXXzQ), [large model downstream performance](https://postimg.cc/LJbQYKy6))
- **Additional Downstream Tasks:** We introduced the Nim game as a new downstream task to expand our evaluation, further validating that models pre-trained on higher complexity ECA data achieve superior generalization. ([Nim game performance](https://postimg.cc/ZWfwhy3t)) 
- **Full spatial context:** We validated that the trend of improved downstream performance persists when using the full spatial context, ruling out spatial sampling as a confounding factor. ([Downstream performance of full spatial model](https://postimg.cc/D8Vw5v6B))

In addition to these experiments, we have addressed numerous questions, providing clarifications on key aspects such as the distinction between complexity and diversity, baseline performance comparisons, and connections to related topics like reservoir computing and the principle of computational equivalence. We believe these additions significantly strengthen our work and hope that the reviewers will consider them in their evaluation.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper investigates whether transformers, in particular GPT-2, can learn to predict simple cellular automata, and also how they do it. This is an unexpected and quite interesting topic, and the investigation is well executed. The uncommon topic/perspective makes it worthy of spotlighting, IMHO. There are no particular weaknesses.