Intelligence at the Edge of Chaos
Authors: name: Shiyang Zhang; name: Aakash Patel; name: Syed A Rizvi; name: Nianchen Liu; name: Sizhuang He; name: Amin Karbasi; name: Emanuele Zappala; name: David van Dijk
Published: 2024-10-03 14:42:34

Abstract: We explore the emergence of intelligent behavior in artificial systems by
investigating how the complexity of rule-based systems influences the
capabilities of models trained to predict these rules. Our study focuses on
elementary cellular automata (ECA), simple yet powerful one-dimensional systems
that generate behaviors ranging from trivial to highly complex. By training
distinct Large Language Models (LLMs) on different ECAs, we evaluated the
relationship between the complexity of the rules' behavior and the intelligence
exhibited by the LLMs, as reflected in their performance on downstream tasks.
Our findings reveal that rules with higher complexity lead to models exhibiting
greater intelligence, as demonstrated by their performance on reasoning and
chess move prediction tasks. Both uniform and periodic systems, and often also
highly chaotic systems, resulted in poorer downstream performance, highlighting
a sweet spot of complexity conducive to intelligence. We conjecture that
intelligence arises from the ability to predict complexity and that creating
intelligence may require only exposure to complexity.