Title: The Emergence of Complex Behavior in Large-Scale Ecological Environments
Primary area: transfer learning, meta learning, and lifelong learning
Keywords: Population dynamics, Adaptive agents, Synthetic environment, implicit rewards

Abstract: We explore how physical scale and population size shape the emergence of complex behaviors in open-ended ecological environments.  In our setting, agents are unsupervised and have no explicit rewards or learning objectives but instead evolve over time according to life, death, and reproduction rules.  As they act, they also shape their environment and the population around them in an ongoing dynamic ecology.  Our goal is not to optimize a single high-performance policy, but instead to examine how behaviors emerge and evolve across large populations due to natural competition and environmental pressures.  In an effort to discover how complex behaviors naturally emerge, we conduct experiments in large-scale worlds that reach populations of more than 60,000 individual agents, each with their own evolved neural network policy.  We identify various emergent behaviors such as long-range resource extraction, vision-based foraging, and predation that arise under competitive and survival pressures.  We examine the relationship between sensing, environmental scale, and the emergence of these behaviors, finding that some appear only in large environments and populations, and that larger scales generally increase stability and consistency in these settings.  While there is a rich history of research in evolutionary settings, our scaling results provide promising new directions to explore ecology as an instrument of machine learning in an era of abundant computational resources.

TL;DR: We explore how environmental scale and sensing the emergence complex behaviors in open-ended evolutionary settings.