Reinforcement learning for decision making in agent-based models
Channel: Recording Archaeology
Date: 22/08/2016

Understanding the decision-making processes within past societies is a challenging aspect for archaeological research. In order to validate our understanding of these behaviours, an ideal workflow would be to simulate the behaviours that we imagine are correct. We can then observe, if the imagined behaviours reproduce the evidence collected from the terrain.
ABM has proved to be an efficient approach toward the realisation of this vision. However, many of the phenomenons we wish to study, require the adaptation of the agents to the context they live. For example, it would be interesting to study the behaviors of gatherer agents in front of changing environments. To face this type of situations, there is a clear need for adaptation abilities. Unfortunately, most of the current behavioural architectures used in ABMs do not let the agent adapt continuously its behaviour so as to fit the environment, thus restricting the modeling possibilities.
Reinforcement learning algorithms have been proposed to address learning problems within multi-agent settings. Once applied to past-societies models, this learning method faces two main challenges: each agent observes only part of the world and the number of states and actions an agent can face is potentially extremely large. Similar challenges have been previously been encountered by researcher applying reinforcement learning methods to multi-robots scenarios. Within this article we aim to present which of the solutions previously developed can be applied to create models of past societies.
Moreover, the use of a UCT architecture has been previously proposed to address the challenge of learning in past-societies models. We will draw a comparison between reinforcement learning and UCT approaches. This comparison will highlight the difficulties each approach is facing, specifically for an application to past-societies models.

Jean-Marc Montanier, Xavier Rubio-Campillo