  Model-based planning and prospection are widely studied in both cognitive
neuroscience and artificial intelligence (AI), but from different perspectives
- and with different desiderata in mind (biological realism versus scalability)
that are difficult to reconcile. Here, we introduce a novel method to plan in
large POMDPs - Active Tree Search - that combines the normative character and
biological realism of a leading planning theory in neuroscience (Active
Inference) and the scalability of Monte-Carlo methods in AI. This unification
is beneficial for both approaches. On the one hand, using Monte-Carlo planning
permits scaling up the biologically grounded approach of Active Inference to
large-scale problems. On the other hand, the theory of Active Inference
provides a principled solution to the balance of exploration and exploitation,
which is often addressed heuristically in Monte-Carlo methods. Our simulations
show that Active Tree Search successfully navigates binary trees that are
challenging for sampling-based methods, problems that require adaptive
exploration, and the large POMDP problem Rocksample. Furthermore, we illustrate
how Active Tree Search can be used to simulate neurophysiological responses
(e.g., in the hippocampus and prefrontal cortex) of humans and other animals
that contain large planning problems. These simulations show that Active Tree
Search is a principled realisation of neuroscientific and AI theories of
planning, which offers both biological realism and scalability.
