  To make good decisions in the real world people need efficient planning
strategies because their computational resources are limited. Knowing which
planning strategies would work best for people in different situations would be
very useful for understanding and improving human decision-making. But our
ability to compute those strategies used to be limited to very small and very
simple planning tasks. To overcome this computational bottleneck, we introduce
a cognitively-inspired reinforcement learning method that can overcome this
limitation by exploiting the hierarchical structure of human behavior. The
basic idea is to decompose sequential decision problems into two sub-problems:
setting a goal and planning how to achieve it. This hierarchical decomposition
enables us to discover optimal strategies for human planning in larger and more
complex tasks than was previously possible. The discovered strategies
outperform existing planning algorithms and achieve a super-human level of
computational efficiency. We demonstrate that teaching people to use those
strategies significantly improves their performance in sequential
decision-making tasks that require planning up to eight steps ahead. By
contrast, none of the previous approaches was able to improve human performance
on these problems. These findings suggest that our cognitively-informed
approach makes it possible to leverage reinforcement learning to improve human
decision-making in complex sequential decision-problems. Future work can
leverage our method to develop decision support systems that improve human
decision making in the real world.
