Cognitive Neurorobotics Research Unit
認知脳ロボティクス研究ユニット

Goal-Directed Planning for Habituated Agents by Active Inference
Using a Variational Recurrent Neural Network
Takazumi Matsumoto & Jun Tani**Corresponding author: tani1216jp@gmail.com

Motivation

Latent variables

How can agents achieve goals given only partial knowledge of the world? It isgenerally assumed that agents can never have complete knowledge of theirenvironment because the experience that can be gained in a finite amount oftime is limited. However, some agents such as humans are able to generalize
from experience to form action plans that accomplish unfamiliar tasks.
In our proposed model, generalization is achieved by learning probabilistic
patterns from well habituated sensory-motor trajectories. These priorPrediction error
distributions are stored in a low dimensional latent state space. Goal-directed
planning is accomplished by inferring latent variables which maximizes theestimated lower bound, following the principle of Free Energy Minimization [1].

Goal directed planning using PV-RNN

MethodFM

PC + AIF

Our proposed model (GLean) uses the frameworks of predictive coding (PC) [2] and activeinference (AIF) [3], and leverages the PV-RNN architecture [4] to learn probabilistic patterns as aprior distribution zp. For plan generation, given zp, a known initial condition and desired goal, we
use an estimate of the evidence based lower bound to infer a posterior distribution zq that leads
to generation of a plausible action plan and sensory prediction.
In comparison, the forward model (FM) [5] is a conventional approach for sensory-motor systems
and uses the current state and motor command to predict the next state and associated sensory
state. In theory it is possible to infer a sequence of optimal motor commands to reach a desired
goal state, however in practice it is impractical to learn such a combination with limited training.

Experimental Results

Accuracy at t=1 (start) and t=T (goal)

Our model (GLean) was evaluated with simulated robot tasks inprobabilistic settings and demonstrated generalization with limited
training data by setting an appropriate regularization coefficient.
GLean also outperformed both a conventional forward model (FM),and a model with stochasticity only in the initial state (SI) [6] in goaldirected planning, due to the learned prior directing the search ofmotor plans within the range of habituated trajectories.
ModelGLean
Stochastic initial state (SI)
Forward model (FM)

Success rate
86.0%
68.0%
0.0%

Avg. error at goal
1.52±0.07cm
2.02cm±0.14cm-Success rate and error for GLean and two alternative methods

Simulated robot arm grasping a block and moving it to the goal

Complexity at t=1 (start) and t=T (goal)

Estimated lower boundConclusion & Ongoing Work
This work demonstrates our approach to generating actionplans and sensory predictions for a robot to achieve untrained
goals by generalizing from limited experience.
We are currently working on real time planning using physicalrobot hardware, with the robot able to dynamically alter its
plan in response to changes in its environment.

GLeanSIFM

Sensory state predictions (solid) compared to ground truth (dotted)[1] K. Friston, J. Kilner, and L. Harrison. A free energy principle for the brain. Journal of Physiology, 100(1–3):70–87, 2006
[2] K. Friston. Does predictive coding have a future? Nature Neuroscience, 21:1019–1021, 2018
[3] G. Oliver, P. Lanillos, and G. Cheng. Active inference body perception and action for humanoid robots. arXiv:1906.03022, 2019[4] A. Ahmadi and J. Tani. A novel predictive-coding-inspired variational RNN model for online prediction and recognition. NeuralComputation, 31(11):2025–2074, 2019[5] M. Kawato. Internal models for motor control and trajectory planning. Current Opinion in Neurobiology, 9(6):718–727, 1999
[6] M. Jung, T. Matsumoto, and J. Tani. Goal-directed behavior under variational predictive coding: Dynamic organization of visualattention and working memory. IROS 2019, 1040–1047