Scoring-Aggregating-Planning: Learning task-agnostic priors from interactions and sparse rewards for zero-shot generalization
Humans can learn task-agnostic priors from interactive experience and utilize the priors for novel tasks without any finetuning.In this paper, we propose Scoring-Aggregating-Planning, a framework that can learn task-agnostic semantics and dynamics priors from arbitrary quality interactions as well as the corresponding sparse rewards and then plan on unseen tasks in zero-shot condition.The framework finds a neural score function for local regional state and action pairs that can be aggregated to approximate the quality of a full trajectory; moreover, a dynamics model that is learned with self-supervision can be incorporated for planning.Many of previous works that leverage interactive data for policy learning either need massive on-policy environmental interactions or assume access to expert data while we can achieve a similar goal with pure off-policy imperfect data.Instantiating our framework results in a generalizable policy to unseen tasks.Experiments demonstrate that the proposed method can outperform baseline methods on a wide range of applications including gridworld, robotics tasks and video games.
We learn dense scores and dynamics model as priors from exploration data and use them to induce a good policy in new tasks in zero-shot condition.This paper discusses zero shot generalization into new environments, and proposes an approach with results on Grid-World, Super Mario Bros, and 3D Robotics.A method aiming to learn task-agnostic priors for zero-shot generalization, with the idea to employ a modeling approach on top of the model-based RL framework.