Transfer learning can greatly speed up reinforcement learning for a new task by leveraging policies of relevant tasks. Existing works of policy reuse either focus on selecting a single best source policy for reuse without considering contexts, or fail to guarantee learning an optimal policy for a target task. To improve transfer efficiency and guarantee optimality, we develop a novel policy reuse method, called Context-Aware Policy reuSe (CAPS), that enables multi-policy reuse. Our method learns when and which source policy is best for reuse, as well as when to terminate its reuse. CAPS provides theoretical guarantees in convergence and optimality for both source policy selection and target task learning. Empirical results on a gridbased navigation domain and the Pygame Learning Environment demonstrate that CAPS significantly outperforms other state-ofthe-art policy reuse methods.
The Context-Aware Policy Reuse algorithm of REF used options to represent selection of source policies as well as target actions, learning Q-values and termination conditions simultaneously.
Context-Aware Policy Reuse
ArXiv
ArXiv
Computer Science