PAE: Reinforcement Learning from External Knowledge for Efficient Exploration
Authors: Zhe Wu,Haofei Lu,Junliang Xing,You Wu,Renye Yan,Yaozhong Gan,Yuanchun Shi
Presentation: ICLR 2024,Poster

Abstract: Human intelligence is adept at absorbing valuable insights from external knowledge.
This capability is equally crucial for artificial intelligence. 
In contrast, classical reinforcement learning agents lack such capabilities and often resort to extensive trial and error to explore the environment. 
This paper introduces $\textbf{PAE}$: $\textbf{P}$lanner-$\textbf{A}$ctor-$\textbf{E}$valuator, a novel framework for teaching agents to $\textit{learn to absorb external knowledge}$. 
PAE integrates the Planner's knowledge-state alignment mechanism, the Actor's mutual information skill control, and the Evaluator's adaptive intrinsic exploration reward to achieve 1) effective cross-modal information fusion, 2) enhanced linkage between knowledge and state, and 3) hierarchical mastery of complex tasks.
Comprehensive experiments across
 11 challenging tasks from the BabyAI and MiniHack environment suites demonstrate PAE's superior exploration efficiency with good interpretability.