Title: From Objects to Skills: Interpretable Meta-Policies for Neural Control
Primary area: reinforcement learning
Keywords: Reinforcement Learning, Interpretable RL, Hierarchical RL

Abstract: Despite its success in learning high-performing policies for diverse control and decision-making tasks, deep reinforcement learning remains difficult to interpret and align due to the black-box nature of its neural network representations. Neuro-symbolic approaches improve transparency by incorporating symbolic reasoning, but when applied to low-level actions, they result in overly complex policies. We introduce LENS, a hierarchical Reinforcement Learning framework that integrates neural skills with neuro-symbolic meta-policies to balance efficiency and interpretability. In its core, it allows transparent reasoning on disentangled high-level actions (i.e. interpretable skills), greatly reducing complexity of symbolic policies. Object-centric representations enable extracting rewards and meta-policies from language models, while the hierarchical structure allow reasoning over skills rather than atomic actions. We experimentally demonstrate that LENS agents are interpretable, less prone to reward hacking, and more robust to distribution shifts. We further evaluate how differing levels of meta-policy interpretability (i.e. purely neural or symbolic) influences performance. Overall, LENS enables interpretable and robust control via neuro-symbolic reasoning over high-level skills.