Title: DT-Pro: Proactive Decision Transformers with Implicit Latent Space Planning
Primary area: reinforcement learning
Keywords: Reinforcement Learning, Sequential Decision Making

Abstract: Decision Transformers (DTs) address decision making problems through sequence modeling and have achieved surprisingly strong results. However, DTs still struggle in long-horizon tasks due to their poor planning ability. Existing works have demonstrated that subgoal prediction helps to guide DTs' decision making in complex and long-horizon tasks. However, explicit planning via subgoal prediction suffers from suboptimality, inefficiency and instability. In this paper, we present DT-Pro, a variant of DT that enhances its planning ability by integrating a natural implicit planning step into sequence modeling. Compared with explicit planning via subgoal prediction, the implicit planning works by inferring a latent plan from a structured plan space. Through this way, DT-Pro enables high-quality adaptive plan generation and efficient stepwise replanning with only a marginal increase in the computational cost.  Extensive experimental results show that DT-Pro achieves strong performance on a variety of widely used control and navigation benchmarks.

TL;DR: In this paper, we present DT-Pro, a variant of DT that enhances its planning ability by integrating a natural implicit planning step into sequence modeling.