When robots operate in the real world, they need to handle uncertainties in sensing, acting, and the environment dynamics. Many tasks also require reasoning about long-term consequences of robot decisions. The partially observable Markov decision process (POMDP) offers a principled approach for planning under uncertainty. However, its computational complexitygrows exponentially with the planning horizon. We propose to use temporally-extended macro-actions  to cut down the effective planning horizon and thus the exponential factor of the complexity. We propose Macro-Action Generator-Critic (MAGIC), an algorithm that learns a macro-action generator using feedback from a planner, and in turn uses the learned macro-actions to condition long-horizon planning. Importantly , the generator is learned to directly maximize the down-stream planning performance. We evaluate MAGIC on several long-term planning tasks , showing that it significantly outperforms planning using primitive actions and hand-crafted macro-actions in both simulation and on a real robot  .
 The partially observable Markov decision process (POMDP) is a principled general framework for robot decision making under uncertainty, but POMDP planning suffers from high computational complexity, when long-term planning is required. While temporally-extended macro-actions help to cut down the effective planning horizon and significantly improve computational efficiency, how do we acquire good macro-actions? This paper proposes Macro-Action Generator-Critic (MAGIC), which performs offline learning of macro-actions optimized for online POMDP planning. Specifically, MAGIC learns a macro-action generator end-to-end, using an online planner's performance as the feedback. During online planning , the generator generates on the fly situation-aware macro-actions conditioned on the robot's belief and the environment context. We evaluated MAGIC on several long-horizon planning tasks both in simulation and on a real robot . The experimental results show that the learned macro-actions offer significant benefits in online planning performance, compared with primitive actions and handcrafted macro-actions .
D
When robots operate in the real world, they need to handle uncertainties in sensing, acting, and the environment dynamics. Many tasks also require reasoning about long-term consequences of robot decisions.
R
offers a principled approach for planning under uncertainty. However, its computational complexitygrows exponentially with the planning horizon. We propose to use
is a principled general framework for robot decision making under uncertainty, but POMDP planning suffers from high computational complexity, when long-term planning is required. While
A
help
R
thus the exponential factor of the complexity. We propose
significantly improve computational efficiency, how do we acquire good macro-actions? This paper proposes
R
an algorithm that
which performs offline learning of macro-actions optimized for online POMDP planning. Specifically, MAGIC
R
using feedback from a planner, and in turn uses the learned macro-actions to condition long-horizon planning. Importantly
end-to-end, using an online planner's performance as the feedback. During online planning
R
is learned to directly maximize the down-stream planning performance. We evaluate
generates on the fly situation-aware macro-actions conditioned on the robot's belief and the environment context. We evaluated
R
long-term planning tasks , showing that it significantly outperforms planning using primitive actions and hand-crafted macro-actions in both
long-horizon planning tasks both in
A
. The experimental results show that the learned macro-actions offer significant benefits in online planning performance, compared with primitive actions and handcrafted macro-actions