The MAXQ Method for Hierarchical Reinforcement Learning
Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms