Prioritizing Bellman backups without a priority queue
Topological Value Iteration Algorithm for Markov Decision Processes