This paper presents two new approaches to de composing and solving large Markov decision problems (MDPs), a partial decoupling method and a complete decoupling method. In these ap proaches, a large, stochastic decision problem is divided into smaller pieces. The first approach builds a cache of policies for each part of the problem independently, and then combines the pieces in a separate, light-weight step. A second approach also divides the problem into smaller pieces, but information is communicated be tween the different problem pieces, allowing in telligent decisions to be made about which piece requires the most attention. Both approaches can be used to find optimal policies or approximately optimal policies with provable bounds. These al gorithms also provide a framework for the effi cient transfer of knowledge across problems that share similar structure.
Other methods incrementally build a cache of policies for a decomposed MDP REF , but also following a hierarchical approach.
Flexible Decomposition Algorithms for Weakly Coupled Markov Decision Problems
ArXiv
ArXiv
Computer Science