Most algorithms for solving POMDPs itera tively improve a value function that implic itly represents a policy and are said to search in value function space. This paper presents an approach to solving POMDPs that repre sents a policy explicitly as a finite-state con troller and iteratively improves the controller by search in policy space. Two related al gorithms illustrate this approach. The first is a policy iteration algorithm that can out perform value iteration in solving infinite horizon POMDPs. It provides the founda tion for a new heuristic search algorithm that promises further speedup by focusing compu tational effort on regions of the problem space that are reachable, or likely to be reached, from a start state.
REF discussed an approach to repeatedly evaluate and improve a finite-state controller.
Solving POMDPs by Searching in Policy Space
ArXiv
ArXiv
Computer Science