Policy iteration for bounded-parameter POMDPs
POMDP is considered as a basic model for decision making under uncertainty. As a generalization of the exact POMDP model, the bounded-parameter POMDP (BPOMDP) provides only upper and lower bounds on the state-transition probabilities, observation probabilities and rewards, which is particularly suitable for characterizing the situations where the underlying model is imprecisely given or time-varying. This paper presents the optimistic criterion for optimality for solving BPOMDPs, under which the optimistically optimal value function is defined. By representing a policy explicitly as a finite-state controller, we propose a policy iteration approach that is shown to converge to an \(\epsilon\)-optimal policy under the optimistic optimality criterion.
policy iteration bounded-parameter pomdp decision making under uncertainty optimistic optimality finite-state controller \(\epsilon\)-optimal policy