This prior work considers models with bounded cost functions, and imposes a strong form of uniform ergodicity in order to show that a relative value function exists and is bounded. P. W. Glynn and S. P. Meyn. A Lyapunov bound for solutions of Poisson's equation . 24, April 1996. A Lyapunov bound for solutions of Poisson's equation .
Value iteration converges to form a regular policy which is optimal, and non idling.
P. W. Glynn and S. P. Meyn, A Lyapunov bound for solutions of Poisson's equation , Ann. We establish in this paper the following bounds on the relative value functions fh n g. Results of this kind are further developed in the appendix. A Lyapunov bound for solutions of Poisson's equation . 24, April 1996.