Unfortunately, there is still a significant gap between this potential and actual applications, primarily due to the lack of effective solution methods. The sequence converges to the optimal value function.
Value iteration terminates when a predetermined convergence condition is met.
Value iteration performs typically a large number of DP updates before it converges and DP updates are notoriously expensive. In this paper, we develop a technique for reducing the number of DP updates. The output value function is closer to the optimal than the input value function. In this sense, we say that DP update improves its input. We propose an approximation to DP update called point-based DP update .