Before revision:  To do this, we show that a Partial Differential Equation (PDE) characterizes the evolution of the value function under each policy . The data enables us to obtain a sample version of the PDEthat provides estimates of these value functions. The estimated policy rule is the one with the maximal estimated value function. Using the theory of viscosity solutions to PDEs we show that the policy regret decays at a n^{-1/2} rate in most examples;
After revision:  To do this, we show that a Partial Differential Equation (PDE) characterizes the evolution of the value function under each policy in a Partial Differential Equation (PDE) form and using the theory of viscosity solutions to PDEs we show that the policy regret decays at a n^{-1/2} rate in most examples;