Before revision:  Using the theory of viscosity solutions to PDEs we show that the policy regret decays at a n^{-1/2} rate in most examples;
After revision:  Using the theory of viscosity solutions to PDEs . We find that the policy regret decays at a n^{-1/2} rate in most examples;