Before revision:  The first is computationally expensive but provably obtains (\sqrt{B_\star^3 d^3 K/c_{min , where B_\star is a (known)upper bound on the optimal cost-to-go function, d is the feature dimension, K is the number of episodes, and c_{min .
After revision:  The first is computationally expensive but provably obtains (SSP) and bound their regret over K episodes. When all stationary policies are proper, our first algorithm obtains sublinear regret (K^{3/4 , where B_\star is a (known)upper bound on the optimal cost-to-go function, d is the feature dimension, K is the number of episodes, and c_{min .