Before revision:  This paper establishes a new convergence analysis of neural TD learningwithout any projection . We show that the projection-free TD learning equipped with a two-layer ReLU network of any width exceeding poly( , 1/\epsilon) converges to the true value function with error \epsilon given poly( , 1/\epsilon) iterations or samples, where is an upper bound on the RKHS norm of the value function induced by the neural tangent kernel.
After revision:  This paper establishes a new convergence analysis of neural TD learningwithout any projection  , 1/\epsilon) converges to the true value function with error \epsilon given poly( , 1/\epsilon) iterations or samples, where is an upper bound on the RKHS norm of the value function induced by the neural tangent kernel.