Before revision:  Existing analysis of neural TD learningrelies on either infinite width-analysis or constraining the network parameters in a (random) compact set; as a result, an extra projection step is required at each iteration. This paper establishes a new convergence analysis of neural TD learningwithout any projection .
After revision:  without any projection .