Seems to be working at least partially. It is quite slow (compared to model-free), which is expected. I'm getting huge number of warnings from sklearn about the nearest neighbors calls. And although it is doing much much better than sarsa does with the same basis (trivial), it doesn't do nearly as well as sarsa when sarsa has a fourier basis.