Okay finally have the basic model based RL working on gridworld. Still needs more testing, but should be equivalent to fitted-rmax.