Value iteration seems to work?
Merge branch 'master' of https://github.com/sagieske/AA1