Fixed qlearning and sarsa to handle discrete as well as continuous observations in a semi-reasonable way.