In the hybrid model, the states and decisions are recorded in a dynamically managed history as decisions are made. The decision that returns the highest Q value is the optimal choice. However, there are several research issues like perceptual aliasing and generalization that must be resolved before Q learning can be considered effective for large scale tasks. These methods have been effective at disambiguating states in initial work. An important future research question is whether similar methods can resolve all of the hidden states in a large, real world application.
Memory approaches to reinforcement learning in nonmarkovian domains .
Carnegie Mellon University, School of Computer Science. The desired quantity at each step of a backgammon game is the continuous valued probability of winning, whereas the desired output of a phoneme recognition problem is a classification decision among 50 or more discrete choices. L.-J. Lin and T. Mitchell. Memory approaches to reinforcement learning in nonMarkovian domains . It seems to work effectively on simple problems, but can suffer from convergence to local optima on more complex problems.