This approach works well on small problems but may suffer from local optima in complex problems. Memory approaches to reinforcement learning in non-Markovian domains . One way to include memory in a reinforcement learning algorithm like Q learning is to use a recurrent neural network to represent the Q values. Memory approaches to reinforcement learning in non-Markovian domains . These methods have been e ective at disambiguating states for TD methods in initial studies, but further research is required to determine the extent to which similar methods can resolve signi cant hidden state information in realistic applications.
Memory approaches to reinforcement learning in non-Markovian domains .
In these situations, the only chance at being able to distinguish certain states is by remembering past perceptions and actions. This is called the memorybased approach. It is an attempt to extend the Q learning algorithm to non Markovian domains. For input, it uses not only the current observation, but also a sliding window over the past N observations and actions. In addition, the size of the training problem grows exponentially in the size of the window.