In this paper, we focus on the case where successive examples are dependent, and we analyze how these dependencies affect the learning process. We define the representation error and the prediction error. In the neighborhood of a local minimum of the error surface, we calculate these errors. We find that the more predictable the example presentation,the higher the representation error, i.e. the less accurate the asymptotic representation of the whole environment. Furthermore we study the learning process in the presence of a plateau.
Plateaus are flat spots on the error surface, which can severely slow down the learning process.
In particular, they are notorious in applications with multilayer perceptrons. Our results, which are confirmed by simulations of a multilayer perceptron learning a chaotic time series using backpropagation, explain how dependencies between examples can help the learning process to escape from a plateau. All Rights Reserved.