Predicting Conditional Probability Densities of Stationary Stochastic Time Series
Feedforward neural networks applied to time series prediction are usually trained to predict the next time step x(t + 1) as a function of m previous values, x(t) := (x(t), x(t + 1),, x(t ? m + 1)), which, if a sum-of-squares error function is chosen, results in predicting the conditional mean ?y|x(t)?. However, further information about the distribution is lost, which is a serious drawback especially in the case of multimodality, where the conditional mean alone turns out to be a rather insufficient or even misleading quantity. The only satisfactory approach in the general case is therefore to predict the whole conditional probability density for the time series, P(x(t + 1)|x(t), x(t ? m 1),, x(t ? + 1)). We deduce here a two-hidden-layer universal approximator network for modelling this function, and develop a training algorithm from maximum likelihood. The method is tested on three time series of different nature, which will demonstrate how state-space dependent variances and multimodal transitions can be learned. We will finally note comparisons with other recent neural network approaches to this problem, and will state results on a benchmark problem.  1997 Elsevier Science Ltd. All Rights Reserved.
conditional probability density universal maximum likelihood stationary markov process moment generating function approximation theorem bayes rule.