The analysis treats realizable stochastic neural networks, trained with Kullback-Leibler loss in the asymptotic case. Our ongoing research focusses on avoiding overfitting in the unrealizable case, which is closer to practical considerations. Competing neural networks can improve short-term prediction of time series which originate from systems with multiple modes of behaviour. We developed a method, where each expert network specializes on a different dynamical mode and the time series will be segmented accordingly. In order to obtain a maximal specialization, the competition is adiabatically increased during training.
Memory is included in order to resolve ambiguities of input-output relations.
We investigated the properties of the method in the case of switching chaotic dynamics and speech. As a further application of our approach we are now considering data streams which originate from biological systems. We are recently considering the incorporation of drift into our algorithm. Future work will be focussed on the blind separation of superimposed time series. We are very interested in this approach since it's mathematical beauty and high practical performance.