The analysis treats realizable stochastic neural networks, trained with Kullback-Leibler loss in the asymptotic case. Our ongoing research focusses on avoiding overfitting in the unrealizable case, which is closer to practical considerations. Competing neural networks can improve short-term prediction of time series which originate from systems with multiple modes of behaviour. We developed a method, where each expert network specializes on a different dynamical mode and the time series will be segmented accordingly. In order to obtain a maximal specialization, the competition is adiabatically increased during training.
Memory is included in order to resolve ambiguities of input-output relations.
We investigated the properties of the method in the case of switching chaotic dynamics and speech. As a further application of our approach we are now considering data streams which originate from biological systems. We are recently considering the incorporation of drift into our algorithm. Future work will be focussed on the blind separation of superimposed time series. Recently we started to apply ICA techniques to physiological data from MEG measurements in the peripheral nerve system and extracted biological and measurement artifacts.