I will present some dynamic Bayesian network models which can capture much richer structure than HMMs and Kalman filters, including spatial and temporal multiresolution structure, distributed hidden state representations, and multiple switching linear regimes. While exact probabilistic inference is intractable in these networks, one can obtain tractable variational approximations which call as subroutines the forward-backward and Kalman filter recursions. These approximations can be used to learn the model parameters by maximizing a lower bound on the likelihood. Other examples or behaviors or systems will be given as time permits. Recently, the generic architecture has been proven to be a universal approximator of mappings between rooted labeled ordered trees and real vector spaces.
All proofs are carried out in a constructive way, an analysis of the space complexity is provided.
The computation of recursive networks rely on on a set of real-valued state and input variables which are processed using a set of continuous operators such as multipliers, adders and sigmoidal functions. When processing each node in a graph, a real valued state vector is computed using the node label and the state vector of each child. Thus the state memorizes the relevant information extracted from the node descendants and the information available at the current node. On the other side when the graph nodes are labelled with symbols, classical symbolic processing models can be used. This category of computational devices is characterized by a finite number of internal states and by a finite input alphabet.