Learning nonlinear constraints with contrastive backpropagation.
Published: 2004
Language: eng
Number of pages: 72
Description: In this thesis, we investigate a class of neural network EBMs that model distributions of data vectors using products of nonlinear constraints. We train the EBMs on trajectories of idealized particles using contrastive backpropagation, evaluate the resulting models, and compare their performance to that of Gaussian mixture models. The results show that, unlike mixture models, EBMs are capable of modelling complex distributions efficiently by taking advantage of the independence between subsets of input variables.If the data distribution has widely separated modes, contrastive divergence learning can fail to learn the relative heights of the modes. We compare three methods of using long-range jumps to overcome this problem.Energy-based models (EBMs) with deterministic hidden variables provide an alternative to causal generative models for density estimation. Features of such EBMs include fast exact inference and the ability to learn distributed representations nonlinearly related to data vectors.
Notes: Adviser:  Geoffrey Minton.

Thesis (M.Sc.)--University of Toronto, 2004.

Electronic version licensed for access by U. of T. users.

Source: Masters Abstracts International, Volume: 43-03, page: 0885.

MICR copy on microfiche (1 microfiche).