Above are papers related to the following paper: [Paper_ID] Paper #400356. 	[Title] Factor Analysis Using Delta-Rule Wake-Sleep Learning [Abstract] Technical Report No. 9607, Department of Statistics, University of Toronto We describe a linear network that models correlations between real-valued visible variables using one or more real-valued hidden variables a factor analysis model. This model can be seen as a linear version of the Helmholtz machine, and its parameters can be learned using the wake-sleep method, in which learning of the primary generative model is assisted by a recognition model, whose role is to fill in the values of hidden variables based on the values of visible variables. The generative and recognition models are jointly learned in wake and sleep phases, using just the delta rule. This learning procedure is comparable in simplicity to Oja's version of Hebbian learning, which produces a somewhat different representation of correlations in terms of principal components. We argue that the simplicity of wake-sleep learning makes factor analysis a plau sible alternative to Hebbian learning as a model of activity-dependent cortical plasticity. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #126912. 	[Title] Continuous sigmoidal belief networks trained using slice sampling [Abstract] Real-valued random hidden variables can be useful for modelling latent structure that explains correlations among observed variables. I propose a simple unit that adds zero-mean Gaussian noise to its input before passing it through a sigmoidal squashing function. Such units can produce a variety of useful behaviors, ranging from deterministic to binary stochastic to continuous stochastic. I show how "slice sampling" can be used for inference and learning in top-down networks of these units and demonstrate learning on two simple problems. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #949217. 	[Title] Comparison of Neural and Statistical Classifiers| Theory and Practice [Abstract] Research Reports A13 January 1996 Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #395553. 	[Title] Mixtures of Probabilistic Principal Component Analysers [Abstract] Principal component analysis (PCA) is one of the most popular techniques for processing, compressing and visualising data, although its effectiveness is limited by its global linearity. While nonlinear variants of PCA have been proposed, an alternative paradigm is to capture data complexity by a combination of local linear PCA projections. However, conventional PCA does not correspond to a probability density, and so there is no unique way to combine PCA models. Previous attempts to formulate mixture models for PCA have therefore to some extent been ad hoc. In this paper, PCA is formulated within a maximum-likelihood framework, based on a specific form of Gaussian latent variable model. This leads to a well-defined mixture model for probabilistic principal component analysers, whose parameters can be determined using an EM algorithm. We discuss the advantages of this model in the context of clustering, density modelling and local dimensionality reduction, and we demonstrate its application to image compression and handwritten digit recognition. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1153861. 	[Title] Data Mining for Association Rules with Unsupervised Neural Networks [Abstract] results for Gaussian mixture models and factor analysis are discussed. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #19621. 	[Title] References "Using Neural Networks to Identify Jets", Kohonen, "Self Organized Formation of Topologically Correct Feature [Abstract] 2] D. E. Rumelhart, G. E. Hinton and R. J. Williams, "Learning Internal Representations by Error Propagation", in D. E. Rumelhart and J. L. McClelland (eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition (Vol. 1), MIT Press (1986). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #213279. 	[Title] Constraint Tangent Distance for On-line Character Recognition [Abstract] In on-line character recognition we can observe two kinds of intra-class variations: small geometric deformations and completely different writing styles. We propose a new approach to deal with these problems by defining an extension of tangent distance [9], well known in off-line character recognition. The system has been implemented with a k-nearest neighbor classifier and a so called diabolo classifier [6] respectively. Both classifiers are invariant under transformations like rotation, scale or slope and can deal with variations in stroke order and writing direction. Results are presented for our digit database with more than 200 writers. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #390889. 	[Title] A HIERARCHICAL COMMUNITY OF EXPERTS [Abstract] We describe a directed acyclic graphical model that contains a hierarchy of linear units and a mechanism for dynamically selecting an appropriate subset of these units to model each observation. The non-linear selection mechanism is a hierarchy of binary units each of which gates the output of one of the linear units. There are no connections from linear units to binary units, so the generative model can be viewed as a logistic belief net (Neal 1992) which selects a skeleton linear model from among the available linear units. We show that Gibbs sampling can be used to learn the parameters of the linear and binary units even when the sampling is so brief that the Markov chain is far from equilibrium. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #39210. 	[Title] Pattern analysis and synthesis in attractor neural networks [Abstract] The representation of hidden variable models by attractor neural networks is studied. Memories are stored in a dynamical attractor that is a continuous manifold of fixed points, as illustrated by linear and nonlinear networks with hidden neurons. Pattern analysis and synthesis are forms of pattern completion by recall of a stored memory. Analysis and synthesis in the linear network are performed by bottom-up and top-down connections. In the nonlinear network, the analysis computation additionally requires rectification nonlinearity and inner product inhibition between hidden neurons. One popular approach to sensory processing is based on generative models, which assume that sensory input patterns are synthesized from some underlying hidden variables. For example, the sounds of speech can be synthesized from a sequence of phonemes, and images of a face can be synthesized from pose and lighting variables. Hidden variables are useful because they constitute a simpler representation of the variables that are visible in the sensory input. Using a generative model for sensory processing requires a method of pattern analysis. Given a sensory input pattern, analysis is the recovery of the hidden variables from which it was synthesized. In other words, analysis and synthesis are inverses of each other. There are a number of approaches to pattern analysis. In analysis-by-synthesis, the synthetic model is embedded inside a negative feedback loop[1]. Another approach is to construct a separate analysis model[2]. This paper explores a third approach, in which visible-hidden pairs are embedded as attractive fixed points, or attractors, in the state space of a recurrent neural network. The attractors can be regarded as memories stored in the network, and analysis and synthesis as forms of pattern completion by recall of a memory. The approach is illustrated with linear and nonlinear network architectures. In both networks, the synthetic model is linear, as in principal Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #12330. 	[Title] Recognizing Handwritten Digits Using Mixtures of Linear Models [Abstract] We construct a mixture of locally linear generative models of a collection of pixel-based images of digits, and use them for recognition. Different models of a given digit are used to capture different styles of writing, and new images are classified by evaluating their log-likelihoods under each model. We use an EM-based algorithm in which the M-step is computationally straightforward principal components analysis (PCA). Incorporating tangent-plane information [12] about expected local deformations only requires adding tangent vectors into the sample covariance matrices for the PCA, and it demonstrably improves performance. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1152490. 	[Title] Data Mining for Association Rules with Unsupervised Neural Networks [Abstract] results for Gaussian mixture models and factor analysis are discussed. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1120777. 	[Title] Modelling the Manifolds of Images of Handwritten Digits [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #66990. 	[Title] Generative Models for Discovering Sparse Distributed Representations [Abstract] We describe a hierarchical, generative model that can be viewed as a non-linear generalization of factor analysis and can be implemented in a neural network. The model uses bottom-up, top-down and lateral connections to perform Bayesian perceptual inference correctly. Once perceptual inference has been performed the connection strengths can be updated using a very simple learning rule that only requires locally available information. We demon strate that the network learns to extract sparse, distributed, hierarchical representations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2702. 	[Title] In  Fast Non-Linear Dimension Reduction [Abstract] We present a fast algorithm for non-linear dimension reduction. The algorithm builds a local linear model of the data by merging PCA with clustering based on a new distortion measure. Experiments with speech and image data indicate that the local linear algorithm produces encodings with lower distortion than those built by five layer auto-associative networks. The local linear algorithm is also more than an order of magnitude faster to train. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #107569. 	[Title] Using generative models for handwritten digit recognition [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #156977. 	[Title] The EM Algorithm for Mixtures of Factor Analyzers [Abstract] Technical Report CRG-TR-96-1 May 21, 1996 (revised Feb 27, 1997) Abstract Factor analysis, a statistical method for modeling the covariance structure of high dimensional data using a small number of latent variables, can be extended by allowing different local factor models in different regions of the input space. This results in a model which concurrently performs clustering and dimensionality reduction, and can be thought of as a reduced dimension mixture of Gaussians. We present an exact Expectation-Maximization algorithm for fitting the parameters of this mixture of factor analyzers. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1117501. 	[Title] The Rectified Gaussian Distribution [Abstract] A simple but powerful modification of the standard Gaussian distribution is studied. The variables of the rectified Gaussian are constrained to be nonnegative, enabling the use of nonconvex energy functions. Two multimodal examples, the competitive and cooperative distributions, illustrate the representational power of the rectified Gaussian. Since the cooperative distribution can represent the translations of a pattern, it demonstrates the potential of the rectified Gaussian for modeling pattern manifolds. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #39199. 	[Title] Unsupervised Learning by Convex and Conic Coding [Abstract] Unsupervised learning algorithms based on convex and conic encoders are proposed. The encoders find the closest convex or conic combination of basis vectors to the input. The learning algorithms produce basis vectors that minimize the reconstruction error of the encoders. The convex algorithm develops locally linear models of the input, while the conic algorithm discovers features. Both algorithms are used to model handwritten digits and compared with vector quantization and principal component analysis. The neural network implementations involve feedback connections that project a reconstruction back to the input layer. Please summarise all the given information.

Paper #400356 has the title `Factor Analysis Using Delta-Rule Wake-Sleep Learning  
` and the abstract `Technical Report No. 9607, Department of Statistics, University of Toronto We describe a linear network that models correlations between real-valued visible variables using one or more real-valued hidden variables a factor analysis model. This model can be seen as a linear version of the Helmholtz machine, and its parameters can be learned using the wake-sleep method, in which learning of the primary generative model is assisted by a recognition model, whose role is to fill in the values of hidden variables based on the values of visible variables. The generative and recognition models are jointly learned in wake and sleep phases, using just the delta rule. This learning procedure is comparable in simplicity to Oja's version of Hebbian learning, which produces a somewhat different representation of correlations in terms of principal components. We argue that the simplicity of wake-sleep learning makes factor analysis a plau sible alternative to Hebbian learning as a model of activity-dependent cortical plasticity.
`.
Question: Which category should Paper #400356 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #400356 has the title `Factor Analysis Using Delta-Rule Wake-Sleep Learning  
` and the abstract `Technical Report No. 9607, Department of Statistics, University of Toronto We describe a linear network that models correlations between real-valued visible variables using one or more real-valued hidden variables a factor analysis model. This model can be seen as a linear version of the Helmholtz machine, and its parameters can be learned using the wake-sleep method, in which learning of the primary generative model is assisted by a recognition model, whose role is to fill in the values of hidden variables based on the values of visible variables. The generative and recognition models are jointly learned in wake and sleep phases, using just the delta rule. This learning procedure is comparable in simplicity to Oja's version of Hebbian learning, which produces a somewhat different representation of correlations in terms of principal components. We argue that the simplicity of wake-sleep learning makes factor analysis a plau sible alternative to Hebbian learning as a model of activity-dependent cortical plasticity.
`.
Question: Which category should Paper #400356 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Neural_Networks