Above are papers related to the following paper: [Paper_ID] Paper #851968. 	[Title] Natural Gradient Descent for Training Multi-Layer Perceptrons [Abstract] The main difficulty in implementing the natural gradient learning rule is to compute the inverse of the Fisher information matrix when the input dimension is large. We have found a new scheme to represent the Fisher information matrix. Based on this scheme, we have designed an algorithm to compute the inverse of the Fisher information matrix. When the input dimension n is much larger than the number of hidden neurons, the complexity of this algorithm is of order O(n 2 ) while the complexity of conventional algorithms for the same purpose is of order O(n 3 ). The simulation has confirmed the efficience and robustness of the natural gradient learning rule. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #578309. 	[Title] Simple Neuron Models for Independent Component Analysis [Abstract] Recently, several neural algorithms have been introduced for Independent Component Analysis. Here we approach the problem from the point of view of a single neuron. First, simple Hebbian-like learning rules are introduced for estimating one of the independent components from sphered data. Some of the learning rules can be used to estimate an independent component which has a negative kurtosis, and the others estimate a component of positive kurtosis. Next, a two-unit system is introduced to estimate an independent component of any kurtosis. The results are then generalized to estimate independent components from non-sphered (raw) mixtures. To separate several independent components, a system of several neurons with linear negative feedback is used. The convergence of the learning rules is rigorously proven without any unnecessary hypotheses on the distributions of the independent components. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1140230. 	[Title] NIPS*97 Multiplicative Updating Rule for Blind Separation Derived from the Method of Scoring [Abstract] For blind source separation, when the Fisher information matrix is used as the Riemannian metric tensor for the parameter space, the steepest descent algorithm to maximize the likelihood function in this Riemannian parameter space becomes the serial updating rule with equivariant property. This algorithm can be further simplified by using the asymptotic form of the Fisher information matrix around the equilibrium. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1140231. 	[Title] NIPS*97 The Efficiency and The Robustness of Natural Gradient Descent Learning Rule [Abstract] We have discovered a new scheme to represent the Fisher information matrix of a stochastic multi-layer perceptron. Based on this scheme, we have designed an algorithm to compute the inverse of the Fisher information matrix. When the input dimension n is much larger than the number of hidden neurons, the complexity of this algorithm is of order O(n 2 ) while the complexity of conventional algorithms for the same purpose is of order O(n 3 ). The inverse of the Fisher information matrix is used in the natural gradient descent algorithm to train single-layer or multi-layer perceptrons. It is confirmed by simulation that the natural gradient Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #205192. 	[Title] Principal and Independent Components in Neural Networks Recent Developments [Abstract] Nonlinear extensions of one-unit and multi-unit Principal Component Analysis (PCA) neural networks, introduced earlier by the authors, are reviewed. The networks and their nonlinear Hebbian learning rules are related to other signal expansions like Projection Pursuit (PP) and Independent Component Analysis (ICA). Separation results for mixtures of real world signals and im ages are given. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1132948. 	[Title] Edges are the `Independent Components' of Natural Scenes. [Abstract] Field (1994) has suggested that neurons with line and edge selectivities found in primary visual cortex of cats and monkeys form a sparse, distributed representation of natural scenes, and Barlow (1989) has reasoned that such responses should emerge from an unsupervised learning algorithm that attempts to find a factorial code of independent visual features. We show here that non-linear `infomax', when applied to an ensemble of natural scenes, produces sets of visual filters that are localised and oriented. Some of these filters are Gabor-like and resemble those produced by the sparseness-maximisation network of Olshausen & Field (1996). In addition, the outputs of these filters are as independent as possible, since the info-max network is able to perform Independent Components Analysis (ICA). We compare the resulting ICA filters and their associated basis functions, with other decorrelating filters produced by Principal Components Analysis (PCA) and zero-phase whitening filters (ZCA). The ICA filters have more sparsely distributed (kurtotic) outputs on natural scenes. They also resemble the receptive fields of simple cells in visual cortex, which suggests that these neurons form an information-theoretic co-ordinate system for images. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #696342. 	[Title] A Blind Identification and Separation Technique via Multi-layer Neural Networks [Abstract] This paper deals with the problem of blind identification and source separation which consists of estimation of the mixing matrix and/or the separation of a mixture of stochastically independent sources without a priori knowledge on the mixing matrix . The method we propose here estimates the mixture matrix by a recurrent Input-Output (IO) Identification using as inputs a nonlinear transformation of the estimated sources. Herein, the nonlinear transformation (distortion) consists in constraining the modulus of the inputs of the IO-Identification device to be a constant. In contrast to other existing approaches, the covariance of the additive noise do not need to be modeled and can be estimated as a regular parameter if needed. The proposed approach is implemented using multi-layer neural networks in order to improve performance of separation. New associated on-line un-supervised adaptive learning rules are also developed. The effectiveness of the proposed method is illustrated by some computer simulations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #696343. 	[Title] On the performance of orthogonal source separation algorithms [Abstract] Source separation consists in recovering a set of n independent signals from m n observed instantaneous mixtures of these signals, possibly corrupted by additive noise. Many source separation algorithms use second order information in a whitening operation which reduces the non trivial part of the separation to determining a unitary matrix. Most of them further show a kind of invariance property which can be exploited to predict some general results about their performance. Our first contribution is to exhibit a lower bound to the performance in terms of accuracy of the separation. This bound is independent of the algorithm and, in the i.i.d. case, of the distribution of the source signals. Second, we show that the performance of invariant algorithms depends on the mixing matrix and on the noise level in a specific way. A consequence is that at low noise levels, the performance does not depend on the mixture but only on the distribution of the sources, via a function which is characteristic of the given source separation algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #696345. 	[Title] SELF-ADAPTIVE NEURAL NETWORKS FOR BLIND SEPARATION OF SOURCES [Abstract] Novel on-line learning algorithms with self adaptive learning rates (parameters) for blind separation of signals are proposed. The main motivation for development of new learning rules is to improve convergence speed and to reduce cross-talking, especially for non-stationary signals. Furthermore, we have discovered that under some conditions the proposed neural network models with associated learning algorithms exhibit a random switch of attention, i.e. they have ability of chaotic or random switching or cross-over of output signals in such way that a specified separated signal may appear at various outputs at different time windows. Validity, performance and dynamic properties of the proposed learning algorithms are investigated by computer simulation experiments. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #696346. 	[Title] LOCAL ADAPTIVE LEARNING ALGORITHMS FOR BLIND SEPARATION OF NATURAL IMAGES [Abstract] In this paper a neural network approach for reconstruction of natural highly correlated images from linear (additive) mixture of them is proposed. A multi-layer architecture with local on-line learning rules is developed to solve the problem of blind separation of sources. The main motivation for using a multi-layer network instead of a single-layer one is to improve the performance and robustness of separation, while applying a very simple local learning rule, which is biologically plausible. Moreover such architecture with on-chip learning is relatively easy implementable using VLSI electronic circuits. Furthermore it enables the extraction of source signals sequentially one after the other, starting from the strongest signal and finishing with the weakest one. The experimental part focuses on separating highly correlated human faces from mixture of them, with additive noise and under unknown number of sources. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #578337. 	[Title] Signal Separation by Nonlinear Hebbian Learning [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #751408. 	[Title] Blind separation of delayed and convolved sources. [Abstract] We address the difficult problem of separating multiple speakers with multiple microphones in a real room. We combine the work of Torkkola and Amari, Cichocki and Yang, to give Natural Gradient information maximisation rules for recurrent (IIR) networks, blindly adjusting delays, separating and deconvolving mixed signals. While they work well on simulated data, these rules fail in real rooms which usually involve non-minimum phase transfer functions, not-invertible using stable IIR filters. An approach that sidesteps this problem is to perform infomax on a feedforward architecture in the frequency domain (Lambert 1996). We demonstrate real-room separation of two natural signals using this approach. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1130808. 	[Title] Independent Component Analysis of Simulated EEG Using a Three-Shell Spherical Head Model 1 [Abstract] 1 This report was supported in part by the Navy Medical Research and Development Command and the Office of Naval Research, Department of the Navy under work unit ONR.Reimb-6429. The views expressed in this article are those of the authors and do not reflect the official policy or position of the Department of the Navy, Department of Defense, or the U.S. Government. Approved for public release, distribution unlimited. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1132486. 	[Title] Working Paper IS-97-22 (Information Systems) A First Application of Independent Component Analysis to Extracting Structure [Abstract] This paper discusses the application of a modern signal processing technique known as independent component analysis (ICA) or blind source separation to multivariate financial time series such as a portfolio of stocks. The key idea of ICA is to linearly map the observed multivariate time series into a new space of statistically independent components (ICs). This can be viewed as a factorization of the portfolio since joint probabilities become simple products in the coordinate system of the ICs. We apply ICA to three years of daily returns of the 28 largest Japanese stocks and compare the results with those obtained using principal component analysis. The results indicate that the estimated ICs fall into two categories, (i) infrequent but large shocks (responsible for the major changes in the stock prices), and (ii) frequent smaller fluctuations (contributing little to the overall level of the stocks). We show that the overall stock price can be reconstructed surprisingly well by using a small number of thresholded weighted ICs. In contrast, when using shocks derived from principal components instead of independent components, the reconstructed price is less similar to the original one. Independent component analysis is a potentially powerful method of analyzing and understanding driving mechanisms in financial markets. There are further promising applications to risk management since ICA focuses on higher order statistics. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #627024. 	[Title] Early Stopping but when? [Abstract] Validation can be used to detect when overfitting starts during supervised training of a neural network; training is then stopped before convergence to avoid the overfitting ("early stopping"). The exact criterion used for validation-based early stopping, however, is usually chosen in an ad-hoc fashion or training is stopped interactively. This trick describes how to select a stopping criterion in a systematic fashion; it is a trick for either speeding learning procedures or improving generalization, whichever is more important in the particular situation. An empirical investigation on multi-layer perceptrons shows that there exists a tradeoff between training time and generalization: From the given mix of 1296 training runs using different 12 problems and 24 different network architectures I conclude slower stopping criteria allow for small improvements in generalization (here: about 4% on average), but cost much more training time (here: about factor 4 longer on average). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #99030. 	[Title] The Nonlinear PCA Learning Rule and Signal Separation Mathematical Analysis [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #236759. 	[Title] NEURAL NETWORK APPROACH TO BLIND SEPARATION AND ENHANCEMENT OF IMAGES [Abstract] In this contribution we propose a new solution for the problem of blind separation of sources (for one dimensional signals and images) in the case that not only the waveform of sources is unknown, but also their number. For this purpose multi-layer neural networks with associated adaptive learning algorithms are developed. The primary source signals can have any non-Gaussian distribution, i.e. they can be sub-Gaussian and/or super-Gaussian. Computer experiments are presented which demonstrate the validity and high performance of the proposed approach. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #230879. 	[Title] Equivariant adaptive source separation [Abstract] Source separation consists in recovering a set of independent signals when only mixtures with unknown coefficients are observed. This paper introduces a class of adaptive algorithms for source separation which implements an adaptive version of equivariant estimation and is henceforth called EASI (Equivariant Adaptive Separation via Independence). The EASI algorithms are based on the idea of serial updating: this specific form of matrix updates systematically yields algorithms with a simple, parallelizable structure, for both real and complex mixtures. Most importantly, the performance of an EASI algorithm does not depend on the mixing matrix. In particular, convergence rates, stability conditions and interference rejection levels depend only on the (normalized) distributions of the source signals. Close form expressions of these quantities are given via an asymptotic performance analysis. This is completed by some numerical experiments illustrating the effectiveness of the proposed approach. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2658. 	[Title] A New Learning Algorithm for Blind Signal Separation [Abstract] A new on-line learning algorithm which minimizes a statistical dependency among outputs is derived for blind separation of mixed signals. The dependency is measured by the average mutual information (MI) of the outputs. The source signals and the mixing matrix are unknown except for the number of the sources. The Gram-Charlier expansion instead of the Edgeworth expansion is used in evaluating the MI. The natural gradient approach is used to minimize the MI. A novel activation function is proposed for the on-line learning algorithm which has an equivariant property and is easily implemented on a neural network like model. The validity of the new learning algorithm is verified by computer simulations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #230884. 	[Title] BLIND SEPARATION OF DELAYED SOURCES BASED ON INFORMATION MAXIMIZATION [Abstract] Recently, Bell and Sejnowski have presented an approach to blind source separation based on the information maximization principle. We extend this approach into more general cases where the sources may have been delayed with respect to each other. We present a network architecture capable of coping with such sources, and we derive the adaptation equations for the delays and the weights in the network by maximizing the information transferred through the network. Examples using wideband sources such as speech are presented to illustrate the algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1133428. 	[Title] Maximum likelihood source separation for discrete sources [Abstract] This communication deals with the source separation problem which consists in the separation of a noisy mixture of independent sources without a priori knowledge of the mixture coefficients. In this paper, we consider the maximum likelihood (ML) approach for discrete source signals with known probability distributions. An important feature of the ML approach in Gaussian noise is that the covariance matrix of the additive noise can be treated as a parameter. Hence, it is not necessary to know or to model the spatial structure of the noise. Another striking feature offered in the case of discrete sources is that, under mild assumptions, it is possible to separate more sources than sensors. In this paper, we consider maximization of the likelihood via the Expectation-Maximization (EM) algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #278394. 	[Title] Statistical Theory of Overtraining Is Cross-Validation Asymptotically Effective? [Abstract] A statistical theory for overtraining is proposed. The analysis treats realizable stochastic neural networks, trained with Kullback-Leibler loss in the asymptotic case. It is shown that the asymptotic gain in the generalization error is small if we perform early stopping, even if we have access to the optimal stopping time. Considering cross-validation stopping we answer the question: In what ratio the examples should be divided into training and testing sets in order to obtain the optimum performance. In the non-asymptotic region cross-validated early stopping always decreases the generalization error. Our large scale simulations done on a CM5 are in nice agreement with our analytical findings. Please summarise all the given information.

Paper #851968 has the title `Natural Gradient Descent for Training Multi-Layer Perceptrons  
` and the abstract `The main difficulty in implementing the natural gradient learning rule is to compute the inverse of the Fisher information matrix when the input dimension is large. We have found a new scheme to represent the Fisher information matrix. Based on this scheme, we have designed an algorithm to compute the inverse of the Fisher information matrix. When the input dimension n is much larger than the number of hidden neurons, the complexity of this algorithm is of order O(n 2 ) while the complexity of conventional algorithms for the same purpose is of order O(n 3 ). The simulation has confirmed the efficience and robustness of the natural gradient learning rule.
`.
Question: Which category should Paper #851968 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #851968 has the title `Natural Gradient Descent for Training Multi-Layer Perceptrons  
` and the abstract `The main difficulty in implementing the natural gradient learning rule is to compute the inverse of the Fisher information matrix when the input dimension is large. We have found a new scheme to represent the Fisher information matrix. Based on this scheme, we have designed an algorithm to compute the inverse of the Fisher information matrix. When the input dimension n is much larger than the number of hidden neurons, the complexity of this algorithm is of order O(n 2 ) while the complexity of conventional algorithms for the same purpose is of order O(n 3 ). The simulation has confirmed the efficience and robustness of the natural gradient learning rule.
`.
Question: Which category should Paper #851968 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Neural_Networks