Above are papers related to the following paper: [Paper_ID] Paper #696342. 	[Title] A Blind Identification and Separation Technique via Multi-layer Neural Networks [Abstract] This paper deals with the problem of blind identification and source separation which consists of estimation of the mixing matrix and/or the separation of a mixture of stochastically independent sources without a priori knowledge on the mixing matrix . The method we propose here estimates the mixture matrix by a recurrent Input-Output (IO) Identification using as inputs a nonlinear transformation of the estimated sources. Herein, the nonlinear transformation (distortion) consists in constraining the modulus of the inputs of the IO-Identification device to be a constant. In contrast to other existing approaches, the covariance of the additive noise do not need to be modeled and can be estimated as a regular parameter if needed. The proposed approach is implemented using multi-layer neural networks in order to improve performance of separation. New associated on-line un-supervised adaptive learning rules are also developed. The effectiveness of the proposed method is illustrated by some computer simulations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #851968. 	[Title] Natural Gradient Descent for Training Multi-Layer Perceptrons [Abstract] The main difficulty in implementing the natural gradient learning rule is to compute the inverse of the Fisher information matrix when the input dimension is large. We have found a new scheme to represent the Fisher information matrix. Based on this scheme, we have designed an algorithm to compute the inverse of the Fisher information matrix. When the input dimension n is much larger than the number of hidden neurons, the complexity of this algorithm is of order O(n 2 ) while the complexity of conventional algorithms for the same purpose is of order O(n 3 ). The simulation has confirmed the efficience and robustness of the natural gradient learning rule. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #578309. 	[Title] Simple Neuron Models for Independent Component Analysis [Abstract] Recently, several neural algorithms have been introduced for Independent Component Analysis. Here we approach the problem from the point of view of a single neuron. First, simple Hebbian-like learning rules are introduced for estimating one of the independent components from sphered data. Some of the learning rules can be used to estimate an independent component which has a negative kurtosis, and the others estimate a component of positive kurtosis. Next, a two-unit system is introduced to estimate an independent component of any kurtosis. The results are then generalized to estimate independent components from non-sphered (raw) mixtures. To separate several independent components, a system of several neurons with linear negative feedback is used. The convergence of the learning rules is rigorously proven without any unnecessary hypotheses on the distributions of the independent components. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1140230. 	[Title] NIPS*97 Multiplicative Updating Rule for Blind Separation Derived from the Method of Scoring [Abstract] For blind source separation, when the Fisher information matrix is used as the Riemannian metric tensor for the parameter space, the steepest descent algorithm to maximize the likelihood function in this Riemannian parameter space becomes the serial updating rule with equivariant property. This algorithm can be further simplified by using the asymptotic form of the Fisher information matrix around the equilibrium. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #205192. 	[Title] Principal and Independent Components in Neural Networks Recent Developments [Abstract] Nonlinear extensions of one-unit and multi-unit Principal Component Analysis (PCA) neural networks, introduced earlier by the authors, are reviewed. The networks and their nonlinear Hebbian learning rules are related to other signal expansions like Projection Pursuit (PP) and Independent Component Analysis (ICA). Separation results for mixtures of real world signals and im ages are given. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #395540. 	[Title] Maximum Likelihood and Covariant Algorithms for Independent Component Analysis somewhat more biologically plausible, involving no [Abstract] Bell and Sejnowski (1995) have derived a blind signal processing algorithm for a non-linear feedforward network from an information maximization viewpoint. This paper first shows that the same algorithm can be viewed as a maximum likelihood algorithm for the optimization of a linear generative model. Third, this paper gives a partial proof of the `folk-theorem' that any mixture of sources with high-kurtosis histograms is separable by the classic ICA algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1132948. 	[Title] Edges are the `Independent Components' of Natural Scenes. [Abstract] Field (1994) has suggested that neurons with line and edge selectivities found in primary visual cortex of cats and monkeys form a sparse, distributed representation of natural scenes, and Barlow (1989) has reasoned that such responses should emerge from an unsupervised learning algorithm that attempts to find a factorial code of independent visual features. We show here that non-linear `infomax', when applied to an ensemble of natural scenes, produces sets of visual filters that are localised and oriented. Some of these filters are Gabor-like and resemble those produced by the sparseness-maximisation network of Olshausen & Field (1996). In addition, the outputs of these filters are as independent as possible, since the info-max network is able to perform Independent Components Analysis (ICA). We compare the resulting ICA filters and their associated basis functions, with other decorrelating filters produced by Principal Components Analysis (PCA) and zero-phase whitening filters (ZCA). The ICA filters have more sparsely distributed (kurtotic) outputs on natural scenes. They also resemble the receptive fields of simple cells in visual cortex, which suggests that these neurons form an information-theoretic co-ordinate system for images. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #696343. 	[Title] On the performance of orthogonal source separation algorithms [Abstract] Source separation consists in recovering a set of n independent signals from m n observed instantaneous mixtures of these signals, possibly corrupted by additive noise. Many source separation algorithms use second order information in a whitening operation which reduces the non trivial part of the separation to determining a unitary matrix. Most of them further show a kind of invariance property which can be exploited to predict some general results about their performance. Our first contribution is to exhibit a lower bound to the performance in terms of accuracy of the separation. This bound is independent of the algorithm and, in the i.i.d. case, of the distribution of the source signals. Second, we show that the performance of invariant algorithms depends on the mixing matrix and on the noise level in a specific way. A consequence is that at low noise levels, the performance does not depend on the mixture but only on the distribution of the sources, via a function which is characteristic of the given source separation algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #696345. 	[Title] SELF-ADAPTIVE NEURAL NETWORKS FOR BLIND SEPARATION OF SOURCES [Abstract] Novel on-line learning algorithms with self adaptive learning rates (parameters) for blind separation of signals are proposed. The main motivation for development of new learning rules is to improve convergence speed and to reduce cross-talking, especially for non-stationary signals. Furthermore, we have discovered that under some conditions the proposed neural network models with associated learning algorithms exhibit a random switch of attention, i.e. they have ability of chaotic or random switching or cross-over of output signals in such way that a specified separated signal may appear at various outputs at different time windows. Validity, performance and dynamic properties of the proposed learning algorithms are investigated by computer simulation experiments. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #696346. 	[Title] LOCAL ADAPTIVE LEARNING ALGORITHMS FOR BLIND SEPARATION OF NATURAL IMAGES [Abstract] In this paper a neural network approach for reconstruction of natural highly correlated images from linear (additive) mixture of them is proposed. A multi-layer architecture with local on-line learning rules is developed to solve the problem of blind separation of sources. The main motivation for using a multi-layer network instead of a single-layer one is to improve the performance and robustness of separation, while applying a very simple local learning rule, which is biologically plausible. Moreover such architecture with on-chip learning is relatively easy implementable using VLSI electronic circuits. Furthermore it enables the extraction of source signals sequentially one after the other, starting from the strongest signal and finishing with the weakest one. The experimental part focuses on separating highly correlated human faces from mixture of them, with additive noise and under unknown number of sources. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1135899. 	[Title] BLIND SEPARATION OF REAL WORLD AUDIO SIGNALS USING OVERDETERMINED MIXTURES [Abstract] We discuss the advantages of using overdetermined mixtures to improve upon blind source separation algorithms that are designed to extract sound sources from acoustic mixtures. A study of the nature of room impulse responses helps us choose an adaptive filter architecture. We use ideal inverses of acquired room impulse responses to compare the effectiveness of different-sized separating filter configurations of various filter lengths. Using a multi-channel blind least-mean-square algorithm (MBLMS), we show that, by adding additional sensors, we can improve upon the separation of signals mixed with real world filters. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #578337. 	[Title] Signal Separation by Nonlinear Hebbian Learning [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #578347. 	[Title] Independent Component Analysis by General Non-linear Hebbian-like Learning Rules [Abstract] A number of neural learning rules have been recently proposed for Independent Component Analysis (ICA). The rules are usually derived from information-theoretic criteria such as maximum entropy or minimum mutual information. In this paper, we show that in fact, ICA can be performed by very simple Hebbian or anti-Hebbian learning rules, which may have only weak relations to such information-theoretical quantities. Rather suprisingly, practically any non-linear function can be used in the learning rule, provided only that the sign of the Hebbian/anti-Hebbian term is chosen correctly. In addition to the Hebbian-like mechanism, the weight vector is here constrained to have unit norm, and the data is preprocessed by prewhitening, or sphering. These results imply that one can choose the non-linearity so as to optimize desired statistical or numerical criteria. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #751408. 	[Title] Blind separation of delayed and convolved sources. [Abstract] We address the difficult problem of separating multiple speakers with multiple microphones in a real room. We combine the work of Torkkola and Amari, Cichocki and Yang, to give Natural Gradient information maximisation rules for recurrent (IIR) networks, blindly adjusting delays, separating and deconvolving mixed signals. While they work well on simulated data, these rules fail in real rooms which usually involve non-minimum phase transfer functions, not-invertible using stable IIR filters. An approach that sidesteps this problem is to perform infomax on a feedforward architecture in the frequency domain (Lambert 1996). We demonstrate real-room separation of two natural signals using this approach. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1130676. 	[Title] Learning overcomplete representations [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1130808. 	[Title] Independent Component Analysis of Simulated EEG Using a Three-Shell Spherical Head Model 1 [Abstract] 1 This report was supported in part by the Navy Medical Research and Development Command and the Office of Naval Research, Department of the Navy under work unit ONR.Reimb-6429. The views expressed in this article are those of the authors and do not reflect the official policy or position of the Department of the Navy, Department of Defense, or the U.S. Government. Approved for public release, distribution unlimited. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1132486. 	[Title] Working Paper IS-97-22 (Information Systems) A First Application of Independent Component Analysis to Extracting Structure [Abstract] This paper discusses the application of a modern signal processing technique known as independent component analysis (ICA) or blind source separation to multivariate financial time series such as a portfolio of stocks. The key idea of ICA is to linearly map the observed multivariate time series into a new space of statistically independent components (ICs). This can be viewed as a factorization of the portfolio since joint probabilities become simple products in the coordinate system of the ICs. We apply ICA to three years of daily returns of the 28 largest Japanese stocks and compare the results with those obtained using principal component analysis. The results indicate that the estimated ICs fall into two categories, (i) infrequent but large shocks (responsible for the major changes in the stock prices), and (ii) frequent smaller fluctuations (contributing little to the overall level of the stocks). We show that the overall stock price can be reconstructed surprisingly well by using a small number of thresholded weighted ICs. In contrast, when using shocks derived from principal components instead of independent components, the reconstructed price is less similar to the original one. Independent component analysis is a potentially powerful method of analyzing and understanding driving mechanisms in financial markets. There are further promising applications to risk management since ICA focuses on higher order statistics. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #282700. 	[Title] LEARNING LINEAR, SPARSE, FACTORIAL CODES [Abstract] In previous work (Olshausen & Field 1996), an algorithm was described for learning linear sparse codes which, when trained on natural images, produces a set of basis functions that are spatially localized, oriented, and bandpass (i.e., wavelet-like). This note shows how the algorithm may be interpreted within a maximum-likelihood framework. Several useful insights emerge from this connection: it makes explicit the relation to statistical independence (i.e., factorial coding), it shows a formal relationship to the algorithm of Bell and Sejnowski (1995), and it suggests how to adapt parameters that were previously fixed. This report describes research done within the Center for Biological and Computational Learning in the Department of Brain and Cognitive Sciences at the Massachusetts Institute of Technology. This research is sponsored by an Individual National Research Service Award to B.A.O. (NIMH F32-MH11062) and by a grant from the National Science Foundation under contract ASC-9217041 (this award includes funds from ARPA provided under the HPCC program) to CBCL. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #99023. 	[Title] A Fast Fixed-Point Algorithm for Independent Component Analysis [Abstract] This paper will appear in Neural Computation, 9:1483-1492, 1997. Abstract We introduce a novel fast algorithm for Independent Component Analysis, which can be used for blind source separation and feature extraction. It is shown how a neural network learning rule can be transformed into a txed-point iteration, which provides an algorithm that is very simple, does not depend on any user-detned parameters, and is fast to converge to the most accurate solution allowed by the data. The algorithm tnds, one at a time, all non-Gaussian independent components, regardless of their probability distributions. The computations can be performed either in batch mode or in a semi-adaptive manner. The convergence of the algorithm is rigorously proven, and the convergence speed is shown to be cubic. Some comparisons to gradient based algorithms are made, showing that the new algorithm is usually 10 to 100 times faster, sometimes giving the solution in just a few iterations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #99030. 	[Title] The Nonlinear PCA Learning Rule and Signal Separation Mathematical Analysis [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #236759. 	[Title] NEURAL NETWORK APPROACH TO BLIND SEPARATION AND ENHANCEMENT OF IMAGES [Abstract] In this contribution we propose a new solution for the problem of blind separation of sources (for one dimensional signals and images) in the case that not only the waveform of sources is unknown, but also their number. For this purpose multi-layer neural networks with associated adaptive learning algorithms are developed. The primary source signals can have any non-Gaussian distribution, i.e. they can be sub-Gaussian and/or super-Gaussian. Computer experiments are presented which demonstrate the validity and high performance of the proposed approach. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1131607. 	[Title] A Context-Sensitive Generalization of ICA [Abstract] Source separation arises in a surprising number of signal processing applications, from speech recognition to EEG analysis. In the square linear blind source separation problem without time delays, one must find an unmixing matrix which can detangle the result of mixing n unknown independent sources through an unknown n fi n mixing matrix. The recently introduced ICA blind source separation algorithm (Baram and Roth 1994; Bell and Sejnowski 1995) is a powerful and surprisingly simple technique for solving this problem. ICA is all the more remarkable for performing so well despite making absolutely no use of the temporal structure of its input! This paper presents a new algorithm, contextual ICA, which derives from a maximum likelihood density estimation formulation of the problem. cICA can incorporate arbitrarily complex adaptive history-sensitive source models, and thereby make use of the temporal structure of its input. This allows it to separate in a number of situations where standard ICA cannot, including sources with low kurtosis, colored gaussian sources, and sources which have gaussian histograms. Since ICA is a special case of cICA, the MLE derivation provides as a corollary a rigorous derivation of classic ICA. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #230879. 	[Title] Equivariant adaptive source separation [Abstract] Source separation consists in recovering a set of independent signals when only mixtures with unknown coefficients are observed. This paper introduces a class of adaptive algorithms for source separation which implements an adaptive version of equivariant estimation and is henceforth called EASI (Equivariant Adaptive Separation via Independence). The EASI algorithms are based on the idea of serial updating: this specific form of matrix updates systematically yields algorithms with a simple, parallelizable structure, for both real and complex mixtures. Most importantly, the performance of an EASI algorithm does not depend on the mixing matrix. In particular, convergence rates, stability conditions and interference rejection levels depend only on the (normalized) distributions of the source signals. Close form expressions of these quantities are given via an asymptotic performance analysis. This is completed by some numerical experiments illustrating the effectiveness of the proposed approach. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2658. 	[Title] A New Learning Algorithm for Blind Signal Separation [Abstract] A new on-line learning algorithm which minimizes a statistical dependency among outputs is derived for blind separation of mixed signals. The dependency is measured by the average mutual information (MI) of the outputs. The source signals and the mixing matrix are unknown except for the number of the sources. The Gram-Charlier expansion instead of the Edgeworth expansion is used in evaluating the MI. The natural gradient approach is used to minimize the MI. A novel activation function is proposed for the on-line learning algorithm which has an equivariant property and is easily implemented on a neural network like model. The validity of the new learning algorithm is verified by computer simulations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #230884. 	[Title] BLIND SEPARATION OF DELAYED SOURCES BASED ON INFORMATION MAXIMIZATION [Abstract] Recently, Bell and Sejnowski have presented an approach to blind source separation based on the information maximization principle. We extend this approach into more general cases where the sources may have been delayed with respect to each other. We present a network architecture capable of coping with such sources, and we derive the adaptation equations for the delays and the weights in the network by maximizing the information transferred through the network. Examples using wideband sources such as speech are presented to illustrate the algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2665. 	[Title] An information-maximisation approach to blind separation and blind deconvolution [Abstract] We derive a new self-organising learning algorithm which maximises the information transferred in a network of non-linear units. The algorithm does not assume any knowledge of the input distributions, and is defined here for the zero-noise limit. Under these conditions, information maximisation has extra properties not found in the linear case (Linsker 1989). The non-linearities in the transfer function are able to pick up higher-order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation. This enables the network to separate statistically independent components in the inputs: a higher-order generalisation of Principal Components Analysis. We apply the network to the source separation (or cocktail party) problem, successfully separating unknown mixtures of up to ten speakers. We also show that a variant on the network architecture is able to perform blind deconvolution (cancellation of unknown echoes and reverberation in a speech signal). Finally, we derive dependencies of information transfer on time delays. We suggest that information max-imisation provides a unifying framework for problems in `blind' signal processing. fl Please send comments to tony@salk.edu. This paper will appear as Neural Computation, 7, 6, 1004-1034 (1995). The reference for this version is: Technical Report no. INC-9501, February 1995, Institute for Neural Computation, UCSD, San Diego, CA 92093-0523. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1115375. 	[Title] Analyzing Hyperspectral Data with Independent Component Analysis [Abstract] Hyperspectral image sensors provide images with a large number of contiguous spectral channels per pixel and enable information about different materials within a pixel to be obtained. The problem of spectrally unmixing materials may be viewed as a specific case of the blind source separation problem where data consists of mixed signals (in this case minerals) and the goal is to determine the contribution of each mineral to the mix without prior knowledge of the minerals in the mix. The technique of Independent Component Analysis (ICA) assumes that the spectral components are close to statistically independent and provides an unsupervised method for blind source separation. We introduce contextual ICA in the context of hyperspectral data analysis and apply the method to mineral data from synthetically mixed minerals and real image signatures. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1133428. 	[Title] Maximum likelihood source separation for discrete sources [Abstract] This communication deals with the source separation problem which consists in the separation of a noisy mixture of independent sources without a priori knowledge of the mixture coefficients. In this paper, we consider the maximum likelihood (ML) approach for discrete source signals with known probability distributions. An important feature of the ML approach in Gaussian noise is that the covariance matrix of the additive noise can be treated as a parameter. Hence, it is not necessary to know or to model the spatial structure of the noise. Another striking feature offered in the case of discrete sources is that, under mild assumptions, it is possible to separate more sources than sensors. In this paper, we consider maximization of the likelihood via the Expectation-Maximization (EM) algorithm. Please summarise all the given information.

Paper #696342 has the title `A Blind Identification and Separation Technique via Multi-layer Neural Networks  
` and the abstract `This paper deals with the problem of blind identification and source separation which consists of estimation of the mixing matrix and/or the separation of a mixture of stochastically independent sources without a priori knowledge on the mixing matrix . The method we propose here estimates the mixture matrix by a recurrent Input-Output (IO) Identification using as inputs a nonlinear transformation of the estimated sources. Herein, the nonlinear transformation (distortion) consists in constraining the modulus of the inputs of the IO-Identification device to be a constant. In contrast to other existing approaches, the covariance of the additive noise do not need to be modeled and can be estimated as a regular parameter if needed. The proposed approach is implemented using multi-layer neural networks in order to improve performance of separation. New associated on-line un-supervised adaptive learning rules are also developed. The effectiveness of the proposed method is illustrated by some computer simulations. 
`.
Question: Which category should Paper #696342 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #696342 has the title `A Blind Identification and Separation Technique via Multi-layer Neural Networks  
` and the abstract `This paper deals with the problem of blind identification and source separation which consists of estimation of the mixing matrix and/or the separation of a mixture of stochastically independent sources without a priori knowledge on the mixing matrix . The method we propose here estimates the mixture matrix by a recurrent Input-Output (IO) Identification using as inputs a nonlinear transformation of the estimated sources. Herein, the nonlinear transformation (distortion) consists in constraining the modulus of the inputs of the IO-Identification device to be a constant. In contrast to other existing approaches, the covariance of the additive noise do not need to be modeled and can be estimated as a regular parameter if needed. The proposed approach is implemented using multi-layer neural networks in order to improve performance of separation. New associated on-line un-supervised adaptive learning rules are also developed. The effectiveness of the proposed method is illustrated by some computer simulations. 
`.
Question: Which category should Paper #696342 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Neural_Networks