Above are papers related to the following paper: [Paper_ID] Paper #1132948. 	[Title] Edges are the `Independent Components' of Natural Scenes. [Abstract] Field (1994) has suggested that neurons with line and edge selectivities found in primary visual cortex of cats and monkeys form a sparse, distributed representation of natural scenes, and Barlow (1989) has reasoned that such responses should emerge from an unsupervised learning algorithm that attempts to find a factorial code of independent visual features. We show here that non-linear `infomax', when applied to an ensemble of natural scenes, produces sets of visual filters that are localised and oriented. Some of these filters are Gabor-like and resemble those produced by the sparseness-maximisation network of Olshausen & Field (1996). In addition, the outputs of these filters are as independent as possible, since the info-max network is able to perform Independent Components Analysis (ICA). We compare the resulting ICA filters and their associated basis functions, with other decorrelating filters produced by Principal Components Analysis (PCA) and zero-phase whitening filters (ZCA). The ICA filters have more sparsely distributed (kurtotic) outputs on natural scenes. They also resemble the receptive fields of simple cells in visual cortex, which suggests that these neurons form an information-theoretic co-ordinate system for images. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #851968. 	[Title] Natural Gradient Descent for Training Multi-Layer Perceptrons [Abstract] The main difficulty in implementing the natural gradient learning rule is to compute the inverse of the Fisher information matrix when the input dimension is large. We have found a new scheme to represent the Fisher information matrix. Based on this scheme, we have designed an algorithm to compute the inverse of the Fisher information matrix. When the input dimension n is much larger than the number of hidden neurons, the complexity of this algorithm is of order O(n 2 ) while the complexity of conventional algorithms for the same purpose is of order O(n 3 ). The simulation has confirmed the efficience and robustness of the natural gradient learning rule. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #763009. 	[Title] Generalization and Exclusive Allocation of Credit in Unsupervised Category Learning [Abstract] Acknowledgements: This research was supported in part by the Office of Naval Research (Cognitive and Neural Sciences, N00014-93-1-0208) and by the Whitaker Foundation (Special Opportunity Grant). We thank George Kalarickal, Charles Schmitt, William Ross, and Douglas Kelly for valuable discussions. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #578306. 	[Title] A FAMILY OF FIXED-POINT ALGORITHMS FOR INDEPENDENT COMPONENT ANALYSIS [Abstract] Independent Component Analysis (ICA) is a statistical signal processing technique whose main applications are blind source separation, blind deconvolution, and feature extraction. Estimation of ICA is usually performed by optimizing a 'contrast' function based on higher-order cumulants. In this paper, it is shown how almost any error function can be used to construct a contrast function to perform the ICA estimation. In particular, this means that one can use contrast functions that are robust against outliers. As a practical method for tnding the relevant extrema of such contrast functions, a txed-point iteration scheme is then introduced. The resulting algorithms are quite simple and converge fast and reliably. These algorithms also enable estimation of the independent components one-by-one, using a simple deation scheme. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #578309. 	[Title] Simple Neuron Models for Independent Component Analysis [Abstract] Recently, several neural algorithms have been introduced for Independent Component Analysis. Here we approach the problem from the point of view of a single neuron. First, simple Hebbian-like learning rules are introduced for estimating one of the independent components from sphered data. Some of the learning rules can be used to estimate an independent component which has a negative kurtosis, and the others estimate a component of positive kurtosis. Next, a two-unit system is introduced to estimate an independent component of any kurtosis. The results are then generalized to estimate independent components from non-sphered (raw) mixtures. To separate several independent components, a system of several neurons with linear negative feedback is used. The convergence of the learning rules is rigorously proven without any unnecessary hypotheses on the distributions of the independent components. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #8581. 	[Title] LEARNING FACTORIAL CODES BY PREDICTABILITY MINIMIZATION (Neural Computation, 4(6):863-879, 1992) [Abstract] I propose a novel general principle for unsupervised learning of distributed non-redundant internal representations of input patterns. The principle is based on two opposing forces. For each representational unit there is an adaptive predictor which tries to predict the unit from the remaining units. In turn, each unit tries to react to the environment such that it minimizes its predictability. This encourages each unit to filter `abstract concepts' out of the environmental input such that these concepts are statistically independent of those upon which the other units focus. I discuss various simple yet potentially powerful implementations of the principle which aim at finding binary factorial codes (Bar-low et al., 1989), i.e. codes where the probability of the occurrence of a particular input is simply the product of the probabilities of the corresponding code symbols. Such codes are potentially relevant for (1) segmentation tasks, (2) speeding up supervised learning, (3) novelty detection. Methods for finding factorial codes automatically implement Occam's razor for finding codes using a minimal number of units. Unlike previous methods the novel principle has a potential for removing not only linear but also non-linear output redundancy. Illustrative experiments show that algorithms based on the principle of predictability minimization are practically feasible. The final part of this paper describes an entirely local algorithm that has a potential for learning unique representations of extended input sequences. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1140230. 	[Title] NIPS*97 Multiplicative Updating Rule for Blind Separation Derived from the Method of Scoring [Abstract] For blind source separation, when the Fisher information matrix is used as the Riemannian metric tensor for the parameter space, the steepest descent algorithm to maximize the likelihood function in this Riemannian parameter space becomes the serial updating rule with equivariant property. This algorithm can be further simplified by using the asymptotic form of the Fisher information matrix around the equilibrium. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #205192. 	[Title] Principal and Independent Components in Neural Networks Recent Developments [Abstract] Nonlinear extensions of one-unit and multi-unit Principal Component Analysis (PCA) neural networks, introduced earlier by the authors, are reviewed. The networks and their nonlinear Hebbian learning rules are related to other signal expansions like Projection Pursuit (PP) and Independent Component Analysis (ICA). Separation results for mixtures of real world signals and im ages are given. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #631052. 	[Title] Constrained Optimization for Neural Map Formation: A Unifying Framework for Weight Growth and Normalization [Abstract] Computational models of neural map formation can be considered on at least three different levels of abstraction: detailed models including neural activity dynamics, weight dynamics that abstract from the neural activity dynamics by an adiabatic approximation, and constrained optimization from which equations governing weight dynamics can be derived. Constrained optimization uses an objective function, from which a weight growth rule can be derived as a gradient flow, and some constraints, from which normalization rules are derived. In this paper we present an example of how an optimization problem can be derived from detailed non-linear neural dynamics. A systematic investigation reveals how different weight dynamics introduced previously can be derived from two types of objective function terms and two types of constraints. This includes dynamic link matching as a special case of neural map formation. We focus in particular on the role of coordinate transformations to derive different weight dynamics from the same optimization problem. Several examples illustrate how the constrained optimization framework can help in understanding, generating, and comparing different models of neural map formation. The techniques used in this analysis may also be useful in investigating other types of neural dynamics. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1122704. 	[Title] Independent Component Analysis of Electroencephalographic Data [Abstract] Because of the distance between the skull and brain and their different resistivities, electroencephalographic (EEG) data collected from any point on the human scalp includes activity generated within a large brain area. This spatial smearing of EEG data by volume conduction does not involve significant time delays, however, suggesting that the Independent Component Analysis (ICA) algorithm of Bell and Sejnowski [1] is suitable for performing blind source separation on EEG data. The ICA algorithm separates the problem of source identification from that of source localization. First results of applying the ICA algorithm to EEG and event-related potential (ERP) data collected during a sustained auditory detection task show: (1) ICA training is insensitive to different random seeds. (2) ICA may be used to segregate obvious artifactual EEG components (line and muscle noise, eye movements) from other sources. (3) ICA is capable of isolating overlapping EEG phenomena, including alpha and theta bursts and spatially-separable ERP components, to separate ICA channels. (4) Nonstationarities in EEG and behavioral state can be tracked using ICA via changes in the amount of residual correlation between ICA-filtered output channels. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #395540. 	[Title] Maximum Likelihood and Covariant Algorithms for Independent Component Analysis somewhat more biologically plausible, involving no [Abstract] Bell and Sejnowski (1995) have derived a blind signal processing algorithm for a non-linear feedforward network from an information maximization viewpoint. This paper first shows that the same algorithm can be viewed as a maximum likelihood algorithm for the optimization of a linear generative model. Third, this paper gives a partial proof of the `folk-theorem' that any mixture of sources with high-kurtosis histograms is separable by the classic ICA algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #696342. 	[Title] A Blind Identification and Separation Technique via Multi-layer Neural Networks [Abstract] This paper deals with the problem of blind identification and source separation which consists of estimation of the mixing matrix and/or the separation of a mixture of stochastically independent sources without a priori knowledge on the mixing matrix . The method we propose here estimates the mixture matrix by a recurrent Input-Output (IO) Identification using as inputs a nonlinear transformation of the estimated sources. Herein, the nonlinear transformation (distortion) consists in constraining the modulus of the inputs of the IO-Identification device to be a constant. In contrast to other existing approaches, the covariance of the additive noise do not need to be modeled and can be estimated as a regular parameter if needed. The proposed approach is implemented using multi-layer neural networks in order to improve performance of separation. New associated on-line un-supervised adaptive learning rules are also developed. The effectiveness of the proposed method is illustrated by some computer simulations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #696343. 	[Title] On the performance of orthogonal source separation algorithms [Abstract] Source separation consists in recovering a set of n independent signals from m n observed instantaneous mixtures of these signals, possibly corrupted by additive noise. Many source separation algorithms use second order information in a whitening operation which reduces the non trivial part of the separation to determining a unitary matrix. Most of them further show a kind of invariance property which can be exploited to predict some general results about their performance. Our first contribution is to exhibit a lower bound to the performance in terms of accuracy of the separation. This bound is independent of the algorithm and, in the i.i.d. case, of the distribution of the source signals. Second, we show that the performance of invariant algorithms depends on the mixing matrix and on the noise level in a specific way. A consequence is that at low noise levels, the performance does not depend on the mixture but only on the distribution of the sources, via a function which is characteristic of the given source separation algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #696345. 	[Title] SELF-ADAPTIVE NEURAL NETWORKS FOR BLIND SEPARATION OF SOURCES [Abstract] Novel on-line learning algorithms with self adaptive learning rates (parameters) for blind separation of signals are proposed. The main motivation for development of new learning rules is to improve convergence speed and to reduce cross-talking, especially for non-stationary signals. Furthermore, we have discovered that under some conditions the proposed neural network models with associated learning algorithms exhibit a random switch of attention, i.e. they have ability of chaotic or random switching or cross-over of output signals in such way that a specified separated signal may appear at various outputs at different time windows. Validity, performance and dynamic properties of the proposed learning algorithms are investigated by computer simulation experiments. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #696346. 	[Title] LOCAL ADAPTIVE LEARNING ALGORITHMS FOR BLIND SEPARATION OF NATURAL IMAGES [Abstract] In this paper a neural network approach for reconstruction of natural highly correlated images from linear (additive) mixture of them is proposed. A multi-layer architecture with local on-line learning rules is developed to solve the problem of blind separation of sources. The main motivation for using a multi-layer network instead of a single-layer one is to improve the performance and robustness of separation, while applying a very simple local learning rule, which is biologically plausible. Moreover such architecture with on-chip learning is relatively easy implementable using VLSI electronic circuits. Furthermore it enables the extraction of source signals sequentially one after the other, starting from the strongest signal and finishing with the weakest one. The experimental part focuses on separating highly correlated human faces from mixture of them, with additive noise and under unknown number of sources. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1135899. 	[Title] BLIND SEPARATION OF REAL WORLD AUDIO SIGNALS USING OVERDETERMINED MIXTURES [Abstract] We discuss the advantages of using overdetermined mixtures to improve upon blind source separation algorithms that are designed to extract sound sources from acoustic mixtures. A study of the nature of room impulse responses helps us choose an adaptive filter architecture. We use ideal inverses of acquired room impulse responses to compare the effectiveness of different-sized separating filter configurations of various filter lengths. Using a multi-channel blind least-mean-square algorithm (MBLMS), we show that, by adding additional sensors, we can improve upon the separation of signals mixed with real world filters. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #630817. 	[Title] Inferring sparse, overcomplete image codes using an efficient coding framework [Abstract] We apply a general technique for learning overcomplete bases to the problem of finding efficient image codes. The bases learned by the algorithm are localized, oriented, and bandpass, consistent with earlier results obtained using related methods. We show that the learned bases are Gabor-like in structure and that higher degrees of overcompleteness produce greater sampling density in position, orientation, and scale. The efficient coding framework provides a method for comparing different bases objectively by calculating their probability given the observed data or by measuring the entropy of the basis function coefficients. Compared to complete and overcomplete Fourier and wavelet bases, the learned bases have much better coding efficiency. We demonstrate the improvement in the representation of the learned bases by showing superior performance in image denoising and filling-in of missing pixels. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #578337. 	[Title] Signal Separation by Nonlinear Hebbian Learning [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1105574. 	[Title] Learning Viewpoint Invariant Representations of Faces in an Attractor Network [Abstract] In natural visual experience, different views of an object tend to appear in close temporal proximity as an animal manipulates the object or navigates around it. We investigated the ability of an attractor network to acquire view invariant visual representations by associating first neighbors in a pattern sequence. The pattern sequence contains successive views of faces of ten individuals as they change pose. Under the network dynamics developed by Griniasty, Tsodyks & Amit (1993), multiple views of a given subject fall into the same basin of attraction. We use an independent component (ICA) representation of the faces for the input patterns (Bell & Sejnowski, 1995). The ICA representation has advantages over the principal component representation (PCA) for viewpoint-invariant recognition both with and without the attractor network, suggesting that ICA is a better representation than PCA for object recognition. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1132968. 	[Title] Local Feature Analysis: A general statistical theory for object representation [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #578347. 	[Title] Independent Component Analysis by General Non-linear Hebbian-like Learning Rules [Abstract] A number of neural learning rules have been recently proposed for Independent Component Analysis (ICA). The rules are usually derived from information-theoretic criteria such as maximum entropy or minimum mutual information. In this paper, we show that in fact, ICA can be performed by very simple Hebbian or anti-Hebbian learning rules, which may have only weak relations to such information-theoretical quantities. Rather suprisingly, practically any non-linear function can be used in the learning rule, provided only that the sign of the Hebbian/anti-Hebbian term is chosen correctly. In addition to the Hebbian-like mechanism, the weight vector is here constrained to have unit norm, and the data is preprocessed by prewhitening, or sphering. These results imply that one can choose the non-linearity so as to optimize desired statistical or numerical criteria. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #751408. 	[Title] Blind separation of delayed and convolved sources. [Abstract] We address the difficult problem of separating multiple speakers with multiple microphones in a real room. We combine the work of Torkkola and Amari, Cichocki and Yang, to give Natural Gradient information maximisation rules for recurrent (IIR) networks, blindly adjusting delays, separating and deconvolving mixed signals. While they work well on simulated data, these rules fail in real rooms which usually involve non-minimum phase transfer functions, not-invertible using stable IIR filters. An approach that sidesteps this problem is to perform infomax on a feedforward architecture in the frequency domain (Lambert 1996). We demonstrate real-room separation of two natural signals using this approach. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #237489. 	[Title] A Non-linear Information Maximisation Algorithm that Performs Blind Separation. [Abstract] A new learning algorithm is derived which performs online stochastic gradient ascent in the mutual information between outputs and inputs of a network. In the absence of a priori knowledge about the `signal' and `noise' components of the input, propagation of information depends on calibrating network non-linearities to the detailed higher-order moments of the input density functions. By incidentally minimising mutual information between outputs, as well as maximising their individual entropies, the network `fac-torises' the input into independent components. As an example application, we have achieved near-perfect separation of ten digitally mixed speech signals. Our simulations lead us to believe that our network performs better at blind separation than the Herault-Jutten network, reflecting the fact that it is derived rigorously from the mutual information objective. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1130676. 	[Title] Learning overcomplete representations [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1130808. 	[Title] Independent Component Analysis of Simulated EEG Using a Three-Shell Spherical Head Model 1 [Abstract] 1 This report was supported in part by the Navy Medical Research and Development Command and the Office of Naval Research, Department of the Navy under work unit ONR.Reimb-6429. The views expressed in this article are those of the authors and do not reflect the official policy or position of the Department of the Navy, Department of Defense, or the U.S. Government. Approved for public release, distribution unlimited. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1132486. 	[Title] Working Paper IS-97-22 (Information Systems) A First Application of Independent Component Analysis to Extracting Structure [Abstract] This paper discusses the application of a modern signal processing technique known as independent component analysis (ICA) or blind source separation to multivariate financial time series such as a portfolio of stocks. The key idea of ICA is to linearly map the observed multivariate time series into a new space of statistically independent components (ICs). This can be viewed as a factorization of the portfolio since joint probabilities become simple products in the coordinate system of the ICs. We apply ICA to three years of daily returns of the 28 largest Japanese stocks and compare the results with those obtained using principal component analysis. The results indicate that the estimated ICs fall into two categories, (i) infrequent but large shocks (responsible for the major changes in the stock prices), and (ii) frequent smaller fluctuations (contributing little to the overall level of the stocks). We show that the overall stock price can be reconstructed surprisingly well by using a small number of thresholded weighted ICs. In contrast, when using shocks derived from principal components instead of independent components, the reconstructed price is less similar to the original one. Independent component analysis is a potentially powerful method of analyzing and understanding driving mechanisms in financial markets. There are further promising applications to risk management since ICA focuses on higher order statistics. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1113035. 	[Title] Empirical Entropy Manipulation for Real-World Problems [Abstract] No finite sample is sufficient to determine the density, and therefore the entropy, of a signal directly. Some assumption about either the functional form of the density or about its smoothness is necessary. Both amount to a prior over the space of possible density functions. By far the most common approach is to assume that the density has a parametric form. By contrast we derive a differential learning rule called EMMA that optimizes entropy by way of kernel density estimation. Entropy and its derivative can then be calculated by sampling from this density estimate. The resulting parameter update rule is surprisingly simple and efficient. We will show how EMMA can be used to detect and correct corruption in magnetic resonance images (MRI). This application is beyond the scope of existing parametric entropy models. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #282700. 	[Title] LEARNING LINEAR, SPARSE, FACTORIAL CODES [Abstract] In previous work (Olshausen & Field 1996), an algorithm was described for learning linear sparse codes which, when trained on natural images, produces a set of basis functions that are spatially localized, oriented, and bandpass (i.e., wavelet-like). This note shows how the algorithm may be interpreted within a maximum-likelihood framework. Several useful insights emerge from this connection: it makes explicit the relation to statistical independence (i.e., factorial coding), it shows a formal relationship to the algorithm of Bell and Sejnowski (1995), and it suggests how to adapt parameters that were previously fixed. This report describes research done within the Center for Biological and Computational Learning in the Department of Brain and Cognitive Sciences at the Massachusetts Institute of Technology. This research is sponsored by an Individual National Research Service Award to B.A.O. (NIMH F32-MH11062) and by a grant from the National Science Foundation under contract ASC-9217041 (this award includes funds from ARPA provided under the HPCC program) to CBCL. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #99023. 	[Title] A Fast Fixed-Point Algorithm for Independent Component Analysis [Abstract] This paper will appear in Neural Computation, 9:1483-1492, 1997. Abstract We introduce a novel fast algorithm for Independent Component Analysis, which can be used for blind source separation and feature extraction. It is shown how a neural network learning rule can be transformed into a txed-point iteration, which provides an algorithm that is very simple, does not depend on any user-detned parameters, and is fast to converge to the most accurate solution allowed by the data. The algorithm tnds, one at a time, all non-Gaussian independent components, regardless of their probability distributions. The computations can be performed either in batch mode or in a semi-adaptive manner. The convergence of the algorithm is rigorously proven, and the convergence speed is shown to be cubic. Some comparisons to gradient based algorithms are made, showing that the new algorithm is usually 10 to 100 times faster, sometimes giving the solution in just a few iterations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #237521. 	[Title] Competitive Anti-Hebbian Learning of Invariants [Abstract] Although the detection of invariant structure in a given set of input patterns is vital to many recognition tasks, connectionist learning rules tend to focus on directions of high variance (principal components). The prediction paradigm is often used to reconcile this dichotomy; here we suggest a more direct approach to invariant learning based on an anti-Hebbian learning rule. An unsupervised two-layer network implementing this method in a competitive setting learns to extract coherent depth information from random-dot stereograms. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #99030. 	[Title] The Nonlinear PCA Learning Rule and Signal Separation Mathematical Analysis [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #236759. 	[Title] NEURAL NETWORK APPROACH TO BLIND SEPARATION AND ENHANCEMENT OF IMAGES [Abstract] In this contribution we propose a new solution for the problem of blind separation of sources (for one dimensional signals and images) in the case that not only the waveform of sources is unknown, but also their number. For this purpose multi-layer neural networks with associated adaptive learning algorithms are developed. The primary source signals can have any non-Gaussian distribution, i.e. they can be sub-Gaussian and/or super-Gaussian. Computer experiments are presented which demonstrate the validity and high performance of the proposed approach. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1131607. 	[Title] A Context-Sensitive Generalization of ICA [Abstract] Source separation arises in a surprising number of signal processing applications, from speech recognition to EEG analysis. In the square linear blind source separation problem without time delays, one must find an unmixing matrix which can detangle the result of mixing n unknown independent sources through an unknown n fi n mixing matrix. The recently introduced ICA blind source separation algorithm (Baram and Roth 1994; Bell and Sejnowski 1995) is a powerful and surprisingly simple technique for solving this problem. ICA is all the more remarkable for performing so well despite making absolutely no use of the temporal structure of its input! This paper presents a new algorithm, contextual ICA, which derives from a maximum likelihood density estimation formulation of the problem. cICA can incorporate arbitrarily complex adaptive history-sensitive source models, and thereby make use of the temporal structure of its input. This allows it to separate in a number of situations where standard ICA cannot, including sources with low kurtosis, colored gaussian sources, and sources which have gaussian histograms. Since ICA is a special case of cICA, the MLE derivation provides as a corollary a rigorous derivation of classic ICA. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #230879. 	[Title] Equivariant adaptive source separation [Abstract] Source separation consists in recovering a set of independent signals when only mixtures with unknown coefficients are observed. This paper introduces a class of adaptive algorithms for source separation which implements an adaptive version of equivariant estimation and is henceforth called EASI (Equivariant Adaptive Separation via Independence). The EASI algorithms are based on the idea of serial updating: this specific form of matrix updates systematically yields algorithms with a simple, parallelizable structure, for both real and complex mixtures. Most importantly, the performance of an EASI algorithm does not depend on the mixing matrix. In particular, convergence rates, stability conditions and interference rejection levels depend only on the (normalized) distributions of the source signals. Close form expressions of these quantities are given via an asymptotic performance analysis. This is completed by some numerical experiments illustrating the effectiveness of the proposed approach. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2658. 	[Title] A New Learning Algorithm for Blind Signal Separation [Abstract] A new on-line learning algorithm which minimizes a statistical dependency among outputs is derived for blind separation of mixed signals. The dependency is measured by the average mutual information (MI) of the outputs. The source signals and the mixing matrix are unknown except for the number of the sources. The Gram-Charlier expansion instead of the Edgeworth expansion is used in evaluating the MI. The natural gradient approach is used to minimize the MI. A novel activation function is proposed for the on-line learning algorithm which has an equivariant property and is easily implemented on a neural network like model. The validity of the new learning algorithm is verified by computer simulations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #230884. 	[Title] BLIND SEPARATION OF DELAYED SOURCES BASED ON INFORMATION MAXIMIZATION [Abstract] Recently, Bell and Sejnowski have presented an approach to blind source separation based on the information maximization principle. We extend this approach into more general cases where the sources may have been delayed with respect to each other. We present a network architecture capable of coping with such sources, and we derive the adaptation equations for the delays and the weights in the network by maximizing the information transferred through the network. Examples using wideband sources such as speech are presented to illustrate the algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2665. 	[Title] An information-maximisation approach to blind separation and blind deconvolution [Abstract] We derive a new self-organising learning algorithm which maximises the information transferred in a network of non-linear units. The algorithm does not assume any knowledge of the input distributions, and is defined here for the zero-noise limit. Under these conditions, information maximisation has extra properties not found in the linear case (Linsker 1989). The non-linearities in the transfer function are able to pick up higher-order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation. This enables the network to separate statistically independent components in the inputs: a higher-order generalisation of Principal Components Analysis. We apply the network to the source separation (or cocktail party) problem, successfully separating unknown mixtures of up to ten speakers. We also show that a variant on the network architecture is able to perform blind deconvolution (cancellation of unknown echoes and reverberation in a speech signal). Finally, we derive dependencies of information transfer on time delays. We suggest that information max-imisation provides a unifying framework for problems in `blind' signal processing. fl Please send comments to tony@salk.edu. This paper will appear as Neural Computation, 7, 6, 1004-1034 (1995). The reference for this version is: Technical Report no. INC-9501, February 1995, Institute for Neural Computation, UCSD, San Diego, CA 92093-0523. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1115375. 	[Title] Analyzing Hyperspectral Data with Independent Component Analysis [Abstract] Hyperspectral image sensors provide images with a large number of contiguous spectral channels per pixel and enable information about different materials within a pixel to be obtained. The problem of spectrally unmixing materials may be viewed as a specific case of the blind source separation problem where data consists of mixed signals (in this case minerals) and the goal is to determine the contribution of each mineral to the mix without prior knowledge of the minerals in the mix. The technique of Independent Component Analysis (ICA) assumes that the spectral components are close to statistically independent and provides an unsupervised method for blind source separation. We introduce contextual ICA in the context of hyperspectral data analysis and apply the method to mineral data from synthetically mixed minerals and real image signatures. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1133428. 	[Title] Maximum likelihood source separation for discrete sources [Abstract] This communication deals with the source separation problem which consists in the separation of a noisy mixture of independent sources without a priori knowledge of the mixture coefficients. In this paper, we consider the maximum likelihood (ML) approach for discrete source signals with known probability distributions. An important feature of the ML approach in Gaussian noise is that the covariance matrix of the additive noise can be treated as a parameter. Hence, it is not necessary to know or to model the spatial structure of the noise. Another striking feature offered in the case of discrete sources is that, under mild assumptions, it is possible to separate more sources than sensors. In this paper, we consider maximization of the likelihood via the Expectation-Maximization (EM) algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #43639. 	[Title] Natural image statistics and efficient coding [Abstract] Natural images contain characteristic statistical regularities that set them apart from purely random images. Understanding what these regularities are can enable natural images to be coded more efficiently. In this paper, we describe some of the forms of structure that are contained in natural images, and we show how these are related to the response properties of neurons at early stages of the visual system. Many of the important forms of structure require higher-order (i.e., more than linear, pairwise) statistics to characterize, which makes models based on linear Hebbian learning, or principal components analysis, inappropriate for finding efficient codes for natural images. We suggest that a good objective for an efficient coding of natural scenes is to maximize the sparseness of the representation, and we show that a network that learns sparse codes of natural scenes succeeds in developing localized, oriented, bandpass receptive fields similar to those in the primate striate cortex. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #582139. 	[Title] Viewpoint invariant face recognition using independent component analysis and attractor networks [Abstract] We have explored two approaches to recognizing faces across changes in pose. First, we developed a representation of face images based on independent component analysis (ICA) and compared it to a principal component analysis (PCA) representation for face recognition. The ICA basis vectors for this data set were more spatially local than the PCA basis vectors and the ICA representation had greater invariance to changes in pose. Second, we present a model for the development of viewpoint invariant responses to faces from visual experience in a biological system. The temporal continuity of natural visual experience was incorporated into an attractor network model by Hebbian learning following a lowpass temporal filter on unit activities. When combined with the temporal filter, a basic Hebbian update rule became a generalization of Griniasty et al. (1993), which associates temporally proximal input patterns into basins of attraction. The system acquired rep resentations of faces that were largely independent of pose. Please summarise all the given information.

Paper #1132948 has the title `Edges are the `Independent Components' of Natural Scenes.  
` and the abstract `Field (1994) has suggested that neurons with line and edge selectivities found in primary visual cortex of cats and monkeys form a sparse, distributed representation of natural scenes, and Barlow (1989) has reasoned that such responses should emerge from an unsupervised learning algorithm that attempts to find a factorial code of independent visual features. We show here that non-linear `infomax', when applied to an ensemble of natural scenes, produces sets of visual filters that are localised and oriented. Some of these filters are Gabor-like and resemble those produced by the sparseness-maximisation network of Olshausen & Field (1996). In addition, the outputs of these filters are as independent as possible, since the info-max network is able to perform Independent Components Analysis (ICA). We compare the resulting ICA filters and their associated basis functions, with other decorrelating filters produced by Principal Components Analysis (PCA) and zero-phase whitening filters (ZCA). The ICA filters have more sparsely distributed (kurtotic) outputs on natural scenes. They also resemble the receptive fields of simple cells in visual cortex, which suggests that these neurons form an information-theoretic co-ordinate system for images.
`.
Question: Which category should Paper #1132948 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #1132948 has the title `Edges are the `Independent Components' of Natural Scenes.  
` and the abstract `Field (1994) has suggested that neurons with line and edge selectivities found in primary visual cortex of cats and monkeys form a sparse, distributed representation of natural scenes, and Barlow (1989) has reasoned that such responses should emerge from an unsupervised learning algorithm that attempts to find a factorial code of independent visual features. We show here that non-linear `infomax', when applied to an ensemble of natural scenes, produces sets of visual filters that are localised and oriented. Some of these filters are Gabor-like and resemble those produced by the sparseness-maximisation network of Olshausen & Field (1996). In addition, the outputs of these filters are as independent as possible, since the info-max network is able to perform Independent Components Analysis (ICA). We compare the resulting ICA filters and their associated basis functions, with other decorrelating filters produced by Principal Components Analysis (PCA) and zero-phase whitening filters (ZCA). The ICA filters have more sparsely distributed (kurtotic) outputs on natural scenes. They also resemble the receptive fields of simple cells in visual cortex, which suggests that these neurons form an information-theoretic co-ordinate system for images.
`.
Question: Which category should Paper #1132948 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Neural_Networks