Above are papers related to the following paper: [Paper_ID] Paper #75695. 	[Title] Unsupervised Discrimination of Clustered Data via Optimization of Binary Information Gain [Abstract] We present the information-theoretic derivation of a learning algorithm that clusters unlabelled data with linear discriminants. In contrast to methods that try to preserve information about the input patterns, we maximize the information gained from observing the output of robust binary discriminators implemented with sigmoid nodes. We derive a local weight adaptation rule via gradient ascent in this objective, demonstrate its dynamics on some simple data sets, relate our approach to previous work and suggest directions in which it may be extended. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1118083. 	[Title] An integrated approach to the study of object features in visual recognition [Abstract] We propose to assess the relevance of theories of synaptic modification as models of feature extraction in human vision, by using masks derived from synaptic weight patterns to occlude parts of the stimulus images in psychophysical experiments. In the experiment reported here, we found that a mask derived from principal component analysis of object images was more effective in reducing the generalization performance of human subjects than a mask derived from another method of feature extraction (BCM), based on higher-order statistics of the images. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #8581. 	[Title] LEARNING FACTORIAL CODES BY PREDICTABILITY MINIMIZATION (Neural Computation, 4(6):863-879, 1992) [Abstract] I propose a novel general principle for unsupervised learning of distributed non-redundant internal representations of input patterns. The principle is based on two opposing forces. For each representational unit there is an adaptive predictor which tries to predict the unit from the remaining units. In turn, each unit tries to react to the environment such that it minimizes its predictability. This encourages each unit to filter `abstract concepts' out of the environmental input such that these concepts are statistically independent of those upon which the other units focus. I discuss various simple yet potentially powerful implementations of the principle which aim at finding binary factorial codes (Bar-low et al., 1989), i.e. codes where the probability of the occurrence of a particular input is simply the product of the probabilities of the corresponding code symbols. Such codes are potentially relevant for (1) segmentation tasks, (2) speeding up supervised learning, (3) novelty detection. Methods for finding factorial codes automatically implement Occam's razor for finding codes using a minimal number of units. Unlike previous methods the novel principle has a potential for removing not only linear but also non-linear output redundancy. Illustrative experiments show that algorithms based on the principle of predictability minimization are practically feasible. The final part of this paper describes an entirely local algorithm that has a potential for learning unique representations of extended input sequences. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2695. 	[Title] Combining Exploratory Projection Pursuit And Projection Pursuit Regression With Application To Neural Networks [Abstract] We present a novel classification and regression method that combines exploratory projection pursuit (unsupervised training) with projection pursuit regression (supervised training), to yield a new family of cost/complexity penalty terms. Some improved generalization properties are demonstrated on real world problems. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2696. 	[Title] Objective Function Formulation of the BCM Theory of Visual Cortical Plasticity: Statistical Connections, Stability Conditions [Abstract] In this paper, we present an objective function formulation of the BCM theory of visual cortical plasticity that permits us to demonstrate the connection between the unsupervised BCM learning procedure and various statistical methods, in particular, that of Projection Pursuit. This formulation provides a general method for stability analysis of the fixed points of the theory and enables us to analyze the behavior and the evolution of the network under various visual rearing conditions. It also allows comparison with many existing unsupervised methods. This model has been shown successful in various applications such as phoneme and 3D object recognition. We thus have the striking and possibly highly significant result that a biological neuron is performing a sophisticated statistical procedure. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #342802. 	[Title] PHONETIC CLASSIFICATION OF TIMIT SEGMENTS PREPROCESSED WITH LYON'S COCHLEAR MODEL USING A SUPERVISED/UNSUPERVISED HYBRID NEURAL NETWORK [Abstract] We report results on vowel and stop consonant recognition with tokens extracted from the TIMIT database. Our current system differs from others doing similar tasks in that we do not use any specific time normalization techniques. We use a very detailed biologically motivated input representation of the speech tokens - Lyon's cochlear model as implemented by Slaney [20]. This detailed, high dimensional representation, known as a cochleagram, is classified by either a back-propagation or by a hybrid supervised/unsupervised neural network classifier. The hybrid network is composed of a biologically motivated unsupervised network and a supervised back-propagation network. This approach produces results comparable to those obtained by others without the addition of time normalization. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #139547. 	[Title] On Centering Neural Network Weight Updates [Abstract] Technical Report IDSIA-19-97 Abstract. It has long been known that neural networks can learn faster when their input and hidden unit activities are centered about zero; recently we have extended this approach to also encompass the centering of error signals (Schraudolph and Sejnowski, 1996). Here we generalize this notion to all factors involved in the weight update, leading us to propose centering the slope of hidden unit activation functions as well. Slope centering removes the linear component of backpropagated error; this improves credit assignment in networks with shortcut connections. Benchmark results show that this can speed up learning significantly without adversely affecting the trained network's generalization ability. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1123239. 	[Title] 3D Object Recognition Using Unsupervised Feature Extraction [Abstract] Intrator (1990) proposed a feature extraction method that is related to recent statistical theory (Huber, 1985; Friedman, 1987), and is based on a biologically motivated model of neuronal plasticity (Bienenstock et al., 1982). This method has been recently applied to feature extraction in the context of recognizing 3D objects from single 2D views (Intrator and Gold, 1991). Here we describe experiments designed to analyze the nature of the extracted features, and their relevance to the theory and psychophysics of object recognition. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1132968. 	[Title] Local Feature Analysis: A general statistical theory for object representation [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #135464. 	[Title] Unsupervised Neural Network Learning Procedures For Feature Extraction and Classification [Abstract] Technical report CNS-TR-95-1 Center for Neural Systems McMaster University Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #75691. 	[Title] SEMILINEAR PREDICTABILITY MINIMIZATION PRODUCES WELL-KNOWN FEATURE DETECTORS Neural Computation, 1996 (accepted) [Abstract] Predictability minimization (PM | Schmidhuber, 1992) exhibits various intuitive and theoretical advantages over many other methods for unsupervised redundancy reduction. So far, however, there were only toy applications of PM. In this paper, we apply semilinear PM to static real world images and find: without a teacher and without any significant preprocessing, the system automatically learns to generate distributed representations based on well-known feature detectors, such as orientation sensitive edge detectors and off-center-on-surround-like structures, thus extracting simple features related to those considered useful for image pre-processing and compression. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #237489. 	[Title] A Non-linear Information Maximisation Algorithm that Performs Blind Separation. [Abstract] A new learning algorithm is derived which performs online stochastic gradient ascent in the mutual information between outputs and inputs of a network. In the absence of a priori knowledge about the `signal' and `noise' components of the input, propagation of information depends on calibrating network non-linearities to the detailed higher-order moments of the input density functions. By incidentally minimising mutual information between outputs, as well as maximising their individual entropies, the network `fac-torises' the input into independent components. As an example application, we have achieved near-perfect separation of ten digitally mixed speech signals. Our simulations lead us to believe that our network performs better at blind separation than the Herault-Jutten network, reflecting the fact that it is derived rigorously from the mutual information objective. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #626999. 	[Title] Tempering Backpropagation Networks: Not All Weights are Created Equal approach yields hitherto unparalleled performance on [Abstract] Backpropagation learning algorithms typically collapse the network's structure into a single vector of weight parameters to be optimized. We suggest that their performance may be improved by utilizing the structural information instead of discarding it, and introduce a framework for tempering each weight accordingly. In the tempering model, activation and error signals are treated as approximately independent random variables. The characteristic scale of weight changes is then matched to that of the residuals, allowing structural properties such as a node's fan-in and fan-out to affect the local learning rate and backpropagated error. The model also permits calculation of an upper bound on the global learning rate for batch updates, which in turn leads to different update rules for bias vs. non-bias weights. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1120059. 	[Title] Plasticity-Mediated Competitive Learning [Abstract] Differentiation between the nodes of a competitive learning network is conventionally achieved through competition on the basis of neural activity. Simple inhibitory mechanisms are limited to sparse representations, while decorrelation and factorization schemes that support distributed representations are computation-ally unattractive. By letting neural plasticity mediate the competitive interaction instead, we obtain diffuse, nonadaptive alternatives for fully distributed representations. We use this technique to simplify and improve our binary information gain optimization algorithm for feature extraction (Schraudolph and Sejnowski, 1993); the same approach could be used to improve other learning algorithms. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1120197. 	[Title] On the Combination of Supervised and Unsupervised Learning  reducing the overall error measure of a classifier. [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1113035. 	[Title] Empirical Entropy Manipulation for Real-World Problems [Abstract] No finite sample is sufficient to determine the density, and therefore the entropy, of a signal directly. Some assumption about either the functional form of the density or about its smoothness is necessary. Both amount to a prior over the space of possible density functions. By far the most common approach is to assume that the density has a parametric form. By contrast we derive a differential learning rule called EMMA that optimizes entropy by way of kernel density estimation. Entropy and its derivative can then be calculated by sampling from this density estimate. The resulting parameter update rule is surprisingly simple and efficient. We will show how EMMA can be used to detect and correct corruption in magnetic resonance images (MRI). This application is beyond the scope of existing parametric entropy models. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #69198. 	[Title] LEARNING COMPLEX, EXTENDED SEQUENCES USING THE PRINCIPLE OF HISTORY COMPRESSION (Neural Computation, 4(2):234-242, 1992) [Abstract] Previous neural network learning algorithms for sequence processing are computationally expensive and perform poorly when it comes to long time lags. This paper first introduces a simple principle for reducing the descriptions of event sequences without loss of information. A consequence of this principle is that only unexpected inputs can be relevant. This insight leads to the construction of neural architectures that learn to `divide and conquer' by recursively decomposing sequences. I describe two architectures. The first functions as a self-organizing multi-level hierarchy of recurrent networks. The second, involving only two recurrent networks, tries to collapse a multi-level predictor hierarchy into a single recurrent net. Experiments show that the system can require less computation per time step and many fewer training sequences than conventional training algorithms for recurrent nets. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #627024. 	[Title] Early Stopping but when? [Abstract] Validation can be used to detect when overfitting starts during supervised training of a neural network; training is then stopped before convergence to avoid the overfitting ("early stopping"). The exact criterion used for validation-based early stopping, however, is usually chosen in an ad-hoc fashion or training is stopped interactively. This trick describes how to select a stopping criterion in a systematic fashion; it is a trick for either speeding learning procedures or improving generalization, whichever is more important in the particular situation. An empirical investigation on multi-layer perceptrons shows that there exists a tradeoff between training time and generalization: From the given mix of 1296 training runs using different 12 problems and 24 different network architectures I conclude slower stopping criteria allow for small improvements in generalization (here: about 4% on average), but cost much more training time (here: about factor 4 longer on average). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1114192. 	[Title] Multimodality Exploration in Training an Unsupervised Projection Pursuit Neural Network [Abstract] Graphical inspection of multimodality is demonstrated using unsupervised lateral-inhibition neural networks. Three projection pursuit indices are compared on low dimensional simulated and real-world data: principal components [22], Legendre poly nomial [6] and projection pursuit network [16]. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1108050. 	[Title] Theory of Synaptic Plasticity in Visual Cortex [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #99025. 	[Title] Neuronal Goals: Efficient Coding and Coincidence Detection [Abstract] Barlow's seminal work on minimal entropy codes and unsupervised learning is reiterated. In particular, the need to transmit the probability of events is put in a practical neuronal framework for detecting suspicious events. A variant of the BCM learning rule [15] is presented together with some mathematical results suggesting optimal minimal entropy coding. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1120858. 	[Title] Three-Dimensional Object Recognition Using an Unsupervised BCM Network: The Usefulness of Distinguishing Features [Abstract] We propose an object recognition scheme based on a method for feature extraction from gray level images that corresponds to recent statistical theory, called projection pursuit, and is derived from a biologically motivated feature extracting neuron. To evaluate the performance of this method we use a set of very detailed psychophysical 3D object recognition experiments (Bulthoff and Edelman, 1992). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #911198. 	[Title] Centering Neural Network Gradient Factors [Abstract] Technical Report IDSIA-19-97 Abstract. It has long been known that neural networks can learn faster when their input and hidden unit activities are centered about zero; recently we have extended this approach to also encompass the centering of error signals [2]. Here we generalize this notion to all factors involved in the network's gradient, leading us to propose centering the slope of hidden unit activation functions as well. Slope centering removes the linear component of backpropagated error; this improves credit assignment in networks with shortcut connections. Benchmark results show that this can speed up learning significantly without adversely affecting the trained network's generalization ability. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #5348. 	[Title] Feature Extraction Using an Unsupervised Neural Network [Abstract] A novel unsupervised neural network for dimensionality reduction that seeks directions emphasizing multimodality is presented, and its connection to exploratory projection pursuit methods is discussed. This leads to a new statistical insight into the synaptic modification equations governing learning in Bienenstock, Cooper, and Munro (BCM) neurons (1982). The importance of a dimensionality reduction principle based solely on distinguishing features is demonstrated using a phoneme recognition experiment. The extracted features are compared with features extracted using a back-propagation network. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2665. 	[Title] An information-maximisation approach to blind separation and blind deconvolution [Abstract] We derive a new self-organising learning algorithm which maximises the information transferred in a network of non-linear units. The algorithm does not assume any knowledge of the input distributions, and is defined here for the zero-noise limit. Under these conditions, information maximisation has extra properties not found in the linear case (Linsker 1989). The non-linearities in the transfer function are able to pick up higher-order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation. This enables the network to separate statistically independent components in the inputs: a higher-order generalisation of Principal Components Analysis. We apply the network to the source separation (or cocktail party) problem, successfully separating unknown mixtures of up to ten speakers. We also show that a variant on the network architecture is able to perform blind deconvolution (cancellation of unknown echoes and reverberation in a speech signal). Finally, we derive dependencies of information transfer on time delays. We suggest that information max-imisation provides a unifying framework for problems in `blind' signal processing. fl Please send comments to tony@salk.edu. This paper will appear as Neural Computation, 7, 6, 1004-1034 (1995). The reference for this version is: Technical Report no. INC-9501, February 1995, Institute for Neural Computation, UCSD, San Diego, CA 92093-0523. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #256106. 	[Title] Classification of Underwater Mammals using Feature Extraction Based on Time-Frequency Analysis and BCM Theory [Abstract] Underwater mammal sound classification is demonstrated using a novel application of wavelet time/frequency decomposition and feature extraction using a BCM unsupervised network. Different feature extraction methods and different wavelet representations are studied. The system achieves outstanding classification performance even when tested with mammal sounds recorded at very different locations (from those used for training). The improved results suggest that nonlinear feature extraction from wavelet representations outperforms different linear choices of basis functions. Please summarise all the given information.

Paper #75695 has the title `Unsupervised Discrimination of Clustered Data via Optimization of Binary Information Gain  
` and the abstract `We present the information-theoretic derivation of a learning algorithm that clusters unlabelled data with linear discriminants. In contrast to methods that try to preserve information about the input patterns, we maximize the information gained from observing the output of robust binary discriminators implemented with sigmoid nodes. We derive a local weight adaptation rule via gradient ascent in this objective, demonstrate its dynamics on some simple data sets, relate our approach to previous work and suggest directions in which it may be extended.
`.
Question: Which category should Paper #75695 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #75695 has the title `Unsupervised Discrimination of Clustered Data via Optimization of Binary Information Gain  
` and the abstract `We present the information-theoretic derivation of a learning algorithm that clusters unlabelled data with linear discriminants. In contrast to methods that try to preserve information about the input patterns, we maximize the information gained from observing the output of robust binary discriminators implemented with sigmoid nodes. We derive a local weight adaptation rule via gradient ascent in this objective, demonstrate its dynamics on some simple data sets, relate our approach to previous work and suggest directions in which it may be extended.
`.
Question: Which category should Paper #75695 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Neural_Networks