Above are papers related to the following paper: [Paper_ID] Paper #911198. 	[Title] Centering Neural Network Gradient Factors [Abstract] Technical Report IDSIA-19-97 Abstract. It has long been known that neural networks can learn faster when their input and hidden unit activities are centered about zero; recently we have extended this approach to also encompass the centering of error signals [2]. Here we generalize this notion to all factors involved in the network's gradient, leading us to propose centering the slope of hidden unit activation functions as well. Slope centering removes the linear component of backpropagated error; this improves credit assignment in networks with shortcut connections. Benchmark results show that this can speed up learning significantly without adversely affecting the trained network's generalization ability. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #278403. 	[Title] Fast Pruning Using Principal Components [Abstract] We present a new algorithm for eliminating excess parameters and improving network generalization after supervised training. The method, "Principal Components Pruning (PCP)", is based on principal component analysis of the node activations of successive layers of the network. It is simple, cheap to implement, and effective. It requires no network retraining, and does not involve calculating the full Hessian of the cost function. Only the weight and the node activity correlation matrices for each layer of nodes are required. We demonstrate the efficacy of the method on a regression problem using polynomial basis functions, and on an economic time series prediction problem using a two-layer, feedforward network. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1118083. 	[Title] An integrated approach to the study of object features in visual recognition [Abstract] We propose to assess the relevance of theories of synaptic modification as models of feature extraction in human vision, by using masks derived from synaptic weight patterns to occlude parts of the stimulus images in psychophysical experiments. In the experiment reported here, we found that a mask derived from principal component analysis of object images was more effective in reducing the generalization performance of human subjects than a mask derived from another method of feature extraction (BCM), based on higher-order statistics of the images. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #8581. 	[Title] LEARNING FACTORIAL CODES BY PREDICTABILITY MINIMIZATION (Neural Computation, 4(6):863-879, 1992) [Abstract] I propose a novel general principle for unsupervised learning of distributed non-redundant internal representations of input patterns. The principle is based on two opposing forces. For each representational unit there is an adaptive predictor which tries to predict the unit from the remaining units. In turn, each unit tries to react to the environment such that it minimizes its predictability. This encourages each unit to filter `abstract concepts' out of the environmental input such that these concepts are statistically independent of those upon which the other units focus. I discuss various simple yet potentially powerful implementations of the principle which aim at finding binary factorial codes (Bar-low et al., 1989), i.e. codes where the probability of the occurrence of a particular input is simply the product of the probabilities of the corresponding code symbols. Such codes are potentially relevant for (1) segmentation tasks, (2) speeding up supervised learning, (3) novelty detection. Methods for finding factorial codes automatically implement Occam's razor for finding codes using a minimal number of units. Unlike previous methods the novel principle has a potential for removing not only linear but also non-linear output redundancy. Illustrative experiments show that algorithms based on the principle of predictability minimization are practically feasible. The final part of this paper describes an entirely local algorithm that has a potential for learning unique representations of extended input sequences. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2695. 	[Title] Combining Exploratory Projection Pursuit And Projection Pursuit Regression With Application To Neural Networks [Abstract] We present a novel classification and regression method that combines exploratory projection pursuit (unsupervised training) with projection pursuit regression (supervised training), to yield a new family of cost/complexity penalty terms. Some improved generalization properties are demonstrated on real world problems. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2696. 	[Title] Objective Function Formulation of the BCM Theory of Visual Cortical Plasticity: Statistical Connections, Stability Conditions [Abstract] In this paper, we present an objective function formulation of the BCM theory of visual cortical plasticity that permits us to demonstrate the connection between the unsupervised BCM learning procedure and various statistical methods, in particular, that of Projection Pursuit. This formulation provides a general method for stability analysis of the fixed points of the theory and enables us to analyze the behavior and the evolution of the network under various visual rearing conditions. It also allows comparison with many existing unsupervised methods. This model has been shown successful in various applications such as phoneme and 3D object recognition. We thus have the striking and possibly highly significant result that a biological neuron is performing a sophisticated statistical procedure. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #342802. 	[Title] PHONETIC CLASSIFICATION OF TIMIT SEGMENTS PREPROCESSED WITH LYON'S COCHLEAR MODEL USING A SUPERVISED/UNSUPERVISED HYBRID NEURAL NETWORK [Abstract] We report results on vowel and stop consonant recognition with tokens extracted from the TIMIT database. Our current system differs from others doing similar tasks in that we do not use any specific time normalization techniques. We use a very detailed biologically motivated input representation of the speech tokens - Lyon's cochlear model as implemented by Slaney [20]. This detailed, high dimensional representation, known as a cochleagram, is classified by either a back-propagation or by a hybrid supervised/unsupervised neural network classifier. The hybrid network is composed of a biologically motivated unsupervised network and a supervised back-propagation network. This approach produces results comparable to those obtained by others without the addition of time normalization. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #139547. 	[Title] On Centering Neural Network Weight Updates [Abstract] Technical Report IDSIA-19-97 Abstract. It has long been known that neural networks can learn faster when their input and hidden unit activities are centered about zero; recently we have extended this approach to also encompass the centering of error signals (Schraudolph and Sejnowski, 1996). Here we generalize this notion to all factors involved in the weight update, leading us to propose centering the slope of hidden unit activation functions as well. Slope centering removes the linear component of backpropagated error; this improves credit assignment in networks with shortcut connections. Benchmark results show that this can speed up learning significantly without adversely affecting the trained network's generalization ability. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1123239. 	[Title] 3D Object Recognition Using Unsupervised Feature Extraction [Abstract] Intrator (1990) proposed a feature extraction method that is related to recent statistical theory (Huber, 1985; Friedman, 1987), and is based on a biologically motivated model of neuronal plasticity (Bienenstock et al., 1982). This method has been recently applied to feature extraction in the context of recognizing 3D objects from single 2D views (Intrator and Gold, 1991). Here we describe experiments designed to analyze the nature of the extracted features, and their relevance to the theory and psychophysics of object recognition. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1132968. 	[Title] Local Feature Analysis: A general statistical theory for object representation [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #75695. 	[Title] Unsupervised Discrimination of Clustered Data via Optimization of Binary Information Gain [Abstract] We present the information-theoretic derivation of a learning algorithm that clusters unlabelled data with linear discriminants. In contrast to methods that try to preserve information about the input patterns, we maximize the information gained from observing the output of robust binary discriminators implemented with sigmoid nodes. We derive a local weight adaptation rule via gradient ascent in this objective, demonstrate its dynamics on some simple data sets, relate our approach to previous work and suggest directions in which it may be extended. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #626999. 	[Title] Tempering Backpropagation Networks: Not All Weights are Created Equal approach yields hitherto unparalleled performance on [Abstract] Backpropagation learning algorithms typically collapse the network's structure into a single vector of weight parameters to be optimized. We suggest that their performance may be improved by utilizing the structural information instead of discarding it, and introduce a framework for tempering each weight accordingly. In the tempering model, activation and error signals are treated as approximately independent random variables. The characteristic scale of weight changes is then matched to that of the residuals, allowing structural properties such as a node's fan-in and fan-out to affect the local learning rate and backpropagated error. The model also permits calculation of an upper bound on the global learning rate for batch updates, which in turn leads to different update rules for bias vs. non-bias weights. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1120059. 	[Title] Plasticity-Mediated Competitive Learning [Abstract] Differentiation between the nodes of a competitive learning network is conventionally achieved through competition on the basis of neural activity. Simple inhibitory mechanisms are limited to sparse representations, while decorrelation and factorization schemes that support distributed representations are computation-ally unattractive. By letting neural plasticity mediate the competitive interaction instead, we obtain diffuse, nonadaptive alternatives for fully distributed representations. We use this technique to simplify and improve our binary information gain optimization algorithm for feature extraction (Schraudolph and Sejnowski, 1993); the same approach could be used to improve other learning algorithms. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1120197. 	[Title] On the Combination of Supervised and Unsupervised Learning  reducing the overall error measure of a classifier. [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1113035. 	[Title] Empirical Entropy Manipulation for Real-World Problems [Abstract] No finite sample is sufficient to determine the density, and therefore the entropy, of a signal directly. Some assumption about either the functional form of the density or about its smoothness is necessary. Both amount to a prior over the space of possible density functions. By far the most common approach is to assume that the density has a parametric form. By contrast we derive a differential learning rule called EMMA that optimizes entropy by way of kernel density estimation. Entropy and its derivative can then be calculated by sampling from this density estimate. The resulting parameter update rule is surprisingly simple and efficient. We will show how EMMA can be used to detect and correct corruption in magnetic resonance images (MRI). This application is beyond the scope of existing parametric entropy models. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1114192. 	[Title] Multimodality Exploration in Training an Unsupervised Projection Pursuit Neural Network [Abstract] Graphical inspection of multimodality is demonstrated using unsupervised lateral-inhibition neural networks. Three projection pursuit indices are compared on low dimensional simulated and real-world data: principal components [22], Legendre poly nomial [6] and projection pursuit network [16]. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #627024. 	[Title] Early Stopping but when? [Abstract] Validation can be used to detect when overfitting starts during supervised training of a neural network; training is then stopped before convergence to avoid the overfitting ("early stopping"). The exact criterion used for validation-based early stopping, however, is usually chosen in an ad-hoc fashion or training is stopped interactively. This trick describes how to select a stopping criterion in a systematic fashion; it is a trick for either speeding learning procedures or improving generalization, whichever is more important in the particular situation. An empirical investigation on multi-layer perceptrons shows that there exists a tradeoff between training time and generalization: From the given mix of 1296 training runs using different 12 problems and 24 different network architectures I conclude slower stopping criteria allow for small improvements in generalization (here: about 4% on average), but cost much more training time (here: about factor 4 longer on average). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1108050. 	[Title] Theory of Synaptic Plasticity in Visual Cortex [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #99025. 	[Title] Neuronal Goals: Efficient Coding and Coincidence Detection [Abstract] Barlow's seminal work on minimal entropy codes and unsupervised learning is reiterated. In particular, the need to transmit the probability of events is put in a practical neuronal framework for detecting suspicious events. A variant of the BCM learning rule [15] is presented together with some mathematical results suggesting optimal minimal entropy coding. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #35797. 	[Title] Proben1 A Set of Neural Network Benchmark Problems and Benchmarking Rules [Abstract] Proben1 is a collection of problems for neural network learning in the realm of pattern classification and function approximation plus a set of rules and conventions for carrying out benchmark tests with these or similar problems. Proben1 contains 15 data sets from 12 different domains. All datasets represent realistic problems which could be called diagnosis tasks and all but one consist of real world data. The datasets are all presented in the same simple format, using an attribute representation that can directly be used for neural network training. Along with the datasets, Proben1 defines a set of rules for how to conduct and how to document neural network benchmarking. The purpose of the problem and rule collection is to give researchers easy access to data for the evaluation of their algorithms and networks and to make direct comparison of the published results feasible. This report describes the datasets and the benchmarking rules. It also gives some basic performance measures indicating the difficulty of the various problems. These measures can be used as baselines for comparison. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1120858. 	[Title] Three-Dimensional Object Recognition Using an Unsupervised BCM Network: The Usefulness of Distinguishing Features [Abstract] We propose an object recognition scheme based on a method for feature extraction from gray level images that corresponds to recent statistical theory, called projection pursuit, and is derived from a biologically motivated feature extracting neuron. To evaluate the performance of this method we use a set of very detailed psychophysical 3D object recognition experiments (Bulthoff and Edelman, 1992). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #5348. 	[Title] Feature Extraction Using an Unsupervised Neural Network [Abstract] A novel unsupervised neural network for dimensionality reduction that seeks directions emphasizing multimodality is presented, and its connection to exploratory projection pursuit methods is discussed. This leads to a new statistical insight into the synaptic modification equations governing learning in Bienenstock, Cooper, and Munro (BCM) neurons (1982). The importance of a dimensionality reduction principle based solely on distinguishing features is demonstrated using a phoneme recognition experiment. The extracted features are compared with features extracted using a back-propagation network. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #256106. 	[Title] Classification of Underwater Mammals using Feature Extraction Based on Time-Frequency Analysis and BCM Theory [Abstract] Underwater mammal sound classification is demonstrated using a novel application of wavelet time/frequency decomposition and feature extraction using a BCM unsupervised network. Different feature extraction methods and different wavelet representations are studied. The system achieves outstanding classification performance even when tested with mammal sounds recorded at very different locations (from those used for training). The improved results suggest that nonlinear feature extraction from wavelet representations outperforms different linear choices of basis functions. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #278394. 	[Title] Statistical Theory of Overtraining Is Cross-Validation Asymptotically Effective? [Abstract] A statistical theory for overtraining is proposed. The analysis treats realizable stochastic neural networks, trained with Kullback-Leibler loss in the asymptotic case. It is shown that the asymptotic gain in the generalization error is small if we perform early stopping, even if we have access to the optimal stopping time. Considering cross-validation stopping we answer the question: In what ratio the examples should be divided into training and testing sets in order to obtain the optimum performance. In the non-asymptotic region cross-validated early stopping always decreases the generalization error. Our large scale simulations done on a CM5 are in nice agreement with our analytical findings. Please summarise all the given information.

Paper #911198 has the title `Centering Neural Network Gradient Factors  
` and the abstract `Technical Report IDSIA-19-97 Abstract. It has long been known that neural networks can learn faster when their input and hidden unit activities are centered about zero; recently we have extended this approach to also encompass the centering of error signals [2]. Here we generalize this notion to all factors involved in the network's gradient, leading us to propose centering the slope of hidden unit activation functions as well. Slope centering removes the linear component of backpropagated error; this improves credit assignment in networks with shortcut connections. Benchmark results show that this can speed up learning significantly without adversely affecting the trained network's generalization ability. 
`.
Question: Which category should Paper #911198 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #911198 has the title `Centering Neural Network Gradient Factors  
` and the abstract `Technical Report IDSIA-19-97 Abstract. It has long been known that neural networks can learn faster when their input and hidden unit activities are centered about zero; recently we have extended this approach to also encompass the centering of error signals [2]. Here we generalize this notion to all factors involved in the network's gradient, leading us to propose centering the slope of hidden unit activation functions as well. Slope centering removes the linear component of backpropagated error; this improves credit assignment in networks with shortcut connections. Benchmark results show that this can speed up learning significantly without adversely affecting the trained network's generalization ability. 
`.
Question: Which category should Paper #911198 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Neural_Networks