Above are papers related to the following paper: [Paper_ID] Paper #6784. 	[Title] A Quantitative Study of Experimental Evaluations of Neural Network Learning Algorithms: Current Research Practice [Abstract] 190 articles about neural network learning algorithms published in 1993 and 1994 are examined for the amount of experimental evaluation they contain. 29% of them employ not even a single realistic or real learning problem. Only 8% of the articles present results for more than one problem using real world data. Furthermore, one third of all articles do not present any quantitative comparison with a previously known algorithm. These results suggest that we should strive for better assessment practices in neural network learning algorithm research. For the long-term benefit of the field, the publication standards should be raised in this respect and easily accessible collections of benchmark problems should be built. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #60169. 	[Title] A Parallel Programming Model for Irregular Dynamic Neural Networks a programming model that allows to [Abstract] A compiler for CuPit has been built for the MasPar MP-1/MP-2 using compilation techniques that can also be applied to most other parallel machines. The paper shortly presents the main ideas of the techniques used and results obtained by the various optimizations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #60170. 	[Title] Connection Pruning with Static and Adaptive Pruning Schedules [Abstract] Neural network pruning methods on the level of individual network parameters (e.g. connection weights) can improve generalization, as is shown in this empirical study. However, an open problem in the pruning methods known today (e.g. OBD, OBS, autoprune, epsiprune) is the selection of the number of parameters to be removed in each pruning step (pruning strength). This work presents a pruning method lprune that automatically adapts the pruning strength to the evolution of weights and loss of generalization during training. The method requires no algorithm parameter adjustment by the user. Results of statistical significance tests comparing autoprune, lprune, and static networks with early stopping are given, based on extensive experimentation with 14 different problems. The results indicate that training with pruning is often significantly better and rarely significantly worse than training with early stopping without pruning. Furthermore, lprune is often superior to autoprune (which is superior to OBD) on diagnosis tasks unless severe pruning early in the training process is required. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #399370. 	[Title] CuPit-2: Portable and Efficient High-Level Parallel Programming of Neural Networks for the Systems Analysis Modelling [Abstract] CuPit-2 is a special-purpose programming language designed for expressing dynamic neural network learning algorithms. It provides most of the flexibility of general-purpose languages such as C or C ++ , but is more expressive. It allows writing much clearer and more elegant programs, in particular for algorithms that change the network topology dynamically (constructive algorithms, pruning algorithms). In contrast to other languages, CuPit-2 programs can be compiled into efficient code for parallel machines without any changes in the source program, thus providing an easy start for using parallel platforms. This article analyzes the circumstances under which the CuPit-2 approach is the most useful one, presents a description of most language constructs and reports performance results for CuPit-2 on symmetric multiprocessors (SMPs). It concludes that in many cases CuPit-2 is a good basis for neural learning algorithm research on small-scale parallel machines. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #193932. 	[Title] Lessons in Neural Network Training: Overfitting Lessons in Neural Network Training: Overfitting May be Harder [Abstract] For many reasons, neural networks have become very popular AI machine learning models. Two of the most important aspects of machine learning models are how well the model generalizes to unseen data, and how well the model scales with problem complexity. Using a controlled task with known optimal training error, we investigate the convergence of the backpropagation (BP) algorithm. We find that the optimal solution is typically not found. Furthermore, we observe that networks larger than might be expected can result in lower training and generalization error. This result is supported by another real world example. We further investigate the training behavior by analyzing the weights in trained networks (excess degrees of freedom are seen to do little harm and to aid convergence), and contrasting the interpolation characteristics of multi-layer perceptron neural networks (MLPs) and polynomial models (overfitting behavior is very different the MLP is often biased towards smoother solutions). Finally, we analyze relevant theory outlining the reasons for significant practical differences. These results bring into question common beliefs about neural network training regarding convergence and optimal network size, suggest alternate guidelines for practical use (lower fear of excess degrees of freedom), and help to direct future work (e.g. methods for creation of more parsimonious solutions, importance of the MLP/BP bias and possibly worse performance of improved training algorithms). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1128974. 	[Title] An Evolutionary Method to Find Good Building-Blocks for Architectures of Artificial Neural Networks [Abstract] This paper deals with the combination of Evolutionary Algorithms and Artificial Neural Networks (ANN). A new method is presented, to find good building-blocks for architectures of Artificial Neural Networks. The method is based on Cellular Encoding, a representation scheme by F. Gruau, and on Genetic Programming by J. Koza. First it will be shown that a modified Cellular Encoding technique is able to find good architectures even for non-boolean networks. With the help of a graph-database and a new graph-rewriting method, it is secondly possible to build architectures from modular structures. The information about building-blocks for architectures is obtained by statistically analyzing the data in the graph-database. Simulation results for two real world problems are given. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #188318. 	[Title] Avoiding Overfitting with BP-SOM [Abstract] Overfitting is a well-known problem in the fields of symbolic and connectionist machine learning. It describes the deterioration of gen-eralisation performance of a trained model. In this paper, we investigate the ability of a novel artificial neural network, bp-som, to avoid overfitting. bp-som is a hybrid neural network which combines a multi-layered feed-forward network (mfn) with Kohonen's self-organising maps (soms). During training, supervised back-propagation learning and unsupervised som learning cooperate in finding adequate hidden-layer representations. We show that bp-som outperforms standard backpropagation, and also back-propagation with a weight decay when dealing with the problem of overfitting. In addition, we show that bp-som succeeds in preserving generalisation performance under hidden-unit pruning, where both other methods fail. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #560936. 	[Title] Monte Carlo Comparison of Non-hierarchical Unsupervised Classifiers [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #36140. 	[Title] Error-Correcting Output Codes: A General Method for Improving Multiclass Inductive Learning Programs [Abstract] Multiclass learning problems involve finding a definition for an unknown function f(x) whose range is a discrete set containing k &gt; 2 values (i.e., k "classes"). The definition is acquired by studying large collections of training examples of the form hx i ; f(x i )i. Existing approaches to this problem include (a) direct application of multiclass algorithms such as the decision-tree algorithms ID3 and CART, (b) application of binary concept learning algorithms to learn individual binary functions for each of the k classes, and (c) application of binary concept learning algorithms with distributed output codes such as those employed by Sejnowski and Rosenberg in the NETtalk system. This paper compares these three approaches to a new technique in which BCH error-correcting codes are employed as a distributed output representation. We show that these output representations improve the performance of ID3 on the NETtalk task and of backpropagation on an isolated-letter speech-recognition task. These results demonstrate that error-correcting output codes provide a general-purpose method for improving the performance of inductive learning programs on multiclass problems. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1050679. 	[Title] Diplomarbeit A Genetic Algorithm for the Topological Optimization of Neural Networks [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #91581. 	[Title] Finding Overlapping Distributions with MML [Abstract] This paper considers an aspect of mixture modelling. Significantly overlapping distributions require more data for their parameters to be accurately estimated than well separated distributions. For example, two Gaussian distributions are considered to significantly overlap when their means are within three standard deviations of each other. If insufficient data is available, only a single component distribution will be estimated, although the data originates from two component distributions. We consider how much data is required to distinguish two component distributions from one distribution in mixture modelling using the minimum message length (MML) criterion. First, we perform experiments which show the MML criterion performs well relative to other Bayesian criteria. Second, we make two improvements to the existing MML estimates, that improve its performance with overlapping distributions. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1111240. 	[Title] On the Distribution of Performance from Multiple Neural Network Trials, On the Distribution of Performance [Abstract] Andrew D. Back was with the Department of Electrical and Computer Engineering, University of Queensland. St. Lucia, Australia. He is now with the Brain Information Processing Group, Frontier Research Program, RIKEN, The Institute of Physical and Chemical Research, 2-1 Hirosawa, Wako-shi, Saitama 351-01, Japan Abstract The performance of neural network simulations is often reported in terms of the mean and standard deviation of a number of simulations performed with different starting conditions. However, in many cases, the distribution of the individual results does not approximate a Gaussian distribution, may not be symmetric, and may be multimodal. We present the distribution of results for practical problems and show that assuming Gaussian distributions can significantly affect the interpretation of results, especially those of comparison studies. For a controlled task which we consider, we find that the distribution of performance is skewed towards better performance for smoother target functions and skewed towards worse performance Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #562123. 	[Title] Comparison of Bayesian and Neural Net Unsupervised Classification Techniques [Abstract] Unsupervised classification is the classification of data into a number of classes in such a way that data in each class are all similar to each other. In the past there have been few if any studies done to compare the performance of different unsupervised classification techniques. In this paper we review Bayesian and neural net approaches to unsupervised classification and present results of experiments that we did to compare Autoclass, a Bayesian classification system, and ART2, a neural net classification algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #627024. 	[Title] Early Stopping but when? [Abstract] Validation can be used to detect when overfitting starts during supervised training of a neural network; training is then stopped before convergence to avoid the overfitting ("early stopping"). The exact criterion used for validation-based early stopping, however, is usually chosen in an ad-hoc fashion or training is stopped interactively. This trick describes how to select a stopping criterion in a systematic fashion; it is a trick for either speeding learning procedures or improving generalization, whichever is more important in the particular situation. An empirical investigation on multi-layer perceptrons shows that there exists a tradeoff between training time and generalization: From the given mix of 1296 training runs using different 12 problems and 24 different network architectures I conclude slower stopping criteria allow for small improvements in generalization (here: about 4% on average), but cost much more training time (here: about factor 4 longer on average). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1365. 	[Title] Cholinergic suppression of transmission may allow combined associative memory function and self-organization in the neocortex. [Abstract] Selective suppression of transmission at feedback synapses during learning is proposed as a mechanism for combining associative feedback with self-organization of feedforward synapses. Experimental data demonstrates cholinergic suppression of synaptic transmission in layer I (feedback synapses), and a lack of suppression in layer IV (feed-forward synapses). A network with this feature uses local rules to learn mappings which are not linearly separable. During learning, sensory stimuli and desired response are simultaneously presented as input. Feedforward connections form self-organized representations of input, while suppressed feedback connections learn the transpose of feedfor-ward connectivity. During recall, suppression is removed, sensory input activates the self-organized representation, and activity generates the learned response. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1116629. 	[Title] Comparing Adaptive and Non-Adaptive Connection Pruning With Pure Early Stopping [Abstract] Neural network pruning methods on the level of individual network parameters (e.g. connection weights) can improve generalization, as is shown in this empirical study. However, an open problem in the pruning methods known today (OBD, OBS, autoprune, epsiprune) is the selection of the number of parameters to be removed in each pruning step (pruning strength). This work presents a pruning method lprune that automatically adapts the pruning strength to the evolution of weights and loss of generalization during training. The method requires no algorithm parameter adjustment by the user. Results of statistical significance tests comparing autoprune, lprune, and static networks with early stopping are given, based on extensive experimentation with 14 different problems. The results indicate that training with pruning is often significantly better and rarely significantly worse than training with early stopping without pruning. Furthermore, lprune is often superior to autoprune (which is superior to OBD) on diagnosis tasks unless severe pruning early in the training process is required. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #35797. 	[Title] Proben1 A Set of Neural Network Benchmark Problems and Benchmarking Rules [Abstract] Proben1 is a collection of problems for neural network learning in the realm of pattern classification and function approximation plus a set of rules and conventions for carrying out benchmark tests with these or similar problems. Proben1 contains 15 data sets from 12 different domains. All datasets represent realistic problems which could be called diagnosis tasks and all but one consist of real world data. The datasets are all presented in the same simple format, using an attribute representation that can directly be used for neural network training. Along with the datasets, Proben1 defines a set of rules for how to conduct and how to document neural network benchmarking. The purpose of the problem and rule collection is to give researchers easy access to data for the evaluation of their algorithms and networks and to make direct comparison of the published results feasible. This report describes the datasets and the benchmarking rules. It also gives some basic performance measures indicating the difficulty of the various problems. These measures can be used as baselines for comparison. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1110998. 	[Title] Interpretable Neural Networks with BP-SOM [Abstract] Interpretation of models induced by artificial neural networks is often a difficult task. In this paper we focus on a relatively novel neural network architecture and learning algorithm, bp-som, that offers possibilities to overcome this difficulty. It is shown that networks trained with bp-som show interesting regularities, in that hidden-unit activations become restricted to discrete values, and that the som part can be exploited for automatic rule extraction. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #100961. 	[Title] Adaptive Parameter Pruning in Neural Networks [Abstract] Neural network pruning methods on the level of individual network parameters (e.g. connection weights) can improve generalization. An open problem in the pruning methods known today (OBD, OBS, autoprune, epsiprune) is the selection of the number of parameters to be removed in each pruning step (pruning strength). This paper presents a pruning method lprune that automatically adapts the pruning strength to the evolution of weights and loss of generalization during training. The method requires no algorithm parameter adjustment by the user. The results of extensive experimentation indicate that lprune is often superior to autoprune (which is superior to OBD) on diagnosis tasks unless severe pruning early in the training process is required. Results of statistical significance tests comparing autoprune to the new method lprune as well as to backpropagation with early stopping are given for 14 different problems. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #4330. 	[Title] Hierarchical Mixtures of Experts and the EM Algorithm [Abstract] We present a tree-structured architecture for supervised learning. The statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models (GLIM's). Learning is treated as a maximum likelihood problem; in particular, we present an Expectation-Maximization (EM) algorithm for adjusting the parameters of the architecture. We also develop an on-line learning algorithm in which the parameters are updated incrementally. Comparative simulation results are presented in the robot dynamics domain. This report describes research done at the Dept. of Brain and Cognitive Sciences, the Center for Biological and Computational Learning, and the Artificial Intelligence Laboratory of the Massachusetts Institute of Technology. Support for CBCL is provided in part by a grant from the NSF (ASC-9217041). Support for the laboratory's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Dept. of Defense. The authors were supported by a grant from the McDonnell-Pew Foundation, by a grant from ATR Human Information Processing Research Laboratories, by a grant from Siemens Corporation, by by grant IRI-9013991 from the National Science Foundation, by grant N00014-90-J-1942 from the Office of Naval Research, and by NSF grant ECS-9216531 to support an Initiative in Intelligent Control at MIT. Michael I. Jordan is a NSF Presidential Young Investigator. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #350319. 	[Title] Averaging and Data Snooping [Abstract] Presenting and Analyzing the Results of AI Experiments: Data Averaging and Data Snooping, Proceedings of the Fourteenth National Conference on Artificial Intelligence, AAAI-97, AAAI Press, Menlo Park, California, pp. 362367, 1997. Copyright AAAI. Presenting and Analyzing the Results of AI Experiments: Abstract Experimental results reported in the machine learning AI literature can be misleading. This paper investigates the common processes of data averaging (reporting results in terms of the mean and standard deviation of the results from multiple trials) and data snooping in the context of neural networks, one of the most popular AI machine learning models. Both of these processes can result in misleading results and inaccurate conclusions. We demonstrate how easily this can happen and propose techniques for avoiding these very important problems. For data averaging, common presentation assumes that the distribution of individual results is Gaussian. However, we investigate the distribution for common problems and find that it often does not approximate the Gaussian distribution, may not be symmetric, and may be multimodal. We show that assuming Gaussian distributions can significantly affect the interpretation of results, especially those of comparison studies. For a controlled task, we find that the distribution of performance is skewed towards better performance for smoother target functions and skewed towards worse performance for more complex target functions. We propose new guidelines for reporting performance which provide more information about the actual distribution (e.g. box-whiskers plots). For data snooping, we demonstrate that optimization of performance via experimentation with multiple parameters can lead to significance being assigned to results which are due to chance. We suggest that precise descriptions of experimental techniques can be very important to the evaluation of results, and that we need to be aware of potential data snooping biases when formulating these experimental techniques (e.g. selecting the test procedure). Additionally, it is important to only rely on appropriate statistical tests and to ensure that any assumptions made in the tests are valid (e.g. normality of the distribution). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6775. 	[Title] Statistical Evaluation of Neural Network Experiments: Minimum Requirements and Current Practice [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1272. 	[Title] Learning a set of primitive actions with an Induction of decision trees. Machine Learning, 1(1):81-106, [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1786. 	[Title] A BENCHMARK FOR CLASSIFIER LEARNING [Abstract] Technical Report 474 November 1993 Please summarise all the given information.

Paper #6784 has the title `A Quantitative Study of Experimental Evaluations of Neural Network Learning Algorithms: Current Research Practice  
` and the abstract `190 articles about neural network learning algorithms published in 1993 and 1994 are examined for the amount of experimental evaluation they contain. 29% of them employ not even a single realistic or real learning problem. Only 8% of the articles present results for more than one problem using real world data. Furthermore, one third of all articles do not present any quantitative comparison with a previously known algorithm. These results suggest that we should strive for better assessment practices in neural network learning algorithm research. For the long-term benefit of the field, the publication standards should be raised in this respect and easily accessible collections of benchmark problems should be built. 
`.
Question: Which category should Paper #6784 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #6784 has the title `A Quantitative Study of Experimental Evaluations of Neural Network Learning Algorithms: Current Research Practice  
` and the abstract `190 articles about neural network learning algorithms published in 1993 and 1994 are examined for the amount of experimental evaluation they contain. 29% of them employ not even a single realistic or real learning problem. Only 8% of the articles present results for more than one problem using real world data. Furthermore, one third of all articles do not present any quantitative comparison with a previously known algorithm. These results suggest that we should strive for better assessment practices in neural network learning algorithm research. For the long-term benefit of the field, the publication standards should be raised in this respect and easily accessible collections of benchmark problems should be built. 
`.
Question: Which category should Paper #6784 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Neural_Networks