Above are papers related to the following paper: [Paper_ID] Paper #1109542. 	[Title] The covariance inflation criterion for adaptive model selection [Abstract] We propose a new criterion for model selection in prediction problems. The covariance inflation criterion adjusts the training error by the average covariance of the predictions and responses, when the prediction rule is applied to permuted versions of the dataset. This criterion can be applied to general prediction problems (for example regression or classification), and to general prediction rules (for example stepwise regression, tree-based models and neural nets). As a byproduct we obtain a measure of the effective number of parameters used by an adaptive procedure. We relate the covariance inflation criterion to other model selection procedures and illustrate its use in some regression and classification problems. We also revisit the conditional bootstrap approach to model selection. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #3233. 	[Title] A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection [Abstract] We review accuracy estimation methods and compare the two most common methods: cross-validation and bootstrap. Recent experimental results on artificial data and theoretical results in restricted settings have shown that for selecting a good classifier from a set of classifiers (model selection), ten-fold cross-validation may be better than the more expensive leave-one-out cross-validation. We report on a large-scale experiment|over half a million runs of C4.5 and a Naive-Bayes algorithm|to estimate the effects of different parameters on these algorithms on real-world datasets. For cross-validation, we vary the number of folds and whether the folds are stratified or not; for bootstrap, we vary the number of bootstrap samples. Our results indicate that for real-word datasets similar to ours, the best method to use for model selection is ten-fold stratified cross validation, even if computation power allows using more folds. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #154982. 	[Title] Bias, variance and prediction error for classification rules [Abstract] We study the notions of bias and variance for classification rules. Following Efron (1978) we develop a decomposition of prediction error into its natural components. Then we derive bootstrap estimates of these components and illustrate how they can be used to describe the error behaviour of a classifier in practice. In the process we also obtain a bootstrap estimate of the error of a "bagged" classifier. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1108551. 	[Title] From: Computational Learning Theory and Natural Systems, Chapter 18, "Cross-validation and Modal Theories", Cross-Validation and [Abstract] Cross-validation is a frequently used, intuitively pleasing technique for estimating the accuracy of theories learned by machine learning algorithms. During testing of a machine learning algorithm (foil) on new databases of prokaryotic RNA transcription promoters which we have developed, cross-validation displayed an interesting phenomenon. One theory is found repeatedly and is responsible for very little of the cross-validation error, whereas other theories are found very infrequently which tend to be responsible for the majority of the cross-validation error. It is tempting to believe that the most frequently found theory (the "modal theory") may be more accurate as a classifier of unseen data than the other theories. However, experiments showed that modal theories are not more accurate on unseen data than the other theories found less frequently during cross-validation. Modal theories may be useful in predicting when cross-validation is a poor estimate of true accuracy. We offer explanations 1 For correspondence: Department of Computer Science and Engineering, University of California, San Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #129897. 	[Title] The out-of-bootstrap method for model averaging and selection [Abstract] We propose a bootstrap-based method for model averaging and selection that focuses on training points that are left out of individual bootstrap samples. This information can be used to estimate optimal weighting factors for combining estimates from different bootstrap samples, and also for finding the best subsets the linear model setting. These proposals provide alternatives to Bayesian approaches to model averaging and selection, requiring less computation and fewer subjective choices. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1102442. 	[Title] Unbiased Assessment of Learning Algorithms [Abstract] In order to rank the performance of machine learning algorithms, many researchers conduct experiments on benchmark data sets. Since most learning algorithms have domain-specific parameters, it is a popular custom to adapt these parameters to obtain a minimal error rate on the test set. The same rate is then used to rank the algorithm, which causes an optimistic bias. We quantify this bias, showing, in particular, that an algorithm with more parameters will probably be ranked higher than an equally good algorithm with fewer parameters. We demonstrate this result, showing the number of parameters and trials required in order to pretend to outperform C4.5 or FOIL, respectively, for various benchmark problems. We then describe out how unbiased ranking experiments should be conducted. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #25772. 	[Title] Flexible Metric Nearest Neighbor Classiflcation [Abstract] The K-nearest-neighbor decision rule assigns an object of unknown class to the plurality class among the K labeled "training" objects that are closest to it. Closeness is usually deflned in terms of a metric distance on the Euclidean space with the input measurement variables as axes. The metric chosen to deflne this distance can strongly efiect performance. An optimal choice depends on the problem at hand as characterized by the respective class distributions on the input measurement space, and within a given problem, on the location of the unknown object in that space. In this paper new types of K-nearest-neighbor procedures are described that estimate the local relevance of each input variable, or their linear combinations, for each individual point to be classifled. This information is then used to separately customize the metric used to deflne distance from that object in flnding its nearest neighbors. These procedures are a hybrid between regular K-nearest-neighbor methods and treestructured recursive partitioning techniques popular in statistics and machine learning. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1997. 	[Title] Cross-Validation and the Bootstrap: Estimating the Error Rate of a Prediction Rule [Abstract] A training set of data has been used to construct a rule for predicting future responses. What is the error rate of this rule? The traditional answer to this question is given by cross-validation. The cross-validation estimate of prediction error is nearly unbiased, but can be highly variable. This article discusses bootstrap estimates of prediction error, which can be thought of as smoothed versions of cross-validation. A particular bootstrap method, the 632+ rule, is shown to substantially outperform cross-validation in a catalog of 24 simulation experiments. Besides providing point estimates, we also consider estimating the variability of an error rate estimate. All of the results here are nonparametric, and apply to any possible prediction rule: however we only study classification problems with 0-1 loss in detail. Our simulations include "smooth" prediction rules like Fisher's Linear Discriminant Function, and unsmooth ones like Nearest Neighbors. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #49811. 	[Title] Estimating the Accuracy of Learned Concepts [Abstract] This paper investigates alternative estimators of the accuracy of concepts learned from examples. In particular, the cross-validation and 632 bootstrap estimators are studied, using synthetic training data and the foil learning algorithm. Our experimental results contradict previous papers in statistics, which advocate the 632 bootstrap method as superior to cross-validation. Nevertheless, our results also suggest that conclusions based on cross-validation in previous machine learning papers are unreliable. Specifically, our observations are that (i) the true error of the concept learned by foil from independently drawn sets of examples of the same concept varies widely, (ii) the estimate of true error provided by cross-validation has high variability but is approximately unbiased, and (iii) the 632 bootstrap estimator has lower variability than cross-validation, but is systematically biased. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #7032. 	[Title] Combining estimates in regression and classification [Abstract] We consider the problem of how to combine a collection of general regression fit vectors in order to obtain a better predictive model. The individual fits may be from subset linear regression, ridge regression, or something more complex like a neural network. We develop a general framework for this problem and examine a recent cross-validation-based proposal called "stacking" in this context. Combination methods based on the bootstrap and analytic methods are also derived and compared in a number of examples, including best subsets regression and regression trees. Finally, we apply these ideas to classification problems where the estimated combination weights can yield insight into the structure of the problem. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1109439. 	[Title] Classifying Seismic Signals by Integrating Ensembles of Neural Networks [Abstract] This paper proposes a classification scheme based on integration of multiple Ensembles of ANNs. It is demonstrated on a classification problem, in which seismic signals of Natural Earthquakes must be distinguished from seismic signals of Artificial Explosions. A Redundant Classification Environment consists of several Ensembles of Neural Networks is created and trained on Bootstrap Sample Sets, using various data representations and architectures. The ANNs within the Ensembles are aggregated (as in Bagging) while the Ensembles are integrated non-linearly, in a signal adaptive manner, using a posterior confidence measure based on the agreement (variance) within the Ensembles. The proposed Integrated Classification Machine achieved 92.1% correct classifications on the seismic test data. Cross Validation evaluations and comparisons indicate that such integration of a collection of ANN's Ensembles is a robust way for handling high dimensional problems with a complex non-stationary signal space as in the current Seismic Classification problem. Please summarise all the given information.

Paper #1109542 has the title `The covariance inflation criterion for adaptive model selection  
` and the abstract `We propose a new criterion for model selection in prediction problems. The covariance inflation criterion adjusts the training error by the average covariance of the predictions and responses, when the prediction rule is applied to permuted versions of the dataset. This criterion can be applied to general prediction problems (for example regression or classification), and to general prediction rules (for example stepwise regression, tree-based models and neural nets). As a byproduct we obtain a measure of the effective number of parameters used by an adaptive procedure. We relate the covariance inflation criterion to other model selection procedures and illustrate its use in some regression and classification problems. We also revisit the conditional bootstrap approach to model selection. 
`.
Question: Which category should Paper #1109542 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #1109542 has the title `The covariance inflation criterion for adaptive model selection  
` and the abstract `We propose a new criterion for model selection in prediction problems. The covariance inflation criterion adjusts the training error by the average covariance of the predictions and responses, when the prediction rule is applied to permuted versions of the dataset. This criterion can be applied to general prediction problems (for example regression or classification), and to general prediction rules (for example stepwise regression, tree-based models and neural nets). As a byproduct we obtain a measure of the effective number of parameters used by an adaptive procedure. We relate the covariance inflation criterion to other model selection procedures and illustrate its use in some regression and classification problems. We also revisit the conditional bootstrap approach to model selection. 
`.
Question: Which category should Paper #1109542 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Probabilistic_Methods