Above are papers related to the following paper: [Paper_ID] Paper #46470. 	[Title] On the Boosting Ability of Top-Down Decision Tree Learning Algorithms provably optimal for decision tree [Abstract] We analyze the performance of top-down algorithms for decision tree learning, such as those employed by the widely used C4.5 and CART software packages. Our main result is a proof that such algorithms are boosting algorithms. By this we mean that if the functions that label the internal nodes of the decision tree can weakly approximate the unknown target function, then the top-down algorithms we study will amplify this weak advantage to build a tree achieving any desired level of accuracy. The bounds we obtain for this amplification show an interesting dependence on the splitting criterion used by the top-down algorithm. More precisely, if the functions used to label the internal nodes have error 1=2 fl as approximations to the target function, then for the splitting criteria used by CART and C4.5, trees of size (1=*) O(1=fl 2 * 2 ) and (1=*) O(log(1=*)=fl 2 ) (respectively) suffice to drive the error below *. Thus (for example), a small constant advantage over random guessing is amplified to any larger constant advantage with trees of constant size. For a new splitting criterion suggested by our analysis, the much stronger fl A preliminary version of this paper appears in Proceedings of the Twenty-Eighth Annual ACM Symposium on the Theory of Computing, pages 459-468, ACM Press, 1996. Authors' addresses: M. Kearns, AT&T Research, 600 Mountain Avenue, Room 2A-423, Murray Hill, New Jersey 07974; electronic mail mkearns@research.att.com. Y. Mansour, Department of Computer Science, Tel Aviv University, Tel Aviv, Israel; electronic mail mansour@math.tau.ac.il. Y. Mansour was supported in part by the Israel Science Foundation, administered by the Israel Academy of Science and Humanities, and by a grant of the Israeli Ministry of Science and Technology. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #45189. 	[Title] An Experimental and Theoretical Comparison of Model Selection Methods on simple model selection problems, the [Abstract] We investigate the problem of model selection in the setting of supervised learning of boolean functions from independent random examples. More precisely, we compare methods for finding a balance between the complexity of the hypothesis chosen and its observed error on a random training sample of limited size, when the goal is that of minimizing the resulting generalization error. We undertake a detailed comparison of three well-known model selection methods | a variation of Vapnik's Guaranteed Risk Minimization (GRM), an instance of Rissanen's Minimum Description Length Principle (MDL), and cross validation (CV). We introduce a general class of model selection methods (called penalty-based methods) that includes both GRM and MDL, and provide general methods for analyzing such rules. We provide both controlled experimental evidence and formal theorems to support the following conclusions: * The class of penalty-based methods is fundamentally handicapped in the sense that there exist two types of model selection problems for which every penalty-based method must incur large generalization error on at least one, while CV enjoys small generalization error Despite the inescapable incomparability of model selection methods under certain circumstances, we conclude with a discussion of our belief that the balance of the evidence provides specific reasons to prefer CV to other methods, unless one is in possession of detailed problem-specific information. on both. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1105450. 	[Title] A Fast, Bottom-Up Decision Tree Pruning Algorithm with Near-Optimal Generalization [Abstract] In this work, we present a new bottom-up algorithm for decision tree pruning that is very efficient (requiring only a single pass through the given tree), and prove a strong performance guarantee for the generalization error of the resulting pruned tree. We work in the typical setting in which the given tree T may have been derived from the given training sample S, and thus may badly overfit S. In this setting, we give bounds on the amount of additional generalization error that our pruning suffers compared to the optimal pruning of T . More generally, our results show that if there is a pruning of T with small error, and whose size is small compared to jSj, then our algorithm will find a pruning whose error is not much larger. This style of result has been called an index of resolvability result by Barron and Cover in the context of density estimation. A novel feature of our algorithm is its locality | the decision to prune a subtree is based entirely on properties of that subtree and the sample reaching it. To analyze our algorithm, we develop tools of local uniform convergence, a generalization of the standard notion that may prove useful in other settings. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #65650. 	[Title] Machine Learning 27(1):51-68, 1997. Predicting nearly as well as the best pruning of a decision tree [Abstract] Many algorithms for inferring a decision tree from data involve a two-phase process: First, a very large decision tree is grown which typically ends up "over-fitting" the data. To reduce over-fitting, in the second phase, the tree is pruned using one of a number of available methods. The final tree is then output and used for classification on test data. In this paper, we suggest an alternative approach to the pruning phase. Using a given unpruned decision tree, we present a new method of making predictions on test data, and we prove that our algorithm's performance will not be "much worse" (in a precise technical sense) than the predictions made by the best reasonably small pruning of the given decision tree. Thus, our procedure is guaranteed to be competitive (in terms of the quality of its predictions) with any pruning algorithm. We prove that our procedure is very efficient and highly robust. Our method can be viewed as a synthesis of two previously studied techniques. First, we apply Cesa-Bianchi et al.'s [4] results on predicting using "expert advice" (where we view each pruning as an "expert") to obtain an algorithm that has provably low prediction loss, but that is com-putationally infeasible. Next, we generalize and apply a method developed by Buntine [3], [2] and Willems, Shtarkov and Tjalkens [20], [21] to derive a very efficient implementation of this procedure. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #65653. 	[Title] Pessimistic decision tree pruning based on tree size [Abstract] In this work we develop a new criteria to perform pessimistic decision tree pruning. Our method is theoretically sound and is based on theoretical concepts such as uniform convergence and the Vapnik-Chervonenkis dimension. We show that our criteria is very well motivated, from the theory side, and performs very well in practice. The accuracy of the new criteria is comparable to that of the current method used in C4.5. Please summarise all the given information.

Paper #46470 has the title `On the Boosting Ability of Top-Down Decision Tree Learning Algorithms provably optimal for decision tree
` and the abstract `We analyze the performance of top-down algorithms for decision tree learning, such as those employed by the widely used C4.5 and CART software packages. Our main result is a proof that such algorithms are boosting algorithms. By this we mean that if the functions that label the internal nodes of the decision tree can weakly approximate the unknown target function, then the top-down algorithms we study will amplify this weak advantage to build a tree achieving any desired level of accuracy. The bounds we obtain for this amplification show an interesting dependence on the splitting criterion used by the top-down algorithm. More precisely, if the functions used to label the internal nodes have error 1=2 fl as approximations to the target function, then for the splitting criteria used by CART and C4.5, trees of size (1=*) O(1=fl 2 * 2 ) and (1=*) O(log(1=*)=fl 2 ) (respectively) suffice to drive the error below *. Thus (for example), a small constant advantage over random guessing is amplified to any larger constant advantage with trees of constant size. For a new splitting criterion suggested by our analysis, the much stronger fl A preliminary version of this paper appears in Proceedings of the Twenty-Eighth Annual ACM Symposium on the Theory of Computing, pages 459-468, ACM Press, 1996. Authors' addresses: M. Kearns, AT&T Research, 600 Mountain Avenue, Room 2A-423, Murray Hill, New Jersey 07974; electronic mail mkearns@research.att.com. Y. Mansour, Department of Computer Science, Tel Aviv University, Tel Aviv, Israel; electronic mail mansour@math.tau.ac.il. Y. Mansour was supported in part by the Israel Science Foundation, administered by the Israel Academy of Science and Humanities, and by a grant of the Israeli Ministry of Science and Technology. 
`.
Question: Which category should Paper #46470 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #46470 has the title `On the Boosting Ability of Top-Down Decision Tree Learning Algorithms provably optimal for decision tree
` and the abstract `We analyze the performance of top-down algorithms for decision tree learning, such as those employed by the widely used C4.5 and CART software packages. Our main result is a proof that such algorithms are boosting algorithms. By this we mean that if the functions that label the internal nodes of the decision tree can weakly approximate the unknown target function, then the top-down algorithms we study will amplify this weak advantage to build a tree achieving any desired level of accuracy. The bounds we obtain for this amplification show an interesting dependence on the splitting criterion used by the top-down algorithm. More precisely, if the functions used to label the internal nodes have error 1=2 fl as approximations to the target function, then for the splitting criteria used by CART and C4.5, trees of size (1=*) O(1=fl 2 * 2 ) and (1=*) O(log(1=*)=fl 2 ) (respectively) suffice to drive the error below *. Thus (for example), a small constant advantage over random guessing is amplified to any larger constant advantage with trees of constant size. For a new splitting criterion suggested by our analysis, the much stronger fl A preliminary version of this paper appears in Proceedings of the Twenty-Eighth Annual ACM Symposium on the Theory of Computing, pages 459-468, ACM Press, 1996. Authors' addresses: M. Kearns, AT&T Research, 600 Mountain Avenue, Room 2A-423, Murray Hill, New Jersey 07974; electronic mail mkearns@research.att.com. Y. Mansour, Department of Computer Science, Tel Aviv University, Tel Aviv, Israel; electronic mail mansour@math.tau.ac.il. Y. Mansour was supported in part by the Israel Science Foundation, administered by the Israel Academy of Science and Humanities, and by a grant of the Israeli Ministry of Science and Technology. 
`.
Question: Which category should Paper #46470 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Theory