Above are papers related to the following paper: [Paper_ID] Paper #11342. 	[Title] On Pruning and Averaging Decision Trees [Abstract] Pruning a decision tree is considered by some researchers to be the most important part of tree building in noisy domains. While, there are many approaches to pruning, an alternative approach of averaging over decision trees has not received as much attention. We perform an empirical comparison of pruning with the approach of averaging over decision trees. For this comparison we use a computa-tionally efficient method of averaging, namely averaging over the extended fanned set of a tree. Since there are a wide range of approaches to pruning, we compare tree averaging with a traditional pruning approach, along with an optimal pruning approach. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #24966. 	[Title] A Practical Bayesian Framework for Backprop Networks [Abstract] A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks. The framework makes possible: (1) objective comparisons between solutions using alternative network architectures; (2) objective stopping rules for network pruning or growing procedures; (3) objective choice of magnitude and type of weight decay terms or additive regularisers (for penalising large weights, etc.); (4) a measure of the effective number of well-determined parameters in a model; (5) quantified estimates of the error bars on network parameters and on network output; (6) objective comparisons with alternative learning and interpolation models such as splines and radial basis functions. The Bayesian `evidence' automatically embodies `Occam's razor,' penalising over-flexible and over-complex models. The Bayesian approach helps detect poor underlying assumptions in learning models. For learning models well matched to a problem, a good correlation between generalisation ability and the Bayesian evidence is obtained. This paper makes use of the Bayesian framework for regularisation and model comparison described in the companion paper `Bayesian interpolation' (MacKay, 1991a). This framework is due to Gull and Skilling (Gull, 1989a). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #189574. 	[Title] Theory Refinement on Bayesian Networks [Abstract] Theory refinement is the task of updating a domain theory in the light of new cases, to be done automatically or with some expert assistance. The problem of theory refinement under uncertainty is reviewed here in the context of Bayesian statistics, a theory of belief revision. The problem is reduced to an incremental learning task as follows: the learning system is initially primed with a partial theory supplied by a domain expert, and thereafter maintains its own internal representation of alternative theories which is able to be interrogated by the domain expert and able to be incrementally refined from data. Algorithms for refinement of Bayesian networks are presented to illustrate what is meant by "partial theory", "alternative theory representation", etc. The algorithms are an incremental variant of batch learning algorithms from the literature so can work well in batch and incremental mode. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1132815. 	[Title] Fast Bounded Smooth Regression with Lazy Neural Trees [Abstract] We propose the lazy neural tree (LNT) as the appropriate architecture for the realization of smooth regression systems. The LNT is a hybrid of a decision tree and a neural network. From the neural network it inherits smoothness of the generated function, incremental adaptability, and conceptual simplicity. From the decision tree it inherits the topology and initial parameter setting as well as a very efficient sequential implementation that out-performs traditional neural network simulations by the order of magnitudes. The enormous speed is achieved by lazy evaluation. A further speed-up can be obtained by the application of a window-ing scheme if the region of interesting results is restricted. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #49811. 	[Title] Estimating the Accuracy of Learned Concepts [Abstract] This paper investigates alternative estimators of the accuracy of concepts learned from examples. In particular, the cross-validation and 632 bootstrap estimators are studied, using synthetic training data and the foil learning algorithm. Our experimental results contradict previous papers in statistics, which advocate the 632 bootstrap method as superior to cross-validation. Nevertheless, our results also suggest that conclusions based on cross-validation in previous machine learning papers are unreliable. Specifically, our observations are that (i) the true error of the concept learned by foil from independently drawn sets of examples of the same concept varies widely, (ii) the estimate of true error provided by cross-validation has high variability but is approximately unbiased, and (iii) the 632 bootstrap estimator has lower variability than cross-validation, but is systematically biased. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #12182. 	[Title] How to Use Expert Advice (Extended Abstract) [Abstract] We analyze algorithms that predict a binary value by combining the predictions of several prediction strategies, called experts. Our analysis is for worst-case situations, i.e., we make no assumptions about the way the sequence of bits to be predicted is generated. We measure the performance of the algorithm by the difference between the expected number of mistakes it makes on the bit sequence and the expected number of mistakes made by the best expert on this sequence, where the expectation is taken with respect to the randomization in the predictions. We show that the minimum achievable difference is on the order of the square root of the number of mistakes of the best expert, and we give efficient algorithms that achieve this. Our upper and lower bounds have matching leading constants in most cases. We give implications of this result on the performance of batch learning algorithms in a PAC setting which improve on the best results currently known in this context. We also extend our analysis to the case in which log loss is used instead of the expected number of mistakes. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1110426. 	[Title] Incremental Reduced Error Pruning [Abstract] This paper outlines some problems that may occur with Reduced Error Pruning in relational learning algorithms, most notably efficiency. Thereafter a new method, Incremental Reduced Error Pruning, is proposed that attempts to address all of these problems. Experiments show that in many noisy domains this method is much more efficient than alternative algorithms, along with a slight gain in accuracy. However, the experiments show as well that the use of the algorithm cannot be recommended for domains which require a very specific concept description. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #101662. 	[Title] Learning from positive data [Abstract] Gold showed in 1967 that not even regular grammars can be exactly identified from positive examples alone. Since it is known that children learn natural grammars almost exclusively from positives examples, Gold's result has been used as a theoretical support for Chomsky's theory of innate human linguistic abilities. In this paper new results are presented which show that within a Bayesian framework not only grammars, but also logic programs are learnable with arbitrarily low expected error from positive examples only. In addition, we show that the upper bound for expected error of a learner which maximises the Bayes' posterior probability when learning from positive examples is within a small additive term of one which does the same from a mixture of positive and negative examples. An Inductive Logic Programming implementation is described which avoids the pitfalls of greedy search by global optimisation of this function during the local construction of individual clauses of the hypothesis. Results of testing this implementation on artificially-generated data-sets are reported. These results are in agreement with the theoretical predictions. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #22431. 	[Title] An investigation of noise-tolerant relational concept learning algorithms [Abstract] We discuss the types of noise that may occur in relational learning systems and describe two approaches to addressing noise in a relational concept learning algorithm. We then evaluate each approach experimentally. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #8224. 	[Title] Mingers, 1989 J. Mingers. An empirical comparison of pruning methods for decision tree induction. Machine [Abstract] Ourston and Mooney, 1990b ] D. Ourston and R. J. Mooney. Improving shared rules in multiple category domain theories. Technical Report AI90-150, Artificial Intelligence Labora tory, University of Texas, Austin, TX, December 1990. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #22563. 	[Title] A Comparative Study of ID3 and Backpropagation for English Text-to-Speech Mapping [Abstract] The performance of the error backpropagation (BP) and ID3 learning algorithms was compared on the task of mapping English text to phonemes and stresses. Under the distributed output code developed by Sejnowski and Rosenberg, it is shown that BP consistently out-performs ID3 on this task by several percentage points. Three hypotheses explaining this difference were explored: (a) ID3 is overfitting the training data, (b) BP is able to share hidden units across several output units and hence can learn the output units better, and (c) BP captures statistical information that ID3 does not. We conclude that only hypothesis (c) is correct. By augmenting ID3 with a simple statistical learning procedure, the performance of BP can be approached but not matched. More complex statistical procedures can improve the performance of both BP and ID3 substantially. A study of the residual errors suggests that there is still substantial room for improvement in learning methods for text-to-speech mapping. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6311. 	[Title] Dynamic Automatic Model Selection [Abstract] COINS Technical Report 92-30 February 1992 Abstract The problem of how to learn from examples has been studied throughout the history of machine learning, and many successful learning algorithms have been developed. A problem that has received less attention is how to select which algorithm to use for a given learning task. The ability of a chosen algorithm to induce a good generalization depends on how appropriate the model class underlying the algorithm is for the given task. We define an algorithm's model class to be the representation language it uses to express a generalization of the examples. Supervised learning algorithms differ in their underlying model class and in how they search for a good generalization. Given this characterization, it is not surprising that some algorithms find better generalizations for some, but not all tasks. Therefore, in order to find the best generalization for each task, an automated learning system must search for the appropriate model class in addition to searching for the best generalization within the chosen class. This thesis proposal investigates the issues involved in automating the selection of the appropriate model class. The presented approach has two facets. Firstly, the approach combines different model classes in the form of a model combination decision tree, which allows the best representation to be found for each subconcept of the learning task. Secondly, which model class is the most appropriate is determined dynamically using a set of heuristic rules. Explicit in each rule are the conditions in which a particular model class is appropriate and if it is not, what should be done next. In addition to describing the approach, this proposal describes how the approach will be evaluated in order to demonstrate that it is both an efficient and effective method for automatic model selection. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1153064. 	[Title] Stochastic Logic Programs [Abstract] One way to represent a machine learning algorithm's bias over the hypothesis and instance space is as a pair of probability distributions. This approach has been taken both within Bayesian learning schemes and the framework of U-learnability. However, it is not obvious how an Inductive Logic Programming (ILP) system should best be provided with a probability distribution. This paper extends the results of a previous paper by the author which introduced stochastic logic programs as a means of providing a structured definition of such a probability distribution. Stochastic logic programs are a generalisation of stochastic grammars. A stochastic logic program consists of a set of labelled clauses p : C where p is from the interval [0; 1] and C is a range-restricted definite clause. A stochastic logic program P has a distributional semantics, that is one which assigns a probability distribution to the atoms of each predicate in the Herbrand base of the clauses in P . These probabilities are assigned to atoms according to an SLD-resolution strategy which employs a stochastic selection rule. It is shown that the probabilities can be computed directly for fail-free logic programs and by normalisation for arbitrary logic programs. The stochastic proof strategy can be used to provide three distinct functions: 1) a method of sampling from the Herbrand base which can be used to provide selected targets or example sets for ILP experiments, 2) a measure of the information content of examples or hypotheses; this can be used to guide the search in an ILP system and 3) a simple method for conditioning a given stochastic logic program on samples of data. Functions 1) and 3) are used to measure the generality of hypotheses in the ILP system Progol4.2. This supports an implementation of a Bayesian technique for learning from positive examples only. fl This paper is an extension of a paper with the same title which appeared in [12] Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1105450. 	[Title] A Fast, Bottom-Up Decision Tree Pruning Algorithm with Near-Optimal Generalization [Abstract] In this work, we present a new bottom-up algorithm for decision tree pruning that is very efficient (requiring only a single pass through the given tree), and prove a strong performance guarantee for the generalization error of the resulting pruned tree. We work in the typical setting in which the given tree T may have been derived from the given training sample S, and thus may badly overfit S. In this setting, we give bounds on the amount of additional generalization error that our pruning suffers compared to the optimal pruning of T . More generally, our results show that if there is a pruning of T with small error, and whose size is small compared to jSj, then our algorithm will find a pruning whose error is not much larger. This style of result has been called an index of resolvability result by Barron and Cover in the context of density estimation. A novel feature of our algorithm is its locality | the decision to prune a subtree is based entirely on properties of that subtree and the sample reaching it. To analyze our algorithm, we develop tools of local uniform convergence, a generalization of the standard notion that may prove useful in other settings. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1118764. 	[Title] An Efficient Extension to Mixture Techniques for Prediction and Decision Trees [Abstract] We present a method for maintaining mixtures of prunings of a prediction or decision tree that extends the "node-based" prunings of [Bun90, WST95, HS97] to the larger class of edge-based prunings. The method includes an efficient online weight allocation algorithm that can be used for prediction, compression and classification. Although the set of edge-based prunings of a given tree is much larger than that of node-based prunings, our algorithm has similar space and time complexity to that of previous mixture algorithms for trees. Using the general on-line framework of Freund and Schapire [FS97], we prove that our algorithm maintains correctly the mixture weights for edge-based prunings with any bounded loss function. We also give a similar algorithm for the logarithmic loss function with a corresponding weight allocation algorithm. Finally, we describe experiments comparing node-based and edge-based mixture models for estimating the probability of the next word in English text, which show the ad vantages of edge-based models. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #5038. 	[Title] Using and combining predictors that specialize [Abstract] We study online learning algorithms that predict by combining the predictions of several subordinate prediction algorithms, sometimes called experts. These simple algorithms belong to the multiplicative weights family of algorithms. The performance of these algorithms degrades only logarithmically with the number of experts, making them particularly useful in applications where the number of experts is very large. However, in applications such as text categorization, it is often natural for some of the experts to abstain from making predictions on some of the instances. We show how to transform algorithms that assume that all experts are always awake to algorithms that do not require this assumption. We also show how to derive corresponding loss bounds. Our method is very general, and can be applied to a large family of online learning algorithms. We also give applications to various prediction models including decision graphs and switching experts. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #368431. 	[Title] Decision Tree Induction: How Effective is the Greedy Heuristic? [Abstract] Most existing decision tree systems use a greedy approach to induce trees | locally optimal splits are induced at every node of the tree. Although the greedy approach is suboptimal, it is believed to produce reasonably good trees. In the current work, we attempt to verify this belief. We quantify the goodness of greedy tree induction empirically, using the popular decision tree algorithms, C4.5 and CART. We induce decision trees on thousands of synthetic data sets and compare them to the corresponding optimal trees, which in turn are found using a novel map coloring idea. We measure the effect on greedy induction of variables such as the underlying concept complexity, training set size, noise and dimensionality. Our experiments show, among other things, that the expected classification cost of a greedily induced tree is consistently very close to that of the optimal tree. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #30895. 	[Title] Induction of Oblique Decision Trees [Abstract] This paper introduces a randomized technique for partitioning examples using oblique hyperplanes. Standard decision tree techniques, such as ID3 and its descendants, partition a set of points with axis-parallel hyper-planes. Our method, by contrast, attempts to find hyperplanes at any orientation. The purpose of this more general technique is to find smaller but equally accurate decision trees than those created by other methods. We have tested our algorithm on both real and simulated data, and found that in some cases it produces surprisingly small trees without losing predictive accuracy. Small trees allow us, in turn, to obtain simple qualitative descriptions of each problem domain. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1153275. 	[Title]  [Abstract] Tech Report 4-94 Department of Statistics, Open University, Walton Hall, MK7 6AA, UK Tech Report 205 Department of Computer Science, Monash University, Clayton, Vic. 3168, Australia Abstract: This paper examines the minimum encoding approaches to inference, Minimum Message Length (MML) and Minimum Description Length (MDL). This paper was written with the objective of providing an introduction to this area for statisticians. We describe coding techniques for data, and examine how these techniques can be applied to perform inference and model selection. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #101811. 	[Title] Programming Research Group A LEARNABILITY MODEL FOR UNIVERSAL REPRESENTATIONS [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #84020. 	[Title] Classifiers: A Theoretical and Empirical Study [Abstract] This paper describes how a competitive tree learning algorithm can be derived from first principles. The algorithm approximates the Bayesian decision theoretic solution to the learning task. Comparative experiments with the algorithm and the several mature AI and statistical families of tree learning algorithms currently in use show the derived Bayesian algorithm is consistently as good or better, although sometimes at computational cost. Using the same strategy, we can design algorithms for many other supervised and model learning tasks given just a probabilistic representation for the kind of knowledge to be learned. As an illustration, a second learning algorithm is derived for learning Bayesian networks from data. Implications to incremental learning and the use of multiple models are also discussed. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #49844. 	[Title] A Comparison of Pruning Methods for Relational Concept Learning [Abstract] Pre-Pruning and Post-Pruning are two standard methods of dealing with noise in concept learning. Pre-Pruning methods are very efficient, while Post-Pruning methods typically are more accurate, but much slower, because they have to generate an overly specific concept description first. We have experimented with a variety of pruning methods, including two new methods that try to combine and integrate pre- and post-pruning in order to achieve both accuracy and efficiency. This is verified with test series in a chess position classification task. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #49843. 	[Title] Fossil: A Robust Relational Learner [Abstract] The research reported in this paper describes Fossil, an ILP system that uses a search heuristic based on statistical correlation. This algorithm implements a new method for learning useful concepts in the presence of noise. In contrast to Foil's stopping criterion, which allows theories to grow in complexity as the size of the training sets increases, we propose a new stopping criterion that is independent of the number of training examples. Instead, Fossil's stopping criterion depends on a search heuristic that estimates the utility of literals on a uniform scale. In addition we outline how this feature can be used for top-down pruning and present some preliminary results. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #49847. 	[Title] Top-Down Pruning in Relational Learning [Abstract] Pruning is an effective method for dealing with noise in Machine Learning. Recently pruning algorithms, in particular Reduced Error Pruning, have also attracted interest in the field of Inductive Logic Programming. However, it has been shown that these methods can be very inefficient, because most of the time is wasted for generating clauses that explain noisy examples and subsequently pruning these clauses. We introduce a new method which searches for good theories in a top-down fashion to get a better starting point for the pruning algorithm. Experiments show that this approach can significantly lower the complexity of the task without losing predictive accuracy. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #85688. 	[Title] Finding new rules for incomplete theories: Explicit biases for induction with contextual information. In Proceedings [Abstract] addressed in KBANN (which translates a theory into a neural-net, refines it using backpropagation, and then retranslates the result back into rules) by adding extra hidden units and connections to the initial network; however, this would require predetermining the num In this paper, we have presented constructive induction techniques recently added to the EITHER theory refinement system. Intermediate concept utilization employs existing rules in the theory to derive higher-level features for use in induction. Intermediate concept creation employs inverse resolution to introduce new intermediate concepts in order to fill gaps in a theory than span multiple levels. These revisions allow EITHER to make use of imperfect domain theories in the ways typical of previous work in both constructive induction and theory refinement. As a result, EITHER is able to handle a wider range of theory imperfections than any other existing theory refinement system. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #62389. 	[Title] New Roles for Machine Learning in Design for Design of Educational Computing New roles for [Abstract] Research on machine learning in design has concentrated on the use and development of techniques that can solve simple well-defined problems. Invariably, this effort, while important at the early stages of the development of the field, cannot scale up to address real design problems since all existing techniques are based on simplifying assumptions that do not hold for real design. In particular they do not address the dependence on context and multiple, often conflicting, interests that are constitutive of design. This paper analyzes the present situation and criticizes a number of prevailing views. Subsequently, the paper offers an alternative approach whose goal is to advance the use of machine learning in design practice. The approach is partially integrated into a modeling system called n-dim. The use of machine learning in n-dim is presented and open research issues are outlined. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #23738. 	[Title] A THEORY OF LEARNING CLASSIFICATION RULES [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #73146. 	[Title] LEARNING BAYESIAN NETWORKS WITH LOCAL STRUCTURE [Abstract] We examine a novel addition to the known methods for learning Bayesian networks from data that improves the quality of the learned networks. Our approach explicitly represents and learns the local structure in the conditional probability distributions (CPDs) that quantify these networks. This increases the space of possible models, enabling the representation of CPDs with a variable number of parameters. The resulting learning procedure induces models that better emulate the interactions present in the data. We describe the theoretical foundations and practical aspects of learning local structures and provide an empirical evaluation of the proposed learning procedure. This evaluation indicates that learning curves characterizing this procedure converge faster, in the number of training instances, than those of the standard procedure, which ignores the local structure of the CPDs. Our results also show that networks learned with local structures tend to be more complex (in terms of arcs), yet require fewer parameters. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #27895. 	[Title] Bibliography "SMART: Support Management Automated Reasoning Technology for COMPAQ Customer Service," "Instance-Based Learning Algorithms," Machine [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #11325. 	[Title] On Learning Multiple Descriptions of a Concept [Abstract] In sparse data environments, greater classification accuracy can be achieved by learning several concept descriptions of the data and combining their classifications. Stochastic search is a general tool which can be used to generate many good concept descriptions (rule sets) for each class in the data. Bayesian probability theory offers an optimal strategy for combining classifications of the individual concept descriptions, and here we use an approximation of that theory. This strategy is most useful when additional data is difficult to obtain and every increase in classification accuracy is important. The primary result of this paper is that multiple concept descriptions are particularly helpful in "flat" hypothesis spaces in which there are many equally good ways to grow a rule, each having similar gain. Another result is experimental evidence that learning multiple rule sets yields more accurate classifications than learning multiple rules for some domains. To demonstrate these behaviors, we learn multiple concept descriptions by adapting HYDRA, a noise-tolerant relational learning algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #91581. 	[Title] Finding Overlapping Distributions with MML [Abstract] This paper considers an aspect of mixture modelling. Significantly overlapping distributions require more data for their parameters to be accurately estimated than well separated distributions. For example, two Gaussian distributions are considered to significantly overlap when their means are within three standard deviations of each other. If insufficient data is available, only a single component distribution will be estimated, although the data originates from two component distributions. We consider how much data is required to distinguish two component distributions from one distribution in mixture modelling using the minimum message length (MML) criterion. First, we perform experiments which show the MML criterion performs well relative to other Bayesian criteria. Second, we make two improvements to the existing MML estimates, that improve its performance with overlapping distributions. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #79809. 	[Title] MDL and MML Similarities and Differences (Introduction to Minimum Encoding Inference Part III) [Abstract] Tech Report 207 Department of Computer Science, Monash University, Clayton, Vic. 3168, Australia Abstract: This paper continues the introduction to minimum encoding inductive inference given by Oliver and Hand. This series of papers was written with the objective of providing an introduction to this area for statisticians. We describe the message length estimates used in Wallace's Minimum Message Length (MML) inference and Rissanen's Minimum Description Length (MDL) inference. The differences in the message length estimates of the two approaches are explained. The implications of these differences for applications are discussed. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #38722. 	[Title] Learning Probabilistic Automata with Variable Memory Length [Abstract] We propose and analyze a distribution learning algorithm for variable memory length Markov processes. These processes can be described by a subclass of probabilistic finite automata which we name Probabilistic Finite Suffix Automata. The learning algorithm is motivated by real applications in man-machine interaction such as handwriting and speech recognition. Conventionally used fixed memory Markov and hidden Markov models have either severe practical or theoretical drawbacks. Though general hardness results are known for learning distributions generated by sources with similar structure, we prove that our algorithm can indeed efficiently learn distributions generated by our more restricted sources. In Particular, we show that the KL-divergence between the distribution generated by the target source and the distribution generated by our hypothesis can be made small with high confidence in polynomial time and sample complexity. We demonstrate the applicability of our algorithm by learning the structure of natural English text and using our hy pothesis for the correction of corrupted text. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #79817. 	[Title] Bayesian and Information-Theoretic Priors for Bayesian Network Parameters [Abstract] We consider Bayesian and information-theoretic approaches for determining non-informative prior distributions in a parametric model family. The information-theoretic approaches are based on the recently modified definition of stochastic complexity by Rissanen, and on the Minimum Message Length (MML) approach by Wallace. The Bayesian alternatives include the uniform prior, and the equivalent sample size priors. In order to be able to empirically compare the different approaches in practice, the methods are instantiated for a model family of practical importance, the family of Bayesian networks. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1153866. 	[Title] Learning Decision Trees from Decision Rules: [Abstract] A method and initial results from a comparative study ABSTRACT A standard approach to determining decision trees is to learn them from examples. A disadvantage of this approach is that once a decision tree is learned, it is difficult to modify it to suit different decision making situations. Such problems arise, for example, when an attribute assigned to some node cannot be measured, or there is a significant change in the costs of measuring attributes or in the frequency distribution of events from different decision classes. An attractive approach to resolving this problem is to learn and store knowledge in the form of decision rules, and to generate from them, whenever needed, a decision tree that is most suitable in a given situation. An additional advantage of such an approach is that it facilitates building compact decision trees , which can be much simpler than the logically equivalent conventional decision trees (by compact trees are meant decision trees that may contain branches assigned a set of values , and nodes assigned derived attributes, i.e., attributes that are logical or mathematical functions of the original ones). The paper describes an efficient method, AQDT-1, that takes decision rules generated by an AQ-type learning system (AQ15 or AQ17), and builds from them a decision tree optimizing a given optimality criterion. The method can work in two modes: the standard mode , which produces conventional decision trees, and compact mode, which produces compact decision trees. The preliminary experiments with AQDT-1 have shown that the decision trees generated by it from decision rules (conventional and compact) have outperformed those generated from examples by the well-known C4.5 program both in terms of their simplicity and their predictive accuracy. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1131466. 	[Title] Stochastic Complexity Based Estimation of Missing Elements in Questionnaire Data [Abstract] In this paper we study a new information-theoretically justified approach to missing data estimation for multivariate categorical data. The approach discussed is a model-based imputation procedure relative to a model class (i.e., a functional form for the probability distribution of the complete data matrix), which in our case is the set of multinomial models with some independence assumptions. Based on the given model class assumption an information-theoretic criterion can be derived to select between the different complete data matrices. Intuitively this general criterion, called stochastic complexity, represents the shortest code length needed for coding the complete data matrix relative to the model class chosen. Using this information-theoretic criteria, the missing data problem is reduced to a search problem, i.e., finding the data completion with minimal stochastic complexity. In the experimental part of the paper we present empirical results of the approach using two real data sets, and compare these results to those achived by commonly used techniques such as case deletion and imputating sample averages. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #463825. 	[Title] On the Induction of Intelligible Ensembles [Abstract] Ensembles of classifiers, e.g. decision trees, often exhibit greater predictive accuracy than single classifiers alone. Bagging and boosting are two standard ways of generating and combining multiple classifiers. Unfortunately, the increase in predictive performance is usually linked to a dramatic decrease in intelligibility: ensembles are more or less black boxes comparable to neural networks. So far attempts at pruning of ensembles have not been very successful, approximately reducing ensembles into half. This paper describes a different approach which both tries to keep ensemble-sizes small during induction already and also limits the complexity of single classifiers rigorously. Single classifiers are decision-stumps of a prespecified maximal depth. They are combined by majority voting. Ensembles are induced and pruned by a simple hill-climbing procedure. These ensembles can reasonably be transformed into equivalent decision trees. We conduct some empirical evaluation to investigate both predictive accuracies and classifier complexities. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #358866. 	[Title] BAYESIAN ESTIMATION OF THE VON MISES CONCENTRATION PARAMETER [Abstract] The von Mises distribution is a maximum entropy distribution. It corresponds to the distribution of an angle of a compass needle in a uniform magnetic field of direction, , with concentration parameter, . The concentration parameter, , is the ratio of the field strength to the temperature of thermal fluctuations. Previously, we obtained a Bayesian estimator for the von Mises distribution parameters using the information-theoretic Minimum Message Length (MML) principle. Here, we examine a variety of Bayesian estimation techniques by examining the posterior distribution in both polar and Cartesian co-ordinates. We compare the MML estimator with these fellow Bayesian techniques, and a range of Classical estimators. We find that the Bayesian estimators outperform the Classical estimators. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1128531. 	[Title] LEARNING FOR DECISION MAKING: The FRD Approach and a Comparative Study  Machine Learning and Inference Laboratory [Abstract] This paper concerns the issue of what is the best form for learning, representing and using knowledge for decision making. The proposed answer is that such knowledge should be learned and represented in a declarative form. When needed for decision making, it should be efficiently transferred to a procedural form that is tailored to the specific decision making situation. Such an approach combines advantages of the declarative representation, which facilitates learning and incremental knowledge modification, and the procedural representation, which facilitates the use of knowledge for decision making. This approach also allows one to determine decision structures that may avoid attributes that unavailable or difficult to measure in any given situation. Experimental investigations of the system, FRD-1, have demonstrated that decision structures obtained via the declarative route often have not only higher predictive accuracy but are also are simpler than those learned directly from facts. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #411092. 	[Title] LEARNING LOGICAL EXCEPTIONS IN CHESS [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2653. 	[Title] A decision-theoretic generalization of on-line learning and an application to boosting how the weight-update rule [Abstract] We consider the problem of dynamically apportioning resources among a set of options in a worst-case on-line framework. The model we study can be interpreted as a broad, abstract extension of the well-studied on-line prediction model to a general decision-theoretic setting. We show that the multiplicative weight-update rule of Littlestone and Warmuth [10] can be adapted to this model yielding bounds that are slightly weaker in some cases, but applicable to a considerably more general class of learning problems. We show how the resulting learning algorithm can be applied to a variety of problems, including gambling, multiple-outcome prediction, repeated games and prediction of points in R n Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #5086. 	[Title] Approximate Bayes Factors and Accounting for Model Uncertainty in Generalized Linear Models [Abstract] Technical Report no. 255 Department of Statistics, University of Washington August 1993; Revised March 1994 Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #601567. 	[Title] Geometric Comparison of Classifications and Rule Sets* [Abstract] We present a technique for evaluating classifications by geometric comparison of rule sets. Rules are represented as objects in an n-dimensional hyperspace. The similarity of classes is computed from the overlap of the geometric class descriptions. The system produces a correlation matrix that indicates the degree of similarity between each pair of classes. The technique can be applied to classifications generated by different algorithms, with different numbers of classes and different attribute sets. Experimental results from a case study in a medical domain are included. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2654. 	[Title] Experiments with a New Boosting Algorithm [Abstract] In an earlier paper, we introduced a new boosting algorithm called AdaBoost which, theoretically, can be used to significantly reduce the error of any learning algorithm that consistently generates classifiers whose performance is a little better than random guessing. We also introduced the related notion of a pseudo-loss which is a method for forcing a learning algorithm of multi-label concepts to concentrate on the labels that are hardest to discriminate. In this paper, we describe experiments we carried out to assess how well AdaBoost with and without pseudo-loss, performs on real learning problems. We performed two sets of experiments. The first set compared boosting to Breiman's bagging method when used to aggregate various classifiers (including decision trees and single attribute-value tests). We compared the performance of the two methods on a collection of machine-learning benchmarks. In the second set of experiments, we studied in more detail the performance of boosting using a nearest-neighbor classifier on an OCR problem. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #97377. 	[Title] Induction of One-Level Decision Trees [Abstract] In recent years, researchers have made considerable progress on the worst-case analysis of inductive learning tasks, but for theoretical results to have impact on practice, they must deal with the average case. In this paper we present an average-case analysis of a simple algorithm that induces one-level decision trees for concepts defined by a single relevant attribute. Given knowledge about the number of training instances, the number of irrelevant attributes, the amount of class and attribute noise, and the class and attribute distributions, we derive the expected classification accuracy over the entire instance space. We then examine the predictions of this analysis for different settings of these domain parameters, comparing them to exper imental results to check our reasoning. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #309476. 	[Title] Adaptive Mixtures of Probabilistic Transducers [Abstract] We describe and analyze a mixture model for supervised learning of probabilistic transducers. We devise an on-line learning algorithm that efficiently infers the structure and estimates the parameters of each probabilistic transducer in the mixture. Theoretical analysis and comparative simulations indicate that the learning algorithm tracks the best transducer from an arbitrarily large (possibly infinite) pool of models. We also present an application of the model for inducing a noun phrase recognizer. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #358887. 	[Title] SINGLE FACTOR ANALYSIS BY MML ESTIMATION [Abstract] The Minimum Message Length (MML) technique is applied to the problem of estimating the parameters of a multivariate Gaussian model in which the correlation structure is modelled by a single common factor. Implicit estimator equations are derived and compared with those obtained from a Maximum Likelihood (ML) analysis. Unlike ML, the MML estimators remain consistent when used to estimate both the factor loadings and factor scores. Tests on simulated data show the MML estimates to be on av erage more accurate than the ML estimates when the former exist. If the data show little evidence for a factor, the MML estimate collapses. It is shown that the condition for the existence of an MML estimate is essentially that the log likelihood ratio in favour of the factor model exceed the value expected under the null (no-factor) hypotheses. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #55403. 	[Title] Learning Problem-Oriented Decision Structures from Decision Rules: The AQDT-2 System [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1111788. 	[Title] Multivariate Decision Trees [Abstract] COINS Technical Report 92-82 December 1992 Abstract Multivariate decision trees overcome a representational limitation of univariate decision trees: univariate decision trees are restricted to splits of the instance space that are orthogonal to the feature's axis. This paper discusses the following issues for constructing multivariate decision trees: representing a multivariate test, including symbolic and numeric features, learning the coefficients of a multivariate test, selecting the features to include in a test, and pruning of multivariate decision trees. We present some new and review some well-known methods for forming multivariate decision trees. The methods are compared across a variety of learning tasks to assess each method's ability to find concise, accurate decision trees. The results demonstrate that some multivariate methods are more effective than others. In addition, the experiments confirm that allowing multivariate tests improves the accuracy of the resulting decision tree over univariate trees. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #133615. 	[Title] The Estimation of Probabilities in Attribute Selection Measures for Decision Tree Induction [Abstract] In this paper we analyze two well-known measures for attribute selection in decision tree induction, informativity and gini index. In particular, we are interested in the influence of different methods for estimating probabilities on these two measures. The results of experiments show that different measures, which are obtained by different probability estimation methods, determine the preferential order of attributes in a given node. Therefore, they determine the structure of a constructed decision tree. This feature can be very beneficial, especially in real-world applications where several different trees are often required. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6639. 	[Title] Stochastic search in inductive concept learning [Abstract] Concept learning can be viewed as search of the space of concept descriptions. The hypothesis language determines the search space. In standard inductive learning algorithms, the structure of the search space is determined by generalization/specialization operators. Algorithms perform locally optimal search by using a hill-climbing and/or a beam-search strategy. To overcome this limitation, concept learning can be viewed as stochastic search of the space of concept descriptions. The proposed stochastic search method is based on simulated annealing which is known as a successful means for solving combinatorial optimization problems. The stochastic search method, implemented in a rule learning system ATRIS, is based on a compact and efficient representation of the problem and the appropriate operators for structuring the search space. Furthermore, by heuristic pruning of the search space, the method enables also handling of imperfect data. The paper introduces the stochastic search method, describes the ATRIS learning algorithm and gives results of the experiments. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #27249. 	[Title] Machine Learning Bias, Statistical Bias, and Statistical Variance of Decision Tree Algorithms [Abstract] The term "bias" is widely used|and with different meanings|in the fields of machine learning and statistics. This paper clarifies the uses of this term and shows how to measure and visualize the statistical bias and variance of learning algorithms. Statistical bias and variance can be applied to diagnose problems with machine learning bias, and the paper shows four examples of this. Finally, the paper discusses methods of reducing bias and variance. Methods based on voting can reduce variance, and the paper compares Breiman's bagging method and our own tree randomization method for voting decision trees. Both methods uniformly improve performance on data sets from the Irvine repository. Tree randomization yields perfect performance on the Letter Recognition task. A weighted nearest neighbor algorithm based on the infinite bootstrap is also introduced. In general, decision tree algorithms have moderate-to-high variance, so an important implication of this work is that variance|rather than appropriate or inappropriate machine learning bias|is an important cause of poor performance for decision tree algorithms. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #27250. 	[Title] Why Does Bagging Work? A Bayesian Account and its Implications bagging's success, both in a [Abstract] The error rate of decision-tree and other classification learners can often be much reduced by bagging: learning multiple models from bootstrap samples of the database, and combining them by uniform voting. In this paper we empirically test two alternative explanations for this, both based on Bayesian learning theory: (1) bagging works because it is an approximation to the optimal procedure of Bayesian model averaging, with an appropriate implicit prior; (2) bagging works because it effectively shifts the prior to a more appropriate region of model space. All the experimental evidence contradicts the first hypothesis, and confirms the second. Bagging (Breiman 1996a) is a simple and effective way to reduce the error rate of many classification learning algorithms. For example, in the empirical study described below, it reduces the error of a decision-tree learner in 19 of 26 databases, by 4% on average. In the bagging procedure, given a training set of size s, a "bootstrap" replicate of it is constructed by taking s samples with replacement from the training set. Thus a new training set of the same size is produced, where each of the original examples may appear once, more than once, or not. On average, 63% of the original examples will appear in the bootstrap sample. The learning algorithm is then applied to this training set. This procedure is repeated m times, and the resulting m models are aggregated by uniform voting. Bagging is one of several "multiple model" approaches that have recently received much attention (see, for example, (Chan, Stolfo, & Wolpert 1996)). Other procedures of this type include boosting (Freund & Schapire 1996) and stacking (Wolpert 1992). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #65650. 	[Title] Machine Learning 27(1):51-68, 1997. Predicting nearly as well as the best pruning of a decision tree [Abstract] Many algorithms for inferring a decision tree from data involve a two-phase process: First, a very large decision tree is grown which typically ends up "over-fitting" the data. To reduce over-fitting, in the second phase, the tree is pruned using one of a number of available methods. The final tree is then output and used for classification on test data. In this paper, we suggest an alternative approach to the pruning phase. Using a given unpruned decision tree, we present a new method of making predictions on test data, and we prove that our algorithm's performance will not be "much worse" (in a precise technical sense) than the predictions made by the best reasonably small pruning of the given decision tree. Thus, our procedure is guaranteed to be competitive (in terms of the quality of its predictions) with any pruning algorithm. We prove that our procedure is very efficient and highly robust. Our method can be viewed as a synthesis of two previously studied techniques. First, we apply Cesa-Bianchi et al.'s [4] results on predicting using "expert advice" (where we view each pruning as an "expert") to obtain an algorithm that has provably low prediction loss, but that is com-putationally infeasible. Next, we generalize and apply a method developed by Buntine [3], [2] and Willems, Shtarkov and Tjalkens [20], [21] to derive a very efficient implementation of this procedure. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #263279. 	[Title] Data Analyses Using Simulated Breeding and Inductive Learning Methods [Abstract] Marketing decision making tasks require the acquisition of efficient decision rules from noisy questionnaire data. Unlike popular learning-from-example methods, in such tasks, we must interpret the characteristics of the data without clear features of the data nor pre-determined evaluation criteria. The problem is how domain experts get simple, easy-to-understand, and accurate knowledge from noisy data. This paper describes a novel method to acquire efficient decision rules from questionnaire data using both simulated breeding and inductive learning techniques. The basic ideas of the method are that simulated breeding is used to get the effective features from the questionnaire data and that inductive learning is used to acquire simple decision rules from the data. The simulated breeding is one of the Genetic Algorithm based techniques to subjectively or interactively evaluate the qualities of offspring generated by genetic operations. The proposed method has been qualitatively and quantitatively validated by a case study on consumer product questionnaire data: the acquired rules are simpler than the results from the direct application of inductive learning; a domain expert admits that they are easy to understand; and they are at the same level on the accuracy compared with the other methods. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #65653. 	[Title] Pessimistic decision tree pruning based on tree size [Abstract] In this work we develop a new criteria to perform pessimistic decision tree pruning. Our method is theoretically sound and is based on theoretical concepts such as uniform convergence and the Vapnik-Chervonenkis dimension. We show that our criteria is very well motivated, from the theory side, and performs very well in practice. The accuracy of the new criteria is comparable to that of the current method used in C4.5. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #31863. 	[Title] Unsupervised Learning Using MML [Abstract] This paper discusses the unsupervised learning problem. An important part of the unsupervised learning problem is determining the number of constituent groups (components or classes) which best describes some data. We apply the Minimum Message Length (MML) criterion to the unsupervised learning problem, modifying an earlier such MML application. We give an empirical comparison of criteria prominent in the literature for estimating the number of components in a data set. We conclude that the Minimum Message Length criterion performs better than the alternatives on the data considered here for unsupervised learning tasks. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1108728. 	[Title] Proceedings of the First International Workshop on Intelligent Adaptive Systems (IAS-95) Constructive Induction-based Learning Agents: [Abstract] This paper introduces a new type of intelligent agent called a constructive induction-based learning agent (CILA). This agent differs from other adaptive agents because it has the ability to not only learn how to assist a user in some task, but also to incrementally adapt its knowledge representation space to better fit the given learning task. The agents ability to autonomously make problem-oriented modifications to the originally given representation space is due to its constructive induction (CI) learning method. Selective induction (SI) learning methods, and agents based on these methods, rely on a good representation space. A good representation space has no misclassification noise, inter-correlated attributes or irrelevant attributes. Our proposed CILA has methods for overcoming all of these problems. In agent domains with poor representations, the CI-based learning agent will learn more accurate rules and be more useful than an SI-based learning agent. This paper gives an architecture for a CI-based learning agent and gives an empirical comparison of a CI and SI for a set of six abstract domains involving DNF-type (disjunctive normal form) descriptions. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1121659. 	[Title] Causal Discovery via MML [Abstract] Automating the learning of causal models from sample data is a key step toward incorporating machine learning in the automation of decision-making and reasoning under uncertainty. This paper presents a Bayesian approach to the discovery of causal models, using a Minimum Message Length (MML) method. We have developed encoding and search methods for discovering linear causal models. The initial experimental results presented in this paper show that the MML induction approach can recover causal models from generated data which are quite accurate reflections of the original models; our results compare favorably with those of the TETRAD II program of Spirtes et al. [25] even when their algorithm is supplied with prior temporal information and MML is not. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1153148. 	[Title] Learning Classification Trees [Abstract] Algorithms for learning classification trees have had successes in artificial intelligence and statistics over many years. This paper outlines how a tree learning algorithm can be derived using Bayesian statistics. This introduces Bayesian techniques for splitting, smoothing, and tree averaging. The splitting rule is similar to Quinlan's information gain, while smoothing and averaging replace pruning. Comparative experiments with reimplementations of a minimum encoding approach, Quinlan's C4 (Quinlan et al., 1987) and Breiman et al.'s CART (Breiman et al., 1984) show the full Bayesian algorithm can produce Publication: This paper is a final draft submitted for publication to the Statistics and Computing journal; a version with some minor changes appeared in Volume 2, 1992, pages 63-73. more accurate predictions than versions of these other approaches, though pay a computational price. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6782. 	[Title] Oblivious Decision Trees, Graphs, and Top-Down Pruning [Abstract] We describe a supervised learning algorithm, EODG, that uses mutual information to build an oblivious decision tree. The tree is then converted to an Oblivious read-Once Decision Graph (OODG) by merging nodes at the same level of the tree. For domains that are appropriate for both decision trees and OODGs, performance is approximately the same as that of C4.5, but the number of nodes in the OODG is much smaller. The merging phase that converts the oblivious decision tree to an OODG provides a new way of dealing with the replication problem and a new pruning mechanism that works top down starting from the root. The pruning mechanism is well suited for finding symmetries and aids in recovering from splits on irrelevant features that may happen during the tree construction. Please summarise all the given information.

Paper #11342 has the title `On Pruning and Averaging Decision Trees  
` and the abstract `Pruning a decision tree is considered by some researchers to be the most important part of tree building in noisy domains. While, there are many approaches to pruning, an alternative approach of averaging over decision trees has not received as much attention. We perform an empirical comparison of pruning with the approach of averaging over decision trees. For this comparison we use a computa-tionally efficient method of averaging, namely averaging over the extended fanned set of a tree. Since there are a wide range of approaches to pruning, we compare tree averaging with a traditional pruning approach, along with an optimal pruning approach.
`.
Question: Which category should Paper #11342 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #11342 has the title `On Pruning and Averaging Decision Trees  
` and the abstract `Pruning a decision tree is considered by some researchers to be the most important part of tree building in noisy domains. While, there are many approaches to pruning, an alternative approach of averaging over decision trees has not received as much attention. We perform an empirical comparison of pruning with the approach of averaging over decision trees. For this comparison we use a computa-tionally efficient method of averaging, namely averaging over the extended fanned set of a tree. Since there are a wide range of approaches to pruning, we compare tree averaging with a traditional pruning approach, along with an optimal pruning approach.
`.
Question: Which category should Paper #11342 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Theory