Above are papers related to the following paper: [Paper_ID] Paper #4330. 	[Title] Hierarchical Mixtures of Experts and the EM Algorithm [Abstract] We present a tree-structured architecture for supervised learning. The statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models (GLIM's). Learning is treated as a maximum likelihood problem; in particular, we present an Expectation-Maximization (EM) algorithm for adjusting the parameters of the architecture. We also develop an on-line learning algorithm in which the parameters are updated incrementally. Comparative simulation results are presented in the robot dynamics domain. This report describes research done at the Dept. of Brain and Cognitive Sciences, the Center for Biological and Computational Learning, and the Artificial Intelligence Laboratory of the Massachusetts Institute of Technology. Support for CBCL is provided in part by a grant from the NSF (ASC-9217041). Support for the laboratory's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Dept. of Defense. The authors were supported by a grant from the McDonnell-Pew Foundation, by a grant from ATR Human Information Processing Research Laboratories, by a grant from Siemens Corporation, by by grant IRI-9013991 from the National Science Foundation, by grant N00014-90-J-1942 from the Office of Naval Research, and by NSF grant ECS-9216531 to support an Initiative in Intelligent Control at MIT. Michael I. Jordan is a NSF Presidential Young Investigator. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1135108. 	[Title] A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian [Abstract] We describe the maximum-likelihood parameter estimation problem and how the Expectation-Maximization (EM) algorithm can be used for its solution. We first describe the abstract form of the EM algorithm as it is often given in the literature. We then develop the EM parameter estimation procedure for two applications: 1) finding the parameters of a mixture of Gaussian densities, and 2) finding the parameters of a hidden Markov model (HMM) (i.e., the Baum-Welch algorithm) for both discrete and Gaussian mixture observation models. We derive the update equations in fairly explicit detail but we do not prove any convergence properties. We try to emphasize intuition rather than mathematical rigor. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1120786. 	[Title] Trees and Splines in Survival Analysis [Abstract] Technical Report No. 275 Revised March 30, 1995 University of Washington Department of Statistics Seattle, Washington 98195 Abstract During the past few years several nonparametric alternatives to the Cox proportional hazards model have appeared in the literature. These methods extend techniques that are well known from regression analysis to the analysis of censored survival data. In this paper we discuss methods based on (partition) trees and (polynomial) splines, analyze two datasets using both Survival Trees[1] and HARE[2], and compare the strengths and weaknesses of the two methods. One of the strengths of HARE is that its model fitting procedure has an implicit check for proportionality of the underlying hazards model. It also provides an explicit model for the conditional hazards function, which makes it very convenient to obtain graphical summaries. On the other hand, the tree-based methods automatically partition a dataset into groups of cases that are similar in survival history. Results obtained by survival trees and HARE are often complimentary. Trees and splines in survival analysis should provide the data analyst with two useful tools when analyzing survival data. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #260121. 	[Title] The wake-sleep algorithm for unsupervised neural networks [Abstract] We describe an unsupervised learning algorithm for a multilayer network of stochastic neurons. Bottom-up recognition connections convert the input into representations in successive hidden layers and top-down generative connections reconstruct the representation in one layer from the representation in the layer above. In the wake phase, neurons are driven by recognition connections, and generative connections are adapted to increase the probability that they would reconstruct the correct activity vector in the layer below. In the sleep phase, neurons are driven by generative connections and recognition connections are adapted to increase the probability that they would produce Supervised learning algorithms for multilayer neural networks face two problems: They require a teacher to specify the desired output of the network and they require some method of communicating error information to all of the connections. The wake-sleep algorithm finesses both these problems. When there is no teaching signal to be matched, some other goal is required to force the hidden units to extract underlying structure. In the wake-sleep algorithm the goal is to learn representations that are economical to describe but allow the input to be reconstructed accurately. Each input vector could be communicated to a receiver by first sending its hidden representation and then sending the difference between the input vector and its top-down reconstruction from the hidden representation. The aim of learning is to minimize the description length which is the total number of bits that would be required to communicate the input vectors in this way [1]. No communication actually takes place, but minimizing the description length that would be required forces the network to learn economical representations that capture the underlying regularities in the data [2]. the correct activity vector in the layer above. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #671269. 	[Title] Predicting Conditional Probability Distributions: A Connectionist Approach [Abstract] Most traditional prediction techniques deliver the mean of the probability distribution (a single point). For multimodal processes, instead of predicting the mean of the probability distribution, it is important to predict the full distribution. This article presents a new connectionist method to predict the conditional probability distribution in response to an input. The main idea is to transform the problem from a regression to a classification problem. The conditional probability distribution network can perform both direct predictions and iterated predictions, a task which is specific for time series problems. We compare our method to fuzzy logic and discuss important differences, and also demonstrate the architecture on two time series. The first is the benchmark laser series used in the Santa Fe competition, a deterministic chaotic system. The second is a time series from a Markov process which exhibits structure on two time scales. The network produces multimodal predictions for this series. We compare the predictions of the network with a nearest-neighbor predictor and find that the conditional probability network is more than twice as likely a model. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #20528. 	[Title] Efficient Locally Weighted Polynomial Regression Predictions [Abstract] Locally weighted polynomial regression (LWPR) is a popular instance-based algorithm for learning continuous non-linear mappings. For more than two or three inputs and for more than a few thousand dat-apoints the computational expense of predictions is daunting. We discuss drawbacks with previous approaches to dealing with this problem, and present a new algorithm based on a multiresolution search of a quickly-constructible augmented kd-tree. Without needing to rebuild the tree, we can make fast predictions with arbitrary local weighting functions, arbitrary kernel widths and arbitrary queries. The paper begins with a new, faster, algorithm for exact LWPR predictions. Next we introduce an approximation that achieves up to a two-orders-of-magnitude speedup with negligible accuracy losses. Increasing a certain approximation parameter achieves greater speedups still, but with a correspondingly larger accuracy degradation. This is nevertheless useful during operations such as the early stages of model selection and locating optima of a fitted surface. We also show how the approximations can permit real-time query-specific optimization of the kernel width. We conclude with a brief discussion of potential extensions for tractable instance-based learning on datasets that are too large to fit in a com puter's main memory. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6209. 	[Title] Transfer of Learning by Composing Solutions of Elemental Sequential Tasks [Abstract] Although building sophisticated learning agents that operate in complex environments will require learning to perform multiple tasks, most applications of reinforcement learning have focussed on single tasks. In this paper I consider a class of sequential decision tasks (SDTs), called composite sequential decision tasks, formed by temporally concatenating a number of elemental sequential decision tasks. Elemental SDTs cannot be decomposed into simpler SDTs. I consider a learning agent that has to learn to solve a set of elemental and composite SDTs. I assume that the structure of the composite tasks is unknown to the learning agent. The straightforward application of reinforcement learning to multiple tasks requires learning the tasks separately, which can waste computational resources, both memory and time. I present a new learning algorithm and a modular architecture that learns the decomposition of composite SDTs, and achieves transfer of learning by sharing the solutions of elemental SDTs across multiple composite SDTs. The solution of a composite SDT is constructed by computationally inexpensive modifications of the solutions of its constituent elemental SDTs. I provide a proof of one aspect of the learning algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #428610. 	[Title] Non-linear Models for Time Series Using Mixtures of Experts [Abstract] We consider a novel non-linear model for time series analysis. The study of this model emphasizes both theoretical aspects as well as practical applicability. The architecture of the model is demonstrated to be sufficiently rich, in the sense of approximating unknown functional forms, yet it retains some of the simple and intuitive characteristics of linear models. A comparison to some more established non-linear models will be emphasized, and theoretical issues are backed by prediction results for benchmark time series, as well as computer generated data sets. Efficient estimation algorithms are seen to be applicable, made possible by the mixture based structure of the model. Large sample properties of the estimators are discussed as well, in both well specified as well as misspecified settings. We also demonstrate how inference pertaining to the data structure may be made from the parameterization of the model, resulting in a better, more intuitive, understanding of the structure and performance of the model. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #16451. 	[Title] Using Recurrent Neural Networks to Learn the Structure of Interconnection Networks [Abstract] A modified Recurrent Neural Network (RNN) is used to learn a Self-Routing Interconnection Network (SRIN) from a set of routing examples. The RNN is modified so that it has several distinct initial states. This is equivalent to a single RNN learning multiple different synchronous sequential machines. We define such a sequential machine structure as augmented and show that a SRIN is essentially an Augmented Synchronous Sequential Machine (ASSM). As an example, we learn a small six-switch SRIN. After training we extract the net-work's internal representation of the ASSM and corresponding SRIN. fl This paper is adapted from ( Goudreau, 1993, Chapter 6 ) . A shortened version of this paper was published in ( Goudreau & Giles, 1993 ) . Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1104449. 	[Title] Using Neural Networks for Descriptive Statistical Analysis of Educational Data [Abstract] In this paper we discuss the methodological issues of using a class of neural networks called Mixture Density Networks (MDN) for discriminant analysis. MDN models have the advantage of having a rigorous probabilistic interpretation, and they have proven to be a viable alternative as a classification procedure in discrete domains. We will address both the classification and interpretive aspects of discriminant analysis, and compare the approach to the traditional method of linear discrimin- ants as implemented in standard statistical packages. We show that the MDN approach adopted performs well in both aspects. Many of the observations made are not restricted to the particular case at hand, and are applicable to most applications of discriminant analysis in educational research. fl URL: http://www.cs.Helsinki.FI/research/cosco/ Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6213. 	[Title] Machine Learning Learning to Predict by the Methods of Temporal Differences Keywords: Incremental learning, prediction, [Abstract] This article introduces a class of incremental learning procedures specialized for prediction|that is, for using past experience with an incompletely known system to predict its future behavior. Whereas conventional prediction-learning methods assign credit by means of the difference between predicted and actual outcomes, the new methods assign credit by means of the difference between temporally successive predictions. Although such temporal-difference methods have been used in Samuel's checker player, Holland's bucket brigade, and the author's Adaptive Heuristic Critic, they have remained poorly understood. Here we prove their convergence and optimality for special cases and relate them to supervised-learning methods. For most real-world prediction problems, temporal-difference methods require less memory and less peak computation than conventional methods; and they produce more accurate predictions. We argue that most problems to which supervised learning is currently applied are really prediction problems of the sort to which temporal-difference methods can be applied to advantage. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #2654. 	[Title] Experiments with a New Boosting Algorithm [Abstract] In an earlier paper, we introduced a new boosting algorithm called AdaBoost which, theoretically, can be used to significantly reduce the error of any learning algorithm that consistently generates classifiers whose performance is a little better than random guessing. We also introduced the related notion of a pseudo-loss which is a method for forcing a learning algorithm of multi-label concepts to concentrate on the labels that are hardest to discriminate. In this paper, we describe experiments we carried out to assess how well AdaBoost with and without pseudo-loss, performs on real learning problems. We performed two sets of experiments. The first set compared boosting to Breiman's bagging method when used to aggregate various classifiers (including decision trees and single attribute-value tests). We compared the performance of the two methods on a collection of machine-learning benchmarks. In the second set of experiments, we studied in more detail the performance of boosting using a nearest-neighbor classifier on an OCR problem. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #431206. 	[Title] Backfitting in Smoothing Spline ANOVA [Abstract] A scheme to compute smoothing spline ANOVA estimates for large data sets with a (near) tensor-product structure is proposed. Such data sets are common in spatial-temporal analysis and image analysis. This scheme combines backfitting algorithm with iterative imputation algorithm in order to save both computational space and time. The convergence of this algorithm and various ways to further speed it up, such as collapsing component functions and successive over-relaxation, are discussed. Issues related to its application in spatial-temporal analysis are discussed too. An application to a global analysis of historical surface temperature data is described. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #25702. 	[Title] Nonlinear Prediction of Chaotic Time Series Using Support Vector Machines [Abstract] A novel method for regression has been recently proposed by V. Vapnik et al. [8, 9]. The technique, called Support Vector Machine (SVM), is very well founded from the mathematical point of view and seems to provide a new insight in function approximation. We implemented the SVM and tested it on the same data base of chaotic time series that was used in [1] to compare the performances of different approximation techniques, including polynomial and rational approximation, local polynomial techniques, Radial Basis Functions, and Neural Networks. The SVM performs better than the approaches presented in [1]. We also study, for a particular time series, the variability in performance with respect to the few free parameters of SVM. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #753265. 	[Title] Validation of Average Error Rate Over Classifiers [Abstract] We examine methods to estimate the average and variance of test error rates over a set of classifiers. We begin with the process of drawing a classifier at random for each example. Given validation data, the average test error rate can be estimated as if validating a single classifier. Given the test example inputs, the variance can be computed exactly. Next, we consider the process of drawing a classifier at random and using it on all examples. Once again, the expected test error rate can be validated as if validating a single classifier. However, the variance must be estimated by validating all classifers, which yields loose or uncertain bounds. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1105011. 	[Title] On the Connection Between Stochastic Smoothing, Filtering and Estimation with Incomplete Data [Abstract] Connections between stochastic smoothing/filtering and estimation with incomplete data are investigated. It is shown, under the right censoring scheme, that the Kaplan-Meier estimator can be characterized as a moment estimate based on a stochastic filter/smoother(a pseudo-filter/smoother). Motivated by this result, a potentially useful martingale approach for estimation and convergence with incomplete data is proposed: estimators are characterized as pseudo-stochastic smoothers (which sometimes reduce to filters), which are described by a (system of) stochastic integral equation(s); recent results in convergence of stochastic integrals and stochastic differential equations are then applied to address convergence issues. As an illustration, the double censoring problem is revisited under this framework, a closed form estimator is proposed and convergence properties studied. Martingale theory plays a vital role in the entire analysis. This approach is in essence a self-consistency method. 1 Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #3191. 	[Title] Model Selection and Accounting for Model Uncertainty in Linear Regression Models [Abstract] 1 Adrian E. Raftery is Professor of Statistics and Sociology, David Madigan is Assistant Professor of Statistics, and Jennifer Hoeting is a Ph.D. Candidate, all at the Department of Statistics, GN-22, University of Washington, Seattle, WA 98195. The research of Raftery and Hoeting was supported by ONR Contract N-00014-91-J-1074. Madigan's research was partially supported by NSF grant no. DMS 92111627. The authors are grateful to Danika Lew for research assistance. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #646286. 	[Title] Rate of Convergence of the Gibbs Sampler by Gaussian Approximation  SUMMARY [Abstract] In this article we approximate the rate of convergence of the Gibbs sampler by a normal approximation of the target distribution. Based on this approximation, we consider many implementational issues for the Gibbs sampler, e.g., updating strategy, parameterization and blocking. We give theoretical results to justify our approximation and illustrate our methods in a number of realistic examples. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #646289. 	[Title] Bayesian Estimation and Model Choice in Item Response Models [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #137873. 	[Title] Stacked Density Estimation [Abstract] Technical Report No. 97-36, Information and Computer Science Department, University of California, Irvine Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #12946. 	[Title] Learning from incomplete data [Abstract] Real-world learning tasks often involve high-dimensional data sets with complex patterns of missing features. In this paper we review the problem of learning from incomplete data from two statistical perspectives|the likelihood-based and the Bayesian. The goal is two-fold: to place current neural network approaches to missing data within a statistical framework, and to describe a set of algorithms, derived from the likelihood-based framework, that handle clustering, classification, and function approximation from incomplete data in a principled and efficient manner. These algorithms are based on mixture modeling and make two distinct appeals to the Expectation-Maximization (EM) principle (Dempster et al., 1977)|both for the estimation of mixture components and for coping with the missing data. This report describes research done at the Center for Biological and Computational Learning and the Artificial Intelligence Laboratory of the Massachusetts Institute of Technology. Support for the Center is provided in part by a grant from the National Science Foundation under contract ASC-9217041. Support for the laboratory's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Department of Defense. The authors were supported in part by a grant from ATR Auditory and Visual Perception Research Laboratories, by a grant from Siemens Corporation, by grant IRI-9013991 from the National Science Foundation, and by grant N00014-90-J-1942 from the Office of Naval Research. Zoubin Ghahramani was supported by a grant from the McDonnell-Pew Foundation. Michael I. Jordan is a NSF Presidential Young Investigator. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #119956. 	[Title] Function Approximation with Neural Networks and Local Methods: Bias, Variance and Smoothness [Abstract] We review the use of global and local methods for estimating a function mapping R m ) R n from samples of the function containing noise. The relationship between the methods is examined and an empirical comparison is performed using the multi-layer perceptron (MLP) global neural network model, the single nearest-neighbour model, a linear local approximation (LA) model, and the following commonly used datasets: the Mackey-Glass chaotic time series, the Sunspot time series, British English Vowel data, TIMIT speech phonemes, building energy prediction data, and the sonar dataset. We find that the simple local approximation models often outperform the MLP. No criterion such as classification/prediction, size of the training set, dimensionality of the training set, etc. can be used to distinguish whether the MLP or the local approximation method will be superior. However, we find that if we consider histograms of the k-NN density estimates for the training datasets then we can choose the best performing method a priori by selecting local approximation when the spread of the density histogram is large and choosing the MLP otherwise. This result correlates with the hypothesis that the global MLP model is less appropriate when the characteristics of the function to be approximated varies throughout the input space. We discuss the results, the smoothness assumption often made in function approximation, and the bias/variance dilemma. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #3222. 	[Title] Beyond Independence: Conditions for the Optimality of the Simple Bayesian Classifier [Abstract] The simple Bayesian classifier (SBC) is commonly thought to assume that attributes are independent given the class, but this is apparently contradicted by the surprisingly good performance it exhibits in many domains that contain clear attribute dependences. No explanation for this has been proposed so far. In this paper we show that the SBC does not in fact assume attribute independence, and can be optimal even when this assumption is violated by a wide margin. The key to this finding lies in the distinction between classification and probability estimation: correct classification can be achieved even when the probability estimates used contain large errors. We show that the previously-assumed region of optimality of the SBC is a second-order infinitesimal fraction of the actual one. This is followed by the derivation of several necessary and several sufficient conditions for the optimality of the SBC. For example, the SBC is optimal for learning arbitrary conjunctions and disjunctions, even though they violate the independence assumption. The paper also reports empirical evidence of the SBC's competitive performance in domains containing substantial degrees of attribute dependence. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #118424. 	[Title] Routing in Optical Multistage Interconnection Networks: a Neural Network Solution [Abstract] There has been much interest in using optics to implement computer interconnection networks. However, there has been little discussion of any routing methodologies besides those already used in electronics. In this paper, a neural network routing methodology is proposed that can generate control bits for a broad range of optical multistage interconnection networks (OMINs). Though we present no optical implementation of this methodology, we illustrate its control for an optical interconnection network. These OMINs can be used as communication media for distributed computing systems. The routing methodology makes use of an Artificial Neural Network (ANN) that functions as a parallel computer for generating the routes. The neural network routing scheme can be applied to electrical as well as optical interconnection networks. However, since the ANN can be implemented using optics, this routing approach is especially appealing for an optical computing environment. Although the ANN does not always generate the best solution, the parallel nature of the ANN computation may make this routing scheme faster than conventional routing approaches, especially for OMINs that have an irregular structure. Furthermore, the ANN router is fault-tolerant. Results are shown for generating routes in a 16 fi 16, 3-stage OMIN. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1115291. 	[Title] How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis 1 [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #151708. 	[Title] Using the Stochastic Gradient Method to Fit Polychotomous Regression Models [Abstract] Technical Report No. 319 April 7, 1997 University of Washington Department of Statistics Seattle, Washington 98195-4322 Abstract Kooperberg, Bose, and Stone (1997) introduced polyclass, a methodology that uses adaptively selected linear splines and their tensor products to model conditional class probabilities. The authors attempted to develop a methodology that would work well on small and moderate size problems and would scale up to large problems. However, the version of polyclass that was developed for large problems was impractical in that it required two months of cpu time to apply it to a large data set. A modification to this methodology involving the use of the stochastic gradient (on-line) method in fitting polyclass models to given sets of basis functions is developed here that makes the methodology applicable to large data sets. In particular, it is successfully applied to a phoneme recognition problem involving 45 phonemes, 81 features, 150,000 cases in the training sample, 1000 basis functions, and 44,000 unknown parameters. Comparisons with neural networks are made both on the original problem and on a three-vowel subproblem. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #45212. 	[Title] Gain Adaptation Beats Least Squares? [Abstract] I present computational results suggesting that gain-adaptation algorithms based in part on connectionist learning methods may improve over least squares and other classical parameter-estimation methods for stochastic time-varying linear systems. The new algorithms are evaluated with respect to classical methods along three dimensions: asymptotic error, computational complexity, and required prior knowledge about the system. The new algorithms are all of the same order of complexity as LMS methods, O(n), where n is the dimensionality of the system, whereas least-squares methods and the Kalman filter are O(n 2 ). The new methods also improve over the Kalman filter in that they do not require a complete statistical model of how the system varies over time. In a simple computational experiment, the new methods are shown to produce asymptotic error levels near that of the optimal Kalman filter and significantly below those of least-squares and LMS methods. The new methods may perform better even than the Kalman filter if there is any error in the filter's model of how the system varies over time. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #3229. 	[Title] Book Review  Introduction to the Theory of Neural Computation Reviewed by: 2 [Abstract] Neural computation, also called connectionism, parallel distributed processing, neural network modeling or brain-style computation, has grown rapidly in the last decade. Despite this explosion, and ultimately because of impressive applications, there has been a dire need for a concise introduction from a theoretical perspective, analyzing the strengths and weaknesses of connectionist approaches and establishing links to other disciplines, such as statistics or control theory. The Introduction to the Theory of Neural Computation by Hertz, Krogh and Palmer (subsequently referred to as HKP) is written from the perspective of physics, the home discipline of the authors. The book fulfills its mission as an introduction for neural network novices, provided that they have some background in calculus, linear algebra, and statistics. It covers a number of models that are often viewed as disjoint. Critical analyses and fruitful comparisons between these models Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #3231. 	[Title] Irrelevant Features and the Subset Selection Problem [Abstract] We address the problem of finding a subset of features that allows a supervised induction algorithm to induce small high-accuracy concepts. We examine notions of relevance and irrelevance, and show that the definitions used in the machine learning literature do not adequately partition the features into useful categories of relevance. We present definitions for irrelevance and for two degrees of relevance. These definitions improve our understanding of the behavior of previous subset selection algorithms, and help define the subset of features that should be sought. The features selected should depend not only on the features and the target concept, but also on the induction algorithm. We describe a method for feature subset selection using cross-validation that is applicable to any induction algorithm, and discuss experiments conducted with ID3 and C4.5 on artificial and real datasets. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #3233. 	[Title] A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection [Abstract] We review accuracy estimation methods and compare the two most common methods: cross-validation and bootstrap. Recent experimental results on artificial data and theoretical results in restricted settings have shown that for selecting a good classifier from a set of classifiers (model selection), ten-fold cross-validation may be better than the more expensive leave-one-out cross-validation. We report on a large-scale experiment|over half a million runs of C4.5 and a Naive-Bayes algorithm|to estimate the effects of different parameters on these algorithms on real-world datasets. For cross-validation, we vary the number of folds and whether the folds are stratified or not; for bootstrap, we vary the number of bootstrap samples. Our results indicate that for real-word datasets similar to ours, the best method to use for model selection is ten-fold stratified cross validation, even if computation power allows using more folds. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #118435. 	[Title] Predictive Control of Opto-Electronic Reconfigurable Interconnection Networks Using Neural Networks [Abstract] Opto-electronic reconfigurable interconnection networks are limited by significant control latency when used in large multiprocessor systems. This latency is the time required to analyze the current traffic and reconfigure the network to establish the required paths. The goal of latency hiding is to minimize the effect of this control overhead. In this paper, we introduce a technique that performs latency hiding by learning the patterns of communication traffic and using that information to anticipate the need for communication paths. Hence, the network provides the required communication paths before a request for a path is made. In this study, the communication patterns (memory accesses) of a parallel program are used as input to a time delay neural network (TDNN) to perform online training and prediction. These predicted communication patterns are used by the interconnection network controller that provides routes for the memory requests. Based on our experiments, the neural network was able to learn highly repetitive communication patterns, and was thus able to predict the allocation of communication paths, resulting in a reduction of communication latency. Communication latency is a significant issue in the design of lar ge scale multiprocessor systems. Point-to-point interconnection networks, which directly connect all processors and/or memories, provide minimum communication latency but suffer from high cost and limited scalability. A plethora of electr onic singlestage and multistage networks have been pr oposed, designed and built [Siegel90, Leighton93]. An alternative is the use of opto-electr onic reconfigurable interconnection networks which offer a limited number of high bandwidth communication channels configured on demand, to satisfy the r equired communication traffic [CLMQ94b]. A network controller determines the network configuration based on processor requests. Once the controller provides the optical communication paths requested, the communication proceeds at high speeds. Hence, the end-to-end latency incurred by such networks can be characterized by three components: control time, which is the time needed to determine the new network configuration and to physically establish the paths; launch time, the time to transmit the data into the network; and y time, the time needed for the message to travel through the network to its final destination. For high bandwidth short distance networks, the control time dominates the overall Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #118436. 	[Title] Performance of On-Line Learning Methods in Predicting Multiprocessor Memory Access Patterns [Abstract] Technical Report UMIACS-TR-96-59 and CS-TR-3676 Institute for Advanced Computer Studies University of Maryland College Park, MD 20742 Abstract Shared memory multiprocessors require reconfigurable interconnection networks (INs) for scalability. These INs are reconfigured by an IN control unit. However, these INs are often plagued by undesirable reconfiguration time that is primarily due to control latency, the amount of time delay that the control unit takes to decide on a desired new IN configuration. To reduce control latency, a trainable prediction unit (PU) was devised and added to the IN controller. The PUs job is to anticipate and reduce control configuration time, the major component of the control latency. Three different on-line prediction techniques were tested to learn and predict repetitive memory access patterns for three typical parallel processing applications, the 2-D relaxation algorithm, matrix multiply and Fast Fourier Transform. The predictions were then used by a routing control algorithm to reduce control latency by configuring the IN to provide needed memory access paths before they were requested. Three prediction techniques were used and tested: 1). a Markov predictor, 2). a linear predictor and 3). a time delay neural network (TDNN) predictor. As expected, different predictors performed best on different applications, however, the TDNN produced the best overall results. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1116336. 	[Title] On Computing the Largest Fraction of Missing Information for the EM Algorithm and the Worst [Abstract] We address the problem of computing the largest fraction of missing information for the EM algorithm and the worst linear function for data augmentation. These are the largest eigenvalue and its associated eigenvector for the Jacobian of the EM operator at a maximum likelihood estimate, which are important for assessing convergence in iterative simulation. An estimate of the largest fraction of missing information is available from the EM iterates; this is often adequate since only a few figures of accuracy are needed. In some instances the EM iteration also gives an estimate of the worst linear function. We show that the power method for eigencomputation can be used to compute efficient and accurate estimates of both quantities. Unlike eigenvalue decomposition, the power method computes only the largest eigenvalue and eigenvector of a matrix, it can take advantage of a good eigenvector estimate as an initial value and it can be terminated after only a few figures of accuracy are obtained. Moreover, the matrix products needed in the power method can be computed by extrapolation, obviating the need to form the Jacobian of the EM operator. We give results of simultation studies on multivariate normal data showing this approach becomes more efficient as the data dimension increases than methods that use a finite-difference approximation to the Jacobian, which is the only general-purpose alternative available. fl Funded by National Institutes of Health Small Business Innovation Reseach Grant 5R44CA65147-03, and by Office of Naval Research contracts N00014-96-1-0192 and N00014-96-1-0330. We are indebted to Tim Hesterberg, Jim Schimert, Doug Clarkson, Anne Greenbaum, and Adrian Raftery for comments and discussion that helped advance this research and improve this paper. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #328370. 	[Title] Expectation-Based Selective Attention for Visual Monitoring and Control of a Robot Vehicle [Abstract] Reliable vision-based control of an autonomous vehicle requires the ability to focus attention on the important features in an input scene. Previous work with an autonomous lane following system, ALVINN [Pomerleau, 1993], has yielded good results in uncluttered conditions. This paper presents an artificial neural network based learning approach for handling difficult scenes which will confuse the ALVINN system. This work presents a mechanism for achieving task-specific focus of attention by exploiting temporal coherence. A saliency map, which is based upon a computed expectation of the contents of the inputs in the next time step, indicates which regions of the input retina are important for performing the task. The saliency map can be used to accentuate the features which are important for the task, and de-emphasize those which are not. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1717. 	[Title] On Convergence Properties of the EM Algorithm for Gaussian Mixtures [Abstract] We build up the mathematical connection between the "Expectation-Maximization" (EM) algorithm and gradient-based approaches for maximum likelihood learning of finite Gaussian mixtures. We show that the EM step in parameter space is obtained from the gradient via a projection matrix P , and we provide an explicit expression for the matrix. We then analyze the convergence of EM in terms of special properties of P and provide new results analyzing the effect that P has on the likelihood surface. Based on these mathematical results, we present a comparative discussion of the advantages and disadvantages of EM and other algorithms for the learning of Gaussian mixture models. This report describes research done at the Center for Biological and Computational Learning and the Artificial Intelligence Laboratory of the Massachusetts Institute of Technology. Support for the Center is provided in part by a grant from the National Science Foundation under contract ASC-9217041. Support for the laboratory's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Department of Defense under Office of Naval Research contract N00000-00-A-0000. The authors were also supported by the HK RGC Earmarked Grant CUHK250/94E, by a grant from the McDonnell-Pew Foundation, by a grant from ATR Human Information Processing Research Laboratories, by a grant from Siemens Corporation, and by grant N00014-90-1-0777 from the Office of Naval Research. Michael I. Jordan is an NSF Presidential Young Investigator. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #299197. 	[Title] TD Learning of Game Evaluation Functions with Hierarchical Neural Architectures [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #240321. 	[Title] Nonlinear gated experts for time series: discovering regimes and avoiding overfitting [Abstract] In: International Journal of Neural Systems 6 (1995) p. 373 - 399. URL of this paper: ftp://ftp.cs.colorado.edu/pub/Time-Series/MyPapers/experts.ps.Z, or http://www.cs.colorado.edu/~andreas/Time-Series/MyPapers/experts.ps.Z University of Colorado Computer Science Technical Report CU-CS-798-95. In the analysis and prediction of real-world systems, two of the key problems are nonstationarity(often in the form of switching between regimes), and overfitting (particularly serious for noisy processes). This article addresses these problems using gated experts, consisting of a (nonlinear) gating network, and several (also nonlinear) competing experts. Each expert learns to predict the conditional mean, and each expert adapts its width to match the noise level in its regime. The gating network learns to predict the probability of each expert, given the input. This article focuses on the case where the gating network bases its decision on information from the inputs. This can be contrasted to hidden Markov models where the decision is based on the previous state(s) (i.e., on the output of the gating network at the previous time step), as well as to averaging over several predictors. In contrast, gated experts soft-partition the input space. This article discusses the underlying statistical assumptions, derives the weight update rules, and compares the performance of gated experts to standard methods on three time series: (1) a computer-generated series, obtained by randomly switching between two nonlinear processes, (2) a time series from the Santa Fe Time Series Competition (the light intensity of a laser in chaotic state), and (3) the daily electricity demand of France, a real-world multivariate problem with structure on several time scales. The main results are (1) the gating network correctly discovers the different regimes of the process, (2) the widths associated with each expert are important for the segmentation task (and they can be used to characterize the sub-processes), and (3) there is less overfitting compared to single networks (homogeneous multi-layer perceptrons), since the experts learn to match their variances to the (local) noise levels. This can be viewed as matching the local complexity of the model to the local complexity of the data. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #120013. 	[Title] From Isolation to Cooperation: An Alternative View of a System of Experts [Abstract] We introduce a constructive, incremental learning system for regression problems that models data by means of locally linear experts. In contrast to other approaches, the experts are trained independently and do not compete for data during learning. Only when a prediction for a query is required do the experts cooperate by blen ding their individual predictions. Each expert is trained by minimizing a penalized local cross validation error using second order methods. In this way, an expert is able to adjust the size and shape of the receptive field in which its predictions are valid, and also to adjust its bias on the importance of individual input dimensions. The size and shape adjustment corresponds to finding a local distance metric, while the bias adjustment accomplishes local dimensio n-ality reduction. We derive asymptotic results for our method. In a variety of simulations we demonstrate the properties of the algorithm with respect to interference, learning speed, prediction accuracy, feature detection, and task or i-ented incremental learning. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #50381. 	[Title] Training Algorithms for Hidden Markov Models Using Entropy Based Distance Functions [Abstract] We present new algorithms for parameter estimation of HMMs. By adapting a framework used for supervised learning, we construct iterative algorithms that maximize the likelihood of the observations while also attempting to stay close to the current estimated parameters. We use a bound on the relative entropy between the two HMMs as a distance measure between them. The result is new iterative training algorithms which are similar to the EM (Baum-Welch) algorithm for training HMMs. The proposed algorithms are composed of a step similar to the expectation step of Baum-Welch and a new update of the parameters which replaces the maximization (re-estimation) step. The algorithm takes only negligibly more time per iteration and an approximated version uses the same expectation step as Baum-Welch. We evaluate experimentally the new algorithms on synthetic and natural speech pronunciation data. For sparse models, i.e. models with relatively small number of non-zero parameters, the proposed algorithms require significantly fewer iterations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #23258. 	[Title] Exploration and Model Building in Mobile Robot Domains [Abstract] I present first results on COLUMBUS, an autonomous mobile robot. COLUMBUS operates in initially unknown, structured environments. Its task is to explore and model the environment efficiently while avoiding collisions with obstacles. COLUMBUS uses an instance-based learning technique for modeling its environment. Real-world experiences are generalized via two artificial neural networks that encode the characteristics of the robot's sensors, as well as the characteristics of typical environments the robot is assumed to face. Once trained, these networks allow for knowledge transfer across different environments the robot will face over its lifetime. COLUMBUS' models represent both the expected reward and the confidence in these expectations. Exploration is achieved by navigating to low confidence regions. An efficient dynamic programming method is employed in background to find minimal-cost paths that, executed by the robot, maximize exploration. COLUMBUS operates in real-time. It has been operating successfully in an office building environment for periods up to hours. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1131230. 	[Title] Identifiability, Improper Priors and Gibbs Sampling for Generalized Linear Models [Abstract] Alan E. Gelfand is a Professor in the Department of Statistics at the University of Connecti-cut, Storrs, CT 06269. Sujit K. Sahu is a Lecturer at the School of Mathematics, University of Wales, Cardiff, CF2 4YH, UK. The research of the first author was supported in part by NSF grant DMS 9301316 while the second author was supported in part by an EPSRC grant from UK. The authors thank Brad Carlin, Kate Cowles, Gareth Roberts and an anonymous referee for valuable comments. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #4329. 	[Title] Supervised learning from incomplete data via an EM approach [Abstract] Real-world learning tasks may involve high-dimensional data sets with arbitrary patterns of missing data. In this paper we present a framework based on maximum likelihood density estimation for learning from such data sets. We use mixture models for the density estimates and make two distinct appeals to the Expectation-Maximization (EM) principle (Dempster et al., 1977) in deriving a learning algorithm|EM is used both for the estimation of mixture components and for coping with missing data. The resulting algorithm is applicable to a wide range of supervised as well as unsupervised learning problems. Results from a classification benchmark|the iris data set|are presented. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #390889. 	[Title] A HIERARCHICAL COMMUNITY OF EXPERTS [Abstract] We describe a directed acyclic graphical model that contains a hierarchy of linear units and a mechanism for dynamically selecting an appropriate subset of these units to model each observation. The non-linear selection mechanism is a hierarchy of binary units each of which gates the output of one of the linear units. There are no connections from linear units to binary units, so the generative model can be viewed as a logistic belief net (Neal 1992) which selects a skeleton linear model from among the available linear units. We show that Gibbs sampling can be used to learn the parameters of the linear and binary units even when the sampling is so brief that the Markov chain is far from equilibrium. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #4335. 	[Title] A VIEW OF THE EM ALGORITHM THAT JUSTIFIES INCREMENTAL, SPARSE, AND OTHER VARIANTS [Abstract] The EM algorithm performs maximum likelihood estimation for data in which some variables are unobserved. We present a function that resembles negative free energy and show that the M step maximizes this function with respect to the model parameters and the E step maximizes it with respect to the distribution over the unobserved variables. From this perspective, it is easy to justify an incremental variant of the EM algorithm in which the distribution for only one of the unobserved variables is recalculated in each E step. This variant is shown empirically to give faster convergence in a mixture estimation problem. A variant of the algorithm that exploits sparse conditional distributions is also described, and a wide range of other variant algorithms are also seen to be possible. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1114352. 	[Title] Cortical Mechanisms of Visual Recognition and Learning: A Hierarchical Kalman Filter Model [Abstract] We describe a biologically plausible model of dynamic recognition and learning in the visual cortex based on the statistical theory of Kalman filtering from optimal control theory. The model utilizes a hierarchical network whose successive levels implement Kalman filters operating over successively larger spatial and temporal scales. Each hierarchical level in the network predicts the current visual recognition state at a lower level and adapts its own recognition state using the residual error between the prediction and the actual lower-level state. Simultaneously, the network also learns an internal model of the spatiotemporal dynamics of the input stream by adapting the synaptic weights at each hierarchical level in order to minimize prediction errors. The Kalman filter model respects key neuroanatomical data such as the reciprocity of connections between visual cortical areas, and assigns specific computational roles to the inter-laminar connections known to exist between neurons in the visual cortex. Previous work elucidated the usefulness of this model in explaining neurophysiological phenomena such as endstopping and other related extra-classical receptive field effects. In this paper, in addition to providing a more detailed exposition of the model, we present a variety of experimental results demonstrating the ability of this model to perform robust spatiotemporal segmentation and recognition of objects and image sequences in the presence of varying amounts of occlusion, background clutter, and noise. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6898. 	[Title] Sample Size Calculations for Smoothing Splines Based on Bayesian Confidence Intervals [Abstract] Bayesian confidence intervals of a smoothing spline are often used to distinguish two curves. In this paper, we provide an asymptotic formula for sample size calculations based on Bayesian confidence intervals. Approximations and simulations on special functions indicate that this asymptotic formula is reasonably accurate. Key Words: Bayesian confidence intervals; sample size; smoothing spline. fl Address: Department of Statistics and Applied Probability, University of California, Santa Barbara, CA 93106-3110. Tel.: (805)893-4870. Fax: (805)893-2334. E-mail: yuedong@pstat.ucsb.edu. Supported by the National Institute of Health under Grants R01 EY09946, P60 DK20572 and P30 HD18258. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #27895. 	[Title] Bibliography "SMART: Support Management Automated Reasoning Technology for COMPAQ Customer Service," "Instance-Based Learning Algorithms," Machine [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6910. 	[Title] Smoothing Spline ANOVA for Exponential Families, with Application to the Wisconsin Epidemiological Study of Diabetic [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6913. 	[Title] On Convergence of the EM Algorithm and the Gibbs Sampler  SUMMARY [Abstract] In this article we investigate the relationship between the two popular algorithms, the EM algorithm and the Gibbs sampler. We show that the approximate rate of convergence of the Gibbs sampler by Gaussian approximation is equal to that of the corresponding EM type algorithm. This helps in implementing either of the algorithms as improvement strategies for one algorithm can be directly transported to the other. In particular, by running the EM algorithm we know approximately how many iterations are needed for convergence of the Gibbs sampler. We also obtain a result that under conditions, the EM algorithm used for finding the maximum likelihood estimates can be slower to converge than the corresponding Gibbs sampler for Bayesian inference which uses proper prior distributions. We illustrate our results in a number of realistic examples all based on the generalized linear mixed models. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1136393. 	[Title] Data-driven Modeling and Synthesis of Acoustical Instruments [Abstract] We present a framework for the analysis and synthesis of acoustical instruments based on data-driven probabilistic inference modeling. Audio time series and boundary conditions of a played instrument are recorded and the non-linear mapping from the control data into the audio space is inferred using the general inference framework of Cluster-Weighted Modeling. The resulting model is used for real-time synthesis of audio sequences from new input data. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1136397. 	[Title] State Reconstruction for Determining Predictability in Driven Nonlinear Acoustical Systems [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #69392. 	[Title] Forecasting electricity demand using nonlinear mixture of experts [Abstract] In this paper we study a forecasting model based on mixture of experts for predicting the French electric daily consumption energy. We split the task into two parts. Using mixture of experts, a first model predicts the electricity demand from the exogenous variables (such as temperature and degree of cloud cover) and can be viewed as a nonlinear regression model of mixture of Gaussians. Using a single neural network, a second model predicts the evolution of the residual error of the first one, and can be viewed as an nonlinear autoregression model. We analyze the splitting of the input space generated by the mixture of experts model, and compare the performance to models presently used. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #39199. 	[Title] Unsupervised Learning by Convex and Conic Coding [Abstract] Unsupervised learning algorithms based on convex and conic encoders are proposed. The encoders find the closest convex or conic combination of basis vectors to the input. The learning algorithms produce basis vectors that minimize the reconstruction error of the encoders. The convex algorithm develops locally linear models of the input, while the conic algorithm discovers features. Both algorithms are used to model handwritten digits and compared with vector quantization and principal component analysis. The neural network implementations involve feedback connections that project a reconstruction back to the input layer. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #395553. 	[Title] Mixtures of Probabilistic Principal Component Analysers [Abstract] Principal component analysis (PCA) is one of the most popular techniques for processing, compressing and visualising data, although its effectiveness is limited by its global linearity. While nonlinear variants of PCA have been proposed, an alternative paradigm is to capture data complexity by a combination of local linear PCA projections. However, conventional PCA does not correspond to a probability density, and so there is no unique way to combine PCA models. Previous attempts to formulate mixture models for PCA have therefore to some extent been ad hoc. In this paper, PCA is formulated within a maximum-likelihood framework, based on a specific form of Gaussian latent variable model. This leads to a well-defined mixture model for probabilistic principal component analysers, whose parameters can be determined using an EM algorithm. We discuss the advantages of this model in the context of clustering, density modelling and local dimensionality reduction, and we demonstrate its application to image compression and handwritten digit recognition. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #69418. 	[Title] Constructive Algorithms for Hierarchical Mixtures of Experts [Abstract] We present two additions to the hierarchical mixture of experts (HME) architecture. We view the HME as a tree structured classifier. Firstly, by applying a likelihood splitting criteria to each expert in the HME we "grow" the tree adaptively during training. Secondly, by considering only the most probable path through the tree we may "prune" branches away, either temporarily, or permanently if they become redundant. We demonstrate results for the growing and pruning algorithms which show significant speed ups and more efficient use of parameters over the conventional algorithms in discriminating between two interlocking spirals and classifying 8-bit parity patterns. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #116528. 	[Title] Selection of Relevant Features and Examples in Machine Learning [Abstract] In this survey, we review work in machine learning on methods for handling data sets containing large amounts of irrelevant information. We focus on two key issues: the problem of selecting relevant features, and the problem of selecting relevant examples. We describe the advances that have been made on these topics in both empirical and theoretical work in machine learning, and we present a general framework that we use to compare different methods. We close with some challenges for future work in this area. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #85299. 	[Title] LEARNING BY ERROR-DRIVEN DECOMPOSITION [Abstract] In this paper we describe a new selforganizing decomposition technique for learning high-dimensional mappings. Problem decomposition is performed in an error-driven manner, such that the resulting subtasks (patches) are equally well approximated. Our method combines an unsupervised learning scheme (Feature Maps [Koh84]) with a nonlinear approximator (Backpropagation [RHW86]). The resulting learning system is more stable and effective in changing environments than plain backpropagation and much more powerful than extended feature maps as proposed by [RS88, RMS89]. Extensions of our method give rise to active exploration strategies for autonomous agents facing unknown environments. The appropriateness of our general purpose method will be demonstrated with an ex ample from mathematical function approximation. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1103676. 	[Title] Memory-based Time Series Recognition  A New Methodology and Real World Applications [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #28485. 	[Title] The Efficient Learning of Multiple Task Sequences [Abstract] I present a modular network architecture and a learning algorithm based on incremental dynamic programming that allows a single learning agent to learn to solve multiple Markovian decision tasks (MDTs) with significant transfer of learning across the tasks. I consider a class of MDTs, called composite tasks, formed by temporally concatenating a number of simpler, elemental MDTs. The architecture is trained on a set of composite and elemental MDTs. The temporal structure of a composite task is assumed to be unknown and the architecture learns to produce a temporal decomposition. It is shown that under certain conditions the solution of a composite MDT can be constructed by computationally inexpensive modifications of the solutions of its constituent elemental MDTs. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #672071. 	[Title] Predictions with Confidence Intervals (Local Error Bars) [Abstract] We present a new method for obtaining local error bars, i.e., estimates of the confidence in the predicted value that depend on the input. We approach this problem of nonlinear regression in a maximum likelihood framework. We demonstrate our technique first on computer generated data with locally varying, normally distributed target noise. We then apply it to the laser data from the Santa Fe Time Series Competition. Finally, we extend the technique to estimate error bars for iterated predictions, and apply it to the exact competition task where it gives the best performance to date. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #85324. 	[Title] The optimal number of learning samples and hidden units in function approximation with a feedforward network [Abstract] This paper presents a methodology to estimate the optimal number of learning samples and the number of hidden units needed to obtain a desired accuracy of a function approximation by a feedforward network. The representation error and the generalization error, components of the total approximation error are analyzed and the approximation accuracy of a feedforward network is investigated as a function of the number of hidden units and the number of learning samples. Based on the asymptotical behavior of the approximation error, an asymptotical model of the error function (AMEF) is introduced of which the parameters can be determined experimentally. An alternative model of the error function, which include theoretical results about general bounds of approximation, is also analyzed. In combination with knowledge about the computational complexity of the learning rule an optimal learning set size and number of hidden units can be found resulting in a minimum computation time for a given desired precision of the approximation. This approach was applied to optimize the learning of the camera-robot mapping of a visually guided robot arm and a complex logarithm function approximation. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1365. 	[Title] Cholinergic suppression of transmission may allow combined associative memory function and self-organization in the neocortex. [Abstract] Selective suppression of transmission at feedback synapses during learning is proposed as a mechanism for combining associative feedback with self-organization of feedforward synapses. Experimental data demonstrates cholinergic suppression of synaptic transmission in layer I (feedback synapses), and a lack of suppression in layer IV (feed-forward synapses). A network with this feature uses local rules to learn mappings which are not linearly separable. During learning, sensory stimuli and desired response are simultaneously presented as input. Feedforward connections form self-organized representations of input, while suppressed feedback connections learn the transpose of feedfor-ward connectivity. During recall, suppression is removed, sensory input activates the self-organized representation, and activity generates the learned response. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #5462. 	[Title] Priors, Stabilizers and Basis Functions: from regularization to radial, tensor and additive splines [Abstract] We had previously shown that regularization principles lead to approximation schemes which are equivalent to networks with one layer of hidden units, called Regularization Networks. In particular we had discussed how standard smoothness functionals lead to a subclass of regularization networks, the well-known Radial Basis Functions approximation schemes. In this paper we show that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models and some of the neural networks. In particular we introduce new classes of smoothness functionals that lead to different classes of basis functions. Additive splines as well as some tensor product splines can be obtained from appropriate classes of smoothness functionals. Furthermore, the same extension that leads from Radial Basis Functions (RBF) to Hyper Basis Functions (HBF) also leads from additive models to ridge approximation models, containing as special cases Breiman's hinge functions and some forms of Projection Pursuit Regression. We propose to use the term Generalized Regularization Networks for this broad class of approximation schemes that follow from an extension of regularization. In the probabilistic interpretation of regularization, the different classes of basis functions correspond to different classes of prior probabilities on the approximating function spaces, and therefore to different types of smoothness assumptions. In the final part of the paper, we show the relation between activation functions of the Gaussian and sigmoidal type by considering the simple case of the kernel G(x) = jxj. In summary, different multilayer networks with one hidden layer, which we collectively call Generalized Regularization Networks, correspond to different classes of priors and associated smoothness functionals in a classical regularization principle. Three broad classes are a) Radial Basis Functions that generalize into Hyper Basis Functions, b) some tensor product splines, and c) additive splines that generalize into schemes of the type of ridge approximation, hinge functions and one-hidden-layer perceptrons. This paper describes research done within the Center for Biological and Computational Learning in the Department of Brain and Cognitive Sciences and at the Artificial Intelligence Laboratory. This research is sponsored by grants from the Office of Naval Research under contracts N00014-91-J-1270 and N00014-92-J-1879; by a grant from the National Science Foundation under contract ASC-9217041 (which includes funds from DARPA provided under the HPCC program); and by a grant from the National Institutes of Health under contract NIH 2-S07-RR07047. Additional support is provided by the North Atlantic Treaty Organization, ATR Audio and Visual Perception Research Laboratories, Mitsubishi Electric Corporation, Sumitomo Metal Industries, and Siemens AG. Support for the A.I. Laboratory's artificial intelligence research is provided by ONR contract N00014-91-J-4038. Tomaso Poggio is supported by the Uncas and Helen Whitaker Chair at the Whitaker College, Massachusetts Institute of Technology. c fl Massachusetts Institute of Technology, 1993 Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #12631. 	[Title] Smoothing Spline ANOVA with Component-Wise Bayesian "Confidence Intervals" To Appear, J. Computational and Graphical Statistics [Abstract] We study a multivariate smoothing spline estimate of a function of several variables, based on an ANOVA decomposition as sums of main effect functions (of one variable), two-factor interaction functions (of two variables), etc. We derive the Bayesian "confidence intervals" for the components of this decomposition and demonstrate that, even with multiple smoothing parameters, they can be efficiently computed using the publicly available code RKPACK, which was originally designed just to compute the estimates. We carry out a small Monte Carlo study to see how closely the actual properties of these component-wise confidence intervals match their nominal confidence levels. Lastly, we analyze some lake acidity data as a function of calcium concentration, latitude, and longitude, using both polynomial and thin plate spline main effects in the same model. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #11093. 	[Title] Factorial Learning and the EM Algorithm [Abstract] Many real world learning problems are best characterized by an interaction of multiple independent causes or factors. Discovering such causal structure from the data is the focus of this paper. Based on Zemel and Hinton's cooperative vector quantizer (CVQ) architecture, an unsupervised learning algorithm is derived from the Expectation-Maximization (EM) framework. Due to the combinatorial nature of the data generation process, the exact E-step is computationally intractable. Two alternative methods for computing the E-step are proposed: Gibbs sampling and mean-field approximation, and some promising empirical results are presented. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #12638. 	[Title] Soft Classification, a.k.a. Risk Estimation, via Penalized Log Likelihood and Smoothing Spline Analysis of Variance [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1112929. 	[Title] Combining Linear Discriminant Functions with Neural Networks for Supervised Learning [Abstract] A novel supervised learning method is presented by combining linear discriminant functions with neural networks. The proposed method results in a tree-structured hybrid architecture. Due to constructive learning, the binary tree hierarchical architecture is automatically generated by a controlled growing process for a specific supervised learning task. Unlike the classic decision tree, the linear discriminant functions are merely employed in the intermediate level of the tree for heuristically partitioning a large and complicated task into several smaller and simpler subtasks in the proposed method. These subtasks are dealt with by component neural networks at the leaves of the tree accordingly. For constructive learning, growing and credit-assignment algorithms are developed to serve for the hybrid architecture. The proposed architecture provides an efficient way to apply existing neural networks (e.g. multi-layered perceptron) for solving a large scale problem. We have already applied the proposed method to a universal approximation problem and several benchmark classification problems in order to evaluate its performance. Simulation results have shown that the proposed method yields better results and faster training in comparison with the multi-layered perceptron. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1108834. 	[Title] ce: [39] <author> Yoda, M. </author> <year> (1994). </year> <title> Predicting the Tokyo stock market. </title> <editor> In Deboeck, G.J. (Ed.) </editor> <year> (1994). </year> <title> Trading on the Edge. </title> <address> New York: </address> <publisher> Wiley., </publisher> <pages> 66-79. </pages> <institution> VITA Graduate School Southern Illinois University Daniel Nikolaev Nikovski Date of Birth: </institution> <address> April 13, 1969 606 West College Street, Apt.4, Rm. 6, Carbondale, Illinois 62901 150 Hristo Botev Boulevard, Apt. </address> <month> 54, </month> <title> 4004 Plovdiv, Bulgaria Technical University - Sofia, Bulgaria Engineer of Computer Systems and Control Thesis Title: Adaptive Computation Techniques for Time Series Analysis Major Professor: </title> <journal> Dr. Mehdi Zargham </journal> [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1123188. 	[Title] GRKPACK: FITTING SMOOTHING SPLINE ANOVA MODELS FOR EXPONENTIAL FAMILIES [Abstract] Wahba, Wang, Gu, Klein and Klein (1995) introduced Smoothing Spline ANalysis of VAriance (SS ANOVA) method for data from exponential families. Based on RKPACK, which fits SS ANOVA models to Gaussian data, we introduce GRKPACK: a collection of Fortran subroutines for binary, binomial, Poisson and Gamma data. We also show how to calculate Bayesian confidence intervals for SS ANOVA estimates. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #887. 	[Title] Some studies in machine learning using the game of checkers. IBM Journal, 3(3):211-229, 1959. Some [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #7032. 	[Title] Combining estimates in regression and classification [Abstract] We consider the problem of how to combine a collection of general regression fit vectors in order to obtain a better predictive model. The individual fits may be from subset linear regression, ridge regression, or something more complex like a neural network. We develop a general framework for this problem and examine a recent cross-validation-based proposal called "stacking" in this context. Combination methods based on the bootstrap and analytic methods are also derived and compared in a number of examples, including best subsets regression and regression trees. Finally, we apply these ideas to classification problems where the estimated combination weights can yield insight into the structure of the problem. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1103737. 	[Title] On learning hierarchical classifications [Abstract] Many significant real-world classification tasks involve a large number of categories which are arranged in a hierarchical structure; for example, classifying documents into subject categories under the library of congress scheme, or classifying world-wide-web documents into topic hierarchies. We investigate the potential benefits of using a given hierarchy over base classes to learn accurate multi-category classifiers for these domains. First, we consider the possibility of exploiting a class hierarchy as prior knowledge that can help one learn a more accurate classifier. We explore the benefits of learning category-discriminants in a hard top-down fashion and compare this to a soft approach which shares training data among sibling categories. In doing so, we verify that hierarchies have the potential to improve prediction accuracy. But we argue that the reasons for this can be subtle. Sometimes, the improvement is only because using a hierarchy happens to constrain the expressiveness of a hypothesis class in an appropriate manner. However, various controlled experiments show that in other cases the performance advantage associated with using a hierarchy really does seem to be due to the prior knowledge it encodes. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #18815. 	[Title] Probabilistic Instance-Based Learning [Abstract] Traditional instance-based learning methods base their predictions directly on (training) data that has been stored in the memory. The predictions are based on weighting the contributions of the individual stored instances by a distance function implementing a domain-dependent similarity metrics. This basic approach suffers from three drawbacks: com-putationally expensive prediction when the database grows large, overfitting in the presence of noisy data, and sensitivity to the selection of a proper distance function. We address all these issues by giving a probabilistic interpretation to instance-based learning, where the goal is to approximate predictive distributions of the attributes of interest. In this probabilistic view the instances are not individual data items but probability distributions, and we perform Bayesian inference with a mixture of such prototype distributions. We demonstrate the feasibility of the method empirically for a wide variety of public domain classification data sets. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #263553. 	[Title] A Generalized Approximate Cross Validation for Smoothing Splines with Non-Gaussian Data 1 [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #46468. 	[Title] A Statistical Approach to Decision Tree Modeling [Abstract] A statistical approach to decision tree modeling is described. In this approach, each decision in the tree is modeled parametrically as is the process by which an output is generated from an input and a sequence of decisions. The resulting model yields a likelihood measure of goodness of fit, allowing ML and MAP estimation techniques to be utilized. An efficient algorithm is presented to estimate the parameters in the tree. The model selection problem is presented and several alternative proposals are considered. A hidden Markov version of the tree is described for data sequences that have temporal dependencies. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #24966. 	[Title] A Practical Bayesian Framework for Backprop Networks [Abstract] A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks. The framework makes possible: (1) objective comparisons between solutions using alternative network architectures; (2) objective stopping rules for network pruning or growing procedures; (3) objective choice of magnitude and type of weight decay terms or additive regularisers (for penalising large weights, etc.); (4) a measure of the effective number of well-determined parameters in a model; (5) quantified estimates of the error bars on network parameters and on network output; (6) objective comparisons with alternative learning and interpolation models such as splines and radial basis functions. The Bayesian `evidence' automatically embodies `Occam's razor,' penalising over-flexible and over-complex models. The Bayesian approach helps detect poor underlying assumptions in learning models. For learning models well matched to a problem, a good correlation between generalisation ability and the Bayesian evidence is obtained. This paper makes use of the Bayesian framework for regularisation and model comparison described in the companion paper `Bayesian interpolation' (MacKay, 1991a). This framework is due to Gull and Skilling (Gull, 1989a). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #755082. 	[Title] DRAFT Cluster-Weighted Modeling for Time Series Prediction and Characterization [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #906. 	[Title] Regularization Theory and Neural Networks Architectures [Abstract] We had previously shown that regularization principles lead to approximation schemes which are equivalent to networks with one layer of hidden units, called Regularization Networks. In particular, standard smoothness functionals lead to a subclass of regularization networks, the well known Radial Basis Functions approximation schemes. This paper shows that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models and some of the neural networks. In particular, we introduce new classes of smoothness functionals that lead to different classes of basis functions. Additive splines as well as some tensor product splines can be obtained from appropriate classes of smoothness functionals. Furthermore, the same generalization that extends Radial Basis Functions (RBF) to Hyper Basis Functions (HBF) also leads from additive models to ridge approximation models, containing as special cases Breiman's hinge functions, some forms of Projection Pursuit Regression and several types of neural networks. We propose to use the term Generalized Regularization Networks for this broad class of approximation schemes that follow from an extension of regularization. In the probabilistic interpretation of regularization, the different classes of basis functions correspond to different classes of prior probabilities on the approximating function spaces, and therefore to different types of smoothness assumptions. In summary, different multilayer networks with one hidden layer, which we collectively call Generalized Regularization Networks, correspond to different classes of priors and associated smoothness functionals in a classical regularization principle. Three broad classes are a) Radial Basis Functions that can be generalized to Hyper Basis Functions, b) some tensor product splines, and c) additive splines that can be generalized to schemes of the type of ridge approximation, hinge functions and several perceptron-like neural networks with one-hidden layer. 1 This paper will appear on Neural Computation, vol. 7, pages 219-269, 1995. An earlier version of Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #205196. 	[Title] Constructive Training Methods for Feedforward Neural Networks with Binary Weights [Abstract] DIMACS Technical Report 95-35 August 1995 Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #910. 	[Title] Learning networks for face analysis and synthesis [Abstract] This paper presents an overview of the face-related projects in our group. The unifying theme underlying our work is the use of example-based learning methods for both analyzing and synthesizing face images. We label the example face images (and for the problem of face detection, "near miss" faces as well) with descriptive parameters for pose, expression, identity, and face vs. non-face. Then, by using example-based learning techniques, we develop networks for performing analysis tasks such as pose and expression estimation, face recognition, and face detection in cluttered scenes. In addition to these analysis applications, we show how the example-based technique can also be used as a novel method for image synthesis that is for computer graphics. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #8079. 	[Title] Bias Plus Variance Decomposition for Zero-One Loss Functions [Abstract] We present a bias-variance decomposition of expected misclassification rate, the most commonly used loss function in supervised classification learning. The bias-variance decomposition for quadratic loss functions is well known and serves as an important tool for analyzing learning algorithms, yet no decomposition was offered for the more commonly used zero-one (misclassification) loss functions until the recent work of Kong & Dietterich (1995) and Breiman (1996). Their decomposition suffers from some major shortcomings though (e.g., potentially negative variance), which our decomposition avoids. We show that, in practice, the naive frequency-based estimation of the decomposition terms is by itself biased and show how to correct for this bias. We illustrate the decomposition on various algorithms and datasets from the UCI repository. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #662416. 	[Title] USING SMOOTHING SPLINE ANOVA TO EXAMINE THE RELATION OF RISK FACTORS TO THE INCIDENCE AND [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1152917. 	[Title]  [Abstract] Convergence Results for the EM Approach to Abstract The Expectation-Maximization (EM) algorithm is an iterative approach to maximum likelihood parameter estimation. Jordan and Jacobs (1993) recently proposed an EM algorithm for the mixture of experts architecture of Jacobs, Jordan, Nowlan and Hinton (1991) and the hierarchical mixture of experts architecture of Jordan and Jacobs (1992). They showed empirically that the EM algorithm for these architectures yields significantly faster convergence than gradient ascent. In the current paper we provide a theoretical analysis of this algorithm. We show that the algorithm can be regarded as a variable metric algorithm with its searching direction having a positive projection on the gradient of the log likelihood. We also analyze the convergence of the algorithm and provide an explicit expression for the convergence rate. In addition, we describe an acceleration technique that yields a significant speedup in simulation experiments. This report describes research done at the Dept. of Brain and Cognitive Sciences, the Center for Biological and Computational Learning, and the Artificial Intelligence Laboratory of the Massachusetts Institute of Technology. Support for CBCL is provided in part by a grant from the NSF (ASC-9217041). Support for the laboratory's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Dept. of Defense. The authors were supported by a grant from the McDonnell-Pew Foundation, by a grant from ATR Human Information Processing Research Laboratories, by a grant from Siemens Corporation, by by grant IRI-9013991 from the National Science Foundation, by grant N00014-90-J-1942 from the Office of Naval Research, and by NSF grant ECS-9216531 to support an Initiative in Intelligent Control at MIT. Michael I. Jordan is a NSF Presidential Young Investigator. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1110438. 	[Title] A Method of Combining Multiple Probabilistic Classifiers through Soft Competition on Different Feature Sets [Abstract] A novel method is proposed for combining multiple probabilistic classifiers on different feature sets. In order to achieve the improved classification performance, a generalized finite mixture model is proposed as a linear combination scheme and implemented based on radial basis function networks. In the linear combination scheme, soft competition on different feature sets is adopted as an automatic feature rank mechanism so that different feature sets can be always simultaneously used in an optimal way to determine linear combination weights. For training the linear combination scheme, a learning algorithm is developed based on Expectation-Maximization (EM) algorithm. The proposed method has been applied to a typical real world problem, viz. speaker identification, in which different feature sets often need consideration simultaneously for robustness. Simulation results show that the proposed method yields good performance in speaker identification. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1132459. 	[Title] First experiments using a mixture of nonlinear experts for time series prediction [Abstract] This paper investigates the advantages and disadvantages of the mixture of experts (ME) model (introduced to the connectionist community in [JJNH91] and applied to time series analysis in [WM95]) on two time series where the dynamics is well understood. The first series is a computer-generated series, consisting of a mixture between a noise-free process (the quadratic map) and a noisy process (a composition of a noisy linear autoregressive and a hyperbolic tangent). There are three main results: (1) the ME model produces significantly better results than single networks; (2) it discovers the regimes correctly and also allows us to characterize the sub-processes through their variances. (3) due to the correct matching of the noise level of the model to that of the data it avoids overfitting. The second series is the laser series used in the Santa Fe competition; the ME model also obtains excellent out-of-sample predictions, allows for analysis and shows no overfitting. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1132461. 	[Title] Avoiding overfitting by locally matching the noise level of the data gating network discovers the [Abstract] When trying to forecast the future behavior of a real-world system, two of the key problems are nonstationarity of the process (e.g., regime switching) and overfitting of the model (particularly serious for noisy processes). This articles shows how gated experts can point to solutions to these problems. The architecture, also called society of experts and mixture of experts consists of a (nonlinear) gating network and several (nonlinear) competing experts. Each expert learns a conditional mean (as usual), but each expert also has its own adaptive width. The gating network learns to assign a probability to each expert that depends on the input. This article first discusses the assumptions underlying this architecture and derives the weight update rules. It then evaluates the performance of gated experts in comparison to that of single networks, as well as to networks with two outputs, one predicting the mean, the other one the local error bar. This article also investigates the ability of gated experts to discover and characterize underlying the regimes. The results are: * there is significantly less overfitting compared to single nets, for two reasons: only subsets of the potential inputs are given to the experts and gating network (less of a curse of dimensionality), and the experts learn to match their variances to the (local) noise levels, thus only learning as This article focuses on the architecture and the overfitting problem. Applications to a computer-generated toy problem and the laser data from Santa Fe Competition are given in [Mangeas and Weigend, 1995], and the application to the real-world problem of predicting the electricity demand of France are given in [Mangeas et al., 1995]. much as the data support. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #32688. 	[Title] CLASSIFICATION USING HIERARCHICAL MIXTURES OF EXPERTS [Abstract] There has recently been widespread interest in the use of multiple models for classification and regression in the statistics and neural networks communities. The Hierarchical Mixture of Experts (HME) [1] has been successful in a number of regression problems, yielding significantly faster training through the use of the Expectation Maximisation algorithm. In this paper we extend the HME to classification and results are reported for three common classification benchmark tests: Exclusive-Or, N-input Parity and Two Spirals. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #684986. 	[Title] A Hierarchical Latent Variable Model for Data Visualization [Abstract] Visualization has proven to be a powerful and widely-applicable tool for the analysis and interpretation of multi-variate data. Most visualization algorithms aim to find a projection from the data space down to a two-dimensional visualization space. However, for complex data sets living in a high-dimensional space it is unlikely that a single two-dimensional projection can reveal all of the interesting structure. We therefore introduce a hierarchical visualization algorithm which allows the complete data set to be visualized at the top level, with clusters and sub-clusters of data points visualized at deeper levels. The algorithm is based on a hierarchical mixture of latent variable models, whose parameters are estimated using the expectation-maximization algorithm. We demonstrate the principle of the approach on a toy data set, and we then apply the algorithm to the visualization of a synthetic data set in 12 dimensions obtained from a simulation of multi-phase flows in oil pipelines, and to data in 36 dimensions derived from satellite images. A Matlab software implementation of the algorithm is publicly available from the world-wide web. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1109439. 	[Title] Classifying Seismic Signals by Integrating Ensembles of Neural Networks [Abstract] This paper proposes a classification scheme based on integration of multiple Ensembles of ANNs. It is demonstrated on a classification problem, in which seismic signals of Natural Earthquakes must be distinguished from seismic signals of Artificial Explosions. A Redundant Classification Environment consists of several Ensembles of Neural Networks is created and trained on Bootstrap Sample Sets, using various data representations and architectures. The ANNs within the Ensembles are aggregated (as in Bagging) while the Ensembles are integrated non-linearly, in a signal adaptive manner, using a posterior confidence measure based on the agreement (variance) within the Ensembles. The proposed Integrated Classification Machine achieved 92.1% correct classifications on the seismic test data. Cross Validation evaluations and comparisons indicate that such integration of a collection of ANN's Ensembles is a robust way for handling high dimensional problems with a complex non-stationary signal space as in the current Seismic Classification problem. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1997. 	[Title] Cross-Validation and the Bootstrap: Estimating the Error Rate of a Prediction Rule [Abstract] A training set of data has been used to construct a rule for predicting future responses. What is the error rate of this rule? The traditional answer to this question is given by cross-validation. The cross-validation estimate of prediction error is nearly unbiased, but can be highly variable. This article discusses bootstrap estimates of prediction error, which can be thought of as smoothed versions of cross-validation. A particular bootstrap method, the 632+ rule, is shown to substantially outperform cross-validation in a catalog of 24 simulation experiments. Besides providing point estimates, we also consider estimating the variability of an error rate estimate. All of the results here are nonparametric, and apply to any possible prediction rule: however we only study classification problems with 0-1 loss in detail. Our simulations include "smooth" prediction rules like Fisher's Linear Discriminant Function, and unsmooth ones like Nearest Neighbors. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #119761. 	[Title] Learning from Examples, Agent Teams and the Concept of Reflection [Abstract] In International Journal of Pattern Recognition and AI, 10(3):251-272, 1996 Also available as GMD report #766 Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #703953. 	[Title] Convergence in Norm for Alternating Expectation-Maximization (EM) Type Algorithms 1 [Abstract] We provide a sufficient condition for convergence of a general class of alternating estimation-maximization (EM) type continuous-parameter estimation algorithms with respect to a given norm. This class includes EM, penalized EM, Green's OSL-EM, and other approximate EM algorithms. The convergence analysis can be extended to include alternating coordinate-maximization EM algorithms such as Meng and Rubin's ECM and Fessler and Hero's SAGE. The condition for monotone convergence can be used to establish norms under which the distance between successive iterates and the limit point of the EM-type algorithm approaches zero monotonically. For illustration, we apply our results to estimation of Poisson rate parameters in emission tomography and establish that in the final iterations the logarithm of the EM iterates converge monotonically in a weighted Euclidean norm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #35797. 	[Title] Proben1 A Set of Neural Network Benchmark Problems and Benchmarking Rules [Abstract] Proben1 is a collection of problems for neural network learning in the realm of pattern classification and function approximation plus a set of rules and conventions for carrying out benchmark tests with these or similar problems. Proben1 contains 15 data sets from 12 different domains. All datasets represent realistic problems which could be called diagnosis tasks and all but one consist of real world data. The datasets are all presented in the same simple format, using an attribute representation that can directly be used for neural network training. Along with the datasets, Proben1 defines a set of rules for how to conduct and how to document neural network benchmarking. The purpose of the problem and rule collection is to give researchers easy access to data for the evaluation of their algorithms and networks and to make direct comparison of the published results feasible. This report describes the datasets and the benchmarking rules. It also gives some basic performance measures indicating the difficulty of the various problems. These measures can be used as baselines for comparison. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #139738. 	[Title] A Radial Basis Function Approach to Financial Time Series Analysis [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #168410. 	[Title] A Modular Q-Learning Architecture for Manipulator Task Decomposition `Data storage in the cerebellar model ar [Abstract] Compositional Q-Learning (CQ-L) (Singh 1992) is a modular approach to learning to perform composite tasks made up of several elemental tasks by reinforcement learning. Skills acquired while performing elemental tasks are also applied to solve composite tasks. Individual skills compete for the right to act and only winning skills are included in the decomposition of the composite task. We extend the original CQ-L concept in two ways: (1) a more general reward function, and (2) the agent can have more than one actuator. We use the CQ-L architecture to acquire skills for performing composite tasks with a simulated two-linked manipulator having large state and action spaces. The manipulator is a non-linear dynamical system and we require its end-effector to be at specific positions in the workspace. Fast function approximation in each of the Q-modules is achieved through the use of an array of Cerebellar Model Articulation Controller (CMAC) (Albus Our research interests involve the scaling up of machine learning methods, especially reinforcement learning, for autonomous robot control. We are interested in function approximators suitable for reinforcement learning in problems with large state spaces, such as the Cerebellar Model Articulation Controller (CMAC) (Albus 1975) which permit fast, online learning and good local generalization. In addition, we are interested in task decomposition by reinforcement learning and the use of hierarchical and modular function approximator architectures. We are examining the effectiveness of a modified Hierarchical Mixtures of Experts (HME) (Jordan & Jacobs 1993) approach for reinforcement learning since the original HME was developed mainly for supervised learning and batch learning tasks. The incorporation of domain knowledge into reinforcement learning agents is an important way of extending their capabilities. Default policies can be specified, and domain knowledge can also be used to restrict the size of the state-action space, leading to faster learning. We are investigating the use of Q-Learning (Watkins 1989) in planning tasks, using a classifier system (Holland 1986) to encode the necessary condition-action rules. Jordan, M. & Jacobs, R. (1993), Hierarchical mixtures of experts and the EM algorithm, Technical Report 9301, MIT Computational Cognitive Science. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #949217. 	[Title] Comparison of Neural and Statistical Classifiers| Theory and Practice [Abstract] Research Reports A13 January 1996 Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #4584. 	[Title] References elements that can solve difficult learning control problems. on Simulation of Adaptive Behavior, pages [Abstract] Miller, G. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review, 63(2):81-97. Schmidhuber, J. (1990b). Towards compositional learning with dynamic neural networks. Technical Report FKI-129-90, Technische Universitat Munchen, Institut fu Informatik. Servan-Schreiber, D., Cleermans, A., and McClelland, J. (1988). Encoding sequential structure in simple recurrent networks. Technical Report CMU-CS-88-183, Carnegie Mellon University, Computer Science Department. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #733167. 	[Title] Soft Vector Quantization and the EM Algorithm Running Title: Soft Vector Quantization and EM Section: [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6130. 	[Title] ``Learning Local Error Bars for Nonlinear Regression.''  Learning Local Error Bars for Nonlinear Regression [Abstract] We present a new method for obtaining local error bars for nonlinear regression, i.e., estimates of the confidence in predicted values that depend on the input. We approach this problem by applying a maximum-likelihood framework to an assumed distribution of errors. We demonstrate our method first on computer-generated data with locally varying, normally distributed target noise. We then apply it to laser data from the Santa Fe Time Series Competition where the underlying system noise is known quantization error and the error bars give local estimates of model misspecification. In both cases, the method also provides a weighted-regression effect that improves generalization performance. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #37884. 	[Title] Hidden Markov decision trees [Abstract] We study a time series model that can be viewed as a decision tree with Markov temporal structure. The model is intractable for exact calculations, thus we utilize variational approximations. We consider three different distributions for the approximation: one in which the Markov calculations are performed exactly and the layers of the decision tree are decoupled, one in which the decision tree calculations are performed exactly and the time steps of the Markov chain are decoupled, and one in which a Viterbi-like assumption is made to pick out a single most likely state sequence. We present simulation results for artificial data and the Bach chorales. Accepted for oral presentation at NIPS*96. Please summarise all the given information.

Paper #4330 has the title `Hierarchical Mixtures of Experts and the EM Algorithm  
` and the abstract `We present a tree-structured architecture for supervised learning. The statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models (GLIM's). Learning is treated as a maximum likelihood problem; in particular, we present an Expectation-Maximization (EM) algorithm for adjusting the parameters of the architecture. We also develop an on-line learning algorithm in which the parameters are updated incrementally. Comparative simulation results are presented in the robot dynamics domain. This report describes research done at the Dept. of Brain and Cognitive Sciences, the Center for Biological and Computational Learning, and the Artificial Intelligence Laboratory of the Massachusetts Institute of Technology. Support for CBCL is provided in part by a grant from the NSF (ASC-9217041). Support for the laboratory's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Dept. of Defense. The authors were supported by a grant from the McDonnell-Pew Foundation, by a grant from ATR Human Information Processing Research Laboratories, by a grant from Siemens Corporation, by by grant IRI-9013991 from the National Science Foundation, by grant N00014-90-J-1942 from the Office of Naval Research, and by NSF grant ECS-9216531 to support an Initiative in Intelligent Control at MIT. Michael I. Jordan is a NSF Presidential Young Investigator. 
`.
Question: Which category should Paper #4330 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #4330 has the title `Hierarchical Mixtures of Experts and the EM Algorithm  
` and the abstract `We present a tree-structured architecture for supervised learning. The statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models (GLIM's). Learning is treated as a maximum likelihood problem; in particular, we present an Expectation-Maximization (EM) algorithm for adjusting the parameters of the architecture. We also develop an on-line learning algorithm in which the parameters are updated incrementally. Comparative simulation results are presented in the robot dynamics domain. This report describes research done at the Dept. of Brain and Cognitive Sciences, the Center for Biological and Computational Learning, and the Artificial Intelligence Laboratory of the Massachusetts Institute of Technology. Support for CBCL is provided in part by a grant from the NSF (ASC-9217041). Support for the laboratory's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Dept. of Defense. The authors were supported by a grant from the McDonnell-Pew Foundation, by a grant from ATR Human Information Processing Research Laboratories, by a grant from Siemens Corporation, by by grant IRI-9013991 from the National Science Foundation, by grant N00014-90-J-1942 from the Office of Naval Research, and by NSF grant ECS-9216531 to support an Initiative in Intelligent Control at MIT. Michael I. Jordan is a NSF Presidential Young Investigator. 
`.
Question: Which category should Paper #4330 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Probabilistic_Methods