A genetic algorithm for tuning variable orderings in Bayesian network structure learning
In the last two decades or so, Bayesian networks (BNs) [Pe88] have become a prevalent method for uncertain knowledge representation and reasoning. BNs are directed acyclic graphs (DAGs) where nodes represent random variables, and edges represent conditional dependence between random variables. Each node has a conditional probabilistic table (CPT) that contains probabilities of that node being a specific value given the values of its parents. The problem of learning a BN from data is important but hard. Finding the optimal structure of a BN from data has been shown to be NP-hard [HGC95], even without considering unobserved or irrelevant variables. In recent years, many Bayesian network learning algorithms have been developed. Generally these algorithms fall into two groups, score-based search and dependency analysis (conditional independence tests and constraint solving). Many previous approaches require that a node ordering is available before learning. Unfortunately, this is usually not the case in many real-world applications. To make greedy search usable when node orderings are unknown, we have developed a permutation genetic algorithm (GA) wrapper to tune the variable ordering given as input to K2 [CH92], a score-based BN learning algorithm. In our continuing project, we have used a probabilistic inference criterion as the GA’s fitness function and we are also trying some other criterion to evaluate the learning result such as the learning fixed-point property.

title: Genetic Wrappers for Constructive Induction in High-Performance Data Mining
abstract: We present an application of genetic algorithm-based design to configuration of high-level optimization systems, or wrappers, for relevance determination and constructive induction. Our system combines genetic wrappers with elicited knowledge on attribute relevance and synthesis. We discuss decision support issues in a large-scale commercial data mining project (cost prediction for multiple automobile insurance markets), and report experiments using D2K, a Java-based visual programming system for data mining and information visualization, and several commercial and research tools. Our GA system, Jenesis [HWRC00], is deployed on several network-of-workstation systems (Beowulf clusters). It achieves a linear speedup, due to a high degree of task parallelism, and improved test set accuracy, compared to decision tree learning with only constructive induction and state-space search-based wrappers [KJ97].

title: Relational Graphical Models of Computational Workflows for Data Mining
abstract: Collaborative recommendation is the problem of analyzing the content of an information retrieval system and actions of its users, to predict additional topics or products a new user may find useful. Developing this capability poses several challenges to machine learning and reasoning under uncertainty. Recent systems such as CiteSeer  have succeeded in providing some specialized but comprehensive indices of full documents, but the kind of probabilistic models used in such indexing do not extend easily to information Grid databases and computational Grid workflows. The collection of user data from Grid portals [4] provides a test bed for the underlying IR technology, including learning and inference systems. To model workflows created using the TAVERNA editor  and SCUFL description language, the DESCRIBER system, shown in Figure 1, applies score-based structure learning algorithms, including Bayesian model selection and greedy search (cf. the K2 algorithm) adapted to relational graphical models. Figure 2 illustrates how the decision support front-end of DESCRIBER interacts with modules that learn and reason using probabilistic relational models,. The purpose is to discover interrelationships among, and thereby recommend, components used in workflows developed by other Grid users.

title: Probabilistic Prediction of Protein Secondary Structure Using Causal Networks (Extended Abstract)
abstract: In this paper we present a probabilistic approach to analysis and prediction of protein structure. We argue that this approach provides a flexible and convenient mechanism to perform general scientific data analysis in molecular biology. We apply our approach to an important problem in molecular biology--predicting the secondary structure of proteins--and obtain experimental results comparable to several other methods. The causal networks that we use provide a very convenient medium for the scientist to experiment with different empirical models and obtain possibly important insights about the problem being studied.

title: Text Extraction from the Web via Text-to-Tag Ratio
abstract: We describe a method to extract content text from diverse Web pages by using the HTML document's text-to-tag ratio rather than specific HTML cues that may not be constant across various Web pages. We describe how to compute the text-to-tag ratio on a line-by-line basis and then cluster the results into content and non-content areas. With this approach we then show surprisingly high levels of recall for all levels of precision, and a large space savings.

title: Genetic wrappers for feature selection in decision tree induction and variable ordering in Bayesian network structure learning
abstract: In this paper, we address the automated tuning of input specification for supervised inductive learning and develop combinatorial optimization solutions for two such tuning problems. First, we present a framework for selection and reordering of input variables to reduce generalization error in classification and probabilistic inference. One purpose of selection is to control overfitting using validation set accuracy as a criterion for relevance. Similarly, some inductive learning algorithms, such as greedy algorithms for learning probabilistic networks, are sensitive to the evaluation order of variables. We design a generic fitness function for validation of input specification, then use it to develop two genetic algorithm wrappers: one for the variable selection problem for decision tree inducers and one for the variable ordering problem for Bayesian network structure learning. We evaluate the wrappers, using real-world data for the selection wrapper and synthetic data for both, and discuss their limitations and generalizability to other inducers.

title: The Clusnet Algorithm And Time Series Prediction
abstract: This paper describes a novel neural network architecture named ClusNet. This network is designed to study the trade-offs between the simplicity of instance-based methods and the accuracy of the more computational intensive learning methods. The features that make this network different from existing learning algorithms are outlined. A simple proof of convergence of the ClusNet algorithm is given. Experimental results showing the convergence of the algorithm on a specific problem is also presented. In this paper, ClusNet is applied to predict the temporal continuation of the Mackey-Glass chaotic time series. A comparison between the results obtained with ClusNet and other neural network algorithms is made. For example, ClusNet requires one-tenth the computing resources of the instance-based local linear method for this application while achieving comparable accuracy in this task. The sensitivity of ClusNet prediction accuracies on specific clustering algorithms is examined for an application. The simplicity and fast convergence of ClusNet makes it ideal as a rapid prototyping tool for applications where on-line learning is required.

title: Parameter significance estimation and financial prediction
abstract: This paper deals with the problem of parameter significance estimation, and its application to currency exchange rate prediction. The basic problem is that over the years, practitioners in the field of financial engineering have developed dozens of technical and fundamental indicators on the basis of which they try to predict financial time series. The practitioners are now faced with the problem of finding out which combinations of those indicators are most significant or relevant, and how their significance changes over time. The authors propose a novel neural architecture calledSupNet for estimating the significance of various parameters. The methodology is based on the principle of penalizing those features that are the largest contributors to the error term. Two algorithms based on this principle are proposed. This approach is different from related methodologies, which are based on the principle of removing parameters with the least significance. The proposed methodology is demonstrated on the next day returns of the DM-US currency exchange rate, and promising results are obtained.