nils-herrmann/clean_scirepeval huggingface dataset:
Stratiied Exponential Families: Graphical Models and Model Selection

We provide a classi cation of graphical models according to their representation as exponential families. Undirected graphical models with no hidden variables are linear exponential families (LEFs), directed acyclic graphical (DAG) models and chain graphs with no hidden variables, including DAG models with several families of local distributions, are curved exponential families (CEFs) and graphical models with hidden variables are strati ed exponential families (SEFs). A SEF is a nite union of CEFs of various dimensions satisfying some regularity conditions. The main results of this paper are that graphical models are SEFs and that many graphical models are not CEFs. That is, roughly speaking, graphical models when viewed as exponential families correspond to a set of smooth manifolds of various dimensions and usually not to a single smooth manifold. These results are discussed in the context of model selection.
Empirical Analysis of Predictive Algorithms for Collaborative Filtering

Collaborative filtering or recommender systems use a database about user preferences to predict additional topics or products a new user might like. In this paper we describe several algorithms designed for this task, including techniques based on correlation coefficients, vector-based similarity calculations, and statistical Bayesian methods. We compare the predictive accuracy of the various methods in a set of representative problem domains. We use two basic classes of evaluation metrics. The first characterizes accuracy over a set of individual predictions in terms of average absolute deviation. The second estimates the utility of a ranked list of suggested items. This metric uses an estimate of the probability that a user will see a recommendation in an ordered list. 
 
Experiments were run for datasets associated with 3 application areas, 4 experimental protocols, and the 2 evaluation metr rics for the various algorithms. Results indicate that for a wide range of conditions, Bayesian networks with decision trees at each node and correlation methods outperform Bayesian-clustering and vector-similarity methods. Between correlation and Bayesian networks, the preferred method depends on the nature of the dataset, nature of the application (ranked versus one-by-one presentation), and the availability of votes with which to make predictions. Other considerations include the size of database, speed of predictions, and learning time.
Leaning (1991). Cytotoxic chemotherapy monitoring using stochastic simulation

The belief network framework for reasoning with uncertainty in knowledgebased systems has been around for some time now. As more and more practical applications employing the framework are being developed, it becomes apparent that the framework lacks with regard to explicit means for exerting control over reasoning. In this paper, we extend the belief network framework with a method for selective gathering of evidence for diagnostic applications. To this end, a belief network architecture is developed consisting of two layers: a probabilistic layer specifying a belief network and its associated algorithms, and a control layer providing the method for evidence gathering.
A Bayesian Approach to Filtering Junk E-Mail

In addressing the growing problem of junk E-mail on the Internet, we examine methods for the automated construction of filters to eliminate such unwanted messages from a user’s mail stream. By casting this problem in a decision theoretic framework, we are able to make use of probabilistic learning methods in conjunction with a notion of differential misclassification cost to produce filters Which are especially appropriate for the nuances of this task. While this may appear, at first, to be a straight-forward text classification problem, we show that by considering domain-specific features of this problem in addition to the raw text of E-mail messages, we can produce much more accurate filters. Finally, we show the efficacy of such filters in a real world usage scenario, arguing that this technology is mature enough for deployment.
The Lumière Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users

The Lumiere Project centers on harnessing probability and utility to provide assistance to computer software users. We review work on Bayesian user models that can be employed to infer a user's needs by considering a user's background, actions, and queries. Several problems were tackled in Lumiere research, including (1) the construction of Bayesian models for reasoning about the time-varying goals of computer users from their observed actions and queries, (2) gaining access to a stream of events from software applications, (3) developing a language for transforming system events into observational variables represented in Bayesian user models, (4) developing persistent profiles to capture changes in a user's expertise, and (5) the development of an overall architecture for an intelligent user interface. Lumiere prototypes served as the basis for the Ofice Assistant in the Microsoft Office '97 suite of productivity applications.
Learning Mixtures of DAG Models

We describe computationally efficient methods for learning mixtures in which each component is a directed acyclic graphical model (mixtures of DAGs or MDAGs). We argue that simple search-and-score algorithms are infeasible for a variety of problems, and introduce a feasible approach in which parameter and structure search is interleaved and expected data is treated as real data. Our approach can be viewed as a combination of (1) the Cheeseman-Stutz asymptotic approximation for model posterior probability and (2) the Expectation-Maximization algorithm. We evaluate our procedure for selecting among MDAGs on synthetic and real examples.
An Experimental Comparison of Several Clustering and Initialization Methods

We examine methods for clustering in high dimensions. In the first part of the paper, we perform an experimental comparison between three batch clustering algorithms: the Expectation-Maximization (EM) algorithm, a "winner take all" version of the EM algorithm reminiscent of the K-means algorithm, and model-based hierarchical agglomerative clustering. We learn naive-Bayes models with a hidden root node, using high-dimensional discrete-variable data sets (both real and synthetic). We find that the EM algorithm significantly outperforms the other methods, and proceed to investigate the effect of various initialization schemes on the final solution produced by the EM algorithm. The initializations that we consider are (1) parameters sampled from an uninformative prior, (2) random perturbations of the marginal distribution of the data, and (3) the output of hierarchical agglomerative clustering. Although the methods are substantially different, they lead to learned models that are strikingly similar in quality.
Probabilistic relevance relations

The intuition behind the construction of Bayesian networks and other graph-based representations of joint probability distributions from expert judgments is based on the assumed relationship between "connectedness" in the graphical model and "relatedness" among the variables involved. We show that several plausible definitions of relatedness do not adhere to such an equivalence. We then provide a definition of probabilistic relatedness that is closely related to connectedness in the graphical model and prove that the two concepts are equivalent whenever the model uses only propositional variables and assuming every combination of value assignment to these variables is feasible. We conjecture that the equivalence established holds also when these restrictions are lifted.
Inferring Informational Goals from Free-Text Queries: A Bayesian Approach

People using consumer software applications typically do not use technical jargon when querying an online database of help topics. Rather, they attempt to communicate their goals with common words and phrases that describe software functionality in terms of structure and objects they understand. We describe a Bayesian approach to modeling the relationship between words in a user's query for assistance and the informational goals of the user. After reviewing the general method, we describe several extensions that center on integrating additional distinctions and structure about language usage and user goals into the Bayesian models.
Challenge: What is the Impact of Bayesian Networks on Learning?

In recent years, there has been much interest in learning Bayesian networks from data. Learning such models is desirable simply because there is a wide array of off-the-shelf tools that can apply the learned models as expert systems, diagnosis engines, and decision support systems. Practitioners also claim that adaptive Bayesian networks have advantages in their own right as a non-parametric method for density estimation, data analysis, pattern classification, and modeling. Among the reasons cited we find: their semantic clarity and understandability by humans, the ease of acquisition and incorporation of prior knowledge, the ease of integration with optimal decision-making methods, the possibility of causal interpretation of learned models, and the automatic handling of noisy and missing data. In spite of these claims, and the initial success reported recently, methods that learn Bayesian networks have yet to make the impact that other techniques such as neural networks and hidden Markov models have made in applications such as pattern and speech recognition. In this paper, we challenge the research community to identify and characterize domains where induction of Bayesian networks makes the critical difference, and to quantify the factors that are responsible for that difference. In addition to formalizing the challenge, we identify research problems whose solution is, in our view, crucial for meeting this challenge.
A Bayesian Approach to Learning Bayesian Networks with Local Structure

Recently several researchers have investigated techniques for using data to learn Bayesian networks containing compact representations for the conditional probability distributions (CPDs) stored at each node. The majority of this work has concentrated on using decision-tree representations for the CPDs. In addition, researchers typically apply non-Bayesian (or asymptotically Bayesian) scoring functions such as MDL to evaluate the goodness-of-fit of networks to the data. 
 
In this paper we investigate a Bayesian approach to learning Bayesian networks that contain the more general decision-graph representations of the CPDs. First, we describe how to evaluate the posterior probability-- that is, the Bayesian score--of such a network, given a database of observed cases. Second, we describe various search spaces that can be used, in conjunction with a scoring function and a search procedure, to identify one or more high-scoring networks. Finally, we present an experimentd evaluation of the search spaces, using a greedy algorithm and a Bayesian scoring function.
Models and Selection Criteria for Regression and Classification

When performing regression or classification, we are interested in the conditional probability distribution for an outcome or class variable Y given a set of explanatory or input variables X. We consider Bayesian models for this task. In particular, we examine a special class of models, which we call Bayesian regression/classification (BRC) models, that can be factored into independent conditional (y[x) and input (x) models. These models are convenient, because the conditional model (the portion of the full model that we care about) can be analyzed by itself. We examine the practice of transforming arbitrary Bayesian models to BRC models, and argue that this practice is often inappropriate because it ignores prior knowledge that may be important for learning. In addition, we examine Bayesian methods for learning models from data. We discuss two criteria for Bayesian model selection that are appropriate for repression/classification: one described by Spiegelhalter etah (1993), and other by Buntine (1993). We contrast these two criteria using the prequentia] framework of Dawid (1984), and give sufficient conditions under which the criteria agree.
Structure and Parameter Learning for Causal Independence and Causal Interaction Models

We begin by discussing causal independence models and generalize these models to causal interaction models. Causal interaction models are models that have independent mechanisms where mechanisms can have several causes. In addition to introducing several particular types of causal interaction models, we show how we can apply the Bayesian approach to learning causal interaction models obtaining approximate posterior distributions for the models and obtain MAP and ML estimates for the parameters. We illustrate the approach with a simulation study of learning model posteriors.
Learning Mixtures of Bayesian Networks

We describe a heuristic method for learning mixtures of Bayesian Networks (MBNs) from possibly incomplete data. The considered class of models is mixtures in which each mixture component is a Bayesian network encoding a conditional Gaussian distribution over a xed set of variables. Some variables may be hidden or otherwise have missing observations. A key idea in our approach is to treat expected data as real data. This allows us to interleave structure and parameter search and to take advantage of closed form approximations for marginal likelihood. In addition, by treating expected data as real data, the search criterion factors by variable, making the search processes more e cient. We evaluate our approach on synthetic and real-world data sets.
Probabilistic Independence Networks for Hidden Markov Probability Models

Graphical techniques for modeling the dependencies of random variables have been explored in a variety of different areas, including statistics, statistical physics, artificial intelligence, speech recognition, image processing, and genetics. Formalisms for manipulating these models have been developed relatively independently in these research communities. In this paper we explore hidden Markov models (HMMs) and related structures within the general framework of probabilistic independence networks (PINs). The paper presents a self-contained review of the basic principles of PINs. It is shown that the well-known forward-backward (F-B) and Viterbi algorithms for HMMs are special cases of more general inference algorithms for arbitrary PINs. Furthermore, the existence of inference and estimation algorithms for more general graphical models provides a set of analysis tools for HMM practitioners who wish to explore a richer class of HMM structures. Examples of relatively complex models to handle sensor fusion and coarticulation in speech recognition are introduced and treated within the graphical model framework to illustrate the advantages of the general approach.
A characterization of the Dirichlet distribution through global and local parameter independence

We provide a new characterization of the Dirichlet distribution. Let θ ij , 1 ≤ i < k, 1 ≤ j ≤ n, be positive random variables that sum to unity. Define θ i = Σ j n = 1 θ ij , θ 1 = {θ i } i=1 k=1 , θ j/i = θ ij /Σj θ ij and θ j/i = {θ j/i } j=1 n-1 . We prove that if {θi, θ j/1 ,..., θ j/k } are mutually independent and {θ. j , θ l/1 ,..., θ I/n } are mutually independent (where θ. j and θ I/j are defined analogously), and each parameter set has a strictly positive pdf, then the pdf of θ ij is Dirichlet. This characterization implies that under assumptions made by several previous authors for selecting a Bayesian network structure out of a set of candidate structures, a Dirichlet prior on the parameters is inevitable.
Probabilistic and Bayesian Representations of Uncertainty in Information Systems: A Pragmatic Introduction

A great deal has been written about the underlying principles of alternative methods of representing uncertainty—not the least about probability and Bayesian methods. While we cannot entirely resist discussing basic principles, we will focus on the pragmatic issues, which too often get lost under the mass of philosophy and mathematics. We will address such questions as: How can we use probability to represent the various types of uncertainty? How can we quantify these uncertainties? How much effort is necessary to do so? How can we obtain the greatest benefits from representing uncertainty while minimizing the effort? There are a variety of reasons to represent uncertainty and a variety of probabilistic and Bayesian ways to do so, requiring varying amounts of effort. We discuss an approach to resolve these issues, so that the costs will be commensurate with the benefits.
Efficient Approximations for the Marginal Likelihood of Incomplete Data Given a Bayesian Network

We examine asymptotic approximations for the marginal likelihood of incomplete data given a Bayesian network. We consider the Laplace approximation and the less accurate but more efficient BIC/MDL approximation. We also consider approximations proposed by Draper (1993) and Cheeseman and Stutz (1995). These approximations are as efficient as BIC/MDL, but their accuracy has not been studied in any depth. We compare the accuracy of these approximations under the assumption that the Laplace approximation is the most accurate. In experiments using synthetic data generated from discrete naive-Bayes models having a hidden root node, we find that (1) the BIC/MDL measure is the least accurate, having a bias in favor of simple models, and (2) the Draper and CS measures are the most accurate, having a bias in favor of simple and complex models, respectively.
Asymptotic Model Selection for Directed Networks with Hidden Variables

We extend the Bayesian Information Criterion (BIC), an asymptotic approximation for tile marginal likelihood, to Bayesian networks with hidden variables. This approximation can be used to select models given large sampies of data. Tile standard BIC as well as our extension punishes the complexity of a model according to tile dimension of its parameters. We argue that the dimension of a Bayesian uetwork with hidden variables is tile rank of the Jacobian matrix of the transformation between the parameters of the network and the parameters of the observable variables. We compute the dimensions of several networks including the naive Bayes model with a hidden root node.
Decision-Theoretic Troubleshooting: A Framework for Repair and Experiment

We develop and extend existing decision-theoretic methods for troubleshooting a nonfunctioning device. Traditionally, diagnosis with Bayesian networks has focused on belief updating--determining the probabilities of various faults given current observations. In this paper, we extend this paradigm to include taking actions. In particular, we consider three classes of actions: (1) we can make observations regarding the behavior of a device and infer likely faults as in traditional diagnosis, (2) we can repair a component and then observe the behavior of the device to infer likely faults, and (3) we can change the configuration of the device, observe its new behavior, and infer the likelihood of faults. Analysis of latter two classes of troubleshooting actions requires incorporating notions of persistence into the belief-network formalism used for probabilistic inference.
Causal independence for probability assessment and inference using Bayesian networks

A Bayesian network is a probabilistic representation for uncertain relationships, which has proven to be useful for modeling real-world problems. When there are many potential causes of a given effect, however, both probability assessment and inference using a Bayesian network can be difficult. In this paper, we describe causal independence, a collection of conditional independence assertions and functional relationships that are often appropriate to apply to the representation of the uncertain interactions between causes and effect. We show how the use of causal independence in a Bayesian network can greatly simplify probability assessment as well as probabilistic inference.
Learning Bayesian Networks: Search Methods and Experimental Results

We discuss Bayesian approaches for learning Bayesian networks from data. First, we review a metric for computing the relative posterior probability of a network structure given data developed by Heckerman et al. (1994a,b,c). We see that the metric has a property useful for inferring causation from data. Next, we describe search methods for identifying network structures with high posterior probabilities. We describe polynomial algorithms for nding the highestscoring network structures in the special case where every node has at most k = 1 parent. We show that the general case (k > 1) is NP-hard, and review heuristic search algorithms for this general case. Finally, we describe a methodology for evaluating learning algorithms, and use this methodology to evaluate various scoring metrics and search
A Definition and Graphical Representation for Causality

We present a precise definition of cause and effect in terms of a fundamental notion called unresponsiveness. Our definition is based on Savage's (1954) formulation of decision theory and departs from the traditional view of causation in that our causal assertions are made relative to a set of decisions. An important consequence of this departure is that we can reason about cause locally, not requiring a causal explanation for every dependency. Such local reasoning can be beneficial because it may not be necessary to determine whether a particular dependency is causal to make a decision. Also in this paper, we examine the graphical encoding of causal relationships. We show that influence diagrams in canonical form are an accurate and efficient representation of causal relationships. In addition, we establish a correspondence between canonical form and Pearl's causal theory.
A Characterization of the Bivariate Normal-Wishart Distribution

We provide a new characterization of the Bivariate normal-Wishart distribution. Let ~ x = fx 1 x 2 g have a non-singular Bivariate normal pdf f(~ x) = N(~ W) with unknown mean vector ~ and unknown precision matrix W. L e t f(~ x) = f(x 1)f(x 2 jx 1) where f(x 1) = N(m 1 1=v 1) and f(x 2 jx 1) = N(m 2j1 +b 12 x 1 1=v 2j1). Similarly, deene fv 2 v 1j2 b 21 m 2 m 1j2 g using the factorization f(~ x) = f(x 2)f(x 1 jx 2). Assume ~ and W have a strictly positive joint pdf f ~ W (~ W). Then f ~ W is a normal-Wishart pdf if and only if global independence holds, namely, fv 1 m 1 g?fv 2j1 b 12 m 2j1 g and fv 2 m 2 g?fv 1j2 b 21 m 1j2 g and local independence holds, namely, ?fv 1 m 1 g, ?fv 2j1 b 12 m 2j1 g and ?fv 2 m 2 g, ?fv 1j2 b 21 m 1j2 g (where x denotes the standardized r.v x and ? stands for independence). We also characterize the Bivariate pdfs that satisfy global independence alone. Such pdfs are termed Hyper-Markov laws and they are used for a decomposable prior-to-posterior analysis of Bayesian networks.
Bayesian networks

This brief tutorial on Bayesian networks serves to introduce readers to some of the concepts, terminology, and notation employed by articles in this special section. In a Bayesian network, a variable takes on values from a collection of mutually exclusive and collective exhaustive states. A variable may be discrete, having a finite or countable number of states, or it may be continuous. Often the choice of states itself presents an interesting modeling question. For example, in a system for troubleshooting a problem with printing, we may choose to model the variable “print output” with two states—“present” and “absent”—or we may want to model the variable with finer distinctions such as “absent,” “blurred ,” “cut off,” and “ok.”