nils-herrmann/clean_scirepeval huggingface dataset:
Bayesian Models for Massive Multimedia Databases: a New Frontier

Modelling the increasing number of digital databases (the web, photo-libraries, music collections, news archives, medical databases) is one of the greatest challenges of statisticians in the new century. Despite the larg e amounts of data, the models are so large that they motivate the use of Bayesian models. In particular, the Bayesian perspective allows us to perform automatic regularisation to obtain sparse and coherent models. It also enables us to encode a priori knowledge, such as word, music and image preferences. The learned models can be used for browsing digital databases, information retrieval with image, m usic and/or text queries, image annotation (adding words to an image), text illustrat ion (adding images to a text), and object recognition.
An MDP-Based Recommender System

Typical Recommender systems adopt a static view of the recommendation process and treat it as a prediction problem. We argue that it is more appropriate to view the problem of generating recommendations as a sequential decision problem and, consequently, that Markov decision processes (MDP) provide a more appropriate model for Recommender systems. MDPs introduce two benefits: they take into account the long-term effects of each recommendation, and they take into account the expected value of each recommendation. To succeed in practice, an MDP-based Recommender system must employ a strong initial model; and the bulk of this paper is concerned with the generation of such a model. In particular, we suggest the use of an n-gram predictive model for generating the initial MDP. Our n-gram model induces a Markovchain model of user behavior whose predictive accuracy is greater than that of existing predictive models. We describe our predictive model in detail and evaluate its performance on real data. In addition, we show how the model can be used in an MDP-based Recommender system.
CFW: A Collaborative Filtering System Using Posteriors over Weights of Evidence

We describe CFW, a computationally efficient algorithm for collaborative filtering that uses posteriors over weights of evidence. In experiments on real data, we show that this method predicts as well or better than other methods in situations where the size of the user query is small. The new approach works particularly well when the user's query contains low frequency (unpopular) items. The approach complements that of dependency networks which perform well when the size of the query is large. Also in this paper, we argue that the use of posteriors over weights of evidence is a natural way to recommend similar items--a task that is somewhat different from the usual collaborative-filtering task.
The Learning-Curve Sampling Method Applied to Model-Based Clustering

We examine the learning-curve sampling method, an approach for applying machine-learning algorithms to large data sets. The approach is based on the observation that the computational cost of learning a model increases as a function of the sample size of the training data, whereas the accuracy of a model has diminishing improvements as a function of sample size. Thus, the learning-curve sampling method monitors the increasing costs and performance as larger and larger amounts of data are used for training, and terminates learning when future costs outweigh future benefits. In this paper, we formalize the learning-curve sampling method and its associated cost-benefit tradeoff in terms of decision theory. In addition, we describe the application of the learning-curve sampling method to the task of model-based clustering via the expectation-maximization (EM) algorithm. In experiments on three real data sets, we show that the learning-curve sampling method produces models that are nearly as accurate as those trained on complete data sets, but with dramatically reduced learning times. Finally, we describe an extension of the basic learning-curve approach for model-based clustering that results in an additional speedup. This extension is based on the observation that the shape of the learning curve for a given model and data set is roughly independent of the number of EM iterations used during training. Thus, we run EM for only a few iterations to decide how many cases to use for training, and then run EM to full convergence once the number of cases is selected.
Graphical Models: Structure Learning

The article GRAPHICAL MODELS: PARAMETER LEARNING discussed the learning of parameters for a xed graphical model. In this article, we discuss the simultaneous learning of parameters and structure. Real-world applications of such learning abound and can be found in (e.g.) the Proceedings of the Conference on Uncertainty in Arti cial Intelligence (1991 and after). An index to software for parameter and structure learning can be found at http://www.cs.berkeley.edu/ murphyk/Bayes/bnsoft.html. For simplicity, we concentrate on directed-acyclic graphical models (DAG models), but the basic principles described here can be applied more generally. We describe the Bayesian approach in detail and mention several common non-Bayesian approaches. We use the same notation as the article on parameter learning. In particular, we use X = (X1; : : : ; Xn) to denote the n variables that we are modeling, x to denote a con guration or observation of X, d = (x1; : : : ;x) to denote a random sample of N observations of X. In addition, we use Pai to denote the variables corresponding to the parents of Xi in a DAG model and pai to denote a con guration of those variables. Finally, we shall use the terms \model" and \structure" interchangeably. In particular, a DAG model (and hence its structure) is described by (1) its nodes and arcs, and (2) the distribution class of each of its local distributions p(xijpai).
The Markov Modulated Poisson Process and Markov Poisson Cascade with Applications to Web Traffic Modeling

A Markov modulated Poisson Process (MMPP) is a Poisson process whose rate varies according to a Markov process. The nonhomogeneous MMPP developed in this article is a natural model for point processes whose events combine irregular bursts of activity with predictable (e.g. daily and hourly) patterns. We show how the MMPP may be viewed as a superposition of unobserved Poisson processes that are activated and deactivated by an unobserved Markov process. The MMPP is a continuous time model which may also be viewed as a discretely indexed nonstationary hidden Markov model by viewing intervals between events as a sequence of dependent random variables. The HMM representation allows one to probabilistically reconstruct the latent Markov and Poisson processes using a set of forward-backward recursions. The recursions allow MMPP parameters to be estimated either by an EM algorithm or by a rapidly mixing Markov chain Monte Carlo algorithm which uses the recursions for data augmentation. The Markov-Poisson cascade (MPC) is an MMPP whose underlying Markov process obeys certain restrictions which uniquely order the event rates for the observed process. The ordering avoids a possible label switching issue without slowing down the rapidly mixing algorithms we use to implement the model. We apply the MPC to a data set containing click rate data for individual computer users browsing through the World Wide Web. Because the complete data posterior distribution for the MPC is a product of exponential family distributions we are able to incorporate data from multiple users into a hierarchical model using existing methods from hierarchical Poisson regression.
Direct Bayes for Interest Parameters

For an interest parameterψ(θ) the Bayesian method eliminates the nuisance parameterλ(θ) by integrating with respect to a conditional prior for λ givenψ. This conditional prior may be difficult to specify in both the subjective and objective Bayesian contexts. We propose the use of results from likelihood theory that give highly accurate third order determinations of various marginal distributions in the continuous case. The appropriate conditional prior exists as part of the calculations for the marginal distributions and corresponds to an integration used in the marginalization. To third order however the integrated likelihood for the interest parameter can be obtained directly; it needs no model information beyond that commonly available for accurate analyses of likelihood. Simple examples are given where the steps are analytically available to illustrate this direct Bayesian calculation.
Stage Mixture Modeling and Boosting

In this paper, we introduce and evaluate a data-driven staged mixture modeling technique for building density, regression, and classi cation models. Our basic approach is to sequentially add components to a nite mixture model using the structural expectation maximization (SEM) algorithm. We show that our technique is qualitatively similar to boosting. This correspondence is a natural byproduct of the fact that we use the SEM algorithm to sequentially t the mixture model. Finally, in our experimental evaluation, we demonstrate the e ectiveness of our approach on a variety of prediction and density estimation tasks using real-world data.
Discrimination Based on an Odds Ratio Parameterization

It is argued that the association of any two random elements with positive joint probability density function is characterized by its odds ratio function. The impact of this fundamental result is explored in two applications. In Bayesian analyses it is the association between an observable random variable and a random parameter that is of primary interest. In multivariate analysis it is the association between two random vectors that is investigated. If two random elements are strongly related the corresponding conditional distributions can well be separated. A concept of dependence therefore implies an approach to solving problems of discrimination. Discrimination based on the characterization of association by the odds ratio function is exemplified in Bayesian inference and in multivariate analysis.
Stratified exponential families: Graphical models and model selection

We describe a hierarchy of exponential families which is useful for distinguishing types of graphical models. Undirected graphical models with no hidden variables are linear exponential families (LEFs). Directed acyclic graphical (DAG) models and chain graphs with no hidden variables, including DAG models with several families of local distributions, are curved exponential families (CEFs). Graphical models with hidden variables are what we term stratified exponential families (SEFs). A SEF is a finite union of CEFs of various dimensions satisfying some regularity conditions. We also show that this hierarchy of exponential families is noncollapsing with respect to graphical models by providing a graphical model which is a CEF but not a LEF and a graphical model that is a SEF but not a CEF. Finally, we show how to compute the dimension of a stratified exponential family. These results are discussed in the context of model selection of graphical models.
Separating appearance from deformation

By representing images and image prototypes by linear subspaces spanned by "tangent vectors" (derivatives of an image with respect to translation, rotation, etc.), impressive invariance to known types of uniform distortion can be built into feedforward discriminators. We describe a new probability model that can jointly cluster data and learn mixtures of nonuniform, smooth deformation fields. Our fields are based on low-frequency wavelets, so they use very few parameters to model a wide range of smooth deformations (unlike, e.g., factor analysis, which uses a large number of parameters to model deformations). In spirit, our ideas are most similar to the idea of separating content from style published by Tenenbaum and Freeman. However, our models do not need labeled data for training, and thus allow for unsupervised separation of appearance from deformation. We give results on handwritten digit recognition and face recognition.
Learning mixtures of smooth, nonuniform deformation models for probabilistic image matching

By representing images and image prototypes by linear subspaces spanned by “tangent vectors” (derivatives of an image with respect to translation, rotation, etc.), impressive invariance to known types of uniform distortion can be built into feedforward discriminators. We describe a new probability model that can jointly cluster data and learn mixtures of nonuniform, smooth deformation fields. Our fields are based on low-frequency wavelets, so they use very few parameters to model a wide range of smooth deformations (unlike, e.g., factor analysis, which uses a large number of parameters to model deformations). We give results on handwritten digit recognition and face recognition.
Dependency Networks for Density Estimation, Collaborative Filtering, and Data Visualization

We describe a graphical model for probabilistic relationships – an alternative to the Bayesian network – called a dependency network. The graph of a dependency network, unlike a Bayesian network, is potentially cyclic. The probability component of a dependency network, like a Bayesian network, is a set of conditional distributions, one for each node given its parents. We identify several basic properties of this representation and describe a computationally efficient procedure for learning the graph and probability components from data. We describe the application of this representation to probabilistic inference, collaborative filtering (the task of predicting preferences), and the visualization of acausal predictive relationships.
Dependency Networks for Inference, Collaborative Filtering, and Data Visualization

We describe a graphical model for probabilistic relationships--an alternative to the Bayesian network--called a dependency network. The graph of a dependency network, unlike a Bayesian network, is potentially cyclic. The probability component of a dependency network, like a Bayesian network, is a set of conditional distributions, one for each node given its parents. We identify several basic properties of this representation and describe a computationally efficient procedure for learning the graph and probability components from data. We describe the application of this representation to probabilistic inference, collaborative filtering (the task of predicting preferences), and the visualization of acausal predictive relationships.
A Decision Theoretic Approach to Targeted Advertising

A simple advertising strategy that can be used to help increase sales of a product is to mail out special offers to selected potential customers. Because there is a cost associated with sending each offer, the optimal mailing strategy depends on both the benefit obtained from a purchase and how the offer affects the buying behavior of the customers. In this paper, we describe two methods for partitioning the potential customers into groups, and show how to perform a simple cost-benefit analysis to decide which, if any, of the groups should be targeted. In particular, we consider two decision-tree learning algorithms. The first is an "off the shelf" algorithm used to model the probability that groups of customers will buy the product. The second is a new algorithm that is similar to the first, except that for each group, it explicitly models the probability of purchase under the two mailing scenarios: (1) the mail is sent to members of that group and (2) the mail is not sent to members of that group. Using data from a real-world advertising experiment, we compare the algorithms to each other and to a naive mail-to-all strategy.
Targeted advertising with inventory management

Companies that maintain web sites can make considerable revenue through advertising, and consequently attracting advertisers has become an important and competitive endeavor. A property that can attract advertisers is high clickthrough rates; and therefore companies can bene t from delivery systems that serve advertisements selectively to those visitors most likely to click. In order to satisfy contractual obligations, however, these systems must simultaneously perform inventory management. For example, if a company has agreed to serve a certain number of a particular advertisement, it must do so regardless of how likely it is to be clicked. In this paper, we describe how to use a linear program to identify a schedule, based on known attributes of each visitor, that maximizes the expected number of clicks given all of the inventory-management constraints. We present experimental results using real data that demonstrate that a delivery schedule from our system realizes more clicks than a schedule that was hand constructed.
A comparison of scientific and engineering criteria for Bayesian model selection

Given a set of possible models for variables X and a set of possible parameters for each model, the Bayesian “estimate” of the probability distribution for X given observed data is obtained by averaging over the possible models and their parameters. An often-used approximation for this estimate is obtained by selecting a single model and averaging over its parameters. The approximation is useful because it is computationally efficient, and because it provides a model that facilitates understanding of the domain. A common criterion for model selection is the posterior probability of the model. Another criterion for model selection, proposed by San Martini and Spezzafari (1984), is the predictive performance of a model for the next observation to be seen. From the standpoint of domain understanding, both criteria are useful, because one identifies the model that is most likely, whereas the other identifies the model that is the best predictor of the next observation. To highlight the difference, we refer to the posterior-probability and alternative criteria as the scientific criterion (SC) and engineering criterion (EC), respectively. When we are interested in predicting the next observation, the model-averaged estimate is at least as good as that produced by EC, which itself is at least as good as the estimate produced by SC. We show experimentally that, for Bayesian-network models containing discrete variables only, the predictive performance of the model average can be significantly better than those of single models selected by either criterion, and that differences between models selected by the two criterion can be substantial.
Visualization of navigation patterns on a Web site using model-based clustering

We present a new methodology for visualizing navigation patterns on a Web site. In our approach, we rst partition site users into clusters such that only users with similar navigation paths through the site are placed into the same cluster. Then, for each cluster, we display these paths for users within that cluster. The clustering approach we employ is model based (as opposed to distance based) and partitions users according to the order in which they request Web pages. In particular, we cluster users by learning a mixture of rst-order Markov models using the ExpectationMaximization algorithm. Our algorithm scales linearly with both number of users and number of clusters, and our implementation easily handles millions of users and thousands of clusters in memory. In the paper, we describe the details of our technology and a tool based on it called WebCANVAS. We illustrate the use of our technology on user-traAEc data from msnbc.com.
Dependency Networks for Collaborative Filtering and Data Visualization

We describe a graphical representation of probabilistic relationships--an alternative to the Bayesian network--called a dependency network. Like a Bayesian network, a dependency network has a graph and a probability component. The graph component is a (cyclic) directed graph such that a node's parents render that node independent of all other nodes in the network. The probability component consists of the probability of a node given its parents for each node (as in a Bayesian network). We identify several basic properties of this representation, and describe its use in collaborative filtering (the task of predicting preferences) and the visualization of predictive relationships.
Parameter Priors for Directed Acyclic Graphical Models and the Characteriration of Several Probability Distributions

We show that the only parameter prior for complete Gaussian DAG models that satisfies global parameter independence, complete model equivalence, and some weak regularity assumptions, is the normal-Wishart distribution. Our analysis is based on the following new characterization of the Wishart distribution: let W be an n × n, n > 3, positive-definite symmetric matrix of random variables and f(W) be a pdf of W. Then, f(W) is a Wishart distribution if and only if W11 - W12W22-1, is independent of {W12, W22) for every block partitioning W11, W12, W12, W22 of W. Similar characterizations of the normal and normal-Wishart distributions are provided as well. We also show how to construct a prior for every DAG model over X from the prior of a single regression model.
Computationally Efficient Methods For Selecting Among Mixtures Of Graphical Models, With Discussion

We describe computationally efficient methods for Bayesian model selection. The methods select among mixtures in which each component is a directed acyclic graphical model (mixtures of DAGs or MDAGs), and can be applied to data sets in which some of the random variables are not always observed. The model-selection criterion that we consider is the posterior probability of the model (structure) given data. Our model-selection problem is difficult because (1) the number of possible model structures grows super-exponentially with the number of random variables and (2) missing data necessitates the use of computationally slow approximations of model posterior probability. We argue that simple search-and-score algorithms are infeasible for a variety of problems, and introduce a feasible approach in which parameter and structure search is interleaved and expected data is treated as real data. Our approach can be viewed as a combination of the Cheeseman-Stutz asymptotic approximation for model posterior probability and the Expectation-Maximization algorithm. We evaluate our procedure for selecting among MDAGs on synthetic and real examples.
Geometry, moments and Bayesian networks with hidden variables

The purpose of this paper is to present a systematic way of analysing the geometry of the probability spaces for a particular class of Bayesian networks with hidden variables. It will be shown that the conditional independence statements implicit in such graphical models can be neatly expressed as simple polynomial relationships among central moments. This algebraic framework will enable us to explore and identify the structural constraints on the sample space induced by models with tree strcutures and therefore characterise the families of distributions consistent with such conditional independence assumptions.
Fast Learning from Sparse Data

We describe two techniques that significantly improve the running time of several standard machine-learning algorithms when data is sparse. The first technique is an algorithm that efficiently extracts one-way and two-way counts-either real or expected-from discrete data. Extracting such counts is a fundamental step in learning algorithms for constructing a variety of models including decision trees, decision graphs, Bayesian networks, and naive-Bayes clustering models. The second technique is an algorithm that efficiently performs the E-step of the EM algorithm (i.e., inference) when applied to a naive-Bayes clustering model. Using real-world data sets, we demonstrate a dramatic decrease in running time for algorithms that incorporate these techniques.