pwc-archive/papers-with-abstracts huggingface dataset:
Learning Bayesian Networks: The Combination of Knowledge and Statistical Data
Authors: David Heckerman, Dan Geiger, David Maxwell Chickering
Date: 2013-02-27 00:00:00
Tasks: Heuristic Search

We describe algorithms for learning Bayesian networks from a combination of
user knowledge and statistical data. The algorithms have two components: a
scoring metric and a search procedure. The scoring metric takes a network
structure, statistical data, and a user's prior knowledge, and returns a score
proportional to the posterior probability of the network structure given the
data. The search procedure generates networks for evaluation by the scoring
metric. Our contributions are threefold. First, we identify two important
properties of metrics, which we call event equivalence and parameter
modularity. These properties have been mostly ignored, but when combined,
greatly simplify the encoding of a user's prior knowledge. In particular, a
user can express her knowledge-for the most part-as a single prior Bayesian
network for the domain. Second, we describe local search and annealing
algorithms to be used in conjunction with scoring metrics. In the special case
where each node has at most one parent, we show that heuristic search can be
replaced with a polynomial algorithm to identify the networks with the highest
score. Third, we describe a methodology for evaluating Bayesian-network
learning algorithms. We apply this approach to a comparison of metrics and
search procedures.
A Definition and Graphical Representation for Causality
Authors: David Heckerman, Ross D. Shachter
Date: 2013-02-20 00:00:00
Tasks: Form

We present a precise definition of cause and effect in terms of a fundamental
notion called unresponsiveness. Our definition is based on Savage's (1954)
formulation of decision theory and departs from the traditional view of
causation in that our causal assertions are made relative to a set of
decisions. An important consequence of this departure is that we can reason
about cause locally, not requiring a causal explanation for every dependency.
Such local reasoning can be beneficial because it may not be necessary to
determine whether a particular dependency is causal to make a decision. Also in
this paper, we examine the graphical encoding of causal relationships. We show
that influence diagrams in canonical form are an accurate and efficient
representation of causal relationships. In addition, we establish a
correspondence between canonical form and Pearl's causal theory.
A Bayesian Approach to Learning Causal Networks
Authors: David Heckerman
Date: 2013-02-20 00:00:00

Whereas acausal Bayesian networks represent probabilistic independence,
causal Bayesian networks represent causal relationships. In this paper, we
examine Bayesian methods for learning both types of networks. Bayesian methods
for learning acausal networks are fairly well developed. These methods often
employ assumptions to facilitate the construction of priors, including the
assumptions of parameter independence, parameter modularity, and likelihood
equivalence. We show that although these assumptions also can be appropriate
for learning causal networks, we need additional assumptions in order to learn
causal networks. We introduce two sufficient assumptions, called {em mechanism
independence} and {em component independence}. We show that these new
assumptions, when combined with parameter independence, parameter modularity,
and likelihood equivalence, allow us to apply methods for learning acausal
networks to learn causal networks.
Asymptotic Model Selection for Directed Networks with Hidden Variables
Authors: Dan Geiger, David Heckerman, Christopher Meek
Date: 2013-02-13 00:00:00
Tasks: Model Selection

We extend the Bayesian Information Criterion (BIC), an asymptotic
approximation for the marginal likelihood, to Bayesian networks with hidden
variables. This approximation can be used to select models given large samples
of data. The standard BIC as well as our extension punishes the complexity of a
model according to the dimension of its parameters. We argue that the dimension
of a Bayesian network with hidden variables is the rank of the Jacobian matrix
of the transformation between the parameters of the network and the parameters
of the observable variables. We compute the dimensions of several networks
including the naive Bayes model with a hidden root node.
Structure and Parameter Learning for Causal Independence and Causal Interaction Models
Authors: Christopher Meek, David Heckerman
Date: 2013-02-06 00:00:00

This paper discusses causal independence models and a generalization of these
models called causal interaction models. Causal interaction models are models
that have independent mechanisms where a mechanism can have several causes. In
addition to introducing several particular types of causal interaction models,
we show how we can apply the Bayesian approach to learning causal interaction
models obtaining approximate posterior distributions for the models and obtain
MAP and ML estimates for the parameters. We illustrate the approach with a
simulation study of learning model posteriors.
A Bayesian Approach to Learning Bayesian Networks with Local Structure
Authors: David Maxwell Chickering, David Heckerman, Christopher Meek
Date: 2013-02-06 00:00:00

Recently several researchers have investigated techniques for using data to
learn Bayesian networks containing compact representations for the conditional
probability distributions (CPDs) stored at each node. The majority of this work
has concentrated on using decision-tree representations for the CPDs. In
addition, researchers typically apply non-Bayesian (or asymptotically Bayesian)
scoring functions such as MDL to evaluate the goodness-of-fit of networks to
the data. In this paper we investigate a Bayesian approach to learning Bayesian
networks that contain the more general decision-graph representations of the
CPDs. First, we describe how to evaluate the posterior probability that is, the
Bayesian score of such a network, given a database of observed cases. Second,
we describe various search spaces that can be used, in conjunction with a
scoring function and a search procedure, to identify one or more high-scoring
networks. Finally, we present an experimental evaluation of the search spaces,
using a greedy algorithm and a Bayesian scoring function.
Learning Mixtures of DAG Models
Authors: Bo Thiesson, Christopher Meek, David Maxwell Chickering, David Heckerman
Date: 2013-01-30 00:00:00

We describe computationally efficient methods for learning mixtures in which
each component is a directed acyclic graphical model (mixtures of DAGs or
MDAGs). We argue that simple search-and-score algorithms are infeasible for a
variety of problems, and introduce a feasible approach in which parameter and
structure search is interleaved and expected data is treated as real data. Our
approach can be viewed as a combination of (1) the Cheeseman--Stutz asymptotic
approximation for model posterior probability and (2) the
Expectation--Maximization algorithm. We evaluate our procedure for selecting
among MDAGs on synthetic and real examples.
An Experimental Comparison of Several Clustering and Initialization Methods
Authors: Marina Meila, David Heckerman
Date: 2013-01-30 00:00:00
Tasks: Clustering

We examine methods for clustering in high dimensions. In the first part of
the paper, we perform an experimental comparison between three batch clustering
algorithms: the Expectation-Maximization (EM) algorithm, a winner take all
version of the EM algorithm reminiscent of the K-means algorithm, and
model-based hierarchical agglomerative clustering. We learn naive-Bayes models
with a hidden root node, using high-dimensional discrete-variable data sets
(both real and synthetic). We find that the EM algorithm significantly
outperforms the other methods, and proceed to investigate the effect of various
initialization schemes on the final solution produced by the EM algorithm. The
initializations that we consider are (1) parameters sampled from an
uninformative prior, (2) random perturbations of the marginal distribution of
the data, and (3) the output of hierarchical agglomerative clustering. Although
the methods are substantially different, they lead to learned models that are
strikingly similar in quality.
Inferring Informational Goals from Free-Text Queries: A Bayesian Approach
Authors: David Heckerman, Eric J. Horvitz
Date: 2013-01-30 00:00:00

People using consumer software applications typically do not use technical
jargon when querying an online database of help topics. Rather, they attempt to
communicate their goals with common words and phrases that describe software
functionality in terms of structure and objects they understand. We describe a
Bayesian approach to modeling the relationship between words in a user's query
for assistance and the informational goals of the user. After reviewing the
general method, we describe several extensions that center on integrating
additional distinctions and structure about language usage and user goals into
the Bayesian models.
Fast Learning from Sparse Data
Authors: David Maxwell Chickering, David Heckerman
Date: 2013-01-23 00:00:00
Tasks: Clustering

We describe two techniques that significantly improve the running time of
several standard machine-learning algorithms when data is sparse. The first
technique is an algorithm that effeciently extracts one-way and two-way
counts--either real or expected-- from discrete data. Extracting such counts is
a fundamental step in learning algorithms for constructing a variety of models
including decision trees, decision graphs, Bayesian networks, and naive-Bayes
clustering models. The second technique is an algorithm that efficiently
performs the E-step of the EM algorithm (i.e. inference) when applied to a
naive-Bayes clustering model. Using real-world data sets, we demonstrate a
dramatic decrease in running time for algorithms that incorporate these
techniques.
Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributions
Authors: Dan Geiger, David Heckerman
Date: 2013-01-23 00:00:00

We show that the only parameter prior for complete Gaussian DAG models that satisfies global parameter independence, complete model equivalence, and some weak regularity assumptions, is the normal-Wishart distribution. Our analysis is based on the following new characterization of the Wishart distribution: let W be an n x n, n >= 3, positive-definite symmetric matrix of random variables and f(W) be a pdf of W. Then, f(W) is a Wishart distribution if and only if W_{11}-W_{12}W_{22}^{-1}W_{12}' is independent of {W_{12}, W_{22}} for every block partitioning W_{11}, W_{12}, W_{12}', W_{22} of W. Similar characterizations of the normal and normal-Wishart distributions are provided as well. We also show how to construct a prior for every DAG model over X from the prior of a single regression model.
CFW: A Collaborative Filtering System Using Posteriors Over Weights Of Evidence
Authors: Carl Kadie, Christopher Meek, David Heckerman
Date: 2012-12-12 00:00:00
Tasks: Collaborative Filtering

We describe CFW, a computationally efficient algorithm for collaborative
filtering that uses posteriors over weights of evidence. In experiments on real
data, we show that this method predicts as well or better than other methods in
situations where the size of the user query is small. The new approach works
particularly well when the user s query CONTAINS low frequency(unpopular)
items.The approach complements that OF dependency networks which perform well
WHEN the size OF the query IS large.Also IN this paper, we argue that the USE
OF posteriors OVER weights OF evidence IS a natural way TO recommend similar
items collaborative - filtering task.
An MDP-based Recommender System
Authors: Guy Shani, Ronen I. Brafman, David Heckerman
Date: 2012-12-12 00:00:00
Tasks: Recommendation Systems

Typical Recommender systems adopt a static view of the recommendation process
and treat it as a prediction problem. We argue that it is more appropriate to
view the problem of generating recommendations as a sequential decision problem
and, consequently, that Markov decision processes (MDP) provide a more
appropriate model for Recommender systems. MDPs introduce two benefits: they
take into account the long-term effects of each recommendation, and they take
into account the expected value of each recommendation. To succeed in practice,
an MDP-based Recommender system must employ a strong initial model; and the
bulk of this paper is concerned with the generation of such a model. In
particular, we suggest the use of an n-gram predictive model for generating the
initial MDP. Our n-gram model induces a Markov-chain model of user behavior
whose predictive accuracy is greater than that of existing predictive models.
We describe our predictive model in detail and evaluate its performance on real
data. In addition, we show how the model can be used in an MDP-based
Recommender system.
Continuous Time Dynamic Topic Models
Authors: Chong Wang, David Blei, David Heckerman
Date: 2012-06-13 00:00:00
Tasks: Topic Models, Variational Inference

In this paper, we develop the continuous time dynamic topic model (cDTM). The
cDTM is a dynamic topic model that uses Brownian motion to model the latent
topics through a sequential collection of documents, where a "topic" is a
pattern of word use that we expect to evolve over the course of the collection.
We derive an efficient variational approximate inference algorithm that takes
advantage of the sparsity of observations in text, a property that lets us
easily handle many time points. In contrast to the cDTM, the original
discrete-time dynamic topic model (dDTM) requires that time be discretized.
Moreover, the complexity of variational inference for the dDTM grows quickly as
time granularity increases, a drawback which limits fine-grained
discretization. We demonstrate the cDTM on two news corpora, reporting both
predictive perplexity and the novel task of time stamp prediction.
Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity
Authors: Sham M. Kakade, Ohad Shamir, Karthik Sridharan, Ambuj Tewari
Date: 2009-10-31 00:00:00
Tasks: Vocal Bursts Intensity Prediction

The versatility of exponential families, along with their attendant convexity
properties, make them a popular and effective statistical model. A central
issue is learning these models in high-dimensions, such as when there is some
sparsity pattern of the optimal parameter. This work characterizes a certain
strong convexity property of general exponential families, which allow their
generalization ability to be quantified. In particular, we show how this
property can be used to analyze generic exponential families under L_1
regularization.
A type-theoretical approach to Universal Grammar
Authors: Erkki Luuk
Date: 2015-05-16 00:00:00
Tasks: Vocal Bursts Type Prediction

The idea of Universal Grammar (UG) as the hypothetical linguistic structure
shared by all human languages harkens back at least to the 13th century. The
best known modern elaborations of the idea are due to Chomsky. Following a
devastating critique from theoretical, typological and field linguistics, these
elaborations, the idea of UG itself and the more general idea of language
universals stand untenable and are largely abandoned. The proposal tackles the
hypothetical contents of UG using dependent and polymorphic type theory in a
framework very different from the Chomskyan ones. We introduce a type logic for
a precise, universal and parsimonious representation of natural language
morphosyntax and compositional semantics. The logic handles grammatical
ambiguity (with polymorphic types), selectional restrictions and diverse kinds
of anaphora (with dependent types), and features a partly universal set of
morphosyntactic types (by the Curry-Howard isomorphism).
Robust Real-time Extraction of Fiducial Facial Feature Points using Haar-like Features
Authors: Harry Commin
Date: 2015-05-16 00:00:00

In this paper, we explore methods of robustly extracting fiducial facial
feature points - an important process for numerous facial image processing
tasks. We consider various methods to first detect face, then facial features
and finally salient facial feature points. Colour-based models are analysed and
their overall unsuitability for this task is summarised. The bulk of the report
is then dedicated to proposing a learning-based method centred on the
Viola-Jones algorithm. The specific difficulties and considerations relating to
feature point detection are laid out in this context and a novel approach is
established to address these issues. On a sequence of clear and unobstructed
face images, our proposed system achieves average detection rates of over 90%.
Then, using a more varied sample dataset, we identify some possible areas for
future development of our system.
Cognitive Development of the Web
Authors: Viktoras Veitas, David Weinbaum
Date: 2015-05-16 00:00:00
Tasks: Cultural Vocal Bursts Intensity Prediction

The sociotechnological system is a system constituted of human individuals
and their artifacts: technological artifacts, institutions, conceptual and
representational systems, worldviews, knowledge systems, culture and the whole
biosphere as a volutionary niche. In our view the sociotechnological system as
a super-organism is shaped and determined both by the characteristics of the
agents involved and the characteristics emergent in their interactions at
multiple scales. Our approach to sociotechnological dynamics will maintain a
balance between perspectives: the individual and the collective. Accordingly,
we analyze dynamics of the Web as a sociotechnological system made of people,
computers and digital artifacts (Web pages, databases, search engines, etc.).
Making sense of the sociotechnological system while being part of it, is also a
constant interplay between pragmatic and value based approaches. The first is
focusing on the actualities of the system while the second highlights the
observer's projections. In our attempt to model sociotechnological dynamics and
envision its future, we take special care to make explicit our values as part
of the analysis. In sociotechnological systems with a high degree of
reflexivity (coupling between the perception of the system and the system's
behavior), highlighting values is of critical importance. In this essay, we
choose to see the future evolution of the web as facilitating a basic value,
that is, continuous open-ended intelligence expansion. By that we mean that we
see intelligence expansion as the determinant of the 'greater good' and 'well
being' of both of individuals and collectives at all scales. Our working
definition of intelligence here is the progressive process of sense-making of
self, other, environment and universe. Intelligence expansion, therefore, means
an increasing ability of sense-making.
The color of smiling: computational synaesthesia of facial expressions
Authors: Vittorio Cuculo, Raffaella Lanzarotti, Giuseppe Boccignone
Date: 2015-05-16 00:00:00

This note gives a preliminary account of the transcoding or rechanneling
problem between different stimuli as it is of interest for the natural
interaction or affective computing fields. By the consideration of a simple
example, namely the color response of an affective lamp to a sensed facial
expression, we frame the problem within an information- theoretic perspective.
A full justification in terms of the Information Bottleneck principle promotes
a latent affective space, hitherto surmised as an appealing and intuitive
solution, as a suitable mediator between the different stimuli.
A New Perspective on Boosting in Linear Regression via Subgradient Optimization and Relatives
Authors: Robert M. Freund, Paul Grigas, Rahul Mazumder
Date: 2015-05-16 00:00:00
Tasks: regression

In this paper we analyze boosting algorithms in linear regression from a new
perspective: that of modern first-order methods in convex optimization. We show
that classic boosting algorithms in linear regression, namely the incremental
forward stagewise algorithm (FS$_\varepsilon$) and least squares boosting
(LS-Boost($\varepsilon$)), can be viewed as subgradient descent to minimize the
loss function defined as the maximum absolute correlation between the features
and residuals. We also propose a modification of FS$_\varepsilon$ that yields
an algorithm for the Lasso, and that may be easily extended to an algorithm
that computes the Lasso path for different values of the regularization
parameter. Furthermore, we show that these new algorithms for the Lasso may
also be interpreted as the same master algorithm (subgradient descent), applied
to a regularized version of the maximum absolute correlation loss function. We
derive novel, comprehensive computational guarantees for several boosting
algorithms in linear regression (including LS-Boost($\varepsilon$) and
FS$_\varepsilon$) by using techniques of modern first-order methods in convex
optimization. Our computational guarantees inform us about the statistical
properties of boosting algorithms. In particular they provide, for the first
time, a precise theoretical description of the amount of data-fidelity and
regularization imparted by running a boosting algorithm with a prespecified
learning rate for a fixed but arbitrary number of iterations, for any dataset.
Automatic Photo Adjustment Using Deep Neural Networks
Authors: Zhicheng Yan, Hao Zhang, Baoyuan Wang, Sylvain Paris, Yizhou Yu
Date: 2014-12-24 00:00:00
Tasks: Photo Retouching

Photo retouching enables photographers to invoke dramatic visual impressions
by artistically enhancing their photos through stylistic color and tone
adjustments. However, it is also a time-consuming and challenging task that
requires advanced skills beyond the abilities of casual photographers. Using an
automated algorithm is an appealing alternative to manual work but such an
algorithm faces many hurdles. Many photographic styles rely on subtle
adjustments that depend on the image content and even its semantics. Further,
these adjustments are often spatially varying. Because of these
characteristics, existing automatic algorithms are still limited and cover only
a subset of these challenges. Recently, deep machine learning has shown unique
abilities to address hard problems that resisted machine algorithms for long.
This motivated us to explore the use of deep learning in the context of photo
editing. In this paper, we explain how to formulate the automatic photo
adjustment problem in a way suitable for this approach. We also introduce an
image descriptor that accounts for the local semantics of an image. Our
experiments demonstrate that our deep learning formulation applied using these
descriptors successfully capture sophisticated photographic styles. In
particular and unlike previous techniques, it can model local adjustments that
depend on the image semantics. We show on several examples that this yields
results that are qualitatively and quantitatively better than previous work.