pwc-archive/papers-with-abstracts huggingface dataset:
Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models
Authors: Michael U. Gutmann, Jukka Corander
Date: 2015-01-14 00:00:00
Tasks: Bayesian Optimization

Our paper deals with inferring simulator-based statistical models given some
observed data. A simulator-based model is a parametrized mechanism which
specifies how data are generated. It is thus also referred to as generative
model. We assume that only a finite number of parameters are of interest and
allow the generative process to be very general; it may be a noisy nonlinear
dynamical system with an unrestricted number of hidden variables. This weak
assumption is useful for devising realistic models but it renders statistical
inference very difficult. The main challenge is the intractability of the
likelihood function. Several likelihood-free inference methods have been
proposed which share the basic idea of identifying the parameters by finding
values for which the discrepancy between simulated and observed data is small.
A major obstacle to using these methods is their computational cost. The cost
is largely due to the need to repeatedly simulate data sets and the lack of
knowledge about how the parameters affect the discrepancy. We propose a
strategy which combines probabilistic modeling of the discrepancy with
optimization to facilitate likelihood-free inference. The strategy is
implemented using Bayesian optimization and is shown to accelerate the
inference through a reduction in the number of required simulations by several
orders of magnitude.
Strategies and Principles of Distributed Machine Learning on Big Data
Authors: Eric P. Xing, Qirong Ho, Pengtao Xie, Wei Dai
Date: 2015-12-31 00:00:00
Tasks: BIG-bench Machine Learning

The rise of Big Data has led to new demands for Machine Learning (ML) systems
to learn complex models with millions to billions of parameters, that promise
adequate capacity to digest massive datasets and offer powerful predictive
analytics thereupon. In order to run ML algorithms at such scales, on a
distributed cluster with 10s to 1000s of machines, it is often the case that
significant engineering efforts are required --- and one might fairly ask if
such engineering truly falls within the domain of ML research or not. Taking
the view that Big ML systems can benefit greatly from ML-rooted statistical and
algorithmic insights --- and that ML researchers should therefore not shy away
from such systems design --- we discuss a series of principles and strategies
distilled from our recent efforts on industrial-scale ML solutions. These
principles and strategies span a continuum from application, to engineering,
and to theoretical research and development of Big ML systems and
architectures, with the goal of understanding how to make them efficient,
generally-applicable, and supported with convergence and scaling guarantees.
They concern four key questions which traditionally receive little attention in
ML research: How to distribute an ML program over a cluster? How to bridge ML
computation with inter-machine communication? How to perform such
communication? What should be communicated between machines? By exposing
underlying statistical and algorithmic characteristics unique to ML programs
but not typically seen in traditional computer programs, and by dissecting
successful cases to reveal how we have harnessed these principles to design and
develop both high-performance distributed ML software as well as
general-purpose ML frameworks, we present opportunities for ML researchers and
practitioners to further shape and grow the area that lies between ML and
systems.
Evolving Non-linear Stacking Ensembles for Prediction of Go Player Attributes
Authors: Josef Moudřík, Roman Neruda
Date: 2015-12-31 00:00:00
Tasks: Prediction

The paper presents an application of non-linear stacking ensembles for
prediction of Go player attributes. An evolutionary algorithm is used to form a
diverse ensemble of base learners, which are then aggregated by a stacking
ensemble. This methodology allows for an efficient prediction of different
attributes of Go players from sets of their games. These attributes can be
fairly general, in this work, we used the strength and style of the players.
Solving the G-problems in less than 500 iterations: Improved efficient constrained optimization by surrogate modeling and adaptive parameter control
Authors: Samineh Bagheri, Wolfgang Konen, Michael Emmerich, Thomas Bäck
Date: 2015-12-31 00:00:00

Constrained optimization of high-dimensional numerical problems plays an
important role in many scientific and industrial applications. Function
evaluations in many industrial applications are severely limited and no
analytical information about objective function and constraint functions is
available. For such expensive black-box optimization tasks, the constraint
optimization algorithm COBRA was proposed, making use of RBF surrogate modeling
for both the objective and the constraint functions. COBRA has shown remarkable
success in solving reliably complex benchmark problems in less than 500
function evaluations. Unfortunately, COBRA requires careful adjustment of
parameters in order to do so.
  In this work we present a new self-adjusting algorithm SACOBRA, which is
based on COBRA and capable to achieve high-quality results with very few
function evaluations and no parameter tuning. It is shown with the help of
performance profiles on a set of benchmark problems (G-problems, MOPTA08) that
SACOBRA consistently outperforms any COBRA algorithm with fixed parameter
setting. We analyze the importance of the several new elements in SACOBRA and
find that each element of SACOBRA plays a role to boost up the overall
optimization performance. We discuss the reasons behind and get in this way a
better understanding of high-quality RBF surrogate modeling.
Denoising and Completion of 3D Data via Multidimensional Dictionary Learning
Authors: Zemin Zhang, Shuchin Aeron
Date: 2015-12-31 00:00:00
Tasks: Denoising, Dictionary Learning, Image Denoising

In this paper a new dictionary learning algorithm for multidimensional data
is proposed. Unlike most conventional dictionary learning methods which are
derived for dealing with vectors or matrices, our algorithm, named KTSVD,
learns a multidimensional dictionary directly via a novel algebraic approach
for tensor factorization as proposed in [3, 12, 13]. Using this approach one
can define a tensor-SVD and we propose to extend K-SVD algorithm used for 1-D
data to a K-TSVD algorithm for handling 2-D and 3-D data. Our algorithm, based
on the idea of sparse coding (using group-sparsity over multidimensional
coefficient vectors), alternates between estimating a compact representation
and dictionary learning. We analyze our KTSVD algorithm and demonstrate its
result on video completion and multispectral image denoising.
Recovery of Sparse Signals Using Multiple Orthogonal Least Squares
Authors: Jian Wang, Ping Li
Date: 2014-10-09 00:00:00
Tasks: Computational Efficiency

We study the problem of recovering sparse signals from compressed linear
measurements. This problem, often referred to as sparse recovery or sparse
reconstruction, has generated a great deal of interest in recent years. To
recover the sparse signals, we propose a new method called multiple orthogonal
least squares (MOLS), which extends the well-known orthogonal least squares
(OLS) algorithm by allowing multiple $L$ indices to be chosen per iteration.
Owing to inclusion of multiple support indices in each selection, the MOLS
algorithm converges in much fewer iterations and improves the computational
efficiency over the conventional OLS algorithm. Theoretical analysis shows that
MOLS ($L > 1$) performs exact recovery of all $K$-sparse signals within $K$
iterations if the measurement matrix satisfies the restricted isometry property
(RIP) with isometry constant $\delta_{LK} < \frac{\sqrt{L}}{\sqrt{K} + 2
\sqrt{L}}.$ The recovery performance of MOLS in the noisy scenario is also
studied. It is shown that stable recovery of sparse signals can be achieved
with the MOLS algorithm when the signal-to-noise ratio (SNR) scales linearly
with the sparsity level of input signals.
Nonparametric mixture of Gaussian graphical models
Authors: Kevin Lee, Lingzhou Xue
Date: 2015-12-31 00:00:00
Tasks: Functional Connectivity

Graphical model has been widely used to investigate the complex dependence
structure of high-dimensional data, and it is common to assume that observed
data follow a homogeneous graphical model. However, observations usually come
from different resources and have heterogeneous hidden commonality in
real-world applications. Thus, it is of great importance to estimate
heterogeneous dependencies and discover subpopulation with certain commonality
across the whole population. In this work, we introduce a novel regularized
estimation scheme for learning nonparametric mixture of Gaussian graphical
models, which extends the methodology and applicability of Gaussian graphical
models and mixture models. We propose a unified penalized likelihood approach
to effectively estimate nonparametric functional parameters and heterogeneous
graphical parameters. We further design an efficient generalized effective EM
algorithm to address three significant challenges: high-dimensionality,
non-convexity, and label switching. Theoretically, we study both the
algorithmic convergence of our proposed algorithm and the asymptotic properties
of our proposed estimators. Numerically, we demonstrate the performance of our
method in simulation studies and a real application to estimate human brain
functional connectivity from ADHD imaging data, where two heterogeneous
conditional dependencies are explained through profiling demographic variables
and supported by existing scientific findings.
Bayes-Optimal Effort Allocation in Crowdsourcing: Bounds and Index Policies
Authors: Weici Hu, Peter I. Frazier
Date: 2015-12-31 00:00:00

We consider effort allocation in crowdsourcing, where we wish to assign
labeling tasks to imperfect homogeneous crowd workers to maximize overall
accuracy in a continuous-time Bayesian setting, subject to budget and time
constraints. The Bayes-optimal policy for this problem is the solution to a
partially observable Markov decision process, but the curse of dimensionality
renders the computation infeasible. Based on the Lagrangian Relaxation
technique in Adelman & Mersereau (2008), we provide a computationally tractable
instance-specific upper bound on the value of this Bayes-optimal policy, which
can in turn be used to bound the optimality gap of any other sub-optimal
policy. In an approach similar in spirit to the Whittle index for restless
multiarmed bandits, we provide an index policy for effort allocation in
crowdsourcing and demonstrate numerically that it outperforms other stateof-
arts and performs close to optimal solution.
Benders Decomposition for the Design of a Hub and Shuttle Public Transit System
Authors: Arthur Maheo, Philip Kilby, Pascal Van Hentenryck
Date: 2015-12-30 00:00:00

The BusPlus project aims at improving the off-peak hours public transit
service in Canberra, Australia. To address the difficulty of covering a large
geographic area, BusPlus proposes a hub and shuttle model consisting of a
combination of a few high-frequency bus routes between key hubs and a large
number of shuttles that bring passengers from their origin to the closest hub
and take them from their last bus stop to their destination. This paper focuses
on the design of bus network and proposes an efficient solving method to this
multimodal network design problem based on the Benders decomposition method.
Starting from a MIP formulation of the problem, the paper presents a Benders
decomposition approach using dedicated solution techniques for solving
independent sub-problems, Pareto optimal cuts, cut bundling, and core point
update. Computational results on real-world data from Canberra's public transit
system justify the design choices and show that the approach outperforms the
MIP formulation by two orders of magnitude. Moreover, the results show that the
hub and shuttle model may decrease transit time by a factor of 2, while staying
within the costs of the existing transit system.
LIBSVX: A Supervoxel Library and Benchmark for Early Video Processing
Authors: Chenliang Xu, Jason J. Corso
Date: 2015-12-30 00:00:00
Tasks: Boundary Detection, Segmentation, Superpixels

Supervoxel segmentation has strong potential to be incorporated into early
video analysis as superpixel segmentation has in image analysis. However, there
are many plausible supervoxel methods and little understanding as to when and
where each is most appropriate. Indeed, we are not aware of a single
comparative study on supervoxel segmentation. To that end, we study seven
supervoxel algorithms, including both off-line and streaming methods, in the
context of what we consider to be a good supervoxel: namely, spatiotemporal
uniformity, object/region boundary detection, region compression and parsimony.
For the evaluation we propose a comprehensive suite of seven quality metrics to
measure these desirable supervoxel characteristics. In addition, we evaluate
the methods in a supervoxel classification task as a proxy for subsequent
high-level uses of the supervoxels in video analysis. We use six existing
benchmark video datasets with a variety of content-types and dense human
annotations. Our findings have led us to conclusive evidence that the
hierarchical graph-based (GBH), segmentation by weighted aggregation (SWA) and
temporal superpixels (TSP) methods are the top-performers among the seven
methods. They all perform well in terms of segmentation accuracy, but vary in
regard to the other desiderata: GBH captures object boundaries best; SWA has
the best potential for region compression; and TSP achieves the best
undersegmentation error.
Actor-Action Semantic Segmentation with Grouping Process Models
Authors: Chenliang Xu, Jason J. Corso
Date: 2015-12-30 00:00:00
Proceeding: CVPR 2016 6
Tasks: Semantic Segmentation, Video Understanding

Actor-action semantic segmentation made an important step toward advanced
video understanding problems: what action is happening; who is performing the
action; and where is the action in space-time. Current models for this problem
are local, based on layered CRFs, and are unable to capture long-ranging
interaction of video parts. We propose a new model that combines these local
labeling CRFs with a hierarchical supervoxel decomposition. The supervoxels
provide cues for possible groupings of nodes, at various scales, in the CRFs to
encourage adaptive, high-order groups for more effective labeling. Our model is
dynamic and continuously exchanges information during inference: the local CRFs
influence what supervoxels in the hierarchy are active, and these active nodes
influence the connectivity in the CRF; we hence call it a grouping process
model. The experimental results on a recent large-scale video dataset show a
large margin of 60% relative improvement over the state of the art, which
demonstrates the effectiveness of the dynamic, bidirectional flow between
labeling and grouping.
Recursive Partitioning for Heterogeneous Causal Effects
Authors: Susan Athey, Guido Imbens
Date: 2015-04-05 00:00:00
Tasks: Re-Ranking

In this paper we study the problems of estimating heterogeneity in causal
effects in experimental or observational studies and conducting inference about
the magnitude of the differences in treatment effects across subsets of the
population. In applications, our method provides a data-driven approach to
determine which subpopulations have large or small treatment effects and to
test hypotheses about the differences in these effects. For experiments, our
method allows researchers to identify heterogeneity in treatment effects that
was not specified in a pre-analysis plan, without concern about invalidating
inference due to multiple testing. In most of the literature on supervised
machine learning (e.g. regression trees, random forests, LASSO, etc.), the goal
is to build a model of the relationship between a unit's attributes and an
observed outcome. A prominent role in these methods is played by
cross-validation which compares predictions to actual outcomes in test samples,
in order to select the level of complexity of the model that provides the best
predictive power. Our method is closely related, but it differs in that it is
tailored for predicting causal effects of a treatment rather than a unit's
outcome. The challenge is that the "ground truth" for a causal effect is not
observed for any individual unit: we observe the unit with the treatment, or
without the treatment, but not both at the same time. Thus, it is not obvious
how to use cross-validation to determine whether a causal effect has been
accurately predicted. We propose several novel cross-validation criteria for
this problem and demonstrate through simulations the conditions under which
they perform better than standard methods for the problem of causal effects. We
then apply the method to a large-scale field experiment re-ranking results on a
search engine.
Nonparametric Bayesian Factor Analysis for Dynamic Count Matrices
Authors: Ayan Acharya, Joydeep Ghosh, Mingyuan Zhou
Date: 2015-12-30 00:00:00
Tasks: Data Augmentation

A gamma process dynamic Poisson factor analysis model is proposed to
factorize a dynamic count matrix, whose columns are sequentially observed count
vectors. The model builds a novel Markov chain that sends the latent gamma
random variables at time $(t-1)$ as the shape parameters of those at time $t$,
which are linked to observed or latent counts under the Poisson likelihood. The
significant challenge of inferring the gamma shape parameters is fully
addressed, using unique data augmentation and marginalization techniques for
the negative binomial distribution. The same nonparametric Bayesian model also
applies to the factorization of a dynamic binary matrix, via a
Bernoulli-Poisson link that connects a binary observation to a latent count,
with closed-form conditional posteriors for the latent counts and efficient
computation for sparse observations. We apply the model to text and music
analysis, with state-of-the-art results.
Infinite Edge Partition Models for Overlapping Community Detection and Link Prediction
Authors: Mingyuan Zhou
Date: 2015-01-25 00:00:00
Tasks: Community Detection, Data Augmentation, Link Prediction

A hierarchical gamma process infinite edge partition model is proposed to
factorize the binary adjacency matrix of an unweighted undirected relational
network under a Bernoulli-Poisson link. The model describes both homophily and
stochastic equivalence, and is scalable to big sparse networks by focusing its
computation on pairs of linked nodes. It can not only discover overlapping
communities and inter-community interactions, but also predict missing edges. A
simplified version omitting inter-community interactions is also provided and
we reveal its interesting connections to existing models. The number of
communities is automatically inferred in a nonparametric Bayesian manner, and
efficient inference via Gibbs sampling is derived using novel data augmentation
techniques. Experimental results on four real networks demonstrate the models'
scalability and state-of-the-art performance.
Technical Report: a tool for measuring Prosodic Accommodation
Authors: Sucheta Ghosh
Date: 2015-12-30 00:00:00

This article has been withdrawn by arXiv administrators because the submitter
did not have the legal authority to grant the license applied to the work.
The Poisson Gamma Belief Network
Authors: Mingyuan Zhou, Yulai Cong, Bo Chen
Date: 2015-11-06 00:00:00
Proceeding: NeurIPS 2015 12

To infer a multilayer representation of high-dimensional count vectors, we
propose the Poisson gamma belief network (PGBN) that factorizes each of its
layers into the product of a connection weight matrix and the nonnegative real
hidden units of the next layer. The PGBN's hidden layers are jointly trained
with an upward-downward Gibbs sampler, each iteration of which upward samples
Dirichlet distributed connection weight vectors starting from the first layer
(bottom data layer), and then downward samples gamma distributed hidden units
starting from the top hidden layer. The gamma-negative binomial process
combined with a layer-wise training strategy allows the PGBN to infer the width
of each layer given a fixed budget on the width of the first layer. The PGBN
with a single hidden layer reduces to Poisson factor analysis. Example results
on text analysis illustrate interesting relationships between the width of the
first layer and the inferred network structure, and demonstrate that the PGBN,
whose hidden units are imposed with correlated gamma priors, can add more
layers to increase its performance gains over Poisson factor analysis, given
the same limit on the width of the first layer.
Evaluating Go Game Records for Prediction of Player Attributes
Authors: Josef Moudřík, Petr Baudiš, Roman Neruda
Date: 2015-12-30 00:00:00
Tasks: BIG-bench Machine Learning, Prediction

We propose a way of extracting and aggregating per-move evaluations from sets
of Go game records. The evaluations capture different aspects of the games such
as played patterns or statistic of sente/gote sequences. Using machine learning
algorithms, the evaluations can be utilized to predict different relevant
target variables. We apply this methodology to predict the strength and playing
style of the player (e.g. territoriality or aggressivity) with good accuracy.
We propose a number of possible applications including aiding in Go study,
seeding real-work ranks of internet players or tuning of Go-playing programs.
Assessing binary classifiers using only positive and unlabeled data
Authors: Marc Claesen, Jesse Davis, Frank De Smet, Bart De Moor
Date: 2015-04-26 00:00:00

Assessing the performance of a learned model is a crucial part of machine
learning. However, in some domains only positive and unlabeled examples are
available, which prohibits the use of most standard evaluation metrics. We
propose an approach to estimate any metric based on contingency tables,
including ROC and PR curves, using only positive and unlabeled data. Estimating
these performance metrics is essentially reduced to estimating the fraction of
(latent) positives in the unlabeled set, assuming known positives are a random
sample of all positives. We provide theoretical bounds on the quality of our
estimates, illustrate the importance of estimating the fraction of positives in
the unlabeled set and demonstrate empirically that we are able to reliably
estimate ROC and PR curves on real data.
Online Keyword Spotting with a Character-Level Recurrent Neural Network
Authors: Kyuyeon Hwang, Minjae Lee, Wonyong Sung
Date: 2015-12-30 00:00:00
Tasks: General Classification, Keyword Spotting

In this paper, we propose a context-aware keyword spotting model employing a
character-level recurrent neural network (RNN) for spoken term detection in
continuous speech. The RNN is end-to-end trained with connectionist temporal
classification (CTC) to generate the probabilities of character and
word-boundary labels. There is no need for the phonetic transcription, senone
modeling, or system dictionary in training and testing. Also, keywords can
easily be added and modified by editing the text based keyword list without
retraining the RNN. Moreover, the unidirectional RNN processes an infinitely
long input audio streams without pre-segmentation and keywords are detected
with low-latency before the utterance is finished. Experimental results show
that the proposed keyword spotter significantly outperforms the deep neural
network (DNN) and hidden Markov model (HMM) based keyword-filler model even
with less computations.