pwc-archive/papers-with-abstracts huggingface dataset:
Sparsistency and agnostic inference in sparse PCA
Authors: Jing Lei, Vincent Q. Vu
Date: 2014-01-27 00:00:00

The presence of a sparse "truth" has been a constant assumption in the
theoretical analysis of sparse PCA and is often implicit in its methodological
development. This naturally raises questions about the properties of sparse PCA
methods and how they depend on the assumption of sparsity. Under what
conditions can the relevant variables be selected consistently if the truth is
assumed to be sparse? What can be said about the results of sparse PCA without
assuming a sparse and unique truth? We answer these questions by investigating
the properties of the recently proposed Fantope projection and selection (FPS)
method in the high-dimensional setting. Our results provide general sufficient
conditions for sparsistency of the FPS estimator. These conditions are weak and
can hold in situations where other estimators are known to fail. On the other
hand, without assuming sparsity or identifiability, we show that FPS provides a
sparse, linear dimension-reducing transformation that is close to the best
possible in terms of maximizing the predictive covariance.
Convex Optimization for Parallel Energy Minimization
Authors: K. S. Sesh Kumar, Alvaro Barbero, Stefanie Jegelka, Suvrit Sra, Francis Bach
Date: 2015-03-05 00:00:00
Tasks: Denoising

Energy minimization has been an intensely studied core problem in computer
vision. With growing image sizes (2D and 3D), it is now highly desirable to run
energy minimization algorithms in parallel. But many existing algorithms, in
particular, some efficient combinatorial algorithms, are difficult to
par-allelize. By exploiting results from convex and submodular theory, we
reformulate the quadratic energy minimization problem as a total variation
denoising problem, which, when viewed geometrically, enables the use of
projection and reflection based convex methods. The resulting min-cut algorithm
(and code) is conceptually very simple, and solves a sequence of TV denoising
problems. We perform an extensive empirical evaluation comparing
state-of-the-art combinatorial algorithms and convex optimization techniques.
On small problems the iterative convex methods match the combinatorial max-flow
algorithms, while on larger problems they offer other flexibility and important
gains: (a) their memory footprint is small; (b) their straightforward
parallelizability fits multi-core platforms; (c) they can easily be
warm-started; and (d) they quickly reach approximately good solutions, thereby
enabling faster "inexact" solutions. A key consequence of our approach based on
submodularity and convexity is that it is allows to combine any arbitrary
combinatorial or convex methods as subroutines, which allows one to obtain
hybrid combinatorial and convex optimization algorithms that benefit from the
strengths of both.
Visualization of Clandestine Labs from Seizure Reports: Thematic Mapping and Data Mining Research Directions
Authors: William Hsu, Mohammed Abduljabbar, Ryuichi Osuga, Max Lu, Wesam Elshamy
Date: 2015-03-05 00:00:00
Tasks: Dynamic Topic Modeling, Event Extraction, Information Retrieval, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Relationship Extraction (Distant Supervised), Retrieval

The problem of spatiotemporal event visualization based on reports entails
subtasks ranging from named entity recognition to relationship extraction and
mapping of events. We present an approach to event extraction that is driven by
data mining and visualization goals, particularly thematic mapping and trend
analysis. This paper focuses on bridging the information extraction and
visualization tasks and investigates topic modeling approaches. We develop a
static, finite topic model and examine the potential benefits and feasibility
of extending this to dynamic topic modeling with a large number of topics and
continuous time. We describe an experimental test bed for event mapping that
uses this end-to-end information retrieval system, and report preliminary
results on a geoinformatics problem: tracking of methamphetamine lab seizure
events across time and space.
A Statistical Parsing Framework for Sentiment Classification
Authors: Li Dong, Furu Wei, Shujie Liu, Ming Zhou, Ke Xu
Date: 2014-01-24 00:00:00
Proceeding: CL 2015 6
Tasks: Classification, General Classification, Negation, Sentence, Sentiment Analysis, Sentiment Classification

We present a statistical parsing framework for sentence-level sentiment
classification in this article. Unlike previous works that employ syntactic
parsing results for sentiment analysis, we develop a statistical parser to
directly analyze the sentiment structure of a sentence. We show that
complicated phenomena in sentiment analysis (e.g., negation, intensification,
and contrast) can be handled the same as simple and straightforward sentiment
expressions in a unified and probabilistic way. We formulate the sentiment
grammar upon Context-Free Grammars (CFGs), and provide a formal description of
the sentiment parsing framework. We develop the parsing model to obtain
possible sentiment parse trees for a sentence, from which the polarity model is
proposed to derive the sentiment strength and polarity, and the ranking model
is dedicated to selecting the best sentiment tree. We train the parser directly
from examples of sentences annotated only with sentiment polarity labels but
without any syntactic annotations or polarity annotations of constituents
within sentences. Therefore we can obtain training data easily. In particular,
we train a sentiment parser, s.parser, from a large amount of review sentences
with users' ratings as rough sentiment polarity labels. Extensive experiments
on existing benchmark datasets show significant improvements over baseline
sentiment classification approaches.
Learning to rank in person re-identification with metric ensembles
Authors: Sakrapee Paisitkriangkrai, Chunhua Shen, Anton Van Den Hengel
Date: 2015-03-05 00:00:00
Proceeding: CVPR 2015 6
Tasks: Learning-To-Rank, Person Re-Identification

We propose an effective structured learning based approach to the problem of
person re-identification which outperforms the current state-of-the-art on most
benchmark data sets evaluated. Our framework is built on the basis of multiple
low-level hand-crafted and high-level visual features. We then formulate two
optimization algorithms, which directly optimize evaluation measures commonly
used in person re-identification, also known as the Cumulative Matching
Characteristic (CMC) curve. Our new approach is practical to many real-world
surveillance applications as the re-identification performance can be
concentrated in the range of most practical importance. The combination of
these factors leads to a person re-identification system which outperforms most
existing algorithms. More importantly, we advance state-of-the-art results on
person re-identification by improving the rank-$1$ recognition rates from
$40\%$ to $50\%$ on the iLIDS benchmark, $16\%$ to $18\%$ on the PRID2011
benchmark, $43\%$ to $46\%$ on the VIPeR benchmark, $34\%$ to $53\%$ on the
CUHK01 benchmark and $21\%$ to $62\%$ on the CUHK03 benchmark.
Pyrcca: regularized kernel canonical correlation analysis in Python and its applications to neuroimaging
Authors: Natalia Y. Bilenko, Jack L. Gallant
Date: 2015-03-05 00:00:00
Tasks: Anatomy

Canonical correlation analysis (CCA) is a valuable method for interpreting
cross-covariance across related datasets of different dimensionality. There are
many potential applications of CCA to neuroimaging data analysis. For instance,
CCA can be used for finding functional similarities across fMRI datasets
collected from multiple subjects without resampling individual datasets to a
template anatomy. In this paper, we introduce Pyrcca, an open-source Python
module for executing CCA between two or more datasets. Pyrcca can be used to
implement CCA with or without regularization, and with or without linear or a
Gaussian kernelization of the datasets. We demonstrate an application of CCA
implemented with Pyrcca to neuroimaging data analysis. We use CCA to find a
data-driven set of functional response patterns that are similar across
individual subjects in a natural movie experiment. We then demonstrate how this
set of response patterns discovered by CCA can be used to accurately predict
subject responses to novel natural movie stimuli.
Deep Temporal Appearance-Geometry Network for Facial Expression Recognition
Authors: Heechul Jung, Sihaeng Lee, Sunjeong Park, Injae Lee, Chunghyun Ahn, Junmo Kim
Date: 2015-03-05 00:00:00
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER)

Temporal information can provide useful features for recognizing facial
expressions. However, to manually design useful features requires a lot of
effort. In this paper, to reduce this effort, a deep learning technique which
is regarded as a tool to automatically extract useful features from raw data,
is adopted. Our deep network is based on two different models. The first deep
network extracts temporal geometry features from temporal facial landmark
points, while the other deep network extracts temporal appearance features from
image sequences . These two models are combined in order to boost the
performance of the facial expression recognition. Through several experiments,
we showed that the two models cooperate with each other. As a result, we
achieved superior performance to other state-of-the-art methods in CK+ and
Oulu-CASIA databases. Furthermore, one of the main contributions of this paper
is that our deep network catches the facial action points automatically.
Spectral Clustering by Ellipsoid and Its Connection to Separable Nonnegative Matrix Factorization
Authors: Tomohiko Mizutani
Date: 2015-03-05 00:00:00
Tasks: Clustering

This paper proposes a variant of the normalized cut algorithm for spectral
clustering. Although the normalized cut algorithm applies the K-means algorithm
to the eigenvectors of a normalized graph Laplacian for finding clusters, our
algorithm instead uses a minimum volume enclosing ellipsoid for them. We show
that the algorithm shares similarity with the ellipsoidal rounding algorithm
for separable nonnegative matrix factorization. Our theoretical insight implies
that the algorithm can serve as a bridge between spectral clustering and
separable NMF. The K-means algorithm has the issues in that the choice of
initial points affects the construction of clusters and certain choices result
in poor clustering performance. The normalized cut algorithm inherits these
issues since K-means is incorporated in it, whereas the algorithm proposed here
does not. An empirical study is presented to examine the performance of the
algorithm.
Genetic optimization of the Hyperloop route through the Grapevine
Authors: Casey J. Handmer
Date: 2015-03-05 00:00:00

We demonstrate a genetic algorithm that employs a versatile fitness function
to optimize route selection for the Hyperloop, a proposed high speed passenger
transportation system.
Dependence space of matroids and its application to attribute reduction
Authors: Aiping Huang, William Zhu
Date: 2013-12-16 00:00:00
Tasks: Attribute

Attribute reduction is a basic issue in knowledge representation and data
mining. Rough sets provide a theoretical foundation for the issue. Matroids
generalized from matrices have been widely used in many fields, particularly
greedy algorithm design, which plays an important role in attribute reduction.
Therefore, it is meaningful to combine matroids with rough sets to solve the
optimization problems. In this paper, we introduce an existing algebraic
structure called dependence space to study the reduction problem in terms of
matroids. First, a dependence space of matroids is constructed. Second, the
characterizations for the space such as consistent sets and reducts are studied
through matroids. Finally, we investigate matroids by the means of the space
and present two expressions for their bases. In a word, this paper provides new
approaches to study attribute reduction.
Do We Need More Training Data?
Authors: Xiangxin Zhu, Carl Vondrick, Charless Fowlkes, Deva Ramanan
Date: 2015-03-05 00:00:00
Tasks: Object Recognition

Datasets for training object recognition systems are steadily increasing in
size. This paper investigates the question of whether existing detectors will
continue to improve as data grows, or saturate in performance due to limited
model complexity and the Bayes risk associated with the feature spaces in which
they operate. We focus on the popular paradigm of discriminatively trained
templates defined on oriented gradient features. We investigate the performance
of mixtures of templates as the number of mixture components and the amount of
training data grows. Surprisingly, even with proper treatment of regularization
and "outliers", the performance of classic mixture models appears to saturate
quickly ($\sim$10 templates and $\sim$100 positive training examples per
template). This is not a limitation of the feature space as compositional
mixtures that share template parameters via parts and that can synthesize new
templates not encountered during training yield significantly better
performance. Based on our analysis, we conjecture that the greatest gains in
detection performance will continue to derive from improved representations and
learning algorithms that can make efficient use of large datasets.
Local Expectation Gradients for Doubly Stochastic Variational Inference
Authors: Michalis K. Titsias
Date: 2015-03-04 00:00:00
Tasks: Variational Inference

We introduce local expectation gradients which is a general purpose
stochastic variational inference algorithm for constructing stochastic
gradients through sampling from the variational distribution. This algorithm
divides the problem of estimating the stochastic gradients over multiple
variational parameters into smaller sub-tasks so that each sub-task exploits
intelligently the information coming from the most relevant part of the
variational distribution. This is achieved by performing an exact expectation
over the single random variable that mostly correlates with the variational
parameter of interest resulting in a Rao-Blackwellized estimate that has low
variance and can work efficiently for both continuous and discrete random
variables. Furthermore, the proposed algorithm has interesting similarities
with Gibbs sampling but at the same time, unlike Gibbs sampling, it can be
trivially parallelized.
Random Serial Dictatorship versus Probabilistic Serial Rule: A Tale of Two Random Mechanisms
Authors: Hadi Hosseini, Kate Larson, Robin Cohen
Date: 2015-03-04 00:00:00

For assignment problems where agents, specifying ordinal preferences, are
allocated indivisible objects, two widely studied randomized mechanisms are the
Random Serial Dictatorship (RSD) and Probabilistic Serial Rule (PS). These two
mechanisms both have desirable economic and computational properties, but the
outcomes they induce can be incomparable in many instances, thus creating
challenges in deciding which mechanism to adopt in practice. In this paper we
first look at the space of lexicographic preferences and show that, as opposed
to the general preference domain, RSD satisfies envyfreeness. Moreover, we show
that although under lexicographic preferences PS is strategyproof when the
number of objects is less than or equal agents, it is strictly manipulable when
there are more objects than agents. In the space of general preferences, we
provide empirical results on the (in)comparability of RSD and PS, analyze
economic properties, and provide further insights on the applicability of each
mechanism in different application domains.
An Entropy Search Portfolio for Bayesian Optimization
Authors: Bobak Shahriari, Ziyu Wang, Matthew W. Hoffman, Alexandre Bouchard-Côté, Nando de Freitas
Date: 2014-06-18 00:00:00
Tasks: Bayesian Optimization, global-optimization

Bayesian optimization is a sample-efficient method for black-box global
optimization. How- ever, the performance of a Bayesian optimization method very
much depends on its exploration strategy, i.e. the choice of acquisition
function, and it is not clear a priori which choice will result in superior
performance. While portfolio methods provide an effective, principled way of
combining a collection of acquisition functions, they are often based on
measures of past performance which can be misleading. To address this issue, we
introduce the Entropy Search Portfolio (ESP): a novel approach to portfolio
construction which is motivated by information theoretic considerations. We
show that ESP outperforms existing portfolio methods on several real and
synthetic problems, including geostatistical datasets and simulated control
tasks. We not only show that ESP is able to offer performance as good as the
best, but unknown, acquisition function, but surprisingly it often gives better
performance. Finally, over a wide range of conditions we find that ESP is
robust to the inclusion of poor acquisition functions.
Guaranteed Non-Orthogonal Tensor Decomposition via Alternating Rank-$1$ Updates
Authors: Animashree Anandkumar, Rong Ge, Majid Janzamin
Date: 2014-02-21 00:00:00
Tasks: Tensor Decomposition

In this paper, we provide local and global convergence guarantees for
recovering CP (Candecomp/Parafac) tensor decomposition. The main step of the
proposed algorithm is a simple alternating rank-$1$ update which is the
alternating version of the tensor power iteration adapted for asymmetric
tensors. Local convergence guarantees are established for third order tensors
of rank $k$ in $d$ dimensions, when $k=o \bigl( d^{1.5} \bigr)$ and the tensor
components are incoherent. Thus, we can recover overcomplete tensor
decomposition. We also strengthen the results to global convergence guarantees
under stricter rank condition $k \le \beta d$ (for arbitrary constant $\beta >
1$) through a simple initialization procedure where the algorithm is
initialized by top singular vectors of random tensor slices. Furthermore, the
approximate local convergence guarantees for $p$-th order tensors are also
provided under rank condition $k=o \bigl( d^{p/2} \bigr)$. The guarantees also
include tight perturbation analysis given noisy tensor.
Predicting opponent team activity in a RoboCup environment
Authors: Selene Baez
Date: 2015-03-04 00:00:00
Tasks: Position

The goal of this project is to predict the opponent's configuration in a
RoboCup SSL environment. For simplicity, a Markov model assumption is made such
that the predicted formation of the opponent team only depends on its current
formation. The field is divided into a grid and a robot state per player is
created with information about its position and its velocity. To gather a more
general sense of what the opposing team is doing, the state also incorporates
the team's average position (centroid). All possible state transitions are
stored in a hash table that requires minimum storage space. The table is
populated with transition probabilities that are learned by reading vision
packages and counting the state transitions regardless of the specific robot
player. Therefore, the computation during the game is reduced to interpreting a
given vision package to assign each player to a state, and looking for the most
likely state it will transition to. The confidence of the predicted team's
formation is the product of each individual player's probability. The project
is noteworthy in that it minimizes the time and space complexity requirements
for opponent's moves prediction.
Toxicity Prediction using Deep Learning
Authors: Thomas Unterthiner, Andreas Mayr, Günter Klambauer, Sepp Hochreiter
Date: 2015-03-04 00:00:00
Tasks: Deep Learning, Prediction

Everyday we are exposed to various chemicals via food additives, cleaning and
cosmetic products and medicines -- and some of them might be toxic. However
testing the toxicity of all existing compounds by biological experiments is
neither financially nor logistically feasible. Therefore the government
agencies NIH, EPA and FDA launched the Tox21 Data Challenge within the
"Toxicology in the 21st Century" (Tox21) initiative. The goal of this challenge
was to assess the performance of computational methods in predicting the
toxicity of chemical compounds. State of the art toxicity prediction methods
build upon specifically-designed chemical descriptors developed over decades.
Though Deep Learning is new to the field and was never applied to toxicity
prediction before, it clearly outperformed all other participating methods. In
this application paper we show that deep nets automatically learn features
resembling well-established toxicophores. In total, our Deep Learning approach
won both of the panel-challenges (nuclear receptors and stress response) as
well as the overall Grand Challenge, and thereby sets a new standard in tox
prediction.
Heteroscedastic Treed Bayesian Optimisation
Authors: John-Alexander M. Assael, Ziyu Wang, Bobak Shahriari, Nando de Freitas
Date: 2014-10-27 00:00:00
Tasks: Bayesian Optimisation, BIG-bench Machine Learning

Optimising black-box functions is important in many disciplines, such as
tuning machine learning models, robotics, finance and mining exploration.
Bayesian optimisation is a state-of-the-art technique for the global
optimisation of black-box functions which are expensive to evaluate. At the
core of this approach is a Gaussian process prior that captures our belief
about the distribution over functions. However, in many cases a single Gaussian
process is not flexible enough to capture non-stationarity in the objective
function. Consequently, heteroscedasticity negatively affects performance of
traditional Bayesian methods. In this paper, we propose a novel prior model
with hierarchical parameter learning that tackles the problem of
non-stationarity in Bayesian optimisation. Our results demonstrate substantial
improvements in a wide range of applications, including automatic machine
learning and mining exploration.
Quantifying Uncertainty in Stochastic Models with Parametric Variability
Authors: Kyle S. Hickmann, James M. Hyman, Sara Y. Del Valle
Date: 2015-03-04 00:00:00
Tasks: Uncertainty Quantification

We present a method to quantify uncertainty in the predictions made by
simulations of mathematical models that can be applied to a broad class of
stochastic, discrete, and differential equation models. Quantifying uncertainty
is crucial for determining how accurate the model predictions are and
identifying which input parameters affect the outputs of interest. Most of the
existing methods for uncertainty quantification require many samples to
generate accurate results, are unable to differentiate where the uncertainty is
coming from (e.g., parameters or model assumptions), or require a lot of
computational resources. Our approach addresses these challenges and
opportunities by allowing different types of uncertainty, that is, uncertainty
in input parameters as well as uncertainty created through stochastic model
components. This is done by combining the Karhunen-Loeve decomposition,
polynomial chaos expansion, and Bayesian Gaussian process regression to create
a statistical surrogate for the stochastic model. The surrogate separates the
analysis of variation arising through stochastic simulation and variation
arising through uncertainty in the model parameterization. We illustrate our
approach by quantifying the uncertainty in a stochastic ordinary differential
equation epidemic model. Specifically, we estimate four quantities of interest
for the epidemic model and show agreement between the surrogate and the actual
model results.