pwc-archive/papers-with-abstracts huggingface dataset:
Robust PCA via Nonconvex Rank Approximation
Authors: Zhao Kang, Chong Peng, Qiang Cheng
Date: 2015-11-17 00:00:00

Numerous applications in data mining and machine learning require recovering
a matrix of minimal rank. Robust principal component analysis (RPCA) is a
general framework for handling this kind of problems. Nuclear norm based convex
surrogate of the rank function in RPCA is widely investigated. Under certain
assumptions, it can recover the underlying true low rank matrix with high
probability. However, those assumptions may not hold in real-world
applications. Since the nuclear norm approximates the rank by adding all
singular values together, which is essentially a $\ell_1$-norm of the singular
values, the resulting approximation error is not trivial and thus the resulting
matrix estimator can be significantly biased. To seek a closer approximation
and to alleviate the above-mentioned limitations of the nuclear norm, we
propose a nonconvex rank approximation. This approximation to the matrix rank
is tighter than the nuclear norm. To solve the associated nonconvex
minimization problem, we develop an efficient augmented Lagrange multiplier
based optimization algorithm. Experimental results demonstrate that our method
outperforms current state-of-the-art algorithms in both accuracy and
efficiency.
Visualizing and Understanding Recurrent Networks
Authors: Andrej Karpathy, Justin Johnson, Li Fei-Fei
Date: 2015-06-05 00:00:00

Recurrent Neural Networks (RNNs), and specifically a variant with Long
Short-Term Memory (LSTM), are enjoying renewed interest as a result of
successful applications in a wide range of machine learning problems that
involve sequential data. However, while LSTMs provide exceptional results in
practice, the source of their performance and their limitations remain rather
poorly understood. Using character-level language models as an interpretable
testbed, we aim to bridge this gap by providing an analysis of their
representations, predictions and error types. In particular, our experiments
reveal the existence of interpretable cells that keep track of long-range
dependencies such as line lengths, quotes and brackets. Moreover, our
comparative analysis with finite horizon n-gram models traces the source of the
LSTM improvements to long-range structural dependencies. Finally, we provide
analysis of the remaining errors and suggests areas for further study.
An extension of McDiarmid's inequality
Authors: Richard Combes
Date: 2015-11-17 00:00:00

We generalize McDiarmid's inequality for functions with bounded differences on a high probability set, using an extension argument. Those functions concentrate around their conditional expectations. We further extend the results to concentration in general metric spaces.
Learning Expressionlets via Universal Manifold Model for Dynamic Facial Expression Recognition
Authors: Mengyi Liu, Shiguang Shan, Ruiping Wang, Xilin Chen
Date: 2015-11-16 00:00:00
Tasks: Dynamic Facial Expression Recognition, Facial Expression Recognition, Facial Expression Recognition (FER)

Facial expression is temporally dynamic event which can be decomposed into a
set of muscle motions occurring in different facial regions over various time
intervals. For dynamic expression recognition, two key issues, temporal
alignment and semantics-aware dynamic representation, must be taken into
account. In this paper, we attempt to solve both problems via manifold modeling
of videos based on a novel mid-level representation, i.e.
\textbf{expressionlet}. Specifically, our method contains three key stages: 1)
each expression video clip is characterized as a spatial-temporal manifold
(STM) formed by dense low-level features; 2) a Universal Manifold Model (UMM)
is learned over all low-level features and represented as a set of local modes
to statistically unify all the STMs. 3) the local modes on each STM can be
instantiated by fitting to UMM, and the corresponding expressionlet is
constructed by modeling the variations in each local mode. With above strategy,
expression videos are naturally aligned both spatially and temporally. To
enhance the discriminative power, the expressionlet-based STM representation is
further processed with discriminant embedding. Our method is evaluated on four
public expression databases, CK+, MMI, Oulu-CASIA, and FERA. In all cases, our
method outperforms the known state-of-the-art by a large margin.
Binary Classifier Calibration using an Ensemble of Near Isotonic Regression Models
Authors: Mahdi Pakdaman Naeini, Gregory F. Cooper
Date: 2015-11-16 00:00:00
Tasks: Binary Classification, Classifier calibration, General Classification, regression

Learning accurate probabilistic models from data is crucial in many practical
tasks in data mining. In this paper we present a new non-parametric calibration
method called \textit{ensemble of near isotonic regression} (ENIR). The method
can be considered as an extension of BBQ, a recently proposed calibration
method, as well as the commonly used calibration method based on isotonic
regression. ENIR is designed to address the key limitation of isotonic
regression which is the monotonicity assumption of the predictions. Similar to
BBQ, the method post-processes the output of a binary classifier to obtain
calibrated probabilities. Thus it can be combined with many existing
classification models. We demonstrate the performance of ENIR on synthetic and
real datasets for the commonly used binary classification models. Experimental
results show that the method outperforms several common binary classifier
calibration methods. In particular on the real data, ENIR commonly performs
statistically significantly better than the other methods, and never worse. It
is able to improve the calibration power of classifiers, while retaining their
discrimination power. The method is also computationally tractable for large
scale datasets, as it is $O(N \log N)$ time, where $N$ is the number of
samples.
Neural Adaptive Sequential Monte Carlo
Authors: Shixiang Gu, Zoubin Ghahramani, Richard E. Turner
Date: 2015-06-10 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Variational Inference

Sequential Monte Carlo (SMC), or particle filtering, is a popular class of
methods for sampling from an intractable target distribution using a sequence
of simpler intermediate distributions. Like other importance sampling-based
methods, performance is critically dependent on the proposal distribution: a
bad proposal can lead to arbitrarily inaccurate estimates of the target
distribution. This paper presents a new method for automatically adapting the
proposal using an approximation of the Kullback-Leibler divergence between the
true posterior and the proposal distribution. The method is very flexible,
applicable to any parameterized proposal distribution and it supports online
and batch variants. We use the new framework to adapt powerful proposal
distributions with rich parameterizations based upon neural networks leading to
Neural Adaptive Sequential Monte Carlo (NASMC). Experiments indicate that NASMC
significantly improves inference in a non-linear state space model
outperforming adaptive proposal methods including the Extended Kalman and
Unscented Particle Filters. Experiments also indicate that improved inference
translates into improved parameter learning when NASMC is used as a subroutine
of Particle Marginal Metropolis Hastings. Finally we show that NASMC is able to
train a latent variable recurrent neural network (LV-RNN) achieving results
that compete with the state-of-the-art for polymorphic music modelling. NASMC
can be seen as bridging the gap between adaptive SMC methods and the recent
work in scalable, black-box variational inference.
Nonlinear Local Metric Learning for Person Re-identification
Authors: Siyuan Huang, Jiwen Lu, Jie zhou, Anil K. Jain
Date: 2015-11-16 00:00:00
Tasks: Metric Learning, Person Re-Identification

Person re-identification aims at matching pedestrians observed from
non-overlapping camera views. Feature descriptor and metric learning are two
significant problems in person re-identification. A discriminative metric
learning method should be capable of exploiting complex nonlinear
transformations due to the large variations in feature space. In this paper, we
propose a nonlinear local metric learning (NLML) method to improve the
state-of-the-art performance of person re-identification on public datasets.
Motivated by the fact that local metric learning has been introduced to handle
the data which varies locally and deep neural network has presented outstanding
capability in exploiting the nonlinearity of samples, we utilize the merits of
both local metric learning and deep neural network to learn multiple sets of
nonlinear transformations. By enforcing a margin between the distances of
positive pedestrian image pairs and distances of negative pairs in the
transformed feature subspace, discriminative information can be effectively
exploited in the developed neural networks. Our experiments show that the
proposed NLML method achieves the state-of-the-art results on the widely used
VIPeR, GRID, and CUHK 01 datasets.
Resolving the Geometric Locus Dilemma for Support Vector Learning Machines
Authors: Denise M. Reeves
Date: 2015-11-16 00:00:00

Capacity control, the bias/variance dilemma, and learning unknown functions
from data, are all concerned with identifying effective and consistent fits of
unknown geometric loci to random data points. A geometric locus is a curve or
surface formed by points, all of which possess some uniform property. A
geometric locus of an algebraic equation is the set of points whose coordinates
are solutions of the equation. Any given curve or surface must pass through
each point on a specified locus. This paper argues that it is impossible to fit
random data points to algebraic equations of partially configured geometric
loci that reference arbitrary Cartesian coordinate systems. It also argues that
the fundamental curve of a linear decision boundary is actually a principal
eigenaxis. It is shown that learning principal eigenaxes of linear decision
boundaries involves finding a point of statistical equilibrium for which
eigenenergies of principal eigenaxis components are symmetrically balanced with
each other. It is demonstrated that learning linear decision boundaries
involves strong duality relationships between a statistical eigenlocus of
principal eigenaxis components and its algebraic forms, in primal and dual,
correlated Hilbert spaces. Locus equations are introduced and developed that
describe principal eigen-coordinate systems for lines, planes, and hyperplanes.
These equations are used to introduce and develop primal and dual statistical
eigenlocus equations of principal eigenaxes of linear decision boundaries.
Important generalizations for linear decision boundaries are shown to be
encoded within a dual statistical eigenlocus of principal eigenaxis components.
Principal eigenaxes of linear decision boundaries are shown to encode Bayes'
likelihood ratio for common covariance data and a robust likelihood ratio for
all other data.
How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
Authors: Ferenc Huszár
Date: 2015-11-16 00:00:00
Tasks: Image Captioning

Modern applications and progress in deep learning research have created
renewed interest for generative models of text and of images. However, even
today it is unclear what objective functions one should use to train and
evaluate these models. In this paper we present two contributions.
  Firstly, we present a critique of scheduled sampling, a state-of-the-art
training method that contributed to the winning entry to the MSCOCO image
captioning benchmark in 2015. Here we show that despite this impressive
empirical performance, the objective function underlying scheduled sampling is
improper and leads to an inconsistent learning algorithm.
  Secondly, we revisit the problems that scheduled sampling was meant to
address, and present an alternative interpretation. We argue that maximum
likelihood is an inappropriate training objective when the end-goal is to
generate natural-looking samples. We go on to derive an ideal objective
function to use in this situation instead. We introduce a generalisation of
adversarial training, and show how such method can interpolate between maximum
likelihood training and our ideal training objective. To our knowledge this is
the first theoretical analysis that explains why adversarial training tends to
produce samples with higher perceived quality.
Accurate estimation of influenza epidemics using Google search data via ARGO
Authors: Shihao Yang, Mauricio Santillana, S. C. Kou
Date: 2015-05-05 00:00:00

Accurate real-time tracking of influenza outbreaks helps public health
officials make timely and meaningful decisions that could save lives. We
propose an influenza tracking model, ARGO (AutoRegression with GOogle search
data), that uses publicly available online search data. In addition to having a
rigorous statistical foundation, ARGO outperforms all previously available
Google-search-based tracking models, including the latest version of Google Flu
Trends, even though it uses only low-quality search data as input from publicly
available Google Trends and Google Correlate websites. ARGO not only
incorporates the seasonality in influenza epidemics but also captures changes
in people's online search behavior over time. ARGO is also flexible,
self-correcting, robust, and scalable, making it a potentially powerful tool
that can be used for real-time tracking of other social events at multiple
temporal and spatial resolutions.
Convex Optimization: Algorithms and Complexity
Authors: Sébastien Bubeck
Date: 2014-05-20 00:00:00
Tasks: Stochastic Optimization

This monograph presents the main complexity theorems in convex optimization
and their corresponding algorithms. Starting from the fundamental theory of
black-box optimization, the material progresses towards recent advances in
structural optimization and stochastic optimization. Our presentation of
black-box optimization, strongly influenced by Nesterov's seminal book and
Nemirovski's lecture notes, includes the analysis of cutting plane methods, as
well as (accelerated) gradient descent schemes. We also pay special attention
to non-Euclidean settings (relevant algorithms include Frank-Wolfe, mirror
descent, and dual averaging) and discuss their relevance in machine learning.
We provide a gentle introduction to structural optimization with FISTA (to
optimize a sum of a smooth and a simple non-smooth term), saddle-point mirror
prox (Nemirovski's alternative to Nesterov's smoothing), and a concise
description of interior point methods. In stochastic optimization we discuss
stochastic gradient descent, mini-batches, random coordinate descent, and
sublinear algorithms. We also briefly touch upon convex relaxation of
combinatorial problems and the use of randomness to round solutions, as well as
random walks based methods.
Topic Modeling of Behavioral Modes Using Sensor Data
Authors: Yehezkel S. Resheff, Shay Rotics, Ran Nathan, Daphna Weinshall
Date: 2015-11-16 00:00:00
Tasks: Clustering

The field of Movement Ecology, like so many other fields, is experiencing a
period of rapid growth in availability of data. As the volume rises,
traditional methods are giving way to machine learning and data science, which
are playing an increasingly large part it turning this data into
science-driving insights. One rich and interesting source is the bio-logger.
These small electronic wearable devices are attached to animals free to roam in
their natural habitats, and report back readings from multiple sensors,
including GPS and accelerometer bursts. A common use of accelerometer data is
for supervised learning of behavioral modes. However, we need unsupervised
analysis tools as well, in order to overcome the inherent difficulties of
obtaining a labeled dataset, which in some cases is either infeasible or does
not successfully encompass the full repertoire of behavioral modes of interest.
Here we present a matrix factorization based topic-model method for
accelerometer bursts, derived using a linear mixture property of patch
features. Our method is validated via comparison to a labeled dataset, and is
further compared to standard clustering algorithms.
Latent Dirichlet Allocation Based Organisation of Broadcast Media Archives for Deep Neural Network Adaptation
Authors: Mortaza Doulaty, Oscar Saz, Raymond W. M. Ng, Thomas Hain
Date: 2015-11-16 00:00:00
Tasks: Acoustic Modelling, Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

This paper presents a new method for the discovery of latent domains in
diverse speech data, for the use of adaptation of Deep Neural Networks (DNNs)
for Automatic Speech Recognition. Our work focuses on transcription of
multi-genre broadcast media, which is often only categorised broadly in terms
of high level genres such as sports, news, documentary, etc. However, in terms
of acoustic modelling these categories are coarse. Instead, it is expected that
a mixture of latent domains can better represent the complex and diverse
behaviours within a TV show, and therefore lead to better and more robust
performance. We propose a new method, whereby these latent domains are
discovered with Latent Dirichlet Allocation, in an unsupervised manner. These
are used to adapt DNNs using the Unique Binary Code (UBIC) representation for
the LDA domains. Experiments conducted on a set of BBC TV broadcasts, with more
than 2,000 shows for training and 47 shows for testing, show that the use of
LDA-UBIC DNNs reduces the error up to 13% relative compared to the baseline
hybrid DNN models.
An Empirical Study of Recent Face Alignment Methods
Authors: Heng Yang, Xuhui Jia, Chen Change Loy, Peter Robinson
Date: 2015-11-16 00:00:00
Tasks: Face Alignment, Face Detection

The problem of face alignment has been intensively studied in the past years.
A large number of novel methods have been proposed and reported very good
performance on benchmark dataset such as 300W. However, the differences in the
experimental setting and evaluation metric, missing details in the description
of the methods make it hard to reproduce the results reported and evaluate the
relative merits. For instance, most recent face alignment methods are built on
top of face detection but from different face detectors. In this paper, we
carry out a rigorous evaluation of these methods by making the following
contributions: 1) we proposes a new evaluation metric for face alignment on a
set of images, i.e., area under error distribution curve within a threshold,
AUC$_\alpha$, given the fact that the traditional evaluation measure (mean
error) is very sensitive to big alignment error. 2) we extend the 300W database
with more practical face detections to make fair comparison possible. 3) we
carry out face alignment sensitivity analysis w.r.t. face detection, on both
synthetic and real data, using both off-the-shelf and re-retrained models. 4)
we study factors that are particularly important to achieve good performance
and provide suggestions for practical applications. Most of the conclusions
drawn from our comparative analysis cannot be inferred from the original
publications.
Performance Analysis of Multiclass Support Vector Machine Classification for Diagnosis of Coronary Heart Diseases
Authors: Wiharto Wiharto, Hari Kusnanto, Herianto Herianto
Date: 2015-11-07 00:00:00
Tasks: Binary Classification, Classification, Decision Making, General Classification

Automatic diagnosis of coronary heart disease helps the doctor to support in
decision making a diagnosis. Coronary heart disease have some types or levels.
Referring to the UCI Repository dataset, it divided into 4 types or levels that
are labeled numbers 1-4 (low, medium, high and serious). The diagnosis models
can be analyzed with multiclass classification approach. One of multiclass
classification approach used, one of which is a support vector machine (SVM).
The SVM use due to strong performance of SVM in binary classification. This
research study multiclass performance classification support vector machine to
diagnose the type or level of coronary heart disease. Coronary heart disease
patient data taken from the UCI Repository. Stages in this study is
preprocessing, which consist of, to normalizing the data, divide the data into
data training and testing. The next stage of multiclass classification and
performance analysis. This study uses multiclass SVM algorithm, namely: Binary
Tree Support Vector Machine (BTSVM), One-Against-One (OAO), One-Against-All
(OAA), Decision Direct Acyclic Graph (DDAG) and Exhaustive Output Error
Correction Code (ECOC). Performance parameter used is recall, precision,
F-measure and Overall accuracy.
Learning visual biases from human imagination
Authors: Carl Vondrick, Hamed Pirsiavash, Aude Oliva, Antonio Torralba
Date: 2014-10-17 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Object Recognition

Although the human visual system can recognize many concepts under
challenging conditions, it still has some biases. In this paper, we investigate
whether we can extract these biases and transfer them into a machine
recognition system. We introduce a novel method that, inspired by well-known
tools in human psychophysics, estimates the biases that the human visual system
might use for recognition, but in computer vision feature spaces. Our
experiments are surprising, and suggest that classifiers from the human visual
system can be transferred into a machine with some success. Since these
classifiers seem to capture favorable biases in the human visual system, we
further present an SVM formulation that constrains the orientation of the SVM
hyperplane to agree with the bias from human visual system. Our results suggest
that transferring this human bias into machines may help object recognition
systems generalize across datasets and perform better when very little training
data is available.
Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting
Authors: Jakub Konečný, Jie Liu, Peter Richtárik, Martin Takáč
Date: 2015-04-16 00:00:00

We propose mS2GD: a method incorporating a mini-batching scheme for improving
the theoretical complexity and practical performance of semi-stochastic
gradient descent (S2GD). We consider the problem of minimizing a strongly
convex function represented as the sum of an average of a large number of
smooth convex functions, and a simple nonsmooth convex regularizer. Our method
first performs a deterministic step (computation of the gradient of the
objective function at the starting point), followed by a large number of
stochastic steps. The process is repeated a few times with the last iterate
becoming the new starting point. The novelty of our method is in introduction
of mini-batching into the computation of stochastic steps. In each step,
instead of choosing a single function, we sample $b$ functions, compute their
gradients, and compute the direction based on this. We analyze the complexity
of the method and show that it benefits from two speedup effects. First, we
prove that as long as $b$ is below a certain threshold, we can reach any
predefined accuracy with less overall work than without mini-batching. Second,
our mini-batching scheme admits a simple parallel implementation, and hence is
suitable for further acceleration by parallelization.