pwc-archive/papers-with-abstracts huggingface dataset:
Extraction of clinical information from the non-invasive fetal electrocardiogram
Authors: Joachim Behar
Date: 2016-05-27 00:00:00
Tasks: Benchmarking, Heart Rate Variability, Morphological Analysis

Estimation of the fetal heart rate (FHR) has gained interest in the last
century, low heart rate variability has been studied to identify intrauterine
growth restricted fetuses (prepartum), and abnormal FHR patterns have been
associated with fetal distress during delivery (intrapartum). Several
monitoring techniques have been proposed for FHR estimation, including
auscultation and Doppler ultrasound. This thesis focuses on the extraction of
the non-invasive fetal electrocardiogram (NI-FECG) recorded from a limited set
of abdominal sensors. The main challenge with NI-FECG extraction techniques is
the low signal-to-noise ratio of the FECG signal on the abdominal mixture
signal which consists of a dominant maternal ECG component, FECG and noise.
However the NI-FECG offers many advantages over the alternative fetal
monitoring techniques, the most important one being the opportunity to enable
morphological analysis of the FECG which is vital for determining whether an
observed FHR event is normal or pathological. In order to advance the field of
NI-FECG signal processing, the development of standardised public databases and
benchmarking of a number of published and novel algorithms was necessary.
Ladder Variational Autoencoders
Authors: Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, Ole Winther
Date: 2016-02-06 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Unsupervised MNIST

Variational Autoencoders are powerful models for unsupervised learning.
However deep models with several layers of dependent stochastic variables are
difficult to train which limits the improvements obtained using these highly
expressive models. We propose a new inference model, the Ladder Variational
Autoencoder, that recursively corrects the generative distribution by a data
dependent approximate likelihood in a process resembling the recently proposed
Ladder Network. We show that this model provides state of the art predictive
log-likelihood and tighter log-likelihood lower bound compared to the purely
bottom-up inference in layered Variational Autoencoders and other generative
models. We provide a detailed analysis of the learned hierarchical latent
representation and show that our new inference model is qualitatively different
and utilizes a deeper more distributed hierarchy of latent variables. Finally,
we observe that batch normalization and deterministic warm-up (gradually
turning on the KL-term) are crucial for training variational models with many
stochastic layers.
Lazy Evaluation of Convolutional Filters
Authors: Sam Leroux, Steven Bohez, Cedric De Boom, Elias De Coninck, Tim Verbelen, Bert Vankeirsbilck, Pieter Simoens, Bart Dhoedt
Date: 2016-05-27 00:00:00

In this paper we propose a technique which avoids the evaluation of certain
convolutional filters in a deep neural network. This allows to trade-off the
accuracy of a deep neural network with the computational and memory
requirements. This is especially important on a constrained device unable to
hold all the weights of the network in memory.
Stochastic Optimization for Large-scale Optimal Transport
Authors: Genevay Aude, Marco Cuturi, Gabriel Peyré, Francis Bach
Date: 2016-05-27 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Stochastic Optimization

Optimal transport (OT) defines a powerful framework to compare probability
distributions in a geometrically faithful way. However, the practical impact of
OT is still limited because of its computational burden. We propose a new class
of stochastic optimization algorithms to cope with large-scale problems
routinely encountered in machine learning applications. These methods are able
to manipulate arbitrary distributions (either discrete or continuous) by simply
requiring to be able to draw samples from them, which is the typical setup in
high-dimensional learning problems. This alleviates the need to discretize
these densities, while giving access to provably convergent methods that output
the correct distance without discretization error. These algorithms rely on two
main ideas: (a) the dual OT problem can be re-cast as the maximization of an
expectation ; (b) entropic regularization of the primal OT problem results in a
smooth dual optimization optimization which can be addressed with algorithms
that have a provably faster convergence. We instantiate these ideas in three
different setups: (i) when comparing a discrete distribution to another, we
show that incremental stochastic optimization schemes can beat Sinkhorn's
algorithm, the current state-of-the-art finite dimensional OT solver; (ii) when
comparing a discrete distribution to a continuous density, a semi-discrete
reformulation of the dual program is amenable to averaged stochastic gradient
descent, leading to better performance than approximately solving the problem
by discretization ; (iii) when dealing with two continuous densities, we
propose a stochastic gradient descent over a reproducing kernel Hilbert space
(RKHS). This is currently the only known method to solve this problem, apart
from computing OT on finite samples. We backup these claims on a set of
discrete, semi-discrete and continuous benchmark problems.
SNN: Stacked Neural Networks
Authors: Milad Mohammadi, Subhasis Das
Date: 2016-05-27 00:00:00
Tasks: Transfer Learning

It has been proven that transfer learning provides an easy way to achieve
state-of-the-art accuracies on several vision tasks by training a simple
classifier on top of features obtained from pre-trained neural networks. The
goal of this work is to generate better features for transfer learning from
multiple publicly available pre-trained neural networks. To this end, we
propose a novel architecture called Stacked Neural Networks which leverages the
fast training time of transfer learning while simultaneously being much more
accurate. We show that using a stacked NN architecture can result in up to 8%
improvements in accuracy over state-of-the-art techniques using only one
pre-trained network for transfer learning. A second aim of this work is to make
network fine- tuning retain the generalizability of the base network to unseen
tasks. To this end, we propose a new technique called "joint fine-tuning" that
is able to give accuracies comparable to finetuning the same network
individually over two datasets. We also show that a jointly finetuned network
generalizes better to unseen tasks when compared to a network finetuned over a
single task.
Expressiveness of Rectifier Networks
Authors: Xingyuan Pan, Vivek Srikumar
Date: 2015-11-18 00:00:00

Rectified Linear Units (ReLUs) have been shown to ameliorate the vanishing
gradient problem, allow for efficient backpropagation, and empirically promote
sparsity in the learned parameters. They have led to state-of-the-art results
in a variety of applications. However, unlike threshold and sigmoid networks,
ReLU networks are less explored from the perspective of their expressiveness.
This paper studies the expressiveness of ReLU networks. We characterize the
decision boundary of two-layer ReLU networks by constructing functionally
equivalent threshold networks. We show that while the decision boundary of a
two-layer ReLU network can be captured by a threshold network, the latter may
require an exponentially larger number of hidden units. We also formulate
sufficient conditions for a corresponding logarithmic reduction in the number
of hidden units to represent a sign network as a ReLU network. Finally, we
experimentally compare threshold networks and their much smaller ReLU
counterparts with respect to their ability to learn from synthetically
generated data.
Structured Sparse Regression via Greedy Hard-Thresholding
Authors: Prateek Jain, Nikhil Rao, Inderjit Dhillon
Date: 2016-02-19 00:00:00
Tasks: regression

Several learning applications require solving high-dimensional regression
problems where the relevant features belong to a small number of (overlapping)
groups. For very large datasets and under standard sparsity constraints, hard
thresholding methods have proven to be extremely efficient, but such methods
require NP hard projections when dealing with overlapping groups. In this
paper, we show that such NP-hard projections can not only be avoided by
appealing to submodular optimization, but such methods come with strong
theoretical guarantees even in the presence of poorly conditioned data (i.e.
say when two features have correlation $\geq 0.99$), which existing analyses
cannot handle. These methods exhibit an interesting computation-accuracy
trade-off and can be extended to significantly harder problems such as sparse
overlapping groups. Experiments on both real and synthetic data validate our
claims and demonstrate that the proposed methods are orders of magnitude faster
than other greedy and convex relaxation techniques for learning with
group-structured sparsity.
ARock: an Algorithmic Framework for Asynchronous Parallel Coordinate Updates
Authors: Zhimin Peng, Yangyang Xu, Ming Yan, Wotao Yin
Date: 2015-06-08 00:00:00

Finding a fixed point to a nonexpansive operator, i.e., $x^*=Tx^*$, abstracts
many problems in numerical linear algebra, optimization, and other areas of
scientific computing. To solve fixed-point problems, we propose ARock, an
algorithmic framework in which multiple agents (machines, processors, or cores)
update $x$ in an asynchronous parallel fashion. Asynchrony is crucial to
parallel computing since it reduces synchronization wait, relaxes communication
bottleneck, and thus speeds up computing significantly. At each step of ARock,
an agent updates a randomly selected coordinate $x_i$ based on possibly
out-of-date information on $x$. The agents share $x$ through either global
memory or communication. If writing $x_i$ is atomic, the agents can read and
write $x$ without memory locks.
  Theoretically, we show that if the nonexpansive operator $T$ has a fixed
point, then with probability one, ARock generates a sequence that converges to
a fixed points of $T$. Our conditions on $T$ and step sizes are weaker than
comparable work. Linear convergence is also obtained.
  We propose special cases of ARock for linear systems, convex optimization,
machine learning, as well as distributed and decentralized consensus problems.
Numerical experiments of solving sparse logistic regression problems are
presented.
Local Region Sparse Learning for Image-on-Scalar Regression
Authors: Yao Chen, Xiao Wang, Linglong Kong, Hongtu Zhu
Date: 2016-05-27 00:00:00
Tasks: regression, Sparse Learning

Identification of regions of interest (ROI) associated with certain disease
has a great impact on public health. Imposing sparsity of pixel values and
extracting active regions simultaneously greatly complicate the image analysis.
We address these challenges by introducing a novel region-selection penalty in
the framework of image-on-scalar regression. Our penalty combines the Smoothly
Clipped Absolute Deviation (SCAD) regularization, enforcing sparsity, and the
SCAD of total variation (TV) regularization, enforcing spatial contiguity, into
one group, which segments contiguous spatial regions against zero-valued
background. Efficient algorithm is based on the alternative direction method of
multipliers (ADMM) which decomposes the non-convex problem into two iterative
optimization problems with explicit solutions. Another virtue of the proposed
method is that a divide and conquer learning algorithm is developed, thereby
allowing scaling to large images. Several examples are presented and the
experimental results are compared with other state-of-the-art approaches.
Recommendations as Treatments: Debiasing Learning and Evaluation
Authors: Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, Thorsten Joachims
Date: 2016-02-17 00:00:00
Tasks: Causal Inference, Recommendation Systems

Most data for evaluating and training recommender systems is subject to
selection biases, either through self-selection by the users or through the
actions of the recommendation system itself. In this paper, we provide a
principled approach to handling selection biases, adapting models and
estimation techniques from causal inference. The approach leads to unbiased
performance estimators despite biased data, and to a matrix factorization
method that provides substantially improved prediction performance on
real-world data. We theoretically and empirically characterize the robustness
of the approach, finding that it is highly practical and scalable.
Universum Learning for SVM Regression
Authors: Sauptik Dhar, Vladimir Cherkassky
Date: 2016-05-27 00:00:00
Tasks: regression

This paper extends the idea of Universum learning [18, 19] to regression
problems. We propose new Universum-SVM formulation for regression problems that
incorporates a priori knowledge in the form of additional data samples. These
additional data samples or Universum belong to the same application domain as
the training samples, but they follow a different distribution. Several
empirical comparisons are presented to illustrate the utility of the proposed
approach.
Combinatorial Topic Models using Small-Variance Asymptotics
Authors: Ke Jiang, Suvrit Sra, Brian Kulis
Date: 2016-04-07 00:00:00
Tasks: Combinatorial Optimization, Topic Models

Topic models have emerged as fundamental tools in unsupervised machine
learning. Most modern topic modeling algorithms take a probabilistic view and
derive inference algorithms based on Latent Dirichlet Allocation (LDA) or its
variants. In contrast, we study topic modeling as a combinatorial optimization
problem, and propose a new objective function derived from LDA by passing to
the small-variance limit. We minimize the derived objective by using ideas from
combinatorial optimization, which results in a new, fast, and high-quality
topic modeling algorithm. In particular, we show that our results are
competitive with popular LDA-based topic modeling approaches, and also discuss
the (dis)similarities between our approach and its probabilistic counterparts.
Probabilistic Inference Modulo Theories
Authors: Rodrigo de Salvo Braz, Ciaran O'Reilly, Vibhav Gogate, Rina Dechter
Date: 2016-05-26 00:00:00

We present SGDPLL(T), an algorithm that solves (among many other problems)
probabilistic inference modulo theories, that is, inference problems over
probabilistic models defined via a logic theory provided as a parameter
(currently, propositional, equalities on discrete sorts, and inequalities, more
specifically difference arithmetic, on bounded integers). While many solutions
to probabilistic inference over logic representations have been proposed,
SGDPLL(T) is simultaneously (1) lifted, (2) exact and (3) modulo theories, that
is, parameterized by a background logic theory. This offers a foundation for
extending it to rich logic languages such as data structures and relational
data. By lifted, we mean algorithms with constant complexity in the domain size
(the number of values that variables can take). We also detail a solver for
summations with difference arithmetic and show experimental results from a
scenario in which SGDPLL(T) is much faster than a state-of-the-art
probabilistic solver.
Provable Algorithms for Inference in Topic Models
Authors: Sanjeev Arora, Rong Ge, Frederic Koehler, Tengyu Ma, Ankur Moitra
Date: 2016-05-27 00:00:00
Tasks: Topic Models

Recently, there has been considerable progress on designing algorithms with
provable guarantees -- typically using linear algebraic methods -- for
parameter learning in latent variable models. But designing provable algorithms
for inference has proven to be more challenging. Here we take a first step
towards provable inference in topic models. We leverage a property of topic
models that enables us to construct simple linear estimators for the unknown
topic proportions that have small variance, and consequently can work with
short documents. Our estimators also correspond to finding an estimate around
which the posterior is well-concentrated. We show lower bounds that for shorter
documents it can be information theoretically impossible to find the hidden
topics. Finally, we give empirical results that demonstrate that our algorithm
works on realistic topic models. It yields good solutions on synthetic data and
runs in time comparable to a {\em single} iteration of Gibbs sampling.
Evasion and Hardening of Tree Ensemble Classifiers
Authors: Alex Kantchelian, J. D. Tygar, Anthony D. Joseph
Date: 2015-09-25 00:00:00

Classifier evasion consists in finding for a given instance $x$ the nearest
instance $x'$ such that the classifier predictions of $x$ and $x'$ are
different. We present two novel algorithms for systematically computing
evasions for tree ensembles such as boosted trees and random forests. Our first
algorithm uses a Mixed Integer Linear Program solver and finds the optimal
evading instance under an expressive set of constraints. Our second algorithm
trades off optimality for speed by using symbolic prediction, a novel algorithm
for fast finite differences on tree ensembles. On a digit recognition task, we
demonstrate that both gradient boosted trees and random forests are extremely
susceptible to evasions. Finally, we harden a boosted tree model without loss
of predictive accuracy by augmenting the training set of each boosting round
with evading instances, a technique we call adversarial boosting.
Global Optimality of Local Search for Low Rank Matrix Recovery
Authors: Srinadh Bhojanapalli, Behnam Neyshabur, Nathan Srebro
Date: 2016-05-23 00:00:00
Proceeding: NeurIPS 2016 12

We show that there are no spurious local minima in the non-convex factorized
parametrization of low-rank matrix recovery from incoherent linear
measurements. With noisy measurements we show all local minima are very close
to a global optimum. Together with a curvature bound at saddle points, this
yields a polynomial time global convergence guarantee for stochastic gradient
descent {\em from random initialization}.
Open Problem: Best Arm Identification: Almost Instance-Wise Optimality and the Gap Entropy Conjecture
Authors: Lijie Chen, Jian Li
Date: 2016-05-27 00:00:00
Tasks: Multi-Armed Bandits

The best arm identification problem (BEST-1-ARM) is the most basic pure
exploration problem in stochastic multi-armed bandits. The problem has a long
history and attracted significant attention for the last decade. However, we do
not yet have a complete understanding of the optimal sample complexity of the
problem: The state-of-the-art algorithms achieve a sample complexity of
$O(\sum_{i=2}^{n} \Delta_{i}^{-2}(\ln\delta^{-1} + \ln\ln\Delta_i^{-1}))$
($\Delta_{i}$ is the difference between the largest mean and the $i^{th}$
mean), while the best known lower bound is $\Omega(\sum_{i=2}^{n}
\Delta_{i}^{-2}\ln\delta^{-1})$ for general instances and $\Omega(\Delta^{-2}
\ln\ln \Delta^{-1})$ for the two-arm instances. We propose to study the
instance-wise optimality for the BEST-1-ARM problem. Previous work has proved
that it is impossible to have an instance optimal algorithm for the 2-arm
problem. However, we conjecture that modulo the additive term
$\Omega(\Delta_2^{-2} \ln\ln \Delta_2^{-1})$ (which is an upper bound and worst
case lower bound for the 2-arm problem), there is an instance optimal algorithm
for BEST-1-ARM. Moreover, we introduce a new quantity, called the gap entropy
for a best-arm problem instance, and conjecture that it is the instance-wise
lower bound. Hence, resolving this conjecture would provide a final answer to
the old and basic problem.
Structure Learning of Partitioned Markov Networks
Authors: Song Liu, Taiji Suzuki, Masashi Sugiyama, Kenji Fukumizu
Date: 2015-04-02 00:00:00
Tasks: Time Series, Time Series Analysis

We learn the structure of a Markov Network between two groups of random
variables from joint observations. Since modelling and learning the full MN
structure may be hard, learning the links between two groups directly may be a
preferable option. We introduce a novel concept called the \emph{partitioned
ratio} whose factorization directly associates with the Markovian properties of
random variables across two groups. A simple one-shot convex optimization
procedure is proposed for learning the \emph{sparse} factorizations of the
partitioned ratio and it is theoretically guaranteed to recover the correct
inter-group structure under mild conditions. The performance of the proposed
method is experimentally compared with the state of the art MN structure
learning methods using ROC curves. Real applications on analyzing
bipartisanship in US congress and pairwise DNA/time-series alignments are also
reported.
Revisiting Semi-Supervised Learning with Graph Embeddings
Authors: Zhilin Yang, William W. Cohen, Ruslan Salakhutdinov
Date: 2016-03-29 00:00:00
Tasks: Document Classification, Entity Extraction using GAN, General Classification, Node Classification, Text Classification

We present a semi-supervised learning framework based on graph embeddings.
Given a graph between instances, we train an embedding for each instance to
jointly predict the class label and the neighborhood context in the graph. We
develop both transductive and inductive variants of our method. In the
transductive variant of our method, the class labels are determined by both the
learned embeddings and input feature vectors, while in the inductive variant,
the embeddings are defined as a parametric function of the feature vectors, so
predictions can be made on instances not seen during training. On a large and
diverse set of benchmark tasks, including text classification, distantly
supervised entity extraction, and entity classification, we show improved
performance over many of the existing models.
Model-Free Imitation Learning with Policy Optimization
Authors: Jonathan Ho, Jayesh K. Gupta, Stefano Ermon
Date: 2016-05-26 00:00:00
Tasks: Imitation Learning, model, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In imitation learning, an agent learns how to behave in an environment with
an unknown cost function by mimicking expert demonstrations. Existing imitation
learning algorithms typically involve solving a sequence of planning or
reinforcement learning problems. Such algorithms are therefore not directly
applicable to large, high-dimensional environments, and their performance can
significantly degrade if the planning problems are not solved to optimality.
Under the apprenticeship learning formalism, we develop alternative model-free
algorithms for finding a parameterized stochastic policy that performs at least
as well as an expert policy on an unknown cost function, based on sample
trajectories from the expert. Our approach, based on policy gradients, scales
to large continuous environments with guaranteed convergence to local minima.
A Feature based Approach for Video Compression
Authors: Rajer Sindhu
Date: 2016-05-26 00:00:00
Tasks: Image Stitching, Translation, Video Compression

It is a high cost problem for panoramic image stitching via image matching
algorithm and not practical for real-time performance. In this paper, we take
full advantage ofHarris corner invariant characterization method light
intensity parallel meaning, translation and rotation, and made a realtime
panoramic image stitching algorithm. According to the basic characteristics and
performance FPGA classical algorithm, several modules such as the feature point
extraction, and matching description is to optimize the feature-based logic.
Real-time optimization system to achieve high precision match. The new
algorithm process the image from pixel domain and obtained from CCD camera
Xilinx Spartan-6 hardware platform. After the image stitching algorithm, will
eventually form a portable interface to output high-definition content on the
display. The results showed that, the proposed algorithm has higher precision
with good real-time performance and robustness.