pwc-archive/papers-with-abstracts huggingface dataset:
The Matrix Generalized Inverse Gaussian Distribution: Properties and Applications
Authors: Farideh Fazayeli, Arindam Banerjee
Date: 2016-04-12 00:00:00

While the Matrix Generalized Inverse Gaussian ($\mathcal{MGIG}$) distribution
arises naturally in some settings as a distribution over symmetric positive
semi-definite matrices, certain key properties of the distribution and
effective ways of sampling from the distribution have not been carefully
studied. In this paper, we show that the $\mathcal{MGIG}$ is unimodal, and the
mode can be obtained by solving an Algebraic Riccati Equation (ARE) equation
[7]. Based on the property, we propose an importance sampling method for the
$\mathcal{MGIG}$ where the mode of the proposal distribution matches that of
the target. The proposed sampling method is more efficient than existing
approaches [32, 33], which use proposal distributions that may have the mode
far from the $\mathcal{MGIG}$'s mode. Further, we illustrate that the the
posterior distribution in latent factor models, such as probabilistic matrix
factorization (PMF) [25], when marginalized over one latent factor has the
$\mathcal{MGIG}$ distribution. The characterization leads to a novel Collapsed
Monte Carlo (CMC) inference algorithm for such latent factor models. We
illustrate that CMC has a lower log loss or perplexity than MCMC, and needs
fewer samples.
Computational and Statistical Tradeoffs in Learning to Rank
Authors: Ashish Khetan, Sewoong Oh
Date: 2016-08-22 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Learning-To-Rank

For massive and heterogeneous modern datasets, it is of fundamental interest
to provide guarantees on the accuracy of estimation when computational
resources are limited. In the application of learning to rank, we provide a
hierarchy of rank-breaking mechanisms ordered by the complexity in thus
generated sketch of the data. This allows the number of data points collected
to be gracefully traded off against computational resources available, while
guaranteeing the desired level of accuracy. Theoretical guarantees on the
proposed generalized rank-breaking implicitly provide such trade-offs, which
can be explicitly characterized under certain canonical scenarios on the
structure of the data.
CrowdNet: A Deep Convolutional Network for Dense Crowd Counting
Authors: Lokesh Boominathan, Srinivas S. S. Kruthiventi, R. Venkatesh Babu
Date: 2016-08-22 00:00:00
Tasks: Crowd Counting, Data Augmentation, Deep Learning

Our work proposes a novel deep learning framework for estimating crowd
density from static images of highly dense crowds. We use a combination of deep
and shallow, fully convolutional networks to predict the density map for a
given crowd image. Such a combination is used for effectively capturing both
the high-level semantic information (face/body detectors) and the low-level
features (blob detectors), that are necessary for crowd counting under large
scale variations. As most crowd datasets have limited training samples (<100
images) and deep learning based approaches require large amounts of training
data, we perform multi-scale data augmentation. Augmenting the training samples
in such a manner helps in guiding the CNN to learn scale invariant
representations. Our method is tested on the challenging UCF_CC_50 dataset, and
shown to outperform the state of the art methods.
Cox process representation and inference for stochastic reaction-diffusion processes
Authors: David Schnoerr, Ramon Grima, Guido Sanguinetti
Date: 2016-01-08 00:00:00
Tasks: Epidemiology, Model Selection

Complex behaviour in many systems arises from the stochastic interactions of
spatially distributed particles or agents. Stochastic reaction-diffusion
processes are widely used to model such behaviour in disciplines ranging from
biology to the social sciences, yet they are notoriously difficult to simulate
and calibrate to observational data. Here we use ideas from statistical physics
and machine learning to provide a solution to the inverse problem of learning a
stochastic reaction-diffusion process from data. Our solution relies on a
non-trivial connection between stochastic reaction-diffusion processes and
spatio-temporal Cox processes, a well-studied class of models from
computational statistics. This connection leads to an efficient and flexible
algorithm for parameter inference and model selection. Our approach shows
excellent accuracy on numeric and real data examples from systems biology and
epidemiology. Our work provides both insights into spatio-temporal stochastic
systems, and a practical solution to a long-standing problem in computational
modelling.
Efficient Continuous Relaxations for Dense CRF
Authors: Alban Desmaison, Rudy Bunel, Pushmeet Kohli, Philip H. S. Torr, M. Pawan Kumar
Date: 2016-08-22 00:00:00
Tasks: Semantic Segmentation, Variational Inference

Dense conditional random fields (CRF) with Gaussian pairwise potentials have
emerged as a popular framework for several computer vision applications such as
stereo correspondence and semantic segmentation. By modeling long-range
interactions, dense CRFs provide a more detailed labelling compared to their
sparse counterparts. Variational inference in these dense models is performed
using a filtering-based mean-field algorithm in order to obtain a
fully-factorized distribution minimising the Kullback-Leibler divergence to the
true distribution. In contrast to the continuous relaxation-based energy
minimisation algorithms used for sparse CRFs, the mean-field algorithm fails to
provide strong theoretical guarantees on the quality of its solutions. To
address this deficiency, we show that it is possible to use the same filtering
approach to speed-up the optimisation of several continuous relaxations.
Specifically, we solve a convex quadratic programming (QP) relaxation using the
efficient Frank-Wolfe algorithm. This also allows us to solve
difference-of-convex relaxations via the iterative concave-convex procedure
where each iteration requires solving a convex QP. Finally, we develop a novel
divide-and-conquer method to compute the subgradients of a linear programming
relaxation that provides the best theoretical bounds for energy minimisation.
We demonstrate the advantage of continuous relaxations over the widely used
mean-field algorithm on publicly available datasets.
Multi-Sensor Prognostics using an Unsupervised Health Index based on LSTM Encoder-Decoder
Authors: Pankaj Malhotra, Vishnu Tv, Anusha Ramakrishnan, Gaurangi Anand, Lovekesh Vig, Puneet Agarwal, Gautam Shroff
Date: 2016-08-22 00:00:00
Tasks: Decoder, Exponential degradation, Time Series, Time Series Analysis

Many approaches for estimation of Remaining Useful Life (RUL) of a machine,
using its operational sensor data, make assumptions about how a system degrades
or a fault evolves, e.g., exponential degradation. However, in many domains
degradation may not follow a pattern. We propose a Long Short Term Memory based
Encoder-Decoder (LSTM-ED) scheme to obtain an unsupervised health index (HI)
for a system using multi-sensor time-series data. LSTM-ED is trained to
reconstruct the time-series corresponding to healthy state of a system. The
reconstruction error is used to compute HI which is then used for RUL
estimation. We evaluate our approach on publicly available Turbofan Engine and
Milling Machine datasets. We also present results on a real-world industry
dataset from a pulverizer mill where we find significant correlation between
LSTM-ED based HI and maintenance costs.
Reconstructing Neural Parameters and Synapses of arbitrary interconnected Neurons from their Simulated Spiking Activity
Authors: J. Fischer, P. Manoonpong, S. Lackner
Date: 2016-08-22 00:00:00
Tasks: Computational Efficiency

To understand the behavior of a neural circuit it is a presupposition that we
have a model of the dynamical system describing this circuit. This model is
determined by several parameters, including not only the synaptic weights, but
also the parameters of each neuron. Existing works mainly concentrate on either
the synaptic weights or the neural parameters. In this paper we present an
algorithm to reconstruct all parameters including the synaptic weights of a
spiking neuron model. The model based on works of Eugene M. Izhikevich
(Izhikevich 2007) consists of two differential equations and covers different
types of cortical neurons. It combines the dynamical properties of
Hodgkin-Huxley-type dynamics with a high computational efficiency. The
presented algorithm uses the recordings of the corresponding membrane
potentials of the model for the reconstruction and consists of two main
components. The first component is a rank based Genetic Algorithm (GA) which is
used to find the neural parameters of the model. The second one is a Least Mean
Squares approach which computes the synaptic weights of all interconnected
neurons by minimizing the squared error between the calculated and the measured
membrane potentials for each time step. In preparation for the reconstruction
of the neural parameters and of the synaptic weights from real measured
membrane potentials, promising results based on simulated data generated with a
randomly parametrized Izhikevich model are presented. The reconstruction does
not only converge to a global minimum of neural parameters, but also
approximates the synaptic weights with high precision.
Actor-critic versus direct policy search: a comparison based on sample complexity
Authors: Arnaud de Froissard de Broissia, Olivier Sigaud
Date: 2016-06-29 00:00:00
Tasks: Deep Reinforcement Learning, Reinforcement Learning

Sample efficiency is a critical property when optimizing policy parameters
for the controller of a robot. In this paper, we evaluate two state-of-the-art
policy optimization algorithms. One is a recent deep reinforcement learning
method based on an actor-critic algorithm, Deep Deterministic Policy Gradient
(DDPG), that has been shown to perform well on various control benchmarks. The
other one is a direct policy search method, Covariance Matrix Adaptation
Evolution Strategy (CMA-ES), a black-box optimization method that is widely
used for robot learning. The algorithms are evaluated on a continuous version
of the mountain car benchmark problem, so as to compare their sample
complexity. From a preliminary analysis, we expect DDPG to be more sample
efficient than CMA-ES, which is confirmed by our experimental results.
The Evolution of Sex through the Baldwin Effect
Authors: Larry Bull
Date: 2016-07-01 00:00:00

This paper suggests that the fundamental haploid-diploid cycle of eukaryotic
sex exploits a rudimentary form of the Baldwin effect. With this explanation
for the basic cycle, the other associated phenomena can be explained as
evolution tuning the amount and frequency of learning experienced by an
organism. Using the well-known NK model of fitness landscapes it is shown that
varying landscape ruggedness varies the benefit of the haploid-diploid cycle,
whether based upon endomitosis or syngamy. The utility of pre-meiotic doubling
and recombination during the cycle are also shown to vary with landscape
ruggedness. This view is suggested as underpinning, rather than contradicting,
many existing explanations for sex.
Greedy methods, randomization approaches and multi-arm bandit algorithms for efficient sparsity-constrained optimization
Authors: A Rakotomamonjy, S Koço, Liva Ralaivola
Date: 2015-08-26 00:00:00

Several sparsity-constrained algorithms such as Orthogonal Matching Pursuit
or the Frank-Wolfe algorithm with sparsity constraints work by iteratively
selecting a novel atom to add to the current non-zero set of variables. This
selection step is usually performed by computing the gradient and then by
looking for the gradient component with maximal absolute entry. This step can
be computationally expensive especially for large-scale and high-dimensional
data. In this work, we aim at accelerating these sparsity-constrained
optimization algorithms by exploiting the key observation that, for these
algorithms to work, one only needs the coordinate of the gradient's top entry.
Hence, we introduce algorithms based on greedy methods and randomization
approaches that aim at cheaply estimating the gradient and its top entry.
Another of our contribution is to cast the problem of finding the best gradient
entry as a best arm identification in a multi-armed bandit problem. Owing to
this novel insight, we are able to provide a bandit-based algorithm that
directly estimates the top entry in a very efficient way. Theoretical
observations stating that the resulting inexact Frank-Wolfe or Orthogonal
Matching Pursuit algorithms act, with high probability, similarly to their
exact versions are also given. We have carried out several experiments showing
that the greedy deterministic and the bandit approaches we propose can achieve
an acceleration of an order of magnitude while being as efficient as the exact
gradient when used in algorithms such as OMP, Frank-Wolfe or CoSaMP.
Survey of resampling techniques for improving classification performance in unbalanced datasets
Authors: Ajinkya More
Date: 2016-08-22 00:00:00
Tasks: Classification, General Classification

A number of classification problems need to deal with data imbalance between
classes. Often it is desired to have a high recall on the minority class while
maintaining a high precision on the majority class. In this paper, we review a
number of resampling techniques proposed in literature to handle unbalanced
datasets and study their effect on classification performance.
Connotation Frames: A Data-Driven Investigation
Authors: Hannah Rashkin, Sameer Singh, Yejin Choi
Date: 2015-06-09 00:00:00
Proceeding: ACL 2016 8

Through a particular choice of a predicate (e.g., "x violated y"), a writer
can subtly connote a range of implied sentiments and presupposed facts about
the entities x and y: (1) writer's perspective: projecting x as an
"antagonist"and y as a "victim", (2) entities' perspective: y probably dislikes
x, (3) effect: something bad happened to y, (4) value: y is something valuable,
and (5) mental state: y is distressed by the event. We introduce connotation
frames as a representation formalism to organize these rich dimensions of
connotation using typed relations. First, we investigate the feasibility of
obtaining connotative labels through crowdsourcing experiments. We then present
models for predicting the connotation frames of verb predicates based on their
distributional word representations and the interplay between different types
of connotative relations. Empirical results confirm that connotation frames can
be induced from various data sources that reflect how people use language and
give rise to the connotative meanings. We conclude with analytical results that
show the potential use of connotation frames for analyzing subtle biases in
online news media.
Domain Separation Networks
Authors: Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, Dumitru Erhan
Date: 2016-08-22 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Domain Adaptation, Domain Generalization, Unsupervised Domain Adaptation

The cost of large scale data collection and annotation often makes the
application of machine learning algorithms to new tasks or datasets
prohibitively expensive. One approach circumventing this cost is training
models on synthetic data where annotations are provided automatically. Despite
their appeal, such models often fail to generalize from synthetic to real
images, necessitating domain adaptation algorithms to manipulate these models
before they can be successfully applied. Existing approaches focus either on
mapping representations from one domain to the other, or on learning to extract
features that are invariant to the domain from which they were extracted.
However, by focusing only on creating a mapping or shared representation
between the two domains, they ignore the individual characteristics of each
domain. We suggest that explicitly modeling what is unique to each domain can
improve a model's ability to extract domain-invariant features. Inspired by
work on private-shared component analysis, we explicitly learn to extract image
representations that are partitioned into two subspaces: one component which is
private to each domain and one which is shared across domains. Our model is
trained not only to perform the task we care about in the source domain, but
also to use the partitioned representation to reconstruct the images from both
domains. Our novel architecture results in a model that outperforms the
state-of-the-art on a range of unsupervised domain adaptation scenarios and
additionally produces visualizations of the private and shared representations
enabling interpretation of the domain adaptation process.
AI Evaluation: past, present and future
Authors: Jose Hernandez-Orallo
Date: 2014-08-29 00:00:00

Artificial intelligence develops techniques and systems whose performance
must be evaluated on a regular basis in order to certify and foster progress in
the discipline. We will describe and critically assess the different ways AI
systems are evaluated. We first focus on the traditional task-oriented
evaluation approach. We see that black-box (behavioural evaluation) is becoming
more and more common, as AI systems are becoming more complex and
unpredictable. We identify three kinds of evaluation: Human discrimination,
problem benchmarks and peer confrontation. We describe the limitations of the
many evaluation settings and competitions in these three categories and propose
several ideas for a more systematic and robust evaluation. We then focus on a
less customary (and challenging) ability-oriented evaluation approach, where a
system is characterised by its (cognitive) abilities, rather than by the tasks
it is designed to solve. We discuss several possibilities: the adaptation of
cognitive tests used for humans and animals, the development of tests derived
from algorithmic information theory or more general approaches under the
perspective of universal psychometrics.
Screening Tests for Lasso Problems
Authors: Zhen James Xiang, Yun Wang, Peter J. Ramadge
Date: 2014-05-19 00:00:00

This paper is a survey of dictionary screening for the lasso problem. The
lasso problem seeks a sparse linear combination of the columns of a dictionary
to best match a given target vector. This sparse representation has proven
useful in a variety of subsequent processing and decision tasks. For a given
target vector, dictionary screening quickly identifies a subset of dictionary
columns that will receive zero weight in a solution of the corresponding lasso
problem. These columns can be removed from the dictionary prior to solving the
lasso problem without impacting the optimality of the solution obtained. This
has two potential advantages: it reduces the size of the dictionary, allowing
the lasso problem to be solved with less resources, and it may speed up
obtaining a solution. Using a geometrically intuitive framework, we provide
basic insights for understanding useful lasso screening tests and their
limitations. We also provide illustrative numerical studies on several
datasets.
Pointing the Unknown Words
Authors: Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, Bo-Wen Zhou, Yoshua Bengio
Date: 2016-03-26 00:00:00
Proceeding: ACL 2016 8
Tasks: Machine Translation, Sentence, Text Summarization, Translation

The problem of rare and unknown words is an important issue that can
potentially influence the performance of many NLP systems, including both the
traditional count-based and the deep learning models. We propose a novel way to
deal with the rare and unseen words for the neural network models using
attention. Our model uses two softmax layers in order to predict the next word
in conditional language models: one predicts the location of a word in the
source sentence, and the other predicts a word in the shortlist vocabulary. At
each time-step, the decision of which softmax layer to use choose adaptively
made by an MLP which is conditioned on the context.~We motivate our work from a
psychological evidence that humans naturally have a tendency to point towards
objects in the context or the environment when the name of an object is not
known.~We observe improvements on two tasks, neural machine translation on the
Europarl English to French parallel corpora and text summarization on the
Gigaword dataset using our proposed model.
Spatial Modeling of Oil Exploration Areas Using Neural Networks and ANFIS in GIS
Authors: Nouraddin Misagh, Mohammadreza Ashouri
Date: 2016-08-21 00:00:00

Exploration of hydrocarbon resources is a highly complicated and expensive
process where various geological, geochemical and geophysical factors are
developed then combined together. It is highly significant how to design the
seismic data acquisition survey and locate the exploratory wells since
incorrect or imprecise locations lead to waste of time and money during the
operation. The objective of this study is to locate high-potential oil and gas
field in 1: 250,000 sheet of Ahwaz including 20 oil fields to reduce both time
and costs in exploration and production processes. In this regard, 17 maps were
developed using GIS functions for factors including: minimum and maximum of
total organic carbon (TOC), yield potential for hydrocarbons production (PP),
Tmax peak, production index (PI), oxygen index (OI), hydrogen index (HI) as
well as presence or proximity to high residual Bouguer gravity anomalies,
proximity to anticline axis and faults, topography and curvature maps obtained
from Asmari Formation subsurface contours. To model and to integrate maps, this
study employed artificial neural network and adaptive neuro-fuzzy inference
system (ANFIS) methods. The results obtained from model validation demonstrated
that the 17x10x5 neural network with R=0.8948, RMS=0.0267, and kappa=0.9079 can
be trained better than other models such as ANFIS and predicts the potential
areas more accurately. However, this method failed to predict some oil fields
and wrongly predict some areas as potential zones.
Piece-wise quadratic approximations of arbitrary error functions for fast and robust machine learning
Authors: A. N. Gorban, E. M. Mirkes, A. Zinovyev
Date: 2016-05-20 00:00:00
Tasks: BIG-bench Machine Learning, regression

Most of machine learning approaches have stemmed from the application of
minimizing the mean squared distance principle, based on the computationally
efficient quadratic optimization methods. However, when faced with
high-dimensional and noisy data, the quadratic error functionals demonstrated
many weaknesses including high sensitivity to contaminating factors and
dimensionality curse. Therefore, a lot of recent applications in machine
learning exploited properties of non-quadratic error functionals based on $L_1$
norm or even sub-linear potentials corresponding to quasinorms $L_p$ ($0<p<1$).
The back side of these approaches is increase in computational cost for
optimization. Till so far, no approaches have been suggested to deal with {\it
arbitrary} error functionals, in a flexible and computationally efficient
framework. In this paper, we develop a theory and basic universal data
approximation algorithms ($k$-means, principal components, principal manifolds
and graphs, regularized and sparse regression), based on piece-wise quadratic
error potentials of subquadratic growth (PQSQ potentials). We develop a new and
universal framework to minimize {\it arbitrary sub-quadratic error potentials}
using an algorithm with guaranteed fast convergence to the local or global
error minimum. The theory of PQSQ potentials is based on the notion of the cone
of minorant functions, and represents a natural approximation formalism based
on the application of min-plus algebra. The approach can be applied in most of
existing machine learning methods, including methods of data approximation and
regularized and sparse regression, leading to the improvement in the
computational cost/accuracy trade-off. We demonstrate that on synthetic and
real-life datasets PQSQ-based machine learning methods achieve orders of
magnitude faster computational performance than the corresponding
state-of-the-art methods.