pwc-archive/papers-with-abstracts huggingface dataset:
From Paraphrase Database to Compositional Paraphrase Model and Back
Authors: John Wieting, Mohit Bansal, Kevin Gimpel, Karen Livescu, Dan Roth
Date: 2015-06-10 00:00:00
Proceeding: TACL 2015 1
Tasks: Word Embeddings

The Paraphrase Database (PPDB; Ganitkevitch et al., 2013) is an extensive
semantic resource, consisting of a list of phrase pairs with (heuristic)
confidence estimates. However, it is still unclear how it can best be used, due
to the heuristic nature of the confidences and its necessarily incomplete
coverage. We propose models to leverage the phrase pairs from the PPDB to build
parametric paraphrase models that score paraphrase pairs more accurately than
the PPDB's internal scores while simultaneously improving its coverage. They
allow for learning phrase embeddings as well as improved word embeddings.
Moreover, we introduce two new, manually annotated datasets to evaluate
short-phrase paraphrasing models. Using our paraphrase model trained using
PPDB, we achieve state-of-the-art results on standard word and bigram
similarity tasks and beat strong baselines on our new short phrase paraphrase
tasks.
Automatic Relevance Determination For Deep Generative Models
Authors: Theofanis Karaletsos, Gunnar Rätsch
Date: 2015-05-28 00:00:00
Tasks: Model Selection, Variational Inference

A recurring problem when building probabilistic latent variable models is
regularization and model selection, for instance, the choice of the
dimensionality of the latent space. In the context of belief networks with
latent variables, this problem has been adressed with Automatic Relevance
Determination (ARD) employing Monte Carlo inference. We present a variational
inference approach to ARD for Deep Generative Models using doubly stochastic
variational inference to provide fast and scalable learning. We show empirical
results on a standard dataset illustrating the effects of contracting the
latent space automatically. We show that the resulting latent representations
are significantly more compact without loss of expressive power of the learned
models.
Financial Market Modeling with Quantum Neural Networks
Authors: Carlos Pedro Gonçalves
Date: 2015-08-26 00:00:00

Econophysics has developed as a research field that applies the formalism of
Statistical Mechanics and Quantum Mechanics to address Economics and Finance
problems. The branch of Econophysics that applies of Quantum Theory to
Economics and Finance is called Quantum Econophysics. In Finance, Quantum
Econophysics' contributions have ranged from option pricing to market dynamics
modeling, behavioral finance and applications of Game Theory, integrating the
empirical finding, from human decision analysis, that shows that nonlinear
update rules in probabilities, leading to non-additive decision weights, can be
computationally approached from quantum computation, with resulting quantum
interference terms explaining the non-additive probabilities. The current work
draws on these results to introduce new tools from Quantum Artificial
Intelligence, namely Quantum Artificial Neural Networks as a way to build and
simulate financial market models with adaptive selection of trading rules,
leading to turbulence and excess kurtosis in the returns distributions for a
wide range of parameters.
The Relation Between Acausality and Interference in Quantum-Like Bayesian Networks
Authors: Catarina Moreira, Andreas Wichert
Date: 2015-08-26 00:00:00
Tasks: Relation

We analyse a quantum-like Bayesian Network that puts together cause/effect
relationships and semantic similarities between events. These semantic
similarities constitute acausal connections according to the Synchronicity
principle and provide new relationships to quantum like probabilistic graphical
models. As a consequence, beliefs (or any other event) can be represented in
vector spaces, in which quantum parameters are determined by the similarities
that these vectors share between them. Events attached by a semantic meaning do
not need to have an explanation in terms of cause and effect.
Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
Authors: Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Pei-Hao Su, David Vandyke, Steve Young
Date: 2015-08-07 00:00:00
Proceeding: EMNLP 2015 9
Tasks: Informativeness, Sentence, Spoken Dialogue Systems, Text Generation

Natural language generation (NLG) is a critical component of spoken dialogue
and it has a significant impact both on usability and perceived quality. Most
NLG systems in common use employ rules and heuristics and tend to generate
rigid and stylised responses without the natural variation of human language.
They are also not easily scaled to systems covering multiple domains and
languages. This paper presents a statistical language generator based on a
semantically controlled Long Short-term Memory (LSTM) structure. The LSTM
generator can learn from unaligned data by jointly optimising sentence planning
and surface realisation using a simple cross entropy training criterion, and
language variation can be easily achieved by sampling from output candidates.
With fewer heuristics, an objective evaluation in two differing test domains
showed the proposed method improved performance compared to previous methods.
Human judges scored the LSTM system higher on informativeness and naturalness
and overall preferred it to the other systems.
A review of homomorphic encryption and software tools for encrypted statistical machine learning
Authors: Louis J. M. Aslett, Pedro M. Esperança, Chris C. Holmes
Date: 2015-08-26 00:00:00
Tasks: BIG-bench Machine Learning

Recent advances in cryptography promise to enable secure statistical
computation on encrypted data, whereby a limited set of operations can be
carried out without the need to first decrypt. We review these homomorphic
encryption schemes in a manner accessible to statisticians and machine
learners, focusing on pertinent limitations inherent in the current state of
the art. These limitations restrict the kind of statistics and machine learning
algorithms which can be implemented and we review those which have been
successfully applied in the literature. Finally, we document a high performance
R package implementing a recent homomorphic scheme in a general framework.
Deep Convolutional Neural Networks for Smile Recognition
Authors: Patrick O. Glauner
Date: 2015-08-26 00:00:00
Tasks: CPU, Facial Expression Recognition, Facial Expression Recognition (FER), GPU, Model Selection, Smile Recognition

This thesis describes the design and implementation of a smile detector based
on deep convolutional neural networks. It starts with a summary of neural
networks, the difficulties of training them and new training methods, such as
Restricted Boltzmann Machines or autoencoders. It then provides a literature
review of convolutional neural networks and recurrent neural networks. In order
to select databases for smile recognition, comprehensive statistics of
databases popular in the field of facial expression recognition were generated
and are summarized in this thesis. It then proposes a model for smile
detection, of which the main part is implemented. The experimental results are
discussed in this thesis and justified based on a comprehensive model selection
performed. All experiments were run on a Tesla K40c GPU benefiting from a
speedup of up to factor 10 over the computations on a CPU. A smile detection
test accuracy of 99.45% is achieved for the Denver Intensity of Spontaneous
Facial Action (DISFA) database, significantly outperforming existing approaches
with accuracies ranging from 65.55% to 79.67%. This experiment is re-run under
various variations, such as retaining less neutral images or only the low or
high intensities, of which the results are extensively compared.
Real-time Top-K Predictive Query Processing over Event Streams
Authors: Saurav Acharya, Byung Suk Lee, Paul Hines
Date: 2015-08-26 00:00:00
Tasks: Causal Inference

This paper addresses the problem of predicting the k events that are most
likely to occur next, over historical real-time event streams. Existing
approaches to causal prediction queries have a number of limitations. First,
they exhaustively search over an acyclic causal network to find the most likely
k effect events; however, data from real event streams frequently reflect
cyclic causality. Second, they contain conservative assumptions intended to
exclude all possible non-causal links in the causal network; it leads to the
omission of many less-frequent but important causal links. We overcome these
limitations by proposing a novel event precedence model and a run-time causal
inference mechanism. The event precedence model constructs a first order
absorbing Markov chain incrementally over event streams, where an edge between
two events signifies a temporal precedence relationship between them, which is
a necessary condition for causality. Then, the run-time causal inference
mechanism learns causal relationships dynamically during query processing. This
is done by removing some of the temporal precedence relationships that do not
exhibit causality in the presence of other events in the event precedence
model. This paper presents two query processing algorithms -- one performs
exhaustive search on the model and the other performs a more efficient reduced
search with early termination. Experiments using two real datasets (cascading
blackouts in power systems and web page views) verify the effectiveness of the
probabilistic top-k prediction queries and the efficiency of the algorithms.
Specifically, the reduced search algorithm reduced runtime, relative to
exhaustive search, by 25-80% (depending on the application) with only a small
reduction in accuracy.
A Multiple-Expert Binarization Framework for Multispectral Images
Authors: Reza Farrahi Moghaddam, Mohamed Cheriet
Date: 2015-02-04 00:00:00
Tasks: Binarization

In this work, a multiple-expert binarization framework for multispectral
images is proposed. The framework is based on a constrained subspace selection
limited to the spectral bands combined with state-of-the-art gray-level
binarization methods. The framework uses a binarization wrapper to enhance the
performance of the gray-level binarization. Nonlinear preprocessing of the
individual spectral bands is used to enhance the textual information. An
evolutionary optimizer is considered to obtain the optimal and some suboptimal
3-band subspaces from which an ensemble of experts is then formed. The
framework is applied to a ground truth multispectral dataset with promising
results. In addition, a generalization to the cross-validation approach is
developed that not only evaluates generalizability of the framework, it also
provides a practical instance of the selected experts that could be then
applied to unseen inputs despite the small size of the given ground truth
dataset.
Population Synthesis via k-Nearest Neighbor Crossover Kernel
Authors: Naoki Hamada, Katsumi Homma, Hiroyuki Higuchi, Hideyuki Kikuchi
Date: 2015-08-26 00:00:00

The recent development of multi-agent simulations brings about a need for
population synthesis. It is a task of reconstructing the entire population from
a sampling survey of limited size (1% or so), supplying the initial conditions
from which simulations begin. This paper presents a new kernel density
estimator for this task. Our method is an analogue of the classical
Breiman-Meisel-Purcell estimator, but employs novel techniques that harness the
huge degree of freedom which is required to model high-dimensional nonlinearly
correlated datasets: the crossover kernel, the k-nearest neighbor restriction
of the kernel construction set and the bagging of kernels. The performance as a
statistical estimator is examined through real and synthetic datasets. We
provide an "optimization-free" parameter selection rule for our method, a
theory of how our method works and a computational cost analysis. To
demonstrate the usefulness as a population synthesizer, our method is applied
to a household synthesis task for an urban micro-simulator.
Dither is Better than Dropout for Regularising Deep Neural Networks
Authors: Andrew J. R. Simpson
Date: 2015-08-19 00:00:00

Regularisation of deep neural networks (DNN) during training is critical to
performance. By far the most popular method is known as dropout. Here, cast
through the prism of signal processing theory, we compare and contrast the
regularisation effects of dropout with those of dither. We illustrate some
serious inherent limitations of dropout and demonstrate that dither provides a
more effective regulariser.
Gaussian Mixture Models with Component Means Constrained in Pre-selected Subspaces
Authors: Mu Qiao, Jia Li
Date: 2015-08-26 00:00:00
Tasks: Clustering, Dimensionality Reduction, General Classification

We investigate a Gaussian mixture model (GMM) with component means
constrained in a pre-selected subspace. Applications to classification and
clustering are explored. An EM-type estimation algorithm is derived. We prove
that the subspace containing the component means of a GMM with a common
covariance matrix also contains the modes of the density and the class means.
This motivates us to find a subspace by applying weighted principal component
analysis to the modes of a kernel density and the class means. To circumvent
the difficulty of deciding the kernel bandwidth, we acquire multiple subspaces
from the kernel densities based on a sequence of bandwidths. The GMM
constrained by each subspace is estimated; and the model yielding the maximum
likelihood is chosen. A dimension reduction property is proved in the sense of
being informative for classification or clustering. Experiments on real and
simulated data sets are conducted to examine several ways of determining the
subspace and to compare with the reduced rank mixture discriminant analysis
(MDA). Our new method with the simple technique of spanning the subspace only
by class means often outperforms the reduced rank MDA when the subspace
dimension is very low, making it particularly appealing for visualization.
SPRIGHT: A Fast and Robust Framework for Sparse Walsh-Hadamard Transform
Authors: Xiao Li, Joseph K. Bradley, Sameer Pawar, Kannan Ramchandran
Date: 2015-08-26 00:00:00

We consider the problem of computing the Walsh-Hadamard Transform (WHT) of
some $N$-length input vector in the presence of noise, where the $N$-point
Walsh spectrum is $K$-sparse with $K = {O}(N^{\delta})$ scaling sub-linearly in
the input dimension $N$ for some $0<\delta<1$. Over the past decade, there has
been a resurgence in research related to the computation of Discrete Fourier
Transform (DFT) for some length-$N$ input signal that has a $K$-sparse Fourier
spectrum. In particular, through a sparse-graph code design, our earlier work
on the Fast Fourier Aliasing-based Sparse Transform (FFAST) algorithm computes
the $K$-sparse DFT in time ${O}(K\log K)$ by taking ${O}(K)$ noiseless samples.
Inspired by the coding-theoretic design framework, Scheibler et al. proposed
the Sparse Fast Hadamard Transform (SparseFHT) algorithm that elegantly
computes the $K$-sparse WHT in the absence of noise using ${O}(K\log N)$
samples in time ${O}(K\log^2 N)$. However, the SparseFHT algorithm explicitly
exploits the noiseless nature of the problem, and is not equipped to deal with
scenarios where the observations are corrupted by noise. Therefore, a question
of critical interest is whether this coding-theoretic framework can be made
robust to noise. Further, if the answer is yes, what is the extra price that
needs to be paid for being robust to noise? In this paper, we show, quite
interestingly, that there is {\it no extra price} that needs to be paid for
being robust to noise other than a constant factor. In other words, we can
maintain the same sample complexity ${O}(K\log N)$ and the computational
complexity ${O}(K\log^2 N)$ as those of the noiseless case, using our SParse
Robust Iterative Graph-based Hadamard Transform (SPRIGHT) algorithm.
Toward a unified theory of sparse dimensionality reduction in Euclidean space
Authors: Jean Bourgain, Sjoerd Dirksen, Jelani Nelson
Date: 2013-11-11 00:00:00
Tasks: compressed sensing, Dimensionality Reduction, LEMMA

Let $\Phi\in\mathbb{R}^{m\times n}$ be a sparse Johnson-Lindenstrauss
transform [KN14] with $s$ non-zeroes per column. For a subset $T$ of the unit
sphere, $\varepsilon\in(0,1/2)$ given, we study settings for $m,s$ required to
ensure $$ \mathop{\mathbb{E}}_\Phi \sup_{x\in T} \left|\|\Phi x\|_2^2 - 1
\right| < \varepsilon , $$ i.e. so that $\Phi$ preserves the norm of every
$x\in T$ simultaneously and multiplicatively up to $1+\varepsilon$. We
introduce a new complexity parameter, which depends on the geometry of $T$, and
show that it suffices to choose $s$ and $m$ such that this parameter is small.
Our result is a sparse analog of Gordon's theorem, which was concerned with a
dense $\Phi$ having i.i.d. Gaussian entries. We qualitatively unify several
results related to the Johnson-Lindenstrauss lemma, subspace embeddings, and
Fourier-based restricted isometries. Our work also implies new results in using
the sparse Johnson-Lindenstrauss transform in numerical linear algebra,
classical and model-based compressed sensing, manifold learning, and
constrained least squares problems such as the Lasso.
Multiple kernel multivariate performance learning using cutting plane algorithm
Authors: Jingbin Wang, Haoxiang Wang, Yihua Zhou, Nancy McDonald
Date: 2015-08-25 00:00:00
Tasks: General Classification

In this paper, we propose a multi-kernel classifier learning algorithm to
optimize a given nonlinear and nonsmoonth multivariate classifier performance
measure. Moreover, to solve the problem of kernel function selection and kernel
parameter tuning, we proposed to construct an optimal kernel by weighted linear
combination of some candidate kernels. The learning of the classifier parameter
and the kernel weight are unified in a single objective function considering to
minimize the upper boundary of the given multivariate performance measure. The
objective function is optimized with regard to classifier parameter and kernel
weight alternately in an iterative algorithm by using cutting plane algorithm.
The developed algorithm is evaluated on two different pattern classification
methods with regard to various multivariate performance measure optimization
problems. The experiment results show the proposed algorithm outperforms the
competing methods.
A Statistical Perspective on Randomized Sketching for Ordinary Least-Squares
Authors: Garvesh Raskutti, Michael Mahoney
Date: 2014-06-23 00:00:00

We consider statistical as well as algorithmic aspects of solving large-scale
least-squares (LS) problems using randomized sketching algorithms. For a LS
problem with input data $(X, Y) \in \mathbb{R}^{n \times p} \times
\mathbb{R}^n$, sketching algorithms use a sketching matrix, $S\in\mathbb{R}^{r
\times n}$ with $r \ll n$. Then, rather than solving the LS problem using the
full data $(X,Y)$, sketching algorithms solve the LS problem using only the
sketched data $(SX, SY)$. Prior work has typically adopted an algorithmic
perspective, in that it has made no statistical assumptions on the input $X$
and $Y$, and instead it has been assumed that the data $(X,Y)$ are fixed and
worst-case (WC). Prior results show that, when using sketching matrices such as
random projections and leverage-score sampling algorithms, with $p < r \ll n$,
the WC error is the same as solving the original problem, up to a small
constant. From a statistical perspective, we typically consider the
mean-squared error performance of randomized sketching algorithms, when data
$(X, Y)$ are generated according to a statistical model $Y = X \beta +
\epsilon$, where $\epsilon$ is a noise process. We provide a rigorous
comparison of both perspectives leading to insights on how they differ. To do
this, we first develop a framework for assessing algorithmic and statistical
aspects of randomized sketching methods. We then consider the statistical
prediction efficiency (PE) and the statistical residual efficiency (RE) of the
sketched LS estimator; and we use our framework to provide upper bounds for
several types of random projection and random sampling sketching algorithms.
Among other results, we show that the RE can be upper bounded when $p < r \ll
n$ while the PE typically requires the sample size $r$ to be substantially
larger. Lower bounds developed in subsequent results show that our upper bounds
on PE can not be improved.
Inferring Passenger Type from Commuter Eigentravel Matrices
Authors: Erika Fille Legara, Christopher Monterola
Date: 2015-08-25 00:00:00
Tasks: Vocal Bursts Type Prediction

A sufficient knowledge of the demographics of a commuting public is essential
in formulating and implementing more targeted transportation policies, as
commuters exhibit different ways of traveling. With the advent of the Automated
Fare Collection system (AFC), probing the travel patterns of commuters has
become less invasive and more accessible. Consequently, numerous transport
studies related to human mobility have shown that these observed patterns allow
one to pair individuals with locations and/or activities at certain times of
the day. However, classifying commuters using their travel signatures is yet to
be thoroughly examined.
  Here, we contribute to the literature by demonstrating a procedure to
characterize passenger types (Adult, Child/Student, and Senior Citizen) based
on their three-month travel patterns taken from a smart fare card system. We
first establish a method to construct distinct commuter matrices, which we
refer to as eigentravel matrices, that capture the characteristic travel
routines of individuals. From the eigentravel matrices, we build classification
models that predict the type of passengers traveling. Among the models
explored, the gradient boosting method (GBM) gives the best prediction accuracy
at 76%, which is 84% better than the minimum model accuracy (41%) required
vis-\`a-vis the proportional chance criterion. In addition, we find that travel
features generated during weekdays have greater predictive power than those on
weekends. This work should not only be useful for transport planners, but for
market researchers as well. With the awareness of which commuter types are
traveling, ads, service announcements, and surveys, among others, can be made
more targeted spatiotemporally. Finally, our framework should be effective in
creating synthetic populations for use in real-world simulations that involve a
metropolitan's public transport system.
Grid-based angle-constrained path planning
Authors: Konstantin Yakovlev, Egor Baskin, Ivan Hramoin
Date: 2015-06-05 00:00:00
Tasks: Heuristic Search

Square grids are commonly used in robotics and game development as spatial
models and well known in AI community heuristic search algorithms (such as A*,
JPS, Theta* etc.) are widely used for path planning on grids. A lot of research
is concentrated on finding the shortest (in geometrical sense) paths while in
many applications finding smooth paths (rather than the shortest ones but
containing sharp turns) is preferable. In this paper we study the problem of
generating smooth paths and concentrate on angle constrained path planning. We
put angle-constrained path planning problem formally and present a new
algorithm tailored to solve it - LIAN. We examine LIAN both theoretically and
empirically. We show that it is sound and complete (under some restrictions).
We also show that LIAN outperforms the analogues when solving numerous path
planning tasks within urban outdoor navigation scenarios.