pwc-archive/papers-with-abstracts huggingface dataset:
On the Theory and Practice of Privacy-Preserving Bayesian Data Analysis
Authors: James Foulds, Joseph Geumlek, Max Welling, Kamalika Chaudhuri
Date: 2016-03-23 00:00:00
Tasks: Bayesian Inference, Privacy Preserving, Time Series, Time Series Analysis

Bayesian inference has great promise for the privacy-preserving analysis of
sensitive data, as posterior sampling automatically preserves differential
privacy, an algorithmic notion of data privacy, under certain conditions
(Dimitrakakis et al., 2014; Wang et al., 2015). While this one posterior sample
(OPS) approach elegantly provides privacy "for free," it is data inefficient in
the sense of asymptotic relative efficiency (ARE). We show that a simple
alternative based on the Laplace mechanism, the workhorse of differential
privacy, is as asymptotically efficient as non-private posterior inference,
under general assumptions. This technique also has practical advantages
including efficient use of the privacy budget for MCMC. We demonstrate the
practicality of our approach on a time-series analysis of sensitive military
records from the Afghanistan and Iraq wars disclosed by the Wikileaks
organization.
Improving Coreference Resolution by Learning Entity-Level Distributed Representations
Authors: Kevin Clark, Christopher D. Manning
Date: 2016-06-04 00:00:00
Proceeding: ACL 2016 8
Tasks: coreference-resolution, Coreference Resolution

A long-standing challenge in coreference resolution has been the
incorporation of entity-level information - features defined over clusters of
mentions instead of mention pairs. We present a neural network based
coreference system that produces high-dimensional vector representations for
pairs of coreference clusters. Using these representations, our system learns
when combining clusters is desirable. We train the system with a
learning-to-search algorithm that teaches it which local decisions (cluster
merges) will lead to a high-scoring final coreference partition. The system
substantially outperforms the current state-of-the-art on the English and
Chinese portions of the CoNLL 2012 Shared Task dataset despite using few
hand-engineered features.
Shaping the Future through Innovations: From Medical Imaging to Precision Medicine
Authors: Dorin Comaniciu, Klaus Engel, Bogdan Georgescu, Tommaso Mansi
Date: 2016-05-01 00:00:00

Medical images constitute a source of information essential for disease
diagnosis, treatment and follow-up. In addition, due to its patient-specific
nature, imaging information represents a critical component required for
advancing precision medicine into clinical practice. This manuscript describes
recently developed technologies for better handling of image information:
photorealistic visualization of medical images with Cinematic Rendering,
artificial agents for in-depth image understanding, support for minimally
invasive procedures, and patient-specific computational models with enhanced
predictive power. Throughout the manuscript we will analyze the capabilities of
such technologies and extrapolate on their potential impact to advance the
quality of medical care, while reducing its cost.
Rotation Invariant Angular Descriptor Via A Bandlimited Gaussian-like Kernel
Authors: Michael T. McCann, Matthew Fickus, Jelena Kovacevic
Date: 2016-06-08 00:00:00
Tasks: Density Estimation, Human Detection, Patch Matching

We present a new smooth, Gaussian-like kernel that allows the kernel density
estimate for an angular distribution to be exactly represented by a finite
number of its Fourier series coefficients. Distributions of angular quantities,
such as gradients, are a central part of several state-of-the-art image
processing algorithms, but these distributions are usually described via
histograms and therefore lack rotation invariance due to binning artifacts.
Replacing histograming with kernel density estimation removes these binning
artifacts and can provide a finite-dimensional descriptor of the distribution,
provided that the kernel is selected to be bandlimited. In this paper, we
present a new band-limited kernel that has the added advantage of being
Gaussian-like in the angular domain. We then show that it compares favorably to
gradient histograms for patch matching, person detection, and texture
segmentation.
Learning Thermodynamics with Boltzmann Machines
Authors: Giacomo Torlai, Roger G. Melko
Date: 2016-06-08 00:00:00

A Boltzmann machine is a stochastic neural network that has been extensively
used in the layers of deep architectures for modern machine learning
applications. In this paper, we develop a Boltzmann machine that is capable of
modelling thermodynamic observables for physical systems in thermal
equilibrium. Through unsupervised learning, we train the Boltzmann machine on
data sets constructed with spin configurations importance-sampled from the
partition function of an Ising Hamiltonian at different temperatures using
Monte Carlo (MC) methods. The trained Boltzmann machine is then used to
generate spin states, for which we compare thermodynamic observables to those
computed by direct MC sampling. We demonstrate that the Boltzmann machine can
faithfully reproduce the observables of the physical system. Further, we
observe that the number of neurons required to obtain accurate results
increases as the system is brought close to criticality.
Efficient Smoothed Concomitant Lasso Estimation for High Dimensional Regression
Authors: Eugene Ndiaye, Olivier Fercoq, Alexandre Gramfort, Vincent Leclère, Joseph Salmon
Date: 2016-06-08 00:00:00
Tasks: regression, Uncertainty Quantification, Vocal Bursts Intensity Prediction

In high dimensional settings, sparse structures are crucial for efficiency,
both in term of memory, computation and performance. It is customary to
consider $\ell_1$ penalty to enforce sparsity in such scenarios. Sparsity
enforcing methods, the Lasso being a canonical example, are popular candidates
to address high dimension. For efficiency, they rely on tuning a parameter
trading data fitting versus sparsity. For the Lasso theory to hold this tuning
parameter should be proportional to the noise level, yet the latter is often
unknown in practice. A possible remedy is to jointly optimize over the
regression parameter as well as over the noise level. This has been considered
under several names in the literature: Scaled-Lasso, Square-root Lasso,
Concomitant Lasso estimation for instance, and could be of interest for
confidence sets or uncertainty quantification. In this work, after illustrating
numerical difficulties for the Smoothed Concomitant Lasso formulation, we
propose a modification we coined Smoothed Concomitant Lasso, aimed at
increasing numerical stability. We propose an efficient and accurate solver
leading to a computational cost no more expansive than the one for the Lasso.
We leverage on standard ingredients behind the success of fast Lasso solvers: a
coordinate descent algorithm, combined with safe screening rules to achieve
speed efficiency, by eliminating early irrelevant features.
Phase Transitions for High Dimensional Clustering and Related Problems
Authors: Jiashun Jin, Zheng Tracy Ke, Wanjie Wang
Date: 2015-02-24 00:00:00
Tasks: Clustering, Two-sample testing, Vocal Bursts Intensity Prediction

Consider a two-class clustering problem where we observe $X_i = \ell_i \mu +
Z_i$, $Z_i \stackrel{iid}{\sim} N(0, I_p)$, $1 \leq i \leq n$. The feature
vector $\mu\in R^p$ is unknown but is presumably sparse. The class labels
$\ell_i\in\{-1, 1\}$ are also unknown and the main interest is to estimate
them.
  We are interested in the statistical limits. In the two-dimensional phase
space calibrating the rarity and strengths of useful features, we find the
precise demarcation for the Region of Impossibility and Region of Possibility.
In the former, useful features are too rare/weak for successful clustering. In
the latter, useful features are strong enough to allow successful clustering.
The results are extended to the case of colored noise using Le Cam's idea on
comparison of experiments.
  We also extend the study on statistical limits for clustering to that for
signal recovery and that for hypothesis testing. We compare the statistical
limits for three problems and expose some interesting insight.
  We propose classical PCA and Important Features PCA (IF-PCA) for clustering.
For a threshold $t > 0$, IF-PCA clusters by applying classical PCA to all
columns of $X$ with an $L^2$-norm larger than $t$. We also propose two
aggregation methods. For any parameter in the Region of Possibility, some of
these methods yield successful clustering. We find an interesting phase
transition for IF-PCA.
  Our results require delicate analysis, especially on post-selection Random
Matrix Theory and on lower bound arguments.
Using Sentence-Level LSTM Language Models for Script Inference
Authors: Karl Pichotta, Raymond J. Mooney
Date: 2016-04-11 00:00:00
Proceeding: ACL 2016 8
Tasks: Sentence

There is a small but growing body of research on statistical scripts, models
of event sequences that allow probabilistic inference of implicit events from
documents. These systems operate on structured verb-argument events produced by
an NLP pipeline. We compare these systems with recent Recurrent Neural Net
models that directly operate on raw tokens to predict sentences, finding the
latter to be roughly comparable to the former in terms of predicting missing
events in documents.
Continuously Learning Neural Dialogue Management
Authors: Pei-Hao Su, Milica Gasic, Nikola Mrksic, Lina Rojas-Barahona, Stefan Ultes, David Vandyke, Tsung-Hsien Wen, Steve Young
Date: 2016-06-08 00:00:00
Tasks: Dialogue Management, Management, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Spoken Dialogue Systems, User Simulation

We describe a two-step approach for dialogue management in task-oriented
spoken dialogue systems. A unified neural network framework is proposed to
enable the system to first learn by supervision from a set of dialogue data and
then continuously improve its behaviour via reinforcement learning, all using
gradient-based algorithms on one single model. The experiments demonstrate the
supervised model's effectiveness in the corpus-based evaluation, with user
simulation, and with paid human subjects. The use of reinforcement learning
further improves the model's performance in both interactive settings,
especially under higher-noise conditions.
First Result on Arabic Neural Machine Translation
Authors: Amjad Almahairi, Kyunghyun Cho, Nizar Habash, Aaron Courville
Date: 2016-06-08 00:00:00
Tasks: Machine Translation, Translation

Neural machine translation has become a major alternative to widely used
phrase-based statistical machine translation. We notice however that much of
research on neural machine translation has focused on European languages
despite its language agnostic nature. In this paper, we apply neural machine
translation to the task of Arabic translation (Ar<->En) and compare it against
a standard phrase-based translation system. We run extensive comparison using
various configurations in preprocessing Arabic script and show that the
phrase-based and neural translation systems perform comparably to each other
and that proper preprocessing of Arabic script has a similar effect on both of
the systems. We however observe that the neural machine translation
significantly outperform the phrase-based system on an out-of-domain test set,
making it attractive for real-world deployment.
A Persona-Based Neural Conversation Model
Authors: Jiwei Li, Michel Galley, Chris Brockett, Georgios P. Spithourakis, Jianfeng Gao, Bill Dolan
Date: 2016-03-19 00:00:00
Proceeding: ACL 2016 8
Tasks: model, Response Generation

We present persona-based models for handling the issue of speaker consistency
in neural response generation. A speaker model encodes personas in distributed
embeddings that capture individual characteristics such as background
information and speaking style. A dyadic speaker-addressee model captures
properties of interactions between two interlocutors. Our models yield
qualitative performance improvements in both perplexity and BLEU scores over
baseline sequence-to-sequence models, with similar gains in speaker consistency
as measured by human judges.
Addressing Limited Data for Textual Entailment Across Domains
Authors: Chaitanya Shivade, Preethi Raghavan, Siddharth Patwardhan
Date: 2016-06-08 00:00:00
Proceeding: ACL 2016 8
Tasks: Active Learning, Natural Language Inference

We seek to address the lack of labeled data (and high cost of annotation) for
textual entailment in some domains. To that end, we first create (for
experimental purposes) an entailment dataset for the clinical domain, and a
highly competitive supervised entailment system, ENT, that is effective (out of
the box) on two domains. We then explore self-training and active learning
strategies to address the lack of labeled data. With self-training, we
successfully exploit unlabeled data to improve over ENT by 15% F-score on the
newswire domain, and 13% F-score on clinical data. On the other hand, our
active learning experiments demonstrate that we can match (and even beat) ENT
using only 6.6% of the training data in the clinical domain, and only 5.8% of
the training data in the newswire domain.
Fast Collaborative Filtering from Implicit Feedback with Provable Guarantees
Authors: Sayantan Dasgupta
Date: 2015-11-03 00:00:00
Tasks: Collaborative Filtering, Learning Theory, PAC learning, Recommendation Systems

Building recommendation algorithms is one of the most challenging tasks in
Machine Learning. Although most of the recommendation systems are built on
explicit feedback available from the users in terms of rating or text, a
majority of the applications do not receive such feedback. Here we consider the
recommendation task where the only available data is the records of user-item
interaction over web applications over time, in terms of subscription or
purchase of items; this is known as implicit feedback recommendation. There is
usually a massive amount of such user-item interaction available for any web
applications. Algorithms like PLSI or Matrix Factorization runs several
iterations through the dataset, and may prove very expensive for large
datasets. Here we propose a recommendation algorithm based on Method of Moment,
which involves factorization of second and third order moments of the dataset.
Our algorithm can be proven to be globally convergent using PAC learning
theory. Further, we show how to extract the parameters using only three passes
through the entire dataset. This results in a highly scalable algorithm that
scales up to million of users even on a machine with a single-core processor
and 8 GB RAM and produces competitive performance in comparison with existing
algorithms.
Specific Differential Entropy Rate Estimation for Continuous-Valued Time Series
Authors: David Darmon
Date: 2016-06-08 00:00:00
Tasks: Heart Rate Variability, Time Series, Time Series Analysis

We introduce a method for quantifying the inherent unpredictability of a
continuous-valued time series via an extension of the differential Shannon
entropy rate. Our extension, the specific entropy rate, quantifies the amount
of predictive uncertainty associated with a specific state, rather than
averaged over all states. We relate the specific entropy rate to popular
`complexity' measures such as Approximate and Sample Entropies. We provide a
data-driven approach for estimating the specific entropy rate of an observed
time series. Finally, we consider three case studies of estimating specific
entropy rate from synthetic and physiological data relevant to the analysis of
heart rate variability.
Fast and Extensible Online Multivariate Kernel Density Estimation
Authors: Jaime Ferreira, David Martins de Matos, Ricardo Ribeiro
Date: 2016-06-08 00:00:00
Tasks: Computational Efficiency, Density Estimation

We present xokde++, a state-of-the-art online kernel density estimation
approach that maintains Gaussian mixture models input data streams. The
approach follows state-of-the-art work on online density estimation, but was
redesigned with computational efficiency, numerical robustness, and
extensibility in mind. Our approach produces comparable or better results than
the current state-of-the-art, while achieving significant computational
performance gains and improved numerical stability. The use of diagonal
covariance Gaussian kernels, which further improve performance and stability,
at a small loss of modelling quality, is also explored. Our approach is up to
40 times faster, while requiring 90\% less memory than the closest
state-of-the-art counterpart.
A Joint Model for Word Embedding and Word Morphology
Authors: Kris Cao, Marek Rei
Date: 2016-06-08 00:00:00
Proceeding: WS 2016 8
Tasks: Morphological Analysis, Word Embeddings

This paper presents a joint model for performing unsupervised morphological
analysis on words, and learning a character-level composition function from
morphemes to word embeddings. Our model splits individual words into segments,
and weights each segment according to its ability to predict context words. Our
morphological analysis is comparable to dedicated morphological analyzers at
the task of morpheme boundary recovery, and also performs better than
word-based embedding models at the task of syntactic analogy answering.
Finally, we show that incorporating morphology explicitly into character-level
models help them produce embeddings for unseen words which correlate better
with human judgments.
Representing Meaning with a Combination of Logical and Distributional Models
Authors: I. Beltagy, Stephen Roller, Pengxiang Cheng, Katrin Erk, Raymond J. Mooney
Date: 2015-05-26 00:00:00
Proceeding: CL 2016 12
Tasks: Knowledge Base Construction, Lexical Entailment, Natural Language Inference, RTE, Sentence

NLP tasks differ in the semantic information they require, and at this time
no single se- mantic representation fulfills all requirements. Logic-based
representations characterize sentence structure, but do not capture the graded
aspect of meaning. Distributional models give graded similarity ratings for
words and phrases, but do not capture sentence structure in the same detail as
logic-based approaches. So it has been argued that the two are complementary.
We adopt a hybrid approach that combines logic-based and distributional
semantics through probabilistic logic inference in Markov Logic Networks
(MLNs). In this paper, we focus on the three components of a practical system
integrating logical and distributional models: 1) Parsing and task
representation is the logic-based part where input problems are represented in
probabilistic logic. This is quite different from representing them in standard
first-order logic. 2) For knowledge base construction we form weighted
inference rules. We integrate and compare distributional information with other
sources, notably WordNet and an existing paraphrase collection. In particular,
we use our system to evaluate distributional lexical entailment approaches. We
use a variant of Robinson resolution to determine the necessary inference
rules. More sources can easily be added by mapping them to logical rules; our
system learns a resource-specific weight that corrects for scaling differences
between resources. 3) In discussing probabilistic inference, we show how to
solve the inference problems efficiently. To evaluate our approach, we use the
task of textual entailment (RTE), which can utilize the strengths of both
logic-based and distributional representations. In particular we focus on the
SICK dataset, where we achieve state-of-the-art results.
Hierarchical learning of grids of microtopics
Authors: Nebojsa Jojic, Alessandro Perina, Dongwoo Kim
Date: 2015-03-12 00:00:00
Tasks: Diversity, General Classification

The counting grid is a grid of microtopics, sparse word/feature
distributions. The generative model associated with the grid does not use these
microtopics individually. Rather, it groups them in overlapping rectangular
windows and uses these grouped microtopics as either mixture or admixture
components. This paper builds upon the basic counting grid model and it shows
that hierarchical reasoning helps avoid bad local minima, produces better
classification accuracy and, most interestingly, allows for extraction of large
numbers of coherent microtopics even from small datasets. We evaluate this in
terms of consistency, diversity and clarity of the indexed content, as well as
in a user study on word intrusion tasks. We demonstrate that these models work
well as a technique for embedding raw images and discuss interesting parallels
between hierarchical CG models and other deep architectures.
Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagery
Authors: Jamie Sherrah
Date: 2016-06-08 00:00:00
Tasks: Scene Understanding, Vocal Bursts Intensity Prediction

The trend towards higher resolution remote sensing imagery facilitates a
transition from land-use classification to object-level scene understanding.
Rather than relying purely on spectral content, appearance-based image features
come into play. In this work, deep convolutional neural networks (CNNs) are
applied to semantic labelling of high-resolution remote sensing data. Recent
advances in fully convolutional networks (FCNs) are adapted to overhead data
and shown to be as effective as in other domains. A full-resolution labelling
is inferred using a deep FCN with no downsampling, obviating the need for
deconvolution or interpolation. To make better use of image features, a
pre-trained CNN is fine-tuned on remote sensing data in a hybrid network
context, resulting in superior results compared to a network trained from
scratch. The proposed approach is applied to the problem of labelling
high-resolution aerial imagery, where fine boundary detail is important. The
dense labelling yields state-of-the-art accuracy for the ISPRS Vaihingen and
Potsdam benchmark data sets.
The Dark Side of Ethical Robots
Authors: Dieter Vanderelst, Alan Winfield
Date: 2016-06-08 00:00:00
Tasks: Ethics

Concerns over the risks associated with advances in Artificial Intelligence
have prompted calls for greater efforts toward robust and beneficial AI,
including machine ethics. Recently, roboticists have responded by initiating
the development of so-called ethical robots. These robots would, ideally,
evaluate the consequences of their actions and morally justify their choices.
This emerging field promises to develop extensively over the next years.
However, in this paper, we point out an inherent limitation of the emerging
field of ethical robots. We show that building ethical robots also necessarily
facilitates the construction of unethical robots. In three experiments, we show
that it is remarkably easy to modify an ethical robot so that it behaves
competitively, or even aggressively. The reason for this is that the specific
AI, required to make an ethical robot, can always be exploited to make
unethical robots. Hence, the development of ethical robots will not guarantee
the responsible deployment of AI. While advocating for ethical robots, we
conclude that preventing the misuse of robots is beyond the scope of
engineering, and requires instead governance frameworks underpinned by
legislation. Without this, the development of ethical robots will serve to
increase the risks of robotic malpractice instead of diminishing it.