pwc-archive/papers-with-abstracts huggingface dataset:
A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systems
Authors: Layla El Asri, Jing He, Kaheer Suleman
Date: 2016-06-30 00:00:00
Tasks: Decoder, Dialogue State Tracking, Spoken Dialogue Systems, User Simulation

User simulation is essential for generating enough data to train a
statistical spoken dialogue system. Previous models for user simulation suffer
from several drawbacks, such as the inability to take dialogue history into
account, the need of rigid structure to ensure coherent user behaviour, heavy
dependence on a specific domain, the inability to output several user
intentions during one dialogue turn, or the requirement of a summarized action
space for tractability. This paper introduces a data-driven user simulator
based on an encoder-decoder recurrent neural network. The model takes as input
a sequence of dialogue contexts and outputs a sequence of dialogue acts
corresponding to user intentions. The dialogue contexts include information
about the machine acts and the status of the user goal. We show on the Dialogue
State Tracking Challenge 2 (DSTC2) dataset that the sequence-to-sequence model
outperforms an agenda-based simulator and an n-gram simulator, according to
F-score. Furthermore, we show how this model can be used on the original action
space and thereby models user behaviour with finer granularity.
Unsupervised Learning with Imbalanced Data via Structure Consolidation Latent Variable Model
Authors: Fariba Yousefi, Zhenwen Dai, Carl Henrik Ek, Neil Lawrence
Date: 2016-06-30 00:00:00
Tasks: Variational Inference

Unsupervised learning on imbalanced data is challenging because, when given
imbalanced data, current model is often dominated by the major category and
ignores the categories with small amount of data. We develop a latent variable
model that can cope with imbalanced data by dividing the latent space into a
shared space and a private space. Based on Gaussian Process Latent Variable
Models, we propose a new kernel formulation that enables the separation of
latent space and derives an efficient variational inference method. The
performance of our model is demonstrated with an imbalanced medical image
dataset.
Towards A Virtual Assistant That Can Be Taught New Tasks In Any Domain By Its End-Users
Authors: I. Dan Melamed, Nobal B. Niraula
Date: 2016-06-30 00:00:00

The challenge stated in the title can be divided into two main problems. The
first problem is to reliably mimic the way that users interact with user
interfaces. The second problem is to build an instructible agent, i.e. one that
can be taught to execute tasks expressed as previously unseen natural language
commands. This paper proposes a solution to the second problem, a system we
call Helpa. End-users can teach Helpa arbitrary new tasks whose level of
complexity is similar to the tasks available from today's most popular virtual
assistants. Teaching Helpa does not involve any programming. Instead, users
teach Helpa by providing just one example of a command paired with a
demonstration of how to execute that command. Helpa does not rely on any
pre-existing domain-specific knowledge. It is therefore completely
domain-independent. Our usability study showed that end-users can teach Helpa
many new tasks in less than a minute each, often much less.
Geometric Learning and Topological Inference with Biobotic Networks: Convergence Analysis
Authors: Alireza Dirafzoon, Alper Bozkurt, Edgar Lobaton
Date: 2016-06-30 00:00:00
Tasks: Topological Data Analysis

In this study, we present and analyze a framework for geometric and
topological estimation for mapping of unknown environments. We consider agents
mimicking motion behaviors of cyborg insects, known as biobots, and exploit
coordinate-free local interactions among them to infer geometric and
topological information about the environment, under minimal sensing and
localization constraints. Local interactions are used to create a graphical
representation referred to as the encounter graph. A metric is estimated over
the encounter graph of the agents in order to construct a geometric point cloud
using manifold learning techniques. Topological data analysis (TDA), in
particular persistent homology, is used in order to extract topological
features of the space and a classification method is proposed to infer robust
features of interest (e.g. existence of obstacles). We examine the asymptotic
behavior of the proposed metric in terms of the convergence to the geodesic
distances in the underlying manifold of the domain, and provide stability
analysis results for the topological persistence. The proposed framework and
its convergences and stability analysis are demonstrated through numerical
simulations and experiments.
Ballpark Learning: Estimating Labels from Rough Group Comparisons
Authors: Tom Hope, Dafna Shahaf
Date: 2016-06-30 00:00:00
Tasks: Sentiment Analysis

We are interested in estimating individual labels given only coarse,
aggregated signal over the data points. In our setting, we receive sets
("bags") of unlabeled instances with constraints on label proportions. We relax
the unrealistic assumption of known label proportions, made in previous work;
instead, we assume only to have upper and lower bounds, and constraints on bag
differences. We motivate the problem, propose an intuitive formulation and
algorithm, and apply our methods to real-world scenarios. Across several
domains, we show how using only proportion constraints and no labeled examples,
we can achieve surprisingly high accuracy. In particular, we demonstrate how to
predict income level using rough stereotypes and how to perform sentiment
analysis using very little information. We also apply our method to guide
exploratory analysis, recovering geographical differences in twitter dialect.
The Simulator: An Engine to Streamline Simulations
Authors: Jacob Bien
Date: 2016-06-30 00:00:00

The simulator is an R package that streamlines the process of performing
simulations by creating a common infrastructure that can be easily used and
reused across projects. Methodological statisticians routinely write
simulations to compare their methods to preexisting ones. While developing
ideas, there is a temptation to write "quick and dirty" simulations to try out
ideas. This approach of rapid prototyping is useful but can sometimes backfire
if bugs are introduced. Using the simulator allows one to remove the "dirty"
without sacrificing the "quick." Coding is quick because the statistician
focuses exclusively on those aspects of the simulation that are specific to the
particular paper being written. Code written with the simulator is succinct,
highly readable, and easily shared with others. The modular nature of
simulations written with the simulator promotes code reusability, which saves
time and facilitates reproducibility. The syntax of the simulator leads to
simulation code that is easily human-readable. Other benefits of using the
simulator include the ability to "step in" to a simulation and change one
aspect without having to rerun the entire simulation from scratch, the
straightforward integration of parallel computing into simulations, and the
ability to rapidly generate plots, tables, and reports with minimal effort.
Lifted Region-Based Belief Propagation
Authors: David Smith, Parag Singla, Vibhav Gogate
Date: 2016-06-30 00:00:00

Due to the intractable nature of exact lifted inference, research has
recently focused on the discovery of accurate and efficient approximate
inference algorithms in Statistical Relational Models (SRMs), such as Lifted
First-Order Belief Propagation. FOBP simulates propositional factor graph
belief propagation without constructing the ground factor graph by identifying
and lifting over redundant message computations. In this work, we propose a
generalization of FOBP called Lifted Generalized Belief Propagation, in which
both the region structure and the message structure can be lifted. This
approach allows more of the inference to be performed intra-region (in the
exact inference step of BP), thereby allowing simulation of propagation on a
graph structure with larger region scopes and fewer edges, while still
maintaining tractability. We demonstrate that the resulting algorithm converges
in fewer iterations to more accurate results on a variety of SRMs.
A Permutation-based Model for Crowd Labeling: Optimal Estimation and Robustness
Authors: Nihar B. Shah, Sivaraman Balakrishnan, Martin J. Wainwright
Date: 2016-06-30 00:00:00
Tasks: Denoising

The task of aggregating and denoising crowd-labeled data has gained increased significance with the advent of crowdsourcing platforms and massive datasets. We propose a permutation-based model for crowd labeled data that is a significant generalization of the classical Dawid-Skene model, and introduce a new error metric by which to compare different estimators. We derive global minimax rates for the permutation-based model that are sharp up to logarithmic factors, and match the minimax lower bounds derived under the simpler Dawid-Skene model. We then design two computationally-efficient estimators: the WAN estimator for the setting where the ordering of workers in terms of their abilities is approximately known, and the OBI-WAN estimator where that is not known. For each of these estimators, we provide non-asymptotic bounds on their performance. We conduct synthetic simulations and experiments on real-world crowdsourcing data, and the experimental results corroborate our theoretical findings.
SnapToGrid: From Statistical to Interpretable Models for Biomedical Information Extraction
Authors: Marco A. Valenzuela-Escarcega, Gus Hahn-Powell, Dane Bell, Mihai Surdeanu
Date: 2016-06-30 00:00:00
Proceeding: WS 2016 8
Tasks: BIG-bench Machine Learning, Event Extraction

We propose an approach for biomedical information extraction that marries the
advantages of machine learning models, e.g., learning directly from data, with
the benefits of rule-based approaches, e.g., interpretability. Our approach
starts by training a feature-based statistical model, then converts this model
to a rule-based variant by converting its features to rules, and "snapping to
grid" the feature weights to discrete votes. In doing so, our proposal takes
advantage of the large body of work in machine learning, but it produces an
interpretable model, which can be directly edited by experts. We evaluate our
approach on the BioNLP 2009 event extraction task. Our results show that there
is a small performance penalty when converting the statistical model to rules,
but the gain in interpretability compensates for that: with minimal effort,
human experts improve this model to have similar performance to the statistical
model that served as starting point.
Exploring Prediction Uncertainty in Machine Translation Quality Estimation
Authors: Daniel Beck, Lucia Specia, Trevor Cohn
Date: 2016-06-30 00:00:00
Proceeding: CONLL 2016 8
Tasks: Machine Translation, Prediction, Translation

Machine Translation Quality Estimation is a notoriously difficult task, which
lessens its usefulness in real-world translation environments. Such scenarios
can be improved if quality predictions are accompanied by a measure of
uncertainty. However, models in this task are traditionally evaluated only in
terms of point estimate metrics, which do not take prediction uncertainty into
account. We investigate probabilistic methods for Quality Estimation that can
provide well-calibrated uncertainty estimates and evaluate them in terms of
their full posterior predictive distributions. We also show how this posterior
information can be useful in an asymmetric risk scenario, which aims to capture
typical situations in translation workflows.
Contextual Symmetries in Probabilistic Graphical Models
Authors: Ankit Anand, Aditya Grover, Mausam, Parag Singla
Date: 2016-06-30 00:00:00

An important approach for efficient inference in probabilistic graphical
models exploits symmetries among objects in the domain. Symmetric variables
(states) are collapsed into meta-variables (meta-states) and inference
algorithms are run over the lifted graphical model instead of the flat one. Our
paper extends existing definitions of symmetry by introducing the novel notion
of contextual symmetry. Two states that are not globally symmetric, can be
contextually symmetric under some specific assignment to a subset of variables,
referred to as the context variables. Contextual symmetry subsumes previous
symmetry definitions and can rep resent a large class of symmetries not
representable earlier. We show how to compute contextual symmetries by reducing
it to the problem of graph isomorphism. We extend previous work on exploiting
symmetries in the MCMC framework to the case of contextual symmetries. Our
experiments on several domains of interest demonstrate that exploiting
contextual symmetries can result in significant computational gains.
Hitting times of local and global optima in genetic algorithms with very high selection pressure
Authors: Anton Eremeev
Date: 2016-06-18 00:00:00

The paper is devoted to upper bounds on the expected first hitting times of
the sets of local or global optima for non-elitist genetic algorithms with very
high selection pressure. The results of this paper extend the range of
situations where the upper bounds on the expected runtime are known for genetic
algorithms and apply, in particular, to the Canonical Genetic Algorithm. The
obtained bounds do not require the probability of fitness-decreasing mutation
to be bounded by a constant less than one.
Cascading Bandits for Large-Scale Recommendation Problems
Authors: Shi Zong, Hao Ni, Kenny Sung, Nan Rosemary Ke, Zheng Wen, Branislav Kveton
Date: 2016-03-17 00:00:00
Tasks: Multi-Armed Bandits, Recommendation Systems, Thompson Sampling

Most recommender systems recommend a list of items. The user examines the
list, from the first item to the last, and often chooses the first attractive
item and does not examine the rest. This type of user behavior can be modeled
by the cascade model. In this work, we study cascading bandits, an online
learning variant of the cascade model where the goal is to recommend $K$ most
attractive items from a large set of $L$ candidate items. We propose two
algorithms for solving this problem, which are based on the idea of linear
generalization. The key idea in our solutions is that we learn a predictor of
the attraction probabilities of items from their features, as opposing to
learning the attraction probability of each item independently as in the
existing work. This results in practical learning algorithms whose regret does
not depend on the number of items $L$. We bound the regret of one algorithm and
comprehensively evaluate the other on a range of recommendation problems. The
algorithm performs well and outperforms all baselines.
Ordering as privileged information
Authors: Thomas Vacek
Date: 2016-06-30 00:00:00
Tasks: Model Selection

We propose to accelerate the rate of convergence of the pattern recognition
task by directly minimizing the variance diameters of certain hypothesis
spaces, which are critical quantities in fast-convergence results.We show that
the variance diameters can be controlled by dividing hypothesis spaces into
metric balls based on a new order metric. This order metric can be minimized as
an ordinal regression problem, leading to a LUPI (Learning Using Privileged
Information) application where we take the privileged information as some
desired ordering, and construct a faster-converging hypothesis space by
empirically restricting some larger hypothesis space according to that
ordering. We give a risk analysis of the approach. We discuss the difficulties
with model selection and give an innovative technique for selecting multiple
model parameters. Finally, we provide some data experiments.
Neural Network-based Word Alignment through Score Aggregation
Authors: Joel Legrand, Michael Auli, Ronan Collobert
Date: 2016-06-30 00:00:00
Proceeding: WS 2016 8
Tasks: Sentence, Word Alignment

We present a simple neural network for word alignment that builds source and
target word window representations to compute alignment scores for sentence
pairs. To enable unsupervised training, we use an aggregation operation that
summarizes the alignment scores for a given target word. A soft-margin
objective increases scores for true target words while decreasing scores for
target words that are not present. Compared to the popular Fast Align model,
our approach improves alignment accuracy by 7 AER on English-Czech, by 6 AER on
Romanian-English and by 1.7 AER on English-French alignment.
Probabilistic Reasoning in the Description Logic ALCP with the Principle of Maximum Entropy (Full Version)
Authors: Rafael Peñaloza, Nico Potyka
Date: 2016-06-30 00:00:00

A central question for knowledge representation is how to encode and handle
uncertain knowledge adequately. We introduce the probabilistic description
logic ALCP that is designed for representing context-dependent knowledge, where
the actual context taking place is uncertain. ALCP allows the expression of
logical dependencies on the domain and probabilistic dependencies on the
possible contexts. In order to draw probabilistic conclusions, we employ the
principle of maximum entropy. We provide reasoning algorithms for this logic,
and show that it satisfies several desirable properties of probabilistic
logics.
A Model Explanation System: Latest Updates and Extensions
Authors: Ryan Turner
Date: 2016-06-30 00:00:00
Tasks: model

We propose a general model explanation system (MES) for "explaining" the
output of black box classifiers. This paper describes extensions to Turner
(2015), which is referred to frequently in the text. We use the motivating
example of a classifier trained to detect fraud in a credit card transaction
history. The key aspect is that we provide explanations applicable to a single
prediction, rather than provide an interpretable set of parameters. We focus on
explaining positive predictions (alerts). However, the presented methodology is
symmetrically applicable to negative predictions.
Programming Patterns in Dataflow Matrix Machines and Generalized Recurrent Neural Nets
Authors: Michael Bukatin, Steve Matthews, Andrey Radul
Date: 2016-06-30 00:00:00

Dataflow matrix machines arise naturally in the context of synchronous
dataflow programming with linear streams. They can be viewed as a rather
powerful generalization of recurrent neural networks. Similarly to recurrent
neural networks, large classes of dataflow matrix machines are described by
matrices of numbers, and therefore dataflow matrix machines can be synthesized
by computing their matrices. At the same time, the evidence is fairly strong
that dataflow matrix machines have sufficient expressive power to be a
convenient general-purpose programming platform. Because of the network nature
of this platform, programming patterns often correspond to patterns of
connectivity in the generalized recurrent neural networks understood as
programs. This paper explores a variety of such programming patterns.
Tractability and Decompositions of Global Cost Functions
Authors: David Allouche, Christian Bessiere, Patrice Boizumault, Simon de Givry, Patricia Gutierrez, Jimmy H. M. Lee, Kam Lun Leung, Samir Loudni, Jean-Philippe Métivier, Thomas Schiex, Yi Wu
Date: 2015-02-09 00:00:00
Tasks: ARC

Enforcing local consistencies in cost function networks is performed by
applying so-called Equivalent Preserving Transformations (EPTs) to the cost
functions. As EPTs transform the cost functions, they may break the property
that was making local consistency enforcement tractable on a global cost
function. A global cost function is called tractable projection-safe when
applying an EPT to it is tractable and does not break the tractability
property. In this paper, we prove that depending on the size r of the smallest
scopes used for performing EPTs, the tractability of global cost functions can
be preserved (r = 0) or destroyed (r > 1). When r = 1, the answer is
indefinite. We show that on a large family of cost functions, EPTs can be
computed via dynamic programming-based algorithms, leading to tractable
projection-safety. We also show that when a global cost function can be
decomposed into a Berge acyclic network of bounded arity cost functions, soft
local consistencies such as soft Directed or Virtual Arc Consistency can
directly emulate dynamic programming. These different approaches to
decomposable cost functions are then embedded in a solver for extensive
experiments that confirm the feasibility and efficiency of our proposal.
Steering a Predator Robot using a Mixed Frame/Event-Driven Convolutional Neural Network
Authors: Diederik Paul Moeys, Federico Corradi, Emmett Kerr, Philip Vance, Gautham Das, Daniel Neil, Dermot Kerr, Tobi Delbruck
Date: 2016-06-30 00:00:00

This paper describes the application of a Convolutional Neural Network (CNN)
in the context of a predator/prey scenario. The CNN is trained and run on data
from a Dynamic and Active Pixel Sensor (DAVIS) mounted on a Summit XL robot
(the predator), which follows another one (the prey). The CNN is driven by both
conventional image frames and dynamic vision sensor "frames" that consist of a
constant number of DAVIS ON and OFF events. The network is thus "data driven"
at a sample rate proportional to the scene activity, so the effective sample
rate varies from 15 Hz to 240 Hz depending on the robot speeds. The network
generates four outputs: steer right, left, center and non-visible. After
off-line training on labeled data, the network is imported on the on-board
Summit XL robot which runs jAER and receives steering directions in real time.
Successful results on closed-loop trials, with accuracies up to 87% or 92%
(depending on evaluation criteria) are reported. Although the proposed approach
discards the precise DAVIS event timing, it offers the significant advantage of
compatibility with conventional deep learning technology without giving up the
advantage of data-driven computing.
Character-based Neural Machine Translation
Authors: Marta R. Costa-jussà, José A. R. Fonollosa
Date: 2016-03-02 00:00:00
Tasks: Machine Translation, Translation, Word Embeddings

Neural Machine Translation (MT) has reached state-of-the-art results.
However, one of the main challenges that neural MT still faces is dealing with
very large vocabularies and morphologically rich languages. In this paper, we
propose a neural MT system using character-based embeddings in combination with
convolutional and highway layers to replace the standard lookup-based word
representations. The resulting unlimited-vocabulary and affix-aware source word
embeddings are tested in a state-of-the-art neural MT based on an
attention-based bidirectional recurrent neural network. The proposed MT scheme
provides improved results even when the source language is not morphologically
rich. Improvements up to 3 BLEU points are obtained in the German-English WMT
task.
Learning Crosslingual Word Embeddings without Bilingual Corpora
Authors: Long Duong, Hiroshi Kanayama, Tengfei Ma, Steven Bird, Trevor Cohn
Date: 2016-06-30 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Bilingual Lexicon Induction, Cross-Lingual Document Classification, Document Classification, General Classification, Word Embeddings, Word Similarity

Crosslingual word embeddings represent lexical items from different languages
in the same vector space, enabling transfer of NLP tools. However, previous
attempts had expensive resource requirements, difficulty incorporating
monolingual data or were unable to handle polysemy. We address these drawbacks
in our method which takes advantage of a high coverage dictionary in an EM
style training algorithm over monolingual corpora in two languages. Our model
achieves state-of-the-art performance on bilingual lexicon induction task
exceeding models using large bilingual corpora, and competitive results on the
monolingual word similarity and cross-lingual document classification task.