pwc-archive/papers-with-abstracts huggingface dataset:
Latent Predictor Networks for Code Generation
Authors: Wang Ling, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiský, Andrew Senior, Fumin Wang, Phil Blunsom
Date: 2016-03-22 00:00:00
Proceeding: ACL 2016 8
Tasks: Card Games, Code Generation, Text Generation

Many language generation tasks require the production of text conditioned on
both structured and unstructured inputs. We present a novel neural network
architecture which generates an output sequence conditioned on an arbitrary
number of input functions. Crucially, our approach allows both the choice of
conditioning context and the granularity of generation, for example characters
or tokens, to be marginalised, thus permitting scalable and effective training.
Using this framework, we address the problem of generating programming code
from a mixed natural language and structured specification. We create two new
data sets for this paradigm derived from the collectible trading card games
Magic the Gathering and Hearthstone. On these, and a third preexisting corpus,
we demonstrate that marginalising multiple predictors allows our model to
outperform strong benchmarks.
Convolution by Evolution: Differentiable Pattern Producing Networks
Authors: Chrisantha Fernando, Dylan Banarse, Malcolm Reynolds, Frederic Besse, David Pfau, Max Jaderberg, Marc Lanctot, Daan Wierstra
Date: 2016-06-08 00:00:00
Tasks: Denoising

In this work we introduce a differentiable version of the Compositional
Pattern Producing Network, called the DPPN. Unlike a standard CPPN, the
topology of a DPPN is evolved but the weights are learned. A Lamarckian
algorithm, that combines evolution and learning, produces DPPNs to reconstruct
an image. Our main result is that DPPNs can be evolved/trained to compress the
weights of a denoising autoencoder from 157684 to roughly 200 parameters, while
achieving a reconstruction accuracy comparable to a fully connected network
with more than two orders of magnitude more parameters. The regularization
ability of the DPPN allows it to rediscover (approximate) convolutional network
architectures embedded within a fully connected architecture. Such
convolutional architectures are the current state of the art for many computer
vision applications, so it is satisfying that DPPNs are capable of discovering
this structure rather than having to build it in by design. DPPNs exhibit
better generalization when tested on the Omniglot dataset after being trained
on MNIST, than directly encoded fully connected autoencoders. DPPNs are
therefore a new framework for integrating learning and evolution.
A moment-matching Ferguson and Klass algorithm
Authors: Julyan Arbel, Igor Prünster
Date: 2016-06-08 00:00:00

Completely random measures (CRM) represent the key building block of a wide
variety of popular stochastic models and play a pivotal role in modern Bayesian
Nonparametrics. A popular representation of CRMs as a random series with
decreasing jumps is due to Ferguson and Klass (1972). This can immediately be
turned into an algorithm for sampling realizations of CRMs or more elaborate
models involving transformed CRMs. However, concrete implementation requires to
truncate the random series at some threshold resulting in an approximation
error. The goal of this paper is to quantify the quality of the approximation
by a moment-matching criterion, which consists in evaluating a measure of
discrepancy between actual moments and moments based on the simulation output.
Seen as a function of the truncation level, the methodology can be used to
determine the truncation level needed to reach a certain level of precision.
The resulting moment-matching \FK algorithm is then implemented and illustrated
on several popular Bayesian nonparametric models.
Clustering with a Reject Option: Interactive Clustering as Bayesian Prior Elicitation
Authors: Akash Srivastava, James Zou, Charles Sutton
Date: 2016-02-22 00:00:00
Tasks: Clustering, Computational Efficiency

A good clustering can help a data analyst to explore and understand a data
set, but what constitutes a good clustering may depend on domain-specific and
application-specific criteria. These criteria can be difficult to formalize,
even when it is easy for an analyst to know a good clustering when she sees
one. We present a new approach to interactive clustering for data exploration,
called \ciif, based on a particularly simple feedback mechanism, in which an
analyst can choose to reject individual clusters and request new ones. The new
clusters should be different from previously rejected clusters while still
fitting the data well. We formalize this interaction in a novel Bayesian prior
elicitation framework. In each iteration, the prior is adapted to account for
all the previous feedback, and a new clustering is then produced from the
posterior distribution. To achieve the computational efficiency necessary for
an interactive setting, we propose an incremental optimization method over data
minibatches using Lagrangian relaxation. Experiments demonstrate that \ciif can
produce accurate and diverse clusterings.
DialPort: Connecting the Spoken Dialog Research Community to Real User Data
Authors: Tiancheng Zhao, Kyusong Lee, Maxine Eskenazi
Date: 2016-06-08 00:00:00

This paper describes a new spoken dialog portal that connects systems
produced by the spoken dialog academic research community and gives them access
to real users. We introduce a distributed, multi-modal, multi-agent prototype
dialog framework that affords easy integration with various remote resources,
ranging from end-to-end dialog systems to external knowledge APIs. To date, the
DialPort portal has successfully connected to the multi-domain spoken dialog
system at Cambridge University, the NOAA (National Oceanic and Atmospheric
Administration) weather API and the Yelp API.
Incorporating Copying Mechanism in Sequence-to-Sequence Learning
Authors: Jiatao Gu, Zhengdong Lu, Hang Li, Victor O. K. Li
Date: 2016-03-21 00:00:00
Proceeding: ACL 2016 8
Tasks: Decoder, Text Summarization

We address an important problem in sequence-to-sequence (Seq2Seq) learning
referred to as copying, in which certain segments in the input sequence are
selectively replicated in the output sequence. A similar phenomenon is
observable in human language communication. For example, humans tend to repeat
entity names or even long phrases in conversation. The challenge with regard to
copying in Seq2Seq is that new machinery is needed to decide when to perform
the operation. In this paper, we incorporate copying into neural network-based
Seq2Seq learning and propose a new model called CopyNet with encoder-decoder
structure. CopyNet can nicely integrate the regular way of word generation in
the decoder with the new copying mechanism which can choose sub-sequences in
the input sequence and put them at proper places in the output sequence. Our
empirical study on both synthetic data sets and real world data sets
demonstrates the efficacy of CopyNet. For example, CopyNet can outperform
regular RNN-based model with remarkable margins on text summarization tasks.
Improving Recurrent Neural Networks For Sequence Labelling
Authors: Marco Dinarelli, Isabelle Tellier
Date: 2016-06-08 00:00:00
Tasks: POS, POS Tagging, Spoken Language Understanding

In this paper we study different types of Recurrent Neural Networks (RNN) for
sequence labeling tasks. We propose two new variants of RNNs integrating
improvements for sequence labeling, and we compare them to the more traditional
Elman and Jordan RNNs. We compare all models, either traditional or new, on
four distinct tasks of sequence labeling: two on Spoken Language Understanding
(ATIS and MEDIA); and two of POS tagging for the French Treebank (FTB) and the
Penn Treebank (PTB) corpora. The results show that our new variants of RNNs are
always more effective than the others.
Globally Normalized Transition-Based Neural Networks
Authors: Daniel Andor, Chris Alberti, David Weiss, Aliaksei Severyn, Alessandro Presta, Kuzman Ganchev, Slav Petrov, Michael Collins
Date: 2016-03-19 00:00:00
Proceeding: ACL 2016 8
Tasks: Dependency Parsing, Part-Of-Speech Tagging, Sentence, Sentence Compression

We introduce a globally normalized transition-based neural network model that
achieves state-of-the-art part-of-speech tagging, dependency parsing and
sentence compression results. Our model is a simple feed-forward neural network
that operates on a task-specific transition system, yet achieves comparable or
better accuracies than recurrent models. We discuss the importance of global as
opposed to local normalization: a key insight is that the label bias problem
implies that globally normalized models can be strictly more expressive than
locally normalized models.
Symbolic Music Data Version 1.0
Authors: Christian Walder
Date: 2016-06-08 00:00:00
Tasks: BIG-bench Machine Learning, Clustering

In this document, we introduce a new dataset designed for training machine
learning models of symbolic music data. Five datasets are provided, one of
which is from a newly collected corpus of 20K midi files. We describe our
preprocessing and cleaning pipeline, which includes the exclusion of a number
of files based on scores from a previously developed probabilistic machine
learning model. We also define training, testing and validation splits for the
new dataset, based on a clustering scheme which we also describe. Some simple
histograms are included.
Coordination Annotation Extension in the Penn Tree Bank
Authors: Jessica Ficler, Yoav Goldberg
Date: 2016-06-08 00:00:00
Proceeding: ACL 2016 8

Coordination is an important and common syntactic construction which is not
handled well by state of the art parsers. Coordinations in the Penn Treebank
are missing internal structure in many cases, do not include explicit marking
of the conjuncts and contain various errors and inconsistencies. In this work,
we initiated manual annotation process for solving these issues. We identify
the different elements in a coordination phrase and label each element with its
function. We add phrase boundaries when these are missing, unify
inconsistencies, and fix errors. The outcome is an extension of the PTB that
includes consistent and detailed structures for coordinations. We make the
coordination annotation publicly available, in hope that they will facilitate
further research into coordination disambiguation.
Representation of linguistic form and function in recurrent neural networks
Authors: Ákos Kádár, Grzegorz Chrupała, Afra Alishahi
Date: 2016-02-29 00:00:00
Proceeding: CL 2017 12
Tasks: Form, Language Modeling, Language Modelling, Sentence, Word Embeddings

We present novel methods for analyzing the activation patterns of RNNs from a
linguistic point of view and explore the types of linguistic structure they
learn. As a case study, we use a multi-task gated recurrent network
architecture consisting of two parallel pathways with shared word embeddings
trained on predicting the representations of the visual scene corresponding to
an input sentence, and predicting the next word in the same sentence. Based on
our proposed method to estimate the amount of contribution of individual tokens
in the input to the final prediction of the networks we show that the image
prediction pathway: a) is sensitive to the information structure of the
sentence b) pays selective attention to lexical categories and grammatical
functions that carry semantic information c) learns to treat the same input
token differently depending on its grammatical functions in the sentence. In
contrast the language model is comparatively more sensitive to words with a
syntactic function. Furthermore, we propose methods to ex- plore the function
of individual hidden units in RNNs and show that the two pathways of the
architecture in our case study contain specialized units tuned to patterns
informative for the task, some of which can carry activations to later time
steps to encode long-term dependencies.
A Modified Vortex Search Algorithm for Numerical Function Optimization
Authors: Berat Doğan
Date: 2016-06-08 00:00:00

The Vortex Search (VS) algorithm is one of the recently proposed
metaheuristic algorithms which was inspired from the vortical flow of the
stirred fluids. Although the VS algorithm is shown to be a good candidate for
the solution of certain optimization problems, it also has some drawbacks. In
the VS algorithm, candidate solutions are generated around the current best
solution by using a Gaussian distribution at each iteration pass. This provides
simplicity to the algorithm but it also leads to some problems along.
Especially, for the functions those have a number of local minimum points, to
select a single point to generate candidate solutions leads the algorithm to
being trapped into a local minimum point. Due to the adaptive step-size
adjustment scheme used in the VS algorithm, the locality of the created
candidate solutions is increased at each iteration pass. Therefore, if the
algorithm cannot escape a local point as quickly as possible, it becomes much
more difficult for the algorithm to escape from that point in the latter
iterations. In this study, a modified Vortex Search algorithm (MVS) is proposed
to overcome above mentioned drawback of the existing VS algorithm. In the MVS
algorithm, the candidate solutions are generated around a number of points at
each iteration pass. Computational results showed that with the help of this
modification the global search ability of the existing VS algorithm is improved
and the MVS algorithm outperformed the existing VS algorithm, PSO2011 and ABC
algorithms for the benchmark numerical function set.
Learning Convolutional Neural Networks for Graphs
Authors: Mathias Niepert, Mohamed Ahmed, Konstantin Kutzkov
Date: 2016-05-17 00:00:00
Tasks: Graph Classification

Numerous important problems can be framed as learning from graph data. We
propose a framework for learning convolutional neural networks for arbitrary
graphs. These graphs may be undirected, directed, and with both discrete and
continuous node and edge attributes. Analogous to image-based convolutional
networks that operate on locally connected regions of the input, we present a
general approach to extracting locally connected regions from graphs. Using
established benchmark data sets, we demonstrate that the learned feature
representations are competitive with state of the art graph kernels and that
their computation is highly efficient.
DefExt: A Semi Supervised Definition Extraction Tool
Authors: Luis Espinosa-Anke, Roberto Carlini, Horacio Saggion, Francesco Ronzano
Date: 2016-06-08 00:00:00
Tasks: Definition Extraction

We present DefExt, an easy to use semi supervised Definition Extraction Tool.
DefExt is designed to extract from a target corpus those textual fragments
where a term is explicitly mentioned together with its core features, i.e. its
definition. It works on the back of a Conditional Random Fields based
sequential labeling algorithm and a bootstrapping approach. Bootstrapping
enables the model to gradually become more aware of the idiosyncrasies of the
target corpus. In this paper we describe the main components of the toolkit as
well as experimental results stemming from both automatic and manual
evaluation. We release DefExt as open source along with the necessary files to
run it in any Unix machine. We also provide access to training and test data
for immediate use.
Exploring Implicit Human Responses to Robot Mistakes in a Learning from Demonstration Task
Authors: Cory J. Hayes, Maryam Moosaei, Laurel D. Riek
Date: 2016-06-08 00:00:00

As robots enter human environments, they will be expected to accomplish a
tremendous range of tasks. It is not feasible for robot designers to
pre-program these behaviors or know them in advance, so one way to address this
is through end-user programming, such as via learning from demonstration (LfD).
While significant work has been done on the mechanics of enabling robot
learning from human teachers, one unexplored aspect is enabling mutual feedback
between both the human teacher and robot during the learning process, i.e.,
implicit learning. In this paper, we explore one aspect of this mutual
understanding, grounding sequences, where both a human and robot provide
non-verbal feedback to signify their mutual understanding during interaction.
We conducted a study where people taught an autonomous humanoid robot a dance,
and performed gesture analysis to measure people's responses to the robot
during correct and incorrect demonstrations.
Point-wise mutual information-based video segmentation with high temporal consistency
Authors: Margret Keuper, Thomas Brox
Date: 2016-06-08 00:00:00
Tasks: Boundary Detection, Optical Flow Estimation, Segmentation, Video Segmentation, Video Semantic Segmentation, Vocal Bursts Intensity Prediction

In this paper, we tackle the problem of temporally consistent boundary
detection and hierarchical segmentation in videos. While finding the best
high-level reasoning of region assignments in videos is the focus of much
recent research, temporal consistency in boundary detection has so far only
rarely been tackled. We argue that temporally consistent boundaries are a key
component to temporally consistent region assignment. The proposed method is
based on the point-wise mutual information (PMI) of spatio-temporal voxels.
Temporal consistency is established by an evaluation of PMI-based point
affinities in the spectral domain over space and time. Thus, the proposed
method is independent of any optical flow computation or previously learned
motion models. The proposed low-level video segmentation method outperforms the
learning-based state of the art in terms of standard region metrics.
Learning Semantically and Additively Compositional Distributional Representations
Authors: Ran Tian, Naoaki Okazaki, Kentaro Inui
Date: 2016-06-08 00:00:00
Proceeding: ACL 2016 8
Tasks: General Classification, Relation Classification, Sentence, Sentence Completion

This paper connects a vector-based composition model to a formal semantics,
the Dependency-based Compositional Semantics (DCS). We show theoretical
evidence that the vector compositions in our model conform to the logic of DCS.
Experimentally, we show that vector-based composition brings a strong ability
to calculate similar phrases as similar vectors, achieving near
state-of-the-art on a wide range of phrase similarity tasks and relation
classification; meanwhile, DCS can guide building vectors for structured
queries that can be directly executed. We evaluate this utility on sentence
completion task and report a new state-of-the-art.
Online optimization and regret guarantees for non-additive long-term constraints
Authors: Rodolphe Jenatton, Jim Huang, Dominik Csiba, Cedric Archambeau
Date: 2016-02-17 00:00:00

We consider online optimization in the 1-lookahead setting, where the
objective does not decompose additively over the rounds of the online game. The
resulting formulation enables us to deal with non-stationary and/or long-term
constraints , which arise, for example, in online display advertising problems.
We propose an on-line primal-dual algorithm for which we obtain dynamic
cumulative regret guarantees. They depend on the convexity and the smoothness
of the non-additive penalty, as well as terms capturing the smoothness with
which the residuals of the non-stationary and long-term constraints vary over
the rounds. We conduct experiments on synthetic data to illustrate the benefits
of the non-additive penalty and show vanishing regret convergence on live
traffic data collected by a display advertising platform in production.
Tree-to-Sequence Attentional Neural Machine Translation
Authors: Akiko Eriguchi, Kazuma Hashimoto, Yoshimasa Tsuruoka
Date: 2016-03-19 00:00:00
Proceeding: ACL 2016 8
Tasks: Decoder, Machine Translation, NMT, Sentence, Translation

Most of the existing Neural Machine Translation (NMT) models focus on the
conversion of sequential data and do not directly use syntactic information. We
propose a novel end-to-end syntactic NMT model, extending a
sequence-to-sequence model with the source-side phrase structure. Our model has
an attention mechanism that enables the decoder to generate a translated word
while softly aligning it with phrases as well as words of the source sentence.
Experimental results on the WAT'15 English-to-Japanese dataset demonstrate that
our proposed model considerably outperforms sequence-to-sequence attentional
NMT models and compares favorably with the state-of-the-art tree-to-string SMT
system.
Multiple-Play Bandits in the Position-Based Model
Authors: Paul Lagrée, Claire Vernade, Olivier Cappé
Date: 2016-06-08 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: model, Position

Sequentially learning to place items in multi-position displays or lists is a
task that can be cast into the multiple-play semi-bandit setting. However, a
major concern in this context is when the system cannot decide whether the user
feedback for each item is actually exploitable. Indeed, much of the content may
have been simply ignored by the user. The present work proposes to exploit
available information regarding the display position bias under the so-called
Position-based click model (PBM). We first discuss how this model differs from
the Cascade model and its variants considered in several recent works on
multiple-play bandits. We then provide a novel regret lower bound for this
model as well as computationally efficient algorithms that display good
empirical and theoretical performance.
Learning Language Games through Interaction
Authors: Sida I. Wang, Percy Liang, Christopher D. Manning
Date: 2016-06-08 00:00:00
Proceeding: ACL 2016 8
Tasks: Semantic Parsing

We introduce a new language learning setting relevant to building adaptive
natural language interfaces. It is inspired by Wittgenstein's language games: a
human wishes to accomplish some task (e.g., achieving a certain configuration
of blocks), but can only communicate with a computer, who performs the actual
actions (e.g., removing all red blocks). The computer initially knows nothing
about language and therefore must learn it from scratch through interaction,
while the human adapts to the computer's capabilities. We created a game in a
blocks world and collected interactions from 100 people playing it. First, we
analyze the humans' strategies, showing that using compositionality and
avoiding synonyms correlates positively with task performance. Second, we
compare computer strategies, showing how to quickly learn a semantic parsing
model from scratch, and that modeling pragmatics further accelerates learning
for successful players.
On the Place of Text Data in Lifelogs, and Text Analysis via Semantic Facets
Authors: Gregory Grefenstette, Lawrence Muchemi
Date: 2016-06-08 00:00:00

Current research in lifelog data has not paid enough attention to analysis of
cognitive activities in comparison to physical activities. We argue that as we
look into the future, wearable devices are going to be cheaper and more
prevalent and textual data will play a more significant role. Data captured by
lifelogging devices will increasingly include speech and text, potentially
useful in analysis of intellectual activities. Analyzing what a person hears,
reads, and sees, we should be able to measure the extent of cognitive activity
devoted to a certain topic or subject by a learner. Test-based lifelog records
can benefit from semantic analysis tools developed for natural language
processing. We show how semantic analysis of such text data can be achieved
through the use of taxonomic subject facets and how these facets might be
useful in quantifying cognitive activity devoted to various topics in a
person's day. We are currently developing a method to automatically create
taxonomic topic vocabularies that can be applied to this detection of
intellectual activity.