pwc-archive/papers-with-abstracts huggingface dataset:
Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation
Authors: Jie Zhou, Ying Cao, Xuguang Wang, Peng Li, Wei Xu
Date: 2016-06-14 00:00:00
Proceeding: TACL 2016 1
Tasks: Machine Translation, NMT, Translation

Neural machine translation (NMT) aims at solving machine translation (MT)
problems using neural networks and has exhibited promising results in recent
years. However, most of the existing NMT models are shallow and there is still
a performance gap between a single NMT model and the best conventional MT
system. In this work, we introduce a new type of linear connections, named
fast-forward connections, based on deep Long Short-Term Memory (LSTM) networks,
and an interleaved bi-directional architecture for stacking the LSTM layers.
Fast-forward connections play an essential role in propagating the gradients
and building a deep topology of depth 16. On the WMT'14 English-to-French task,
we achieve BLEU=37.7 with a single attention model, which outperforms the
corresponding single shallow model by 6.2 BLEU points. This is the first time
that a single NMT model achieves state-of-the-art performance and outperforms
the best conventional model by 0.7 BLEU points. We can still achieve BLEU=36.3
even without using an attention mechanism. After special handling of unknown
words and model ensembling, we obtain the best score reported to date on this
task with BLEU=40.4. Our models are also validated on the more difficult WMT'14
English-to-German task.
Scale-aware Pixel-wise Object Proposal Networks
Authors: Zequn Jie, Xiaodan Liang, Jiashi Feng, Wen Feng Lu, Eng Hock Francis Tay, Shuicheng Yan
Date: 2016-01-19 00:00:00
Tasks: Object, object-detection, Object Detection, Philosophy

Object proposal is essential for current state-of-the-art object detection
pipelines. However, the existing proposal methods generally fail in producing
results with satisfying localization accuracy. The case is even worse for small
objects which however are quite common in practice. In this paper we propose a
novel Scale-aware Pixel-wise Object Proposal (SPOP) network to tackle the
challenges. The SPOP network can generate proposals with high recall rate and
average best overlap (ABO), even for small objects. In particular, in order to
improve the localization accuracy, a fully convolutional network is employed
which predicts locations of object proposals for each pixel. The produced
ensemble of pixel-wise object proposals enhances the chance of hitting the
object significantly without incurring heavy extra computational cost. To solve
the challenge of localizing objects at small scale, two localization networks
which are specialized for localizing objects with different scales are
introduced, following the divide-and-conquer philosophy. Location outputs of
these two networks are then adaptively combined to generate the final proposals
by a large-/small-size weighting network. Extensive evaluations on PASCAL VOC
2007 show the SPOP network is superior over the state-of-the-art models. The
high-quality proposals from SPOP network also significantly improve the mean
average precision (mAP) of object detection with Fast-RCNN framework. Finally,
the SPOP network (trained on PASCAL VOC) shows great generalization performance
when testing it on ILSVRC 2013 validation set.
Image Co-localization by Mimicking a Good Detector's Confidence Score Distribution
Authors: Yao Li, Linqiao Liu, Chunhua Shen, Anton Van Den Hengel
Date: 2016-03-15 00:00:00
Tasks: Object

Given a set of images containing objects from the same category, the task of
image co-localization is to identify and localize each instance. This paper
shows that this problem can be solved by a simple but intriguing idea, that is,
a common object detector can be learnt by making its detection confidence
scores distributed like those of a strongly supervised detector. More
specifically, we observe that given a set of object proposals extracted from an
image that contains the object of interest, an accurate strongly supervised
object detector should give high scores to only a small minority of proposals,
and low scores to most of them. Thus, we devise an entropy-based objective
function to enforce the above property when learning the common object
detector. Once the detector is learnt, we resort to a segmentation approach to
refine the localization. We show that despite its simplicity, our approach
outperforms state-of-the-art methods.
A DEMATEL-Based Completion Method for Incomplete Pairwise Comparison Matrix in AHP
Authors: Xinyi Zhou, Yong Hu, Yong Deng, Felix T. S. Chan, Alessio Ishizak
Date: 2016-07-23 00:00:00
Tasks: Decision Making, Missing Values, Relation

Pairwise comparison matrix as a crucial component of AHP, presents the
prefer- ence relations among alternatives. However, in many cases, the pairwise
comparison matrix is difficult to complete, which obstructs the subsequent
operations of the clas- sical AHP. In this paper, based on DEMATEL which has
ability to derive the total relation matrix from direct relation matrix, a new
completion method for incomplete pairwise comparison matrix is proposed. The
proposed method provides a new per- spective to estimate the missing values
with explicit physical meaning. Besides, the proposed method has low
computational cost. This promising method has a wide application in
multi-criteria decision-making.
Rank Correlation Measure: A Representational Transformation for Biometric Template Protection
Authors: Zhe Jin, Yen-Lung Lai, Andrew Beng Jin Teoh
Date: 2016-07-23 00:00:00

Despite a variety of theoretical-sound techniques have been proposed for
biometric template protection, there is rarely practical solution that
guarantees non-invertibility, cancellability, non-linkability and performance
simultaneously. In this paper, a ranking-based representational transformation
is proposed for fingerprint templates. The proposed method transforms a
real-valued feature vector into index code such that the pairwise-order measure
in the resultant codes are closely correlated with rank similarity measure.
Such a ranking based technique offers two major merits: 1) Resilient to
noises/perturbations in numeric values; and 2) Highly nonlinear embedding based
on partial order statistics. The former takes care of the accuracy performance
mitigating numeric noises/perturbations while the latter offers strong
non-invertible transformation via nonlinear feature embedding from Euclidean to
Rank space that leads to toughness in inversion. The experimental results
demonstrate reasonable accuracy performance on benchmark FVC2002 and FVC2004
fingerprint databases, thus confirm the proposition of the rank correlation.
Moreover, the security and privacy analysis justify the strong capability
against the existing major privacy attacks.
Conducting sparse feature selection on arbitrarily long phrases in text corpora with a focus on interpretability
Authors: Luke Miratrix, Robin Ackerman
Date: 2015-11-20 00:00:00
Tasks: feature selection

We propose a general framework for topic-specific summarization of large text
corpora, and illustrate how it can be used for analysis in two quite different
contexts: an OSHA database of fatality and catastrophe reports (to facilitate
surveillance for patterns in circumstances leading to injury or death) and
legal decisions on workers' compensation claims (to explore relevant case law).
Our summarization framework, built on sparse classification methods, is a
compromise between simple word frequency based methods currently in wide use,
and more heavyweight, model-intensive methods such as Latent Dirichlet
Allocation (LDA). For a particular topic of interest (e.g., mental health
disability, or chemical reactions), we regress a labeling of documents onto the
high-dimensional counts of all the other words and phrases in the documents.
The resulting small set of phrases found as predictive are then harvested as
the summary. Using a branch-and-bound approach, this method can be extended to
allow for phrases of arbitrary length, which allows for potentially rich
summarization. We discuss how focus on the purpose of the summaries can inform
choices of regularization parameters and model constraints. We evaluate this
tool by comparing computational time and summary statistics of the resulting
word lists to three other methods in the literature. We also present a new R
package, textreg. Overall, we argue that sparse methods have much to offer text
analysis, and is a branch of research that should be considered further in this
context.
On the Evaluation of Dialogue Systems with Next Utterance Classification
Authors: Ryan Lowe, Iulian V. Serban, Mike Noseworthy, Laurent Charlin, Joelle Pineau
Date: 2016-05-18 00:00:00
Proceeding: WS 2016 9
Tasks: Classification, General Classification

An open challenge in constructing dialogue systems is developing methods for
automatically learning dialogue strategies from large amounts of unlabelled
data. Recent work has proposed Next-Utterance-Classification (NUC) as a
surrogate task for building dialogue systems from text data. In this paper we
investigate the performance of humans on this task to validate the relevance of
NUC as a method of evaluation. Our results show three main findings: (1) humans
are able to correctly classify responses at a rate much better than chance,
thus confirming that the task is feasible, (2) human performance levels vary
across task domains (we consider 3 datasets) and expertise levels (novice vs
experts), thus showing that a range of performance is possible on this type of
task, (3) automated dialogue systems built using state-of-the-art machine
learning methods have similar performance to the human novices, but worse than
the experts, thus confirming the utility of this class of tasks for driving
further research in automated dialogue systems.
CFGs-2-NLU: Sequence-to-Sequence Learning for Mapping Utterances to Semantics and Pragmatics
Authors: Adam James Summerville, James Ryan, Michael Mateas, Noah Wardrip-Fruin
Date: 2016-07-22 00:00:00
Tasks: Natural Language Understanding

In this paper, we present a novel approach to natural language understanding
that utilizes context-free grammars (CFGs) in conjunction with
sequence-to-sequence (seq2seq) deep learning. Specifically, we take a CFG
authored to generate dialogue for our target application for NLU, a videogame,
and train a long short-term memory (LSTM) recurrent neural network (RNN) to map
the surface utterances that it produces to traces of the grammatical expansions
that yielded them. Critically, this CFG was authored using a tool we have
developed that supports arbitrary annotation of the nonterminal symbols in the
grammar. Because we already annotated the symbols in this grammar for the
semantic and pragmatic considerations that our game's dialogue manager operates
over, we can use the grammatical trace associated with any surface utterance to
infer such information. During gameplay, we translate player utterances into
grammatical traces (using our RNN), collect the mark-up attributed to the
symbols included in that trace, and pass this information to the dialogue
manager, which updates the conversation state accordingly. From an offline
evaluation task, we demonstrate that our trained RNN translates surface
utterances to grammatical traces with great accuracy. To our knowledge, this is
the first usage of seq2seq learning for conversational agents (our game's
characters) who explicitly reason over semantic and pragmatic considerations.
A Latent Variable Model Approach to PMI-based Word Embeddings
Authors: Sanjeev Arora, Yuanzhi Li, YIngyu Liang, Tengyu Ma, Andrej Risteski
Date: 2015-02-12 00:00:00
Proceeding: TACL 2016 1
Tasks: Word Embeddings

Semantic word embeddings represent the meaning of a word via a vector, and are created by diverse methods. Many use nonlinear operations on co-occurrence statistics, and have hand-tuned hyperparameters and reweighting methods. This paper proposes a new generative model, a dynamic version of the log-linear topic model of~\citet{mnih2007three}. The methodological novelty is to use the prior to compute closed form expressions for word statistics. This provides a theoretical justification for nonlinear models like PMI, word2vec, and GloVe, as well as some hyperparameter choices. It also helps explain why low-dimensional semantic embeddings contain linear algebraic structure that allows solution of word analogies, as shown by~\citet{mikolov2013efficient} and many subsequent papers. Experimental support is provided for the generative model assumptions, the most important of which is that latent word vectors are fairly uniformly dispersed in space.
The Fast Bilateral Solver
Authors: Jonathan T. Barron, Ben Poole
Date: 2015-11-10 00:00:00
Tasks: Colorization, Semantic Segmentation

We present the bilateral solver, a novel algorithm for edge-aware smoothing
that combines the flexibility and speed of simple filtering approaches with the
accuracy of domain-specific optimization algorithms. Our technique is capable
of matching or improving upon state-of-the-art results on several different
computer vision tasks (stereo, depth superresolution, colorization, and
semantic segmentation) while being 10-1000 times faster than competing
approaches. The bilateral solver is fast, robust, straightforward to generalize
to new domains, and simple to integrate into deep learning pipelines.
Prior-based Coregistration and Cosegmentation
Authors: Mahsa Shakeri, Enzo Ferrante, Stavros Tsogkas, Sarah Lippe, Samuel Kadoury, Iasonas Kokkinos, Nikos Paragios
Date: 2016-07-22 00:00:00

We propose a modular and scalable framework for dense coregistration and
cosegmentation with two key characteristics: first, we substitute ground truth
data with the semantic map output of a classifier; second, we combine this
output with population deformable registration to improve both alignment and
segmentation. Our approach deforms all volumes towards consensus, taking into
account image similarities and label consistency. Our pipeline can incorporate
any classifier and similarity metric. Results on two datasets, containing
annotations of challenging brain structures, demonstrate the potential of our
method.
Can DMD obtain a Scene Background in Color?
Authors: Santosh Tirunagari, Norman Poh, Miroslaw Bober, David Windridge
Date: 2016-07-22 00:00:00

A background model describes a scene without any foreground objects and has a
number of applications, ranging from video surveillance to computational
photography. Recent studies have introduced the method of Dynamic Mode
Decomposition (DMD) for robustly separating video frames into a background
model and foreground components. While the method introduced operates by
converting color images to grayscale, we in this study propose a technique to
obtain the background model in the color domain. The effectiveness of our
technique is demonstrated using a publicly available Scene Background
Initialisation (SBI) dataset. Our results both qualitatively and quantitatively
show that DMD can successfully obtain a colored background model.
Untangling AdaBoost-based Cost-Sensitive Classification. Part II: Empirical Analysis
Authors: Iago Landesa-Vázquez, José Luis Alba-Castro
Date: 2015-07-15 00:00:00
Tasks: Classification, Clustering, General Classification

A lot of approaches, each following a different strategy, have been proposed
in the literature to provide AdaBoost with cost-sensitive properties. In the
first part of this series of two papers, we have presented these algorithms in
a homogeneous notational framework, proposed a clustering scheme for them and
performed a thorough theoretical analysis of those approaches with a fully
theoretical foundation. The present paper, in order to complete our analysis,
is focused on the empirical study of all the algorithms previously presented
over a wide range of heterogeneous classification problems. The results of our
experiments, confirming the theoretical conclusions, seem to reveal that the
simplest approach, just based on cost-sensitive weight initialization, is the
one showing the best and soundest results, despite having been recurrently
overlooked in the literature.
Untangling AdaBoost-based Cost-Sensitive Classification. Part I: Theoretical Perspective
Authors: Iago Landesa-Vázquez, José Luis Alba-Castro
Date: 2015-07-15 00:00:00
Tasks: Classification, General Classification

Boosting algorithms have been widely used to tackle a plethora of problems.
In the last few years, a lot of approaches have been proposed to provide
standard AdaBoost with cost-sensitive capabilities, each with a different
focus. However, for the researcher, these algorithms shape a tangled set with
diffuse differences and properties, lacking a unifying analysis to jointly
compare, classify, evaluate and discuss those approaches on a common basis. In
this series of two papers we aim to revisit the various proposals, both from
theoretical (Part I) and practical (Part II) perspectives, in order to analyze
their specific properties and behavior, with the final goal of identifying the
algorithm providing the best and soundest results.
Stochastic Neural Networks with Monotonic Activation Functions
Authors: Siamak Ravanbakhsh, Barnabas Poczos, Jeff Schneider, Dale Schuurmans, Russell Greiner
Date: 2016-01-01 00:00:00

We propose a Laplace approximation that creates a stochastic unit from any
smooth monotonic activation function, using only Gaussian noise. This paper
investigates the application of this stochastic approximation in training a
family of Restricted Boltzmann Machines (RBM) that are closely linked to
Bregman divergences. This family, that we call exponential family RBM
(Exp-RBM), is a subset of the exponential family Harmoniums that expresses
family members through a choice of smooth monotonic non-linearity for each
neuron. Using contrastive divergence along with our Gaussian approximation, we
show that Exp-RBM can learn useful representations using novel stochastic
units.
Predicting Enemy's Actions Improves Commander Decision-Making
Authors: Michael Ownby, Alexander Kott
Date: 2016-07-22 00:00:00
Tasks: Decision Making

The Defense Advanced Research Projects Agency (DARPA) Real-time Adversarial
Intelligence and Decision-making (RAID) program is investigating the
feasibility of "reading the mind of the enemy" - to estimate and anticipate, in
real-time, the enemy's likely goals, deceptions, actions, movements and
positions. This program focuses specifically on urban battles at echelons of
battalion and below. The RAID program leverages approximate game-theoretic and
deception-sensitive algorithms to provide real-time enemy estimates to a
tactical commander. A key hypothesis of the program is that these predictions
and recommendations will make the commander more effective, i.e. he should be
able to achieve his operational goals safer, faster, and more efficiently.
Realistic experimentation and evaluation drive the development process using
human-in-the-loop wargames to compare humans and the RAID system. Two
experiments were conducted in 2005 as part of Phase I to determine if the RAID
software could make predictions and recommendations as effectively and
accurately as a 4-person experienced staff. This report discusses the
intriguing and encouraging results of these first two experiments conducted by
the RAID program. It also provides details about the experiment environment and
methodology that were used to demonstrate and prove the research goals.
Validation of Information Fusion
Authors: Alexander Kott, Wes Milks
Date: 2016-07-22 00:00:00

We motivate and offer a formal definition of validation as it applies to
information fusion systems. Common definitions of validation compare the actual
state of the world with that derived by the fusion process. This definition
conflates properties of the fusion system with properties of systems that
intervene between the world and the fusion system. We propose an alternative
definition where validation of an information fusion system references a
standard fusion device, such as recognized human experts. We illustrate the
approach by describing the validation process implemented in RAID, a program
conducted by DARPA and focused on information fusion in adversarial, deceptive
environments.
Labeled compression schemes for extremal classes
Authors: Shay Moran, Manfred K. Warmuth
Date: 2015-05-30 00:00:00

It is a long-standing open problem whether there always exists a compression
scheme whose size is of the order of the Vapnik-Chervonienkis (VC) dimension
$d$. Recently compression schemes of size exponential in $d$ have been found
for any concept class of VC dimension $d$. Previously, compression schemes of
size $d$ have been given for maximum classes, which are special concept classes
whose size equals an upper bound due to Sauer-Shelah. We consider a
generalization of maximum classes called extremal classes. Their definition is
based on a powerful generalization of the Sauer-Shelah bound called the
Sandwich Theorem, which has been studied in several areas of combinatorics and
computer science. The key result of the paper is a construction of a sample
compression scheme for extremal classes of size equal to their VC dimension. We
also give a number of open problems concerning the combinatorial structure of
extremal classes and the existence of unlabeled compression schemes for them.
Parsing Argumentation Structures in Persuasive Essays
Authors: Christian Stab, Iryna Gurevych
Date: 2016-04-25 00:00:00
Proceeding: CL 2017 9

In this article, we present a novel approach for parsing argumentation
structures. We identify argument components using sequence labeling at the
token level and apply a new joint model for detecting argumentation structures.
The proposed model globally optimizes argument component types and
argumentative relations using integer linear programming. We show that our
model considerably improves the performance of base classifiers and
significantly outperforms challenging heuristic baselines. Moreover, we
introduce a novel corpus of persuasive essays annotated with argumentation
structures. We show that our annotation scheme and annotation guidelines
successfully guide human annotators to substantial agreement. This corpus and
the annotation guidelines are freely available for ensuring reproducibility and
to encourage future research in computational argumentation.
Latent Variable Discovery Using Dependency Patterns
Authors: Xuhui Zhang, Kevin B. Korb, Ann E. Nicholson, Steven Mascaro
Date: 2016-07-22 00:00:00
Tasks: Causal Discovery

The causal discovery of Bayesian networks is an active and important research
area, and it is based upon searching the space of causal models for those which
can best explain a pattern of probabilistic dependencies shown in the data.
However, some of those dependencies are generated by causal structures
involving variables which have not been measured, i.e., latent variables. Some
such patterns of dependency "reveal" themselves, in that no model based solely
upon the observed variables can explain them as well as a model using a latent
variable. That is what latent variable discovery is based upon. Here we did a
search for finding them systematically, so that they may be applied in latent
variable discovery in a more rigorous fashion.