pwc-archive/papers-with-abstracts huggingface dataset:
Full-Time Supervision based Bidirectional RNN for Factoid Question Answering
Authors: Dong Xu, Wu-Jun Li
Date: 2016-06-19 00:00:00
Tasks: Question Answering

Recently, bidirectional recurrent neural network (BRNN) has been widely used
for question answering (QA) tasks with promising performance. However, most
existing BRNN models extract the information of questions and answers by
directly using a pooling operation to generate the representation for loss or
similarity calculation. Hence, these existing models don't put supervision
(loss or similarity calculation) at every time step, which will lose some
useful information. In this paper, we propose a novel BRNN model called
full-time supervision based BRNN (FTS-BRNN), which can put supervision at every
time step. Experiments on the factoid QA task show that our FTS-BRNN can
outperform other baselines to achieve the state-of-the-art accuracy.
A Character-Level Decoder without Explicit Segmentation for Neural Machine Translation
Authors: Junyoung Chung, Kyunghyun Cho, Yoshua Bengio
Date: 2016-03-19 00:00:00
Proceeding: ACL 2016 8
Tasks: Decoder, de-en, Machine Translation, Segmentation, Translation

The existing machine translation systems, whether phrase-based or neural,
have relied almost exclusively on word-level modelling with explicit
segmentation. In this paper, we ask a fundamental question: can neural machine
translation generate a character sequence without any explicit segmentation? To
answer this question, we evaluate an attention-based encoder-decoder with a
subword-level encoder and a character-level decoder on four language
pairs--En-Cs, En-De, En-Ru and En-Fi-- using the parallel corpora from WMT'15.
Our experiments show that the models with a character-level decoder outperform
the ones with a subword-level decoder on all of the four language pairs.
Furthermore, the ensembles of neural models with a character-level decoder
outperform the state-of-the-art non-neural machine translation systems on
En-Cs, En-De and En-Fi and perform comparably on En-Ru.
Kernel-based Generative Learning in Distortion Feature Space
Authors: Bo Tang, Paul M. Baggenstoss, Haibo He
Date: 2016-06-21 00:00:00
Tasks: Diversity, General Classification

This paper presents a novel kernel-based generative classifier which is
defined in a distortion subspace using polynomial series expansion, named
Kernel-Distortion (KD) classifier. An iterative kernel selection algorithm is
developed to steadily improve classification performance by repeatedly removing
and adding kernels. The experimental results on character recognition
application not only show that the proposed generative classifier performs
better than many existing classifiers, but also illustrate that it has
different recognition capability compared to the state-of-the-art
discriminative classifier - deep belief network. The recognition diversity
indicates that a hybrid combination of the proposed generative classifier and
the discriminative classifier could further improve the classification
performance. Two hybrid combination methods, cascading and stacking, have been
implemented to verify the diversity and the improvement of the proposed
classifier.
Approachability in Stackelberg Stochastic Games with Vector Costs
Authors: Dileep Kalathil, Vivek Borkar, Rahul Jain
Date: 2014-11-03 00:00:00
Tasks: Decision Making, Reinforcement Learning

The notion of approachability was introduced by Blackwell [1] in the context
of vector-valued repeated games. The famous Blackwell's approachability theorem
prescribes a strategy for approachability, i.e., for `steering' the average
cost of a given agent towards a given target set, irrespective of the
strategies of the other agents. In this paper, motivated by the multi-objective
optimization/decision making problems in dynamically changing environments, we
address the approachability problem in Stackelberg stochastic games with vector
valued cost functions. We make two main contributions. Firstly, we give a
simple and computationally tractable strategy for approachability for
Stackelberg stochastic games along the lines of Blackwell's. Secondly, we give
a reinforcement learning algorithm for learning the approachable strategy when
the transition kernel is unknown. We also recover as a by-product Blackwell's
necessary and sufficient condition for approachability for convex sets in this
set up and thus a complete characterization. We also give sufficient conditions
for non-convex sets.
Contextual Weisfeiler-Lehman Graph Kernel For Malware Detection
Authors: Annamalai Narayanan, Guozhu Meng, Liu Yang, Jinliang Liu, Lihui Chen
Date: 2016-06-21 00:00:00
Tasks: Malware Detection

In this paper, we propose a novel graph kernel specifically to address a
challenging problem in the field of cyber-security, namely, malware detection.
Previous research has revealed the following: (1) Graph representations of
programs are ideally suited for malware detection as they are robust against
several attacks, (2) Besides capturing topological neighbourhoods (i.e.,
structural information) from these graphs it is important to capture the
context under which the neighbourhoods are reachable to accurately detect
malicious neighbourhoods.
  We observe that state-of-the-art graph kernels, such as Weisfeiler-Lehman
kernel (WLK) capture the structural information well but fail to capture
contextual information. To address this, we develop the Contextual
Weisfeiler-Lehman kernel (CWLK) which is capable of capturing both these types
of information. We show that for the malware detection problem, CWLK is more
expressive and hence more accurate than WLK while maintaining comparable
efficiency. Through our large-scale experiments with more than 50,000
real-world Android apps, we demonstrate that CWLK outperforms two
state-of-the-art graph kernels (including WLK) and three malware detection
techniques by more than 5.27% and 4.87% F-measure, respectively, while
maintaining high efficiency. This high accuracy and efficiency make CWLK
suitable for large-scale real-world malware detection.
FSMJ: Feature Selection with Maximum Jensen-Shannon Divergence for Text Categorization
Authors: Bo Tang, Haibo He
Date: 2016-06-20 00:00:00
Tasks: feature selection, Text Categorization

In this paper, we present a new wrapper feature selection approach based on
Jensen-Shannon (JS) divergence, termed feature selection with maximum
JS-divergence (FSMJ), for text categorization. Unlike most existing feature
selection approaches, the proposed FSMJ approach is based on real-valued
features which provide more information for discrimination than binary-valued
features used in conventional approaches. We show that the FSMJ is a greedy
approach and the JS-divergence monotonically increases when more features are
selected. We conduct several experiments on real-life data sets, compared with
the state-of-the-art feature selection approaches for text categorization. The
superior performance of the proposed FSMJ approach demonstrates its
effectiveness and further indicates its wide potential applications on data
mining.
A Probabilistic Generative Grammar for Semantic Parsing
Authors: Abulhair Saparov
Date: 2016-06-20 00:00:00
Proceeding: CONLL 2017 8
Tasks: Natural Language Understanding, Semantic Parsing, Sentence, Structured Prediction

Domain-general semantic parsing is a long-standing goal in natural language processing, where the semantic parser is capable of robustly parsing sentences from domains outside of which it was trained. Current approaches largely rely on additional supervision from new domains in order to generalize to those domains. We present a generative model of natural language utterances and logical forms and demonstrate its application to semantic parsing. Our approach relies on domain-independent supervision to generalize to new domains. We derive and implement efficient algorithms for training, parsing, and sentence generation. The work relies on a novel application of hierarchical Dirichlet processes (HDPs) for structured prediction, which we also present in this manuscript. This manuscript is an excerpt of chapter 4 from the Ph.D. thesis of Saparov (2022), where the model plays a central role in a larger natural language understanding system. This manuscript provides a new simplified and more complete presentation of the work first introduced in Saparov, Saraswat, and Mitchell (2017). The description and proofs of correctness of the training algorithm, parsing algorithm, and sentence generation algorithm are much simplified in this new presentation. We also describe the novel application of hierarchical Dirichlet processes for structured prediction. In addition, we extend the earlier work with a new model of word morphology, which utilizes the comprehensive morphological data from Wiktionary.
Complex Embeddings for Simple Link Prediction
Authors: Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard
Date: 2016-06-20 00:00:00
Tasks: Link Prediction, Prediction, Relational Reasoning

In statistical relational learning, the link prediction problem is key to
automatically understand the structure of large knowledge bases. As in previous
studies, we propose to solve this problem through latent factorization.
However, here we make use of complex valued embeddings. The composition of
complex embeddings can handle a large variety of binary relations, among them
symmetric and antisymmetric relations. Compared to state-of-the-art models such
as Neural Tensor Network and Holographic Embeddings, our approach based on
complex embeddings is arguably simpler, as it only uses the Hermitian dot
product, the complex counterpart of the standard dot product between real
vectors. Our approach is scalable to large datasets as it remains linear in
both space and time, while consistently outperforming alternative approaches on
standard link prediction benchmarks.
A Hierarchical Reinforcement Learning Method for Persistent Time-Sensitive Tasks
Authors: Xiao Li, Calin Belta
Date: 2016-06-20 00:00:00
Tasks: Hierarchical Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning has been applied to many interesting problems such as
the famous TD-gammon and the inverted helicopter flight. However, little effort
has been put into developing methods to learn policies for complex persistent
tasks and tasks that are time-sensitive. In this paper, we take a step towards
solving this problem by using signal temporal logic (STL) as task
specification, and taking advantage of the temporal abstraction feature that
the options framework provide. We show via simulation that a relatively easy to
implement algorithm that combines STL and options can learn a satisfactory
policy with a small number of training cases
Visualizing textual models with in-text and word-as-pixel highlighting
Authors: Abram Handler, Su Lin Blodgett, Brendan O'Connor
Date: 2016-06-20 00:00:00

We explore two techniques which use color to make sense of statistical text
models. One method uses in-text annotations to illustrate a model's view of
particular tokens in particular documents. Another uses a high-level,
"words-as-pixels" graphic to display an entire corpus. Together, these methods
offer both zoomed-in and zoomed-out perspectives into a model's understanding
of text. We show how these interconnected methods help diagnose a classifier's
poor performance on Twitter slang, and make sense of a topic model on
historical political texts.
Twitter as a Source of Global Mobility Patterns for Social Good
Authors: Mark Dredze, Manuel García-Herranz, Alex Rutherford, Gideon Mann
Date: 2016-06-20 00:00:00
Tasks: Humanitarian

Data on human spatial distribution and movement is essential for
understanding and analyzing social systems. However existing sources for this
data are lacking in various ways; difficult to access, biased, have poor
geographical or temporal resolution, or are significantly delayed. In this
paper, we describe how geolocation data from Twitter can be used to estimate
global mobility patterns and address these shortcomings. These findings will
inform how this novel data source can be harnessed to address humanitarian and
development efforts.
SS4MCT: A Statistical Stemmer for Morphologically Complex Texts
Authors: Javid Dadashkarimi, Hossein Nasr Esfahani, Heshaam Faili, Azadeh Shakery
Date: 2016-05-25 00:00:00
Tasks: Information Retrieval, Retrieval

There have been multiple attempts to resolve various inflection matching
problems in information retrieval. Stemming is a common approach to this end.
Among many techniques for stemming, statistical stemming has been shown to be
effective in a number of languages, particularly highly inflected languages. In
this paper we propose a method for finding affixes in different positions of a
word. Common statistical techniques heavily rely on string similarity in terms
of prefix and suffix matching. Since infixes are common in irregular/informal
inflections in morphologically complex texts, it is required to find infixes
for stemming. In this paper we propose a method whose aim is to find
statistical inflectional rules based on minimum edit distance table of word
pairs and the likelihoods of the rules in a language. These rules are used to
statistically stem words and can be used in different text mining tasks.
Experimental results on CLEF 2008 and CLEF 2009 English-Persian CLIR tasks
indicate that the proposed method significantly outperforms all the baselines
in terms of MAP.
A Survey of Signed Network Mining in Social Media
Authors: Jiliang Tang, Yi Chang, Charu Aggarwal, Huan Liu
Date: 2015-11-24 00:00:00
Tasks: Survey

Many real-world relations can be represented by signed networks with positive
and negative links, as a result of which signed network analysis has attracted
increasing attention from multiple disciplines. With the increasing prevalence
of social media networks, signed network analysis has evolved from developing
and measuring theories to mining tasks. In this article, we present a review of
mining signed networks in the context of social media and discuss some
promising research directions and new frontiers. We begin by giving basic
concepts and unique properties and principles of signed networks. Then we
classify and review tasks of signed network mining with representative
algorithms. We also delineate some tasks that have not been extensively studied
with formal definitions and also propose research directions to expand the
field of signed network mining.
Joint Unsupervised Learning of Deep Representations and Image Clusters
Authors: Jianwei Yang, Devi Parikh, Dhruv Batra
Date: 2016-04-13 00:00:00
Proceeding: CVPR 2016 6
Tasks: Clustering, Image Clustering, Representation Learning, Triplet

In this paper, we propose a recurrent framework for Joint Unsupervised
LEarning (JULE) of deep representations and image clusters. In our framework,
successive operations in a clustering algorithm are expressed as steps in a
recurrent process, stacked on top of representations output by a Convolutional
Neural Network (CNN). During training, image clusters and representations are
updated jointly: image clustering is conducted in the forward pass, while
representation learning in the backward pass. Our key idea behind this
framework is that good representations are beneficial to image clustering and
clustering results provide supervisory signals to representation learning. By
integrating two processes into a single model with a unified weighted triplet
loss and optimizing it end-to-end, we can obtain not only more powerful
representations, but also more precise image clusters. Extensive experiments
show that our method outperforms the state-of-the-art on image clustering
across a variety of image datasets. Moreover, the learned representations
generalize well when transferred to other tasks.
A Data-Driven Approach for Semantic Role Labeling from Induced Grammar Structures in Language
Authors: Vivek Datla, David Lin, Max Louwerse, Abhinav Vishnu
Date: 2016-06-20 00:00:00
Tasks: Position, Semantic Role Labeling

Semantic roles play an important role in extracting knowledge from text.
Current unsupervised approaches utilize features from grammar structures, to
induce semantic roles. The dependence on these grammars, however, makes it
difficult to adapt to noisy and new languages. In this paper we develop a
data-driven approach to identifying semantic roles, the approach is entirely
unsupervised up to the point where rules need to be learned to identify the
position the semantic role occurs. Specifically we develop a modified-ADIOS
algorithm based on ADIOS Solan et al. (2005) to learn grammar structures, and
use these grammar structures to learn the rules for identifying the semantic
roles based on the context in which the grammar structures appeared. The
results obtained are comparable with the current state-of-art models that are
inherently dependent on human annotated data.
Detection and Tracking of Liquids with Fully Convolutional Networks
Authors: Connor Schenck, Dieter Fox
Date: 2016-06-20 00:00:00
Tasks: Deep Learning, Image Segmentation, Semantic Segmentation

Recent advances in AI and robotics have claimed many incredible results with
deep learning, yet no work to date has applied deep learning to the problem of
liquid perception and reasoning. In this paper, we apply fully-convolutional
deep neural networks to the tasks of detecting and tracking liquids. We
evaluate three models: a single-frame network, multi-frame network, and a LSTM
recurrent network. Our results show that the best liquid detection results are
achieved when aggregating data over multiple frames, in contrast to standard
image segmentation. They also show that the LSTM network outperforms the other
two in both tasks. This suggests that LSTM-based neural networks have the
potential to be a key component for enabling robots to handle liquids using
robust, closed-loop controllers.
An Empirical Comparison of Sampling Quality Metrics: A Case Study for Bayesian Nonnegative Matrix Factorization
Authors: Arjumand Masood, Weiwei Pan, Finale Doshi-Velez
Date: 2016-06-20 00:00:00
Tasks: Diversity, valid

In this work, we empirically explore the question: how can we assess the
quality of samples from some target distribution? We assume that the samples
are provided by some valid Monte Carlo procedure, so we are guaranteed that the
collection of samples will asymptotically approximate the true distribution.
Most current evaluation approaches focus on two questions: (1) Has the chain
mixed, that is, is it sampling from the distribution? and (2) How independent
are the samples (as MCMC procedures produce correlated samples)? Focusing on
the case of Bayesian nonnegative matrix factorization, we empirically evaluate
standard metrics of sampler quality as well as propose new metrics to capture
aspects that these measures fail to expose. The aspect of sampling that is of
particular interest to us is the ability (or inability) of sampling methods to
move between multiple optima in NMF problems. As a proxy, we propose and study
a number of metrics that might quantify the diversity of a set of NMF
factorizations obtained by a sampler through quantifying the coverage of the
posterior distribution. We compare the performance of a number of standard
sampling methods for NMF in terms of these new metrics.
CNNLab: a Novel Parallel Framework for Neural Networks using GPU and FPGA-a Practical Study with Trade-off Analysis
Authors: Maohua Zhu, Liu Liu, Chao Wang, Yuan Xie
Date: 2016-06-20 00:00:00
Tasks: Deep Learning, GPU, Scheduling

Designing and implementing efficient, provably correct parallel neural
network processing is challenging. Existing high-level parallel abstractions
like MapReduce are insufficiently expressive while low-level tools like MPI and
Pthreads leave ML experts repeatedly solving the same design challenges.
However, the diversity and large-scale data size have posed a significant
challenge to construct a flexible and high-performance implementation of deep
learning neural networks. To improve the performance and maintain the
scalability, we present CNNLab, a novel deep learning framework using GPU and
FPGA-based accelerators. CNNLab provides a uniform programming model to users
so that the hardware implementation and the scheduling are invisible to the
programmers. At runtime, CNNLab leverages the trade-offs between GPU and FPGA
before offloading the tasks to the accelerators. Experimental results on the
state-of-the-art Nvidia K40 GPU and Altera DE5 FPGA board demonstrate that the
CNNLab can provide a universal framework with efficient support for diverse
applications without increasing the burden of the programmers. Moreover, we
analyze the detailed quantitative performance, throughput, power, energy, and
performance density for both approaches. Experimental results leverage the
trade-offs between GPU and FPGA and provide useful practical experiences for
the deep learning research community.
Comparing the hierarchy of keywords in on-line news portals
Authors: Gergely Tibély, David Sousa-Rodrigues, Péter Pollner, Gergely Palla
Date: 2016-06-20 00:00:00
Tasks: TAG

The tagging of on-line content with informative keywords is a widespread
phenomenon from scientific article repositories through blogs to on-line news
portals. In most of the cases, the tags on a given item are free words chosen
by the authors independently. Therefore, relations among keywords in a
collection of news items is unknown. However, in most cases the topics and
concepts described by these keywords are forming a latent hierarchy, with the
more general topics and categories at the top, and more specialised ones at the
bottom. Here we apply a recent, cooccurrence-based tag hierarchy extraction
method to sets of keywords obtained from four different on-line news portals.
The resulting hierarchies show substantial differences not just in the topics
rendered as important (being at the top of the hierarchy) or of less interest
(categorised low in the hierarchy), but also in the underlying network
structure. This reveals discrepancies between the plausible keyword association
frameworks in the studied news portals.
LSTM-Based Predictions for Proactive Information Retrieval
Authors: Petri Luukkonen, Markus Koskela, Patrik Floréen
Date: 2016-06-20 00:00:00
Tasks: Information Retrieval, Retrieval

We describe a method for proactive information retrieval targeted at
retrieving relevant information during a writing task. In our method, the
current task and the needs of the user are estimated, and the potential next
steps are unobtrusively predicted based on the user's past actions. We focus on
the task of writing, in which the user is coalescing previously collected
information into a text. Our proactive system automatically recommends the user
relevant background information. The proposed system incorporates text input
prediction using a long short-term memory (LSTM) network. We present
simulations, which show that the system is able to reach higher precision
values in an exploratory search setting compared to both a baseline and a
comparison system.
The Minimum Cost Connected Subgraph Problem in Medical Image Analysis
Authors: Markus Rempfler, Bjoern Andres, Bjoern H. Menze
Date: 2016-06-20 00:00:00
Tasks: Medical Image Analysis

Several important tasks in medical image analysis can be stated in the form
of an optimization problem whose feasible solutions are connected subgraphs.
Examples include the reconstruction of neural or vascular structures under
connectedness constraints. We discuss the minimum cost connected subgraph
(MCCS) problem and its approximations from the perspective of medical
applications. We propose a) objective-dependent constraints and b) novel
constraint generation schemes to solve this optimization problem exactly by
means of a branch-and-cut algorithm. These are shown to improve scalability and
allow us to solve instances of two medical benchmark datasets to optimality for
the first time. This enables us to perform a quantitative comparison between
exact and approximative algorithms, where we identify the geodesic tree
algorithm as an excellent alternative to exact inference on the examined
datasets.