pwc-archive/papers-with-abstracts huggingface dataset:
Unit Dependency Graph and its Application to Arithmetic Word Problem Solving
Authors: Subhro Roy, Dan Roth
Date: 2016-12-03 00:00:00
Tasks: Math, Natural Language Understanding

Math word problems provide a natural abstraction to a range of natural
language understanding problems that involve reasoning about quantities, such
as interpreting election results, news about casualties, and the financial
section of a newspaper. Units associated with the quantities often provide
information that is essential to support this reasoning. This paper proposes a
principled way to capture and reason about units and shows how it can benefit
an arithmetic word problem solver. This paper presents the concept of Unit
Dependency Graphs (UDGs), which provides a compact representation of the
dependencies between units of numbers mentioned in a given problem. Inducing
the UDG alleviates the brittleness of the unit extraction system and allows for
a natural way to leverage domain knowledge about unit compatibility, for word
problem solving. We introduce a decomposed model for inducing UDGs with minimal
additional annotations, and use it to augment the expressions used in the
arithmetic word problem solver of (Roy and Roth 2015) via a constrained
inference framework. We show that introduction of UDGs reduces the error of the
solver by over 10 %, surpassing all existing systems for solving arithmetic
word problems. In addition, it also makes the system more robust to adaptation
to new vocabulary and equation forms .
Positive blood culture detection in time series data using a BiLSTM network
Authors: Leen De Baets, Joeri Ruyssinck, Thomas Peiffer, Johan Decruyenaere, Filip De Turck, Femke Ongenae, Tom Dhaene
Date: 2016-12-03 00:00:00
Tasks: Cultural Vocal Bursts Intensity Prediction, Time Series, Time Series Analysis

The presence of bacteria or fungi in the bloodstream of patients is abnormal
and can lead to life-threatening conditions. A computational model based on a
bidirectional long short-term memory artificial neural network, is explored to
assist doctors in the intensive care unit to predict whether examination of
blood cultures of patients will return positive. As input it uses nine
monitored clinical parameters, presented as time series data, collected from
2177 ICU admissions at the Ghent University Hospital. Our main goal is to
determine if general machine learning methods and more specific, temporal
models, can be used to create an early detection system. This preliminary
research obtains an area of 71.95% under the precision recall curve, proving
the potential of temporal neural networks in this context.
RecSys Challenge 2016: job recommendations based on preselection of offers and gradient boosting
Authors: Andrzej Pacuk, Piotr Sankowski, Karol Węgrzycki, Adam Witkowski, Piotr Wygocki
Date: 2016-12-03 00:00:00
Tasks: regression

We present the Mim-Solution's approach to the RecSys Challenge 2016, which
ranked 2nd. The goal of the competition was to prepare job recommendations for
the users of the website Xing.com.
  Our two phase algorithm consists of candidate selection followed by the
candidate ranking. We ranked the candidates by the predicted probability that
the user will positively interact with the job offer. We have used Gradient
Boosting Decision Trees as the regression tool.
On the Pitfalls of Nested Monte Carlo
Authors: Tom Rainforth, Robert Cornish, Hongseok Yang, Frank Wood
Date: 2016-12-03 00:00:00

There is an increasing interest in estimating expectations outside of the
classical inference framework, such as for models expressed as probabilistic
programs. Many of these contexts call for some form of nested inference to be
applied. In this paper, we analyse the behaviour of nested Monte Carlo (NMC)
schemes, for which classical convergence proofs are insufficient. We give
conditions under which NMC will converge, establish a rate of convergence, and
provide empirical data that suggests that this rate is observable in practice.
Finally, we prove that general-purpose nested inference schemes are inherently
biased. Our results serve to warn of the dangers associated with naive
composition of inference and models.
Semi-supervised Kernel Metric Learning Using Relative Comparisons
Authors: Ehsan Amid, Aristides Gionis, Antti Ukkonen
Date: 2016-12-01 00:00:00
Tasks: Clustering, Metric Learning

We consider the problem of metric learning subject to a set of constraints on
relative-distance comparisons between the data items. Such constraints are
meant to reflect side-information that is not expressed directly in the feature
vectors of the data items. The relative-distance constraints used in this work
are particularly effective in expressing structures at finer level of detail
than must-link (ML) and cannot-link (CL) constraints, which are most commonly
used for semi-supervised clustering. Relative-distance constraints are thus
useful in settings where providing an ML or a CL constraint is difficult
because the granularity of the true clustering is unknown.
  Our main contribution is an efficient algorithm for learning a kernel matrix
using the log determinant divergence --- a variant of the Bregman divergence
--- subject to a set of relative-distance constraints. The learned kernel
matrix can then be employed by many different kernel methods in a wide range of
applications. In our experimental evaluations, we consider a semi-supervised
clustering setting and show empirically that kernels found by our algorithm
yield clusterings of higher quality than existing approaches that either use
ML/CL constraints or a different means to implement the supervision using
relative comparisons.
Using Discourse Signals for Robust Instructor Intervention Prediction
Authors: Muthu Kumar Chandrasekaran, Carrie Demmans Epp, Min-Yen Kan, Diane Litman
Date: 2016-12-03 00:00:00
Tasks: Prediction

We tackle the prediction of instructor intervention in student posts from
discussion forums in Massive Open Online Courses (MOOCs). Our key finding is
that using automatically obtained discourse relations improves the prediction
of when instructors intervene in student discussions, when compared with a
state-of-the-art, feature-rich baseline. Our supervised classifier makes use of
an automatic discourse parser which outputs Penn Discourse Treebank (PDTB) tags
that represent in-post discourse features. We show PDTB relation-based features
increase the robustness of the classifier and complement baseline features in
recalling more diverse instructor intervention patterns. In comprehensive
experiments over 14 MOOC offerings from several disciplines, the PDTB discourse
features improve performance on average. The resultant models are less
dependent on domain-specific vocabulary, allowing them to better generalize to
new courses.
Theoretical guarantees for approximate sampling from smooth and log-concave densities
Authors: Arnak S. Dalalyan
Date: 2014-12-23 00:00:00

Sampling from various kinds of distributions is an issue of paramount
importance in statistics since it is often the key ingredient for constructing
estimators, test procedures or confidence intervals. In many situations, the
exact sampling from a given distribution is impossible or computationally
expensive and, therefore, one needs to resort to approximate sampling
strategies. However, there is no well-developed theory providing meaningful
nonasymptotic guarantees for the approximate sampling procedures, especially in
the high-dimensional problems. This paper makes some progress in this direction
by considering the problem of sampling from a distribution having a smooth and
log-concave density defined on \(\RR^p\), for some integer \(p>0\). We
establish nonasymptotic bounds for the error of approximating the target
distribution by the one obtained by the Langevin Monte Carlo method and its
variants. We illustrate the effectiveness of the established guarantees with
various experiments. Underlying our analysis are insights from the theory of
continuous-time diffusion processes, which may be of interest beyond the
framework of log-concave densities considered in the present work.
SoftTarget Regularization: An Effective Technique to Reduce Over-Fitting in Neural Networks
Authors: Armen Aghajanyan
Date: 2016-09-21 00:00:00

Deep neural networks are learning models with a very high capacity and
therefore prone to over-fitting. Many regularization techniques such as
Dropout, DropConnect, and weight decay all attempt to solve the problem of
over-fitting by reducing the capacity of their respective models (Srivastava et
al., 2014), (Wan et al., 2013), (Krogh & Hertz, 1992). In this paper we
introduce a new form of regularization that guides the learning problem in a
way that reduces over-fitting without sacrificing the capacity of the model.
The mistakes that models make in early stages of training carry information
about the learning problem. By adjusting the labels of the current epoch of
training through a weighted average of the real labels, and an exponential
average of the past soft-targets we achieved a regularization scheme as
powerful as Dropout without necessarily reducing the capacity of the model, and
simplified the complexity of the learning problem. SoftTarget regularization
proved to be an effective tool in various neural network architectures.
EFANNA : An Extremely Fast Approximate Nearest Neighbor Search Algorithm Based on kNN Graph
Authors: Cong Fu, Deng Cai
Date: 2016-09-23 00:00:00
Tasks: graph construction

Approximate nearest neighbor (ANN) search is a fundamental problem in many
areas of data mining, machine learning and computer vision. The performance of
traditional hierarchical structure (tree) based methods decreases as the
dimensionality of data grows, while hashing based methods usually lack
efficiency in practice. Recently, the graph based methods have drawn
considerable attention. The main idea is that \emph{a neighbor of a neighbor is
also likely to be a neighbor}, which we refer as \emph{NN-expansion}. These
methods construct a $k$-nearest neighbor ($k$NN) graph offline. And at online
search stage, these methods find candidate neighbors of a query point in some
way (\eg, random selection), and then check the neighbors of these candidate
neighbors for closer ones iteratively. Despite some promising results, there
are mainly two problems with these approaches: 1) These approaches tend to
converge to local optima. 2) Constructing a $k$NN graph is time consuming. We
find that these two problems can be nicely solved when we provide a good
initialization for NN-expansion. In this paper, we propose EFANNA, an extremely
fast approximate nearest neighbor search algorithm based on $k$NN Graph. Efanna
nicely combines the advantages of hierarchical structure based methods and
nearest-neighbor-graph based methods. Extensive experiments have shown that
EFANNA outperforms the state-of-art algorithms both on approximate nearest
neighbor search and approximate nearest neighbor graph construction. To the
best of our knowledge, EFANNA is the fastest algorithm so far both on
approximate nearest neighbor graph construction and approximate nearest
neighbor search. A library EFANNA based on this research is released on Github.
Learning Joint Feature Adaptation for Zero-Shot Recognition
Authors: Ziming Zhang, Venkatesh Saligrama
Date: 2016-11-23 00:00:00
Tasks: Zero-Shot Learning

Zero-shot recognition (ZSR) aims to recognize target-domain data instances of
unseen classes based on the models learned from associated pairs of seen-class
source and target domain data. One of the key challenges in ZSR is the relative
scarcity of source-domain features (e.g. one feature vector per class), which
do not fully account for wide variability in target-domain instances. In this
paper we propose a novel framework of learning data-dependent feature
transforms for scoring similarity between an arbitrary pair of source and
target data instances to account for the wide variability in target domain. Our
proposed approach is based on optimizing over a parameterized family of local
feature displacements that maximize the source-target adaptive similarity
functions. Accordingly we propose formulating zero-shot learning (ZSL) using
latent structural SVMs to learn our similarity functions from training data. As
demonstration we design a specific algorithm under the proposed framework
involving bilinear similarity functions and regularized least squares as
penalties for feature displacement. We test our approach on several benchmark
datasets for ZSR and show significant improvement over the state-of-the-art.
For instance, on aP&Y dataset we can achieve 80.89% in terms of recognition
accuracy, outperforming the state-of-the-art by 11.15%.
The Option-Critic Architecture
Authors: Pierre-Luc Bacon, Jean Harb, Doina Precup
Date: 2016-09-16 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Temporal abstraction is key to scaling up learning and planning in
reinforcement learning. While planning with temporally extended actions is well
understood, creating such abstractions autonomously from data has remained
challenging. We tackle this problem in the framework of options [Sutton, Precup
& Singh, 1999; Precup, 2000]. We derive policy gradient theorems for options
and propose a new option-critic architecture capable of learning both the
internal policies and the termination conditions of options, in tandem with the
policy over options, and without the need to provide any additional rewards or
subgoals. Experimental results in both discrete and continuous environments
showcase the flexibility and efficiency of the framework.
Commonly Uncommon: Semantic Sparsity in Situation Recognition
Authors: Mark Yatskar, Vicente Ordonez, Luke Zettlemoyer, Ali Farhadi
Date: 2016-12-03 00:00:00
Proceeding: CVPR 2017 7
Tasks: Grounded Situation Recognition, Situation Recognition, Structured Prediction

Semantic sparsity is a common challenge in structured visual classification
problems; when the output space is complex, the vast majority of the possible
predictions are rarely, if ever, seen in the training set. This paper studies
semantic sparsity in situation recognition, the task of producing structured
summaries of what is happening in images, including activities, objects and the
roles objects play within the activity. For this problem, we find empirically
that most object-role combinations are rare, and current state-of-the-art
models significantly underperform in this sparse data regime. We avoid many
such errors by (1) introducing a novel tensor composition function that learns
to share examples across role-noun combinations and (2) semantically augmenting
our training data with automatically gathered examples of rarely observed
outputs using web data. When integrated within a complete CRF-based structured
prediction model, the tensor-based approach outperforms existing state of the
art by a relative improvement of 2.11% and 4.40% on top-5 verb and noun-role
accuracy, respectively. Adding 5 million images with our semantic augmentation
techniques gives further relative improvements of 6.23% and 9.57% on top-5 verb
and noun-role accuracy.
Success Probability of Exploration: a Concrete Analysis of Learning Efficiency
Authors: Liangpeng Zhang, Ke Tang, Xin Yao
Date: 2016-12-02 00:00:00
Tasks: Reinforcement Learning

Exploration has been a crucial part of reinforcement learning, yet several
important questions concerning exploration efficiency are still not answered
satisfactorily by existing analytical frameworks. These questions include
exploration parameter setting, situation analysis, and hardness of MDPs, all of
which are unavoidable for practitioners. To bridge the gap between the theory
and practice, we propose a new analytical framework called the success
probability of exploration. We show that those important questions of
exploration above can all be answered under our framework, and the answers
provided by our framework meet the needs of practitioners better than the
existing ones. More importantly, we introduce a concrete and practical approach
to evaluating the success probabilities in certain MDPs without the need of
actually running the learning algorithm. We then provide empirical results to
verify our approach, and demonstrate how the success probability of exploration
can be used to analyse and predict the behaviours and possible outcomes of
exploration, which are the keys to the answer of the important questions of
exploration.
Creating a Real-Time, Reproducible Event Dataset
Authors: John Beieler
Date: 2016-12-02 00:00:00

The generation of political event data has remained much the same since the
mid-1990s, both in terms of data acquisition and the process of coding text
into data. Since the 1990s, however, there have been significant improvements
in open-source natural language processing software and in the availability of
digitized news content. This paper presents a new, next-generation event
dataset, named Phoenix, that builds from these and other advances. This dataset
includes improvements in the underlying news collection process and event
coding software, along with the creation of a general processing pipeline
necessary to produce daily-updated data. This paper provides a face validity
checks by briefly examining the data for the conflict in Syria, and a
comparison between Phoenix and the Integrated Crisis Early Warning System data.
Sampled Image Tagging and Retrieval Methods on User Generated Content
Authors: Karl Ni, Kyle Zaragoza, Charles Foster, Carmen Carrano, Barry Chen, Yonas Tesfaye, Alex Gude
Date: 2016-11-21 00:00:00
Tasks: Retrieval, TAG, Word Embeddings

Traditional image tagging and retrieval algorithms have limited value as a
result of being trained with heavily curated datasets. These limitations are
most evident when arbitrary search words are used that do not intersect with
training set labels. Weak labels from user generated content (UGC) found in the
wild (e.g., Google Photos, FlickR, etc.) have an almost unlimited number of
unique words in the metadata tags. Prior work on word embeddings successfully
leveraged unstructured text with large vocabularies, and our proposed method
seeks to apply similar cost functions to open source imagery. Specifically, we
train a deep learning image tagging and retrieval system on large scale, user
generated content (UGC) using sampling methods and joint optimization of word
embeddings. By using the Yahoo! FlickR Creative Commons (YFCC100M) dataset,
such an approach builds robustness to common unstructured data issues that
include but are not limited to irrelevant tags, misspellings, multiple
languages, polysemy, and tag imbalance. As a result, the final proposed
algorithm will not only yield comparable results to state of the art in
conventional image tagging, but will enable new capability to train algorithms
on large, scale unstructured text in the YFCC100M dataset and outperform cited
work in zero-shot capability.
Learning Operations on a Stack with Neural Turing Machines
Authors: Tristan Deleu, Joseph Dureau
Date: 2016-12-02 00:00:00

Multiple extensions of Recurrent Neural Networks (RNNs) have been proposed
recently to address the difficulty of storing information over long time
periods. In this paper, we experiment with the capacity of Neural Turing
Machines (NTMs) to deal with these long-term dependencies on well-balanced
strings of parentheses. We show that not only does the NTM emulate a stack with
its heads and learn an algorithm to recognize such words, but it is also
capable of strongly generalizing to much longer sequences.
A Theoretical Analysis of the BDeu Scores in Bayesian Network Structure Learning
Authors: Joe Suzuki
Date: 2016-07-15 00:00:00

In Bayesian network structure learning (BNSL), we need the prior probability
over structures and parameters. If the former is the uniform distribution, the
latter determines the correctness of BNSL. In this paper, we compare BDeu
(Bayesian Dirichlet equivalent uniform) and Jeffreys' prior w.r.t. their
consistency. When we seek a parent set $U$ of a variable $X$, we require
regularity that if $H(X|U)\leq H(X|U')$ and $U\subsetneq U'$, then $U$ should
be chosen rather than $U'$. We prove that the BDeu scores violate the property
and cause fatal situations in BNSL. This is because for the BDeu scores, for
any sample size $n$,there exists a probability in the form
$P(X,Y,Z)={P(XZ)P(YZ)}/{P(Z)}$ such that the probability of deciding that $X$
and $Y$ are not conditionally independent given $Z$ is more than a half. For
Jeffreys' prior, the false-positive probability uniformly converges to zero
without depending on any parameter values, and no such an inconvenience occurs.
Learning with Hierarchical Gaussian Kernels
Authors: Ingo Steinwart, Philipp Thomann, Nico Schmid
Date: 2016-12-02 00:00:00

We investigate iterated compositions of weighted sums of Gaussian kernels and
provide an interpretation of the construction that shows some similarities with
the architectures of deep neural networks. On the theoretical side, we show
that these kernels are universal and that SVMs using these kernels are
universally consistent. We further describe a parameter optimization method for
the kernel parameters and empirically compare this method to SVMs, random
forests, a multiple kernel learning approach, and to some deep neural networks.
Summary - TerpreT: A Probabilistic Programming Language for Program Induction
Authors: Alexander L. Gaunt, Marc Brockschmidt, Rishabh Singh, Nate Kushman, Pushmeet Kohli, Jonathan Taylor, Daniel Tarlow
Date: 2016-12-02 00:00:00
Tasks: Probabilistic Programming, Program induction, Program Synthesis

We study machine learning formulations of inductive program synthesis; that
is, given input-output examples, synthesize source code that maps inputs to
corresponding outputs. Our key contribution is TerpreT, a domain-specific
language for expressing program synthesis problems. A TerpreT model is composed
of a specification of a program representation and an interpreter that
describes how programs map inputs to outputs. The inference task is to observe
a set of input-output examples and infer the underlying program. From a TerpreT
model we automatically perform inference using four different back-ends:
gradient descent (thus each TerpreT model can be seen as defining a
differentiable interpreter), linear program (LP) relaxations for graphical
models, discrete satisfiability solving, and the Sketch program synthesis
system. TerpreT has two main benefits. First, it enables rapid exploration of a
range of domains, program representations, and interpreter models. Second, it
separates the model specification from the inference algorithm, allowing proper
comparisons between different approaches to inference.
  We illustrate the value of TerpreT by developing several interpreter models
and performing an extensive empirical comparison between alternative inference
algorithms on a variety of program models. To our knowledge, this is the first
work to compare gradient-based search over program space to traditional
search-based alternatives. Our key empirical finding is that constraint solvers
dominate the gradient descent and LP-based formulations.
  This is a workshop summary of a longer report at arXiv:1608.04428