pwc-archive/papers-with-abstracts huggingface dataset:
Learning Explanatory Rules from Noisy Data
Authors: Richard Evans, Edward Grefenstette
Date: 2017-11-13 00:00:00
Tasks: Inductive logic programming

Artificial Neural Networks are powerful function approximators capable of
modelling solutions to a wide variety of problems, both supervised and
unsupervised. As their size and expressivity increases, so too does the
variance of the model, yielding a nearly ubiquitous overfitting problem.
Although mitigated by a variety of model regularisation methods, the common
cure is to seek large amounts of training data---which is not necessarily
easily obtained---that sufficiently approximates the data distribution of the
domain we wish to test on. In contrast, logic programming methods such as
Inductive Logic Programming offer an extremely data-efficient process by which
models can be trained to reason on symbolic domains. However, these methods are
unable to deal with the variety of domains neural networks can be applied to:
they are not robust to noise in or mislabelling of inputs, and perhaps more
importantly, cannot be applied to non-symbolic domains where the data is
ambiguous, such as operating on raw pixels. In this paper, we propose a
Differentiable Inductive Logic framework, which can not only solve tasks which
traditional ILP systems are suited for, but shows a robustness to noise and
error in the training data which ILP cannot cope with. Furthermore, as it is
trained by backpropagation against a likelihood objective, it can be hybridised
by connecting it with neural networks over ambiguous data in order to be
applied to domains which ILP cannot address, while providing data efficiency
and generalisation beyond what neural networks on their own can achieve.
Query Focused Abstractive Summarization: Incorporating Query Relevance, Multi-Document Coverage, and Summary Length Constraints into seq2seq Models
Authors: Tal Baumel, Matan Eyal, Michael Elhadad
Date: 2018-01-23 00:00:00
Tasks: Abstractive Text Summarization, Document Summarization, Query-Based Extractive Summarization, Query-focused Summarization

Query Focused Summarization (QFS) has been addressed mostly using extractive
methods. Such methods, however, produce text which suffers from low coherence.
We investigate how abstractive methods can be applied to QFS, to overcome such
limitations. Recent developments in neural-attention based sequence-to-sequence
models have led to state-of-the-art results on the task of abstractive generic
single document summarization. Such models are trained in an end to end method
on large amounts of training data. We address three aspects to make abstractive
summarization applicable to QFS: (a)since there is no training data, we
incorporate query relevance into a pre-trained abstractive model; (b) since
existing abstractive models are trained in a single-document setting, we design
an iterated method to embed abstractive models within the multi-document
requirement of QFS; (c) the abstractive models we adapt are trained to generate
text of specific length (about 100 words), while we aim at generating output of
a different size (about 250 words); we design a way to adapt the target size of
the generated summaries to a given size ratio. We compare our method (Relevance
Sensitive Attention for QFS) to extractive baselines and with various ways to
combine abstractive models on the DUC QFS datasets and demonstrate solid
improvements on ROUGE performance.
What did you Mention? A Large Scale Mention Detection Benchmark for Spoken and Written Text
Authors: Yosi Mass, Lili Kotlerman, Shachar Mirkin, Elad Venezian, Gera Witzling, Noam Slonim
Date: 2018-01-23 00:00:00

We describe a large, high-quality benchmark for the evaluation of Mention
Detection tools. The benchmark contains annotations of both named entities as
well as other types of entities, annotated on different types of text, ranging
from clean text taken from Wikipedia, to noisy spoken data. The benchmark was
built through a highly controlled crowd sourcing process to ensure its quality.
We describe the benchmark, the process and the guidelines that were used to
build it. We then demonstrate the results of a state-of-the-art system running
on that benchmark.
Using Deep Autoencoders for Facial Expression Recognition
Authors: Muhammad Usman, Siddique Latif, Junaid Qadir
Date: 2018-01-25 00:00:00
Tasks: Dimensionality Reduction, Facial Expression Recognition, Facial Expression Recognition (FER), feature selection

Feature descriptors involved in image processing are generally manually
chosen and high dimensional in nature. Selecting the most important features is
a very crucial task for systems like facial expression recognition. This paper
investigates the performance of deep autoencoders for feature selection and
dimension reduction for facial expression recognition on multiple levels of
hidden layers. The features extracted from the stacked autoencoder outperformed
when compared to other state-of-the-art feature selection and dimension
reduction techniques.
Deep Convolutional Framelets: A General Deep Learning Framework for Inverse Problems
Authors: Jong Chul Ye, Yoseob Han, Eunju Cha
Date: 2017-07-03 00:00:00

Recently, deep learning approaches with various network architectures have
achieved significant performance improvement over existing iterative
reconstruction methods in various imaging problems. However, it is still
unclear why these deep learning architectures work for specific inverse
problems. To address these issues, here we show that the long-searched-for
missing link is the convolution framelets for representing a signal by
convolving local and non-local bases. The convolution framelets was originally
developed to generalize the theory of low-rank Hankel matrix approaches for
inverse problems, and this paper further extends the idea so that we can obtain
a deep neural network using multilayer convolution framelets with perfect
reconstruction (PR) under rectilinear linear unit nonlinearity (ReLU). Our
analysis also shows that the popular deep network components such as residual
block, redundant filter channels, and concatenated ReLU (CReLU) do indeed help
to achieve the PR, while the pooling and unpooling layers should be augmented
with high-pass branches to meet the PR condition. Moreover, by changing the
number of filter channels and bias, we can control the shrinkage behaviors of
the neural network. This discovery leads us to propose a novel theory for deep
convolutional framelets neural network. Using numerical experiments with
various inverse problems, we demonstrated that our deep convolution framelets
network shows consistent improvement over existing deep architectures.This
discovery suggests that the success of deep learning is not from a magical
power of a black-box, but rather comes from the power of a novel signal
representation using non-local basis combined with data-driven local basis,
which is indeed a natural extension of classical signal processing theory.
Information gain ratio correction: Improving prediction with more balanced decision tree splits
Authors: Antonin Leroux, Matthieu Boussard, Remi Dès
Date: 2018-01-25 00:00:00

Decision trees algorithms use a gain function to select the best split during
the tree's induction. This function is crucial to obtain trees with high
predictive accuracy. Some gain functions can suffer from a bias when it
compares splits of different arities. Quinlan proposed a gain ratio in C4.5's
information gain function to fix this bias. In this paper, we present an
updated version of the gain ratio that performs better as it tries to fix the
gain ratio's bias for unbalanced trees and some splits with low predictive
interest.
On the Sample Complexity of the Linear Quadratic Regulator
Authors: Sarah Dean, Horia Mania, Nikolai Matni, Benjamin Recht, Stephen Tu
Date: 2017-10-04 00:00:00

This paper addresses the optimal control problem known as the Linear
Quadratic Regulator in the case when the dynamics are unknown. We propose a
multi-stage procedure, called Coarse-ID control, that estimates a model from a
few experimental trials, estimates the error in that model with respect to the
truth, and then designs a controller using both the model and uncertainty
estimate. Our technique uses contemporary tools from random matrix theory to
bound the error in the estimation procedure. We also employ a recently
developed approach to control synthesis called System Level Synthesis that
enables robust control design by solving a convex optimization problem. We
provide end-to-end bounds on the relative error in control cost that are nearly
optimal in the number of parameters and that highlight salient properties of
the system to be controlled such as closed-loop sensitivity and optimal control
magnitude. We show experimentally that the Coarse-ID approach enables efficient
computation of a stabilizing controller in regimes where simple control schemes
that do not take the model uncertainty into account fail to stabilize the true
system.
Rademacher Complexity Bounds for a Penalized Multiclass Semi-Supervised Algorithm
Authors: Yury Maximov, Massih-Reza Amini, Zaid Harchaoui
Date: 2016-07-02 00:00:00
Tasks: Clustering

We propose Rademacher complexity bounds for multiclass classifiers trained
with a two-step semi-supervised model. In the first step, the algorithm
partitions the partially labeled data and then identifies dense clusters
containing $\kappa$ predominant classes using the labeled training examples
such that the proportion of their non-predominant classes is below a fixed
threshold. In the second step, a classifier is trained by minimizing a margin
empirical loss over the labeled training set and a penalization term measuring
the disability of the learner to predict the $\kappa$ predominant classes of
the identified clusters. The resulting data-dependent generalization error
bound involves the margin distribution of the classifier, the stability of the
clustering technique used in the first step and Rademacher complexity terms
corresponding to partially labeled training data. Our theoretical result
exhibit convergence rates extending those proposed in the literature for the
binary case, and experimental results on different multiclass classification
problems show empirical evidence that supports the theory.
SocialML: machine learning for social media video creators
Authors: Tomasz Trzcinski, Adam Bielski, Paweł Cyrta, Matthew Zak
Date: 2018-01-25 00:00:00
Tasks: BIG-bench Machine Learning

In the recent years, social media have become one of the main places where
creative content is being published and consumed by billions of users. Contrary
to traditional media, social media allow the publishers to receive almost
instantaneous feedback regarding their creative work at an unprecedented scale.
This is a perfect use case for machine learning methods that can use these
massive amounts of data to provide content creators with inspirational ideas
and constructive criticism of their work. In this work, we present a
comprehensive overview of machine learning-empowered tools we developed for
video creators at Group Nine Media - one of the major social media companies
that creates short-form videos with over three billion views per month. Our
main contribution is a set of tools that allow the creators to leverage massive
amounts of data to improve their creation process, evaluate their videos before
the publication and improve content quality. These applications include an
interactive conversational bot that allows access to material archives, a
Web-based application for automatic selection of optimal video thumbnail, as
well as deep learning methods for optimizing headline and predicting video
popularity. Our A/B tests show that deployment of our tools leads to
significant increase of average video view count by 12.9%. Our additional
contribution is a set of considerations collected during the deployment of
those tools that can hel
Automatic Cardiac Disease Assessment on cine-MRI via Time-Series Segmentation and Domain Specific Features
Authors: Fabian Isensee, Paul Jaeger, Peter M. Full, Ivo Wolf, Sandy Engelhardt, Klaus H. Maier-Hein
Date: 2017-07-03 00:00:00
Tasks: Diagnostic, General Classification, Segmentation, Time Series, Time Series Analysis

Cardiac magnetic resonance imaging improves on diagnosis of cardiovascular
diseases by providing images at high spatiotemporal resolution. Manual
evaluation of these time-series, however, is expensive and prone to biased and
non-reproducible outcomes. In this paper, we present a method that addresses
named limitations by integrating segmentation and disease classification into a
fully automatic processing pipeline. We use an ensemble of UNet inspired
architectures for segmentation of cardiac structures such as the left and right
ventricular cavity (LVC, RVC) and the left ventricular myocardium (LVM) on each
time instance of the cardiac cycle. For the classification task, information is
extracted from the segmented time-series in form of comprehensive features
handcrafted to reflect diagnostic clinical procedures. Based on these features
we train an ensemble of heavily regularized multilayer perceptrons (MLP) and a
random forest classifier to predict the pathologic target class. We evaluated
our method on the ACDC dataset (4 pathology groups, 1 healthy group) and
achieve dice scores of 0.945 (LVC), 0.908 (RVC) and 0.905 (LVM) in a
cross-validation over the training set (100 cases) and 0.950 (LVC), 0.923 (RVC)
and 0.911 (LVM) on the test set (50 cases). We report a classification accuracy
of 94% on a training set cross-validation and 92% on the test set. Our results
underpin the potential of machine learning methods for accurate, fast and
reproducible segmentation and computer-assisted diagnosis (CAD).
Class label autoencoder for zero-shot learning
Authors: Guangfeng Lin, Caixia Fan, Wanjun Chen, Yajun Chen, Fan Zhao
Date: 2018-01-25 00:00:00
Tasks: Attribute, Decoder, Diversity, zero-shot-classification, Zero-Shot Learning

Existing zero-shot learning (ZSL) methods usually learn a projection function
between a feature space and a semantic embedding space(text or attribute space)
in the training seen classes or testing unseen classes. However, the projection
function cannot be used between the feature space and multi-semantic embedding
spaces, which have the diversity characteristic for describing the different
semantic information of the same class. To deal with this issue, we present a
novel method to ZSL based on learning class label autoencoder (CLA). CLA can
not only build a uniform framework for adapting to multi-semantic embedding
spaces, but also construct the encoder-decoder mechanism for constraining the
bidirectional projection between the feature space and the class label space.
Moreover, CLA can jointly consider the relationship of feature classes and the
relevance of the semantic classes for improving zero-shot classification. The
CLA solution can provide both unseen class labels and the relation of the
different classes representation(feature or semantic information) that can
encode the intrinsic structure of classes. Extensive experiments demonstrate
the CLA outperforms state-of-art methods on four benchmark datasets, which are
AwA, CUB, Dogs and ImNet-2.
Double-sided probing by map of Asplund's distances using Logarithmic Image Processing in the framework of Mathematical Morphology
Authors: Guillaume Noyel, Michel Jourlin
Date: 2017-01-27 00:00:00

We establish the link between Mathematical Morphology and the map of
Asplund's distances between a probe and a grey scale function, using the
Logarithmic Image Processing scalar multiplication. We demonstrate that the map
is the logarithm of the ratio between a dilation and an erosion of the function
by a structuring function: the probe. The dilations and erosions are mappings
from the lattice of the images into the lattice of the positive functions.
Using a flat structuring element, the expression of the map of Asplund's
distances can be simplified with a dilation and an erosion of the image; these
mappings stays in the lattice of the images. We illustrate our approach by an
example of pattern matching with a non-flat structuring function.
A simple expression for the map of Asplund's distances with the multiplicative Logarithmic Image Processing (LIP) law
Authors: Guillaume Noyel, Michel Jourlin
Date: 2017-08-23 00:00:00

We introduce a simple expression for the map of Asplund's distances with the
multiplicative Logarithmic Image Processing (LIP) law. It is a difference
between a morphological dilation and a morphological erosion with an additive
structuring function which corresponds to a morphological gradient.
Discovering Markov Blanket from Multiple interventional Datasets
Authors: Kui Yu, Lin Liu, Jiuyong Li
Date: 2018-01-25 00:00:00

In this paper, we study the problem of discovering the Markov blanket (MB) of
a target variable from multiple interventional datasets. Datasets attained from
interventional experiments contain richer causal information than passively
observed data (observational data) for MB discovery. However, almost all
existing MB discovery methods are designed for finding MBs from a single
observational dataset. To identify MBs from multiple interventional datasets,
we face two challenges: (1) unknown intervention variables; (2) nonidentical
data distributions. To tackle the challenges, we theoretically analyze (a)
under what conditions we can find the correct MB of a target variable, and (b)
under what conditions we can identify the causes of the target variable via
discovering its MB. Based on the theoretical analysis, we propose a new
algorithm for discovering MBs from multiple interventional datasets, and
present the conditions/assumptions which assure the correctness of the
algorithm. To our knowledge, this work is the first to present the theoretical
analyses about the conditions for MB discovery in multiple interventional
datasets and the algorithm to find the MBs in relation to the conditions. Using
benchmark Bayesian networks and real-world datasets, the experiments have
validated the effectiveness and efficiency of the proposed algorithm in the
paper.
Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition
Authors: Sijie Yan, Yuanjun Xiong, Dahua Lin
Date: 2018-01-23 00:00:00
Tasks: 3D Human Pose Estimation, Action Recognition, Multimodal Activity Recognition, Skeleton Based Action Recognition, Temporal Action Localization

Dynamics of human body skeletons convey significant information for human
action recognition. Conventional approaches for modeling skeletons usually rely
on hand-crafted parts or traversal rules, thus resulting in limited expressive
power and difficulties of generalization. In this work, we propose a novel
model of dynamic skeletons called Spatial-Temporal Graph Convolutional Networks
(ST-GCN), which moves beyond the limitations of previous methods by
automatically learning both the spatial and temporal patterns from data. This
formulation not only leads to greater expressive power but also stronger
generalization capability. On two large datasets, Kinetics and NTU-RGBD, it
achieves substantial improvements over mainstream methods.
A Question-Focused Multi-Factor Attention Network for Question Answering
Authors: Souvik Kundu, Hwee Tou Ng
Date: 2018-01-25 00:00:00
Tasks: Open-Domain Question Answering, Question Answering, Reading Comprehension, Sentence, TriviaQA

Neural network models recently proposed for question answering (QA) primarily
focus on capturing the passage-question relation. However, they have minimal
capability to link relevant facts distributed across multiple sentences which
is crucial in achieving deeper understanding, such as performing multi-sentence
reasoning, co-reference resolution, etc. They also do not explicitly focus on
the question and answer type which often plays a critical role in QA. In this
paper, we propose a novel end-to-end question-focused multi-factor attention
network for answer extraction. Multi-factor attentive encoding using
tensor-based transformation aggregates meaningful facts even when they are
located in multiple sentences. To implicitly infer the answer type, we also
propose a max-attentional question aggregation mechanism to encode a question
vector based on the important words in a question. During prediction, we
incorporate sequence-level encoding of the first wh-word and its immediately
following word as an additional source of question type information. Our
proposed model achieves significant improvements over the best prior
state-of-the-art results on three large-scale challenging QA datasets, namely
NewsQA, TriviaQA, and SearchQA.
The landscape of the spiked tensor model
Authors: Gerard Ben Arous, Song Mei, Andrea Montanari, Mihai Nica
Date: 2017-11-15 00:00:00
Tasks: model

We consider the problem of estimating a large rank-one tensor ${\boldsymbol
u}^{\otimes k}\in({\mathbb R}^{n})^{\otimes k}$, $k\ge 3$ in Gaussian noise.
Earlier work characterized a critical signal-to-noise ratio $\lambda_{Bayes}=
O(1)$ above which an ideal estimator achieves strictly positive correlation
with the unknown vector of interest. Remarkably no polynomial-time algorithm is
known that achieved this goal unless $\lambda\ge C n^{(k-2)/4}$ and even
powerful semidefinite programming relaxations appear to fail for $1\ll
\lambda\ll n^{(k-2)/4}$.
  In order to elucidate this behavior, we consider the maximum likelihood
estimator, which requires maximizing a degree-$k$ homogeneous polynomial over
the unit sphere in $n$ dimensions. We compute the expected number of critical
points and local maxima of this objective function and show that it is
exponential in the dimensions $n$, and give exact formulas for the exponential
growth rate. We show that (for $\lambda$ larger than a constant) critical
points are either very close to the unknown vector ${\boldsymbol u}$, or are
confined in a band of width $\Theta(\lambda^{-1/(k-1)})$ around the maximum
circle that is orthogonal to ${\boldsymbol u}$. For local maxima, this band
shrinks to be of size $\Theta(\lambda^{-1/(k-2)})$. These `uninformative' local
maxima are likely to cause the failure of optimization algorithms.
Using KL-divergence to focus Deep Visual Explanation
Authors: Housam Khalifa Bashier Babiker, Randy Goebel
Date: 2017-11-17 00:00:00
Tasks: General Classification, image-classification, Image Classification, Prediction

We present a method for explaining the image classification predictions of
deep convolution neural networks, by highlighting the pixels in the image which
influence the final class prediction. Our method requires the identification of
a heuristic method to select parameters hypothesized to be most relevant in
this prediction, and here we use Kullback-Leibler divergence to provide this
focus. Overall, our approach helps in understanding and interpreting deep
network predictions and we hope contributes to a foundation for such
understanding of deep learning networks. In this brief paper, our experiments
evaluate the performance of two popular networks in this context of
interpretability.