pwc-archive/papers-with-abstracts huggingface dataset:
Multitask and Multilingual Modelling for Lexical Analysis
Authors: Johannes Bjerva
Date: 2018-09-07 00:00:00
Tasks: Lexical Analysis, Part-Of-Speech Tagging

In Natural Language Processing (NLP), one traditionally considers a single
task (e.g. part-of-speech tagging) for a single language (e.g. English) at a
time. However, recent work has shown that it can be beneficial to take
advantage of relatedness between tasks, as well as between languages. In this
work I examine the concept of relatedness and explore how it can be utilised to
build NLP models that require less manually annotated data. A large selection
of NLP tasks is investigated for a substantial language sample comprising 60
languages. The results show potential for joint multitask and multilingual
modelling, and hints at linguistic insights which can be gained from such
models.
A Primer on Causality in Data Science
Authors: Hachem Saddiki, Laura B. Balzer
Date: 2018-09-07 00:00:00
Tasks: Causal Inference

Many questions in Data Science are fundamentally causal in that our objective
is to learn the effect of some exposure, randomized or not, on an outcome
interest. Even studies that are seemingly non-causal, such as those with the
goal of prediction or prevalence estimation, have causal elements, including
differential censoring or measurement. As a result, we, as Data Scientists,
need to consider the underlying causal mechanisms that gave rise to the data,
rather than simply the pattern or association observed in those data. In this
work, we review the 'Causal Roadmap' of Petersen and van der Laan (2014) to
provide an introduction to some key concepts in causal inference. Similar to
other causal frameworks, the steps of the Roadmap include clearly stating the
scientific question, defining of the causal model, translating the scientific
question into a causal parameter, assessing the assumptions needed to express
the causal parameter as a statistical estimand, implementation of statistical
estimators including parametric and semi-parametric methods, and interpretation
of our findings. We believe that using such a framework in Data Science will
help to ensure that our statistical analyses are guided by the scientific
question driving our research, while avoiding over-interpreting our results. We
focus on the effect of an exposure occurring at a single time point and
highlight the use of targeted maximum likelihood estimation (TMLE) with Super
Learner.
Deep Recurrent Survival Analysis
Authors: Kan Ren, Jiarui Qin, Lei Zheng, Zhengyu Yang, Wei-Nan Zhang, Lin Qiu, Yong Yu
Date: 2018-09-07 00:00:00
Tasks: Survival Analysis

Survival analysis is a hotspot in statistical research for modeling
time-to-event information with data censorship handling, which has been widely
used in many applications such as clinical research, information system and
other fields with survivorship bias. Many works have been proposed for survival
analysis ranging from traditional statistic methods to machine learning models.
However, the existing methodologies either utilize counting-based statistics on
the segmented data, or have a pre-assumption on the event probability
distribution w.r.t. time. Moreover, few works consider sequential patterns
within the feature space. In this paper, we propose a Deep Recurrent Survival
Analysis model which combines deep learning for conditional probability
prediction at fine-grained level of the data, and survival analysis for
tackling the censorship. By capturing the time dependency through modeling the
conditional probability of the event for each sample, our method predicts the
likelihood of the true event occurrence and estimates the survival rate over
time, i.e., the probability of the non-occurrence of the event, for the
censored data. Meanwhile, without assuming any specific form of the event
probability distribution, our model shows great advantages over the previous
works on fitting various sophisticated data distributions. In the experiments
on the three real-world tasks from different fields, our model significantly
outperforms the state-of-the-art solutions under various metrics.
Detecting Potential Local Adversarial Examples for Human-Interpretable Defense
Authors: Xavier Renard, Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Marcin Detyniecki
Date: 2018-09-07 00:00:00
Tasks: BIG-bench Machine Learning

Machine learning models are increasingly used in the industry to make
decisions such as credit insurance approval. Some people may be tempted to
manipulate specific variables, such as the age or the salary, in order to get
better chances of approval. In this ongoing work, we propose to discuss, with a
first proposition, the issue of detecting a potential local adversarial example
on classical tabular data by providing to a human expert the locally critical
features for the classifier's decision, in order to control the provided
information and avoid a fraud.
Deep Feature Learning of Multi-Network Topology for Node Classification
Authors: Hansheng Xue, Jiajie Peng, Xuequn Shang
Date: 2018-09-07 00:00:00
Tasks: General Classification, Network Embedding, Node Classification

Networks are ubiquitous structure that describes complex relationships
between different entities in the real world. As a critical component of
prediction task over nodes in networks, learning the feature representation of
nodes has become one of the most active areas recently. Network Embedding,
aiming to learn non-linear and low-dimensional feature representation based on
network topology, has been proved to be helpful on tasks of network analysis,
especially node classification. For many real-world systems, multiple types of
relations are naturally represented by multiple networks. However, existing
network embedding methods mainly focus on single network embedding and neglect
the information shared among different networks. In this paper, we propose a
novel multiple network embedding method based on semisupervised autoencoder,
named DeepMNE, which captures complex topological structures of multi-networks
and takes the correlation among multi-networks into account. We evaluate
DeepMNE on the task of node classification with two real-world datasets. The
experimental results demonstrate the superior performance of our method over
four state-of-the-art algorithms.
Improving Neural Question Generation using Answer Separation
Authors: Yanghoon Kim, Hwanhee Lee, Joongbo Shin, Kyomin Jung
Date: 2018-09-07 00:00:00
Tasks: Question Generation, Question-Generation

Neural question generation (NQG) is the task of generating a question from a
given passage with deep neural networks. Previous NQG models suffer from a
problem that a significant proportion of the generated questions include words
in the question target, resulting in the generation of unintended questions. In
this paper, we propose answer-separated seq2seq, which better utilizes the
information from both the passage and the target answer. By replacing the
target answer in the original passage with a special token, our model learns to
identify which interrogative word should be used. We also propose a new module
termed keyword-net, which helps the model better capture the key information in
the target answer and generate an appropriate question. Experimental results
demonstrate that our answer separation method significantly reduces the number
of improper questions which include answers. Consequently, our model
significantly outperforms previous state-of-the-art NQG models.
Improving On-policy Learning with Statistical Reward Accumulation
Authors: Yubin Deng, Ke Yu, Dahua Lin, Xiaoou Tang, Chen Change Loy
Date: 2018-09-07 00:00:00
Tasks: Atari Games, Deep Reinforcement Learning, MuJoCo, Reinforcement Learning

Deep reinforcement learning has obtained significant breakthroughs in recent
years. Most methods in deep-RL achieve good results via the maximization of the
reward signal provided by the environment, typically in the form of discounted
cumulative returns. Such reward signals represent the immediate feedback of a
particular action performed by an agent. However, tasks with sparse reward
signals are still challenging to on-policy methods. In this paper, we introduce
an effective characterization of past reward statistics (which can be seen as
long-term feedback signals) to supplement this immediate reward feedback. In
particular, value functions are learned with multi-critics supervision,
enabling complex value functions to be more easily approximated in on-policy
learning, even when the reward signals are sparse. We also introduce a novel
exploration mechanism called "hot-wiring" that can give a boost to seemingly
trapped agents. We demonstrate the effectiveness of our advantage actor
multi-critic (A2MC) method across the discrete domains in Atari games as well
as continuous domains in the MuJoCo environments. A video demo is provided at
https://youtu.be/zBmpf3Yz8tc.
Mixtures of Skewed Matrix Variate Bilinear Factor Analyzers
Authors: Michael P. B. Gallaugher, Paul D. McNicholas
Date: 2018-09-07 00:00:00
Tasks: Clustering, Dimensionality Reduction

In recent years, data have become increasingly higher dimensional and, therefore, an increased need has arisen for dimension reduction techniques for clustering. Although such techniques are firmly established in the literature for multivariate data, there is a relative paucity in the area of matrix variate, or three-way, data. Furthermore, the few methods that are available all assume matrix variate normality, which is not always sensible if cluster skewness or excess kurtosis is present. Mixtures of bilinear factor analyzers using skewed matrix variate distributions are proposed. In all, four such mixture models are presented, based on matrix variate skew-t, generalized hyperbolic, variance-gamma, and normal inverse Gaussian distributions, respectively.
Group-based Learning of Disentangled Representations with Generalizability for Novel Contents
Authors: Haruo Hosoya
Date: 2018-09-07 00:00:00
Tasks: Attribute

Sensory data are often comprised of independent content and transformation factors. For example, face images may have shapes as content and poses as transformation. To infer separately these factors from given data, various ``disentangling'' models have been proposed. However, many of these are supervised or semi-supervised, either requiring attribute labels that are often unavailable or disallowing for generalization over new contents. In this study, we introduce a novel deep generative model, called group-based variational autoencoders. In this, we assume no explicit labels, but a weaker form of structure that groups together data instances having the same content but transformed differently; we thereby separately estimate a group-common factor as content and an instance-specific factor as transformation. This approach allows for learning to represent a general continuous space of contents, which can accommodate unseen contents. Despite the simplicity, our model succeeded in learning, from five datasets, content representations that are highly separate from the transformation representation and generalizable to data with novel contents. We further provide detailed analysis of the latent content code and show insight into how our model obtains the notable transformation invariance and content generalizability.
On2Vec: Embedding-based Relation Prediction for Ontology Population
Authors: Muhao Chen, Yingtao Tian, Xuelu Chen, Zijun Xue, Carlo Zaniolo
Date: 2018-09-07 00:00:00
Tasks: Graph Embedding, Knowledge Graphs, Prediction, Relation, Relation Prediction, Translation

Populating ontology graphs represents a long-standing problem for the
Semantic Web community. Recent advances in translation-based graph embedding
methods for populating instance-level knowledge graphs lead to promising new
approaching for the ontology population problem. However, unlike instance-level
graphs, the majority of relation facts in ontology graphs come with
comprehensive semantic relations, which often include the properties of
transitivity and symmetry, as well as hierarchical relations. These
comprehensive relations are often too complex for existing graph embedding
methods, and direct application of such methods is not feasible. Hence, we
propose On2Vec, a novel translation-based graph embedding method for ontology
population. On2Vec integrates two model components that effectively
characterize comprehensive relation facts in ontology graphs. The first is the
Component-specific Model that encodes concepts and relations into
low-dimensional embedding spaces without a loss of relational properties; the
second is the Hierarchy Model that performs focused learning of hierarchical
relation facts. Experiments on several well-known ontology graphs demonstrate
the promising capabilities of On2Vec in predicting and verifying new relation
facts. These promising results also make possible significant improvements in
related methods.
Monte Carlo Tree Search with Scalable Simulation Periods for Continuously Running Tasks
Authors: Seydou Ba, Takuya Hiraoka, Takashi Onishi, Toru Nakata, Yoshimasa Tsuruoka
Date: 2018-09-07 00:00:00
Tasks: Atari Games

Monte Carlo Tree Search (MCTS) is particularly adapted to domains where the
potential actions can be represented as a tree of sequential decisions. For an
effective action selection, MCTS performs many simulations to build a reliable
tree representation of the decision space. As such, a bottleneck to MCTS
appears when enough simulations cannot be performed between action selections.
This is particularly highlighted in continuously running tasks, for which the
time available to perform simulations between actions tends to be limited due
to the environment's state constantly changing. In this paper, we present an
approach that takes advantage of the anytime characteristic of MCTS to increase
the simulation time when allowed. Our approach is to effectively balance the
prospect of selecting an action with the time that can be spared to perform
MCTS simulations before the next action selection. For that, we considered the
simulation time as a decision variable to be selected alongside an action. We
extended the Hierarchical Optimistic Optimization applied to Tree (HOOT) method
to adapt our approach to environments with a continuous decision space. We
evaluated our approach for environments with a continuous decision space
through OpenAI gym's Pendulum and Continuous Mountain Car environments and for
environments with discrete action space through the arcade learning environment
(ALE) platform. The evaluation results show that, with variable simulation
times, the proposed approach outperforms the conventional MCTS in the evaluated
continuous decision space tasks and improves the performance of MCTS in most of
the ALE tasks.
Infinite Curriculum Learning for Efficiently Detecting Gastric Ulcers in WCE Images
Authors: Xiaolu Zhang, Shiwan Zhao, Lingxi Xie
Date: 2018-09-07 00:00:00
Tasks: Binary Classification

The Wireless Capsule Endoscopy (WCE) is becoming a popular way of screening
gastrointestinal system diseases and cancer. However, the time-consuming
process in inspecting WCE data limits its applications and increases the cost
of examinations. This paper considers WCE-based gastric ulcer detection, in
which the major challenge is to detect the lesions in a local region. We
propose an approach named infinite curriculum learning, which generalizes
curriculum learning to an infinite sampling space by approximately measuring
the difficulty of each patch by its scale. This allows us to adapt our model
from local patches to global images gradually, leading to a consistent accuracy
gain. Experiments are performed on a large dataset with more than 3 million WCE
images. Our approach achieves a binary classification accuracy of 87%, and is
able to detect some lesions mis-annotated by the physicians. In a real-world
application, our approach can reduce the workload of a physician by 90%-98% in
gastric ulcer screening.
A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations
Authors: Philipp Grohs, Fabian Hornung, Arnulf Jentzen, Philippe von Wurstemberger
Date: 2018-09-07 00:00:00
Tasks: image-classification, Image Classification, speech-recognition, Speech Recognition, Time Series Analysis

Artificial neural networks (ANNs) have very successfully been used in numerical simulations for a series of computational problems ranging from image classification/image recognition, speech recognition, time series analysis, game intelligence, and computational advertising to numerical approximations of partial differential equations (PDEs). Such numerical simulations suggest that ANNs have the capacity to very efficiently approximate high-dimensional functions and, especially, indicate that ANNs seem to admit the fundamental power to overcome the curse of dimensionality when approximating the high-dimensional functions appearing in the above named computational problems. There are a series of rigorous mathematical approximation results for ANNs in the scientific literature. Some of them prove convergence without convergence rates and some even rigorously establish convergence rates but there are only a few special cases where mathematical results can rigorously explain the empirical success of ANNs when approximating high-dimensional functions. The key contribution of this article is to disclose that ANNs can efficiently approximate high-dimensional functions in the case of numerical approximations of Black-Scholes PDEs. More precisely, this work reveals that the number of required parameters of an ANN to approximate the solution of the Black-Scholes PDE grows at most polynomially in both the reciprocal of the prescribed approximation accuracy $\varepsilon > 0$ and the PDE dimension $d \in \mathbb{N}$. We thereby prove, for the first time, that ANNs do indeed overcome the curse of dimensionality in the numerical approximation of Black-Scholes PDEs.
Multi-Target Prediction: A Unifying View on Problems and Methods
Authors: Willem Waegeman, Krzysztof Dembczynski, Eyke Huellermeier
Date: 2018-09-07 00:00:00
Tasks: Matrix Completion, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Multi-Task Learning, Prediction, Zero-Shot Learning

Multi-target prediction (MTP) is concerned with the simultaneous prediction
of multiple target variables of diverse type. Due to its enormous application
potential, it has developed into an active and rapidly expanding research field
that combines several subfields of machine learning, including multivariate
regression, multi-label classification, multi-task learning, dyadic prediction,
zero-shot learning, network inference, and matrix completion. In this paper, we
present a unifying view on MTP problems and methods. First, we formally discuss
commonalities and differences between existing MTP problems. To this end, we
introduce a general framework that covers the above subfields as special cases.
As a second contribution, we provide a structured overview of MTP methods. This
is accomplished by identifying a number of key properties, which distinguish
such methods and determine their suitability for different types of problems.
Finally, we also discuss a few challenges for future research.
Exploiting local and global performance of candidate systems for aggregation of summarization techniques
Authors: Parth Mehta, Prasenjit Majumder
Date: 2018-09-07 00:00:00
Tasks: Extractive Summarization

With an ever growing number of extractive summarization techniques being
proposed, there is less clarity then ever about how good each system is
compared to the rest. Several studies highlight the variance in performance of
these systems with change in datasets or even across documents within the same
corpus. An effective way to counter this variance and to make the systems more
robust could be to use inputs from multiple systems when generating a summary.
In the present work, we define a novel way of creating such ensemble by
exploiting similarity between the content of candidate summaries to estimate
their reliability. We define GlobalRank which captures the performance of a
candidate system on an overall corpus and LocalRank which estimates its
performance on a given document cluster. We then use these two scores to assign
a weight to each individual systems, which is then used to generate the new
aggregate ranking. Experiments on DUC2003 and DUC 2004 datasets show a
significant improvement in terms of ROUGE score, over existing sate-of-art
techniques.
A Fast Anderson-Chebyshev Acceleration for Nonlinear Optimization
Authors: Zhize Li, Jian Li
Date: 2018-09-07 00:00:00
Tasks: subspace methods

Anderson acceleration (or Anderson mixing) is an efficient acceleration method for fixed point iterations $x_{t+1}=G(x_t)$, e.g., gradient descent can be viewed as iteratively applying the operation $G(x) \triangleq x-\alpha\nabla f(x)$. It is known that Anderson acceleration is quite efficient in practice and can be viewed as an extension of Krylov subspace methods for nonlinear problems. In this paper, we show that Anderson acceleration with Chebyshev polynomial can achieve the optimal convergence rate $O(\sqrt{\kappa}\ln\frac{1}{\epsilon})$, which improves the previous result $O(\kappa\ln\frac{1}{\epsilon})$ provided by (Toth and Kelley, 2015) for quadratic functions. Moreover, we provide a convergence analysis for minimizing general nonlinear problems. Besides, if the hyperparameters (e.g., the Lipschitz smooth parameter $L$) are not available, we propose a guessing algorithm for guessing them dynamically and also prove a similar convergence rate. Finally, the experimental results demonstrate that the proposed Anderson-Chebyshev acceleration method converges significantly faster than other algorithms, e.g., vanilla gradient descent (GD), Nesterov's Accelerated GD. Also, these algorithms combined with the proposed guessing algorithm (guessing the hyperparameters dynamically) achieve much better performance.
Information-Theoretic Active Learning for Content-Based Image Retrieval
Authors: Björn Barz, Christoph Käding, Joachim Denzler
Date: 2018-09-07 00:00:00
Tasks: Active Learning, Binary Classification, Content-Based Image Retrieval, Diversity, Image Retrieval, Retrieval

We propose Information-Theoretic Active Learning (ITAL), a novel batch-mode
active learning method for binary classification, and apply it for acquiring
meaningful user feedback in the context of content-based image retrieval.
Instead of combining different heuristics such as uncertainty, diversity, or
density, our method is based on maximizing the mutual information between the
predicted relevance of the images and the expected user feedback regarding the
selected batch. We propose suitable approximations to this computationally
demanding problem and also integrate an explicit model of user behavior that
accounts for possible incorrect labels and unnameable instances. Furthermore,
our approach does not only take the structure of the data but also the expected
model output change caused by the user feedback into account. In contrast to
other methods, ITAL turns out to be highly flexible and provides
state-of-the-art performance across various datasets, such as MIRFLICKR and
ImageNet.