pwc-archive/papers-with-abstracts huggingface dataset:
Fast Approximate Spectral Clustering for Dynamic Networks
Authors: Lionel Martin, Andreas Loukas, Pierre Vandergheynst
Date: 2017-06-12 00:00:00
Proceeding: ICML 2018 7
Tasks: Clustering

Spectral clustering is a widely studied problem, yet its complexity is
prohibitive for dynamic graphs of even modest size. We claim that it is
possible to reuse information of past cluster assignments to expedite
computation. Our approach builds on a recent idea of sidestepping the main
bottleneck of spectral clustering, i.e., computing the graph eigenvectors, by
using fast Chebyshev graph filtering of random signals. We show that the
proposed algorithm achieves clustering assignments with quality approximating
that of spectral clustering and that it can yield significant complexity
benefits when the graph dynamics are appropriately bounded.
Enriched Deep Recurrent Visual Attention Model for Multiple Object Recognition
Authors: Artsiom Ablavatski, Shijian Lu, Jianfei Cai
Date: 2017-06-12 00:00:00
Tasks: Object Recognition

We design an Enriched Deep Recurrent Visual Attention Model (EDRAM) - an
improved attention-based architecture for multiple object recognition. The
proposed model is a fully differentiable unit that can be optimized end-to-end
by using Stochastic Gradient Descent (SGD). The Spatial Transformer (ST) was
employed as visual attention mechanism which allows to learn the geometric
transformation of objects within images. With the combination of the Spatial
Transformer and the powerful recurrent architecture, the proposed EDRAM can
localize and recognize objects simultaneously. EDRAM has been evaluated on two
publicly available datasets including MNIST Cluttered (with 70K cluttered
digits) and SVHN (with up to 250k real world images of house numbers).
Experiments show that it obtains superior performance as compared with the
state-of-the-art models.
Action and perception for spatiotemporal patterns
Authors: Martin Biehl, Daniel Polani
Date: 2017-06-12 00:00:00

This is a contribution to the formalization of the concept of agents in
multivariate Markov chains. Agents are commonly defined as entities that act,
perceive, and are goal-directed. In a multivariate Markov chain (e.g. a
cellular automaton) the transition matrix completely determines the dynamics.
This seems to contradict the possibility of acting entities within such a
system. Here we present definitions of actions and perceptions within
multivariate Markov chains based on entity-sets. Entity-sets represent a
largely independent choice of a set of spatiotemporal patterns that are
considered as all the entities within the Markov chain. For example, the
entity-set can be chosen according to operational closure conditions or
complete specific integration. Importantly, the perception-action loop also
induces an entity-set and is a multivariate Markov chain. We then show that our
definition of actions leads to non-heteronomy and that of perceptions
specialize to the usual concept of perception in the perception-action loop.
Learning to Learn without Gradient Descent by Gradient Descent
Authors: Yutian Chen, Matthew W. Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Timothy P. Lillicrap, Matt Botvinick, Nando de Freitas
Date: 2016-11-11 00:00:00
Proceeding: ICML 2017 8
Tasks: Bayesian Optimization, global-optimization

We learn recurrent neural network optimizers trained on simple synthetic
functions by gradient descent. We show that these learned optimizers exhibit a
remarkable degree of transfer in that they can be used to efficiently optimize
a broad range of derivative-free black-box functions, including Gaussian
process bandits, simple control objectives, global optimization benchmarks and
hyper-parameter tuning tasks. Up to the training horizon, the learned
optimizers learn to trade-off exploration and exploitation, and compare
favourably with heavily engineered Bayesian optimization packages for
hyper-parameter tuning.
Exploring the Syntactic Abilities of RNNs with Multi-task Learning
Authors: Emile Enguehard, Yoav Goldberg, Tal Linzen
Date: 2017-06-12 00:00:00
Proceeding: CONLL 2017 8
Tasks: CCG Supertagging, Language Modeling, Language Modelling, Multi-Task Learning, Sentence

Recent work has explored the syntactic abilities of RNNs using the
subject-verb agreement task, which diagnoses sensitivity to sentence structure.
RNNs performed this task well in common cases, but faltered in complex
sentences (Linzen et al., 2016). We test whether these errors are due to
inherent limitations of the architecture or to the relatively indirect
supervision provided by most agreement dependencies in a corpus. We trained a
single RNN to perform both the agreement task and an additional task, either
CCG supertagging or language modeling. Multi-task training led to significantly
lower error rates, in particular on complex sentences, suggesting that RNNs
have the ability to evolve more sophisticated syntactic representations than
shown before. We also show that easily available agreement training data can
improve performance on other syntactic tasks, in particular when only a limited
amount of training data is available for those tasks. The multi-task paradigm
can also be leveraged to inject grammatical knowledge into language models.
Recursive Multikernel Filters Exploiting Nonlinear Temporal Structure
Authors: Steven Van Vaerenbergh, Simone Scardapane, Ignacio Santamaria
Date: 2017-06-12 00:00:00

In kernel methods, temporal information on the data is commonly included by
using time-delayed embeddings as inputs. Recently, an alternative formulation
was proposed by defining a gamma-filter explicitly in a reproducing kernel
Hilbert space, giving rise to a complex model where multiple kernels operate on
different temporal combinations of the input signal. In the original
formulation, the kernels are then simply combined to obtain a single kernel
matrix (for instance by averaging), which provides computational benefits but
discards important information on the temporal structure of the signal.
Inspired by works on multiple kernel learning, we overcome this drawback by
considering the different kernels separately. We propose an efficient strategy
to adaptively combine and select these kernels during the training phase. The
resulting batch and online algorithms automatically learn to process highly
nonlinear temporal information extracted from the input signal, which is
implicitly encoded in the kernel values. We evaluate our proposal on several
artificial and real tasks, showing that it can outperform classical approaches
both in batch and online settings.
Candidate sentence selection for language learning exercises: from a comprehensive framework to an empirical evaluation
Authors: Ildikó Pilán, Elena Volodina, Lars Borin
Date: 2017-06-12 00:00:00
Tasks: Sentence

We present a framework and its implementation relying on Natural Language
Processing methods, which aims at the identification of exercise item
candidates from corpora. The hybrid system combining heuristics and machine
learning methods includes a number of relevant selection criteria. We focus on
two fundamental aspects: linguistic complexity and the dependence of the
extracted sentences on their original context. Previous work on exercise
generation addressed these two criteria only to a limited extent, and a refined
overall candidate sentence selection framework appears also to be lacking. In
addition to a detailed description of the system, we present the results of an
empirical evaluation conducted with language teachers and learners which
indicate the usefulness of the system for educational purposes. We have
integrated our system into a freely available online learning platform.
Exploring the similarity of medical imaging classification problems
Authors: Veronika Cheplygina, Pim Moeskops, Mitko Veta, Behdad Dasht Bozorg, Josien Pluim
Date: 2017-06-12 00:00:00
Tasks: Classification, General Classification, Medical Image Analysis, Meta-Learning

Supervised learning is ubiquitous in medical image analysis. In this paper we
consider the problem of meta-learning -- predicting which methods will perform
well in an unseen classification problem, given previous experience with other
classification problems. We investigate the first step of such an approach: how
to quantify the similarity of different classification problems. We
characterize datasets sampled from six classification problems by performance
ranks of simple classifiers, and define the similarity by the inverse of
Euclidean distance in this meta-feature space. We visualize the similarities in
a 2D space, where meaningful clusters start to emerge, and show that the
proposed representation can be used to classify datasets according to their
origin with 89.3\% accuracy. These findings, together with the observations of
recent trends in machine learning, suggest that meta-learning could be a
valuable tool for the medical imaging community.
SU-RUG at the CoNLL-SIGMORPHON 2017 shared task: Morphological Inflection with Attentional Sequence-to-Sequence Models
Authors: Robert Östling, Johannes Bjerva
Date: 2017-06-12 00:00:00
Proceeding: CONLL 2017 8
Tasks: Lemmatization, Morphological Analysis, Morphological Inflection

This paper describes the Stockholm University/University of Groningen
(SU-RUG) system for the SIGMORPHON 2017 shared task on morphological
inflection. Our system is based on an attentional sequence-to-sequence neural
network model using Long Short-Term Memory (LSTM) cells, with joint training of
morphological inflection and the inverse transformation, i.e. lemmatization and
morphological analysis. Our system outperforms the baseline with a large
margin, and our submission ranks as the 4th best team for the track we
participate in (task 1, high-resource).
A filter based approach for inbetweening
Authors: Yuichi Yagi
Date: 2017-06-12 00:00:00

We present a filter based approach for inbetweening. We train a convolutional
neural network to generate intermediate frames. This network aim to generate
smooth animation of line drawings. Our method can process scanned images
directly. Our method does not need to compute correspondence of lines and
topological changes explicitly. We experiment our method with real animation
production data. The results show that our method can generate intermediate
frames partially.
Random Forests, Decision Trees, and Categorical Predictors: The "Absent Levels" Problem
Authors: Timothy C. Au
Date: 2017-06-12 00:00:00
Tasks: Feature Engineering

One advantage of decision tree based methods like random forests is their
ability to natively handle categorical predictors without having to first
transform them (e.g., by using feature engineering techniques). However, in
this paper, we show how this capability can lead to an inherent "absent levels"
problem for decision tree based methods that has never been thoroughly
discussed, and whose consequences have never been carefully explored. This
problem occurs whenever there is an indeterminacy over how to handle an
observation that has reached a categorical split which was determined when the
observation in question's level was absent during training. Although these
incidents may appear to be innocuous, by using Leo Breiman and Adele Cutler's
random forests FORTRAN code and the randomForest R package (Liaw and Wiener,
2002) as motivating case studies, we examine how overlooking the absent levels
problem can systematically bias a model. Furthermore, by using three real data
examples, we illustrate how absent levels can dramatically alter a model's
performance in practice, and we empirically demonstrate how some simple
heuristics can be used to help mitigate the effects of the absent levels
problem until a more robust theoretical solution is found.
Kernel method for persistence diagrams via kernel embedding and weight factor
Authors: Genki Kusano, Kenji Fukumizu, Yasuaki Hiraoka
Date: 2017-06-12 00:00:00
Tasks: Graph Classification, Topological Data Analysis

Topological data analysis is an emerging mathematical concept for
characterizing shapes in multi-scale data. In this field, persistence diagrams
are widely used as a descriptor of the input data, and can distinguish robust
and noisy topological properties. Nowadays, it is highly desired to develop a
statistical framework on persistence diagrams to deal with practical data. This
paper proposes a kernel method on persistence diagrams. A theoretical
contribution of our method is that the proposed kernel allows one to control
the effect of persistence, and, if necessary, noisy topological properties can
be discounted in data analysis. Furthermore, the method provides a fast
approximation technique. The method is applied into several problems including
practical data in physics, and the results show the advantage compared to the
existing kernel method on persistence diagrams.
Evolutionary Multitasking for Single-objective Continuous Optimization: Benchmark Problems, Performance Metric, and Baseline Results
Authors: Bingshui Da, Yew-Soon Ong, Liang Feng, A. K. Qin, Abhishek Gupta, Zexuan Zhu, Chuan-Kang Ting, Ke Tang, Xin Yao
Date: 2017-06-12 00:00:00

In this report, we suggest nine test problems for multi-task single-objective
optimization (MTSOO), each of which consists of two single-objective
optimization tasks that need to be solved simultaneously. The relationship
between tasks varies between different test problems, which would be helpful to
have a comprehensive evaluation of the MFO algorithms. It is expected that the
proposed test problems will germinate progress the field of the MTSOO research.
Data-Efficient Policy Evaluation Through Behavior Policy Search
Authors: Josiah P. Hanna, Philip S. Thomas, Peter Stone, Scott Niekum
Date: 2017-06-12 00:00:00
Proceeding: ICML 2017 8

We consider the task of evaluating a policy for a Markov decision process
(MDP). The standard unbiased technique for evaluating a policy is to deploy the
policy and observe its performance. We show that the data collected from
deploying a different policy, commonly called the behavior policy, can be used
to produce unbiased estimates with lower mean squared error than this standard
technique. We derive an analytic expression for the optimal behavior policy ---
the behavior policy that minimizes the mean squared error of the resulting
estimates. Because this expression depends on terms that are unknown in
practice, we propose a novel policy evaluation sub-problem, behavior policy
search: searching for a behavior policy that reduces mean squared error. We
present a behavior policy search algorithm and empirically demonstrate its
effectiveness in lowering the mean squared error of policy performance
estimates.
Extract with Order for Coherent Multi-Document Summarization
Authors: Mir Tafseer Nayeem, Yllias Chali
Date: 2017-06-12 00:00:00
Proceeding: WS 2017 8
Tasks: Document Summarization, document understanding, Multi-Document Summarization, Sentence

In this work, we aim at developing an extractive summarizer in the multi-document setting. We implement a rank based sentence selection using continuous vector representations along with key-phrases. Furthermore, we propose a model to tackle summary coherence for increasing readability. We conduct experiments on the Document Understanding Conference (DUC) 2004 datasets using ROUGE toolkit. Our experiments demonstrate that the methods bring significant improvements over the state of the art methods in terms of informativity and coherence.
Factor Graphs for Quantum Probabilities
Authors: Hans-Andrea Loeliger, Pascal O. Vontobel
Date: 2015-08-04 00:00:00

A factor-graph representation of quantum-mechanical probabilities (involving
any number of measurements) is proposed. Unlike standard statistical models,
the proposed representation uses auxiliary variables (state variables) that are
not random variables. All joint probability distributions are marginals of some
complex-valued function $q$, and it is demonstrated how the basic concepts of
quantum mechanics relate to factorizations and marginals of $q$.
Scientific document summarization via citation contextualization and scientific discourse
Authors: Arman Cohan, Nazli Goharian
Date: 2017-06-12 00:00:00
Tasks: Document Summarization, Scientific Document Summarization, Word Embeddings

The rapid growth of scientific literature has made it difficult for the
researchers to quickly learn about the developments in their respective fields.
Scientific document summarization addresses this challenge by providing
summaries of the important contributions of scientific papers. We present a
framework for scientific summarization which takes advantage of the citations
and the scientific discourse structure. Citation texts often lack the evidence
and context to support the content of the cited paper and are even sometimes
inaccurate. We first address the problem of inaccuracy of the citation texts by
finding the relevant context from the cited paper. We propose three approaches
for contextualizing citations which are based on query reformulation, word
embeddings, and supervised learning. We then train a model to identify the
discourse facets for each citation. We finally propose a method for summarizing
scientific papers by leveraging the faceted citations and their corresponding
contexts. We evaluate our proposed method on two scientific summarization
datasets in the biomedical and computational linguistics domains. Extensive
evaluation results show that our methods can improve over the state of the art
by large margins.
Dialog Structure Through the Lens of Gender, Gender Environment, and Power
Authors: Vinodkumar Prabhakaran, Owen Rambow
Date: 2017-06-12 00:00:00

Understanding how the social context of an interaction affects our dialog
behavior is of great interest to social scientists who study human behavior, as
well as to computer scientists who build automatic methods to infer those
social contexts. In this paper, we study the interaction of power, gender, and
dialog behavior in organizational interactions. In order to perform this study,
we first construct the Gender Identified Enron Corpus of emails, in which we
semi-automatically assign the gender of around 23,000 individuals who authored
around 97,000 email messages in the Enron corpus. This corpus, which is made
freely available, is orders of magnitude larger than previously existing gender
identified corpora in the email domain. Next, we use this corpus to perform a
large-scale data-oriented study of the interplay of gender and manifestations
of power. We argue that, in addition to one's own gender, the "gender
environment" of an interaction, i.e., the gender makeup of one's interlocutors,
also affects the way power is manifested in dialog. We focus especially on
manifestations of power in the dialog structure --- both, in a shallow sense
that disregards the textual content of messages (e.g., how often do the
participants contribute, how often do they get replies etc.), as well as the
structure that is expressed within the textual content (e.g., who issues
requests and how are they made, whose requests get responses etc.). We find
that both gender and gender environment affect the ways power is manifested in
dialog, resulting in patterns that reveal the underlying factors. Finally, we
show the utility of gender information in the problem of automatically
predicting the direction of power between pairs of participants in email
interactions.
Learning to Embed Words in Context for Syntactic Tasks
Authors: Lifu Tu, Kevin Gimpel, Karen Livescu
Date: 2017-06-09 00:00:00
Proceeding: WS 2017 8

We present models for embedding words in the context of surrounding words.
Such models, which we refer to as token embeddings, represent the
characteristics of a word that are specific to a given context, such as word
sense, syntactic category, and semantic role. We explore simple, efficient
token embedding models based on standard neural network architectures. We learn
token embeddings on a large amount of unannotated text and evaluate them as
features for part-of-speech taggers and dependency parsers trained on much
smaller amounts of annotated data. We find that predictors endowed with token
embeddings consistently outperform baseline predictors across a range of
context window and training set sizes.
Modeling Multi-Object Configurations via Medial/Skeletal Linking Structures
Authors: James Damon, Ellen Gasparovic
Date: 2017-06-12 00:00:00
Tasks: Object

We introduce a method for modeling a configuration of objects in 2D or 3D
images using a mathematical "skeletal linking structure" which will
simultaneously capture the individual shape features of the objects and their
positional information relative to one another. The objects may either have
smooth boundaries and be disjoint from the others or share common portions of
their boundaries with other objects in a piecewise smooth manner. These
structures include a special class of "Blum medial linking structures," which
are intrinsically associated to the configuration and build upon the Blum
medial axes of the individual objects. We give a classification of the
properties of Blum linking structures for generic configurations. The skeletal
linking structures add increased flexibility for modeling configurations of
objects by relaxing the Blum conditions and they extend in a minimal way the
individual "skeletal structures" which have been previously used for modeling
individual objects and capturing their geometric properties. This allows for
the mathematical methods introduced for single objects to be significantly
extended to the entire configuration of objects. These methods not only capture
the internal shape structures of the individual objects but also the external
structure of the neighboring regions of the objects.
Self-Committee Approach for Image Restoration Problems using Convolutional Neural Network
Authors: Byeongyong Ahn, Nam Ik Cho
Date: 2017-05-12 00:00:00
Tasks: Denoising, Image Denoising, Image Restoration

There have been many discriminative learning methods using convolutional
neural networks (CNN) for several image restoration problems, which learn the
mapping function from a degraded input to the clean output. In this letter, we
propose a self-committee method that can find enhanced restoration results from
the multiple trial of a trained CNN with different but related inputs.
Specifically, it is noted that the CNN sometimes finds different mapping
functions when the input is transformed by a reversible transform and thus
produces different but related outputs with the original. Hence averaging the
outputs for several different transformed inputs can enhance the results as
evidenced by the network committee methods. Unlike the conventional committee
approaches that require several networks, the proposed method needs only a
single network. Experimental results show that adding an additional transform
as a committee always brings additional gain on image denoising and single
image supre-resolution problems.
Inductive Conformal Martingales for Change-Point Detection
Authors: Denis Volkhonskiy, Ilia Nouretdinov, Alexander Gammerman, Vladimir Vovk, Evgeny Burnaev
Date: 2017-06-11 00:00:00
Tasks: Change Point Detection

We consider the problem of quickest change-point detection in data streams.
Classical change-point detection procedures, such as CUSUM, Shiryaev-Roberts
and Posterior Probability statistics, are optimal only if the change-point
model is known, which is an unrealistic assumption in typical applied problems.
Instead we propose a new method for change-point detection based on Inductive
Conformal Martingales, which requires only the independence and identical
distribution of observations. We compare the proposed approach to standard
methods, as well as to change-point detection oracles, which model a typical
practical situation when we have only imprecise (albeit parametric) information
about pre- and post-change data distributions. Results of comparison provide
evidence that change-point detection based on Inductive Conformal Martingales
is an efficient tool, capable to work under quite general conditions unlike
traditional approaches.