pwc-archive/papers-with-abstracts huggingface dataset:
Implicit Discourse Relation Classification via Multi-Task Neural Networks
Authors: Yang Liu, Sujian Li, Xiaodong Zhang, Zhifang Sui
Date: 2016-03-09 00:00:00
Tasks: Classification, General Classification, Implicit Discourse Relation Classification, Multi-Task Learning, Relation, Relation Classification

Without discourse connectives, classifying implicit discourse relations is a
challenging task and a bottleneck for building a practical discourse parser.
Previous research usually makes use of one kind of discourse framework such as
PDTB or RST to improve the classification performance on discourse relations.
Actually, under different discourse annotation frameworks, there exist multiple
corpora which have internal connections. To exploit the combination of
different discourse corpora, we design related discourse classification tasks
specific to a corpus, and propose a novel Convolutional Neural Network embedded
multi-task learning system to synthesize these tasks by learning both unique
and shared representations for each task. The experimental results on the PDTB
implicit discourse relation classification task demonstrate that our model
achieves significant gains over baseline systems.
megaman: Manifold Learning with Millions of points
Authors: James McQueen, Marina Meila, Jacob VanderPlas, Zhongyue Zhang
Date: 2016-03-09 00:00:00

Manifold Learning is a class of algorithms seeking a low-dimensional
non-linear representation of high-dimensional data. Thus manifold learning
algorithms are, at least in theory, most applicable to high-dimensional data
and sample sizes to enable accurate estimation of the manifold. Despite this,
most existing manifold learning implementations are not particularly scalable.
Here we present a Python package that implements a variety of manifold learning
algorithms in a modular and scalable fashion, using fast approximate neighbors
searches and fast sparse eigendecompositions. The package incorporates
theoretical advances in manifold learning, such as the unbiased Laplacian
estimator and the estimation of the embedding distortion by the Riemannian
metric method. In benchmarks, even on a single-core desktop computer, our code
embeds millions of data points in minutes, and takes just 200 minutes to embed
the main sample of galaxy spectra from the Sloan Digital Sky Survey ---
consisting of 0.6 million samples in 3750-dimensions --- a task which has not
previously been possible.
Computing AIC for black-box models using Generalised Degrees of Freedom: a comparison with cross-validation
Authors: Severin Hauenstein, Carsten F. Dormann, Simon N. Wood
Date: 2016-03-09 00:00:00
Tasks: regression

Generalised Degrees of Freedom (GDF), as defined by Ye (1998 JASA
93:120-131), represent the sensitivity of model fits to perturbations of the
data. As such they can be computed for any statistical model, making it
possible, in principle, to derive the number of parameters in machine-learning
approaches. Defined originally for normally distributed data only, we here
investigate the potential of this approach for Bernoulli-data. GDF-values for
models of simulated and real data are compared to model complexity-estimates
from cross-validation. Similarly, we computed GDF-based AICc for randomForest,
neural networks and boosted regression trees and demonstrated its similarity to
cross-validation. GDF-estimates for binary data were unstable and
inconsistently sensitive to the number of data points perturbed simultaneously,
while at the same time being extremely computer-intensive in their calculation.
Repeated 10-fold cross-validation was more robust, based on fewer assumptions
and faster to compute. Our findings suggest that the GDF-approach does not
readily transfer to Bernoulli data and a wider range of regression approaches.
A Kernel Test for Three-Variable Interactions with Random Processes
Authors: Paul K. Rubenstein, Kacper P. Chwialkowski, Arthur Gretton
Date: 2016-03-02 00:00:00

We apply a wild bootstrap method to the Lancaster three-variable interaction
measure in order to detect factorisation of the joint distribution on three
variables forming a stationary random process, for which the existing
permutation bootstrap method fails. As in the i.i.d. case, the Lancaster test
is found to outperform existing tests in cases for which two independent
variables individually have a weak influence on a third, but that when
considered jointly the influence is strong. The main contributions of this
paper are twofold: first, we prove that the Lancaster statistic satisfies the
conditions required to estimate the quantiles of the null distribution using
the wild bootstrap; second, the manner in which this is proved is novel,
simpler than existing methods, and can further be applied to other statistics.
Learning to Blend Computer Game Levels
Authors: Matthew Guzdial, Mark Riedl
Date: 2016-03-08 00:00:00

We present an approach to generate novel computer game levels that blend
different game concepts in an unsupervised fashion. Our primary contribution is
an analogical reasoning process to construct blends between level design models
learned from gameplay videos. The models represent probabilistic relationships
between elements in the game. An analogical reasoning process maps features
between two models to produce blended models that can then generate new level
chunks. As a proof-of-concept we train our system on the classic platformer
game Super Mario Bros. due to its highly-regarded and well understood level
design. We evaluate the extent to which the models represent stylistic level
design knowledge and demonstrate the ability of our system to explain levels
that were blended by human expert designers.
Discriminative models for robust image classification
Authors: Umamahesh Srinivas
Date: 2016-03-08 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

A variety of real-world tasks involve the classification of images into
pre-determined categories. Designing image classification algorithms that
exhibit robustness to acquisition noise and image distortions, particularly
when the available training data are insufficient to learn accurate models, is
a significant challenge. This dissertation explores the development of
discriminative models for robust image classification that exploit underlying
signal structure, via probabilistic graphical models and sparse signal
representations.
  Probabilistic graphical models are widely used in many applications to
approximate high-dimensional data in a reduced complexity set-up. Learning
graphical structures to approximate probability distributions is an area of
active research. Recent work has focused on learning graphs in a discriminative
manner with the goal of minimizing classification error. In the first part of
the dissertation, we develop a discriminative learning framework that exploits
the complementary yet correlated information offered by multiple
representations (or projections) of a given signal/image. Specifically, we
propose a discriminative tree-based scheme for feature fusion by explicitly
learning the conditional correlations among such multiple projections in an
iterative manner. Experiments reveal the robustness of the resulting graphical
model classifier to training insufficiency.
Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks
Authors: Peter Ondruska, Ingmar Posner
Date: 2016-02-02 00:00:00
Tasks: Feature Engineering, Form, Object, Object Tracking

This paper presents to the best of our knowledge the first end-to-end object
tracking approach which directly maps from raw sensor input to object tracks in
sensor space without requiring any feature engineering or system identification
in the form of plant or sensor models. Specifically, our system accepts a
stream of raw sensor data at one end and, in real-time, produces an estimate of
the entire environment state at the output including even occluded objects. We
achieve this by framing the problem as a deep learning task and exploit
sequence models in the form of recurrent neural networks to learn a mapping
from sensor measurements to object tracks. In particular, we propose a learning
method based on a form of input dropout which allows learning in an
unsupervised manner, only based on raw, occluded sensor data without access to
ground-truth annotations. We demonstrate our approach using a synthetic dataset
designed to mimic the task of tracking objects in 2D laser data -- as commonly
encountered in robotics applications -- and show that it learns to track many
dynamic objects despite occlusions and the presence of sensor noise.
Super Mario as a String: Platformer Level Generation Via LSTMs
Authors: Adam Summerville, Michael Mateas
Date: 2016-03-02 00:00:00

The procedural generation of video game levels has existed for at least 30
years, but only recently have machine learning approaches been used to generate
levels without specifying the rules for generation. A number of these have
looked at platformer levels as a sequence of characters and performed
generation using Markov chains. In this paper we examine the use of Long
Short-Term Memory recurrent neural networks (LSTMs) for the purpose of
generating levels trained from a corpus of Super Mario Brothers levels. We
analyze a number of different data representations and how the generated levels
fit into the space of human authored Super Mario Brothers levels.
Parsing Linear Context-Free Rewriting Systems with Fast Matrix Multiplication
Authors: Shay B. Cohen, Daniel Gildea
Date: 2015-04-30 00:00:00
Proceeding: CL 2016 9

We describe a matrix multiplication recognition algorithm for a subset of
binary linear context-free rewriting systems (LCFRS) with running time
$O(n^{\omega d})$ where $M(m) = O(m^{\omega})$ is the running time for $m
\times m$ matrix multiplication and $d$ is the "contact rank" of the LCFRS --
the maximal number of combination and non-combination points that appear in the
grammar rules. We also show that this algorithm can be used as a subroutine to
get a recognition algorithm for general binary LCFRS with running time
$O(n^{\omega d + 1})$. The currently best known $\omega$ is smaller than
$2.38$. Our result provides another proof for the best known result for parsing
mildly context sensitive formalisms such as combinatory categorial grammars,
head grammars, linear indexed grammars, and tree adjoining grammars, which can
be parsed in time $O(n^{4.76})$. It also shows that inversion transduction
grammars can be parsed in time $O(n^{5.76})$. In addition, binary LCFRS
subsumes many other formalisms and types of grammars, for some of which we also
improve the asymptotic complexity of parsing.
A regularization-based approach for unsupervised image segmentation
Authors: Aleksandar Dimitriev, Matej Kristan
Date: 2016-03-08 00:00:00
Tasks: Image Segmentation, Semantic Segmentation, Superpixels, Unsupervised Image Segmentation

We propose a novel unsupervised image segmentation algorithm, which aims to
segment an image into several coherent parts. It requires no user input, no
supervised learning phase and assumes an unknown number of segments. It
achieves this by first over-segmenting the image into several hundred
superpixels. These are iteratively joined on the basis of a discriminative
classifier trained on color and texture information obtained from each
superpixel. The output of the classifier is regularized by a Markov random
field that lends more influence to neighbouring superpixels that are more
similar. In each iteration, similar superpixels fall under the same label,
until only a few coherent regions remain in the image. The algorithm was tested
on a standard evaluation data set, where it performs on par with
state-of-the-art algorithms in term of precision and greatly outperforms the
state of the art by reducing the oversegmentation of the object of interest.
Small ensembles of kriging models for optimization
Authors: Hossein Mohammadi, Rodolphe Le Riche, Eric Touboul
Date: 2016-03-08 00:00:00
Tasks: global-optimization

The Efficient Global Optimization (EGO) algorithm uses a conditional
Gaus-sian Process (GP) to approximate an objective function known at a finite
number of observation points and sequentially adds new points which maximize
the Expected Improvement criterion according to the GP. The important factor
that controls the efficiency of EGO is the GP covariance function (or kernel)
which should be chosen according to the objective function. Traditionally, a
pa-rameterized family of covariance functions is considered whose parameters
are learned through statistical procedures such as maximum likelihood or
cross-validation. However, it may be questioned whether statistical procedures
for learning covariance functions are the most efficient for optimization as
they target a global agreement between the GP and the observations which is not
the ultimate goal of optimization. Furthermore, statistical learning procedures
are computationally expensive. The main alternative to the statistical learning
of the GP is self-adaptation, where the algorithm tunes the kernel parameters
based on their contribution to objective function improvement. After
questioning the possibility of self-adaptation for kriging based optimizers,
this paper proposes a novel approach for tuning the length-scale of the GP in
EGO: At each iteration, a small ensemble of kriging models structured by their
length-scales is created. All of the models contribute to an iterate in an
EGO-like fashion. Then, the set of models is densified around the model whose
length-scale yielded the best iterate and further points are produced.
Numerical experiments are provided which motivate the use of many
length-scales. The tested implementation does not perform better than the
classical EGO algorithm in a sequential context but show the potential of the
approach for parallel implementations.
A Survey on Domain-Specific Languages for Machine Learning in Big Data
Authors: Ivens Portugal, Paulo Alencar, Donald Cowan
Date: 2016-02-24 00:00:00
Tasks: BIG-bench Machine Learning

The amount of data generated in the modern society is increasing rapidly. New
problems and novel approaches of data capture, storage, analysis and
visualization are responsible for the emergence of the Big Data research field.
Machine Learning algorithms can be used in Big Data to make better and more
accurate inferences. However, because of the challenges Big Data imposes, these
algorithms need to be adapted and optimized to specific applications. One
important decision made by software engineers is the choice of the language
that is used in the implementation of these algorithms. Therefore, this
literature survey identifies and describes domain-specific languages and
frameworks used for Machine Learning in Big Data. By doing this, software
engineers can then make more informed choices and beginners have an overview of
the main languages used in this domain.
Observing Trends in Automated Multilingual Media Analysis
Authors: Ralf Steinberger, Aldo Podavini, Alexandra Balahur, Guillaume Jacquet, Hristo Tanev, Jens Linge, Martin Atkinson, Michele Chinosi, Vanni Zavarella, Yaniv Steiner, Erik van der Goot
Date: 2016-03-08 00:00:00
Tasks: Articles

Any large organisation, be it public or private, monitors the media for
information to keep abreast of developments in their field of interest, and
usually also to become aware of positive or negative opinions expressed towards
them. At least for the written media, computer programs have become very
efficient at helping the human analysts significantly in their monitoring task
by gathering media reports, analysing them, detecting trends and - in some
cases - even to issue early warnings or to make predictions of likely future
developments. We present here trend recognition-related functionality of the
Europe Media Monitor (EMM) system, which was developed by the European
Commission's Joint Research Centre (JRC) for public administrations in the
European Union (EU) and beyond. EMM performs large-scale media analysis in up
to seventy languages and recognises various types of trends, some of them
combining information from news articles written in different languages and
from social media posts. EMM also lets users explore the huge amount of
multilingual media data through interactive maps and graphs, allowing them to
examine the data from various view points and according to multiple criteria. A
lot of EMM's functionality is accessibly freely over the internet or via apps
for hand-held devices.
Prediction of Infinite Words with Automata
Authors: Tim Smith
Date: 2016-03-08 00:00:00
Tasks: Prediction

In the classic problem of sequence prediction, a predictor receives a
sequence of values from an emitter and tries to guess the next value before it
appears. The predictor masters the emitter if there is a point after which all
of the predictor's guesses are correct. In this paper we consider the case in
which the predictor is an automaton and the emitted values are drawn from a
finite set; i.e., the emitted sequence is an infinite word. We examine the
predictive capabilities of finite automata, pushdown automata, stack automata
(a generalization of pushdown automata), and multihead finite automata. We
relate our predicting automata to purely periodic words, ultimately periodic
words, and multilinear words, describing novel prediction algorithms for
mastering these sequences.
Batched Lazy Decision Trees
Authors: Mathieu Guillame-Bert, Artur Dubrawski
Date: 2016-03-08 00:00:00
Tasks: General Classification

We introduce a batched lazy algorithm for supervised classification using
decision trees. It avoids unnecessary visits to irrelevant nodes when it is
used to make predictions with either eagerly or lazily trained decision trees.
A set of experiments demonstrate that the proposed algorithm can outperform
both the conventional and lazy decision tree algorithms in terms of computation
time as well as memory consumption, without compromising accuracy.
Network Morphism
Authors: Tao Wei, Changhu Wang, Yong Rui, Chang Wen Chen
Date: 2016-03-05 00:00:00
Tasks: MORPH

We present in this paper a systematic study on how to morph a well-trained
neural network to a new one so that its network function can be completely
preserved. We define this as \emph{network morphism} in this research. After
morphing a parent network, the child network is expected to inherit the
knowledge from its parent network and also has the potential to continue
growing into a more powerful one with much shortened training time. The first
requirement for this network morphism is its ability to handle diverse morphing
types of networks, including changes of depth, width, kernel size, and even
subnet. To meet this requirement, we first introduce the network morphism
equations, and then develop novel morphing algorithms for all these morphing
types for both classic and convolutional neural networks. The second
requirement for this network morphism is its ability to deal with non-linearity
in a network. We propose a family of parametric-activation functions to
facilitate the morphing of any continuous non-linear activation neurons.
Experimental results on benchmark datasets and typical neural networks
demonstrate the effectiveness of the proposed network morphism scheme.
On the inconsistency of $\ell_1$-penalised sparse precision matrix estimation
Authors: Otte Heinävaara, Janne Leppä-aho, Jukka Corander, Antti Honkela
Date: 2016-03-08 00:00:00

Various $\ell_1$-penalised estimation methods such as graphical lasso and
CLIME are widely used for sparse precision matrix estimation. Many of these
methods have been shown to be consistent under various quantitative assumptions
about the underlying true covariance matrix. Intuitively, these conditions are
related to situations where the penalty term will dominate the optimisation. In
this paper, we explore the consistency of $\ell_1$-based methods for a class of
sparse latent variable -like models, which are strongly motivated by several
types of applications. We show that all $\ell_1$-based methods fail
dramatically for models with nearly linear dependencies between the variables.
We also study the consistency on models derived from real gene expression data
and note that the assumptions needed for consistency never hold even for modest
sized gene networks and $\ell_1$-based methods also become unreliable in
practice for larger networks.
A Bayesian non-parametric method for clustering high-dimensional binary data
Authors: Tapesh Santra
Date: 2016-03-08 00:00:00
Tasks: Clustering, Specificity, Vocal Bursts Intensity Prediction

In many real life problems, objects are described by large number of binary
features. For instance, documents are characterized by presence or absence of
certain keywords; cancer patients are characterized by presence or absence of
certain mutations etc. In such cases, grouping together similar
objects/profiles based on such high dimensional binary features is desirable,
but challenging. Here, I present a Bayesian non parametric algorithm for
clustering high dimensional binary data. It uses a Dirichlet Process (DP)
mixture model and simulated annealing to not only cluster binary data, but also
find optimal number of clusters in the data. The performance of the algorithm
was evaluated and compared with other algorithms using simulated datasets. It
outperformed all other clustering methods that were tested in the simulation
studies. It was also used to cluster real datasets arising from document
analysis, handwritten image analysis and cancer research. It successfully
divided a set of documents based on their topics, hand written images based on
different styles of writing digits and identified tissue and mutation
specificity of chemotherapy treatments.
Extracting Arabic Relations from the Web
Authors: Shimaa M. Abd El-salam, Enas M. F. El Houby, A. K. Al Sammak, T. A. El-Shishtawy
Date: 2016-03-08 00:00:00

The goal of this research is to extract a large list or table from named
entities and relations in a specific domain. A small set of a handful of
instance relations is required as input from the user. The system exploits
summaries from Google search engine as a source text. These instances are used
to extract patterns. The output is a set of new entities and their relations.
The results from four experiments show that precision and recall varies
according to relation type. Precision ranges from 0.61 to 0.75 while recall
ranges from 0.71 to 0.83. The best result is obtained for (player, club)
relationship, 0.72 and 0.83 for precision and recall respectively.
A non-extensive entropy feature and its application to texture classification
Authors: Seba Susan, Madasu Hanmandlu
Date: 2016-03-08 00:00:00
Tasks: General Classification, Texture Classification

This paper proposes a new probabilistic non-extensive entropy feature for
texture characterization, based on a Gaussian information measure. The
highlights of the new entropy are that it is bounded by finite limits and that
it is non additive in nature. The non additive property of the proposed entropy
makes it useful for the representation of information content in the
non-extensive systems containing some degree of regularity or correlation. The
effectiveness of the proposed entropy in representing the correlated random
variables is demonstrated by applying it for the texture classification problem
since textures found in nature are random and at the same time contain some
degree of correlation or regularity at some scale. The gray level co-occurrence
probabilities (GLCP) are used for computing the entropy function. The
experimental results indicate high degree of the classification accuracy. The
performance of the new entropy function is found superior to other forms of
entropy such as Shannon, Renyi, Tsallis and Pal and Pal entropies on
comparison. Using the feature based polar interaction maps (FBIM) the proposed
entropy is shown to be the best measure among the entropies compared for
representing the correlated textures.