pwc-archive/papers-with-abstracts huggingface dataset:
Manifold unwrapping using density ridges
Authors: Jonas Nordhaug Myhre, Matineh Shaker, Devrim Kaba, Robert Jenssen, Deniz Erdogmus
Date: 2016-04-06 00:00:00

Research on manifold learning within a density ridge estimation framework has
shown great potential in recent work for both estimation and de-noising of
manifolds, building on the intuitive and well-defined notion of principal
curves and surfaces. However, the problem of unwrapping or unfolding manifolds
has received relatively little attention within the density ridge approach,
despite being an integral part of manifold learning in general. This paper
proposes two novel algorithms for unwrapping manifolds based on estimated
principal curves and surfaces for one- and multi-dimensional manifolds
respectively. The methods of unwrapping are founded in the realization that
both principal curves and principal surfaces will have inherent local maxima of
the probability density function. Following this observation, coordinate
systems that follow the shape of the manifold can be computed by following the
integral curves of the gradient flow of a kernel density estimate on the
manifold. Furthermore, since integral curves of the gradient flow of a kernel
density estimate is inherently local, we propose to stitch together local
coordinate systems using parallel transport along the manifold. We provide
numerical experiments on both real and synthetic data that illustrates clear
and intuitive unwrapping results comparable to state-of-the-art manifold
learning algorithms.
Online Open World Recognition
Authors: Rocco De Rosa, Thomas Mensink, Barbara Caputo
Date: 2016-04-08 00:00:00
Tasks: Incremental Learning, Metric Learning

As we enter into the big data age and an avalanche of images have become
readily available, recognition systems face the need to move from close, lab
settings where the number of classes and training data are fixed, to dynamic
scenarios where the number of categories to be recognized grows continuously
over time, as well as new data providing useful information to update the
system. Recent attempts, like the open world recognition framework, tried to
inject dynamics into the system by detecting new unknown classes and adding
them incrementally, while at the same time continuously updating the models for
the known classes. incrementally adding new classes and detecting instances
from unknown classes, while at the same time continuously updating the models
for the known classes. In this paper we argue that to properly capture the
intrinsic dynamic of open world recognition, it is necessary to add to these
aspects (a) the incremental learning of the underlying metric, (b) the
incremental estimate of confidence thresholds for the unknown classes, and (c)
the use of local learning to precisely describe the space of classes. We extend
three existing metric learning algorithms towards these goals by using online
metric learning. Experimentally we validate our approach on two large-scale
datasets in different learning scenarios. For all these scenarios our proposed
methods outperform their non-online counterparts. We conclude that local and
online learning is important to capture the full dynamics of open world
recognition.
Probabilistic classifiers with low rank indefinite kernels
Authors: Frank-Michael Schleif, Andrej Gisbrecht, Peter Tino
Date: 2016-04-08 00:00:00
Tasks: Image Retrieval, Retrieval

Indefinite similarity measures can be frequently found in bio-informatics by
means of alignment scores, but are also common in other fields like shape
measures in image retrieval. Lacking an underlying vector space, the data are
given as pairwise similarities only. The few algorithms available for such data
do not scale to larger datasets. Focusing on probabilistic batch classifiers,
the Indefinite Kernel Fisher Discriminant (iKFD) and the Probabilistic
Classification Vector Machine (PCVM) are both effective algorithms for this
type of data but, with cubic complexity. Here we propose an extension of iKFD
and PCVM such that linear runtime and memory complexity is achieved for low
rank indefinite kernels. Employing the Nystr\"om approximation for indefinite
kernels, we also propose a new almost parameter free approach to identify the
landmarks, restricted to a supervised learning problem. Evaluations at several
larger similarity data from various domains show that the proposed methods
provides similar generalization capabilities while being easier to parametrize
and substantially faster for large scale data.
Geometric Scene Parsing with Hierarchical LSTM
Authors: Zhanglin Peng, Ruimao Zhang, Xiaodan Liang, Xiaobai Liu, Liang Lin
Date: 2016-04-07 00:00:00
Tasks: 3D Reconstruction, Relation Prediction, Scene Labeling, Scene Parsing

This paper addresses the problem of geometric scene parsing, i.e.
simultaneously labeling geometric surfaces (e.g. sky, ground and vertical
plane) and determining the interaction relations (e.g. layering, supporting,
siding and affinity) between main regions. This problem is more challenging
than the traditional semantic scene labeling, as recovering geometric
structures necessarily requires the rich and diverse contextual information. To
achieve these goals, we propose a novel recurrent neural network model, named
Hierarchical Long Short-Term Memory (H-LSTM). It contains two coupled
sub-networks: the Pixel LSTM (P-LSTM) and the Multi-scale Super-pixel LSTM
(MS-LSTM) for handling the surface labeling and relation prediction,
respectively. The two sub-networks provide complementary information to each
other to exploit hierarchical scene contexts, and they are jointly optimized
for boosting the performance. Our extensive experiments show that our model is
capable of parsing scene geometric structures and outperforming several
state-of-the-art methods by large margins. In addition, we show promising 3D
reconstruction results from the still images based on the geometric parsing.
Sentence Level Recurrent Topic Model: Letting Topics Speak for Themselves
Authors: Fei Tian, Bin Gao, Di He, Tie-Yan Liu
Date: 2016-04-07 00:00:00
Tasks: Sentence, Short-Text Conversation, Topic Models

We propose Sentence Level Recurrent Topic Model (SLRTM), a new topic model
that assumes the generation of each word within a sentence to depend on both
the topic of the sentence and the whole history of its preceding words in the
sentence. Different from conventional topic models that largely ignore the
sequential order of words or their topic coherence, SLRTM gives full
characterization to them by using a Recurrent Neural Networks (RNN) based
framework. Experimental results have shown that SLRTM outperforms several
strong baselines on various tasks. Furthermore, SLRTM can automatically
generate sentences given a topic (i.e., topics to sentences), which is a key
technology for real world applications such as personalized short text
conversation.
Feature-Based Diversity Optimization for Problem Instance Classification
Authors: Wanru Gao, Samadhi Nallaperuma, Frank Neumann
Date: 2015-10-29 00:00:00
Tasks: Classification, Diversity, General Classification, Heuristic Search

Understanding the behaviour of heuristic search methods is a challenge. This even holds for simple local search methods such as 2-OPT for the Traveling Salesperson problem. In this paper, we present a general framework that is able to construct a diverse set of instances that are hard or easy for a given search heuristic. Such a diverse set is obtained by using an evolutionary algorithm for constructing hard or easy instances that are diverse with respect to different features of the underlying problem. Examining the constructed instance sets, we show that many combinations of two or three features give a good classification of the TSP instances in terms of whether they are hard to be solved by 2-OPT.
Characterizing Concept Drift
Authors: Geoffrey I. Webb, Roy Hyde, Hong Cao, Hai Long Nguyen, Francois Petitjean
Date: 2015-11-12 00:00:00

Most machine learning models are static, but the world is dynamic, and
increasing online deployment of learned models gives increasing urgency to the
development of efficient and effective mechanisms to address learning in the
context of non-stationary distributions, or as it is commonly called concept
drift. However, the key issue of characterizing the different types of drift
that can occur has not previously been subjected to rigorous definition and
analysis. In particular, while some qualitative drift categorizations have been
proposed, few have been formally defined, and the quantitative descriptions
required for precise and objective understanding of learner performance have
not existed. We present the first comprehensive framework for quantitative
analysis of drift. This supports the development of the first comprehensive set
of formal definitions of types of concept drift. The formal definitions clarify
ambiguities and identify gaps in previous definitions, giving rise to a new
comprehensive taxonomy of concept drift types and a solid foundation for
research into mechanisms to detect and address concept drift.
A Survey on Soft Subspace Clustering
Authors: Zhaohong Deng, Kup-Sze Choi, Yizhang Jiang, Jun Wang, Shitong Wang
Date: 2014-09-19 00:00:00
Tasks: Clustering, Survey

Subspace clustering (SC) is a promising clustering technology to identify
clusters based on their associations with subspaces in high dimensional spaces.
SC can be classified into hard subspace clustering (HSC) and soft subspace
clustering (SSC). While HSC algorithms have been extensively studied and well
accepted by the scientific community, SSC algorithms are relatively new but
gaining more attention in recent years due to better adaptability. In the
paper, a comprehensive survey on existing SSC algorithms and the recent
development are presented. The SSC algorithms are classified systematically
into three main categories, namely, conventional SSC (CSSC), independent SSC
(ISSC) and extended SSC (XSSC). The characteristics of these algorithms are
highlighted and the potential future development of SSC is also discussed.
Deep Online Convex Optimization by Putting Forecaster to Sleep
Authors: David Balduzzi
Date: 2015-09-06 00:00:00
Tasks: Model Selection

Methods from convex optimization such as accelerated gradient descent are
widely used as building blocks for deep learning algorithms. However, the
reasons for their empirical success are unclear, since neural networks are not
convex and standard guarantees do not apply. This paper develops the first
rigorous link between online convex optimization and error backpropagation on
convolutional networks. The first step is to introduce circadian games, a mild
generalization of convex games with similar convergence properties. The main
result is that error backpropagation on a convolutional network is equivalent
to playing out a circadian game. It follows immediately that the waking-regret
of players in the game (the units in the neural network) controls the overall
rate of convergence of the network. Finally, we explore some implications of
the results: (i) we describe the representations learned by a neural network
game-theoretically, (ii) propose a learning setting at the level of individual
units that can be plugged into deep architectures, and (iii) propose a new
approach to adaptive model selection by applying bandit algorithms to choose
which players to wake on each round.
Collaborative prediction with expert advice
Authors: Paul Christiano
Date: 2016-03-20 00:00:00
Tasks: Learning Theory, Prediction

Many practical learning systems aggregate data across many users, while
learning theory traditionally considers a single learner who trusts all of
their observations. A case in point is the foundational learning problem of
prediction with expert advice. To date, there has been no theoretical study of
the general collaborative version of prediction with expert advice, in which
many users face a similar problem and would like to share their experiences in
order to learn faster. A key issue in this collaborative framework is
robustness: generally algorithms that aggregate data are vulnerable to
manipulation by even a small number of dishonest users.
  We exhibit the first robust collaborative algorithm for prediction with
expert advice. When all users are honest and have similar tastes our algorithm
matches the performance of pooling data and using a traditional algorithm. But
our algorithm also guarantees that adding users never significantly degrades
performance, even if the additional users behave adversarially. We achieve
strong guarantees even when the overwhelming majority of users behave
adversarially. As a special case, our algorithm is extremely robust to
variation amongst the users.
Transfer Learning for Low-Resource Neural Machine Translation
Authors: Barret Zoph, Deniz Yuret, Jonathan May, Kevin Knight
Date: 2016-04-08 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Decoder, Low Resource Neural Machine Translation, Low-Resource Neural Machine Translation, Machine Translation, NMT, Transfer Learning, Translation

The encoder-decoder framework for neural machine translation (NMT) has been
shown effective in large data scenarios, but is much less effective for
low-resource languages. We present a transfer learning method that
significantly improves Bleu scores across a range of low-resource languages.
Our key idea is to first train a high-resource language pair (the parent
model), then transfer some of the learned parameters to the low-resource pair
(the child model) to initialize and constrain training. Using our transfer
learning method we improve baseline NMT models by an average of 5.6 Bleu on
four low-resource language pairs. Ensembling and unknown word replacement add
another 2 Bleu which brings the NMT performance on low-resource machine
translation close to a strong syntax based machine translation (SBMT) system,
exceeding its performance on one language pair. Additionally, using the
transfer learning model for re-scoring, we can improve the SBMT system by an
average of 1.3 Bleu, improving the state-of-the-art on low-resource machine
translation.
Automatic Annotation of Structured Facts in Images
Authors: Mohamed Elhoseiny, Scott Cohen, Walter Chang, Brian Price, Ahmed Elgammal
Date: 2016-04-02 00:00:00
Proceeding: WS 2016 8
Tasks: CPU

Motivated by the application of fact-level image understanding, we present an
automatic method for data collection of structured visual facts from images
with captions. Example structured facts include attributed objects (e.g.,
<flower, red>), actions (e.g., <baby, smile>), interactions (e.g., <man,
walking, dog>), and positional information (e.g., <vase, on, table>). The
collected annotations are in the form of fact-image pairs (e.g.,<man, walking,
dog> and an image region containing this fact). With a language approach, the
proposed method is able to collect hundreds of thousands of visual fact
annotations with accuracy of 83% according to human judgment. Our method
automatically collected more than 380,000 visual fact annotations and more than
110,000 unique visual facts from images with captions and localized them in
images in less than one day of processing time on standard CPU platforms.
A Kernel-Based Nonparametric Test for Anomaly Detection over Line Networks
Authors: Shaofeng Zou, Yingbin Liang, H. Vincent Poor
Date: 2014-04-01 00:00:00
Tasks: Anomaly Detection

The nonparametric problem of detecting existence of an anomalous interval
over a one dimensional line network is studied. Nodes corresponding to an
anomalous interval (if exists) receive samples generated by a distribution q,
which is different from the distribution p that generates samples for other
nodes. If anomalous interval does not exist, then all nodes receive samples
generated by p. It is assumed that the distributions p and q are arbitrary, and
are unknown. In order to detect whether an anomalous interval exists, a test is
built based on mean embeddings of distributions into a reproducing kernel
Hilbert space (RKHS) and the metric of maximummean discrepancy (MMD). It is
shown that as the network size n goes to infinity, if the minimum length of
candidate anomalous intervals is larger than a threshold which has the order
O(log n), the proposed test is asymptotically successful, i.e., the probability
of detection error approaches zero asymptotically. An efficient algorithm to
perform the test with substantial computational complexity reduction is
proposed, and is shown to be asymptotically successful if the condition on the
minimum length of candidate anomalous interval is satisfied. Numerical results
are provided, which are consistent with the theoretical results.
Revising Incompletely Specified Convex Probabilistic Belief Bases
Authors: Gavin Rens, Thomas Meyer, Giovanni Casini
Date: 2016-04-07 00:00:00

We propose a method for an agent to revise its incomplete probabilistic
beliefs when a new piece of propositional information is observed. In this
work, an agent's beliefs are represented by a set of probabilistic formulae --
a belief base. The method involves determining a representative set of
'boundary' probability distributions consistent with the current belief base,
revising each of these probability distributions and then translating the
revised information into a new belief base. We use a version of Lewis Imaging
as the revision operation. The correctness of the approach is proved. The
expressivity of the belief bases under consideration are rather restricted, but
has some applications. We also discuss methods of belief base revision
employing the notion of optimum entropy, and point out some of the benefits and
difficulties in those methods. Both the boundary distribution method and the
optimum entropy method are reasonable, yet yield different results.
On Stochastic Belief Revision and Update and their Combination
Authors: Gavin Rens
Date: 2016-04-07 00:00:00

I propose a framework for an agent to change its probabilistic beliefs when a
new piece of propositional information $\alpha$ is observed. Traditionally,
belief change occurs by either a revision process or by an update process,
depending on whether the agent is informed with $\alpha$ in a static world or,
respectively, whether $\alpha$ is a 'signal' from the environment due to an
event occurring. Boutilier suggested a unified model of qualitative belief
change, which "combines aspects of revision and update, providing a more
realistic characterization of belief change." In this paper, I propose a
unified model of quantitative belief change, where an agent's beliefs are
represented as a probability distribution over possible worlds. As does
Boutilier, I take a dynamical systems perspective. The proposed approach is
evaluated against several rationality postulated, and some properties of the
approach are worked out.
Multilevel Weighted Support Vector Machine for Classification on Healthcare Data with Missing Values
Authors: Talayeh Razzaghi, Oleg Roderick, Ilya Safro, Nicholas Marko
Date: 2016-04-07 00:00:00
Tasks: Classification, General Classification, Imputation, Missing Values, Robust classification

This work is motivated by the needs of predictive analytics on healthcare
data as represented by Electronic Medical Records. Such data is invariably
problematic: noisy, with missing entries, with imbalance in classes of
interests, leading to serious bias in predictive modeling. Since standard data
mining methods often produce poor performance measures, we argue for
development of specialized techniques of data-preprocessing and classification.
In this paper, we propose a new method to simultaneously classify large
datasets and reduce the effects of missing values. It is based on a multilevel
framework of the cost-sensitive SVM and the expected maximization imputation
method for missing values, which relies on iterated regression analyses. We
compare classification results of multilevel SVM-based algorithms on public
benchmark datasets with imbalanced classes and missing values as well as real
data in health applications, and show that our multilevel SVM-based method
produces fast, and more accurate and robust classification results.
A Single Model Explains both Visual and Auditory Precortical Coding
Authors: Honghao Shan, Matthew H. Tong, Garrison W. Cottrell
Date: 2016-02-26 00:00:00

Precortical neural systems encode information collected by the senses, but
the driving principles of the encoding used have remained a subject of debate.
We present a model of retinal coding that is based on three constraints:
information preservation, minimization of the neural wiring, and response
equalization. The resulting novel version of sparse principal components
analysis successfully captures a number of known characteristics of the retinal
coding system, such as center-surround receptive fields, color opponency
channels, and spatiotemporal responses that correspond to magnocellular and
parvocellular pathways. Furthermore, when trained on auditory data, the same
model learns receptive fields well fit by gammatone filters, commonly used to
model precortical auditory coding. This suggests that efficient coding may be a
unifying principle of precortical encoding across modalities.
Trajectory Aligned Features For First Person Action Recognition
Authors: Suriya Singh, Chetan Arora, C. V. Jawahar
Date: 2016-04-07 00:00:00
Tasks: Action Recognition, Point Tracking, Temporal Action Localization

Egocentric videos are characterised by their ability to have the first person
view. With the popularity of Google Glass and GoPro, use of egocentric videos
is on the rise. Recognizing action of the wearer from egocentric videos is an
important problem. Unstructured movement of the camera due to natural head
motion of the wearer causes sharp changes in the visual field of the egocentric
camera causing many standard third person action recognition techniques to
perform poorly on such videos. Objects present in the scene and hand gestures
of the wearer are the most important cues for first person action recognition
but are difficult to segment and recognize in an egocentric video. We propose a
novel representation of the first person actions derived from feature
trajectories. The features are simple to compute using standard point tracking
and does not assume segmentation of hand/objects or recognizing object or hand
pose unlike in many previous approaches. We train a bag of words classifier
with the proposed features and report a performance improvement of more than
11% on publicly available datasets. Although not designed for the particular
case, we show that our technique can also recognize wearer's actions when hands
or objects are not visible.
Planning with Information-Processing Constraints and Model Uncertainty in Markov Decision Processes
Authors: Jordi Grau-Moya, Felix Leibfried, Tim Genewein, Daniel A. Braun
Date: 2016-04-07 00:00:00

Information-theoretic principles for learning and acting have been proposed
to solve particular classes of Markov Decision Problems. Mathematically, such
approaches are governed by a variational free energy principle and allow
solving MDP planning problems with information-processing constraints expressed
in terms of a Kullback-Leibler divergence with respect to a reference
distribution. Here we consider a generalization of such MDP planners by taking
model uncertainty into account. As model uncertainty can also be formalized as
an information-processing constraint, we can derive a unified solution from a
single generalized variational principle. We provide a generalized value
iteration scheme together with a convergence proof. As limit cases, this
generalized scheme includes standard value iteration with a known model,
Bayesian MDP planning, and robust planning. We demonstrate the benefits of this
approach in a grid world simulation.