pwc-archive/papers-with-abstracts huggingface dataset:
Large-Scale Approximate Kernel Canonical Correlation Analysis
Authors: Weiran Wang, Karen Livescu
Date: 2015-11-15 00:00:00
Tasks: Representation Learning, Stochastic Optimization

Kernel canonical correlation analysis (KCCA) is a nonlinear multi-view
representation learning technique with broad applicability in statistics and
machine learning. Although there is a closed-form solution for the KCCA
objective, it involves solving an $N\times N$ eigenvalue system where $N$ is
the training set size, making its computational requirements in both memory and
time prohibitive for large-scale problems. Various approximation techniques
have been developed for KCCA. A commonly used approach is to first transform
the original inputs to an $M$-dimensional random feature space so that inner
products in the feature space approximate kernel evaluations, and then apply
linear CCA to the transformed inputs. In many applications, however, the
dimensionality $M$ of the random feature space may need to be very large in
order to obtain a sufficiently good approximation; it then becomes challenging
to perform the linear CCA step on the resulting very high-dimensional data
matrices. We show how to use a stochastic optimization algorithm, recently
proposed for linear CCA and its neural-network extension, to further alleviate
the computation requirements of approximate KCCA. This approach allows us to
run approximate KCCA on a speech dataset with $1.4$ million training samples
and a random feature space of dimensionality $M=100000$ on a typical
workstation.
Better Computer Go Player with Neural Network and Long-term Prediction
Authors: Yuandong Tian, Yan Zhu
Date: 2015-11-19 00:00:00
Tasks: 4k, Game of Go

Competing with top human players in the ancient game of Go has been a
long-term goal of artificial intelligence. Go's high branching factor makes
traditional search techniques ineffective, even on leading-edge hardware, and
Go's evaluation function could change drastically with one stone change. Recent
works [Maddison et al. (2015); Clark & Storkey (2015)] show that search is not
strictly necessary for machine Go players. A pure pattern-matching approach,
based on a Deep Convolutional Neural Network (DCNN) that predicts the next
move, can perform as well as Monte Carlo Tree Search (MCTS)-based open source
Go engines such as Pachi [Baudis & Gailly (2012)] if its search budget is
limited. We extend this idea in our bot named darkforest, which relies on a
DCNN designed for long-term predictions. Darkforest substantially improves the
win rate for pattern-matching approaches against MCTS-based approaches, even
with looser search budgets. Against human players, the newest versions,
darkfores2, achieve a stable 3d level on KGS Go Server as a ranked bot, a
substantial improvement upon the estimated 4k-5k ranks for DCNN reported in
Clark & Storkey (2015) based on games against other machine players. Adding
MCTS to darkfores2 creates a much stronger player named darkfmcts3: with 5000
rollouts, it beats Pachi with 10k rollouts in all 250 games; with 75k rollouts
it achieves a stable 5d level in KGS server, on par with state-of-the-art Go
AIs (e.g., Zen, DolBaram, CrazyStone) except for AlphaGo [Silver et al.
(2016)]; with 110k rollouts, it won the 3rd place in January KGS Go Tournament.
Even Trolls Are Useful: Efficient Link Classification in Signed Networks
Authors: Géraud Le Falher, Fabio Vitale
Date: 2016-02-29 00:00:00
Tasks: General Classification

We address the problem of classifying the links of signed social networks
given their full structural topology. Motivated by a binary user behaviour
assumption, which is supported by decades of research in psychology, we develop
an efficient and surprisingly simple approach to solve this classification
problem. Our methods operate both within the active and batch settings. We
demonstrate that the algorithms we developed are extremely fast in both
theoretical and practical terms. Within the active setting, we provide a new
complexity measure and a rigorous analysis of our methods that hold for
arbitrary signed networks. We validate our theoretical claims carrying out a
set of experiments on three well known real-world datasets, showing that our
methods outperform the competitors while being much faster.
Clustering Based Feature Learning on Variable Stars
Authors: Cristóbal Mackenzie, Karim Pichara, Pavlos Protopapas
Date: 2016-02-29 00:00:00
Tasks: Classification Of Variable Stars, Clustering, General Classification, Time Series Analysis

The success of automatic classification of variable stars strongly depends on
the lightcurve representation. Usually, lightcurves are represented as a vector
of many statistical descriptors designed by astronomers called features. These
descriptors commonly demand significant computational power to calculate,
require substantial research effort to develop and do not guarantee good
performance on the final classification task. Today, lightcurve representation
is not entirely automatic; algorithms that extract lightcurve features are
designed by humans and must be manually tuned up for every survey. The vast
amounts of data that will be generated in future surveys like LSST mean
astronomers must develop analysis pipelines that are both scalable and
automated. Recently, substantial efforts have been made in the machine learning
community to develop methods that prescind from expert-designed and manually
tuned features for features that are automatically learned from data. In this
work we present what is, to our knowledge, the first unsupervised feature
learning algorithm designed for variable stars. Our method first extracts a
large number of lightcurve subsequences from a given set of photometric data,
which are then clustered to find common local patterns in the time series.
Representatives of these patterns, called exemplars, are then used to transform
lightcurves of a labeled set into a new representation that can then be used to
train an automatic classifier. The proposed algorithm learns the features from
both labeled and unlabeled lightcurves, overcoming the bias generated when the
learning process is done only with labeled data. We test our method on MACHO
and OGLE datasets; the results show that the classification performance we
achieve is as good and in some cases better than the performance achieved using
traditional features, while the computational cost is significantly lower.
Neural Programmer-Interpreters
Authors: Scott Reed, Nando de Freitas
Date: 2015-11-19 00:00:00

We propose the neural programmer-interpreter (NPI): a recurrent and
compositional neural network that learns to represent and execute programs. NPI
has three learnable components: a task-agnostic recurrent core, a persistent
key-value program memory, and domain-specific encoders that enable a single NPI
to operate in multiple perceptually diverse environments with distinct
affordances. By learning to compose lower-level programs to express
higher-level programs, NPI reduces sample complexity and increases
generalization ability compared to sequence-to-sequence LSTMs. The program
memory allows efficient learning of additional tasks by building on existing
programs. NPI can also harness the environment (e.g. a scratch pad with
read-write pointers) to cache intermediate results of computation, lessening
the long-term memory burden on recurrent hidden units. In this work we train
the NPI with fully-supervised execution traces; each program has example
sequences of calls to the immediate subprograms conditioned on the input.
Rather than training on a huge number of relatively weak labels, NPI learns
from a small number of rich examples. We demonstrate the capability of our
model to learn several types of compositional programs: addition, sorting, and
canonicalizing 3D models. Furthermore, a single NPI learns to execute these
programs and all 21 associated subprograms.
Range-based argumentation semantics as 2-valued models
Authors: Mauricio Osorio, Juan Carlos Nieves
Date: 2016-02-29 00:00:00

Characterizations of semi-stable and stage extensions in terms of 2-valued
logical models are presented. To this end, the so-called GL-supported and
GL-stage models are defined. These two classes of logical models are logic
programming counterparts of the notion of range which is an established concept
in argumentation semantics.
Discovering Beaten Paths in Collaborative Ontology-Engineering Projects using Markov Chains
Authors: Simon Walk, Philipp Singer, Markus Strohmaier, Tania Tudorache, Mark A. Musen, Natalya F. Noy
Date: 2014-07-08 00:00:00

Biomedical taxonomies, thesauri and ontologies in the form of the
International Classification of Diseases (ICD) as a taxonomy or the National
Cancer Institute Thesaurus as an OWL-based ontology, play a critical role in
acquiring, representing and processing information about human health. With
increasing adoption and relevance, biomedical ontologies have also
significantly increased in size. For example, the 11th revision of the ICD,
which is currently under active development by the WHO contains nearly 50,000
classes representing a vast variety of different diseases and causes of death.
This evolution in terms of size was accompanied by an evolution in the way
ontologies are engineered. Because no single individual has the expertise to
develop such large-scale ontologies, ontology-engineering projects have evolved
from small-scale efforts involving just a few domain experts to large-scale
projects that require effective collaboration between dozens or even hundreds
of experts, practitioners and other stakeholders. Understanding how these
stakeholders collaborate will enable us to improve editing environments that
support such collaborations. We uncover how large ontology-engineering
projects, such as the ICD in its 11th revision, unfold by analyzing usage logs
of five different biomedical ontology-engineering projects of varying sizes and
scopes using Markov chains. We discover intriguing interaction patterns (e.g.,
which properties users subsequently change) that suggest that large
collaborative ontology-engineering projects are governed by a few general
principles that determine and drive development. From our analysis, we identify
commonalities and differences between different projects that have implications
for project managers, ontology editors, developers and contributors working on
collaborative ontology-engineering projects and tools in the biomedical domain.
Pandora: Description of a Painting Database for Art Movement Recognition with Baselines and Perspectives
Authors: Corneliu Florea, Razvan Condorovici, Constantin Vertan, Raluca Boia, Laura Florea, Ruxandra Vranceanu
Date: 2016-02-29 00:00:00
Tasks: General Classification

To facilitate computer analysis of visual art, in the form of paintings, we
introduce Pandora (Paintings Dataset for Recognizing the Art movement)
database, a collection of digitized paintings labelled with respect to the
artistic movement. Noting that the set of databases available as benchmarks for
evaluation is highly reduced and most existing ones are limited in variability
and number of images, we propose a novel large scale dataset of digital
paintings. The database consists of more than 7700 images from 12 art
movements. Each genre is illustrated by a number of images varying from 250 to
nearly 1000. We investigate how local and global features and classification
systems are able to recognize the art movement. Our experimental results
suggest that accurate recognition is achievable by a combination of various
categories.To facilitate computer analysis of visual art, in the form of
paintings, we introduce Pandora (Paintings Dataset for Recognizing the Art
movement) database, a collection of digitized paintings labelled with respect
to the artistic movement. Noting that the set of databases available as
benchmarks for evaluation is highly reduced and most existing ones are limited
in variability and number of images, we propose a novel large scale dataset of
digital paintings. The database consists of more than 7700 images from 12 art
movements. Each genre is illustrated by a number of images varying from 250 to
nearly 1000. We investigate how local and global features and classification
systems are able to recognize the art movement. Our experimental results
suggest that accurate recognition is achievable by a combination of various
categories.
Exploring the coevolution of predator and prey morphology and behavior
Authors: Randal S. Olson, Arend Hintze, Fred C. Dyer, Jason H. Moore, Christoph Adami
Date: 2016-02-29 00:00:00

A common idiom in biology education states, "Eyes in the front, the animal
hunts. Eyes on the side, the animal hides." In this paper, we explore one
possible explanation for why predators tend to have forward-facing, high-acuity
visual systems. We do so using an agent-based computational model of evolution,
where predators and prey interact and adapt their behavior and morphology to
one another over successive generations of evolution. In this model, we observe
a coevolutionary cycle between prey swarming behavior and the predator's visual
system, where the predator and prey continually adapt their visual system and
behavior, respectively, over evolutionary time in reaction to one another due
to the well-known "predator confusion effect." Furthermore, we provide evidence
that the predator visual system is what drives this coevolutionary cycle, and
suggest that the cycle could be closed if the predator evolves a hybrid visual
system capable of narrow, high-acuity vision for tracking prey as well as
broad, coarse vision for prey discovery. Thus, the conflicting demands imposed
on a predator's visual system by the predator confusion effect could have led
to the evolution of complex eyes in many predators.
Iterative Aggregation Method for Solving Principal Component Analysis Problems
Authors: Vitaly Bulgakov
Date: 2016-02-29 00:00:00

Motivated by the previously developed multilevel aggregation method for
solving structural analysis problems a novel two-level aggregation approach for
efficient iterative solution of Principal Component Analysis (PCA) problems is
proposed. The course aggregation model of the original covariance matrix is
used in the iterative solution of the eigenvalue problem by a power iterations
method. The method is tested on several data sets consisting of large number of
text documents.
A Spectral Algorithm for Inference in Hidden Semi-Markov Models
Authors: Igor Melnyk, Arindam Banerjee
Date: 2014-07-12 00:00:00

Hidden semi-Markov models (HSMMs) are latent variable models which allow
latent state persistence and can be viewed as a generalization of the popular
hidden Markov models (HMMs). In this paper, we introduce a novel spectral
algorithm to perform inference in HSMMs. Unlike expectation maximization (EM),
our approach correctly estimates the probability of given observation sequence
based on a set of training sequences. Our approach is based on estimating
moments from the sample, whose number of dimensions depends only
logarithmically on the maximum length of the hidden state persistence.
Moreover, the algorithm requires only a few matrix inversions and is therefore
computationally efficient. Empirical evaluations on synthetic and real data
demonstrate the advantage of the algorithm over EM in terms of speed and
accuracy, especially for large datasets.
SparkNet: Training Deep Networks in Spark
Authors: Philipp Moritz, Robert Nishihara, Ion Stoica, Michael. I. Jordan
Date: 2015-11-19 00:00:00

Training deep networks is a time-consuming process, with networks for object
recognition often requiring multiple days to train. For this reason, leveraging
the resources of a cluster to speed up training is an important area of work.
However, widely-popular batch-processing computational frameworks like
MapReduce and Spark were not designed to support the asynchronous and
communication-intensive workloads of existing distributed deep learning
systems. We introduce SparkNet, a framework for training deep networks in
Spark. Our implementation includes a convenient interface for reading data from
Spark RDDs, a Scala interface to the Caffe deep learning framework, and a
lightweight multi-dimensional tensor library. Using a simple parallelization
scheme for stochastic gradient descent, SparkNet scales well with the cluster
size and tolerates very high-latency communication. Furthermore, it is easy to
deploy and use with no parameter tuning, and it is compatible with existing
Caffe models. We quantify the dependence of the speedup obtained by SparkNet on
the number of machines, the communication frequency, and the cluster's
communication overhead, and we benchmark our system's performance on the
ImageNet dataset.
Semi-Markov Switching Vector Autoregressive Model-based Anomaly Detection in Aviation Systems
Authors: Igor Melnyk, Arindam Banerjee, Bryan Matthews, Nikunj Oza
Date: 2016-02-21 00:00:00
Tasks: Anomaly Detection, Time Series, Time Series Analysis

In this work we consider the problem of anomaly detection in heterogeneous,
multivariate, variable-length time series datasets. Our focus is on the
aviation safety domain, where data objects are flights and time series are
sensor readings and pilot switches. In this context the goal is to detect
anomalous flight segments, due to mechanical, environmental, or human factors
in order to identifying operationally significant events and provide insights
into the flight operations and highlight otherwise unavailable potential safety
risks and precursors to accidents. For this purpose, we propose a framework
which represents each flight using a semi-Markov switching vector
autoregressive (SMS-VAR) model. Detection of anomalies is then based on
measuring dissimilarities between the model's prediction and data observation.
The framework is scalable, due to the inherent parallel nature of most
computations, and can be used to perform online anomaly detection. Extensive
experimental results on simulated and real datasets illustrate that the
framework can detect various types of anomalies along with the key parameters
involved.
Estimating Structured Vector Autoregressive Model
Authors: Igor Melnyk, Arindam Banerjee
Date: 2016-02-21 00:00:00
Tasks: model

While considerable advances have been made in estimating high-dimensional
structured models from independent data using Lasso-type models, limited
progress has been made for settings when the samples are dependent. We consider
estimating structured VAR (vector auto-regressive models), where the structure
can be captured by any suitable norm, e.g., Lasso, group Lasso, order weighted
Lasso, sparse group Lasso, etc. In VAR setting with correlated noise, although
there is strong dependence over time and covariates, we establish bounds on the
non-asymptotic estimation error of structured VAR parameters. Surprisingly, the
estimation error is of the same order as that of the corresponding Lasso-type
estimator with independent samples, and the analysis holds for any norm. Our
analysis relies on results in generic chaining, sub-exponential martingales,
and spectral representation of VAR models. Experimental results on synthetic
data with a variety of structures as well as real aviation data are presented,
validating theoretical results.
Convergent Learning: Do different neural networks learn the same representations?
Authors: Yixuan Li, Jason Yosinski, Jeff Clune, Hod Lipson, John Hopcroft
Date: 2015-11-24 00:00:00
Tasks: Clustering

Recent success in training deep neural networks have prompted active
investigation into the features learned on their intermediate layers. Such
research is difficult because it requires making sense of non-linear
computations performed by millions of parameters, but valuable because it
increases our ability to understand current models and create improved versions
of them. In this paper we investigate the extent to which neural networks
exhibit what we call convergent learning, which is when the representations
learned by multiple nets converge to a set of features which are either
individually similar between networks or where subsets of features span similar
low-dimensional spaces. We propose a specific method of probing
representations: training multiple networks and then comparing and contrasting
their individual, learned representations at the level of neurons or groups of
neurons. We begin research into this question using three techniques to
approximately align different neural networks on a feature level: a bipartite
matching approach that makes one-to-one assignments between neurons, a sparse
prediction approach that finds one-to-many mappings, and a spectral clustering
approach that finds many-to-many mappings. This initial investigation reveals a
few previously unknown properties of neural networks, and we argue that future
research into the question of convergent learning will yield many more. The
insights described here include (1) that some features are learned reliably in
multiple networks, yet other features are not consistently learned; (2) that
units learn to span low-dimensional subspaces and, while these subspaces are
common to multiple networks, the specific basis vectors learned are not; (3)
that the representation codes show evidence of being a mix between a local code
and slightly, but not fully, distributed codes across multiple units.
Gibberish Semantics: How Good is Russian Twitter in Word Semantic Similarity Task?
Authors: Nikolay N. Vasiliev
Date: 2016-02-28 00:00:00
Tasks: Semantic Similarity, Semantic Textual Similarity

The most studied and most successful language models were developed and
evaluated mainly for English and other close European languages, such as
French, German, etc. It is important to study applicability of these models to
other languages. The use of vector space models for Russian was recently
studied for multiple corpora, such as Wikipedia, RuWac, lib.ru. These models
were evaluated against word semantic similarity task. For our knowledge Twitter
was not considered as a corpus for this task, with this work we fill the gap.
Results for vectors trained on Twitter corpus are comparable in accuracy with
other single-corpus trained models, although the best performance is currently
achieved by combination of multiple corpora.
A Structured Variational Auto-encoder for Learning Deep Hierarchies of Sparse Features
Authors: Tim Salimans
Date: 2016-02-28 00:00:00
Tasks: Variational Inference, Vocal Bursts Type Prediction

In this note we present a generative model of natural images consisting of a
deep hierarchy of layers of latent random variables, each of which follows a
new type of distribution that we call rectified Gaussian. These rectified
Gaussian units allow spike-and-slab type sparsity, while retaining the
differentiability necessary for efficient stochastic gradient variational
inference. To learn the parameters of the new model, we approximate the
posterior of the latent variables with a variational auto-encoder. Rather than
making the usual mean-field assumption however, the encoder parameterizes a new
type of structured variational approximation that retains the prior
dependencies of the generative model. Using this structured posterior
approximation, we are able to perform joint training of deep models with many
layers of latent random variables, without having to resort to stacking or
other layerwise training procedures.