pwc-archive/papers-with-abstracts huggingface dataset:
POS Tagging and its Applications for Mathematics
Authors: Ulf Schöneberg, Wolfram Sperber
Date: 2014-06-11 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, POS, POS Tagging

Content analysis of scientific publications is a nontrivial task, but a
useful and important one for scientific information services. In the Gutenberg
era it was a domain of human experts; in the digital age many machine-based
methods, e.g., graph analysis tools and machine-learning techniques, have been
developed for it. Natural Language Processing (NLP) is a powerful
machine-learning approach to semiautomatic speech and language processing,
which is also applicable to mathematics. The well established methods of NLP
have to be adjusted for the special needs of mathematics, in particular for
handling mathematical formulae. We demonstrate a mathematics-aware part of
speech tagger and give a short overview about our adaptation of NLP methods for
mathematical publications. We show the use of the tools developed for key
phrase extraction and classification in the database zbMATH.
Extract ABox Modules for Efficient Ontology Querying
Authors: Jia Xu, Patrick Shironoshita, Ubbo Visser, Nigel John, Mansur Kabuka
Date: 2013-05-21 00:00:00
Tasks: Retrieval

The extraction of logically-independent fragments out of an ontology ABox can
be useful for solving the tractability problem of querying ontologies with
large ABoxes. In this paper, we propose a formal definition of an ABox module,
such that it guarantees complete preservation of facts about a given set of
individuals, and thus can be reasoned independently w.r.t. the ontology TBox.
With ABox modules of this type, isolated or distributed (parallel) ABox
reasoning becomes feasible, and more efficient data retrieval from ontology
ABoxes can be attained. To compute such an ABox module, we present a
theoretical approach and also an approximation for $\mathcal{SHIQ}$ ontologies.
Evaluation of the module approximation on different types of ontologies shows
that, on average, extracted ABox modules are significantly smaller than the
entire ABox, and the time for ontology reasoning based on ABox modules can be
improved significantly.
Algebraic-Combinatorial Methods for Low-Rank Matrix Completion with Application to Athletic Performance Prediction
Authors: Duncan A. J. Blythe, Louis Theran, Franz Kiraly
Date: 2014-06-11 00:00:00
Tasks: Computational Efficiency, Low-Rank Matrix Completion, Matrix Completion

This paper presents novel algorithms which exploit the intrinsic algebraic
and combinatorial structure of the matrix completion task for estimating
missing en- tries in the general low rank setting. For positive data, we
achieve results out- performing the state of the art nuclear norm, both in
accuracy and computational efficiency, in simulations and in the task of
predicting athletic performance from partially observed data.
Maximum Likelihood-based Online Adaptation of Hyper-parameters in CMA-ES
Authors: Ilya Loshchilov, Marc Schoenauer, Michèle Sebag, Nikolaus Hansen
Date: 2014-06-10 00:00:00

The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is widely
accepted as a robust derivative-free continuous optimization algorithm for
non-linear and non-convex optimization problems. CMA-ES is well known to be
almost parameterless, meaning that only one hyper-parameter, the population
size, is proposed to be tuned by the user. In this paper, we propose a
principled approach called self-CMA-ES to achieve the online adaptation of
CMA-ES hyper-parameters in order to improve its overall performance.
Experimental results show that for larger-than-default population size, the
default settings of hyper-parameters of CMA-ES are far from being optimal, and
that self-CMA-ES allows for dynamically approaching optimal settings.
A Hitchhiker's Guide to Search-Based Software Engineering for Software Product Lines
Authors: Roberto E. Lopez-Herrejon, Javier Ferrer, Francisco Chicano, Lukas Linsbauer, Alexander Egyed, Enrique Alba
Date: 2014-06-11 00:00:00

Search Based Software Engineering (SBSE) is an emerging discipline that
focuses on the application of search-based optimization techniques to software
engineering problems. The capacity of SBSE techniques to tackle problems
involving large search spaces make their application attractive for Software
Product Lines (SPLs). In recent years, several publications have appeared that
apply SBSE techniques to SPL problems. In this paper, we present the results of
a systematic mapping study of such publications. We identified the stages of
the SPL life cycle where SBSE techniques have been used, what case studies have
been employed and how they have been analysed. This mapping study revealed
potential venues for further research as well as common misunderstanding and
pitfalls when applying SBSE techniques that we address by providing a guideline
for researchers and practitioners interested in exploiting these techniques.
Bregman Divergences for Infinite Dimensional Covariance Matrices
Authors: Mehrtash Harandi, Mathieu Salzmann, Fatih Porikli
Date: 2014-03-18 00:00:00
Proceeding: CVPR 2014 6
Tasks: Action Recognition, General Classification, Person Re-Identification, Temporal Action Localization

We introduce an approach to computing and comparing Covariance Descriptors
(CovDs) in infinite-dimensional spaces. CovDs have become increasingly popular
to address classification problems in computer vision. While CovDs offer some
robustness to measurement variations, they also throw away part of the
information contained in the original data by only retaining the second-order
statistics over the measurements. Here, we propose to overcome this limitation
by first mapping the original data to a high-dimensional Hilbert space, and
only then compute the CovDs. We show that several Bregman divergences can be
computed between the resulting CovDs in Hilbert space via the use of kernels.
We then exploit these divergences for classification purposes. Our experiments
demonstrate the benefits of our approach on several tasks, such as material and
texture recognition, person re-identification, and action recognition from
motion capture data.
Provable Tensor Factorization with Missing Data
Authors: Prateek Jain, Sewoong Oh
Date: 2014-06-11 00:00:00
Proceeding: NeurIPS 2014 12

We study the problem of low-rank tensor factorization in the presence of
missing data. We ask the following question: how many sampled entries do we
need, to efficiently and exactly reconstruct a tensor with a low-rank
orthogonal decomposition? We propose a novel alternating minimization based
method which iteratively refines estimates of the singular vectors. We show
that under certain standard assumptions, our method can recover a three-mode
$n\times n\times n$ dimensional rank-$r$ tensor exactly from $O(n^{3/2} r^5
\log^4 n)$ randomly sampled entries. In the process of proving this result, we
solve two challenging sub-problems for tensors with missing data. First, in the
process of analyzing the initialization step, we prove a generalization of a
celebrated result by Szemer\'edie et al. on the spectrum of random graphs.
Next, we prove global convergence of alternating minimization with a good
initialization. Simulations suggest that the dependence of the sample size on
dimensionality $n$ is indeed tight.
Deep Epitomic Convolutional Neural Networks
Authors: George Papandreou
Date: 2014-06-10 00:00:00
Tasks: General Classification, image-classification, Image Classification

Deep convolutional neural networks have recently proven extremely competitive
in challenging image recognition tasks. This paper proposes the epitomic
convolution as a new building block for deep neural networks. An epitomic
convolution layer replaces a pair of consecutive convolution and max-pooling
layers found in standard deep convolutional neural networks. The main version
of the proposed model uses mini-epitomes in place of filters and computes
responses invariant to small translations by epitomic search instead of
max-pooling over image positions. The topographic version of the proposed model
uses large epitomes to learn filter maps organized in translational
topographies. We show that error back-propagation can successfully learn
multiple epitomic layers in a supervised fashion. The effectiveness of the
proposed method is assessed in image classification tasks on standard
benchmarks. Our experiments on Imagenet indicate improved recognition
performance compared to standard convolutional neural networks of similar
architecture. Our models pre-trained on Imagenet perform excellently on
Caltech-101. We also obtain competitive image classification results on the
small-image MNIST and CIFAR-10 datasets.
Learning Latent Variable Gaussian Graphical Models
Authors: Zhaoshi Meng, Brian Eriksson, Alfred O. Hero III
Date: 2014-06-10 00:00:00
Tasks: parameter estimation, Recommendation Systems

Gaussian graphical models (GGM) have been widely used in many
high-dimensional applications ranging from biological and financial data to
recommender systems. Sparsity in GGM plays a central role both statistically
and computationally. Unfortunately, real-world data often does not fit well to
sparse graphical models. In this paper, we focus on a family of latent variable
Gaussian graphical models (LVGGM), where the model is conditionally sparse
given latent variables, but marginally non-sparse. In LVGGM, the inverse
covariance matrix has a low-rank plus sparse structure, and can be learned in a
regularized maximum likelihood framework. We derive novel parameter estimation
error bounds for LVGGM under mild conditions in the high-dimensional setting.
These results complement the existing theory on the structural learning, and
open up new possibilities of using LVGGM for statistical inference.
The Effect of Social Learning on Individual Learning and Evolution
Authors: Chris Marriott, Jobran Chebib
Date: 2014-06-10 00:00:00

We consider the effects of social learning on the individual learning and
genetic evolution of a colony of artificial agents capable of genetic,
individual and social modes of adaptation. We confirm that there is strong
selection pressure to acquire traits of individual learning and social learning
when these are adaptive traits. We show that selection pressure for learning of
either kind can supress selection pressure for reproduction or greater fitness.
We show that social learning differs from individual learning in that it can
support a second evolutionary system that is decoupled from the biological
evolutionary system. This decoupling leads to an emergent interaction where
immature agents are more likely to engage in learning activities than mature
agents.
A Multiplicative Model for Learning Distributed Text-Based Attribute Representations
Authors: Ryan Kiros, Richard S. Zemel, Ruslan Salakhutdinov
Date: 2014-06-10 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Attribute, Authorship Attribution, Cross-Lingual Document Classification, Document Classification, General Classification, Sentence, Sentiment Analysis, Sentiment Classification, Text Generation, Word Embeddings, Word Similarity

In this paper we propose a general framework for learning distributed
representations of attributes: characteristics of text whose representations
can be jointly learned with word embeddings. Attributes can correspond to
document indicators (to learn sentence vectors), language indicators (to learn
distributed language representations), meta-data and side information (such as
the age, gender and industry of a blogger) or representations of authors. We
describe a third-order model where word context and attribute vectors interact
multiplicatively to predict the next word in a sequence. This leads to the
notion of conditional word similarity: how meanings of words change when
conditioned on different attributes. We perform several experimental tasks
including sentiment classification, cross-lingual document classification, and
blog authorship attribution. We also qualitatively evaluate conditional word
neighbours and attribute-conditioned text generation.
Augur: a Modeling Language for Data-Parallel Probabilistic Inference
Authors: Jean-Baptiste Tristan, Daniel Huang, Joseph Tassarotti, Adam Pocock, Stephen J. Green, Guy L. Steele Jr
Date: 2013-12-12 00:00:00
Tasks: Code Completion, GPU, Probabilistic Programming

It is time-consuming and error-prone to implement inference procedures for
each new probabilistic model. Probabilistic programming addresses this problem
by allowing a user to specify the model and having a compiler automatically
generate an inference procedure for it. For this approach to be practical, it
is important to generate inference code that has reasonable performance. In
this paper, we present a probabilistic programming language and compiler for
Bayesian networks designed to make effective use of data-parallel architectures
such as GPUs. Our language is fully integrated within the Scala programming
language and benefits from tools such as IDE support, type-checking, and code
completion. We show that the compiler can generate data-parallel inference code
scalable to thousands of GPU cores by making use of the conditional
independence relationships in the Bayesian network.
Conceptors: an easy introduction
Authors: Herbert Jaeger
Date: 2014-06-10 00:00:00
Tasks: Diversity

Conceptors provide an elementary neuro-computational mechanism which sheds a
fresh and unifying light on a diversity of cognitive phenomena. A number of
demanding learning and processing tasks can be solved with unprecedented ease,
robustness and accuracy. Some of these tasks were impossible to solve before.
This entirely informal paper introduces the basic principles of conceptors and
highlights some of their usages.
Generative Adversarial Networks
Authors: Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio
Date: 2014-06-10 00:00:00
Proceeding: Proceedings of the 27th International Conference on Neural Information Processing Systems 2014 12
Tasks: Graph Property Prediction, Super-Resolution, Time-Series Few-Shot Learning with Heterogeneous Channels

We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. The training procedure for G is to maximize the probability of D making a mistake. This framework corresponds to a minimax two-player game. In the space of arbitrary functions G and D, a unique solution exists, with G recovering the training data distribution and D equal to 1/2 everywhere. In the case where G and D are defined by multilayer perceptrons, the entire system can be trained with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of samples. Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples.
Learning with Cross-Kernels and Ideal PCA
Authors: Franz J. Király, Martin Kreuzer, Louis Theran
Date: 2014-06-10 00:00:00

We describe how cross-kernel matrices, that is, kernel matrices between the
data and a custom chosen set of `feature spanning points' can be used for
learning. The main potential of cross-kernels lies in the fact that (a) only
one side of the matrix scales with the number of data points, and (b)
cross-kernels, as opposed to the usual kernel matrices, can be used to certify
for the data manifold. Our theoretical framework, which is based on a duality
involving the feature space and vanishing ideals, indicates that cross-kernels
have the potential to be used for any kind of kernel learning. We present a
novel algorithm, Ideal PCA (IPCA), which cross-kernelizes PCA. We demonstrate
on real and synthetic data that IPCA allows to (a) obtain PCA-like features
faster and (b) to extract novel and empirically validated features certifying
for the data manifold.
Equivalence of Learning Algorithms
Authors: Julien Audiffren, Hachem Kadri
Date: 2014-06-10 00:00:00
Tasks: BIG-bench Machine Learning, regression

The purpose of this paper is to introduce a concept of equivalence between
machine learning algorithms. We define two notions of algorithmic equivalence,
namely, weak and strong equivalence. These notions are of paramount importance
for identifying when learning prop erties from one learning algorithm can be
transferred to another. Using regularized kernel machines as a case study, we
illustrate the importance of the introduced equivalence concept by analyzing
the relation between kernel ridge regression (KRR) and m-power regularized
least squares regression (M-RLSR) algorithms.
Graph Approximation and Clustering on a Budget
Authors: Ethan Fetaya, Ohad Shamir, Shimon Ullman
Date: 2014-06-10 00:00:00
Tasks: Clustering

We consider the problem of learning from a similarity matrix (such as
spectral clustering and lowd imensional embedding), when computing pairwise
similarities are costly, and only a limited number of entries can be observed.
We provide a theoretical analysis using standard notions of graph
approximation, significantly generalizing previous results (which focused on
spectral clustering with two clusters). We also propose a new algorithmic
approach based on adaptive sampling, which experimentally matches or improves
on previous methods, while being considerably more general and computationally
cheaper.
Identification of Orchid Species Using Content-Based Flower Image Retrieval
Authors: D. H. Apriyanti, A. A. Arymurthy, L. T. Handoko
Date: 2014-06-10 00:00:00
Tasks: Image Retrieval, Retrieval

In this paper, we developed the system for recognizing the orchid species by
using the images of flower. We used MSRM (Maximal Similarity based on Region
Merging) method for segmenting the flower object from the background and
extracting the shape feature such as the distance from the edge to the centroid
point of the flower, aspect ratio, roundness, moment invariant, fractal
dimension and also extract color feature. We used HSV color feature with
ignoring the V value. To retrieve the image, we used Support Vector Machine
(SVM) method. Orchid is a unique flower. It has a part of flower called lip
(labellum) that distinguishes it from other flowers even from other types of
orchids. Thus, in this paper, we proposed to do feature extraction not only on
flower region but also on lip (labellum) region. The result shows that our
proposed method can increase the accuracy value of content based flower image
retrieval for orchid species up to $\pm$ 14%. The most dominant feature is
Centroid Contour Distance, Moment Invariant and HSV Color. The system accuracy
is 85,33% in validation phase and 79,33% in testing phase.
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
Authors: Yann Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, Yoshua Bengio
Date: 2014-06-10 00:00:00
Proceeding: NeurIPS 2014 12

A central challenge to many fields of science and engineering involves
minimizing non-convex error functions over continuous, high dimensional spaces.
Gradient descent or quasi-Newton methods are almost ubiquitously used to
perform such minimizations, and it is often thought that a main source of
difficulty for these local methods to find the global minimum is the
proliferation of local minima with much higher error than the global minimum.
Here we argue, based on results from statistical physics, random matrix theory,
neural network theory, and empirical evidence, that a deeper and more profound
difficulty originates from the proliferation of saddle points, not local
minima, especially in high dimensional problems of practical interest. Such
saddle points are surrounded by high error plateaus that can dramatically slow
down learning, and give the illusory impression of the existence of a local
minimum. Motivated by these arguments, we propose a new approach to
second-order optimization, the saddle-free Newton method, that can rapidly
escape high dimensional saddle points, unlike gradient descent and quasi-Newton
methods. We apply this algorithm to deep or recurrent neural network training,
and provide numerical evidence for its superior optimization performance.
A Flexible Fitness Function for Community Detection in Complex Networks
Authors: Fabricio Olivetti de Franca, Guilherme Palermo Coelho
Date: 2014-06-10 00:00:00
Tasks: Community Detection

Most community detection algorithms from the literature work as optimization
tools that minimize a given \textit{fitness function}, while assuming that each
node belongs to a single community. Since there is no hard concept of what a
community is, most proposed fitness functions focus on a particular definition.
As such, these functions do not always lead to partitions that correspond to
those observed in practice. This paper proposes a new flexible fitness function
that allows the identification of communities with distinct characteristics.
Such flexibility was evaluated through the adoption of an immune-inspired
optimization algorithm, named cob-aiNet[C], to identify both disjoint and
overlapping communities in a set of benchmark networks. The results have shown
that the obtained partitions are much closer to the ground-truth than those
obtained by the optimization of the modularity function.
Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
Authors: José Miguel Hernández-Lobato, Matthew W. Hoffman, Zoubin Ghahramani
Date: 2014-06-10 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Bayesian Optimization, global-optimization

We propose a novel information-theoretic approach for Bayesian optimization
called Predictive Entropy Search (PES). At each iteration, PES selects the next
evaluation point that maximizes the expected information gained with respect to
the global maximum. PES codifies this intractable acquisition function in terms
of the expected reduction in the differential entropy of the predictive
distribution. This reformulation allows PES to obtain approximations that are
both more accurate and efficient than other alternatives such as Entropy Search
(ES). Furthermore, PES can easily perform a fully Bayesian treatment of the
model hyperparameters while ES cannot. We evaluate PES in both synthetic and
real-world applications, including optimization problems in machine learning,
finance, biotechnology, and robotics. We show that the increased accuracy of
PES leads to significant gains in optimization performance.
Maximizing Diversity for Multimodal Optimization
Authors: Fabricio Olivetti de Franca
Date: 2014-06-10 00:00:00
Tasks: Diversity

Most multimodal optimization algorithms use the so called \textit{niching
methods}~\cite{mahfoud1995niching} in order to promote diversity during
optimization, while others, like \textit{Artificial Immune
Systems}~\cite{de2010conceptual} try to find multiple solutions as its main
objective. One of such algorithms, called
\textit{dopt-aiNet}~\cite{de2005artificial}, introduced the Line Distance that
measures the distance between two solutions regarding their basis of
attraction. In this short abstract I propose the use of the Line Distance
measure as the main objective-function in order to locate multiple optima at
once in a population.