pwc-archive/papers-with-abstracts huggingface dataset:
Implicitly Constrained Semi-Supervised Least Squares Classification
Authors: Jesse H. Krijthe, Marco Loog
Date: 2015-07-24 00:00:00
Tasks: Classification, General Classification

We introduce a novel semi-supervised version of the least squares classifier.
This implicitly constrained least squares (ICLS) classifier minimizes the
squared loss on the labeled data among the set of parameters implied by all
possible labelings of the unlabeled data. Unlike other discriminative
semi-supervised methods, our approach does not introduce explicit additional
assumptions into the objective function, but leverages implicit assumptions
already present in the choice of the supervised least squares classifier. We
show this approach can be formulated as a quadratic programming problem and its
solution can be found using a simple gradient descent procedure. We prove that,
in a certain way, our method never leads to performance worse than the
supervised classifier. Experimental results corroborate this theoretical result
in the multidimensional case on benchmark datasets, also in terms of the error
rate.
Clustering of Modal Valued Symbolic Data
Authors: Vladimir Batagelj, Nataša Kejžar, Simona Korenjak-Černe
Date: 2015-07-23 00:00:00
Tasks: Clustering

Symbolic Data Analysis is based on special descriptions of data - symbolic
objects (SO). Such descriptions preserve more detailed information about units
and their clusters than the usual representations with mean values. A special
kind of symbolic object is a representation with frequency or probability
distributions (modal values). This representation enables us to consider in the
clustering process the variables of all measurement types at the same time. In
the paper a clustering criterion function for SOs is proposed such that the
representative of each cluster is again composed of distributions of variables'
values over the cluster. The corresponding leaders clustering method is based
on this result. It is also shown that for the corresponding agglomerative
hierarchical method a generalized Ward's formula holds. Both methods are
compatible - they are solving the same clustering optimization problem. The
leaders method efficiently solves clustering problems with large number of
units; while the agglomerative method can be applied alone on the smaller data
set, or it could be applied on leaders, obtained with compatible
nonhierarchical clustering method. Such a combination of two compatible methods
enables us to decide upon the right number of clusters on the basis of the
corresponding dendrogram. The proposed methods were applied on different data
sets. In the paper, some results of clustering of ESS data are presented.
PCA with Gaussian perturbations
Authors: Wojciech Kotłowski, Manfred K. Warmuth
Date: 2015-06-16 00:00:00

Most of machine learning deals with vector parameters. Ideally we would like
to take higher order information into account and make use of matrix or even
tensor parameters. However the resulting algorithms are usually inefficient.
Here we address on-line learning with matrix parameters. It is often easy to
obtain online algorithm with good generalization performance if you
eigendecompose the current parameter matrix in each trial (at a cost of
$O(n^3)$ per trial). Ideally we want to avoid the decompositions and spend
$O(n^2)$ per trial, i.e. linear time in the size of the matrix data. There is a
core trade-off between the running time and the generalization performance,
here measured by the regret of the on-line algorithm (total gain of the best
off-line predictor minus the total gain of the on-line algorithm). We focus on
the key matrix problem of rank $k$ Principal Component Analysis in
$\mathbb{R}^n$ where $k \ll n$. There are $O(n^3)$ algorithms that achieve the
optimum regret but require eigendecompositions. We develop a simple algorithm
that needs $O(kn^2)$ per trial whose regret is off by a small factor of
$O(n^{1/4})$. The algorithm is based on the Follow the Perturbed Leader
paradigm. It replaces full eigendecompositions at each trial by the problem
finding $k$ principal components of the current covariance matrix that is
perturbed by Gaussian noise.
Optimal Learning Rates for Localized SVMs
Authors: Mona Eberts, Ingo Steinwart
Date: 2015-07-23 00:00:00
Tasks: regression

One of the limiting factors of using support vector machines (SVMs) in large
scale applications are their super-linear computational requirements in terms
of the number of training samples. To address this issue, several approaches
that train SVMs on many small chunks of large data sets separately have been
proposed in the literature. So far, however, almost all these approaches have
only been empirically investigated. In addition, their motivation was always
based on computational requirements. In this work, we consider a localized SVM
approach based upon a partition of the input space. For this local SVM, we
derive a general oracle inequality. Then we apply this oracle inequality to
least squares regression using Gaussian kernels and deduce local learning rates
that are essentially minimax optimal under some standard smoothness assumptions
on the regression function. This gives the first motivation for using local
SVMs that is not based on computational requirements but on theoretical
predictions on the generalization performance. We further introduce a
data-dependent parameter selection method for our local SVM approach and show
that this method achieves the same learning rates as before. Finally, we
present some larger scale experiments for our localized SVM showing that it
achieves essentially the same test performance as a global SVM for a fraction
of the computational requirements. In addition, it turns out that the
computational requirements for the local SVMs are similar to those of a vanilla
random chunk approach, while the achieved test errors are significantly better.
Multi-scale exploration of convex functions and bandit convex optimization
Authors: Sébastien Bubeck, Ronen Eldan
Date: 2015-07-23 00:00:00

We construct a new map from a convex function to a distribution on its
domain, with the property that this distribution is a multi-scale exploration
of the function. We use this map to solve a decade-old open problem in
adversarial bandit convex optimization by showing that the minimax regret for
this problem is $\tilde{O}(\mathrm{poly}(n) \sqrt{T})$, where $n$ is the
dimension and $T$ the number of rounds. This bound is obtained by studying the
dual Bayesian maximin regret via the information ratio analysis of Russo and
Van Roy, and then using the multi-scale exploration to solve the Bayesian
problem.
ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks
Authors: Francesco Visin, Kyle Kastner, Kyunghyun Cho, Matteo Matteucci, Aaron Courville, Yoshua Bengio
Date: 2015-05-03 00:00:00
Tasks: Image Classification, Object Recognition

In this paper, we propose a deep neural network architecture for object
recognition based on recurrent neural networks. The proposed network, called
ReNet, replaces the ubiquitous convolution+pooling layer of the deep
convolutional neural network with four recurrent neural networks that sweep
horizontally and vertically in both directions across the image. We evaluate
the proposed ReNet on three widely-used benchmark datasets; MNIST, CIFAR-10 and
SVHN. The result suggests that ReNet is a viable alternative to the deep
convolutional neural network, and that further investigation is needed.
Learning Weak Constraints in Answer Set Programming
Authors: Mark Law, Alessandra Russo, Krysia Broda
Date: 2015-07-23 00:00:00
Tasks: Inductive logic programming, Scheduling

This paper contributes to the area of inductive logic programming by
presenting a new learning framework that allows the learning of weak
constraints in Answer Set Programming (ASP). The framework, called Learning
from Ordered Answer Sets, generalises our previous work on learning ASP
programs without weak constraints, by considering a new notion of examples as
ordered pairs of partial answer sets that exemplify which answer sets of a
learned hypothesis (together with a given background knowledge) are preferred
to others. In this new learning task inductive solutions are searched within a
hypothesis space of normal rules, choice rules, and hard and weak constraints.
We propose a new algorithm, ILASP2, which is sound and complete with respect to
our new learning framework. We investigate its applicability to learning
preferences in an interview scheduling problem and also demonstrate that when
restricted to the task of learning ASP programs without weak constraints,
ILASP2 can be much more efficient than our previously proposed system.
Manitest: Are classifiers really invariant?
Authors: Alhussein Fawzi, Pascal Frossard
Date: 2015-07-23 00:00:00
Tasks: Data Augmentation

Invariance to geometric transformations is a highly desirable property of
automatic classifiers in many image recognition tasks. Nevertheless, it is
unclear to which extent state-of-the-art classifiers are invariant to basic
transformations such as rotations and translations. This is mainly due to the
lack of general methods that properly measure such an invariance. In this
paper, we propose a rigorous and systematic approach for quantifying the
invariance to geometric transformations of any classifier. Our key idea is to
cast the problem of assessing a classifier's invariance as the computation of
geodesics along the manifold of transformed images. We propose the Manitest
method, built on the efficient Fast Marching algorithm to compute the
invariance of classifiers. Our new method quantifies in particular the
importance of data augmentation for learning invariance from data, and the
increased invariance of convolutional neural networks with depth. We foresee
that the proposed generic tool for measuring invariance to a large class of
geometric transformations and arbitrary classifiers will have many applications
for evaluating and comparing classifiers based on their invariance, and help
improving the invariance of existing classifiers.
Dynamic Matrix Factorization with Priors on Unknown Values
Authors: Robin Devooght, Nicolas Kourtellis, Amin Mantrach
Date: 2015-07-23 00:00:00
Tasks: Collaborative Filtering

Advanced and effective collaborative filtering methods based on explicit
feedback assume that unknown ratings do not follow the same model as the
observed ones (\emph{not missing at random}). In this work, we build on this
assumption, and introduce a novel dynamic matrix factorization framework that
allows to set an explicit prior on unknown values. When new ratings, users, or
items enter the system, we can update the factorization in time independent of
the size of data (number of users, items and ratings). Hence, we can quickly
recommend items even to very recent users. We test our methods on three large
datasets, including two very sparse ones, in static and dynamic conditions. In
each case, we outrank state-of-the-art matrix factorization methods that do not
use a prior on unknown ratings.
Deep Fishing: Gradient Features from Deep Nets
Authors: Albert Gordo, Adrien Gaidon, Florent Perronnin
Date: 2015-07-23 00:00:00

Convolutional Networks (ConvNets) have recently improved image recognition
performance thanks to end-to-end learning of deep feed-forward models from raw
pixels. Deep learning is a marked departure from the previous state of the art,
the Fisher Vector (FV), which relied on gradient-based encoding of local
hand-crafted features. In this paper, we discuss a novel connection between
these two approaches. First, we show that one can derive gradient
representations from ConvNets in a similar fashion to the FV. Second, we show
that this gradient representation actually corresponds to a structured matrix
that allows for efficient similarity computation. We experimentally study the
benefits of transferring this representation over the outputs of ConvNet
layers, and find consistent improvements on the Pascal VOC 2007 and 2012
datasets.
Arbitrariness of peer review: A Bayesian analysis of the NIPS experiment
Authors: Olivier Francois
Date: 2015-07-23 00:00:00

The principle of peer review is central to the evaluation of research, by
ensuring that only high-quality items are funded or published. But peer review
has also received criticism, as the selection of reviewers may introduce biases
in the system. In 2014, the organizers of the ``Neural Information Processing
Systems\rq\rq{} conference conducted an experiment in which $10\%$ of submitted
manuscripts (166 items) went through the review process twice. Arbitrariness
was measured as the conditional probability for an accepted submission to get
rejected if examined by the second committee. This number was equal to $60\%$,
for a total acceptance rate equal to $22.5\%$. Here we present a Bayesian
analysis of those two numbers, by introducing a hidden parameter which measures
the probability that a submission meets basic quality criteria. The standard
quality criteria usually include novelty, clarity, reproducibility, correctness
and no form of misconduct, and are met by a large proportions of submitted
items. The Bayesian estimate for the hidden parameter was equal to $56\%$
($95\%$CI: $ I = (0.34, 0.83)$), and had a clear interpretation. The result
suggested the total acceptance rate should be increased in order to decrease
arbitrariness estimates in future review processes.
Multi-Target Tracking with Time-Varying Clutter Rate and Detection Profile: Application to Time-lapse Cell Microscopy Sequences
Authors: Seyed Hamid Rezatofighi, Stephen Gould, Ba Tuong Vo, Ba-Ngu Vo, Katarina Mele, Richard Hartley
Date: 2015-07-23 00:00:00

Quantitative analysis of the dynamics of tiny cellular and sub-cellular
structures, known as particles, in time-lapse cell microscopy sequences
requires the development of a reliable multi-target tracking method capable of
tracking numerous similar targets in the presence of high levels of noise, high
target density, complex motion patterns and intricate interactions. In this
paper, we propose a framework for tracking these structures based on the random
finite set Bayesian filtering framework. We focus on challenging biological
applications where image characteristics such as noise and background intensity
change during the acquisition process. Under these conditions, detection
methods usually fail to detect all particles and are often followed by missed
detections and many spurious measurements with unknown and time-varying rates.
To deal with this, we propose a bootstrap filter composed of an estimator and a
tracker. The estimator adaptively estimates the required meta parameters for
the tracker such as clutter rate and the detection probability of the targets,
while the tracker estimates the state of the targets. Our results show that the
proposed approach can outperform state-of-the-art particle trackers on both
synthetic and real data in this regime.
Combining Models of Approximation with Partial Learning
Authors: Zi-Yuan Gao, Frank Stephan, Sandra Zilles
Date: 2015-07-05 00:00:00
Tasks: Object

In Gold's framework of inductive inference, the model of partial learning
requires the learner to output exactly one correct index for the target object
and only the target object infinitely often. Since infinitely many of the
learner's hypotheses may be incorrect, it is not obvious whether a partial
learner can be modifed to "approximate" the target object.
  Fulk and Jain (Approximate inference and scientific method. Information and
Computation 114(2):179--191, 1994) introduced a model of approximate learning
of recursive functions. The present work extends their research and solves an
open problem of Fulk and Jain by showing that there is a learner which
approximates and partially identifies every recursive function by outputting a
sequence of hypotheses which, in addition, are also almost all finite variants
of the target function.
  The subsequent study is dedicated to the question how these findings
generalise to the learning of r.e. languages from positive data. Here three
variants of approximate learning will be introduced and investigated with
respect to the question whether they can be combined with partial learning.
Following the line of Fulk and Jain's research, further investigations provide
conditions under which partial language learners can eventually output only
finite variants of the target language. The combinabilities of other partial
learning criteria will also be briefly studied.
Targeting Ultimate Accuracy: Face Recognition via Deep Embedding
Authors: Jingtuo Liu, Yafeng Deng, Tao Bai, Zhengping Wei, Chang Huang
Date: 2015-06-24 00:00:00
Tasks: Face Recognition, Face Verification, Metric Learning

Face Recognition has been studied for many decades. As opposed to traditional
hand-crafted features such as LBP and HOG, much more sophisticated features can
be learned automatically by deep learning methods in a data-driven way. In this
paper, we propose a two-stage approach that combines a multi-patch deep CNN and
deep metric learning, which extracts low dimensional but very discriminative
features for face verification and recognition. Experiments show that this
method outperforms other state-of-the-art methods on LFW dataset, achieving
99.77% pair-wise verification accuracy and significantly better accuracy under
other two more practical protocols. This paper also discusses the importance of
data size and the number of patches, showing a clear path to practical
high-performance face recognition systems in real world.
Forward - Backward Greedy Algorithms for Atomic Norm Regularization
Authors: Nikhil Rao, Parikshit Shah, Stephen Wright
Date: 2014-04-23 00:00:00

In many signal processing applications, the aim is to reconstruct a signal
that has a simple representation with respect to a certain basis or frame.
Fundamental elements of the basis known as "atoms" allow us to define "atomic
norms" that can be used to formulate convex regularizations for the
reconstruction problem. Efficient algorithms are available to solve these
formulations in certain special cases, but an approach that works well for
general atomic norms, both in terms of speed and reconstruction accuracy,
remains to be found. This paper describes an optimization algorithm called
CoGEnT that produces solutions with succinct atomic representations for
reconstruction problems, generally formulated with atomic-norm constraints.
CoGEnT combines a greedy selection scheme based on the conditional gradient
approach with a backward (or "truncation") step that exploits the quadratic
nature of the objective to reduce the basis size. We establish convergence
properties and validate the algorithm via extensive numerical experiments on a
suite of signal processing applications. Our algorithm and analysis also allow
for inexact forward steps and for occasional enhancements of the current
representation to be performed. CoGEnT can outperform the basic conditional
gradient method, and indeed many methods that are tailored to specific
applications, when the enhancement and truncation steps are defined
appropriately. We also introduce several novel applications that are enabled by
the atomic-norm framework, including tensor completion, moment problems in
signal processing, and graph deconvolution.
Evaluation of Spectral Learning for the Identification of Hidden Markov Models
Authors: Robert Mattila, Cristian R. Rojas, Bo Wahlberg
Date: 2015-07-22 00:00:00
Tasks: Open-Ended Question Answering, Time Series, Time Series Analysis

Hidden Markov models have successfully been applied as models of discrete
time series in many fields. Often, when applied in practice, the parameters of
these models have to be estimated. The currently predominating identification
methods, such as maximum-likelihood estimation and especially
expectation-maximization, are iterative and prone to have problems with local
minima. A non-iterative method employing a spectral subspace-like approach has
recently been proposed in the machine learning literature. This paper evaluates
the performance of this algorithm, and compares it to the performance of the
expectation-maximization algorithm, on a number of numerical examples. We find
that the performance is mixed; it successfully identifies some systems with
relatively few available observations, but fails completely for some systems
even when a large amount of observations is available. An open question is how
this discrepancy can be explained. We provide some indications that it could be
related to how well-conditioned some system parameters are.
Part Localization using Multi-Proposal Consensus for Fine-Grained Categorization
Authors: Kevin J. Shih, Arun Mallya, Saurabh Singh, Derek Hoiem
Date: 2015-07-22 00:00:00
Tasks: General Classification, Spatial Reasoning

We present a simple deep learning framework to simultaneously predict
keypoint locations and their respective visibilities and use those to achieve
state-of-the-art performance for fine-grained classification. We show that by
conditioning the predictions on object proposals with sufficient image support,
our method can do well without complicated spatial reasoning. Instead,
inference methods with robustness to outliers, yield state-of-the-art for
keypoint localization. We demonstrate the effectiveness of our accurate
keypoint localization and visibility prediction on the fine-grained bird
recognition task with and without ground truth bird bounding boxes, and
outperform existing state-of-the-art methods by over 2%.
Reservoir Characterization: A Machine Learning Approach
Authors: Soumi Chaki
Date: 2015-06-15 00:00:00
Tasks: BIG-bench Machine Learning, Sand

Reservoir Characterization (RC) can be defined as the act of building a
reservoir model that incorporates all the characteristics of the reservoir that
are pertinent to its ability to store hydrocarbons and also to produce them.It
is a difficult problem due to non-linear and heterogeneous subsurface
properties and associated with a number of complex tasks such as data fusion,
data mining, formulation of the knowledge base, and handling of the
uncertainty.This present work describes the development of algorithms to obtain
the functional relationships between predictor seismic attributes and target
lithological properties. Seismic attributes are available over a study area
with lower vertical resolution. Conversely, well logs and lithological
properties are available only at specific well locations in a study area with
high vertical resolution.Sand fraction, which represents per unit sand volume
within the rock, has a balanced distribution between zero to unity.The thesis
addresses the issues of handling the information content mismatch between
predictor and target variables and proposes regularization of target property
prior to building a prediction model.In this thesis, two Artificial Neural
Network (ANN) based frameworks are proposed to model sand fraction from
multiple seismic attributes without and with well tops information
respectively. The performances of the frameworks are quantified in terms of
Correlation Coefficient, Root Mean Square Error, Absolute Error Mean, etc.
Incremental Variational Inference for Latent Dirichlet Allocation
Authors: Cedric Archambeau, Beyza Ermis
Date: 2015-07-17 00:00:00
Tasks: Variational Inference

We introduce incremental variational inference and apply it to latent
Dirichlet allocation (LDA). Incremental variational inference is inspired by
incremental EM and provides an alternative to stochastic variational inference.
Incremental LDA can process massive document collections, does not require to
set a learning rate, converges faster to a local optimum of the variational
bound and enjoys the attractive property of monotonically increasing it. We
study the performance of incremental LDA on large benchmark data sets. We
further introduce a stochastic approximation of incremental variational
inference which extends to the asynchronous distributed setting. The resulting
distributed algorithm achieves comparable performance as single host
incremental variational inference, but with a significant speed-up.