pwc-archive/papers-with-abstracts huggingface dataset:
Sparse Additive Model using Symmetric Nonnegative Definite Smoothers
Authors: Yan Li
Date: 2014-09-08 00:00:00
Tasks: model, Variable Selection

We introduce a new algorithm, called adaptive sparse backfitting algorithm,
for solving high dimensional Sparse Additive Model (SpAM) utilizing symmetric,
non-negative definite smoothers. Unlike the previous sparse backfitting
algorithm, our method is essentially a block coordinate descent algorithm that
guarantees to converge to the optimal solution. It bridges the gap between the
population backfitting algorithm and that of the data version. We also prove
variable selection consistency under suitable conditions. Numerical studies on
both synthesis and real data are conducted to show that adaptive sparse
backfitting algorithm outperforms previous sparse backfitting algorithm in
fitting and predicting high dimensional nonparametric models.
Bayesian Network Structure Learning Using Quantum Annealing
Authors: Bryan O'Gorman, Alejandro Perdomo-Ortiz, Ryan Babbush, Alan Aspuru-Guzik, Vadim Smelyanskiy
Date: 2014-07-15 00:00:00

We introduce a method for the problem of learning the structure of a Bayesian
network using the quantum adiabatic algorithm. We do so by introducing an
efficient reformulation of a standard posterior-probability scoring function on
graphs as a pseudo-Boolean function, which is equivalent to a system of 2-body
Ising spins, as well as suitable penalty terms for enforcing the constraints
necessary for the reformulation; our proposed method requires $\mathcal O(n^2)$
qubits for $n$ Bayesian network variables. Furthermore, we prove lower bounds
on the necessary weighting of these penalty terms. The logical structure
resulting from the mapping has the appealing property that it is
instance-independent for a given number of Bayesian network variables, as well
as being independent of the number of data cases.
Deep Directed Generative Autoencoders
Authors: Sherjil Ozair, Yoshua Bengio
Date: 2014-10-02 00:00:00
Tasks: Decoder

For discrete data, the likelihood $P(x)$ can be rewritten exactly and
parametrized into $P(X = x) = P(X = x | H = f(x)) P(H = f(x))$ if $P(X | H)$
has enough capacity to put no probability mass on any $x'$ for which $f(x')\neq
f(x)$, where $f(\cdot)$ is a deterministic discrete function. The log of the
first factor gives rise to the log-likelihood reconstruction error of an
autoencoder with $f(\cdot)$ as the encoder and $P(X|H)$ as the (probabilistic)
decoder. The log of the second term can be seen as a regularizer on the encoded
activations $h=f(x)$, e.g., as in sparse autoencoders. Both encoder and decoder
can be represented by a deep neural network and trained to maximize the average
of the optimal log-likelihood $\log p(x)$. The objective is to learn an encoder
$f(\cdot)$ that maps $X$ to $f(X)$ that has a much simpler distribution than
$X$ itself, estimated by $P(H)$. This "flattens the manifold" or concentrates
probability mass in a smaller number of (relevant) dimensions over which the
distribution factorizes. Generating samples from the model is straightforward
using ancestral sampling. One challenge is that regular back-propagation cannot
be used to obtain the gradient on the parameters of the encoder, but we find
that using the straight-through estimator works well here. We also find that
although optimizing a single level of such architecture may be difficult, much
better results can be obtained by pre-training and stacking them, gradually
transforming the data distribution into one that is more easily captured by a
simple parametric model.
Dictionary learning for fast classification based on soft-thresholding
Authors: Alhussein Fawzi, Mike Davies, Pascal Frossard
Date: 2014-02-09 00:00:00
Tasks: Classification, Dictionary Learning, General Classification

Classifiers based on sparse representations have recently been shown to
provide excellent results in many visual recognition and classification tasks.
However, the high cost of computing sparse representations at test time is a
major obstacle that limits the applicability of these methods in large-scale
problems, or in scenarios where computational power is restricted. We consider
in this paper a simple yet efficient alternative to sparse coding for feature
extraction. We study a classification scheme that applies the soft-thresholding
nonlinear mapping in a dictionary, followed by a linear classifier. A novel
supervised dictionary learning algorithm tailored for this low complexity
classification architecture is proposed. The dictionary learning problem, which
jointly learns the dictionary and linear classifier, is cast as a difference of
convex (DC) program and solved efficiently with an iterative DC solver. We
conduct experiments on several datasets, and show that our learning algorithm
that leverages the structure of the classification problem outperforms generic
learning procedures. Our simple classifier based on soft-thresholding also
competes with the recent sparse coding classifiers, when the dictionary is
learned appropriately. The adopted classification scheme further requires less
computational time at the testing stage, compared to other classifiers. The
proposed scheme shows the potential of the adequately trained soft-thresholding
mapping for classification and paves the way towards the development of very
efficient classification methods for vision problems.
A probabilistic evolutionary optimization approach to compute quasiparticle braids
Authors: Roberto Santana, Ross B. McDonald, Helmut G. Katzgraber
Date: 2014-10-02 00:00:00

Topological quantum computing is an alternative framework for avoiding the
quantum decoherence problem in quantum computation. The problem of executing a
gate in this framework can be posed as the problem of braiding quasiparticles.
Because these are not Abelian, the problem can be reduced to finding an optimal
product of braid generators where the optimality is defined in terms of the
gate approximation and the braid's length. In this paper we propose the use of
different variants of estimation of distribution algorithms to deal with the
problem. Furthermore, we investigate how the regularities of the braid
optimization problem can be translated into statistical regularities by means
of the Boltzmann distribution. We show that our best algorithm is able to
produce many solutions that approximates the target gate with an accuracy in
the order of $10^{-6}$, and have lengths up to 9 times shorter than those
expected from braids of the same accuracy obtained with other methods.
Mapping Energy Landscapes of Non-Convex Learning Problems
Authors: Maria Pavlovskaia, Kewei Tu, Song-Chun Zhu
Date: 2014-10-02 00:00:00
Tasks: Clustering

In many statistical learning problems, the target functions to be optimized
are highly non-convex in various model spaces and thus are difficult to
analyze. In this paper, we compute \emph{Energy Landscape Maps} (ELMs) which
characterize and visualize an energy function with a tree structure, in which
each leaf node represents a local minimum and each non-leaf node represents the
barrier between adjacent energy basins. The ELM also associates each node with
the estimated probability mass and volume for the corresponding energy basin.
We construct ELMs by adopting the generalized Wang-Landau algorithm and
multi-domain sampler that simulates a Markov chain traversing the model space
by dynamically reweighting the energy function. We construct ELMs in the model
space for two classic statistical learning problems: i) clustering with
Gaussian mixture models or Bernoulli templates; and ii) bi-clustering. We
propose a way to measure the difficulties (or complexity) of these learning
problems and study how various conditions affect the landscape complexity, such
as separability of the clusters, the number of examples, and the level of
supervision; and we also visualize the behaviors of different algorithms, such
as K-mean, EM, two-step EM and Swendsen-Wang cuts, in the energy landscapes.
Stochastic Discriminative EM
Authors: Andres R. Masegosa
Date: 2014-10-02 00:00:00
Tasks: text-classification, Text Classification

Stochastic discriminative EM (sdEM) is an online-EM-type algorithm for
discriminative training of probabilistic generative models belonging to the
exponential family. In this work, we introduce and justify this algorithm as a
stochastic natural gradient descent method, i.e. a method which accounts for
the information geometry in the parameter space of the statistical model. We
show how this learning algorithm can be used to train probabilistic generative
models by minimizing different discriminative loss functions, such as the
negative conditional log-likelihood and the Hinge loss. The resulting models
trained by sdEM are always generative (i.e. they define a joint probability
distribution) and, in consequence, allows to deal with missing data and latent
variables in a principled way either when being learned or when making
predictions. The performance of this method is illustrated by several text
classification problems for which a multinomial naive Bayes and a latent
Dirichlet allocation based classifier are learned using different
discriminative loss functions.
Deep Sequential Neural Network
Authors: Ludovic Denoyer, Patrick Gallinari
Date: 2014-10-02 00:00:00
Tasks: Reinforcement Learning

Neural Networks sequentially build high-level features through their
successive layers. We propose here a new neural network model where each layer
is associated with a set of candidate mappings. When an input is processed, at
each layer, one mapping among these candidates is selected according to a
sequential decision process. The resulting model is structured according to a
DAG like architecture, so that a path from the root to a leaf node defines a
sequence of transformations. Instead of considering global transformations,
like in classical multilayer networks, this model allows us for learning a set
of local transformations. It is thus able to process data with different
characteristics through specific sequences of such local transformations,
increasing the expression power of this model w.r.t a classical multilayered
network. The learning algorithm is inspired from policy gradient techniques
coming from the reinforcement learning domain and is used here instead of the
classical back-propagation based gradient descent techniques. Experiments on
different datasets show the relevance of this approach.
Generating functionals for computational intelligence: the Fisher information as an objective function for self-limiting Hebbian learning rules
Authors: Rodrigo Echeveste, Claudius Gros
Date: 2014-10-02 00:00:00

Generating functionals may guide the evolution of a dynamical system and
constitute a possible route for handling the complexity of neural networks as
relevant for computational intelligence. We propose and explore a new objective
function, which allows to obtain plasticity rules for the afferent synaptic
weights. The adaption rules are Hebbian, self-limiting, and result from the
minimization of the Fisher information with respect to the synaptic flux. We
perform a series of simulations examining the behavior of the new learning
rules in various circumstances. The vector of synaptic weights aligns with the
principal direction of input activities, whenever one is present. A linear
discrimination is performed when there are two or more principal directions;
directions having bimodal firing-rate distributions, being characterized by a
negative excess kurtosis, are preferred. We find robust performance and full
homeostatic adaption of the synaptic weights results as a by-product of the
synaptic flux minimization. This self-limiting behavior allows for stable
online learning for arbitrary durations. The neuron acquires new information
when the statistics of input activities is changed at a certain point of the
simulation, showing however, a distinct resilience to unlearn previously
acquired knowledge. Learning is fast when starting with randomly drawn synaptic
weights and substantially slower when the synaptic weights are already fully
adapted.
Recognition of Handwritten Bangla Basic Characters and Digits using Convex Hull based Feature Set
Authors: Nibaran Das, Sandip Pramanik, Subhadip Basu, Punam Kumar Saha, Ram Sarkar, Mahantapas Kundu, Mita Nasipuri
Date: 2014-10-02 00:00:00

In dealing with the problem of recognition of handwritten character patterns
of varying shapes and sizes, selection of a proper feature set is important to
achieve high recognition performance. The current research aims to evaluate the
performance of the convex hull based feature set, i.e. 125 features in all
computed over different bays attributes of the convex hull of a pattern, for
effective recognition of isolated handwritten Bangla basic characters and
digits. On experimentation with a database of 10000 samples, the maximum
recognition rate of 76.86% is observed for handwritten Bangla characters. For
Bangla numerals the maximum success rate of 99.45%. is achieved on a database
of 12000 sample. The current work validates the usefulness of a new kind of
feature set for recognition of handwritten Bangla basic characters and
numerals.
PinView: Implicit Feedback in Content-Based Image Retrieval
Authors: Zakria Hussain, Arto Klami, Jussi Kujala, Alex P. Leung, Kitsuchart Pasupa, Peter Auer, Samuel Kaski, Jorma Laaksonen, John Shawe-Taylor
Date: 2014-10-02 00:00:00
Tasks: Content-Based Image Retrieval, Image Retrieval, Retrieval

This paper describes PinView, a content-based image retrieval system that
exploits implicit relevance feedback collected during a search session. PinView
contains several novel methods to infer the intent of the user. From relevance
feedback, such as eye movements or pointer clicks, and visual features of
images, PinView learns a similarity metric between images which depends on the
current interests of the user. It then retrieves images with a specialized
online learning algorithm that balances the tradeoff between exploring new
images and exploiting the already inferred interests of the user. We have
integrated PinView to the content-based image retrieval system PicSOM, which
enables applying PinView to real-world image databases. With the new algorithms
PinView outperforms the original PicSOM, and in online experiments with real
users the combination of implicit and explicit feedback gives the best results.
Identification of Dynamic functional brain network states Through Tensor Decomposition
Authors: Arash Golibagh Mahyari, Selin Aviyente
Date: 2014-10-02 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Functional Connectivity, Tensor Decomposition

With the advances in high resolution neuroimaging, there has been a growing
interest in the detection of functional brain connectivity. Complex network
theory has been proposed as an attractive mathematical representation of
functional brain networks. However, most of the current studies of functional
brain networks have focused on the computation of graph theoretic indices for
static networks, i.e. long-time averages of connectivity networks. It is
well-known that functional connectivity is a dynamic process and the
construction and reorganization of the networks is key to understanding human
cognition. Therefore, there is a growing need to track dynamic functional brain
networks and identify time intervals over which the network is
quasi-stationary. In this paper, we present a tensor decomposition based method
to identify temporally invariant 'network states' and find a common topographic
representation for each state. The proposed methods are applied to
electroencephalogram (EEG) data during the study of error-related negativity
(ERN).
Scalable Nonlinear Learning with Adaptive Polynomial Expansions
Authors: Alekh Agarwal, Alina Beygelzimer, Daniel Hsu, John Langford, Matus Telgarsky
Date: 2014-10-02 00:00:00
Tasks: Computational Efficiency

Can we effectively learn a nonlinear representation in time comparable to
linear learning? We describe a new algorithm that explicitly and adaptively
expands higher-order interaction features over base linear representations. The
algorithm is designed for extreme computational efficiency, and an extensive
experimental study shows that its computation/prediction tradeoff ability
compares very favorably against strong baselines.
Methods and Models for Interpretable Linear Classification
Authors: Berk Ustun, Cynthia Rudin
Date: 2014-05-16 00:00:00
Tasks: Classification, General Classification

We present an integer programming framework to build accurate and
interpretable discrete linear classification models. Unlike existing
approaches, our framework is designed to provide practitioners with the control
and flexibility they need to tailor accurate and interpretable models for a
domain of choice. To this end, our framework can produce models that are fully
optimized for accuracy, by minimizing the 0--1 classification loss, and that
address multiple aspects of interpretability, by incorporating a range of
discrete constraints and penalty functions. We use our framework to produce
models that are difficult to create with existing methods, such as scoring
systems and M-of-N rule tables. In addition, we propose specially designed
optimization methods to improve the scalability of our framework through
decomposition and data reduction. We show that discrete linear classifiers can
attain the training accuracy of any other linear classifier, and provide an
Occam's Razor type argument as to why the use of small discrete coefficients
can provide better generalization. We demonstrate the performance and
flexibility of our framework through numerical experiments and a case study in
which we construct a highly tailored clinical tool for sleep apnea diagnosis.
Learning to Transfer Privileged Information
Authors: Viktoriia Sharmanska, Novi Quadrianto, Christoph H. Lampert
Date: 2014-10-01 00:00:00
Tasks: General Classification

We introduce a learning framework called learning using privileged
information (LUPI) to the computer vision field. We focus on the prototypical
computer vision problem of teaching computers to recognize objects in images.
We want the computers to be able to learn faster at the expense of providing
extra information during training time. As additional information about the
image data, we look at several scenarios that have been studied in computer
vision before: attributes, bounding boxes and image tags. The information is
privileged as it is available at training time but not at test time. We explore
two maximum-margin techniques that are able to make use of this additional
source of information, for binary and multiclass object classification. We
interpret these methods as learning easiness and hardness of the objects in the
privileged space and then transferring this knowledge to train a better
classifier in the original space. We provide a thorough analysis and comparison
of information transfer from privileged to the original data spaces for both
LUPI methods. Our experiments show that incorporating privileged information
can improve the classification accuracy. Finally, we conduct user studies to
understand which samples are easy and which are hard for human learning, and
explore how this information is related to easy and hard samples when learning
a classifier.
The Universe of Minds
Authors: Roman V. Yampolskiy
Date: 2014-10-01 00:00:00
Tasks: Survey

The paper attempts to describe the space of possible mind designs by first
equating all minds to software. Next it proves some interesting properties of
the mind design space such as infinitude of minds, size and representation
complexity of minds. A survey of mind design taxonomies is followed by a
proposal for a new field of investigation devoted to study of minds,
intellectology, a list of open problems for this new field is presented.
Domain adaptation of weighted majority votes via perturbed variation-based self-labeling
Authors: Emilie Morvant
Date: 2014-10-01 00:00:00
Tasks: Domain Adaptation

In machine learning, the domain adaptation problem arrives when the test
(target) and the train (source) data are generated from different
distributions. A key applied issue is thus the design of algorithms able to
generalize on a new distribution, for which we have no label information. We
focus on learning classification models defined as a weighted majority vote
over a set of real-val ued functions. In this context, Germain et al. (2013)
have shown that a measure of disagreement between these functions is crucial to
control. The core of this measure is a theoretical bound--the C-bound (Lacasse
et al., 2007)--which involves the disagreement and leads to a well performing
majority vote learning algorithm in usual non-adaptative supervised setting:
MinCq. In this work, we propose a framework to extend MinCq to a domain
adaptation scenario. This procedure takes advantage of the recent perturbed
variation divergence between distributions proposed by Harel and Mannor (2012).
Justified by a theoretical bound on the target risk of the vote, we provide to
MinCq a target sample labeled thanks to a perturbed variation-based
self-labeling focused on the regions where the source and target marginals
appear similar. We also study the influence of our self-labeling, from which we
deduce an original process for tuning the hyperparameters. Finally, our
framework called PV-MinCq shows very promising results on a rotation and
translation synthetic problem.
Deformable Part Models are Convolutional Neural Networks
Authors: Ross Girshick, Forrest Iandola, Trevor Darrell, Jitendra Malik
Date: 2014-09-18 00:00:00
Proceeding: CVPR 2015 6
Tasks: Object Detection, Rolling Shutter Correction

Deformable part models (DPMs) and convolutional neural networks (CNNs) are
two widely used tools for visual recognition. They are typically viewed as
distinct approaches: DPMs are graphical models (Markov random fields), while
CNNs are "black-box" non-linear classifiers. In this paper, we show that a DPM
can be formulated as a CNN, thus providing a novel synthesis of the two ideas.
Our construction involves unrolling the DPM inference algorithm and mapping
each step to an equivalent (and at times novel) CNN layer. From this
perspective, it becomes natural to replace the standard image features used in
DPM with a learned feature extractor. We call the resulting model DeepPyramid
DPM and experimentally validate it on PASCAL VOC. DeepPyramid DPM significantly
outperforms DPMs based on histograms of oriented gradients features (HOG) and
slightly outperforms a comparable version of the recently introduced R-CNN
detection system, while running an order of magnitude faster.
Using social network graph analysis for interest detection
Authors: Brian Lee Yung Rowe
Date: 2014-10-01 00:00:00
Tasks: Collaborative Filtering

A person's interests exist as an internal state and are difficult to define.
Since only external actions are observable, a proxy must be used that
represents someone's interests. Techniques like collaborative filtering,
behavioral targeting, and hashtag analysis implicitly model an individual's
interests. I argue that these models are limited to shallow, temporary
interests, which do not reflect people's deeper interests or passions. I
propose an alternative model of interests that takes advantage of a user's
social graph. The basic principle is that people only follow those that
interest them, so the social graph is an effective and robust proxy for
people's interests.