pwc-archive/papers-with-abstracts huggingface dataset:
The return of AdaBoost.MH: multi-class Hamming trees
Authors: Balázs Kégl
Date: 2013-12-20 00:00:00
Tasks: Computational Efficiency

Within the framework of AdaBoost.MH, we propose to train vector-valued
decision trees to optimize the multi-class edge without reducing the
multi-class problem to $K$ binary one-against-all classifications. The key
element of the method is a vector-valued decision stump, factorized into an
input-independent vector of length $K$ and label-independent scalar classifier.
At inner tree nodes, the label-dependent vector is discarded and the binary
classifier can be used for partitioning the input space into two regions. The
algorithm retains the conceptual elegance, power, and computational efficiency
of binary AdaBoost. In experiments it is on par with support vector machines
and with the best existing multi-class boosting algorithm AOSOLogitBoost, and
it is significantly better than other known implementations of AdaBoost.MH.
Using Web Co-occurrence Statistics for Improving Image Categorization
Authors: Samy Bengio, Jeff Dean, Dumitru Erhan, Eugene Ie, Quoc Le, Andrew Rabinovich, Jonathon Shlens, Yoram Singer
Date: 2013-12-19 00:00:00
Tasks: Common Sense Reasoning, Image Categorization, Object Recognition

Object recognition and localization are important tasks in computer vision.
The focus of this work is the incorporation of contextual information in order
to improve object recognition and localization. For instance, it is natural to
expect not to see an elephant to appear in the middle of an ocean. We consider
a simple approach to encapsulate such common sense knowledge using
co-occurrence statistics from web documents. By merely counting the number of
times nouns (such as elephants, sharks, oceans, etc.) co-occur in web
documents, we obtain a good estimate of expected co-occurrences in visual data.
We then cast the problem of combining textual co-occurrence statistics with the
predictions of image-based classifiers as an optimization problem. The
resulting optimization problem serves as a surrogate for our inference
procedure. Albeit the simplicity of the resulting optimization problem, it is
effective in improving both recognition and localization accuracy. Concretely,
we observe significant improvements in recognition and localization rates for
both ImageNet Detection 2012 and Sun 2012 datasets.
Occupancy Detection in Vehicles Using Fisher Vector Image Representation
Authors: Yusuf Artan, Peter Paul
Date: 2013-12-20 00:00:00
Tasks: Face Detection, General Classification, image-classification, Image Classification

Due to the high volume of traffic on modern roadways, transportation agencies
have proposed High Occupancy Vehicle (HOV) lanes and High Occupancy Tolling
(HOT) lanes to promote car pooling. However, enforcement of the rules of these
lanes is currently performed by roadside enforcement officers using visual
observation. Manual roadside enforcement is known to be inefficient, costly,
potentially dangerous, and ultimately ineffective. Violation rates up to
50%-80% have been reported, while manual enforcement rates of less than 10% are
typical. Therefore, there is a need for automated vehicle occupancy detection
to support HOV/HOT lane enforcement. A key component of determining vehicle
occupancy is to determine whether or not the vehicle's front passenger seat is
occupied. In this paper, we examine two methods of determining vehicle front
seat occupancy using a near infrared (NIR) camera system pointed at the
vehicle's front windshield. The first method examines a state-of-the-art
deformable part model (DPM) based face detection system that is robust to
facial pose. The second method examines state-of- the-art local aggregation
based image classification using bag-of-visual-words (BOW) and Fisher vectors
(FV). A dataset of 3000 images was collected on a public roadway and is used to
perform the comparison. From these experiments it is clear that the image
classification approach is superior for this problem.
High-Dimensional Regression with Gaussian Mixtures and Partially-Latent Response Variables
Authors: Antoine Deleforge, Florence Forbes, Radu Horaud
Date: 2013-08-10 00:00:00
Tasks: Data Augmentation, Dimensionality Reduction, regression, Vocal Bursts Intensity Prediction

In this work we address the problem of approximating high-dimensional data
with a low-dimensional representation. We make the following contributions. We
propose an inverse regression method which exchanges the roles of input and
response, such that the low-dimensional variable becomes the regressor, and
which is tractable. We introduce a mixture of locally-linear probabilistic
mapping model that starts with estimating the parameters of inverse regression,
and follows with inferring closed-form solutions for the forward parameters of
the high-dimensional regression problem of interest. Moreover, we introduce a
partially-latent paradigm, such that the vector-valued response variable is
composed of both observed and latent entries, thus being able to deal with data
contaminated by experimental artifacts that cannot be explained with noise
models. The proposed probabilistic formulation could be viewed as a
latent-variable augmentation of regression. We devise expectation-maximization
(EM) procedures based on a data augmentation strategy which facilitates the
maximum-likelihood search over the model parameters. We propose two
augmentation schemes and we describe in detail the associated EM inference
procedures that may well be viewed as generalizations of a number of EM
regression, dimension reduction, and factor analysis algorithms. The proposed
framework is validated with both synthetic and real data. We provide
experimental evidence that our method outperforms several existing regression
techniques.
The Sparse Principal Component of a Constant-rank Matrix
Authors: Megasthenis Asteris, Dimitris S. Papailiopoulos, George N. Karystinos
Date: 2013-12-20 00:00:00

The computation of the sparse principal component of a matrix is equivalent
to the identification of its principal submatrix with the largest maximum
eigenvalue. Finding this optimal submatrix is what renders the problem
${\mathcal{NP}}$-hard. In this work, we prove that, if the matrix is positive
semidefinite and its rank is constant, then its sparse principal component is
polynomially computable. Our proof utilizes the auxiliary unit vector technique
that has been recently developed to identify problems that are polynomially
solvable. Moreover, we use this technique to design an algorithm which, for any
sparsity value, computes the sparse principal component with complexity
${\mathcal O}\left(N^{D+1}\right)$, where $N$ and $D$ are the matrix size and
rank, respectively. Our algorithm is fully parallelizable and memory efficient.
Non-parametric Bayesian modeling of complex networks
Authors: Mikkel N. Schmidt, Morten Mørup
Date: 2013-12-20 00:00:00

Modeling structure in complex networks using Bayesian non-parametrics makes
it possible to specify flexible model structures and infer the adequate model
complexity from the observed data. This paper provides a gentle introduction to
non-parametric Bayesian modeling of complex networks: Using an infinite mixture
model as running example we go through the steps of deriving the model as an
infinite limit of a finite parametric model, inferring the model parameters by
Markov chain Monte Carlo, and checking the model's fit and predictive
performance. We explain how advanced non-parametric models for complex networks
can be derived and point out relevant literature.
Optimal parameter selection for unsupervised neural network using genetic algorithm
Authors: suneetha chittineni, Raveendra Babu Bhogapathi
Date: 2013-12-20 00:00:00
Tasks: Clustering

K-means Fast Learning Artificial Neural Network (K-FLANN) is an unsupervised
neural network requires two parameters: tolerance and vigilance. Best
Clustering results are feasible only by finest parameters specified to the
neural network. Selecting optimal values for these parameters is a major
problem. To solve this issue, Genetic Algorithm (GA) is used to determine
optimal parameters of K-FLANN for finding groups in multidimensional data.
K-FLANN is a simple topological network, in which output nodes grows
dynamically during the clustering process on receiving input patterns. Original
K-FLANN is enhanced to select winner unit out of the matched nodes so that
stable clusters are formed with in a less number of epochs. The experimental
results show that the GA is efficient in finding optimal values of parameters
from the large search space and is tested using artificial and synthetic data
sets.
Unsupervised Feature Learning by Deep Sparse Coding
Authors: Yunlong He, Koray Kavukcuoglu, Yun Wang, Arthur Szlam, Yanjun Qi
Date: 2013-12-20 00:00:00
Tasks: Object Recognition

In this paper, we propose a new unsupervised feature learning framework,
namely Deep Sparse Coding (DeepSC), that extends sparse coding to a multi-layer
architecture for visual object recognition tasks. The main innovation of the
framework is that it connects the sparse-encoders from different layers by a
sparse-to-dense module. The sparse-to-dense module is a composition of a local
spatial pooling step and a low-dimensional embedding process, which takes
advantage of the spatial smoothness information in the image. As a result, the
new method is able to learn several levels of sparse representation of the
image which capture features at a variety of abstraction levels and
simultaneously preserve the spatial smoothness between the neighboring image
patches. Combining the feature representations from multiple layers, DeepSC
achieves the state-of-the-art performance on multiple object recognition tasks.
The Value Iteration Algorithm is Not Strongly Polynomial for Discounted Dynamic Programming
Authors: Eugene A. Feinberg, Jefferson Huang
Date: 2013-12-19 00:00:00

This note provides a simple example demonstrating that, if exact computations
are allowed, the number of iterations required for the value iteration
algorithm to find an optimal policy for discounted dynamic programming problems
may grow arbitrarily quickly with the size of the problem. In particular, the
number of iterations can be exponential in the number of actions. Thus, unlike
policy iterations, the value iteration algorithm is not strongly polynomial for
discounted dynamic programming.
Time-varying Learning and Content Analytics via Sparse Factor Analysis
Authors: Andrew S. Lan, Christoph Studer, Richard G. Baraniuk
Date: 2013-12-19 00:00:00
Tasks: Collaborative Filtering, Knowledge Tracing

We propose SPARFA-Trace, a new machine learning-based framework for
time-varying learning and content analytics for education applications. We
develop a novel message passing-based, blind, approximate Kalman filter for
sparse factor analysis (SPARFA), that jointly (i) traces learner concept
knowledge over time, (ii) analyzes learner concept knowledge state transitions
(induced by interacting with learning resources, such as textbook sections,
lecture videos, etc, or the forgetting effect), and (iii) estimates the content
organization and intrinsic difficulty of the assessment questions. These
quantities are estimated solely from binary-valued (correct/incorrect) graded
learner response data and a summary of the specific actions each learner
performs (e.g., answering a question or studying a learning resource) at each
time instance. Experimental results on two online course datasets demonstrate
that SPARFA-Trace is capable of tracing each learner's concept knowledge
evolution over time, as well as analyzing the quality and content organization
of learning resources, the question-concept associations, and the question
intrinsic difficulties. Moreover, we show that SPARFA-Trace achieves comparable
or better performance in predicting unobserved learner responses than existing
collaborative filtering and knowledge tracing approaches for personalized
education.
Flower Pollination Algorithm for Global Optimization
Authors: Xin-She Yang
Date: 2013-12-19 00:00:00
Tasks: global-optimization

Flower pollination is an intriguing process in the natural world. Its
evolutionary characteristics can be used to design new optimization algorithms.
In this paper, we propose a new algorithm, namely, flower pollination
algorithm, inspired by the pollination process of flowers. We first use ten
test functions to validate the new algorithm, and compare its performance with
genetic algorithms and particle swarm optimization. Our simulation results show
the flower algorithm is more efficient than both GA and PSO. We also use the
flower algorithm to solve a nonlinear design benchmark, which shows the
convergence rate is almost exponential.
Abstraction in decision-makers with limited information processing capabilities
Authors: Tim Genewein, Daniel A. Braun
Date: 2013-12-16 00:00:00
Tasks: Decision Making

A distinctive property of human and animal intelligence is the ability to
form abstractions by neglecting irrelevant information which allows to separate
structure from noise. From an information theoretic point of view abstractions
are desirable because they allow for very efficient information processing. In
artificial systems abstractions are often implemented through computationally
costly formations of groups or clusters. In this work we establish the relation
between the free-energy framework for decision making and rate-distortion
theory and demonstrate how the application of rate-distortion for
decision-making leads to the emergence of abstractions. We argue that
abstractions are induced due to a limit in information processing capacity.
Playing Atari with Deep Reinforcement Learning
Authors: Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller
Date: 2013-12-19 00:00:00
Tasks: Atari Games, Deep Reinforcement Learning, Multi-Goal Reinforcement Learning, Q-Learning, Reinforcement Learning, Reinforcement Learning (RL)

We present the first deep learning model to successfully learn control
policies directly from high-dimensional sensory input using reinforcement
learning. The model is a convolutional neural network, trained with a variant
of Q-learning, whose input is raw pixels and whose output is a value function
estimating future rewards. We apply our method to seven Atari 2600 games from
the Arcade Learning Environment, with no adjustment of the architecture or
learning algorithm. We find that it outperforms all previous approaches on six
of the games and surpasses a human expert on three of them.
An Adaptive Dictionary Learning Approach for Modeling Dynamical Textures
Authors: Xian Wei, Hao Shen, Martin Kleinsteuber
Date: 2013-12-19 00:00:00
Tasks: Dictionary Learning

Video representation is an important and challenging task in the computer
vision community. In this paper, we assume that image frames of a moving scene
can be modeled as a Linear Dynamical System. We propose a sparse coding
framework, named adaptive video dictionary learning (AVDL), to model a video
adaptively. The developed framework is able to capture the dynamics of a moving
scene by exploring both sparse properties and the temporal correlations of
consecutive video frames. The proposed method is compared with state of the art
video processing methods on several benchmark data sequences, which exhibit
appearance changes and heavy occlusions.
My First Deep Learning System of 1991 + Deep Learning Timeline 1962-2013
Authors: Jürgen Schmidhuber
Date: 2013-12-19 00:00:00
Tasks: Deep Learning

Deep Learning has attracted significant attention in recent years. Here I
present a brief overview of my first Deep Learner of 1991, and its historic
context, with a timeline of Deep Learning highlights.
Conservative, Proportional and Optimistic Contextual Discounting in the Belief Functions Theory
Authors: Marek Kurdej, Véronique Cherfaoui
Date: 2013-12-19 00:00:00

Information discounting plays an important role in the theory of belief
functions and, generally, in information fusion. Nevertheless, neither
classical uniform discounting nor contextual cannot model certain use cases,
notably temporal discounting. In this article, new contextual discounting
schemes, conservative, proportional and optimistic, are proposed. Some
properties of these discounting operations are examined. Classical discounting
is shown to be a special case of these schemes. Two motivating cases are
discussed: modelling of source reliability and application to temporal
discounting.
Deep Learning Embeddings for Discontinuous Linguistic Units
Authors: Wenpeng Yin, Hinrich Schütze
Date: 2013-12-18 00:00:00
Tasks: coreference-resolution, Coreference Resolution, Deep Learning

Deep learning embeddings have been successfully used for many natural
language processing problems. Embeddings are mostly computed for word forms
although a number of recent papers have extended this to other linguistic units
like morphemes and phrases. In this paper, we argue that learning embeddings
for discontinuous linguistic units should also be considered. In an
experimental evaluation on coreference resolution, we show that such embeddings
perform better than word form embeddings.
Codebook based Audio Feature Representation for Music Information Retrieval
Authors: Yonatan Vaizman, Brian McFee, Gert Lanckriet
Date: 2013-12-19 00:00:00
Tasks: Information Retrieval, Management, Music Information Retrieval, Quantization, Recommendation Systems, Retrieval, TAG

Digital music has become prolific in the web in recent decades. Automated
recommendation systems are essential for users to discover music they love and
for artists to reach appropriate audience. When manual annotations and user
preference data is lacking (e.g. for new artists) these systems must rely on
\emph{content based} methods. Besides powerful machine learning tools for
classification and retrieval, a key component for successful recommendation is
the \emph{audio content representation}.
  Good representations should capture informative musical patterns in the audio
signal of songs. These representations should be concise, to enable efficient
(low storage, easy indexing, fast search) management of huge music
repositories, and should also be easy and fast to compute, to enable real-time
interaction with a user supplying new songs to the system.
  Before designing new audio features, we explore the usage of traditional
local features, while adding a stage of encoding with a pre-computed
\emph{codebook} and a stage of pooling to get compact vectorial
representations. We experiment with different encoding methods, namely
\emph{the LASSO}, \emph{vector quantization (VQ)} and \emph{cosine similarity
(CS)}. We evaluate the representations' quality in two music information
retrieval applications: query-by-tag and query-by-example. Our results show
that concise representations can be used for successful performance in both
applications. We recommend using top-$\tau$ VQ encoding, which consistently
performs well in both applications, and requires much less computation time
than the LASSO.
Delegating Custom Object Detection Tasks to a Universal Classification System
Authors: Andrew Gleibman
Date: 2013-12-19 00:00:00
Tasks: Classification, General Classification, Object, object-detection, Object Detection

In this paper, a concept of multipurpose object detection system, recently
introduced in our previous work, is clarified. The business aspect of this
method is transformation of a classifier into an object detector/locator via an
image grid. This is a universal framework for locating objects of interest
through classification. The framework standardizes and simplifies
implementation of custom systems by doing only a custom analysis of the
classification results on the image grid.
Some Improvements on Deep Convolutional Neural Network Based Image Classification
Authors: Andrew G. Howard
Date: 2013-12-19 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Object Recognition

We investigate multiple techniques to improve upon the current state of the
art deep convolutional neural network based image classification pipeline. The
techiques include adding more image transformations to training data, adding
more transformations to generate additional predictions at test time and using
complementary models applied to higher resolution images. This paper summarizes
our entry in the Imagenet Large Scale Visual Recognition Challenge 2013. Our
system achieved a top 5 classification error rate of 13.55% using no external
data which is over a 20% relative improvement on the previous year's winner.
Missing Value Imputation With Unsupervised Backpropagation
Authors: Michael S. Gashler, Michael R. Smith, Richard Morris, Tony Martinez
Date: 2013-12-19 00:00:00
Tasks: Collaborative Filtering, General Classification, Imputation, Missing Values

Many data mining and data analysis techniques operate on dense matrices or
complete tables of data. Real-world data sets, however, often contain unknown
values. Even many classification algorithms that are designed to operate with
missing values still exhibit deteriorated accuracy. One approach to handling
missing values is to fill in (impute) the missing values. In this paper, we
present a technique for unsupervised learning called Unsupervised
Backpropagation (UBP), which trains a multi-layer perceptron to fit to the
manifold sampled by a set of observed point-vectors. We evaluate UBP with the
task of imputing missing values in datasets, and show that UBP is able to
predict missing values with significantly lower sum-squared error than other
collaborative filtering and imputation techniques. We also demonstrate with 24
datasets and 9 supervised learning algorithms that classification accuracy is
usually higher when randomly-withheld values are imputed using UBP, rather than
with other methods.
On SAT representations of XOR constraints
Authors: Matthew Gwynne, Oliver Kullmann
Date: 2013-09-12 00:00:00
Tasks: ARC, Translation

We study the representation of systems S of linear equations over the
two-element field (aka xor- or parity-constraints) via conjunctive normal forms
F (boolean clause-sets). First we consider the problem of finding an
"arc-consistent" representation ("AC"), meaning that unit-clause propagation
will fix all forced assignments for all possible instantiations of the
xor-variables. Our main negative result is that there is no polysize
AC-representation in general. On the positive side we show that finding such an
AC-representation is fixed-parameter tractable (fpt) in the number of
equations. Then we turn to a stronger criterion of representation, namely
propagation completeness ("PC") --- while AC only covers the variables of S,
now all the variables in F (the variables in S plus auxiliary variables) are
considered for PC. We show that the standard translation actually yields a PC
representation for one equation, but fails so for two equations (in fact
arbitrarily badly). We show that with a more intelligent translation we can
also easily compute a translation to PC for two equations. We conjecture that
computing a representation in PC is fpt in the number of equations.
Detecting Parameter Symmetries in Probabilistic Models
Authors: Robert Nishihara, Thomas Minka, Daniel Tarlow
Date: 2013-12-19 00:00:00
Tasks: Probabilistic Programming

Probabilistic models often have parameters that can be translated, scaled,
permuted, or otherwise transformed without changing the model. These symmetries
can lead to strong correlation and multimodality in the posterior distribution
over the model's parameters, which can pose challenges both for performing
inference and interpreting the results. In this work, we address the automatic
detection of common problematic model symmetries. To do so, we introduce local
symmetries, which cover many common cases and are amenable to automatic
detection. We show how to derive algorithms to detect several broad classes of
local symmetries. Our algorithms are compatible with probabilistic programming
constructs such as arrays, for loops, and if statements, and they scale to
models with many variables.