pwc-archive/papers-with-abstracts huggingface dataset:
BayesOpt: A Library for Bayesian optimization with Robotics Applications
Authors: Ruben Martinez-Cantin
Date: 2013-09-03 00:00:00
Tasks: Active Learning, Bayesian Optimization, Experimental Design

The purpose of this paper is twofold. On one side, we present a general
framework for Bayesian optimization and we compare it with some related fields
in active learning and Bayesian numerical analysis. On the other hand, Bayesian
optimization and related problems (bandits, sequential experimental design) are
highly dependent on the surrogate model that is selected. However, there is no
clear standard in the literature. Thus, we present a fast and flexible toolbox
that allows to test and combine different models and criteria with little
effort. It includes most of the state-of-the-art contributions, algorithms and
models. Its speed also removes part of the stigma that Bayesian optimization
methods are only good for "expensive functions". The software is free and it
can be used in many operating systems and computer languages.
Majority Rule for Belief Evolution in Social Networks
Authors: Yi Zhou
Date: 2013-09-03 00:00:00

In this paper, we study how an agent's belief is affected by her neighbors in
a social network. We first introduce a general framework, where every agent has
an initial belief on a statement, and updates her belief according to her and
her neighbors' current beliefs under some belief evolution functions, which,
arguably, should satisfy some basic properties. Then, we focus on the majority
rule belief evolution function, that is, an agent will (dis)believe the
statement iff more than half of her neighbors (dis)believe it. We consider some
fundamental issues about majority rule belief evolution, for instance, whether
the belief evolution process will eventually converge. The answer is no in
general. However, for random asynchronous belief evolution, this is indeed the
case.
Learning to Rank for Blind Image Quality Assessment
Authors: Fei Gao, DaCheng Tao, Xinbo Gao, Xuelong. Li
Date: 2013-09-01 00:00:00
Tasks: Blind Image Quality Assessment, Image Quality Assessment, Learning-To-Rank

Blind image quality assessment (BIQA) aims to predict perceptual image
quality scores without access to reference images. State-of-the-art BIQA
methods typically require subjects to score a large number of images to train a
robust model. However, subjective quality scores are imprecise, biased, and
inconsistent, and it is challenging to obtain a large scale database, or to
extend existing databases, because of the inconvenience of collecting images,
training the subjects, conducting subjective experiments, and realigning human
quality evaluations. To combat these limitations, this paper explores and
exploits preference image pairs (PIPs) such as "the quality of image $I_a$ is
better than that of image $I_b$" for training a robust BIQA model. The
preference label, representing the relative quality of two images, is generally
precise and consistent, and is not sensitive to image content, distortion type,
or subject identity; such PIPs can be generated at very low cost. The proposed
BIQA method is one of learning to rank. We first formulate the problem of
learning the mapping from the image features to the preference label as one of
classification. In particular, we investigate the utilization of a multiple
kernel learning algorithm based on group lasso (MKLGL) to provide a solution. A
simple but effective strategy to estimate perceptual image quality scores is
then presented. Experiments show that the proposed BIQA method is highly
effective and achieves comparable performance to state-of-the-art BIQA
algorithms. Moreover, the proposed method can be easily extended to new
distortion categories.
A Multi-View Embedding Space for Modeling Internet Images, Tags, and their Semantics
Authors: Yunchao Gong, Qifa Ke, Michael Isard, Svetlana Lazebnik
Date: 2012-12-18 00:00:00
Tasks: Clustering, Image Retrieval, Retrieval, TAG

This paper investigates the problem of modeling Internet images and
associated text or tags for tasks such as image-to-image search, tag-to-image
search, and image-to-tag search (image annotation). We start with canonical
correlation analysis (CCA), a popular and successful approach for mapping
visual and textual features to the same latent space, and incorporate a third
view capturing high-level image semantics, represented either by a single
category or multiple non-mutually-exclusive concepts. We present two ways to
train the three-view embedding: supervised, with the third view coming from
ground-truth labels or search keywords; and unsupervised, with semantic themes
automatically obtained by clustering the tags. To ensure high accuracy for
retrieval tasks while keeping the learning process scalable, we combine
multiple strong visual features and use explicit nonlinear kernel mappings to
efficiently approximate kernel CCA. To perform retrieval, we use a specially
designed similarity function in the embedded space, which substantially
outperforms the Euclidean distance. The resulting system produces compelling
qualitative results and outperforms a number of two-view baselines on retrieval
tasks on three large-scale Internet image datasets.
Scalable Probabilistic Entity-Topic Modeling
Authors: Neil Houlsby, Massimiliano Ciaramita
Date: 2013-09-02 00:00:00
Tasks: Entity Disambiguation

We present an LDA approach to entity disambiguation. Each topic is associated
with a Wikipedia article and topics generate either content words or entity
mentions. Training such models is challenging because of the topic and
vocabulary size, both in the millions. We tackle these problems using a novel
distributed inference and representation framework based on a parallel Gibbs
sampler guided by the Wikipedia link graph, and pipelines of MapReduce allowing
fast and memory-frugal processing of large datasets. We report state-of-the-art
performance on a public dataset.
Parallel Gaussian Process Optimization with Upper Confidence Bound and Pure Exploration
Authors: Emile Contal, David Buffoni, Alexandre Robicquet, Nicolas Vayatis
Date: 2013-04-19 00:00:00

In this paper, we consider the challenge of maximizing an unknown function f
for which evaluations are noisy and are acquired with high cost. An iterative
procedure uses the previous measures to actively select the next estimation of
f which is predicted to be the most useful. We focus on the case where the
function can be evaluated in parallel with batches of fixed size and analyze
the benefit compared to the purely sequential procedure in terms of cumulative
regret. We introduce the Gaussian Process Upper Confidence Bound and Pure
Exploration algorithm (GP-UCB-PE) which combines the UCB strategy and Pure
Exploration in the same batch of evaluations along the parallel iterations. We
prove theoretical upper bounds on the regret with batches of size K for this
procedure which show the improvement of the order of sqrt{K} for fixed
iteration cost over purely sequential versions. Moreover, the multiplicative
constants involved have the property of being dimension-free. We also confirm
empirically the efficiency of GP-UCB-PE on real and synthetic problems compared
to state-of-the-art competitors.
A Study on Unsupervised Dictionary Learning and Feature Encoding for Action Classification
Authors: Xiaojiang Peng, Qiang Peng, Yu Qiao, Junzhou Chen, Mehtab Afzal
Date: 2013-09-02 00:00:00
Tasks: Action Classification, Dictionary Learning, General Classification, Quantization

Many efforts have been devoted to develop alternative methods to traditional
vector quantization in image domain such as sparse coding and soft-assignment.
These approaches can be split into a dictionary learning phase and a feature
encoding phase which are often closely connected. In this paper, we investigate
the effects of these phases by separating them for video-based action
classification. We compare several dictionary learning methods and feature
encoding schemes through extensive experiments on KTH and HMDB51 datasets.
Experimental results indicate that sparse coding performs consistently better
than the other encoding methods in large complex dataset (i.e., HMDB51), and it
is robust to different dictionaries. For small simple dataset (i.e., KTH) with
less variation, however, all the encoding strategies perform competitively. In
addition, we note that the strength of sophisticated encoding approaches comes
not from their corresponding dictionaries but the encoding mechanisms, and we
can just use randomly selected exemplars as dictionaries for video-based action
classification.
Unmixing Incoherent Structures of Big Data by Randomized or Greedy Decomposition
Authors: Tianyi Zhou, DaCheng Tao
Date: 2013-09-02 00:00:00

Learning big data by matrix decomposition always suffers from expensive
computation, mixing of complicated structures and noise. In this paper, we
study more adaptive models and efficient algorithms that decompose a data
matrix as the sum of semantic components with incoherent structures. We firstly
introduce "GO decomposition (GoDec)", an alternating projection method
estimating the low-rank part $L$ and the sparse part $S$ from data matrix
$X=L+S+G$ corrupted by noise $G$. Two acceleration strategies are proposed to
obtain scalable unmixing algorithm on big data: 1) Bilateral random projection
(BRP) is developed to speed up the update of $L$ in GoDec by a closed-form
built from left and right random projections of $X-S$ in lower dimensions; 2)
Greedy bilateral (GreB) paradigm updates the left and right factors of $L$ in a
mutually adaptive and greedy incremental manner, and achieve significant
improvement in both time and sample complexities. Then we proposes three
nontrivial variants of GoDec that generalizes GoDec to more general data type
and whose fast algorithms can be derived from the two strategies......
Parallel machine scheduling with step deteriorating jobs and setup times by a hybrid discrete cuckoo search algorithm
Authors: Peng Guo, Wenming Cheng, Yi Wang
Date: 2013-09-02 00:00:00
Tasks: Scheduling

This article considers the parallel machine scheduling problem with
step-deteriorating jobs and sequence-dependent setup times. The objective is to
minimize the total tardiness by determining the allocation and sequence of jobs
on identical parallel machines. In this problem, the processing time of each
job is a step function dependent upon its starting time. An individual extended
time is penalized when the starting time of a job is later than a specific
deterioration date. The possibility of deterioration of a job makes the
parallel machine scheduling problem more challenging than ordinary ones. A
mixed integer programming model for the optimal solution is derived. Due to its
NP-hard nature, a hybrid discrete cuckoo search algorithm is proposed to solve
this problem. In order to generate a good initial swarm, a modified heuristic
named the MBHG is incorporated into the initialization of population. Several
discrete operators are proposed in the random walk of L\'{e}vy Flights and the
crossover search. Moreover, a local search procedure based on variable
neighborhood descent is integrated into the algorithm as a hybrid strategy in
order to improve the quality of elite solutions. Computational experiments are
executed on two sets of randomly generated test instances. The results show
that the proposed hybrid algorithm can yield better solutions in comparison
with the commercial solver CPLEX with one hour time limit, discrete cuckoo
search algorithm and the existing variable neighborhood search algorithm.
Demodulation of Sparse PPM Signals with Low Samples Using Trained RIP Matrix
Authors: Seyed Hossein Hosseini, Mahrokh G. Shayesteh, Mehdi Chehel Amirani
Date: 2013-09-01 00:00:00
Tasks: compressed sensing, General Classification, Human Dynamics

Compressed sensing (CS) theory considers the restricted isometry property
(RIP) as a sufficient condition for measurement matrix which guarantees the
recovery of any sparse signal from its compressed measurements. The RIP
condition also preserves enough information for classification of sparse
symbols, even with fewer measurements. In this work, we utilize RIP bound as
the cost function for training a simple neural network in order to exploit the
near optimal measurements or equivalently near optimal features for
classification of a known set of sparse symbols. As an example, we consider
demodulation of pulse position modulation (PPM) signals. The results indicate
that the proposed method has much better performance than the random
measurements and requires less samples than the optimum matched filter
demodulator, at the expense of some performance loss. Further, the proposed
approach does not need equalizer for multipath channels in contrast to the
conventional receiver.
Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification
Authors: Dan Cireşan, Jürgen Schmidhuber
Date: 2013-09-01 00:00:00
Tasks: General Classification

Our Multi-Column Deep Neural Networks achieve best known recognition rates on
Chinese characters from the ICDAR 2011 and 2013 offline handwriting
competitions, approaching human performance.
Ensemble approaches for improving community detection methods
Authors: Johan Dahlin, Pontus Svenson
Date: 2013-09-01 00:00:00
Tasks: Clustering, Community Detection

Statistical estimates can often be improved by fusion of data from several
different sources. One example is so-called ensemble methods which have been
successfully applied in areas such as machine learning for classification and
clustering. In this paper, we present an ensemble method to improve community
detection by aggregating the information found in an ensemble of community
structures. This ensemble can found by re-sampling methods, multiple runs of a
stochastic community detection method, or by several different community
detection algorithms applied to the same network. The proposed method is
evaluated using random networks with community structures and compared with two
commonly used community detection methods. The proposed method when applied on
a stochastic community detection algorithm performs well with low computational
complexity, thus offering both a new approach to community detection and an
additional community detection method.
API design for machine learning software: experiences from the scikit-learn project
Authors: Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake Vanderplas, Arnaud Joly, Brian Holt, Gaël Varoquaux
Date: 2013-09-01 00:00:00
Tasks: BIG-bench Machine Learning

Scikit-learn is an increasingly popular machine learning li- brary. Written
in Python, it is designed to be simple and efficient, accessible to
non-experts, and reusable in various contexts. In this paper, we present and
discuss our design choices for the application programming interface (API) of
the project. In particular, we describe the simple and elegant interface shared
by all learning and processing units in the library and then discuss its
advantages in terms of composition and reusability. The paper also comments on
implementation details specific to the Python ecosystem and analyzes obstacles
faced by users and developers of the library.
Non-Asymptotic Convergence Analysis of Inexact Gradient Methods for Machine Learning Without Strong Convexity
Authors: Anthony Man-Cho So
Date: 2013-08-31 00:00:00
Tasks: BIG-bench Machine Learning

Many recent applications in machine learning and data fitting call for the
algorithmic solution of structured smooth convex optimization problems.
Although the gradient descent method is a natural choice for this task, it
requires exact gradient computations and hence can be inefficient when the
problem size is large or the gradient is difficult to evaluate. Therefore,
there has been much interest in inexact gradient methods (IGMs), in which an
efficiently computable approximate gradient is used to perform the update in
each iteration. Currently, non-asymptotic linear convergence results for IGMs
are typically established under the assumption that the objective function is
strongly convex, which is not satisfied in many applications of interest; while
linear convergence results that do not require the strong convexity assumption
are usually asymptotic in nature. In this paper, we combine the best of these
two types of results and establish---under the standard assumption that the
gradient approximation errors decrease linearly to zero---the non-asymptotic
linear convergence of IGMs when applied to a class of structured convex
optimization problems. Such a class covers settings where the objective
function is not necessarily strongly convex and includes the least squares and
logistic regression problems. We believe that our techniques will find further
applications in the non-asymptotic convergence analysis of other first-order
methods.
Artificial Intelligence Based Cognitive Routing for Cognitive Radio Networks
Authors: Junaid Qadir
Date: 2013-08-31 00:00:00
Tasks: Decision Making

Cognitive radio networks (CRNs) are networks of nodes equipped with cognitive
radios that can optimize performance by adapting to network conditions. While
cognitive radio networks (CRN) are envisioned as intelligent networks,
relatively little research has focused on the network level functionality of
CRNs. Although various routing protocols, incorporating varying degrees of
adaptiveness, have been proposed for CRNs, it is imperative for the long term
success of CRNs that the design of cognitive routing protocols be pursued by
the research community. Cognitive routing protocols are envisioned as routing
protocols that fully and seamless incorporate AI-based techniques into their
design. In this paper, we provide a self-contained tutorial on various AI and
machine-learning techniques that have been, or can be, used for developing
cognitive routing protocols. We also survey the application of various classes
of AI techniques to CRNs in general, and to the problem of routing in
particular. We discuss various decision making techniques and learning
techniques from AI and document their current and potential applications to the
problem of routing in CRNs. We also highlight the various inference, reasoning,
modeling, and learning sub tasks that a cognitive routing protocol must solve.
Finally, open research issues and future directions of work are identified.
Derivation of Upper Bounds on Optimization Time of Population-Based Evolutionary Algorithm on a Function with Fitness Plateaus Using Elitism Levels Traverse Mechanism
Authors: Aram Ter-Sarkisov, Stephen Marsland
Date: 2012-04-11 00:00:00

In this article a tool for the analysis of population-based EAs is used to
derive asymptotic upper bounds on the optimization time of the algorithm
solving Royal Roads problem, a test function with plateaus of fitness. In
addition to this, limiting distribution of a certain subset of the population
is approximated.
Mixtures of Common Skew-t Factor Analyzers
Authors: Paula M. Murray, Paul D. McNicholas, Ryan P. Browne
Date: 2013-07-21 00:00:00
Tasks: Clustering, parameter estimation

A mixture of common skew-t factor analyzers model is introduced for
model-based clustering of high-dimensional data. By assuming common component
factor loadings, this model allows clustering to be performed in the presence
of a large number of mixture components or when the number of dimensions is too
large to be well-modelled by the mixtures of factor analyzers model or a
variant thereof. Furthermore, assuming that the component densities follow a
skew-t distribution allows robust clustering of skewed data. The alternating
expectation-conditional maximization algorithm is employed for parameter
estimation. We demonstrate excellent clustering performance when our model is
applied to real and simulated data.This paper marks the first time that skewed
common factors have been used.
Inconsistency of Pitman-Yor process mixtures for the number of components
Authors: Jeffrey W. Miller, Matthew T. Harrison
Date: 2013-08-30 00:00:00
Tasks: Density Estimation

In many applications, a finite mixture is a natural model, but it can be
difficult to choose an appropriate number of components. To circumvent this
choice, investigators are increasingly turning to Dirichlet process mixtures
(DPMs), and Pitman-Yor process mixtures (PYMs), more generally. While these
models may be well-suited for Bayesian density estimation, many investigators
are using them for inferences about the number of components, by considering
the posterior on the number of components represented in the observed data. We
show that this posterior is not consistent --- that is, on data from a finite
mixture, it does not concentrate at the true number of components. This result
applies to a large class of nonparametric mixtures, including DPMs and PYMs,
over a wide variety of families of component distributions, including
essentially all discrete families, as well as continuous exponential families
satisfying mild regularity conditions (such as multivariate Gaussians).
A Hypergraph-Partitioned Vertex Programming Approach for Large-scale Consensus Optimization
Authors: Hui Miao, Xiangyang Liu, Bert Huang, Lise Getoor
Date: 2013-08-30 00:00:00
Tasks: hypergraph partitioning

In modern data science problems, techniques for extracting value from big
data require performing large-scale optimization over heterogenous, irregularly
structured data. Much of this data is best represented as multi-relational
graphs, making vertex programming abstractions such as those of Pregel and
GraphLab ideal fits for modern large-scale data analysis. In this paper, we
describe a vertex-programming implementation of a popular consensus
optimization technique known as the alternating direction of multipliers
(ADMM). ADMM consensus optimization allows elegant solution of complex
objectives such as inference in rich probabilistic models. We also introduce a
novel hypergraph partitioning technique that improves over state-of-the-art
partitioning techniques for vertex programming and significantly reduces the
communication cost by reducing the number of replicated nodes up to an order of
magnitude. We implemented our algorithm in GraphLab and measure scaling
performance on a variety of realistic bipartite graph distributions and a large
synthetic voter-opinion analysis application. In our experiments, we are able
to achieve a 50% improvement in runtime over the current state-of-the-art
GraphLab partitioning scheme.
Concentration Inequalities for Bounded Random Vectors
Authors: Xinjia Chen
Date: 2013-08-30 00:00:00

We derive simple concentration inequalities for bounded random vectors, which
generalize Hoeffding's inequalities for bounded scalar random variables. As
applications, we apply the general results to multinomial and Dirichlet
distributions to obtain multivariate concentration inequalities.