pwc-archive/papers-with-abstracts huggingface dataset:
Learning to Search Better Than Your Teacher
Authors: Kai-Wei Chang, Akshay Krishnamurthy, Alekh Agarwal, Hal Daumé III, John Langford
Date: 2015-02-08 00:00:00
Tasks: Multi-Armed Bandits, Structured Prediction

Methods for learning to search for structured prediction typically imitate a
reference policy, with existing theoretical guarantees demonstrating low regret
compared to that reference. This is unsatisfactory in many applications where
the reference policy is suboptimal and the goal of learning is to improve upon
it. Can learning to search work even when the reference is poor?
  We provide a new learning to search algorithm, LOLS, which does well relative
to the reference policy, but additionally guarantees low regret compared to
deviations from the learned policy: a local-optimality guarantee. Consequently,
LOLS can improve upon the reference policy, unlike previous algorithms. This
enables us to develop structured contextual bandits, a partial information
structured prediction setting with many potential applications.
Benchmarking KAZE and MCM for Multiclass Classification
Authors: Siddharth Srivastava, Prerana Mukherjee, Brejesh lall
Date: 2015-05-20 00:00:00
Tasks: Benchmarking, Classification, General Classification

In this paper, we propose a novel approach for feature generation by
appropriately fusing KAZE and SIFT features. We then use this feature set along
with Minimal Complexity Machine(MCM) for object classification. We show that
KAZE and SIFT features are complementary. Experimental results indicate that an
elementary integration of these techniques can outperform the state-of-the-art
approaches.
Multi-scale recognition with DAG-CNNs
Authors: Songfan Yang, Deva Ramanan
Date: 2015-05-20 00:00:00
Proceeding: ICCV 2015 12
Tasks: Classification, General Classification, image-classification, Image Classification

We explore multi-scale convolutional neural nets (CNNs) for image
classification. Contemporary approaches extract features from a single output
layer. By extracting features from multiple layers, one can simultaneously
reason about high, mid, and low-level features during classification. The
resulting multi-scale architecture can itself be seen as a feed-forward model
that is structured as a directed acyclic graph (DAG-CNNs). We use DAG-CNNs to
learn a set of multiscale features that can be effectively shared between
coarse and fine-grained classification tasks. While fine-tuning such models
helps performance, we show that even "off-the-self" multiscale features perform
quite well. We present extensive analysis and demonstrate state-of-the-art
classification performance on three standard scene benchmarks (SUN397, MIT67,
and Scene15). In terms of the heavily benchmarked MIT67 and Scene15 datasets,
our results reduce the lowest previously-reported error by 23.9% and 9.5%,
respectively.
Bounds on the Minimax Rate for Estimating a Prior over a VC Class from Independent Learning Tasks
Authors: Liu Yang, Steve Hanneke, Jaime Carbonell
Date: 2015-05-20 00:00:00
Tasks: Transfer Learning

We study the optimal rates of convergence for estimating a prior distribution
over a VC class from a sequence of independent data sets respectively labeled
by independent target functions sampled from the prior. We specifically derive
upper and lower bounds on the optimal rates under a smoothness condition on the
correct prior, with the number of samples per data set equal the VC dimension.
These results have implications for the improvements achievable via transfer
learning. We additionally extend this setting to real-valued function, where we
establish consistency of an estimator for the prior, and discuss an additional
application to a preference elicitation problem in algorithmic economics.
Variable subset selection via GA and information complexity in mixtures of Poisson and negative binomial regression models
Authors: T. J. Massaro, H. Bozdogan
Date: 2015-05-20 00:00:00
Tasks: Epidemiology, regression

Count data, for example the number of observed cases of a disease in a city,
often arise in the fields of healthcare analytics and epidemiology. In this
paper, we consider performing regression on multivariate data in which our
outcome is a count. Specifically, we derive log-likelihood functions for finite
mixtures of regression models involving counts that come from a Poisson
distribution, as well as a negative binomial distribution when the counts are
significantly overdispersed. Within our proposed modeling framework, we carry
out optimal component selection using the information criteria scores AIC, BIC,
CAIC, and ICOMP. We demonstrate applications of our approach on simulated data,
as well as on a real data set of HIV cases in Tennessee counties from the year
2010. Finally, using a genetic algorithm within our framework, we perform
variable subset selection to determine the covariates that are most responsible
for categorizing Tennessee counties. This leads to some interesting insights
into the traits of counties that have high HIV counts.
Image aesthetic evaluation using paralleled deep convolution neural network
Authors: Guo Lihua, Li Fudi
Date: 2015-05-20 00:00:00

Image aesthetic evaluation has attracted much attention in recent years.
Image aesthetic evaluation methods heavily depend on the effective aesthetic
feature. Traditional meth-ods always extract hand-crafted features. However,
these hand-crafted features are always designed to adapt particu-lar datasets,
and extraction of them needs special design. Rather than extracting
hand-crafted features, an automati-cally learn of aesthetic features based on
deep convolutional neural network (DCNN) is first adopt in this paper. As we
all know, when the training dataset is given, the DCNN architecture with high
complexity may meet the over-fitting problem. On the other side, the DCNN
architecture with low complexity would not efficiently extract effective
features. For these reasons, we further propose a paralleled convolutional
neural network (PDCNN) with multi-level structures to automatically adapt to
the training dataset. Experimental results show that our proposed PDCNN
architecture achieves better performance than other traditional methods.
Convergence Analysis of Policy Iteration
Authors: Ali Heydari
Date: 2015-05-20 00:00:00

Adaptive optimal control of nonlinear dynamic systems with deterministic and
known dynamics under a known undiscounted infinite-horizon cost function is
investigated. Policy iteration scheme initiated using a stabilizing initial
control is analyzed in solving the problem. The convergence of the iterations
and the optimality of the limit functions, which follows from the established
uniqueness of the solution to the Bellman equation, are the main results of
this study. Furthermore, a theoretical comparison between the speed of
convergence of policy iteration versus value iteration is presented. Finally,
the convergence results are extended to the case of multi-step look-ahead
policy iteration.
Learning with a Drifting Target Concept
Authors: Steve Hanneke, Varun Kanade, Liu Yang
Date: 2015-05-20 00:00:00
Tasks: Active Learning

We study the problem of learning in the presence of a drifting target
concept. Specifically, we provide bounds on the error rate at a given time,
given a learner with access to a history of independent samples labeled
according to a target concept that can change on each round. One of our main
contributions is a refinement of the best previous results for polynomial-time
algorithms for the space of linear separators under a uniform distribution. We
also provide general results for an algorithm capable of adapting to a variable
rate of drift of the target concept. Some of the results also describe an
active learning variant of this setting, and provide bounds on the number of
queries for the labels of points in the sequence sufficient to obtain the
stated bounds on the error rates.
Extrinsic Methods for Coding and Dictionary Learning on Grassmann Manifolds
Authors: Mehrtash Harandi, Richard Hartley, Chunhua Shen, Brian Lovell, Conrad Sanderson
Date: 2014-01-31 00:00:00
Tasks: Action Recognition, Classification, Dictionary Learning, Face Recognition, General Classification, Graph Embedding, Temporal Action Localization, Texture Classification

Sparsity-based representations have recently led to notable results in
various visual recognition tasks. In a separate line of research, Riemannian
manifolds have been shown useful for dealing with features and models that do
not lie in Euclidean spaces. With the aim of building a bridge between the two
realms, we address the problem of sparse coding and dictionary learning over
the space of linear subspaces, which form Riemannian structures known as
Grassmann manifolds. To this end, we propose to embed Grassmann manifolds into
the space of symmetric matrices by an isometric mapping. This in turn enables
us to extend two sparse coding schemes to Grassmann manifolds. Furthermore, we
propose closed-form solutions for learning a Grassmann dictionary, atom by
atom. Lastly, to handle non-linearity in data, we extend the proposed Grassmann
sparse coding and dictionary learning algorithms through embedding into Hilbert
spaces.
  Experiments on several classification tasks (gender recognition, gesture
classification, scene analysis, face recognition, action recognition and
dynamic texture classification) show that the proposed approaches achieve
considerable improvements in discrimination accuracy, in comparison to
state-of-the-art methods such as kernelized Affine Hull Method and
graph-embedding Grassmann discriminant analysis.
Barcode Annotations for Medical Image Retrieval: A Preliminary Investigation
Authors: Hamid. R. Tizhoosh
Date: 2015-05-19 00:00:00
Tasks: General Classification, Image Retrieval, Medical Image Retrieval, Retrieval

This paper proposes to generate and to use barcodes to annotate medical
images and/or their regions of interest such as organs, tumors and tissue
types. A multitude of efficient feature-based image retrieval methods already
exist that can assign a query image to a certain image class. Visual
annotations may help to increase the retrieval accuracy if combined with
existing feature-based classification paradigms. Whereas with annotations we
usually mean textual descriptions, in this paper barcode annotations are
proposed. In particular, Radon barcodes (RBC) are introduced. As well, local
binary patterns (LBP) and local Radon binary patterns (LRBP) are implemented as
barcodes. The IRMA x-ray dataset with 12,677 training images and 1,733 test
images is used to verify how barcodes could facilitate image retrieval.
oASIS: Adaptive Column Sampling for Kernel Matrix Approximation
Authors: Raajen Patel, Thomas A. Goldstein, Eva L. Dyer, Azalia Mirhoseini, Richard G. Baraniuk
Date: 2015-05-19 00:00:00
Tasks: Clustering, Dimensionality Reduction, General Classification

Kernel matrices (e.g. Gram or similarity matrices) are essential for many
state-of-the-art approaches to classification, clustering, and dimensionality
reduction. For large datasets, the cost of forming and factoring such kernel
matrices becomes intractable. To address this challenge, we introduce a new
adaptive sampling algorithm called Accelerated Sequential Incoherence Selection
(oASIS) that samples columns without explicitly computing the entire kernel
matrix. We provide conditions under which oASIS is guaranteed to exactly
recover the kernel matrix with an optimal number of columns selected. Numerical
experiments on both synthetic and real-world datasets demonstrate that oASIS
achieves performance comparable to state-of-the-art adaptive sampling methods
at a fraction of the computational cost. The low runtime complexity of oASIS
and its low memory footprint enable the solution of large problems that are
simply intractable using other adaptive methods.
Image Reconstruction from Bag-of-Visual-Words
Authors: Hiroharu Kato, Tatsuya Harada
Date: 2015-05-19 00:00:00
Proceeding: CVPR 2014 6
Tasks: Image Reconstruction, Retrieval

The objective of this work is to reconstruct an original image from
Bag-of-Visual-Words (BoVW). Image reconstruction from features can be a means
of identifying the characteristics of features. Additionally, it enables us to
generate novel images via features. Although BoVW is the de facto standard
feature for image recognition and retrieval, successful image reconstruction
from BoVW has not been reported yet. What complicates this task is that BoVW
lacks the spatial information for including visual words. As described in this
paper, to estimate an original arrangement, we propose an evaluation function
that incorporates the naturalness of local adjacency and the global position,
with a method to obtain related parameters using an external image database. To
evaluate the performance of our method, we reconstruct images of objects of 101
kinds. Additionally, we apply our method to analyze object classifiers and to
generate novel images via BoVW.
Posterior Contraction Rates of the Phylogenetic Indian Buffet Processes
Authors: Mengjie Chen, Chao GAO, Hongyu Zhao
Date: 2013-07-31 00:00:00

By expressing prior distributions as general stochastic processes,
nonparametric Bayesian methods provide a flexible way to incorporate prior
knowledge and constrain the latent structure in statistical inference. The
Indian buffet process (IBP) is such an example that can be used to define a
prior distribution on infinite binary features, where the exchangeability among
subjects is assumed. The phylogenetic Indian buffet process (pIBP), a
derivative of IBP, enables the modeling of non-exchangeability among subjects
through a stochastic process on a rooted tree, which is similar to that used in
phylogenetics, to describe relationships among the subjects. In this paper, we
study the theoretical properties of IBP and pIBP under a binary factor model.
We establish the posterior contraction rates for both IBP and pIBP and
substantiate the theoretical results through simulation studies. This is the
first work addressing the frequentist property of the posterior behaviors of
IBP and pIBP. We also demonstrated its practical usefulness by applying pIBP
prior to a real data example arising in the field of cancer genomics where the
exchangeability among subjects is violated.
Non-Gaussian Discriminative Factor Models via the Max-Margin Rank-Likelihood
Authors: Xin Yuan, Ricardo Henao, Ephraim L. Tsalik, Raymond J. Langley, Lawrence Carin
Date: 2015-04-28 00:00:00

We consider the problem of discriminative factor analysis for data that are
in general non-Gaussian. A Bayesian model based on the ranks of the data is
proposed. We first introduce a new {\em max-margin} version of the
rank-likelihood. A discriminative factor model is then developed, integrating
the max-margin rank-likelihood and (linear) Bayesian support vector machines,
which are also built on the max-margin principle. The discriminative factor
model is further extended to the {\em nonlinear} case through mixtures of local
linear classifiers, via Dirichlet processes. Fully local conjugacy of the model
yields efficient inference with both Markov Chain Monte Carlo and variational
Bayes approaches. Extensive experiments on benchmark and real data demonstrate
superior performance of the proposed model and its potential for applications
in computational biology.
Vector-Space Markov Random Fields via Exponential Families
Authors: Wesley Tansey, Oscar Hernan Madrid Padilla, Arun Sai Suggala, Pradeep Ravikumar
Date: 2015-05-19 00:00:00

We present Vector-Space Markov Random Fields (VS-MRFs), a novel class of
undirected graphical models where each variable can belong to an arbitrary
vector space. VS-MRFs generalize a recent line of work on scalar-valued,
uni-parameter exponential family and mixed graphical models, thereby greatly
broadening the class of exponential families available (e.g., allowing
multinomial and Dirichlet distributions). Specifically, VS-MRFs are the joint
graphical model distributions where the node-conditional distributions belong
to generic exponential families with general vector space domains. We also
present a sparsistent $M$-estimator for learning our class of MRFs that
recovers the correct set of edges with high probability. We validate our
approach via a set of synthetic data experiments as well as a real-world case
study of over four million foods from the popular diet tracking app
MyFitnessPal. Our results demonstrate that our algorithm performs well
empirically and that VS-MRFs are capable of capturing and highlighting
interesting structure in complex, real-world data. All code for our algorithm
is open source and publicly available.
EpicFlow: Edge-Preserving Interpolation of Correspondences for Optical Flow
Authors: Jerome Revaud, Philippe Weinzaepfel, Zaid Harchaoui, Cordelia Schmid
Date: 2015-01-12 00:00:00
Proceeding: CVPR 2015 6
Tasks: Optical Flow Estimation

We propose a novel approach for optical flow estimation , targeted at large
displacements with significant oc-clusions. It consists of two steps: i) dense
matching by edge-preserving interpolation from a sparse set of matches; ii)
variational energy minimization initialized with the dense matches. The
sparse-to-dense interpolation relies on an appropriate choice of the distance,
namely an edge-aware geodesic distance. This distance is tailored to handle
occlusions and motion boundaries -- two common and difficult issues for optical
flow computation. We also propose an approximation scheme for the geodesic
distance to allow fast computation without loss of performance. Subsequent to
the dense interpolation step, standard one-level variational energy
minimization is carried out on the dense matches to obtain the final flow
estimation. The proposed approach, called Edge-Preserving Interpolation of
Correspondences (EpicFlow) is fast and robust to large displacements. It
significantly outperforms the state of the art on MPI-Sintel and performs on
par on Kitti and Middlebury.
Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
Authors: Tim Salimans, Diederik P. Kingma, Max Welling
Date: 2014-10-23 00:00:00
Tasks: Bayesian Inference, Variational Inference

Recent advances in stochastic gradient variational inference have made it
possible to perform variational Bayesian inference with posterior
approximations containing auxiliary random variables. This enables us to
explore a new synthesis of variational inference and Monte Carlo methods where
we incorporate one or more steps of MCMC into our variational approximation. By
doing so we obtain a rich class of inference algorithms bridging the gap
between variational methods and MCMC, and offering the best of both worlds:
fast posterior approximation through the maximization of an explicit objective,
with the option of trading off additional computation for additional accuracy.
We describe the theoretical foundations that make this possible and show some
promising first results.
Risk and Regret of Hierarchical Bayesian Learners
Authors: Jonathan H. Huggins, Joshua B. Tenenbaum
Date: 2015-05-19 00:00:00
Tasks: feature selection

Common statistical practice has shown that the full power of Bayesian methods
is not realized until hierarchical priors are used, as these allow for greater
"robustness" and the ability to "share statistical strength." Yet it is an
ongoing challenge to provide a learning-theoretically sound formalism of such
notions that: offers practical guidance concerning when and how best to utilize
hierarchical models; provides insights into what makes for a good hierarchical
prior; and, when the form of the prior has been chosen, can guide the choice of
hyperparameter settings. We present a set of analytical tools for understanding
hierarchical priors in both the online and batch learning settings. We provide
regret bounds under log-loss, which show how certain hierarchical models
compare, in retrospect, to the best single model in the model class. We also
show how to convert a Bayesian log-loss regret bound into a Bayesian risk bound
for any bounded loss, a result which may be of independent interest. Risk and
regret bounds for Student's $t$ and hierarchical Gaussian priors allow us to
formalize the concepts of "robustness" and "sharing statistical strength."
Priors for feature selection are investigated as well. Our results suggest that
the learning-theoretic benefits of using hierarchical priors can often come at
little cost on practical problems.
Multi-task additive models with shared transfer functions based on dictionary learning
Authors: Alhussein Fawzi, Mathieu Sinn, Pascal Frossard
Date: 2015-05-19 00:00:00
Tasks: Additive models, Dictionary Learning, Multi-Task Learning, regression

Additive models form a widely popular class of regression models which
represent the relation between covariates and response variables as the sum of
low-dimensional transfer functions. Besides flexibility and accuracy, a key
benefit of these models is their interpretability: the transfer functions
provide visual means for inspecting the models and identifying domain-specific
relations between inputs and outputs. However, in large-scale problems
involving the prediction of many related tasks, learning independently additive
models results in a loss of model interpretability, and can cause overfitting
when training data is scarce. We introduce a novel multi-task learning approach
which provides a corpus of accurate and interpretable additive models for a
large number of related forecasting tasks. Our key idea is to share transfer
functions across models in order to reduce the model complexity and ease the
exploration of the corpus. We establish a connection with sparse dictionary
learning and propose a new efficient fitting algorithm which alternates between
sparse coding and transfer function updates. The former step is solved via an
extension of Orthogonal Matching Pursuit, whose properties are analyzed using a
novel recovery condition which extends existing results in the literature. The
latter step is addressed using a traditional dictionary update rule.
Experiments on real-world data demonstrate that our approach compares favorably
to baseline methods while yielding an interpretable corpus of models, revealing
structure among the individual tasks and being more robust when training data
is scarce. Our framework therefore extends the well-known benefits of additive
models to common regression settings possibly involving thousands of tasks.