pwc-archive/papers-with-abstracts huggingface dataset:
Convolutional Kernel Networks
Authors: Julien Mairal, Piotr Koniusz, Zaid Harchaoui, Cordelia Schmid
Date: 2014-06-12 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Image Classification

An important goal in visual recognition is to devise image representations
that are invariant to particular transformations. In this paper, we address
this goal with a new type of convolutional neural network (CNN) whose
invariance is encoded by a reproducing kernel. Unlike traditional approaches
where neural networks are learned either to represent data or for solving a
classification task, our network learns to approximate the kernel feature map
on training data. Such an approach enjoys several benefits over classical ones.
First, by teaching CNNs to be invariant, we obtain simple network architectures
that achieve a similar accuracy to more complex ones, while being easy to train
and robust to overfitting. Second, we bridge a gap between the neural network
literature and kernels, which are natural tools to model invariance. We
evaluate our methodology on visual recognition tasks where CNNs have proven to
perform well, e.g., digit recognition with the MNIST dataset, and the more
challenging CIFAR-10 and STL-10 datasets, where our accuracy is competitive
with the state of the art.
Deep Belief Network Training Improvement Using Elite Samples Minimizing Free Energy
Authors: Mohammad Ali Keyvanrad, Mohammad Mehdi Homayounpour
Date: 2014-11-14 00:00:00
Tasks: General Classification

Nowadays this is very popular to use deep architectures in machine learning.
Deep Belief Networks (DBNs) are deep architectures that use stack of Restricted
Boltzmann Machines (RBM) to create a powerful generative model using training
data. In this paper we present an improvement in a common method that is
usually used in training of RBMs. The new method uses free energy as a
criterion to obtain elite samples from generative model. We argue that these
samples can more accurately compute gradient of log probability of training
data. According to the results, an error rate of 0.99% was achieved on MNIST
test set. This result shows that the proposed method outperforms the method
presented in the first paper introducing DBN (1.25% error rate) and general
classification methods such as SVM (1.4% error rate) and KNN (with 1.6% error
rate). In another test using ISOLET dataset, letter classification error
dropped to 3.59% compared to 5.59% error rate achieved in those papers using
this dataset. The implemented method is available online at
"http://ceit.aut.ac.ir/~keyvanrad/DeeBNet Toolbox.html".
Supervised mid-level features for word image representation
Authors: Albert Gordo
Date: 2014-10-20 00:00:00
Proceeding: CVPR 2015 6
Tasks: BIG-bench Machine Learning, Descriptive, Retrieval

This paper addresses the problem of learning word image representations:
given the cropped image of a word, we are interested in finding a descriptive,
robust, and compact fixed-length representation. Machine learning techniques
can then be supplied with these representations to produce models useful for
word retrieval or recognition tasks. Although many works have focused on the
machine learning aspect once a global representation has been produced, little
work has been devoted to the construction of those base image representations:
most works use standard coding and aggregation techniques directly on top of
standard computer vision features such as SIFT or HOG.
  We propose to learn local mid-level features suitable for building word image
representations. These features are learnt by leveraging character bounding box
annotations on a small set of training images. However, contrary to other
approaches that use character bounding box information, our approach does not
rely on detecting the individual characters explicitly at testing time. Our
local mid-level features can then be aggregated to produce a global word image
signature. When pairing these features with the recent word attributes
framework of Almaz\'an et al., we obtain results comparable with or better than
the state-of-the-art on matching and recognition tasks using global descriptors
of only 96 dimensions.
A Unified Framework for Probabilistic Component Analysis
Authors: Mihalis A. Nicolaou, Stefanos Zafeiriou, Maja Pantic
Date: 2013-03-13 00:00:00

We present a unifying framework which reduces the construction of
probabilistic component analysis techniques to a mere selection of the latent
neighbourhood, thus providing an elegant and principled framework for creating
novel component analysis models as well as constructing probabilistic
equivalents of deterministic component analysis methods. Under our framework,
we unify many very popular and well-studied component analysis algorithms, such
as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA),
Locality Preserving Projections (LPP) and Slow Feature Analysis (SFA), some of
which have no probabilistic equivalents in literature thus far. We firstly
define the Markov Random Fields (MRFs) which encapsulate the latent
connectivity of the aforementioned component analysis techniques; subsequently,
we show that the projection directions produced by all PCA, LDA, LPP and SFA
are also produced by the Maximum Likelihood (ML) solution of a single joint
probability density function, composed by selecting one of the defined MRF
priors while utilising a simple observation model. Furthermore, we propose
novel Expectation Maximization (EM) algorithms, exploiting the proposed joint
PDF, while we generalize the proposed methodologies to arbitrary connectivities
via parameterizable MRF products. Theoretical analysis and experiments on both
simulated and real world data show the usefulness of the proposed framework, by
deriving methods which well outperform state-of-the-art equivalents.
rFerns: An Implementation of the Random Ferns Method for General-Purpose Machine Learning
Authors: Miron B. Kursa
Date: 2012-02-06 00:00:00
Tasks: Attribute, BIG-bench Machine Learning

In this paper I present an extended implementation of the Random ferns
algorithm contained in the R package rFerns. It differs from the original by
the ability of consuming categorical and numerical attributes instead of only
binary ones. Also, instead of using simple attribute subspace ensemble it
employs bagging and thus produce error approximation and variable importance
measure modelled after Random forest algorithm. I also present benchmarks'
results which show that although Random ferns' accuracy is mostly smaller than
achieved by Random forest, its speed and good quality of importance measure it
provides make rFerns a reasonable choice for a specific applications.
Sample-targeted clinical trial adaptation
Authors: Ognjen Arandjelovic
Date: 2014-11-14 00:00:00

Clinical trial adaptation refers to any adjustment of the trial protocol
after the onset of the trial. The main goal is to make the process of
introducing new medical interventions to patients more efficient by reducing
the cost and the time associated with evaluating their safety and efficacy. The
principal question is how should adaptation be performed so as to minimize the
chance of distorting the outcome of the trial. We propose a novel method for
achieving this. Unlike previous work our approach focuses on trial adaptation
by sample size adjustment. We adopt a recently proposed stratification
framework based on collected auxiliary data and show that this information
together with the primary measured variables can be used to make a
probabilistically informed choice of the particular sub-group a sample should
be removed from. Experiments on simulated data are used to illustrate the
effectiveness of our method and its application in practice.
Learning Fuzzy Controllers in Mobile Robotics with Embedded Preprocessing
Authors: I. Rodríguez-Fdez, M. Mucientes, A. Bugarín
Date: 2014-11-14 00:00:00

The automatic design of controllers for mobile robots usually requires two
stages. In the first stage,sensorial data are preprocessed or transformed into
high level and meaningful values of variables whichare usually defined from
expert knowledge. In the second stage, a machine learning technique is applied
toobtain a controller that maps these high level variables to the control
commands that are actually sent tothe robot. This paper describes an algorithm
that is able to embed the preprocessing stage into the learningstage in order
to get controllers directly starting from sensorial raw data with no expert
knowledgeinvolved. Due to the high dimensionality of the sensorial data, this
approach uses Quantified Fuzzy Rules(QFRs), that are able to transform
low-level input variables into high-level input variables, reducingthe
dimensionality through summarization. The proposed learning algorithm, called
Iterative QuantifiedFuzzy Rule Learning (IQFRL), is based on genetic
programming. IQFRL is able to learn rules with differentstructures, and can
manage linguistic variables with multiple granularities. The algorithm has been
testedwith the implementation of the wall-following behavior both in several
realistic simulated environmentswith different complexity and on a Pioneer 3-AT
robot in two real environments. Results have beencompared with several
well-known learning algorithms combined with different data
preprocessingtechniques, showing that IQFRL exhibits a better and statistically
significant performance. Moreover,three real world applications for which IQFRL
plays a central role are also presented: path and objecttracking with static
and moving obstacles avoidance.
Optimal Cost Almost-sure Reachability in POMDPs
Authors: Krishnendu Chatterjee, Martin Chmelík, Raghav Gupta, Ayush Kanodia
Date: 2014-11-14 00:00:00

We consider partially observable Markov decision processes (POMDPs) with a
set of target states and every transition is associated with an integer cost.
The optimization objective we study asks to minimize the expected total cost
till the target set is reached, while ensuring that the target set is reached
almost-surely (with probability 1). We show that for integer costs
approximating the optimal cost is undecidable. For positive costs, our results
are as follows: (i) we establish matching lower and upper bounds for the
optimal cost and the bound is double exponential; (ii) we show that the problem
of approximating the optimal cost is decidable and present approximation
algorithms developing on the existing algorithms for POMDPs with finite-horizon
objectives. While the worst-case running time of our algorithm is double
exponential, we also present efficient stopping criteria for the algorithm and
show experimentally that it performs well in many examples of interest.
Part Detector Discovery in Deep Convolutional Neural Networks
Authors: Marcel Simon, Erik Rodner, Joachim Denzler
Date: 2014-11-12 00:00:00
Tasks: General Classification

Current fine-grained classification approaches often rely on a robust
localization of object parts to extract localized feature representations
suitable for discrimination. However, part localization is a challenging task
due to the large variation of appearance and pose. In this paper, we show how
pre-trained convolutional neural networks can be used for robust and efficient
object part discovery and localization without the necessity to actually train
the network on the current dataset. Our approach called "part detector
discovery" (PDD) is based on analyzing the gradient maps of the network outputs
and finding activation centers spatially related to annotated semantic parts or
bounding boxes.
  This allows us not just to obtain excellent performance on the CUB200-2011
dataset, but in contrast to previous approaches also to perform detection and
bird classification jointly without requiring a given bounding box annotation
during testing and ground-truth parts during training. The code is available at
http://www.inf-cv.uni-jena.de/part_discovery and
https://github.com/cvjena/PartDetectorDisovery.
Statistical Models for Degree Distributions of Networks
Authors: Kayvan Sadeghi, Alessandro Rinaldo
Date: 2014-11-14 00:00:00
Tasks: parameter estimation

We define and study the statistical models in exponential family form whose
sufficient statistics are the degree distributions and the bi-degree
distributions of undirected labelled simple graphs. Graphs that are constrained
by the joint degree distributions are called $dK$-graphs in the computer
science literature and this paper attempts to provide the first statistically
grounded analysis of this type of models. In addition to formalizing these
models, we provide some preliminary results for the parameter estimation and
the asymptotic behaviour of the model for degree distribution, and discuss the
parameter estimation for the model for bi-degree distribution.
Integrating Fuzzy and Ant Colony System for Fuzzy Vehicle Routing Problem with Time Windows
Authors: Sandhya Bansal, V. Katiyar
Date: 2014-11-14 00:00:00

In this paper fuzzy VRPTW with an uncertain travel time is considered.
Credibility theory is used to model the problem and specifies a preference
index at which it is desired that the travel times to reach the customers fall
into their time windows. We propose the integration of fuzzy and ant colony
system based evolutionary algorithm to solve the problem while preserving the
constraints. Computational results for certain benchmark problems having short
and long time horizons are presented to show the effectiveness of the
algorithm. Comparison between different preferences indexes have been obtained
to help the user in making suitable decisions.
Stochastic Compositional Gradient Descent: Algorithms for Minimizing Compositions of Expected-Value Functions
Authors: Mengdi Wang, Ethan X. Fang, Han Liu
Date: 2014-11-14 00:00:00

Classical stochastic gradient methods are well suited for minimizing
expected-value objective functions. However, they do not apply to the
minimization of a nonlinear function involving expected values or a composition
of two expected-value functions, i.e., problems of the form $\min_x
\mathbf{E}_v [f_v\big(\mathbf{E}_w [g_w(x)]\big)]$. In order to solve this
stochastic composition problem, we propose a class of stochastic compositional
gradient descent (SCGD) algorithms that can be viewed as stochastic versions of
quasi-gradient method. SCGD update the solutions based on noisy sample
gradients of $f_v,g_{w}$ and use an auxiliary variable to track the unknown
quantity $\mathbf{E}_w[g_w(x)]$. We prove that the SCGD converge almost surely
to an optimal solution for convex optimization problems, as long as such a
solution exists. The convergence involves the interplay of two iterations with
different time scales. For nonsmooth convex problems, the SCGD achieve a
convergence rate of $O(k^{-1/4})$ in the general case and $O(k^{-2/3})$ in the
strongly convex case, after taking $k$ samples. For smooth convex problems, the
SCGD can be accelerated to converge at a rate of $O(k^{-2/7})$ in the general
case and $O(k^{-4/5})$ in the strongly convex case. For nonconvex problems, we
prove that any limit point generated by SCGD is a stationary point, for which
we also provide the convergence rate analysis. Indeed, the stochastic setting
where one wants to optimize compositions of expected-value functions is very
common in practice. The proposed SCGD methods find wide applications in
learning, estimation, dynamic programming, etc.
An Approach to Model Checking of Multi-agent Data Analysis
Authors: Natalia Garanina, Eugene Bodin, Elena Sidorova
Date: 2014-11-14 00:00:00

The paper presents an approach to verification of a multi-agent data analysis
algorithm. We base correct simulation of the multi-agent system by a finite
integer model. For verification we use model checking tool SPIN. Protocols of
agents are written in Promela language and properties of the multi-agent data
analysis system are expressed in logic LTL. We run several experiments with
SPIN and the model.
Asymmetric Minwise Hashing
Authors: Anshumali Shrivastava, Ping Li
Date: 2014-11-14 00:00:00
Tasks: Retrieval

Minwise hashing (Minhash) is a widely popular indexing scheme in practice.
Minhash is designed for estimating set resemblance and is known to be
suboptimal in many applications where the desired measure is set overlap (i.e.,
inner product between binary vectors) or set containment. Minhash has inherent
bias towards smaller sets, which adversely affects its performance in
applications where such a penalization is not desirable. In this paper, we
propose asymmetric minwise hashing (MH-ALSH), to provide a solution to this
problem. The new scheme utilizes asymmetric transformations to cancel the bias
of traditional minhash towards smaller sets, making the final "collision
probability" monotonic in the inner product. Our theoretical comparisons show
that for the task of retrieving with binary inner products asymmetric minhash
is provably better than traditional minhash and other recently proposed hashing
algorithms for general inner products. Thus, we obtain an algorithmic
improvement over existing approaches in the literature. Experimental
evaluations on four publicly available high-dimensional datasets validate our
claims and the proposed scheme outperforms, often significantly, other hashing
algorithms on the task of near neighbor retrieval with set containment. Our
proposal is simple and easy to implement in practice.
An Analysis of Random Projections in Cancelable Biometrics
Authors: Devansh Arpit, Ifeoma Nwogu, Gaurav Srivastava, Venu Govindaraju
Date: 2014-01-17 00:00:00
Tasks: Face Recognition

With increasing concerns about security, the need for highly secure physical
biometrics-based authentication systems utilizing \emph{cancelable biometric}
technologies is on the rise. Because the problem of cancelable template
generation deals with the trade-off between template security and matching
performance, many state-of-the-art algorithms successful in generating high
quality cancelable biometrics all have random projection as one of their early
processing steps. This paper therefore presents a formal analysis of why random
projections is an essential step in cancelable biometrics. By formally defining
the notion of an \textit{Independent Subspace Structure} for datasets, it can
be shown that random projection preserves the subspace structure of data
vectors generated from a union of independent linear subspaces. The bound on
the minimum number of random vectors required for this to hold is also derived
and is shown to depend logarithmically on the number of data samples, not only
in independent subspaces but in disjoint subspace settings as well. The
theoretical analysis presented is supported in detail with empirical results on
real-world face recognition datasets.
Tensor decompositions for learning latent variable models
Authors: Anima Anandkumar, Rong Ge, Daniel Hsu, Sham M. Kakade, Matus Telgarsky
Date: 2012-10-29 00:00:00
Tasks: parameter estimation

This work considers a computationally and statistically efficient parameter
estimation method for a wide class of latent variable models---including
Gaussian mixture models, hidden Markov models, and latent Dirichlet
allocation---which exploits a certain tensor structure in their low-order
observable moments (typically, of second- and third-order). Specifically,
parameter estimation is reduced to the problem of extracting a certain
(orthogonal) decomposition of a symmetric tensor derived from the moments; this
decomposition can be viewed as a natural generalization of the singular value
decomposition for matrices. Although tensor decompositions are generally
intractable to compute, the decomposition of these specially structured tensors
can be efficiently obtained by a variety of approaches, including power
iterations and maximization approaches (similar to the case of matrices). A
detailed analysis of a robust tensor power method is provided, establishing an
analogue of Wedin's perturbation theorem for the singular vectors of matrices.
This implies a robust and computationally tractable estimation approach for
several popular latent variable models.
Improved Asymmetric Locality Sensitive Hashing (ALSH) for Maximum Inner Product Search (MIPS)
Authors: Anshumali Shrivastava, Ping Li
Date: 2014-10-20 00:00:00

Recently it was shown that the problem of Maximum Inner Product Search (MIPS)
is efficient and it admits provably sub-linear hashing algorithms. Asymmetric
transformations before hashing were the key in solving MIPS which was otherwise
hard. In the prior work, the authors use asymmetric transformations which
convert the problem of approximate MIPS into the problem of approximate near
neighbor search which can be efficiently solved using hashing. In this work, we
provide a different transformation which converts the problem of approximate
MIPS into the problem of approximate cosine similarity search which can be
efficiently solved using signed random projections. Theoretical analysis show
that the new scheme is significantly better than the original scheme for MIPS.
Experimental evaluations strongly support the theoretical findings.
Jamming Bandits
Authors: SaiDhiraj Amuru, Cem Tekin, Mihaela van der Schaar, R. Michael Buehrer
Date: 2014-11-13 00:00:00
Tasks: Reinforcement Learning

Can an intelligent jammer learn and adapt to unknown environments in an
electronic warfare-type scenario? In this paper, we answer this question in the
positive, by developing a cognitive jammer that adaptively and optimally
disrupts the communication between a victim transmitter-receiver pair. We
formalize the problem using a novel multi-armed bandit framework where the
jammer can choose various physical layer parameters such as the signaling
scheme, power level and the on-off/pulsing duration in an attempt to obtain
power efficient jamming strategies. We first present novel online learning
algorithms to maximize the jamming efficacy against static transmitter-receiver
pairs and prove that our learning algorithm converges to the optimal (in terms
of the error rate inflicted at the victim and the energy used) jamming
strategy. Even more importantly, we prove that the rate of convergence to the
optimal jamming strategy is sub-linear, i.e. the learning is fast in comparison
to existing reinforcement learning algorithms, which is particularly important
in dynamically changing wireless environments. Also, we characterize the
performance of the proposed bandit-based learning algorithm against multiple
static and adaptive transmitter-receiver pairs.
DUM: Diversity-Weighted Utility Maximization for Recommendations
Authors: Azin Ashkan, Branislav Kveton, Shlomo Berkovsky, Zheng Wen
Date: 2014-11-13 00:00:00
Tasks: Diversity, Recommendation Systems

The need for diversification of recommendation lists manifests in a number of
recommender systems use cases. However, an increase in diversity may undermine
the utility of the recommendations, as relevant items in the list may be
replaced by more diverse ones. In this work we propose a novel method for
maximizing the utility of the recommended items subject to the diversity of
user's tastes, and show that an optimal solution to this problem can be found
greedily. We evaluate the proposed method in two online user studies as well as
in an offline analysis incorporating a number of evaluation metrics. The
results of evaluations show the superiority of our method over a number of
baselines.
Handling owl:sameAs via Rewriting
Authors: Boris Motik, Yavor Nenov, Robert Piro, Ian Horrocks
Date: 2014-11-13 00:00:00

Rewriting is widely used to optimise owl:sameAs reasoning in materialisation
based OWL 2 RL systems. We investigate issues related to both the correctness
and efficiency of rewriting, and present an algorithm that guarantees
correctness, improves efficiency, and can be effectively parallelised. Our
evaluation shows that our approach can reduce reasoning times on practical data
sets by orders of magnitude.