pwc-archive/papers-with-abstracts huggingface dataset:
Invariants of objects and their images under surjective maps
Authors: Irina A. Kogan, Peter J. Olver
Date: 2015-09-22 00:00:00

We examine the relationships between the differential invariants of objects
and of their images under a surjective map. We analyze both the case when the
underlying transformation group is projectable and hence induces an action on
the image, and the case when only a proper subgroup of the entire group acts
projectably. In the former case, we establish a constructible isomorphism
between the algebra of differential invariants of the images and the algebra of
fiber-wise constant (gauge) differential invariants of the objects. In the
latter case, we describe residual effects of the full transformation group on
the image invariants. Our motivation comes from the problem of reconstruction
of an object from multiple-view images, with central and parallel projections
of curves from three-dimensional space to the two-dimensional plane serving as
our main examples.
Classification error in multiclass discrimination from Markov data
Authors: Sören Christensen, Albrecht Irle, Lars Willert
Date: 2015-09-22 00:00:00
Tasks: Classification, General Classification

As a model for an on-line classification setting we consider a stochastic
process $(X_{-n},Y_{-n})_{n}$, the present time-point being denoted by 0, with
observables $ \ldots,X_{-n},X_{-n+1},\ldots, X_{-1}, X_0$ from which the
pattern $Y_0$ is to be inferred. So in this classification setting, in addition
to the present observation $X_0$ a number $l$ of preceding observations may be
used for classification, thus taking a possible dependence structure into
account as it occurs e.g. in an ongoing classification of handwritten
characters. We treat the question how the performance of classifiers is
improved by using such additional information. For our analysis, a hidden
Markov model is used. Letting $R_l$ denote the minimal risk of
misclassification using $l$ preceding observations we show that the difference
$\sup_k |R_l - R_{l+k}|$ decreases exponentially fast as $l$ increases. This
suggests that a small $l$ might already lead to a noticeable improvement. To
follow this point we look at the use of past observations for kernel
classification rules. Our practical findings in simulated hidden Markov models
and in the classification of handwritten characters indicate that using $l=1$,
i.e. just the last preceding observation in addition to $X_0$, can lead to a
substantial reduction of the risk of misclassification. So, in the presence of
stochastic dependencies, we advocate to use $ X_{-1},X_0$ for finding the
pattern $Y_0$ instead of only $X_0$ as one would in the independent situation.
Learning quantitative sequence-function relationships from massively parallel experiments
Authors: Gurinder S. Atwal, Justin B. Kinney
Date: 2015-05-30 00:00:00

A fundamental aspect of biological information processing is the ubiquity of
sequence-function relationships -- functions that map the sequence of DNA, RNA,
or protein to a biochemically relevant activity. Most sequence-function
relationships in biology are quantitative, but only recently have experimental
techniques for effectively measuring these relationships been developed. The
advent of such "massively parallel" experiments presents an exciting
opportunity for the concepts and methods of statistical physics to inform the
study of biological systems. After reviewing these recent experimental
advances, we focus on the problem of how to infer parametric models of
sequence-function relationships from the data produced by these experiments.
Specifically, we retrace and extend recent theoretical work showing that
inference based on mutual information, not the standard likelihood-based
approach, is often necessary for accurately learning the parameters of these
models. Closely connected with this result is the emergence of "diffeomorphic
modes" -- directions in parameter space that are far less constrained by data
than likelihood-based inference would suggest. Analogous to Goldstone modes in
physics, diffeomorphic modes arise from an arbitrarily broken symmetry of the
inference problem. An analytically tractable model of a massively parallel
experiment is then described, providing an explicit demonstration of these
fundamental aspects of statistical inference. This paper concludes with an
outlook on the theoretical and computational challenges currently facing
studies of quantitative sequence-function relationships.
Robust Optimization for Deep Regression
Authors: Vasileios Belagiannis, Christian Rupprecht, Gustavo Carneiro, Nassir Navab
Date: 2015-05-25 00:00:00
Proceeding: ICCV 2015 12
Tasks: Age Estimation, object-detection, Object Detection, Pose Estimation, regression

Convolutional Neural Networks (ConvNets) have successfully contributed to
improve the accuracy of regression-based methods for computer vision tasks such
as human pose estimation, landmark localization, and object detection. The
network optimization has been usually performed with L2 loss and without
considering the impact of outliers on the training process, where an outlier in
this context is defined by a sample estimation that lies at an abnormal
distance from the other training sample estimations in the objective space. In
this work, we propose a regression model with ConvNets that achieves robustness
to such outliers by minimizing Tukey's biweight function, an M-estimator robust
to outliers, as the loss function for the ConvNet. In addition to the robust
loss, we introduce a coarse-to-fine model, which processes input images of
progressively higher resolutions for improving the accuracy of the regressed
values. In our experiments, we demonstrate faster convergence and better
generalization of our robust loss function for the tasks of human pose
estimation and age estimation from face images. We also show that the
combination of the robust loss function with the coarse-to-fine model produces
comparable or better results than current state-of-the-art approaches in four
publicly available human pose estimation datasets.
A Compositional Explanation of the Pet Fish Phenomenon
Authors: Bob Coecke, Martha Lewis
Date: 2015-09-22 00:00:00

The `pet fish' phenomenon is often cited as a paradigm example of the
`non-compositionality' of human concept use. We show here how this phenomenon
is naturally accommodated within a compositional distributional model of
meaning. This model describes the meaning of a composite concept by accounting
for interaction between its constituents via their grammatical roles. We give
two illustrative examples to show how the qualitative phenomena are exhibited.
We go on to apply the model to experimental data, and finally discuss
extensions of the formalism.
Graph Kernels exploiting Weisfeiler-Lehman Graph Isomorphism Test Extensions
Authors: Giovanni Da San Martino, Nicolò Navarin, Alessandro Sperduti
Date: 2015-09-22 00:00:00

In this paper we present a novel graph kernel framework inspired the by the
Weisfeiler-Lehman (WL) isomorphism tests. Any WL test comprises a relabelling
phase of the nodes based on test-specific information extracted from the graph,
for example the set of neighbours of a node. We defined a novel relabelling and
derived two kernels of the framework from it. The novel kernels are very fast
to compute and achieve state-of-the-art results on five real-world datasets.
Local Multi-Grouped Binary Descriptor with Ring-based Pooling Configuration and Optimization
Authors: Yongqiang Gao, Weilin Huang, Yu Qiao
Date: 2015-09-22 00:00:00

Local binary descriptors are attracting increasingly attention due to their
great advantages in computational speed, which are able to achieve real-time
performance in numerous image/vision applications. Various methods have been
proposed to learn data-dependent binary descriptors. However, most existing
binary descriptors aim overly at computational simplicity at the expense of
significant information loss which causes ambiguity in similarity measure using
Hamming distance. In this paper, by considering multiple features might share
complementary information, we present a novel local binary descriptor, referred
as Ring-based Multi-Grouped Descriptor (RMGD), to successfully bridge the
performance gap between current binary and floated-point descriptors. Our
contributions are two-fold. Firstly, we introduce a new pooling configuration
based on spatial ring-region sampling, allowing for involving binary tests on
the full set of pairwise regions with different shapes, scales and distances.
This leads to a more meaningful description than existing methods which
normally apply a limited set of pooling configurations. Then, an extended
Adaboost is proposed for efficient bit selection by emphasizing high variance
and low correlation, achieving a highly compact representation. Secondly, the
RMGD is computed from multiple image properties where binary strings are
extracted. We cast multi-grouped features integration as rankSVM or sparse SVM
learning problem, so that different features can compensate strongly for each
other, which is the key to discriminativeness and robustness. The performance
of RMGD was evaluated on a number of publicly available benchmarks, where the
RMGD outperforms the state-of-the-art binary descriptors significantly.
A survey on feature weighting based K-Means algorithms
Authors: Renato Cordeiro de Amorim
Date: 2015-09-22 00:00:00
Tasks: Clustering, Survey

In a real-world data set there is always the possibility, rather high in our
opinion, that different features may have different degrees of relevance. Most
machine learning algorithms deal with this fact by either selecting or
deselecting features in the data preprocessing phase. However, we maintain that
even among relevant features there may be different degrees of relevance, and
this should be taken into account during the clustering process. With over 50
years of history, K-Means is arguably the most popular partitional clustering
algorithm there is. The first K-Means based clustering algorithm to compute
feature weights was designed just over 30 years ago. Various such algorithms
have been designed since but there has not been, to our knowledge, a survey
integrating empirical evidence of cluster recovery ability, common flaws, and
possible directions for future research. This paper elaborates on the concept
of feature weighting and addresses these issues by critically analysing some of
the most popular, or innovative, feature weighting mechanisms based in K-Means.
Modifying iterated Laplace approximations
Authors: Tiep Mai, Simon Wilson
Date: 2015-09-22 00:00:00

In this paper, several modifications are introduced to the functional
approximation method iterLap to reduce the approximation error, including
stopping rule adjustment, proposal of new residual function, starting point
selection for numerical optimisation, scaling of Hessian matrix. Illustrative
examples are also provided to show the trade-off between running time and
accuracy of the original and modified methods.
Bayesian Conditional Density Filtering
Authors: Shaan Qamar, Rajarshi Guhaniyogi, David B. Dunson
Date: 2014-01-15 00:00:00
Tasks: Bayesian Inference

We propose a Conditional Density Filtering (C-DF) algorithm for efficient
online Bayesian inference. C-DF adapts MCMC sampling to the online setting,
sampling from approximations to conditional posterior distributions obtained by
propagating surrogate conditional sufficient statistics (a function of data and
parameter estimates) as new data arrive. These quantities eliminate the need to
store or process the entire dataset simultaneously and offer a number of
desirable features. Often, these include a reduction in memory requirements and
runtime and improved mixing, along with state-of-the-art parameter inference
and prediction. These improvements are demonstrated through several
illustrative examples including an application to high dimensional compressed
regression. Finally, we show that C-DF samples converge to the target posterior
distribution asymptotically as sampling proceeds and more data arrives.
Understand Scene Categories by Objects: A Semantic Regularized Scene Classifier Using Convolutional Neural Networks
Authors: Yiyi Liao, Sarath Kodagoda, Yue Wang, Lei Shi, Yong liu
Date: 2015-09-22 00:00:00
Tasks: Classification, Diversity, General Classification, Object, Scene Classification, Scene Understanding, Semantic Segmentation

Scene classification is a fundamental perception task for environmental
understanding in today's robotics. In this paper, we have attempted to exploit
the use of popular machine learning technique of deep learning to enhance scene
understanding, particularly in robotics applications. As scene images have
larger diversity than the iconic object images, it is more challenging for deep
learning methods to automatically learn features from scene images with less
samples. Inspired by human scene understanding based on object knowledge, we
address the problem of scene classification by encouraging deep neural networks
to incorporate object-level information. This is implemented with a
regularization of semantic segmentation. With only 5 thousand training images,
as opposed to 2.5 million images, we show the proposed deep architecture
achieves superior scene classification results to the state-of-the-art on a
publicly available SUN RGB-D dataset. In addition, performance of semantic
segmentation, the regularizer, also reaches a new record with refinement
derived from predicted scene labels. Finally, we apply our SUN RGB-D dataset
trained model to a mobile robot captured images to classify scenes in our
university demonstrating the generalization ability of the proposed algorithm.
Stochastic gradient descent methods for estimation with large data sets
Authors: Dustin Tran, Panos Toulis, Edoardo M. Airoldi
Date: 2015-09-22 00:00:00
Tasks: parameter estimation

We develop methods for parameter estimation in settings with large-scale data
sets, where traditional methods are no longer tenable. Our methods rely on
stochastic approximations, which are computationally efficient as they maintain
one iterate as a parameter estimate, and successively update that iterate based
on a single data point. When the update is based on a noisy gradient, the
stochastic approximation is known as standard stochastic gradient descent,
which has been fundamental in modern applications with large data sets.
Additionally, our methods are numerically stable because they employ implicit
updates of the iterates. Intuitively, an implicit update is a shrinked version
of a standard one, where the shrinkage factor depends on the observed Fisher
information at the corresponding data point. This shrinkage prevents numerical
divergence of the iterates, which can be caused either by excess noise or
outliers. Our sgd package in R offers the most extensive and robust
implementation of stochastic gradient descent methods. We demonstrate that sgd
dominates alternative software in runtime for several estimation problems with
massive data sets. Our applications include the wide class of generalized
linear models as well as M-estimation for robust regression.
Harmonic Extension
Authors: Zuoqiang Shi, Jian Sun, Minghao Tian
Date: 2015-09-22 00:00:00
Tasks: BIG-bench Machine Learning

In this paper, we consider the harmonic extension problem, which is widely
used in many applications of machine learning. We find that the transitional
method of graph Laplacian fails to produce a good approximation of the
classical harmonic function. To tackle this problem, we propose a new method
called the point integral method (PIM). We consider the harmonic extension
problem from the point of view of solving PDEs on manifolds. The basic idea of
the PIM method is to approximate the harmonicity using an integral equation,
which is easy to be discretized from points. Based on the integral equation, we
explain the reason why the transitional graph Laplacian may fail to approximate
the harmonicity in the classical sense and propose a different approach which
we call the volume constraint method (VCM). Theoretically, both the PIM and the
VCM computes a harmonic function with convergence guarantees, and practically,
they are both simple, which amount to solve a linear system. One important
application of the harmonic extension in machine learning is semi-supervised
learning. We run a popular semi-supervised learning algorithm by Zhu et al.
over a couple of well-known datasets and compare the performance of the
aforementioned approaches. Our experiments show the PIM performs the best.
From Facial Parts Responses to Face Detection: A Deep Learning Approach
Authors: Shuo Yang, Ping Luo, Chen Change Loy, Xiaoou Tang
Date: 2015-09-22 00:00:00
Proceeding: ICCV 2015 12
Tasks: Deep Learning, Face Detection

In this paper, we propose a novel deep convolutional network (DCN) that
achieves outstanding performance on FDDB, PASCAL Face, and AFW. Specifically,
our method achieves a high recall rate of 90.99% on the challenging FDDB
benchmark, outperforming the state-of-the-art method by a large margin of
2.91%. Importantly, we consider finding faces from a new perspective through
scoring facial parts responses by their spatial structure and arrangement. The
scoring mechanism is carefully formulated considering challenging cases where
faces are only partially visible. This consideration allows our network to
detect faces under severe occlusion and unconstrained pose variation, which are
the main difficulty and bottleneck of most existing face detection approaches.
We show that despite the use of DCN, our network can achieve practical runtime
speed.
Efficient Neighborhood Selection for Gaussian Graphical Models
Authors: Yingxiang Yang, Jalal Etesami, Negar Kiyavash
Date: 2015-09-22 00:00:00

This paper addresses the problem of neighborhood selection for Gaussian
graphical models. We present two heuristic algorithms: a forward-backward
greedy algorithm for general Gaussian graphical models based on mutual
information test, and a threshold-based algorithm for walk summable Gaussian
graphical models. Both algorithms are shown to be structurally consistent, and
efficient. Numerical results show that both algorithms work very well.
Twitter User Geolocation Using a Unified Text and Network Prediction Model
Authors: Afshin Rahimi, Trevor Cohn, Timothy Baldwin
Date: 2015-06-27 00:00:00
Proceeding: IJCNLP 2015 7

We propose a label propagation approach to geolocation prediction based on
Modified Adsorption, with two enhancements:(1) the removal of "celebrity" nodes
to increase location homophily and boost tractability, and (2) he incorporation
of text-based geolocation priors for test users. Experiments over three Twitter
benchmark datasets achieve state-of-the-art results, and demonstrate the
effectiveness of the enhancements.
Love Thy Neighbors: Image Annotation by Exploiting Image Metadata
Authors: Justin Johnson, Lamberto Ballan, Fei-Fei Li
Date: 2015-08-30 00:00:00
Proceeding: ICCV 2015 12

Some images that are difficult to recognize on their own may become more
clear in the context of a neighborhood of related images with similar
social-network metadata. We build on this intuition to improve multilabel image
annotation. Our model uses image metadata nonparametrically to generate
neighborhoods of related images using Jaccard similarities, then uses a deep
neural network to blend visual information from the image and its neighbors.
Prior work typically models image metadata parametrically, in contrast, our
nonparametric treatment allows our model to perform well even when the
vocabulary of metadata changes between training and testing. We perform
comprehensive experiments on the NUS-WIDE dataset, where we show that our model
outperforms state-of-the-art methods for multilabel image annotation even when
our model is forced to generalize to new types of metadata.
The Online Coupon-Collector Problem and Its Application to Lifelong Reinforcement Learning
Authors: Emma Brunskill, Lihong Li
Date: 2015-06-10 00:00:00
Tasks: Lifelong learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Transferring knowledge across a sequence of related tasks is an important
challenge in reinforcement learning (RL). Despite much encouraging empirical
evidence, there has been little theoretical analysis. In this paper, we study a
class of lifelong RL problems: the agent solves a sequence of tasks modeled as
finite Markov decision processes (MDPs), each of which is from a finite set of
MDPs with the same state/action sets and different transition/reward functions.
Motivated by the need for cross-task exploration in lifelong learning, we
formulate a novel online coupon-collector problem and give an optimal
algorithm. This allows us to develop a new lifelong RL algorithm, whose overall
sample complexity in a sequence of tasks is much smaller than single-task
learning, even if the sequence of tasks is generated by an adversary. Benefits
of the algorithm are demonstrated in simulated problems, including a recently
introduced human-robot interaction problem.
A Multi-Agent System Approach to Load-Balancing and Resource Allocation for Distributed Computing
Authors: Soumya Banerjee, Joshua Hecker
Date: 2015-09-21 00:00:00
Tasks: CPU, Distributed Computing, Scheduling

In this research we use a decentralized computing approach to allocate and
schedule tasks on a massively distributed grid. Using emergent properties of
multi-agent systems, the algorithm dynamically creates and dissociates clusters
to serve the changing resource demands of a global task queue. The algorithm is
compared to a standard First-in First-out (FIFO) scheduling algorithm.
Experiments done on a simulator show that the distributed resource allocation
protocol (dRAP) algorithm outperforms the FIFO scheduling algorithm on time to
empty queue, average waiting time and CPU utilization. Such a decentralized
computing approach holds promise for massively distributed processing scenarios
like SETI@home and Google MapReduce.
Conservativeness of untied auto-encoders
Authors: Daniel Jiwoong Im, Mohamed Ishmael Diwan Belghazi, Roland Memisevic
Date: 2015-06-25 00:00:00
Tasks: Decoder, Denoising

We discuss necessary and sufficient conditions for an auto-encoder to define
a conservative vector field, in which case it is associated with an energy
function akin to the unnormalized log-probability of the data. We show that the
conditions for conservativeness are more general than for encoder and decoder
weights to be the same ("tied weights"), and that they also depend on the form
of the hidden unit activation function, but that contractive training criteria,
such as denoising, will enforce these conditions locally. Based on these
observations, we show how we can use auto-encoders to extract the conservative
component of a vector field.
Estimating Random Delays in Modbus Network Using Experiments and General Linear Regression Neural Networks with Genetic Algorithm Smoothing
Authors: B. Sreram, F. Bounapane, B. Subathra, Seshadhri Srinivasan
Date: 2015-09-21 00:00:00

Time-varying delays adversely affect the performance of networked control
sys-tems (NCS) and in the worst-case can destabilize the entire system.
Therefore, modelling network delays is important for designing NCS. However,
modelling time-varying delays is challenging because of their dependence on
multiple pa-rameters such as length, contention, connected devices, protocol
employed, and channel loading. Further, these multiple parameters are
inherently random and de-lays vary in a non-linear fashion with respect to
time. This makes estimating ran-dom delays challenging. This investigation
presents a methodology to model de-lays in NCS using experiments and general
regression neural network (GRNN) due to their ability to capture non-linear
relationship. To compute the optimal smoothing parameter that computes the best
estimates, genetic algorithm is used. The objective of the genetic algorithm is
to compute the optimal smoothing pa-rameter that minimizes the mean absolute
percentage error (MAPE). Our results illustrate that the resulting GRNN is able
to predict the delays with less than 3% error. The proposed delay model gives a
framework to design compensation schemes for NCS subjected to time-varying
delays.