pwc-archive/papers-with-abstracts huggingface dataset:
Action Classification with Locality-constrained Linear Coding
Authors: Hossein Rahmani, Arif Mahmood, Du Huynh, Ajmal Mian
Date: 2014-08-17 00:00:00
Tasks: Action Classification, Classification, General Classification, L2 Regularization, regression

We propose an action classification algorithm which uses Locality-constrained
Linear Coding (LLC) to capture discriminative information of human body
variations in each spatiotemporal subsequence of a video sequence. Our proposed
method divides the input video into equally spaced overlapping spatiotemporal
subsequences, each of which is decomposed into blocks and then cells. We use
the Histogram of Oriented Gradient (HOG3D) feature to encode the information in
each cell. We justify the use of LLC for encoding the block descriptor by
demonstrating its superiority over Sparse Coding (SC). Our sequence descriptor
is obtained via a logistic regression classifier with L2 regularization. We
evaluate and compare our algorithm with ten state-of-the-art algorithms on five
benchmark datasets. Experimental results show that, on average, our algorithm
gives better accuracy than these ten algorithms.
HOPC: Histogram of Oriented Principal Components of 3D Pointclouds for Action Recognition
Authors: Hossein Rahmani, Arif Mahmood, Du. Q. Huynh, Ajmal Mian
Date: 2014-08-17 00:00:00
Tasks: 3D Action Recognition, Action Recognition, Keypoint Detection, Temporal Action Localization

Existing techniques for 3D action recognition are sensitive to viewpoint
variations because they extract features from depth images which change
significantly with viewpoint. In contrast, we directly process the pointclouds
and propose a new technique for action recognition which is more robust to
noise, action speed and viewpoint variations. Our technique consists of a novel
descriptor and keypoint detection algorithm. The proposed descriptor is
extracted at a point by encoding the Histogram of Oriented Principal Components
(HOPC) within an adaptive spatio-temporal support volume around that point.
Based on this descriptor, we present a novel method to detect Spatio-Temporal
Key-Points (STKPs) in 3D pointcloud sequences. Experimental results show that
the proposed descriptor and STKP detector outperform state-of-the-art
algorithms on three benchmark human activity datasets. We also introduce a new
multiview public dataset and show the robustness of our proposed method to
viewpoint variations.
Spatially-sparse convolutional neural networks
Authors: Benjamin Graham
Date: 2014-09-22 00:00:00
Tasks: Handwriting Recognition, image-classification, Image Classification

Convolutional neural networks (CNNs) perform well on problems such as
handwriting recognition and image classification. However, the performance of
the networks is often limited by budget and time constraints, particularly when
trying to train deep networks.
  Motivated by the problem of online handwriting recognition, we developed a
CNN for processing spatially-sparse inputs; a character drawn with a one-pixel
wide pen on a high resolution grid looks like a sparse matrix. Taking advantage
of the sparsity allowed us more efficiently to train and test large, deep CNNs.
On the CASIA-OLHWDB1.1 dataset containing 3755 character classes we get a test
error of 3.82%.
  Although pictures are not sparse, they can be thought of as sparse by adding
padding. Applying a deep convolutional network using sparsity has resulted in a
substantial reduction in test error on the CIFAR small picture datasets: 6.28%
on CIFAR-10 and 24.30% for CIFAR-100.
Oblivious Bounds on the Probability of Boolean Functions
Authors: Wolfgang Gatterbauer, Dan Suciu
Date: 2014-09-21 00:00:00
Tasks: Management

This paper develops upper and lower bounds for the probability of Boolean
functions by treating multiple occurrences of variables as independent and
assigning them new individual probabilities. We call this approach dissociation
and give an exact characterization of optimal oblivious bounds, i.e. when the
new probabilities are chosen independent of the probabilities of all other
variables. Our motivation comes from the weighted model counting problem (or,
equivalently, the problem of computing the probability of a Boolean function),
which is #P-hard in general. By performing several dissociations, one can
transform a Boolean formula whose probability is difficult to compute, into one
whose probability is easy to compute, and which is guaranteed to provide an
upper or lower bound on the probability of the original formula by choosing
appropriate probabilities for the dissociated variables. Our new bounds shed
light on the connection between previous relaxation-based and model-based
approximations and unify them as concrete choices in a larger design space. We
also show how our theory allows a standard relational database management
system (DBMS) to both upper and lower bound hard probabilistic queries in
guaranteed polynomial time.
On the Maximum Entropy Property of the First-Order Stable Spline Kernel and its Implications
Authors: Francesca Paola Carli
Date: 2014-06-22 00:00:00
Tasks: Form

A new nonparametric approach for system identification has been recently
proposed where the impulse response is seen as the realization of a zero--mean
Gaussian process whose covariance, the so--called stable spline kernel,
guarantees that the impulse response is almost surely stable. Maximum entropy
properties of the stable spline kernel have been pointed out in the literature.
In this paper we provide an independent proof that relies on the theory of
matrix extension problems in the graphical model literature and leads to a
closed form expression for the inverse of the first order stable spline kernel
as well as to a new factorization in the form $UWU^\top$ with $U$ upper
triangular and $W$ diagonal. Interestingly, all first--order stable spline
kernels share the same factor $U$ and $W$ admits a closed form representation
in terms of the kernel hyperparameter, making the factorization computationally
inexpensive. Maximum likelihood properties of the stable spline kernel are also
highlighted. These results can be applied both to improve the stability and to
reduce the computational complexity associated with the computation of stable
spline estimators.
Approximation errors of online sparsification criteria
Authors: Paul Honeine
Date: 2014-09-21 00:00:00
Tasks: Gaussian Processes

Many machine learning frameworks, such as resource-allocating networks,
kernel-based methods, Gaussian processes, and radial-basis-function networks,
require a sparsification scheme in order to address the online learning
paradigm. For this purpose, several online sparsification criteria have been
proposed to restrict the model definition on a subset of samples. The most
known criterion is the (linear) approximation criterion, which discards any
sample that can be well represented by the already contributing samples, an
operation with excessive computational complexity. Several computationally
efficient sparsification criteria have been introduced in the literature, such
as the distance, the coherence and the Babel criteria. In this paper, we
provide a framework that connects these sparsification criteria to the issue of
approximating samples, by deriving theoretical bounds on the approximation
errors. Moreover, we investigate the error of approximating any feature, by
proposing upper-bounds on the approximation error for each of the
aforementioned sparsification criteria. Two classes of features are described
in detail, the empirical mean and the principal axes in the kernel principal
component analysis.
Analyzing sparse dictionaries for online learning with kernels
Authors: Paul Honeine
Date: 2014-09-21 00:00:00

Many signal processing and machine learning methods share essentially the
same linear-in-the-parameter model, with as many parameters as available
samples as in kernel-based machines. Sparse approximation is essential in many
disciplines, with new challenges emerging in online learning with kernels. To
this end, several sparsity measures have been proposed in the literature to
quantify sparse dictionaries and constructing relevant ones, the most prolific
ones being the distance, the approximation, the coherence and the Babel
measures. In this paper, we analyze sparse dictionaries based on these
measures. By conducting an eigenvalue analysis, we show that these sparsity
measures share many properties, including the linear independence condition and
inducing a well-posed optimization problem. Furthermore, we prove that there
exists a quasi-isometry between the parameter (i.e., dual) space and the
dictionary's induced feature space.
Domain Adaptive Neural Networks for Object Recognition
Authors: Muhammad Ghifary, W. Bastiaan Kleijn, Mengjie Zhang
Date: 2014-09-21 00:00:00
Tasks: Denoising, Domain Adaptation, Object, Object Recognition

We propose a simple neural network model to deal with the domain adaptation
problem in object recognition. Our model incorporates the Maximum Mean
Discrepancy (MMD) measure as a regularization in the supervised learning to
reduce the distribution mismatch between the source and target domains in the
latent space. From experiments, we demonstrate that the MMD regularization is
an effective tool to provide good domain adaptation models on both SURF
features and raw image pixels of a particular image data set. We also show that
our proposed model, preceded by the denoising auto-encoder pretraining,
achieves better performance than recent benchmark models on the same data sets.
This work represents the first study of MMD measure in the context of neural
networks.
A High-Level Model of Neocortical Feedback Based on an Event Window Segmentation Algorithm
Authors: Jerry R. Van Aken
Date: 2014-09-21 00:00:00

The author previously presented an event window segmentation (EWS) algorithm
[5] that uses purely statistical methods to learn to recognize recurring
patterns in an input stream of events. In the following discussion, the EWS
algorithm is first extended to make predictions about future events. Next, this
extended algorithm is used to construct a high-level, simplified model of a
neocortical hierarchy. An event stream enters at the bottom of the hierarchy,
and drives processing activity upward in the hierarchy. Successively higher
regions in the hierarchy learn to recognize successively deeper levels of
patterns in these events as they propagate from the bottom of the hierarchy.
The lower levels in the hierarchy use the predictions from the levels above to
strengthen their own predictions. A C++ source code listing of the model
implementation and test program is included as an appendix.
A KdV-like advection-dispersion equation with some remarkable properties
Authors: Abhijit Sen, Dilip P. Ahalpara, Anantanarayanan Thyagaraja, Govind S. Krishnaswami
Date: 2011-09-17 00:00:00
Tasks: Translation, Unity

We discuss a new non-linear PDE, u_t + (2 u_xx/u) u_x = epsilon u_xxx,
invariant under scaling of dependent variable and referred to here as SIdV. It
is one of the simplest such translation and space-time reflection-symmetric
first order advection-dispersion equations. This PDE (with dispersion
coefficient unity) was discovered in a genetic programming search for equations
sharing the KdV solitary wave solution. It provides a bridge between non-linear
advection, diffusion and dispersion. Special cases include the mKdV and linear
dispersive equations. We identify two conservation laws, though initial
investigations indicate that SIdV does not follow from a polynomial Lagrangian
of the KdV sort. Nevertheless, it possesses solitary and periodic travelling
waves. Moreover, numerical simulations reveal recurrence properties usually
associated with integrable systems. KdV and SIdV are the simplest in an
infinite dimensional family of equations sharing the KdV solitary wave. SIdV
and its generalizations may serve as a testing ground for numerical and
analytical techniques and be a rich source for further explorations.
Capturing "attrition intensifying" structural traits from didactic interaction sequences of MOOC learners
Authors: Tanmay Sinha, Nan Li, Patrick Jermann, Pierre Dillenbourg
Date: 2014-09-20 00:00:00
Proceeding: WS 2014 10

This work is an attempt to discover hidden structural configurations in
learning activity sequences of students in Massive Open Online Courses (MOOCs).
Leveraging combined representations of video clickstream interactions and forum
activities, we seek to fundamentally understand traits that are predictive of
decreasing engagement over time. Grounded in the interdisciplinary field of
network science, we follow a graph based approach to successfully extract
indicators of active and passive MOOC participation that reflect persistence
and regularity in the overall interaction footprint. Using these rich
educational semantics, we focus on the problem of predicting student attrition,
one of the major highlights of MOOC literature in the recent years. Our results
indicate an improvement over a baseline ngram based approach in capturing
"attrition intensifying" features from the learning activities that MOOC
learners engage in. Implications for some compelling future research are
discussed.
R3MC: A Riemannian three-factor algorithm for low-rank matrix completion
Authors: B. Mishra, R. Sepulchre
Date: 2013-06-11 00:00:00
Tasks: Low-Rank Matrix Completion, Matrix Completion, Riemannian optimization

We exploit the versatile framework of Riemannian optimization on quotient
manifolds to develop R3MC, a nonlinear conjugate-gradient method for low-rank
matrix completion. The underlying search space of fixed-rank matrices is
endowed with a novel Riemannian metric that is tailored to the least-squares
cost. Numerical comparisons suggest that R3MC robustly outperforms
state-of-the-art algorithms across different problem instances, especially
those that combine scarcely sampled and ill-conditioned data.
Tight Error Bounds for Structured Prediction
Authors: Amir Globerson, Tim Roughgarden, David Sontag, Cafer Yildirim
Date: 2014-09-19 00:00:00
Tasks: Prediction, Structured Prediction

Structured prediction tasks in machine learning involve the simultaneous
prediction of multiple labels. This is typically done by maximizing a score
function on the space of labels, which decomposes as a sum of pairwise
elements, each depending on two specific labels. Intuitively, the more pairwise
terms are used, the better the expected accuracy. However, there is currently
no theoretical account of this intuition. This paper takes a significant step
in this direction.
  We formulate the problem as classifying the vertices of a known graph
$G=(V,E)$, where the vertices and edges of the graph are labelled and correlate
semi-randomly with the ground truth. We show that the prospects for achieving
low expected Hamming error depend on the structure of the graph $G$ in
interesting ways. For example, if $G$ is a very poor expander, like a path,
then large expected Hamming error is inevitable. Our main positive result shows
that, for a wide class of graphs including 2D grid graphs common in machine
vision applications, there is a polynomial-time algorithm with small and
information-theoretically near-optimal expected error. Our results provide a
first step toward a theoretical justification for the empirical success of the
efficient approximate inference algorithms that are used for structured
prediction in models where exact inference is intractable.
Effects of Coupling in Human-Virtual Agent Body Interaction
Authors: Elisabetta Bevacqua, Sankovic Igor, Maatalaoui Ayoub, A. Nédélec, Pierre De Loor
Date: 2014-09-19 00:00:00

This paper presents a study of the dynamic coupling between a user and a
virtual character during body interaction. Coupling is directly linked with
other dimensions, such as co-presence, engagement, and believability, and was
measured in an experiment that allowed users to describe their subjective
feelings about those dimensions of interest. The experiment was based on a
theatrical game involving the imitation of slow upper-body movements and the
proposal of new movements by the user and virtual agent. The agent's behaviour
varied in autonomy: the agent could limit itself to imitating the user's
movements only, initiate new movements, or combine both behaviours. After the
game, each participant completed a questionnaire regarding their engagement in
the interaction, their subjective feeling about the co-presence of the agent,
etc. Based on four main dimensions of interest, we tested several hypotheses
against our experimental results, which are discussed here.
On the Impact of Multiobjective Scalarizing Functions
Authors: Bilel Derbel, Dimo Brockhoff, Arnaud Liefooghe, Sébastien Verel
Date: 2014-09-19 00:00:00
Tasks: Multiobjective Optimization, Position

Recently, there has been a renewed interest in decomposition-based approaches
for evolutionary multiobjective optimization. However, the impact of the choice
of the underlying scalarizing function(s) is still far from being well
understood. In this paper, we investigate the behavior of different scalarizing
functions and their parameters. We thereby abstract firstly from any specific
algorithm and only consider the difficulty of the single scalarized problems in
terms of the search ability of a (1+lambda)-EA on biobjective NK-landscapes.
Secondly, combining the outcomes of independent single-objective runs allows
for more general statements on set-based performance measures. Finally, we
investigate the correlation between the opening angle of the scalarizing
function's underlying contour lines and the position of the final solution in
the objective space. Our analysis is of fundamental nature and sheds more light
on the key characteristics of multiobjective scalarizing functions.
Hyperspectral and Multispectral Image Fusion based on a Sparse Representation
Authors: Qi Wei, José Bioucas-Dias, Nicolas Dobigeon, Jean-Yves Tourneret
Date: 2014-09-19 00:00:00

This paper presents a variational based approach to fusing hyperspectral and
multispectral images. The fusion process is formulated as an inverse problem
whose solution is the target image assumed to live in a much lower dimensional
subspace. A sparse regularization term is carefully designed, relying on a
decomposition of the scene on a set of dictionaries. The dictionary atoms and
the corresponding supports of active coding coefficients are learned from the
observed images. Then, conditionally on these dictionaries and supports, the
fusion problem is solved via alternating optimization with respect to the
target image (using the alternating direction method of multipliers) and the
coding coefficients. Simulation results demonstrate the efficiency of the
proposed algorithm when compared with the state-of-the-art fusion methods.
Local Optimal Sets and Bounded Archiving on Multi-objective NK-Landscapes with Correlated Objectives
Authors: Manuel López-Ibáñez, Arnaud Liefooghe, Sébastien Verel
Date: 2014-09-19 00:00:00
Tasks: Combinatorial Optimization

The properties of local optimal solutions in multi-objective combinatorial
optimization problems are crucial for the effectiveness of local search
algorithms, particularly when these algorithms are based on Pareto dominance.
Such local search algorithms typically return a set of mutually nondominated
Pareto local optimal (PLO) solutions, that is, a PLO-set. This paper
investigates two aspects of PLO-sets by means of experiments with Pareto local
search (PLS). First, we examine the impact of several problem characteristics
on the properties of PLO-sets for multi-objective NK-landscapes with correlated
objectives. In particular, we report that either increasing the number of
objectives or decreasing the correlation between objectives leads to an
exponential increment on the size of PLO-sets, whereas the variable correlation
has only a minor effect. Second, we study the running time and the quality
reached when using bounding archiving methods to limit the size of the archive
handled by PLS, and thus, the maximum size of the PLO-set found. We argue that
there is a clear relationship between the running time of PLS and the
difficulty of a problem instance.
Statistical Estimation: From Denoising to Sparse Regression and Hidden Cliques
Authors: Eric W. Tramel, Santhosh Kumar, Andrei Giurgiu, Andrea Montanari
Date: 2014-09-19 00:00:00
Tasks: Denoising, regression

These notes review six lectures given by Prof. Andrea Montanari on the topic
of statistical estimation for linear models. The first two lectures cover the
principles of signal recovery from linear measurements in terms of minimax
risk. Subsequent lectures demonstrate the application of these principles to
several practical problems in science and engineering. Specifically, these
topics include denoising of error-laden signals, recovery of compressively
sensed signals, reconstruction of low-rank matrices, and also the discovery of
hidden cliques within large networks.
Learning Negative Mixture Models by Tensor Decompositions
Authors: Guillaume Rabusseau, François Denis
Date: 2014-03-17 00:00:00

This work considers the problem of estimating the parameters of negative
mixture models, i.e. mixture models that possibly involve negative weights. The
contributions of this paper are as follows. (i) We show that every rational
probability distributions on strings, a representation which occurs naturally
in spectral learning, can be computed by a negative mixture of at most two
probabilistic automata (or HMMs). (ii) We propose a method to estimate the
parameters of negative mixture models having a specific tensor structure in
their low order observable moments. Building upon a recent paper on tensor
decompositions for learning latent variable models, we extend this work to the
broader setting of tensors having a symmetric decomposition with positive and
negative weights. We introduce a generalization of the tensor power method for
complex valued tensors, and establish theoretical convergence guarantees. (iii)
We show how our approach applies to negative Gaussian mixture models, for which
we provide some experiments.
Using crowdsourcing system for creating site-specific statistical machine translation engine
Authors: Alexander Kalinin, George Savchenko
Date: 2014-09-19 00:00:00
Tasks: Machine Translation, Sentence, Translation

A crowdsourcing translation approach is an effective tool for globalization
of site content, but it is also an important source of parallel linguistic
data. For the given site, processed with a crowdsourcing system, a
sentence-aligned corpus can be fetched, which covers a very narrow domain of
terminology and language patterns - a site-specific domain. These data can be
used for training and estimation of site-specific statistical machine
translation engine
When Does a Mixture of Products Contain a Product of Mixtures?
Authors: Guido F. Montufar, Jason Morton
Date: 2012-06-02 00:00:00

We derive relations between theoretical properties of restricted Boltzmann
machines (RBMs), popular machine learning models which form the building blocks
of deep learning models, and several natural notions from discrete mathematics
and convex geometry. We give implications and equivalences relating
RBM-representable probability distributions, perfectly reconstructible inputs,
Hamming modes, zonotopes and zonosets, point configurations in hyperplane
arrangements, linear threshold codes, and multi-covering numbers of hypercubes.
As a motivating application, we prove results on the relative representational
power of mixtures of product distributions and products of mixtures of pairs of
product distributions (RBMs) that formally justify widely held intuitions about
distributed representations. In particular, we show that a mixture of products
requiring an exponentially larger number of parameters is needed to represent
the probability distributions which can be obtained as products of mixtures.