pwc-archive/papers-with-abstracts huggingface dataset:
A System View of the Recognition and Interpretation of Observed Human Shape, Pose and Action
Authors: David W. Arathorn
Date: 2015-03-27 00:00:00
Tasks: Motion Planning

There is physiological evidence that our ability to interpret human pose and
action from 2D visual imagery (binocular or monocular) engages the circuitry of
the motor cortices as well as the visual areas of the brain. This implies that
the capability of the motor cortices to solve inverse kinematics is flexible
enough to apply to both motion planning as well as serving as a generative
model for the visual processing of human figures, despite the differing
functional requirements of the two tasks. This paper provides a computational
model of the cooperation between visual and motor areas: in other words, a
system view of an important class of brain computations. The model unifies the
solution of the separate inverse problems involved in the task, visual
transformation discovery, inverse kinematics, and adaptation to morphology
variations, using several instances of the Map-seeking Circuit algorithm. While
the paper is weighted toward the exposition of a neurobiological hypothesis,
from mathematical formalization of the problem to neuronal circuitry, the
algorithmic expression of the solution is also a functional machine vision
system for human figure recognition, and 3D pose and body morphology
reconstruction from monocular, perspective-less input imagery. With an inverse
kinematic generative model capable of imposing a variety of endogenous and
exogenous constraints the machine vision implementation acquires
characteristics currently unique among such systems.
Learning Embedding Representations for Knowledge Inference on Imperfect and Incomplete Repositories
Authors: Miao Fan, Qiang Zhou, Thomas Fang Zheng
Date: 2015-03-27 00:00:00
Tasks: Link Prediction, Triplet

This paper considers the problem of knowledge inference on large-scale
imperfect repositories with incomplete coverage by means of embedding entities
and relations at the first attempt. We propose IIKE (Imperfect and Incomplete
Knowledge Embedding), a probabilistic model which measures the probability of
each belief, i.e. $\langle h,r,t\rangle$, in large-scale knowledge bases such
as NELL and Freebase, and our objective is to learn a better low-dimensional
vector representation for each entity ($h$ and $t$) and relation ($r$) in the
process of minimizing the loss of fitting the corresponding confidence given by
machine learning (NELL) or crowdsouring (Freebase), so that we can use $||{\bf
h} + {\bf r} - {\bf t}||$ to assess the plausibility of a belief when
conducting inference. We use subsets of those inexact knowledge bases to train
our model and test the performances of link prediction and triplet
classification on ground truth beliefs, respectively. The results of extensive
experiments show that IIKE achieves significant improvement compared with the
baseline and state-of-the-art approaches.
Revisable Justified Belief: Preliminary Report
Authors: Alexandru Baltag, Bryan Renne, Sonja Smets
Date: 2015-03-27 00:00:00

The theory $\mathsf{CDL}$ of Conditional Doxastic Logic is the single-agent
version of Board's multi-agent theory $\mathsf{BRSIC}$ of conditional belief.
$\mathsf{CDL}$ may be viewed as a version of AGM belief revision theory in
which Boolean combinations of revisions are expressible in the language. We
introduce a theory $\mathsf{JCDL}$ of Justified Conditional Doxastic Logic that
replaces conditional belief formulas $B^\psi\varphi$ by expressions
$t{\,:^{\psi}}\varphi$ made up of a term $t$ whose syntactic structure suggests
a derivation of the belief $\varphi$ after revision by $\psi$. This allows us
to think of terms $t$ as reasons justifying a belief in various formulas after
a revision takes place. We show that $\mathsf{JCDL}$-theorems are the exact
analogs of $\mathsf{CDL}$-theorems, and that this result holds the other way
around as well. This allows us to think of $\mathsf{JCDL}$ as a theory of
revisable justified belief.
Hyper-Spectral Image Analysis with Partially-Latent Regression and Spatial Markov Dependencies
Authors: Antoine Deleforge, Florence Forbes, Sileye . Ba, Radu Horaud
Date: 2014-09-30 00:00:00
Tasks: regression

Hyper-spectral data can be analyzed to recover physical properties at large
planetary scales. This involves resolving inverse problems which can be
addressed within machine learning, with the advantage that, once a relationship
between physical parameters and spectra has been established in a data-driven
fashion, the learned relationship can be used to estimate physical parameters
for new hyper-spectral observations. Within this framework, we propose a
spatially-constrained and partially-latent regression method which maps
high-dimensional inputs (hyper-spectral images) onto low-dimensional responses
(physical parameters such as the local chemical composition of the soil). The
proposed regression model comprises two key features. Firstly, it combines a
Gaussian mixture of locally-linear mappings (GLLiM) with a partially-latent
response model. While the former makes high-dimensional regression tractable,
the latter enables to deal with physical parameters that cannot be observed or,
more generally, with data contaminated by experimental artifacts that cannot be
explained with noise models. Secondly, spatial constraints are introduced in
the model through a Markov random field (MRF) prior which provides a spatial
structure to the Gaussian-mixture hidden variables. Experiments conducted on a
database composed of remotely sensed observations collected from the Mars
planet by the Mars Express orbiter demonstrate the effectiveness of the
proposed model.
Variational Optimization of Annealing Schedules
Authors: Taichi Kiwaki
Date: 2015-02-18 00:00:00
Tasks: Quantization, Scheduling

Annealed importance sampling (AIS) is a common algorithm to estimate
partition functions of useful stochastic models. One important problem for
obtaining accurate AIS estimates is the selection of an annealing schedule.
Conventionally, an annealing schedule is often determined heuristically or is
simply set as a linearly increasing sequence. In this paper, we propose an
algorithm for the optimal schedule by deriving a functional that dominates the
AIS estimation error and by numerically minimizing this functional. We
experimentally demonstrate that the proposed algorithm mostly outperforms
conventional scheduling schemes with large quantization numbers.
Gibbs Sampling with Low-Power Spiking Digital Neurons
Authors: Srinjoy Das, Bruno Umbria Pedroni, Paul Merolla, John Arthur, Andrew S. Cassidy, Bryan L. Jackson, Dharmendra Modha, Gert Cauwenberghs, Ken Kreutz-Delgado
Date: 2015-03-26 00:00:00
Tasks: General Classification, image-classification, Image Classification, speech-recognition, Speech Recognition

Restricted Boltzmann Machines and Deep Belief Networks have been successfully
used in a wide variety of applications including image classification and
speech recognition. Inference and learning in these algorithms uses a Markov
Chain Monte Carlo procedure called Gibbs sampling. A sigmoidal function forms
the kernel of this sampler which can be realized from the firing statistics of
noisy integrate-and-fire neurons on a neuromorphic VLSI substrate. This paper
demonstrates such an implementation on an array of digital spiking neurons with
stochastic leak and threshold properties for inference tasks and presents some
key performance metrics for such a hardware-based sampler in both the
generative and discriminative contexts.
Sparse graphs using exchangeable random measures
Authors: François Caron, Emily B. Fox
Date: 2014-01-06 00:00:00
Tasks: Efficient Exploration, graph construction

Statistical network modeling has focused on representing the graph as a
discrete structure, namely the adjacency matrix, and considering the
exchangeability of this array. In such cases, the Aldous-Hoover representation
theorem (Aldous, 1981;Hoover, 1979} applies and informs us that the graph is
necessarily either dense or empty. In this paper, we instead consider
representing the graph as a measure on $\mathbb{R}_+^2$. For the associated
definition of exchangeability in this continuous space, we rely on the
Kallenberg representation theorem (Kallenberg, 2005). We show that for certain
choices of such exchangeable random measures underlying our graph construction,
our network process is sparse with power-law degree distribution. In
particular, we build on the framework of completely random measures (CRMs) and
use the theory associated with such processes to derive important network
properties, such as an urn representation for our analysis and network
simulation. Our theoretical results are explored empirically and compared to
common network models. We then present a Hamiltonian Monte Carlo algorithm for
efficient exploration of the posterior distribution and demonstrate that we are
able to recover graphs ranging from dense to sparse--and perform associated
tests--based on our flexible CRM-based formulation. We explore network
properties in a range of real datasets, including Facebook social circles, a
political blogosphere, protein networks, citation networks, and world wide web
networks, including networks with hundreds of thousands of nodes and millions
of edges.
FitNets: Hints for Thin Deep Nets
Authors: Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, Yoshua Bengio
Date: 2014-12-19 00:00:00
Tasks: Knowledge Distillation

While depth tends to improve network performances, it also makes gradient-based training more difficult since deeper networks tend to be more non-linear. The recently proposed knowledge distillation approach is aimed at obtaining small and fast-to-execute models, and it has shown that a student network could imitate the soft output of a larger teacher network or ensemble of networks. In this paper, we extend this idea to allow the training of a student that is deeper and thinner than the teacher, using not only the outputs but also the intermediate representations learned by the teacher as hints to improve the training process and final performance of the student. Because the student intermediate hidden layer will generally be smaller than the teacher's intermediate hidden layer, additional parameters are introduced to map the student hidden layer to the prediction of the teacher hidden layer. This allows one to train deeper students that can generalize better or run faster, a trade-off that is controlled by the chosen student capacity. For example, on CIFAR-10, a deep student network with almost 10.4 times less parameters outperforms a larger, state-of-the-art teacher network.
Real-time multi-view deconvolution
Authors: Benjamin Schmid, Jan Huisken
Date: 2015-03-27 00:00:00
Tasks: GPU

In light-sheet microscopy, overall image content and resolution are improved
by acquiring and fusing multiple views of the sample from different directions.
State-of-the-art multi-view (MV) deconvolution employs the point spread
functions (PSF) of the different views to simultaneously fuse and deconvolve
the images in 3D, but processing takes a multiple of the acquisition time and
constitutes the bottleneck in the imaging pipeline. Here we show that MV
deconvolution in 3D can finally be achieved in real-time by reslicing the
acquired data and processing cross-sectional planes individually on the
massively parallel architecture of a graphics processing unit (GPU).
Discriminative Bayesian Dictionary Learning for Classification
Authors: Naveed Akhtar, Faisal Shafait, Ajmal Mian
Date: 2015-03-27 00:00:00
Tasks: Action Recognition, Classification, Dictionary Learning, General Classification, Temporal Action Localization

We propose a Bayesian approach to learn discriminative dictionaries for
sparse representation of data. The proposed approach infers probability
distributions over the atoms of a discriminative dictionary using a Beta
Process. It also computes sets of Bernoulli distributions that associate class
labels to the learned dictionary atoms. This association signifies the
selection probabilities of the dictionary atoms in the expansion of
class-specific data. Furthermore, the non-parametric character of the proposed
approach allows it to infer the correct size of the dictionary. We exploit the
aforementioned Bernoulli distributions in separately learning a linear
classifier. The classifier uses the same hierarchical Bayesian model as the
dictionary, which we present along the analytical inference solution for Gibbs
sampling. For classification, a test instance is first sparsely encoded over
the learned dictionary and the codes are fed to the classifier. We performed
experiments for face and action recognition; and object and scene-category
classification using five public datasets and compared the results with
state-of-the-art discriminative sparse representation approaches. Experiments
show that the proposed Bayesian approach consistently outperforms the existing
approaches.
Bayesian Cross Validation and WAIC for Predictive Prior Design in Regular Asymptotic Theory
Authors: Sumio Watanabe
Date: 2015-03-27 00:00:00
Tasks: Relation

Prior design is one of the most important problems in both statistics and
machine learning. The cross validation (CV) and the widely applicable
information criterion (WAIC) are predictive measures of the Bayesian
estimation, however, it has been difficult to apply them to find the optimal
prior because their mathematical properties in prior evaluation have been
unknown and the region of the hyperparameters is too wide to be examined. In
this paper, we derive a new formula by which the theoretical relation among CV,
WAIC, and the generalization loss is clarified and the optimal hyperparameter
can be directly found.
  By the formula, three facts are clarified about predictive prior design.
Firstly, CV and WAIC have the same second order asymptotic expansion, hence
they are asymptotically equivalent to each other as the optimizer of the
hyperparameter. Secondly, the hyperparameter which minimizes CV or WAIC makes
the average generalization loss to be minimized asymptotically but does not the
random generalization loss. And lastly, by using the mathematical relation
between priors, the variances of the optimized hyperparameters by CV and WAIC
are made smaller with small computational costs. Also we show that the
optimized hyperparameter by DIC or the marginal likelihood does not minimize
the average or random generalization loss in general.
Competitive Distribution Estimation
Authors: Alon Orlitsky, Ananda Theertha Suresh
Date: 2015-03-27 00:00:00

Estimating an unknown distribution from its samples is a fundamental problem
in statistics. The common, min-max, formulation of this goal considers the
performance of the best estimator over all distributions in a class. It shows
that with $n$ samples, distributions over $k$ symbols can be learned to a KL
divergence that decreases to zero with the sample size $n$, but grows
unboundedly with the alphabet size $k$.
  Min-max performance can be viewed as regret relative to an oracle that knows
the underlying distribution. We consider two natural and modest limits on the
oracle's power. One where it knows the underlying distribution only up to
symbol permutations, and the other where it knows the exact distribution but is
restricted to use natural estimators that assign the same probability to
symbols that appeared equally many times in the sample.
  We show that in both cases the competitive regret reduces to
$\min(k/n,\tilde{\mathcal{O}}(1/\sqrt n))$, a quantity upper bounded uniformly
for every alphabet size. This shows that distributions can be estimated nearly
as well as when they are essentially known in advance, and nearly as well as
when they are completely known in advance but need to be estimated via a
natural estimator. We also provide an estimator that runs in linear time and
incurs competitive regret of $\tilde{\mathcal{O}}(\min(k/n,1/\sqrt n))$, and
show that for natural estimators this competitive regret is inevitable. We also
demonstrate the effectiveness of competitive estimators using simulations.
Generalized K-fan Multimodal Deep Model with Shared Representations
Authors: Gang Chen, Sargur N. Srihari
Date: 2015-03-26 00:00:00
Tasks: Information Retrieval, Object Recognition, Retrieval

Multimodal learning with deep Boltzmann machines (DBMs) is an generative
approach to fuse multimodal inputs, and can learn the shared representation via
Contrastive Divergence (CD) for classification and information retrieval tasks.
However, it is a 2-fan DBM model, and cannot effectively handle multiple
prediction tasks. Moreover, this model cannot recover the hidden
representations well by sampling from the conditional distribution when more
than one modalities are missing. In this paper, we propose a K-fan deep
structure model, which can handle the multi-input and muti-output learning
problems effectively. In particular, the deep structure has K-branch for
different inputs where each branch can be composed of a multi-layer deep model,
and a shared representation is learned in an discriminative manner to tackle
multimodal tasks. Given the deep structure, we propose two objective functions
to handle two multi-input and multi-output tasks: joint visual restoration and
labeling, and the multi-view multi-calss object recognition tasks. To estimate
the model parameters, we initialize the deep model parameters with CD to
maximize the joint distribution, and then we use backpropagation to update the
model according to specific objective function. The experimental results
demonstrate that the model can effectively leverages multi-source information
and predict multiple tasks well over competitive baselines.
Transductive Multi-class and Multi-label Zero-shot Learning
Authors: Yanwei Fu, Yongxin Yang, Timothy M. Hospedales, Tao Xiang, Shaogang Gong
Date: 2015-03-26 00:00:00
Tasks: Multi-label zero-shot learning, Transductive Learning, Transfer Learning, Zero-Shot Learning

Recently, zero-shot learning (ZSL) has received increasing interest. The key
idea underpinning existing ZSL approaches is to exploit knowledge transfer via
an intermediate-level semantic representation which is assumed to be shared
between the auxiliary and target datasets, and is used to bridge between these
domains for knowledge transfer. The semantic representation used in existing
approaches varies from visual attributes to semantic word vectors and semantic
relatedness. However, the overall pipeline is similar: a projection mapping
low-level features to the semantic representation is learned from the auxiliary
dataset by either classification or regression models and applied directly to
map each instance into the same semantic representation space where a zero-shot
classifier is used to recognise the unseen target class instances with a single
known 'prototype' of each target class. In this paper we discuss two related
lines of work improving the conventional approach: exploiting transductive
learning ZSL, and generalising ZSL to the multi-label case.
Averaged Hausdorff Approximations of Pareto Fronts based on Multiobjective Estimation of Distribution Algorithms
Authors: Luis Marti, Christian Grimme, Pascal Kerschke, Heike Trautmann, Günter Rudolph
Date: 2015-03-26 00:00:00
Tasks: Multiobjective Optimization

In the a posteriori approach of multiobjective optimization the Pareto front
is approximated by a finite set of solutions in the objective space. The
quality of the approximation can be measured by different indicators that take
into account the approximation's closeness to the Pareto front and its
distribution along the Pareto front. In particular, the averaged Hausdorff
indicator prefers an almost uniform distribution. An observed drawback of
multiobjective estimation of distribution algorithms (MEDAs) is that - as
common for randomized metaheuristics - the final population usually is not
uniformly distributed along the Pareto front. Therefore, we propose a
postprocessing strategy which consists of applying the averaged Hausdorff
indicator to the complete archive of generated solutions after optimization in
order to select a uniformly distributed subset of nondominated solutions from
the archive. In this paper, we put forward a strategy for extracting the above
described subset. The effectiveness of the proposal is contrasted in a series
of experiments that involve different MEDAs and filtering techniques.
Multi-Labeled Classification of Demographic Attributes of Patients: a case study of diabetics patients
Authors: Naveen Kumar Parachur Cotha, Marina Sokolova
Date: 2015-03-26 00:00:00
Tasks: Binary Classification, General Classification, Multi-Label Learning, Privacy Preserving

Automated learning of patients demographics can be seen as multi-label
problem where a patient model is based on different race and gender groups. The
resulting model can be further integrated into Privacy-Preserving Data Mining,
where it can be used to assess risk of identification of different patient
groups. Our project considers relations between diabetes and demographics of
patients as a multi-labelled problem. Most research in this area has been done
as binary classification, where the target class is finding if a person has
diabetes or not. But very few, and maybe no work has been done in multi-labeled
analysis of the demographics of patients who are likely to be diagnosed with
diabetes. To identify such groups, we applied ensembles of several multi-label
learning algorithms.
Transductive Multi-label Zero-shot Learning
Authors: Yanwei Fu, Yongxin Yang, Tim Hospedales, Tao Xiang, Shaogang Gong
Date: 2015-03-26 00:00:00
Tasks: Multi-label zero-shot learning, regression, Transductive Learning, Zero-Shot Learning

Zero-shot learning has received increasing interest as a means to alleviate
the often prohibitive expense of annotating training data for large scale
recognition problems. These methods have achieved great success via learning
intermediate semantic representations in the form of attributes and more
recently, semantic word vectors. However, they have thus far been constrained
to the single-label case, in contrast to the growing popularity and importance
of more realistic multi-label data. In this paper, for the first time, we
investigate and formalise a general framework for multi-label zero-shot
learning, addressing the unique challenge therein: how to exploit multi-label
correlation at test time with no training data for those classes? In
particular, we propose (1) a multi-output deep regression model to project an
image into a semantic word space, which explicitly exploits the correlations in
the intermediate semantic layer of word vectors; (2) a novel zero-shot learning
algorithm for multi-label data that exploits the unique compositionality
property of semantic word vector representations; and (3) a transductive
learning strategy to enable the regression model learned from seen classes to
generalise well to unseen classes. Our zero-shot learning experiments on a
number of standard multi-label datasets demonstrate that our method outperforms
a variety of baselines.