pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised learning of regression mixture models with unknown number of components
Authors: Faicel Chamroukhi
Date: 2014-09-24 00:00:00
Tasks: Clustering, Model Selection, regression

Regression mixture models are widely studied in statistics, machine learning
and data analysis. Fitting regression mixtures is challenging and is usually
performed by maximum likelihood by using the expectation-maximization (EM)
algorithm. However, it is well-known that the initialization is crucial for EM.
If the initialization is inappropriately performed, the EM algorithm may lead
to unsatisfactory results. The EM algorithm also requires the number of
clusters to be given a priori; the problem of selecting the number of mixture
components requires using model selection criteria to choose one from a set of
pre-estimated candidate models. We propose a new fully unsupervised algorithm
to learn regression mixture models with unknown number of components. The
developed unsupervised learning approach consists in a penalized maximum
likelihood estimation carried out by a robust expectation-maximization (EM)
algorithm for fitting polynomial, spline and B-spline regressions mixtures. The
proposed learning approach is fully unsupervised: 1) it simultaneously infers
the model parameters and the optimal number of the regression mixture
components from the data as the learning proceeds, rather than in a two-fold
scheme as in standard model-based clustering using afterward model selection
criteria, and 2) it does not require accurate initialization unlike the
standard EM for regression mixtures. The developed approach is applied to curve
clustering problems. Numerical experiments on simulated data show that the
proposed robust EM algorithm performs well and provides accurate results in
terms of robustness with regard initialization and retrieving the optimal
partition with the actual number of clusters. An application to real data in
the framework of functional data clustering, confirms the benefit of the
proposed approach for practical applications.
One-class Collaborative Filtering with Random Graphs: Annotated Version
Authors: Ulrich Paquet, Noam Koenigstein
Date: 2013-09-26 00:00:00
Tasks: Collaborative Filtering, Variational Inference

The bane of one-class collaborative filtering is interpreting and modelling
the latent signal from the missing class. In this paper we present a novel
Bayesian generative model for implicit collaborative filtering. It forms a core
component of the Xbox Live architecture, and unlike previous approaches,
delineates the odds of a user disliking an item from simply not considering it.
The latent signal is treated as an unobserved random graph connecting users
with items they might have encountered. We demonstrate how large-scale
distributed learning can be achieved through a combination of stochastic
gradient descent and mean field variational inference over random graph
samples. A fine-grained comparison is done against a state of the art baseline
on real world data.
Recent Progress in Image Deblurring
Authors: Ruxin Wang, DaCheng Tao
Date: 2014-09-24 00:00:00
Tasks: Bayesian Inference, Deblurring, Image Deblurring, Image Restoration

This paper comprehensively reviews the recent development of image
deblurring, including non-blind/blind, spatially invariant/variant deblurring
techniques. Indeed, these techniques share the same objective of inferring a
latent sharp image from one or several corresponding blurry images, while the
blind deblurring techniques are also required to derive an accurate blur
kernel. Considering the critical role of image restoration in modern imaging
systems to provide high-quality images under complex environments such as
motion, undesirable lighting conditions, and imperfect system components, image
deblurring has attracted growing attention in recent years. From the viewpoint
of how to handle the ill-posedness which is a crucial issue in deblurring
tasks, existing methods can be grouped into five categories: Bayesian inference
framework, variational methods, sparse representation-based methods,
homography-based modeling, and region-based methods. In spite of achieving a
certain level of development, image deblurring, especially the blind case, is
limited in its success by complex application conditions which make the blur
kernel hard to obtain and be spatially variant. We provide a holistic
understanding and deep insight into image deblurring in this review. An
analysis of the empirical evidence for representative methods, practical
issues, as well as a discussion of promising future directions are also
presented.
Quantized Estimation of Gaussian Sequence Models in Euclidean Balls
Authors: Yuancheng Zhu, John Lafferty
Date: 2014-09-24 00:00:00
Proceeding: NeurIPS 2014 12

A central result in statistical theory is Pinsker's theorem, which
characterizes the minimax rate in the normal means model of nonparametric
estimation. In this paper, we present an extension to Pinsker's theorem where
estimation is carried out under storage or communication constraints. In
particular, we place limits on the number of bits used to encode an estimator,
and analyze the excess risk in terms of this constraint, the signal size, and
the noise level. We give sharp upper and lower bounds for the case of a
Euclidean ball, which establishes the Pareto-optimal minimax tradeoff between
storage and risk in this setting.
The Application of Differential Privacy for Rank Aggregation: Privacy and Accuracy
Authors: Shang Shang, Tiance Wang, Paul Cuff, Sanjeev Kulkarni
Date: 2014-09-24 00:00:00

The potential risk of privacy leakage prevents users from sharing their
honest opinions on social platforms. This paper addresses the problem of
privacy preservation if the query returns the histogram of rankings. The
framework of differential privacy is applied to rank aggregation. The error
probability of the aggregated ranking is analyzed as a result of noise added in
order to achieve differential privacy. Upper bounds on the error rates for any
positional ranking rule are derived under the assumption that profiles are
uniformly distributed. Simulation results are provided to validate the
probabilistic analysis.
Improving Cross-domain Recommendation through Probabilistic Cluster-level Latent Factor Model--Extended Version
Authors: Siting Ren, Sheng Gao
Date: 2014-09-24 00:00:00
Tasks: Clustering

Cross-domain recommendation has been proposed to transfer user behavior
pattern by pooling together the rating data from multiple domains to alleviate
the sparsity problem appearing in single rating domains. However, previous
models only assume that multiple domains share a latent common rating pattern
based on the user-item co-clustering. To capture diversities among different
domains, we propose a novel Probabilistic Cluster-level Latent Factor (PCLF)
model to improve the cross-domain recommendation performance. Experiments on
several real world datasets demonstrate that our proposed model outperforms the
state-of-the-art methods for the cross-domain recommendation task.
A Concept Learning Approach to Multisensory Object Perception
Authors: Ifeoma Nwogu, Goker Erdogan, Ilker Yildirim, Robert Jacobs
Date: 2014-09-23 00:00:00
Tasks: Bayesian Inference, Object

This paper presents a computational model of concept learning using Bayesian
inference for a grammatically structured hypothesis space, and test the model
on multisensory (visual and haptics) recognition of 3D objects. The study is
performed on a set of artificially generated 3D objects known as fribbles,
which are complex, multipart objects with categorical structures. The goal of
this work is to develop a working multisensory representational model that
integrates major themes on concepts and concepts learning from the cognitive
science literature. The model combines the representational power of a
probabilistic generative grammar with the inferential power of Bayesian
induction.
Local case-control sampling: Efficient subsampling in imbalanced data sets
Authors: William Fithian, Trevor Hastie
Date: 2013-06-16 00:00:00

For classification problems with significant class imbalance, subsampling can
reduce computational costs at the price of inflated variance in estimating
model parameters. We propose a method for subsampling efficiently for logistic
regression by adjusting the class balance locally in feature space via an
accept-reject scheme. Our method generalizes standard case-control sampling,
using a pilot estimate to preferentially select examples whose responses are
conditionally rare given their features. The biased subsampling is corrected by
a post-hoc analytic adjustment to the parameters. The method is simple and
requires one parallelizable scan over the full data set. Standard case-control
sampling is inconsistent under model misspecification for the population
risk-minimizing coefficients $\theta^*$. By contrast, our estimator is
consistent for $\theta^*$ provided that the pilot estimate is. Moreover, under
correct specification and with a consistent, independent pilot estimate, our
estimator has exactly twice the asymptotic variance of the full-sample MLE -
even if the selected subsample comprises a miniscule fraction of the full data
set, as happens when the original data are severely imbalanced. The factor of
two improves to $1+\frac{1}{c}$ if we multiply the baseline acceptance
probabilities by $c>1$ (and weight points with acceptance probability greater
than 1), taking roughly $\frac{1+c}{2}$ times as many data points into the
subsample. Experiments on simulated and real data show that our method can
substantially outperform standard case-control subsampling.
HSR: L1/2 Regularized Sparse Representation for Fast Face Recognition using Hierarchical Feature Selection
Authors: Bo Han, Bo He, Tingting Sun, Mengmeng Ma, Amaury Lendasse
Date: 2014-09-23 00:00:00
Tasks: Face Recognition, feature selection, Sparse Representation-based Classification

In this paper, we propose a novel method for fast face recognition called L1/2 Regularized Sparse Representation using Hierarchical Feature Selection (HSR). By employing hierarchical feature selection, we can compress the scale and dimension of global dictionary, which directly contributes to the decrease of computational cost in sparse representation that our approach is strongly rooted in. It consists of Gabor wavelets and Extreme Learning Machine Auto-Encoder (ELM-AE) hierarchically. For Gabor wavelets part, local features can be extracted at multiple scales and orientations to form Gabor-feature based image, which in turn improves the recognition rate. Besides, in the presence of occluded face image, the scale of Gabor-feature based global dictionary can be compressed accordingly because redundancies exist in Gabor-feature based occlusion dictionary. For ELM-AE part, the dimension of Gabor-feature based global dictionary can be compressed because high-dimensional face images can be rapidly represented by low-dimensional feature. By introducing L1/2 regularization, our approach can produce sparser and more robust representation compared to regularized Sparse Representation based Classification (SRC), which also contributes to the decrease of the computational cost in sparse representation. In comparison with related work such as SRC and Gabor-feature based SRC (GSRC), experimental results on a variety of face databases demonstrate the great advantage of our method for computational cost. Moreover, we also achieve approximate or even better recognition rate.
RMSE-ELM: Recursive Model based Selective Ensemble of Extreme Learning Machines for Robustness Improvement
Authors: Bo Han, Bo He, Mengmeng Ma, Tingting Sun, Tianhong Yan, Amaury Lendasse
Date: 2014-08-09 00:00:00

Extreme learning machine (ELM) as an emerging branch of shallow networks has
shown its excellent generalization and fast learning speed. However, for
blended data, the robustness of ELM is weak because its weights and biases of
hidden nodes are set randomly. Moreover, the noisy data exert a negative
effect. To solve this problem, a new framework called RMSE-ELM is proposed in
this paper. It is a two-layer recursive model. In the first layer, the
framework trains lots of ELMs in different groups concurrently, then employs
selective ensemble to pick out an optimal set of ELMs in each group, which can
be merged into a large group of ELMs called candidate pool. In the second
layer, selective ensemble is recursively used on candidate pool to acquire the
final ensemble. In the experiments, we apply UCI blended datasets to confirm
the robustness of our new approach in two key aspects (mean square error and
standard deviation). The space complexity of our method is increased to some
degree, but the results have shown that RMSE-ELM significantly improves
robustness with slightly computational time compared with representative
methods (ELM, OP-ELM, GASEN-ELM, GASEN-BP and E-GASEN). It becomes a potential
framework to solve robustness issue of ELM for high-dimensional blended data in
the future.
The two-dimensional Gabor function adapted to natural image statistics: A model of simple-cell receptive fields and sparse structure in images
Authors: Peter Loxley
Date: 2013-03-29 00:00:00
Tasks: Image Reconstruction

The two-dimensional Gabor function is adapted to natural image statistics, leading to a tractable probabilistic generative model that can be used to model simple-cell receptive-field profiles, or generate basis functions for sparse coding applications. Learning is found to be most pronounced in three Gabor-function parameters representing the size and spatial frequency of the two-dimensional Gabor function, and characterized by a non-uniform probability distribution with heavy tails. All three parameters are found to be strongly correlated: resulting in a basis of multiscale Gabor functions with similar aspect ratios, and size-dependent spatial frequencies. A key finding is that the distribution of receptive-field sizes is scale-invariant over a wide range of values, so there is no characteristic receptive-field size selected by natural image statistics. The Gabor-function aspect ratio is found to be approximately conserved by the learning rules and is therefore not well-determined by natural image statistics. This allows for three distinct solutions: a basis of Gabor functions with sharp orientation resolution at the expense of spatial-frequency resolution; a basis of Gabor functions with sharp spatial-frequency resolution at the expense of orientation resolution; or a basis with unit aspect ratio. Arbitrary mixtures of all three cases are also possible. Two parameters controlling the shape of the marginal distributions in a probabilistic generative model fully account for all three solutions. The best-performing probabilistic generative model for sparse coding applications is found to be a Gaussian copula with Pareto marginal probability density functions.
Neighborhood Selection and Rules Identification for Cellular Automata: A Rough Sets Approach
Authors: Bartlomiej Placzek
Date: 2014-09-22 00:00:00

In this paper a method is proposed which uses data mining techniques based on
rough sets theory to select neighborhood and determine update rule for cellular
automata (CA). According to the proposed approach, neighborhood is detected by
reducts calculations and a rule-learning algorithm is applied to induce a set
of decision rules that define the evolution of CA. Experiments were performed
with use of synthetic as well as real-world data sets. The results show that
the introduced method allows identification of both deterministic and
probabilistic CA-based models of real-world phenomena.
On tensor rank of conditional probability tables in Bayesian networks
Authors: Jiří Vomlel, Petr Tichavský
Date: 2014-09-22 00:00:00

A difficult task in modeling with Bayesian networks is the elicitation of
numerical parameters of Bayesian networks. A large number of parameters is
needed to specify a conditional probability table (CPT) that has a larger
parent set. In this paper we show that, most CPTs from real applications of
Bayesian networks can actually be very well approximated by tables that require
substantially less parameters. This observation has practical consequence not
only for model elicitation but also for efficient probabilistic reasoning with
these networks.
Unified Structured Learning for Simultaneous Human Pose Estimation and Garment Attribute Classification
Authors: Jie Shen, Guangcan Liu, Jia Chen, Yuqiang Fang, Jianbin Xie, Yong Yu, Shuicheng Yan
Date: 2014-04-19 00:00:00
Tasks: Attribute, General Classification, Pose Estimation

In this paper, we utilize structured learning to simultaneously address two
intertwined problems: human pose estimation (HPE) and garment attribute
classification (GAC), which are valuable for a variety of computer vision and
multimedia applications. Unlike previous works that usually handle the two
problems separately, our approach aims to produce a jointly optimal estimation
for both HPE and GAC via a unified inference procedure. To this end, we adopt a
preprocessing step to detect potential human parts from each image (i.e., a set
of "candidates") that allows us to have a manageable input space. In this way,
the simultaneous inference of HPE and GAC is converted to a structured learning
problem, where the inputs are the collections of candidate ensembles, the
outputs are the joint labels of human parts and garment attributes, and the
joint feature representation involves various cues such as pose-specific
features, garment-specific features, and cross-task features that encode
correlations between human parts and garment attributes. Furthermore, we
explore the "strong edge" evidence around the potential human parts so as to
derive more powerful representations for oriented human parts. Such evidences
can be seamlessly integrated into our structured learning model as a kind of
energy function, and the learning process could be performed by standard
structured Support Vector Machines (SVM) algorithm. However, the joint
structure of the two problems is a cyclic graph, which hinders efficient
inference. To resolve this issue, we compute instead approximate optima by
using an iterative procedure, where in each iteration the variables of one
problem are fixed. In this way, satisfactory solutions can be efficiently
computed by dynamic programming. Experimental results on two benchmark datasets
show the state-of-the-art performance of our approach.
Analyzing the Performance of Multilayer Neural Networks for Object Recognition
Authors: Pulkit Agrawal, Ross Girshick, Jitendra Malik
Date: 2014-07-07 00:00:00
Tasks: Object Recognition

In the last two years, convolutional neural networks (CNNs) have achieved an
impressive suite of results on standard recognition datasets and tasks.
CNN-based features seem poised to quickly replace engineered representations,
such as SIFT and HOG. However, compared to SIFT and HOG, we understand much
less about the nature of the features learned by large CNNs. In this paper, we
experimentally probe several aspects of CNN feature learning in an attempt to
help practitioners gain useful, evidence-backed intuitions about how to apply
CNNs to computer vision problems.
Detecting People in Cubist Art
Authors: Shiry Ginosar, Daniel Haas, Timothy Brown, Jitendra Malik
Date: 2014-09-22 00:00:00
Tasks: Object, object-detection, Object Detection

Although the human visual system is surprisingly robust to extreme distortion
when recognizing objects, most evaluations of computer object detection methods
focus only on robustness to natural form deformations such as people's pose
changes. To determine whether algorithms truly mirror the flexibility of human
vision, they must be compared against human vision at its limits. For example,
in Cubist abstract art, painted objects are distorted by object fragmentation
and part-reorganization, to the point that human vision often fails to
recognize them. In this paper, we evaluate existing object detection methods on
these abstract renditions of objects, comparing human annotators to four
state-of-the-art object detectors on a corpus of Picasso paintings. Our results
demonstrate that while human perception significantly outperforms current
methods, human perception and part-based models exhibit a similarly graceful
degradation in object detection performance as the objects become increasingly
abstract and fragmented, corroborating the theory of part-based object
representation in the brain.
Expectation Propagation
Authors: Jack Raymond, Andre Manoel, Manfred Opper
Date: 2014-09-22 00:00:00
Tasks: Variational Inference

Variational inference is a powerful concept that underlies many iterative
approximation algorithms; expectation propagation, mean-field methods and
belief propagations were all central themes at the school that can be perceived
from this unifying framework. The lectures of Manfred Opper introduce the
archetypal example of Expectation Propagation, before establishing the
connection with the other approximation methods. Corrections by expansion about
the expectation propagation are then explained. Finally some advanced inference
topics and applications are explored in the final sections.
A Bayesian Network View on Acoustic Model-Based Techniques for Robust Speech Recognition
Authors: Roland Maas, Christian Huemmer, Armin Sehr, Walter Kellermann
Date: 2013-10-11 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Robust Speech Recognition, speech-recognition, Speech Recognition

This article provides a unifying Bayesian network view on various approaches
for acoustic model adaptation, missing feature, and uncertainty decoding that
are well-known in the literature of robust automatic speech recognition. The
representatives of these classes can often be deduced from a Bayesian network
that extends the conventional hidden Markov models used in speech recognition.
These extensions, in turn, can in many cases be motivated from an underlying
observation model that relates clean and distorted feature vectors. By
converting the observation models into a Bayesian network representation, we
formulate the corresponding compensation rules leading to a unified view on
known derivations as well as to new formulations for certain approaches. The
generic Bayesian perspective provided in this contribution thus highlights
structural differences and similarities between the analyzed approaches.
Distributed Clustering and Learning Over Networks
Authors: Xiaochuan Zhao, Ali H. Sayed
Date: 2014-09-22 00:00:00
Tasks: Clustering

Distributed processing over networks relies on in-network processing and
cooperation among neighboring agents. Cooperation is beneficial when agents
share a common objective. However, in many applications agents may belong to
different clusters that pursue different objectives. Then, indiscriminate
cooperation will lead to undesired results. In this work, we propose an
adaptive clustering and learning scheme that allows agents to learn which
neighbors they should cooperate with and which other neighbors they should
ignore. In doing so, the resulting algorithm enables the agents to identify
their clusters and to attain improved learning and estimation accuracy over
networks. We carry out a detailed mean-square analysis and assess the error
probabilities of Types I and II, i.e., false alarm and mis-detection, for the
clustering mechanism. Among other results, we establish that these
probabilities decay exponentially with the step-sizes so that the probability
of correct clustering can be made arbitrarily close to one.