pwc-archive/papers-with-abstracts huggingface dataset:
Discrete Equilibrium Sampling with Arbitrary Nonequilibrium Processes
Authors: Firas Hamze, Evgeny Andryash
Date: 2015-12-03 00:00:00

We present a novel framework for performing statistical sampling, expectation
estimation, and partition function approximation using \emph{arbitrary}
heuristic stochastic processes defined over discrete state spaces. Using a
highly parallel construction we call the \emph{sequential constraining
process}, we are able to simultaneously generate states with the heuristic
process and accurately estimate their probabilities, even when they are far too
small to be realistically inferred by direct counting. After showing that both
theoretically correct importance sampling and Markov chain Monte Carlo are
possible using the sequential constraining process, we integrate it into a
methodology called \emph{state space sampling}, extending the ideas of state
space search from computer science to the sampling context. The methodology
comprises a dynamic data structure that constructs a robust Bayesian model of
the statistics generated by the heuristic process subject to an accuracy
constraint, the posterior Kullback-Leibler divergence. Sampling from the
dynamic structure will generally yield partial states, which are completed by
recursively calling the heuristic to refine the structure and resuming the
sampling. Our experiments on various Ising models suggest that state space
sampling enables heuristic state generation with accurate probability
estimates, demonstrated by illustrating the convergence of a simulated
annealing process to the Boltzmann distribution with increasing run length.
Consequently, heretofore unprecedented direct importance sampling using the
\emph{final} (marginal) distribution of a generic stochastic process is
allowed, potentially augmenting the range of algorithms at the Monte Carlo
practitioner's disposal.
Principled Parallel Mean-Field Inference for Discrete Random Fields
Authors: Pierre Baqué, Timur Bagautdinov, François Fleuret, Pascal Fua
Date: 2015-11-19 00:00:00
Proceeding: CVPR 2016 6
Tasks: Variational Inference

Mean-field variational inference is one of the most popular approaches to
inference in discrete random fields. Standard mean-field optimization is based
on coordinate descent and in many situations can be impractical. Thus, in
practice, various parallel techniques are used, which either rely on ad-hoc
smoothing with heuristically set parameters, or put strong constraints on the
type of models. In this paper, we propose a novel proximal gradient-based
approach to optimizing the variational objective. It is naturally
parallelizable and easy to implement. We prove its convergence, and then
demonstrate that, in practice, it yields faster convergence and often finds
better optima than more traditional mean-field optimization techniques.
Moreover, our method is less sensitive to the choice of parameters.
The quasispecies regime for the simple genetic algorithm with roulette-wheel selection
Authors: Raphaël Cerf
Date: 2015-06-30 00:00:00

We introduce a new parameter to discuss the behavior of a genetic algorithm.
This parameter is the mean number of exact copies of the best fit chromosomes
from one generation to the next. We argue that the genetic algorithm should
operate efficiently when this parameter is slightly larger than $1$. We
consider the case of the simple genetic algorithm with the roulette--wheel
selection mechanism. We denote by $\ell$ the length of the chromosomes, by $m$
the population size, by $p_C$ the crossover probability and by $p_M$ the
mutation probability. We start the genetic algorithm with an initial population
whose maximal fitness is equal to $f_0^*$ and whose mean fitness is equal to
${\overline{f_0}}$. We show that, in the limit of large populations, the
dynamics of the genetic algorithm depends in a critical way on the parameter
$\pi \,=\,\big({f_0^*}/{\overline{f_0}}\big) (1-p_C)(1-p_M)^\ell\,.$ Our
results suggest that the mutation and crossover probabilities should be tuned
so that, at each generation, $\text{maximal fitness} \times (1-p_C)
(1-p_M)^\ell > \text{mean fitness}$.
Weighted Schatten $p$-Norm Minimization for Image Denoising with Local and Nonlocal Regularization
Authors: Yuan Xie
Date: 2015-01-07 00:00:00
Tasks: Denoising, Image Denoising

This paper presents a patch-wise low-rank based image denoising method with
constrained variational model involving local and nonlocal regularization. On
one hand, recent patch-wise methods can be represented as a low-rank matrix
approximation problem whose convex relaxation usually depends on nuclear norm
minimization (NNM). Here, we extend the NNM to the nonconvex schatten p-norm
minimization with additional weights assigned to different singular values,
which is referred to as the Weighted Schatten p-Norm Minimization (WSNM). An
efficient algorithm is also proposed to solve the WSNM problem. The proposed
WSNM not only gives better approximation to the original low-rank assumption,
but also considers physical meanings of different data components. On the other
hand, due to the naive aggregation schema which integrates all the denoised
patches into a whole image, current patch-wise denoising methods always produce
various degree of artifacts in denoised results. Therefore, to further reduce
artifacts, a data-driven regularizer called Steering Total Variation (STV)
combined with nonlocal TV is derived for a variational model, which imposes
local and nonlocal consistency constraints on the patch-wise denoised image. A
highly simple but efficient algorithm is proposed to solve this variational
model with convergence guarantee. Both WSNM and local \& nonlocal consistent
regularization are integrated into an iterative restoration framework to
produce final results. Extensive experimental testing shows, both qualitatively
and quantitatively, that the proposed method can effectively remove noise, as
well as reduce artifacts compared with state-of-the-art methods.
Weighted Schatten $p$-Norm Minimization for Image Denoising and Background Subtraction
Authors: Yuan Xie, Shuhang Gu, Yan Liu, WangMeng Zuo, Wensheng Zhang, Lei Zhang
Date: 2015-12-03 00:00:00
Tasks: Denoising, Image Denoising

Low rank matrix approximation (LRMA), which aims to recover the underlying
low rank matrix from its degraded observation, has a wide range of applications
in computer vision. The latest LRMA methods resort to using the nuclear norm
minimization (NNM) as a convex relaxation of the nonconvex rank minimization.
However, NNM tends to over-shrink the rank components and treats the different
rank components equally, limiting its flexibility in practical applications. We
propose a more flexible model, namely the Weighted Schatten $p$-Norm
Minimization (WSNM), to generalize the NNM to the Schatten $p$-norm
minimization with weights assigned to different singular values. The proposed
WSNM not only gives better approximation to the original low-rank assumption,
but also considers the importance of different rank components. We analyze the
solution of WSNM and prove that, under certain weights permutation, WSNM can be
equivalently transformed into independent non-convex $l_p$-norm subproblems,
whose global optimum can be efficiently solved by generalized iterated
shrinkage algorithm. We apply WSNM to typical low-level vision problems, e.g.,
image denoising and background subtraction. Extensive experimental results
show, both qualitatively and quantitatively, that the proposed WSNM can more
effectively remove noise, and model complex and dynamic scenes compared with
state-of-the-art methods.
Bag Reference Vector for Multi-instance Learning
Authors: Hanqiang Song, Zhuotun Zhu, Xinggang Wang
Date: 2015-12-03 00:00:00
Tasks: Text Categorization

Multi-instance learning (MIL) has a wide range of applications due to its
distinctive characteristics. Although many state-of-the-art algorithms have
achieved decent performances, a plurality of existing methods solve the problem
only in instance level rather than excavating relations among bags. In this
paper, we propose an efficient algorithm to describe each bag by a
corresponding feature vector via comparing it with other bags. In other words,
the crucial information of a bag is extracted from the similarity between that
bag and other reference bags. In addition, we apply extensions of Hausdorff
distance to representing the similarity, to a certain extent, overcoming the
key challenge of MIL problem, the ambiguity of instances' labels in positive
bags. Experimental results on benchmarks and text categorization tasks show
that the proposed method outperforms the previous state-of-the-art by a large
margin.
Fast Low-Rank Matrix Learning with Nonconvex Regularization
Authors: Quanming Yao, James T. Kwok, Wenliang Zhong
Date: 2015-12-03 00:00:00
Tasks: Matrix Completion

Low-rank modeling has a lot of important applications in machine learning,
computer vision and social network analysis. While the matrix rank is often
approximated by the convex nuclear norm, the use of nonconvex low-rank
regularizers has demonstrated better recovery performance. However, the
resultant optimization problem is much more challenging. A very recent
state-of-the-art is based on the proximal gradient algorithm. However, it
requires an expensive full SVD in each proximal step. In this paper, we show
that for many commonly-used nonconvex low-rank regularizers, a cutoff can be
derived to automatically threshold the singular values obtained from the
proximal operator. This allows the use of power method to approximate the SVD
efficiently. Besides, the proximal operator can be reduced to that of a much
smaller matrix projected onto this leading subspace. Convergence, with a rate
of O(1/T) where T is the number of iterations, can be guaranteed. Extensive
experiments are performed on matrix completion and robust principal component
analysis. The proposed method achieves significant speedup over the
state-of-the-art. Moreover, the matrix solution obtained is more accurate and
has a lower rank than that of the traditional nuclear norm regularizer.
A Study on Artificial Intelligence IQ and Standard Intelligent Model
Authors: Feng Liu, Yong Shi
Date: 2015-12-03 00:00:00

Currently, potential threats of artificial intelligence (AI) to human have
triggered a large controversy in society, behind which, the nature of the issue
is whether the artificial intelligence (AI) system can be evaluated
quantitatively. This article analyzes and evaluates the challenges that the AI
development level is facing, and proposes that the evaluation methods for the
human intelligence test and the AI system are not uniform; and the key reason
for which is that none of the models can uniformly describe the AI system and
the beings like human. Aiming at this problem, a standard intelligent system
model is established in this study to describe the AI system and the beings
like human uniformly. Based on the model, the article makes an abstract
mathematical description, and builds the standard intelligent machine
mathematical model; expands the Von Neumann architecture and proposes the
Liufeng - Shiyong architecture; gives the definition of the artificial
intelligence IQ, and establishes the artificial intelligence scale and the
evaluation method; conduct the test on 50 search engines and three human
subjects at different ages across the world, and finally obtains the ranking of
the absolute IQ and deviation IQ ranking for artificial intelligence IQ 2014.
Modeling Human Understanding of Complex Intentional Action with a Bayesian Nonparametric Subgoal Model
Authors: Ryo Nakahashi, Chris L. Baker, Joshua B. Tenenbaum
Date: 2015-12-03 00:00:00

Most human behaviors consist of multiple parts, steps, or subtasks. These
structures guide our action planning and execution, but when we observe others,
the latent structure of their actions is typically unobservable, and must be
inferred in order to learn new skills by demonstration, or to assist others in
completing their tasks. For example, an assistant who has learned the subgoal
structure of a colleague's task can more rapidly recognize and support their
actions as they unfold. Here we model how humans infer subgoals from
observations of complex action sequences using a nonparametric Bayesian model,
which assumes that observed actions are generated by approximately rational
planning over unknown subgoal sequences. We test this model with a behavioral
experiment in which humans observed different series of goal-directed actions,
and inferred both the number and composition of the subgoal sequences
associated with each goal. The Bayesian model predicts human subgoal inferences
with high accuracy, and significantly better than several alternative models
and straightforward heuristics. Motivated by this result, we simulate how
learning and inference of subgoals can improve performance in an artificial
user assistance task. The Bayesian model learns the correct subgoals from fewer
observations, and better assists users by more rapidly and accurately inferring
the goal of their actions than alternative approaches.
A New Statistical Framework for Genetic Pleiotropic Analysis of High Dimensional Phenotype Data
Authors: Panpan Wang, Mohammad Rahman, Li Jin, Momiao Xiong
Date: 2015-12-03 00:00:00
Tasks: Computational Efficiency

The widely used genetic pleiotropic analysis of multiple phenotypes are often
designed for examining the relationship between common variants and a few
phenotypes. They are not suited for both high dimensional phenotypes and high
dimensional genotype (next-generation sequencing) data. To overcome these
limitations, we develop sparse structural equation models (SEMs) as a general
framework for a new paradigm of genetic analysis of multiple phenotypes. To
incorporate both common and rare variants into the analysis, we extend the
traditional multivariate SEMs to sparse functional SEMs. To deal with high
dimensional phenotype and genotype data, we employ functional data analysis and
the alternative direction methods of multiplier (ADMM) techniques to reduce
data dimension and improve computational efficiency. Using large scale
simulations we showed that the proposed methods have higher power to detect
true causal genetic pleiotropic structure than other existing methods.
Simulations also demonstrate that the gene-based pleiotropic analysis has
higher power than the single variant-based pleiotropic analysis. The proposed
method is applied to exome sequence data from the NHLBI Exome Sequencing
Project (ESP) with 11 phenotypes, which identifies a network with 137 genes
connected to 11 phenotypes and 341 edges. Among them, 114 genes showed
pleiotropic genetic effects and 45 genes were reported to be associated with
phenotypes in the analysis or other cardiovascular disease (CVD) related
phenotypes in the literature.
A Literature Survey of various Fingerprint De-noising Techniques to justify the need of a new De-noising model based upon Pixel Component Analysis
Authors: Siddharth Choubey, Deepika Banchhor
Date: 2015-12-03 00:00:00

Image Preprocessing is a vital step in the field of image processing for
biometric pattern recognition. This paper studies and reviews various classical
and modern fingerprint image de-noising models. The various model used for
de-noising ranges widely from transform matrix using frequency, histogram model
de-noising, de-noising by introducing Gabor filter and its types to enhance
fingerprint images.
  The output efficiency of various de-noising model proposed earlier is
calculated on the basis of SNR (signal to noise ratio) and MSE (mean square
error rate). Our simulated experimental results indicates that incorporating
the de-noising model based on Gabor filter inside domain of wavelet ranges with
composite method only betters MSE (Mean Square Error). Improved MSE without
significant improvement in SNR improves the fingerprint images only by a little
margin which is non-optimal in nature. Thus the objective of this research
paper is to build an optimal de-noising model for fingerprint images so that
its usage in biometric authentication can be more robust in nature.
Bayesian inference via rejection filtering
Authors: Nathan Wiebe, Christopher Granade, Ashish Kapoor, Krysta M. Svore
Date: 2015-11-20 00:00:00
Tasks: Bayesian Inference, General Classification

We provide a method for approximating Bayesian inference using rejection
sampling. We not only make the process efficient, but also dramatically reduce
the memory required relative to conventional methods by combining rejection
sampling with particle filtering. We also provide an approximate form of
rejection sampling that makes rejection filtering tractable in cases where
exact rejection sampling is not efficient. Finally, we present several
numerical examples of rejection filtering that show its ability to track time
dependent parameters in online settings and also benchmark its performance on
MNIST classification problems.
Compressive hyperspectral imaging via adaptive sampling and dictionary learning
Authors: Mingrui Yang, Frank de Hoog, Yuqi Fan, Wen Hu
Date: 2015-12-02 00:00:00
Tasks: Compressive Sensing, Dictionary Learning

In this paper, we propose a new sampling strategy for hyperspectral signals
that is based on dictionary learning and singular value decomposition (SVD).
Specifically, we first learn a sparsifying dictionary from training spectral
data using dictionary learning. We then perform an SVD on the dictionary and
use the first few left singular vectors as the rows of the measurement matrix
to obtain the compressive measurements for reconstruction. The proposed method
provides significant improvement over the conventional compressive sensing
approaches. The reconstruction performance is further improved by
reconditioning the sensing matrix using matrix balancing. We also demonstrate
that the combination of dictionary learning and SVD is robust by applying them
to different datasets.
Cleaning Schedule Optimization of Heat Exchanger Networks Using Particle Swarm Optimization
Authors: Totok R. Biyanto, Sumitra Wira Suganda, Matraji, Yerry Susatio, Heri Justiono, Sarwono
Date: 2015-12-02 00:00:00

Oil refinery is one of industries that require huge energy consumption. The
today technology advance requires energy saving. Heat integration is a method
used to minimize the energy comsumption though the implementation of Heat
Exchanger Network (HEN). CPT is one of types of Heat Exchanger Network (HEN)
that functions to recover the heat in the flow of product or waste. HEN
comprises a number of heat exchangers (HEs) that are serially connected.
However, the presence of fouling in the heat exchanger has caused the decline
of the performance of both heat exchangers and all heat exchanger networks.
Fouling can not be avoided. However, it can be mitigated. In industry, periodic
heat exchanger cleaning is the most effective and widely used mitigation
technique. On the other side, a very frequent cleaning of heat exchanger can be
much costly in maintenance and lost of production. In this way, an accurate
optimization technique of cleaning schedule interval of heat exchanger is very
essential. Commonly, this technique involves three elements: model to simulate
the heat exchanger network, representative fouling model to describe the
fouling behavior and suitable optimization algorithm to solve the problem of
clening schedule interval for heat exchanger network. This paper describe the
optimization of interval cleaning schedule of HEN within the 44-month period
using PSO (particle swarm optimization). The number of iteration used to
achieve the convergent is 100 iterations and the fitness value in PSO
correlated with the amount of heat recovery, cleaning cost, and additional
pumping cost. The saving after the optimization of cleaning schedule of HEN in
this research achieved at $ 1.236 millions or 23% of maximum potential savings.
The GTR-model: a universal framework for quantum-like measurements
Authors: Diederik Aerts, Massimiliano Sassoli de Bianchi
Date: 2015-12-02 00:00:00

We present a very general geometrico-dynamical description of physical or
more abstract entities, called the 'general tension-reduction' (GTR) model,
where not only states, but also measurement-interactions can be represented,
and the associated outcome probabilities calculated. Underlying the model is
the hypothesis that indeterminism manifests as a consequence of unavoidable
fluctuations in the experimental context, in accordance with the
'hidden-measurements interpretation' of quantum mechanics. When the structure
of the state space is Hilbertian, and measurements are of the 'universal' kind,
i.e., are the result of an average over all possible ways of selecting an
outcome, the GTR-model provides the same predictions of the Born rule, and
therefore provides a natural completed version of quantum mechanics. However,
when the structure of the state space is non-Hilbertian and/or not all possible
ways of selecting an outcome are available to be actualized, the predictions of
the model generally differ from the quantum ones, especially when sequential
measurements are considered. Some paradigmatic examples will be discussed,
taken from physics and human cognition. Particular attention will be given to
some known psychological effects, like question order effects and response
replicability, which we show are able to generate non-Hilbertian statistics. We
also suggest a realistic interpretation of the GTR-model, when applied to human
cognition and decision, which we think could become the generally adopted
interpretative framework in quantum cognition research.
Optimization Monte Carlo: Efficient and Embarrassingly Parallel Likelihood-Free Inference
Authors: Edward Meeds, Max Welling
Date: 2015-06-11 00:00:00
Proceeding: NeurIPS 2015 12

We describe an embarrassingly parallel, anytime Monte Carlo method for
likelihood-free models. The algorithm starts with the view that the
stochasticity of the pseudo-samples generated by the simulator can be
controlled externally by a vector of random numbers u, in such a way that the
outcome, knowing u, is deterministic. For each instantiation of u we run an
optimization procedure to minimize the distance between summary statistics of
the simulator and the data. After reweighing these samples using the prior and
the Jacobian (accounting for the change of volume in transforming from the
space of summary statistics to the space of parameters) we show that this
weighted ensemble represents a Monte Carlo estimate of the posterior
distribution. The procedure can be run embarrassingly parallel (each node
handling one sample) and anytime (by allocating resources to the worst
performing sample). The procedure is validated on six experiments.
Microclustering: When the Cluster Sizes Grow Sublinearly with the Size of the Data Set
Authors: Jeffrey Miller, Brenda Betancourt, Abbas Zaidi, Hanna Wallach, Rebecca C. Steorts
Date: 2015-12-02 00:00:00
Tasks: Clustering, Entity Resolution

Most generative models for clustering implicitly assume that the number of
data points in each cluster grows linearly with the total number of data
points. Finite mixture models, Dirichlet process mixture models, and
Pitman--Yor process mixture models make this assumption, as do all other
infinitely exchangeable clustering models. However, for some tasks, this
assumption is undesirable. For example, when performing entity resolution, the
size of each cluster is often unrelated to the size of the data set.
Consequently, each cluster contains a negligible fraction of the total number
of data points. Such tasks therefore require models that yield clusters whose
sizes grow sublinearly with the size of the data set. We address this
requirement by defining the \emph{microclustering property} and introducing a
new model that exhibits this property. We compare this model to several
commonly used clustering models by checking model fit using real and simulated
data sets.