pwc-archive/papers-with-abstracts huggingface dataset:
Particle Metropolis-Hastings using gradient and Hessian information
Authors: Johan Dahlin, Fredrik Lindsten, Thomas B. Schön
Date: 2013-11-04 00:00:00
Tasks: State Space Models

Particle Metropolis-Hastings (PMH) allows for Bayesian parameter inference in
nonlinear state space models by combining Markov chain Monte Carlo (MCMC) and
particle filtering. The latter is used to estimate the intractable likelihood.
In its original formulation, PMH makes use of a marginal MCMC proposal for the
parameters, typically a Gaussian random walk. However, this can lead to a poor
exploration of the parameter space and an inefficient use of the generated
particles.
  We propose a number of alternative versions of PMH that incorporate gradient
and Hessian information about the posterior into the proposal. This information
is more or less obtained as a byproduct of the likelihood estimation. Indeed,
we show how to estimate the required information using a fixed-lag particle
smoother, with a computational cost growing linearly in the number of
particles. We conclude that the proposed methods can: (i) decrease the length
of the burn-in phase, (ii) increase the mixing of the Markov chain at the
stationary phase, and (iii) make the proposal distribution scale invariant
which simplifies tuning.
SAME but Different: Fast and High-Quality Gibbs Parameter Estimation
Authors: Huasha Zhao, Biye Jiang, John Canny
Date: 2014-09-18 00:00:00
Tasks: Bayesian Inference, parameter estimation, Vocal Bursts Intensity Prediction

Gibbs sampling is a workhorse for Bayesian inference but has several
limitations when used for parameter estimation, and is often much slower than
non-sampling inference methods. SAME (State Augmentation for Marginal
Estimation) \cite{Doucet99,Doucet02} is an approach to MAP parameter estimation
which gives improved parameter estimates over direct Gibbs sampling. SAME can
be viewed as cooling the posterior parameter distribution and allows annealed
search for the MAP parameters, often yielding very high quality (lower loss)
estimates. But it does so at the expense of additional samples per iteration
and generally slower performance. On the other hand, SAME dramatically
increases the parallelism in the sampling schedule, and is an excellent match
for modern (SIMD) hardware. In this paper we explore the application of SAME to
graphical model inference on modern hardware. We show that combining SAME with
factored sample representation (or approximation) gives throughput competitive
with the fastest symbolic methods, but with potentially better quality. We
describe experiments on Latent Dirichlet Allocation, achieving speeds similar
to the fastest reported methods (online Variational Bayes) and lower
cross-validated loss than other LDA implementations. The method is simple to
implement and should be applicable to many other models.
Visual Landmark Recognition from Internet Photo Collections: A Large-Scale Evaluation
Authors: Tobias Weyand, Bastian Leibe
Date: 2014-09-18 00:00:00
Tasks: Clustering, Landmark Recognition

The task of a visual landmark recognition system is to identify photographed
buildings or objects in query photos and to provide the user with relevant
information on them. With their increasing coverage of the world's landmark
buildings and objects, Internet photo collections are now being used as a
source for building such systems in a fully automatic fashion. This process
typically consists of three steps: clustering large amounts of images by the
objects they depict; determining object names from user-provided tags; and
building a robust, compact, and efficient recognition index. To this date,
however, there is little empirical information on how well current approaches
for those steps perform in a large-scale open-set mining and recognition task.
Furthermore, there is little empirical information on how recognition
performance varies for different types of landmark objects and where there is
still potential for improvement. With this paper, we intend to fill these gaps.
Using a dataset of 500k images from Paris, we analyze each component of the
landmark recognition pipeline in order to answer the following questions: How
many and what kinds of objects can be discovered automatically? How can we best
use the resulting image clusters to recognize the object in a query? How can
the object be efficiently represented in memory for recognition? How reliably
can semantic information be extracted? And finally: What are the limiting
factors in the resulting pipeline from query to semantics? We evaluate how
different choices of methods and parameters for the individual pipeline steps
affect overall system performance and examine their effects for different query
categories such as buildings, paintings or sculptures.
Fused Lasso Additive Model
Authors: Ashley Petersen, Daniela Witten, Noah Simon
Date: 2014-09-18 00:00:00
Tasks: model

We consider the problem of predicting an outcome variable using $p$
covariates that are measured on $n$ independent observations, in the setting in
which flexible and interpretable fits are desirable. We propose the fused lasso
additive model (FLAM), in which each additive function is estimated to be
piecewise constant with a small number of adaptively-chosen knots. FLAM is the
solution to a convex optimization problem, for which a simple algorithm with
guaranteed convergence to the global optimum is provided. FLAM is shown to be
consistent in high dimensions, and an unbiased estimator of its degrees of
freedom is proposed. We evaluate the performance of FLAM in a simulation study
and on two data sets.
Belief revision by examples
Authors: Paolo Liberatore
Date: 2014-09-18 00:00:00

A common assumption in belief revision is that the reliability of the
information sources is either given, derived from temporal information, or the
same for all. This article does not describe a new semantics for integration
but the problem of obtaining the reliability of the sources given the result of
a previous merging. As an example, the relative reliability of two sensors can
be assessed given some certain observation, and allows for subsequent mergings
of data coming from them.
Learning and approximation capability of orthogonal super greedy algorithm
Authors: Jian Fang, Shao-Bo Lin, Zongben Xu
Date: 2014-09-18 00:00:00

We consider the approximation capability of orthogonal super greedy
algorithms (OSGA) and its applications in supervised learning. OSGA is
concerned with selecting more than one atoms in each iteration step, which, of
course, greatly reduces the computational burden when compared with the
conventional orthogonal greedy algorithm (OGA). We prove that even for function
classes that are not the convex hull of the dictionary, OSGA does not degrade
the approximation capability of OGA provided the dictionary is incoherent.
Based on this, we deduce a tight generalization error bound for OSGA learning.
Our results show that in the realm of supervised learning, OSGA provides a
possibility to further reduce the computational burden of OGA in the premise of
maintaining its prominent generalization capability.
Virtual Electrode Recording Tool for EXtracellular potentials (VERTEX): Comparing multi-electrode recordings from simulated and biological mammalian cortical tissue
Authors: Richard J. Tomsett, Matt Ainsworth, Alexander Thiele, Mehdi Sanayei, Xing Chen, Alwin Gieselmann, Miles A. Whittington, Mark O. Cunningham, Marcus Kaiser
Date: 2014-09-18 00:00:00

Local field potentials (LFPs) sampled with extracellular electrodes are
frequently used as a measure of population neuronal activity. However, relating
such measurements to underlying neuronal behaviour and connectivity is
non-trivial. To help study this link, we developed the Virtual Electrode
Recording Tool for EXtracellular potentials (VERTEX). We first identified a
reduced neuron model that retained the spatial and frequency filtering
characteristics of extracellular potentials from neocortical neurons. We then
developed VERTEX as an easy-to-use Matlab tool for simulating LFPs from large
populations (>100 000 neurons). A VERTEX-based simulation successfully
reproduced features of the LFPs from an in vitro multi-electrode array
recording of macaque neocortical tissue. Our model, with virtual electrodes
placed anywhere in 3D, allows direct comparisons with the in vitro recording
setup. We envisage that VERTEX will stimulate experimentalists, clinicians, and
computational neuroscientists to use models to understand the mechanisms
underlying measured brain dynamics in health and disease.
A Bayesian model for recognizing handwritten mathematical expressions
Authors: Scott MacLean, George Labahn
Date: 2014-09-18 00:00:00
Tasks: Math, model

Recognizing handwritten mathematics is a challenging classification problem,
requiring simultaneous identification of all the symbols comprising an input as
well as the complex two-dimensional relationships between symbols and
subexpressions. Because of the ambiguity present in handwritten input, it is
often unrealistic to hope for consistently perfect recognition accuracy. We
present a system which captures all recognizable interpretations of the input
and organizes them in a parse forest from which individual parse trees may be
extracted and reported. If the top-ranked interpretation is incorrect, the user
may request alternates and select the recognition result they desire. The tree
extraction step uses a novel probabilistic tree scoring strategy in which a
Bayesian network is constructed based on the structure of the input, and each
joint variable assignment corresponds to a different parse tree. Parse trees
are then reported in order of decreasing probability. Two accuracy evaluations
demonstrate that the resulting recognition system is more accurate than
previous versions (which used non-probabilistic methods) and other academic
math recognizers.
Feature Engineering for Knowledge Base Construction
Authors: Christopher Ré, Amir Abbas Sadeghian, Zifei Shan, Jaeho Shin, Feiran Wang, Sen Wu, Ce Zhang
Date: 2014-07-24 00:00:00
Tasks: Feature Engineering, Knowledge Base Construction

Knowledge base construction (KBC) is the process of populating a knowledge
base, i.e., a relational database together with inference rules, with
information extracted from documents and structured sources. KBC blurs the
distinction between two traditional database problems, information extraction
and information integration. For the last several years, our group has been
building knowledge bases with scientific collaborators. Using our approach, we
have built knowledge bases that have comparable and sometimes better quality
than those constructed by human volunteers. In contrast to these knowledge
bases, which took experts a decade or more human years to construct, many of
our projects are constructed by a single graduate student.
  Our approach to KBC is based on joint probabilistic inference and learning,
but we do not see inference as either a panacea or a magic bullet: inference is
a tool that allows us to be systematic in how we construct, debug, and improve
the quality of such systems. In addition, inference allows us to construct
these systems in a more loosely coupled way than traditional approaches. To
support this idea, we have built the DeepDive system, which has the design goal
of letting the user "think about features---not algorithms." We think of
DeepDive as declarative in that one specifies what they want but not how to get
it. We describe our approach with a focus on feature engineering, which we
argue is an understudied problem relative to its importance to end-to-end
quality.
How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation
Authors: Yoshua Bengio
Date: 2014-07-29 00:00:00

We propose to exploit {\em reconstruction} as a layer-local training signal
for deep learning. Reconstructions can be propagated in a form of target
propagation playing a role similar to back-propagation but helping to reduce
the reliance on derivatives in order to perform credit assignment across many
levels of possibly strong non-linearities (which is difficult for
back-propagation). A regularized auto-encoder tends produce a reconstruction
that is a more likely version of its input, i.e., a small move in the direction
of higher likelihood. By generalizing gradients, target propagation may also
allow to train deep networks with discrete hidden units. If the auto-encoder
takes both a representation of input and target (or of any side information) in
input, then its reconstruction of input representation provides a target
towards a representation that is more likely, conditioned on all the side
information. A deep auto-encoder decoding path generalizes gradient propagation
in a learned way that can could thus handle not just infinitesimal changes but
larger, discrete changes, hopefully allowing credit assignment through a long
chain of non-linear operations. In addition to each layer being a good
auto-encoder, the encoder also learns to please the upper layers by
transforming the data into a space where it is easier to model by them,
flattening manifolds and disentangling factors. The motivations and theoretical
justifications for this approach are laid down in this paper, along with
conjectures that will have to be verified either mathematically or
experimentally, including a hypothesis stating that such auto-encoder mediated
target propagation could play in brains the role of credit assignment through
many non-linear, noisy and discrete transformations.
Deep Regression for Face Alignment
Authors: Baoguang Shi, Xiang Bai, Wenyu Liu, Jingdong Wang
Date: 2014-09-18 00:00:00
Tasks: Face Alignment, regression

In this paper, we present a deep regression approach for face alignment. The deep architecture consists of a global layer and multi-stage local layers. We apply the back-propagation algorithm with the dropout strategy to jointly optimize the regression parameters. We show that the resulting deep regressor gradually and evenly approaches the true facial landmarks stage by stage, avoiding the tendency to yield over-strong early stage regressors while over-weak later stage regressors. Experimental results show that our approach achieves the state-of-the-art
Deontic Logic for Human Reasoning
Authors: Ulrich Furbach, Claudia Schon
Date: 2014-04-28 00:00:00

Deontic logic is shown to be applicable for modelling human reasoning. For
this the Wason selection task and the suppression task are discussed in detail.
Different versions of modelling norms with deontic logic are introduced and in
the case of the Wason selection task it is demonstrated how differences in the
performance of humans in the abstract and in the social contract case can be
explained. Furthermore it is shown that an automated theorem prover can be used
as a reasoning tool for deontic logic.
Why Local Search Excels in Expression Simplification
Authors: Ben Ruijl, Aske Plaat, Jos Vermaseren, Jaap van den Herik
Date: 2014-09-18 00:00:00
Tasks: Numerical Integration

Simplifying expressions is important to make numerical integration of large
expressions from High Energy Physics tractable. To this end, Horner's method
can be used. Finding suitable Horner schemes is assumed to be hard, due to the
lack of local heuristics. Recently, MCTS was reported to be able to find near
optimal schemes. However, several parameters had to be fine-tuned manually. In
this work, we investigate the state space properties of Horner schemes and find
that the domain is relatively flat and contains only a few local minima. As a
result, the Horner space is appropriate to be explored by Stochastic Local
Search (SLS), which has only two parameters: the number of iterations
(computation time) and the neighborhood structure. We found a suitable
neighborhood structure, leaving only the allowed computation time as a
parameter. We performed a range of experiments. The results obtained by SLS are
similar or better than those obtained by MCTS. Furthermore, we show that SLS
obtains the good results at least 10 times faster. Using SLS, we can speed up
numerical integration of many real-world large expressions by at least a factor
of 24. For High Energy Physics this means that numerical integrations that took
weeks can now be done in hours.
Fingerprint Classification Based on Depth Neural Network
Authors: Ruxin Wang, Congying Han, Yanping Wu, Tiande Guo
Date: 2014-09-18 00:00:00
Tasks: Classification, General Classification

Fingerprint classification is an effective technique for reducing the
candidate numbers of fingerprints in the stage of matching in automatic
fingerprint identification system (AFIS). In recent years, deep learning is an
emerging technology which has achieved great success in many fields, such as
image processing, natural language processing and so on. In this paper, we only
choose the orientation field as the input feature and adopt a new method
(stacked sparse autoencoders) based on depth neural network for fingerprint
classification. For the four-class problem, we achieve a classification of 93.1
percent using the depth network structure which has three hidden layers (with
1.8% rejection) in the NIST-DB4 database. And then we propose a novel method
using two classification probabilities for fuzzy classification which can
effectively enhance the accuracy of classification. By only adjusting the
probability threshold, we get the accuracy of classification is 96.1% (setting
threshold is 0.85), 97.2% (setting threshold is 0.90) and 98.0% (setting
threshold is 0.95). Using the fuzzy method, we obtain higher accuracy than
other methods.
A Tabu Search Algorithm for the Multi-period Inspector Scheduling Problem
Authors: Hu Qin, Zizhen Zhang, Yubin Xie, Andrew Lim
Date: 2014-09-17 00:00:00
Tasks: Scheduling

This paper introduces a multi-period inspector scheduling problem (MPISP),
which is a new variant of the multi-trip vehicle routing problem with time
windows (VRPTW). In the MPISP, each inspector is scheduled to perform a route
in a given multi-period planning horizon. At the end of each period, each
inspector is not required to return to the depot but has to stay at one of the
vertices for recuperation. If the remaining time of the current period is
insufficient for an inspector to travel from his/her current vertex $A$ to a
certain vertex B, he/she can choose either waiting at vertex A until the start
of the next period or traveling to a vertex C that is closer to vertex B.
Therefore, the shortest transit time between any vertex pair is affected by the
length of the period and the departure time. We first describe an approach of
computing the shortest transit time between any pair of vertices with an
arbitrary departure time. To solve the MPISP, we then propose several local
search operators adapted from classical operators for the VRPTW and integrate
them into a tabu search framework. In addition, we present a constrained
knapsack model that is able to produce an upper bound for the problem. Finally,
we evaluate the effectiveness of our algorithm with extensive experiments based
on a set of test instances. Our computational results indicate that our
approach generates high-quality solutions.
A Method for Stopping Active Learning Based on Stabilizing Predictions and the Need for User-Adjustable Stopping
Authors: Michael Bloodgood, K. Vijay-Shanker
Date: 2014-09-17 00:00:00
Tasks: Active Learning

A survey of existing methods for stopping active learning (AL) reveals the
needs for methods that are: more widely applicable; more aggressive in saving
annotations; and more stable across changing datasets. A new method for
stopping AL based on stabilizing predictions is presented that addresses these
needs. Furthermore, stopping methods are required to handle a broad range of
different annotation/performance tradeoff valuations. Despite this, the
existing body of work is dominated by conservative methods with little (if any)
attention paid to providing users with control over the behavior of stopping
methods. The proposed method is shown to fill a gap in the level of
aggressiveness available for stopping AL and supports providing users with
control over stopping behavior.
Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
Authors: Jonathan Tompson, Arjun Jain, Yann Lecun, Christoph Bregler
Date: 2014-06-11 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Pose Estimation

This paper proposes a new hybrid architecture that consists of a deep
Convolutional Network and a Markov Random Field. We show how this architecture
is successfully applied to the challenging problem of articulated human pose
estimation in monocular images. The architecture can exploit structural domain
constraints such as geometric relationships between body joint locations. We
show that joint training of these two model paradigms improves performance and
allows us to significantly outperform existing state-of-the-art techniques.
Visual Words for Automatic Lip-Reading
Authors: Ahmad Basheer Hassanat
Date: 2014-09-17 00:00:00
Tasks: Lip Reading, speech-recognition, Speech Recognition, Visual Speech Recognition

Lip reading is used to understand or interpret speech without hearing it, a
technique especially mastered by people with hearing difficulties. The ability
to lip read enables a person with a hearing impairment to communicate with
others and to engage in social activities, which otherwise would be difficult.
Recent advances in the fields of computer vision, pattern recognition, and
signal processing has led to a growing interest in automating this challenging
task of lip reading. Indeed, automating the human ability to lip read, a
process referred to as visual speech recognition, could open the door for other
novel applications. This thesis investigates various issues faced by an
automated lip-reading system and proposes a novel "visual words" based approach
to automatic lip reading. The proposed approach includes a novel automatic face
localisation scheme and a lip localisation method.
Hardness of parameter estimation in graphical models
Authors: Guy Bresler, David Gamarnik, Devavrat Shah
Date: 2014-09-12 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: parameter estimation

We consider the problem of learning the canonical parameters specifying an
undirected graphical model (Markov random field) from the mean parameters. For
graphical models representing a minimal exponential family, the canonical
parameters are uniquely determined by the mean parameters, so the problem is
feasible in principle. The goal of this paper is to investigate the
computational feasibility of this statistical task. Our main result shows that
parameter estimation is in general intractable: no algorithm can learn the
canonical parameters of a generic pair-wise binary graphical model from the
mean parameters in time bounded by a polynomial in the number of variables
(unless RP = NP). Indeed, such a result has been believed to be true (see the
monograph by Wainwright and Jordan (2008)) but no proof was known.
  Our proof gives a polynomial time reduction from approximating the partition
function of the hard-core model, known to be hard, to learning approximate
parameters. Our reduction entails showing that the marginal polytope boundary
has an inherent repulsive property, which validates an optimization procedure
over the polytope that does not use any knowledge of its structure (as required
by the ellipsoid method and others).