pwc-archive/papers-with-abstracts huggingface dataset:
Black box variational inference for state space models
Authors: Evan Archer, Il Memming Park, Lars Buesing, John Cunningham, Liam Paninski
Date: 2015-11-23 00:00:00
Tasks: State Space Models, Time Series, Time Series Analysis, Variational Inference

Latent variable time-series models are among the most heavily used tools from
machine learning and applied statistics. These models have the advantage of
learning latent structure both from noisy observations and from the temporal
ordering in the data, where it is assumed that meaningful correlation structure
exists across time. A few highly-structured models, such as the linear
dynamical system with linear-Gaussian observations, have closed-form inference
procedures (e.g. the Kalman Filter), but this case is an exception to the
general rule that exact posterior inference in more complex generative models
is intractable. Consequently, much work in time-series modeling focuses on
approximate inference procedures for one particular class of models. Here, we
extend recent developments in stochastic variational inference to develop a
`black-box' approximate inference technique for latent variable models with
latent dynamical structure. We propose a structured Gaussian variational
approximate posterior that carries the same intuition as the standard Kalman
filter-smoother but, importantly, permits us to use the same inference approach
to approximate the posterior of much more general, nonlinear latent variable
generative models. We show that our approach recovers accurate estimates in the
case of basic models with closed-form posteriors, and more interestingly
performs well in comparison to variational approaches that were designed in a
bespoke fashion for specific non-conjugate models.
Interpretable Two-level Boolean Rule Learning for Classification
Authors: Guolong Su, Dennis Wei, Kush R. Varshney, Dmitry M. Malioutov
Date: 2015-11-23 00:00:00
Tasks: Classification, General Classification, Vocal Bursts Valence Prediction

This paper proposes algorithms for learning two-level Boolean rules in
Conjunctive Normal Form (CNF, i.e. AND-of-ORs) or Disjunctive Normal Form (DNF,
i.e. OR-of-ANDs) as a type of human-interpretable classification model, aiming
for a favorable trade-off between the classification accuracy and the
simplicity of the rule. Two formulations are proposed. The first is an integer
program whose objective function is a combination of the total number of errors
and the total number of features used in the rule. We generalize a previously
proposed linear programming (LP) relaxation from one-level to two-level rules.
The second formulation replaces the 0-1 classification error with the Hamming
distance from the current two-level rule to the closest rule that correctly
classifies a sample. Based on this second formulation, block coordinate descent
and alternating minimization algorithms are developed. Experiments show that
the two-level rules can yield noticeably better performance than one-level
rules due to their dramatically larger modeling capacity, and the two
algorithms based on the Hamming distance formulation are generally superior to
the other two-level rule learning methods in our comparison. A proposed
approach to binarize any fractional values in the optimal solutions of LP
relaxations is also shown to be effective.
Node Specificity in Convolutional Deep Nets Depends on Receptive Field Position and Size
Authors: Karl Zipser
Date: 2015-11-23 00:00:00
Tasks: Position, Specificity

In convolutional deep neural networks, receptive field (RF) size increases
with hierarchical depth. When RF size approaches full coverage of the input
image, different RF positions result in RFs with different specificity, as
portions of the RF fall out of the input space. This leads to a departure from
the convolutional concept of positional invariance and opens the possibility
for complex forms of context specificity.
Modular Autoencoders for Ensemble Feature Extraction
Authors: Henry W. J. Reeve, Gavin Brown
Date: 2015-11-23 00:00:00

We introduce the concept of a Modular Autoencoder (MAE), capable of learning
a set of diverse but complementary representations from unlabelled data, that
can later be used for supervised tasks. The learning of the representations is
controlled by a trade off parameter, and we show on six benchmark datasets the
optimum lies between two extremes: a set of smaller, independent autoencoders
each with low capacity, versus a single monolithic encoding, outperforming an
appropriate baseline. In the present paper we explore the special case of
linear MAE, and derive an SVD-based algorithm which converges several orders of
magnitude faster than gradient descent.
Unveiling the Dreams of Word Embeddings: Towards Language-Driven Image Generation
Authors: Angeliki Lazaridou, Dat Tien Nguyen, Raffaella Bernardi, Marco Baroni
Date: 2015-06-10 00:00:00
Tasks: Image Generation, Word Embeddings

We introduce language-driven image generation, the task of generating an
image visualizing the semantic contents of a word embedding, e.g., given the
word embedding of grasshopper, we generate a natural image of a grasshopper. We
implement a simple method based on two mapping functions. The first takes as
input a word embedding (as produced, e.g., by the word2vec toolkit) and maps it
onto a high-level visual space (e.g., the space defined by one of the top
layers of a Convolutional Neural Network). The second function maps this
abstract visual representation to pixel space, in order to generate the target
image. Several user studies suggest that the current system produces images
that capture general visual properties of the concepts encoded in the word
embedding, such as color or typical environment, and are sufficient to
discriminate between general categories of objects.
Training Very Deep Networks
Authors: Rupesh Kumar Srivastava, Klaus Greff, Jürgen Schmidhuber
Date: 2015-07-22 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Image Classification

Theoretical and empirical evidence indicates that the depth of neural
networks is crucial for their success. However, training becomes more difficult
as depth increases, and training of very deep networks remains an open problem.
Here we introduce a new architecture designed to overcome this. Our so-called
highway networks allow unimpeded information flow across many layers on
information highways. They are inspired by Long Short-Term Memory recurrent
networks and use adaptive gating units to regulate the information flow. Even
with hundreds of layers, highway networks can be trained directly through
simple gradient descent. This enables the study of extremely deep and efficient
architectures.
Stochastic Parallel Block Coordinate Descent for Large-scale Saddle Point Problems
Authors: Zhanxing Zhu, Amos J. Storkey
Date: 2015-11-23 00:00:00
Tasks: feature selection

We consider convex-concave saddle point problems with a separable structure
and non-strongly convex functions. We propose an efficient stochastic block
coordinate descent method using adaptive primal-dual updates, which enables
flexible parallel optimization for large-scale problems. Our method shares the
efficiency and flexibility of block coordinate descent methods with the
simplicity of primal-dual methods and utilizing the structure of the separable
convex-concave saddle point problem. It is capable of solving a wide range of
machine learning applications, including robust principal component analysis,
Lasso, and feature selection by group Lasso, etc. Theoretically and
empirically, we demonstrate significantly better performance than
state-of-the-art methods in all these applications.
Sparse Recovery via Partial Regularization: Models, Theory and Algorithms
Authors: Zhaosong Lu, Xiaorui Li
Date: 2015-11-23 00:00:00
Tasks: compressed sensing

In the context of sparse recovery, it is known that most of existing
regularizers such as $\ell_1$ suffer from some bias incurred by some leading
entries (in magnitude) of the associated vector. To neutralize this bias, we
propose a class of models with partial regularizers for recovering a sparse
solution of a linear system. We show that every local minimizer of these models
is sufficiently sparse or the magnitude of all its nonzero entries is above a
uniform constant depending only on the data of the linear system. Moreover, for
a class of partial regularizers, any global minimizer of these models is a
sparsest solution to the linear system. We also establish some sufficient
conditions for local or global recovery of the sparsest solution to the linear
system, among which one of the conditions is weaker than the best known
restricted isometry property (RIP) condition for sparse recovery by $\ell_1$.
In addition, a first-order feasible augmented Lagrangian (FAL) method is
proposed for solving these models, in which each subproblem is solved by a
nonmonotone proximal gradient (NPG) method. Despite the complication of the
partial regularizers, we show that each proximal subproblem in NPG can be
solved as a certain number of one-dimensional optimization problems, which
usually have a closed-form solution. We also show that any accumulation point
of the sequence generated by FAL is a first-order stationary point of the
models. Numerical results on compressed sensing and sparse logistic regression
demonstrate that the proposed models substantially outperform the widely used
ones in the literature in terms of solution quality.
Sparse Linear Models applied to Power Quality Disturbance Classification
Authors: Andrés F. López-Lopera, Mauricio A. Álvarez, Ávaro A. Orozco
Date: 2015-11-23 00:00:00
Tasks: Classification, General Classification

Power quality (PQ) analysis describes the non-pure electric signals that are
usually present in electric power systems. The automatic recognition of PQ
disturbances can be seen as a pattern recognition problem, in which different
types of waveform distortion are differentiated based on their features.
Similar to other quasi-stationary signals, PQ disturbances can be decomposed
into time-frequency dependent components by using time-frequency or time-scale
transforms, also known as dictionaries. These dictionaries are used in the
feature extraction step in pattern recognition systems. Short-time Fourier,
Wavelets and Stockwell transforms are some of the most common dictionaries used
in the PQ community, aiming to achieve a better signal representation. To the
best of our knowledge, previous works about PQ disturbance classification have
been restricted to the use of one among several available dictionaries. Taking
advantage of the theory behind sparse linear models (SLM), we introduce a
sparse method for PQ representation, starting from overcomplete dictionaries.
In particular, we apply Group Lasso. We employ different types of
time-frequency (or time-scale) dictionaries to characterize the PQ
disturbances, and evaluate their performance under different pattern
recognition algorithms. We show that the SLM reduce the PQ classification
complexity promoting sparse basis selection, and improving the classification
accuracy.
Über die Klassifizierung von Knoten in dynamischen Netzwerken mit Inhalt
Authors: Martin Thoma
Date: 2015-11-23 00:00:00
Tasks: CPU, Missing Labels

This paper explains the DYCOS-Algorithm as it was introduced in by Aggarwal
and Li in 2011. It operates on graphs whichs nodes are partially labeled and
automatically adds missing labels to nodes. To do so, the DYCOS algorithm makes
use of the structure of the graph as well as content which is assigned to the
node. Aggarwal and Li measured in an experimental analysis that DYCOS adds the
missing labels to a Graph with 19396 nodes of which 14814 are labeled and
another Graph with 806635 nodes of which 18999 are labeld on one core of an
Intel Xeon 2.5 GHz CPU with 32 G RAM within less than a minute. Additionally,
extensions of the DYCOS algorithm are proposed.
  -----
  In dieser Arbeit wird der DYCOS-Algorithmus, wie er 2011 von Aggarwal und Li
vorgestellt wurde, erkl\"art. Er arbeitet auf Graphen, deren Knoten teilweise
mit Beschriftungen versehen sind und erg\"anzt automatisch Beschriftungen f\"ur
Knoten, die bisher noch keine Beschriftung haben. Dieser Vorgang wird
"Klassifizierung" genannt. Dazu verwendet er die Struktur des Graphen sowie
textuelle Informationen, die den Knoten zugeordnet sind. Die von Aggarwal und
Li beschriebene experimentelle Analyse ergab, dass er auch auf dynamischen
Graphen mit 19396 bzw. 806635 Knoten, von denen nur 14814 bzw. 18999
beschriftet waren, innerhalb von weniger als einer Minute auf einem Kern einer
Intel Xeon 2.5 GHz CPU mit 32 G RAM ausgef\"uhrt werden kann. Zus\"atzlich wird
die Ver\"offentlichung von Aggarwal und Li kritisch er\"ortert und und es
werden m\"ogliche Erweiterungen des DYCOS-Algorithmus vorgeschlagen.
Face Alignment Across Large Poses: A 3D Solution
Authors: Xiangyu Zhu, Zhen Lei, Xiaoming Liu, Hailin Shi, Stan Z. Li
Date: 2015-11-23 00:00:00
Proceeding: CVPR 2016 6
Tasks: 3D Face Reconstruction, Face Alignment, Face Model, Head Pose Estimation

Face alignment, which fits a face model to an image and extracts the semantic
meanings of facial pixels, has been an important topic in CV community.
However, most algorithms are designed for faces in small to medium poses (below
45 degree), lacking the ability to align faces in large poses up to 90 degree.
The challenges are three-fold: Firstly, the commonly used landmark-based face
model assumes that all the landmarks are visible and is therefore not suitable
for profile views. Secondly, the face appearance varies more dramatically
across large poses, ranging from frontal view to profile view. Thirdly,
labelling landmarks in large poses is extremely challenging since the invisible
landmarks have to be guessed. In this paper, we propose a solution to the three
problems in an new alignment framework, called 3D Dense Face Alignment (3DDFA),
in which a dense 3D face model is fitted to the image via convolutional neutral
network (CNN). We also propose a method to synthesize large-scale training
samples in profile views to solve the third problem of data labelling.
Experiments on the challenging AFLW database show that our approach achieves
significant improvements over state-of-the-art methods.
DAG-Recurrent Neural Networks For Scene Labeling
Authors: Bing Shuai, Zhen Zuo, Gang Wang, Bing Wang
Date: 2015-09-02 00:00:00
Proceeding: CVPR 2016 6
Tasks: General Classification, Scene Labeling, Semantic Segmentation

In image labeling, local representations for image units are usually
generated from their surrounding image patches, thus long-range contextual
information is not effectively encoded. In this paper, we introduce recurrent
neural networks (RNNs) to address this issue. Specifically, directed acyclic
graph RNNs (DAG-RNNs) are proposed to process DAG-structured images, which
enables the network to model long-range semantic dependencies among image
units. Our DAG-RNNs are capable of tremendously enhancing the discriminative
power of local representations, which significantly benefits the local
classification. Meanwhile, we propose a novel class weighting function that
attends to rare classes, which phenomenally boosts the recognition accuracy for
non-frequent classes. Integrating with convolution and deconvolution layers,
our DAG-RNNs achieve new state-of-the-art results on the challenging SiftFlow,
CamVid and Barcelona benchmarks.
Poisson Subsampling Algorithms for Large Sample Linear Regression in Massive Data
Authors: Rong Zhu
Date: 2015-09-07 00:00:00
Tasks: regression

Large sample size brings the computation bottleneck for modern data analysis.
Subsampling is one of efficient strategies to handle this problem. In previous
studies, researchers make more fo- cus on subsampling with replacement (SSR)
than on subsampling without replacement (SSWR). In this paper we investigate a
kind of SSWR, poisson subsampling (PSS), for fast algorithm in ordinary
least-square problem. We establish non-asymptotic property, i.e, the error
bound of the correspond- ing subsample estimator, which provide a tradeoff
between computation cost and approximation efficiency. Besides the
non-asymptotic result, we provide asymptotic consistency and normality of the
subsample estimator. Methodologically, we propose a two-step subsampling
algorithm, which is efficient with respect to a statistical objective and
independent on the linear model assumption.. Synthetic and real data are used
to empirically study our proposed subsampling strategies. We argue by these
empirical studies that, (1) our proposed two-step algorithm has obvious
advantage when the assumed linear model does not accurate, and (2) the PSS
strategy performs obviously better than SSR when the subsampling ratio
increases.
Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions
Authors: Amar Shah, Zoubin Ghahramani
Date: 2015-11-23 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Bayesian Optimization, global-optimization

We develop parallel predictive entropy search (PPES), a novel algorithm for
Bayesian optimization of expensive black-box objective functions. At each
iteration, PPES aims to select a batch of points which will maximize the
information gain about the global maximizer of the objective. Well known
strategies exist for suggesting a single evaluation point based on previous
observations, while far fewer are known for selecting batches of points to
evaluate in parallel. The few batch selection schemes that have been studied
all resort to greedy methods to compute an optimal batch. To the best of our
knowledge, PPES is the first non-greedy batch Bayesian optimization strategy.
We demonstrate the benefit of this approach in optimization performance on both
synthetic and real world applications, including problems in machine learning,
rocket science and robotics.
What Happened to My Dog in That Network: Unraveling Top-down Generators in Convolutional Neural Networks
Authors: Patrick W. Gallagher, Shuai Tang, Zhuowen Tu
Date: 2015-11-23 00:00:00
Tasks: Data Augmentation, Zero-Shot Learning

Top-down information plays a central role in human perception, but plays
relatively little role in many current state-of-the-art deep networks, such as
Convolutional Neural Networks (CNNs). This work seeks to explore a path by
which top-down information can have a direct impact within current deep
networks. We explore this path by learning and using "generators" corresponding
to the network internal effects of three types of transformation (each a
restriction of a general affine transformation): rotation, scaling, and
translation. We demonstrate how these learned generators can be used to
transfer top-down information to novel settings, as mediated by the "feature
flows" that the transformations (and the associated generators) correspond to
inside the network. Specifically, we explore three aspects: 1) using generators
as part of a method for synthesizing transformed images --- given a previously
unseen image, produce versions of that image corresponding to one or more
specified transformations, 2) "zero-shot learning" --- when provided with a
feature flow corresponding to the effect of a transformation of unknown amount,
leverage learned generators as part of a method by which to perform an accurate
categorization of the amount of transformation, even for amounts never observed
during training, and 3) (inside-CNN) "data augmentation" --- improve the
classification performance of an existing network by using the learned
generators to directly provide additional training "inside the CNN".
Towards Predicting the Likeability of Fashion Images
Authors: Jinghua Wang, Abrar Abdul Nabi, Gang Wang, Chengde Wan, Tian-Tsong Ng
Date: 2015-11-17 00:00:00
Tasks: Attribute

In this paper, we propose a method for ranking fashion images to find the
ones which might be liked by more people. We collect two new datasets from
image sharing websites (Pinterest and Polyvore). We represent fashion images
based on attributes: semantic attributes and data-driven attributes. To learn
semantic attributes from limited training data, we use an algorithm on
multi-task convolutional neural networks to share visual knowledge among
different semantic attribute categories. To discover data-driven attributes
unsupervisedly, we propose an algorithm to simultaneously discover visual
clusters and learn fashion-specific feature representations. Given attributes
as representations, we propose to learn a ranking SPN (sum product networks) to
rank pairs of fashion images. The proposed ranking SPN can capture the
high-order correlations of the attributes. We show the effectiveness of our
method on our two newly collected datasets.
Object-Proposal Evaluation Protocol is 'Gameable'
Authors: Neelima Chavali, Harsh Agrawal, Aroma Mahendru, Dhruv Batra
Date: 2015-05-21 00:00:00
Proceeding: CVPR 2016 6
Tasks: Diagnostic, Object, object-detection, Object Detection, Object Discovery

Object proposals have quickly become the de-facto pre-processing step in a
number of vision pipelines (for object detection, object discovery, and other
tasks). Their performance is usually evaluated on partially annotated datasets.
In this paper, we argue that the choice of using a partially annotated dataset
for evaluation of object proposals is problematic -- as we demonstrate via a
thought experiment, the evaluation protocol is 'gameable', in the sense that
progress under this protocol does not necessarily correspond to a "better"
category independent object proposal algorithm.
  To alleviate this problem, we: (1) Introduce a nearly-fully annotated version
of PASCAL VOC dataset, which serves as a test-bed to check if object proposal
techniques are overfitting to a particular list of categories. (2) Perform an
exhaustive evaluation of object proposal methods on our introduced nearly-fully
annotated PASCAL dataset and perform cross-dataset generalization experiments;
and (3) Introduce a diagnostic experiment to detect the bias capacity in an
object proposal algorithm. This tool circumvents the need to collect a densely
annotated dataset, which can be expensive and cumbersome to collect. Finally,
we plan to release an easy-to-use toolbox which combines various publicly
available implementations of object proposal algorithms which standardizes the
proposal generation and evaluation so that new methods can be added and
evaluated on different datasets. We hope that the results presented in the
paper will motivate the community to test the category independence of various
object proposal methods by carefully choosing the evaluation protocol.
Bayesian Evidence and Model Selection
Authors: Kevin H. Knuth, Michael Habeck, Nabin K. Malakar, Asim M. Mubeen, Ben Placek
Date: 2014-11-11 00:00:00
Tasks: model, Model Selection

In this paper we review the concepts of Bayesian evidence and Bayes factors,
also known as log odds ratios, and their application to model selection. The
theory is presented along with a discussion of analytic, approximate and
numerical techniques. Specific attention is paid to the Laplace approximation,
variational Bayes, importance sampling, thermodynamic integration, and nested
sampling and its recent variants. Analogies to statistical physics, from which
many of these techniques originate, are discussed in order to provide readers
with deeper insights that may lead to new techniques. The utility of Bayesian
model testing in the domain sciences is demonstrated by presenting four
specific practical examples considered within the context of signal processing
in the areas of signal detection, sensor characterization, scientific model
selection and molecular force characterization.