pwc-archive/papers-with-abstracts huggingface dataset:
On some provably correct cases of variational inference for topic models
Authors: Pranjal Awasthi, Andrej Risteski
Date: 2015-03-23 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Clustering, Dictionary Learning, Topic Models, Variational Inference

Variational inference is a very efficient and popular heuristic used in
various forms in the context of latent variable models. It's closely related to
Expectation Maximization (EM), and is applied when exact EM is computationally
infeasible. Despite being immensely popular, current theoretical understanding
of the effectiveness of variaitonal inference based algorithms is very limited.
In this work we provide the first analysis of instances where variational
inference algorithms converge to the global optimum, in the setting of topic
models.
  More specifically, we show that variational inference provably learns the
optimal parameters of a topic model under natural assumptions on the topic-word
matrix and the topic priors. The properties that the topic word matrix must
satisfy in our setting are related to the topic expansion assumption introduced
in (Anandkumar et al., 2013), as well as the anchor words assumption in (Arora
et al., 2012c). The assumptions on the topic priors are related to the well
known Dirichlet prior, introduced to the area of topic modeling by (Blei et
al., 2003).
  It is well known that initialization plays a crucial role in how well
variational based algorithms perform in practice. The initializations that we
use are fairly natural. One of them is similar to what is currently used in
LDA-c, the most popular implementation of variational inference for topic
models. The other one is an overlapping clustering algorithm, inspired by a
work by (Arora et al., 2014) on dictionary learning, which is very simple and
efficient.
  While our primary goal is to provide insights into when variational inference
might work in practice, the multiplicative, rather than the additive nature of
the variational inference updates forces us to use fairly non-standard proof
arguments, which we believe will be of general interest.
Bayesian Hypothesis Testing for Block Sparse Signal Recovery
Authors: Mehdi Korki, Hadi Zayyani, Jingxin Zhang
Date: 2015-08-22 00:00:00
Tasks: Two-sample testing

This letter presents a novel Block Bayesian Hypothesis Testing Algorithm
(Block-BHTA) for reconstructing block sparse signals with unknown block
structures. The Block-BHTA comprises the detection and recovery of the
supports, and the estimation of the amplitudes of the block sparse signal. The
support detection and recovery is performed using a Bayesian hypothesis
testing. Then, based on the detected and reconstructed supports, the nonzero
amplitudes are estimated by linear MMSE. The effectiveness of Block-BHTA is
demonstrated by numerical experiments.
A Self-Adaptive Network Protection System
Authors: Mohamed Hassan
Date: 2014-05-08 00:00:00

In this treatise we aim to build a hybrid network automated (self-adaptive)
security threats discovery and prevention system; by using unconventional
techniques and methods, including fuzzy logic and biological inspired
algorithms under the context of soft computing.
Parallel software implementation of recursive multidimensional digital filters for point-target detection in cluttered infrared scenes
Authors: Hugh L. Kennedy
Date: 2014-08-15 00:00:00
Tasks: CPU, GPU, Optical Flow Estimation

A technique for the enhancement of point targets in clutter is described. The
local 3-D spectrum at each pixel is estimated recursively. An optical
flow-field for the textured background is then generated using the 3-D
autocorrelation function and the local velocity estimates are used to apply
high-pass velocity-selective spatiotemporal filters, with finite impulse
responses (FIRs), to subtract the background clutter signal, leaving the
foreground target signal, plus noise. Parallel software implementations using a
multicore central processing unit (CPU) and a graphical processing unit (GPU)
are investigated.
Random Sampling in an Age of Automation: Minimizing Expenditures through Balanced Collection and Annotation
Authors: Oscar Beijbom
Date: 2014-10-26 00:00:00

Methods for automated collection and annotation are changing the
cost-structures of sampling surveys for a wide range of applications. Digital
samples in the form of images or audio recordings can be collected rapidly, and
annotated by computer programs or crowd workers. We consider the problem of
estimating a population mean under these new cost-structures, and propose a
Hybrid-Offset sampling design. This design utilizes two annotators: a primary,
which is accurate but costly (e.g. a human expert) and an auxiliary which is
noisy but cheap (e.g. a computer program), in order to minimize total sampling
expenditures. Our analysis gives necessary conditions for the Hybrid-Offset
design and specifies optimal sample sizes for both annotators. Simulations on
data from a coral reef survey program indicate that the Hybrid-Offset design
outperforms several alternative sampling designs. In particular, sampling
expenditures are reduced 50% compared to the Conventional design currently
deployed by the coral ecologists.
A large annotated corpus for learning natural language inference
Authors: Samuel R. Bowman, Gabor Angeli, Christopher Potts, Christopher D. Manning
Date: 2015-08-21 00:00:00
Proceeding: EMNLP 2015 9
Tasks: Image Captioning, Natural Language Inference, Sentence

Understanding entailment and contradiction is fundamental to understanding
natural language, and inference about entailment and contradiction is a
valuable testing ground for the development of semantic representations.
However, machine learning research in this area has been dramatically limited
by the lack of large-scale resources. To address this, we introduce the
Stanford Natural Language Inference corpus, a new, freely available collection
of labeled sentence pairs, written by humans doing a novel grounded task based
on image captioning. At 570K pairs, it is two orders of magnitude larger than
all other resources of its type. This increase in scale allows lexicalized
classifiers to outperform some sophisticated existing entailment models, and it
allows a neural network-based model to perform competitively on natural
language inference benchmarks for the first time.
Exemplar Based Deep Discriminative and Shareable Feature Learning for Scene Image Classification
Authors: Zhen Zuo, Gang Wang, Bing Shuai, Lifan Zhao, Qingxiong Yang
Date: 2015-08-21 00:00:00
Tasks: General Classification, image-classification, Image Classification

In order to encode the class correlation and class specific information in
image representation, we propose a new local feature learning approach named
Deep Discriminative and Shareable Feature Learning (DDSFL). DDSFL aims to
hierarchically learn feature transformation filter banks to transform raw pixel
image patches to features. The learned filter banks are expected to: (1) encode
common visual patterns of a flexible number of categories; (2) encode
discriminative information; and (3) hierarchically extract patterns at
different visual levels. Particularly, in each single layer of DDSFL, shareable
filters are jointly learned for classes which share the similar patterns.
Discriminative power of the filters is achieved by enforcing the features from
the same category to be close, while features from different categories to be
far away from each other. Furthermore, we also propose two exemplar selection
methods to iteratively select training data for more efficient and effective
learning. Based on the experimental results, DDSFL can achieve very promising
performance, and it also shows great complementary effect to the
state-of-the-art Caffe features.
Robust Subspace Clustering via Thresholding
Authors: Reinhard Heckel, Helmut Bölcskei
Date: 2013-07-18 00:00:00
Tasks: Clustering

The problem of clustering noisy and incompletely observed high-dimensional
data points into a union of low-dimensional subspaces and a set of outliers is
considered. The number of subspaces, their dimensions, and their orientations
are assumed unknown. We propose a simple low-complexity subspace clustering
algorithm, which applies spectral clustering to an adjacency matrix obtained by
thresholding the correlations between data points. In other words, the
adjacency matrix is constructed from the nearest neighbors of each data point
in spherical distance. A statistical performance analysis shows that the
algorithm exhibits robustness to additive noise and succeeds even when the
subspaces intersect. Specifically, our results reveal an explicit tradeoff
between the affinity of the subspaces and the tolerable noise level. We
furthermore prove that the algorithm succeeds even when the data points are
incompletely observed with the number of missing entries allowed to be (up to a
log-factor) linear in the ambient dimension. We also propose a simple scheme
that provably detects outliers, and we present numerical results on real and
synthetic data.
Representation of Quasi-Monotone Functionals by Families of Separating Hyperplanes
Authors: Ingo Steinwart
Date: 2015-08-21 00:00:00
Tasks: BIG-bench Machine Learning

We characterize when the level sets of a continuous quasi-monotone functional
defined on a suitable convex subset of a normed space can be uniquely
represented by a family of bounded continuous functionals. Furthermore, we
investigate how regularly these functionals depend on the parameterizing level.
Finally, we show how this question relates to the recent problem of property
elicitation that simultaneously attracted interest in machine learning,
statistical evaluation of forecasts, and finance.
Customizable Contraction Hierarchies
Authors: Julian Dibbelt, Ben Strasser, Dorothea Wagner
Date: 2014-02-03 00:00:00

We consider the problem of quickly computing shortest paths in weighted
graphs given auxiliary data derived in an expensive preprocessing phase. By
adding a fast weight-customization phase, we extend Contraction Hierarchies by
Geisberger et al to support the three-phase workflow introduced by Delling et
al. Our Customizable Contraction Hierarchies use nested dissection orders as
suggested by Bauer et al. We provide an in-depth experimental analysis on large
road and game maps that clearly shows that Customizable Contraction Hierarchies
are a very practicable solution in scenarios where edge weights often change.
High-dimensional quadratic classifiers in non-sparse settings
Authors: Makoto Aoshima, Kazuyoshi Yata
Date: 2015-03-16 00:00:00
Tasks: feature selection, General Classification, Vocal Bursts Intensity Prediction

We consider high-dimensional quadratic classifiers in non-sparse settings.
The target of classification rules is not Bayes error rates in the context. The
classifier based on the Mahalanobis distance does not always give a preferable
performance even if the populations are normal distributions having known
covariance matrices. The quadratic classifiers proposed in this paper draw
information about heterogeneity effectively through both the differences of
expanding mean vectors and covariance matrices. We show that they hold a
consistency property in which misclassification rates tend to zero as the
dimension goes to infinity under non-sparse settings. We verify that they are
asymptotically distributed as a normal distribution under certain conditions.
We also propose a quadratic classifier after feature selection by using both
the differences of mean vectors and covariance matrices. Finally, we discuss
performances of the classifiers in actual data analyses. The proposed
classifiers achieve highly accurate classification with very low computational
costs.
Bit-Scalable Deep Hashing with Regularized Similarity Learning for Image Retrieval and Person Re-identification
Authors: Ruimao Zhang, Liang Lin, Rui Zhang, WangMeng Zuo, Lei Zhang
Date: 2015-08-19 00:00:00
Tasks: Deep Hashing, Image Retrieval, Person Re-Identification, Retrieval, Triplet

Extracting informative image features and learning effective approximate
hashing functions are two crucial steps in image retrieval . Conventional
methods often study these two steps separately, e.g., learning hash functions
from a predefined hand-crafted feature space. Meanwhile, the bit lengths of
output hashing codes are preset in most previous methods, neglecting the
significance level of different bits and restricting their practical
flexibility. To address these issues, we propose a supervised learning
framework to generate compact and bit-scalable hashing codes directly from raw
images. We pose hashing learning as a problem of regularized similarity
learning. Specifically, we organize the training images into a batch of triplet
samples, each sample containing two images with the same label and one with a
different label. With these triplet samples, we maximize the margin between
matched pairs and mismatched pairs in the Hamming space. In addition, a
regularization term is introduced to enforce the adjacency consistency, i.e.,
images of similar appearances should have similar codes. The deep convolutional
neural network is utilized to train the model in an end-to-end fashion, where
discriminative image features and hash functions are simultaneously optimized.
Furthermore, each bit of our hashing codes is unequally weighted so that we can
manipulate the code lengths by truncating the insignificant bits. Our framework
outperforms state-of-the-arts on public benchmarks of similar image search and
also achieves promising results in the application of person re-identification
in surveillance. It is also shown that the generated bit-scalable hashing codes
well preserve the discriminative powers with shorter code lengths.
Adaptive Online Learning
Authors: Dylan J. Foster, Alexander Rakhlin, Karthik Sridharan
Date: 2015-08-21 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Model Selection

We propose a general framework for studying adaptive regret bounds in the online learning framework, including model selection bounds and data-dependent bounds. Given a data- or model-dependent bound we ask, "Does there exist some algorithm achieving this bound?" We show that modifications to recently introduced sequential complexity measures can be used to answer this question by providing sufficient conditions under which adaptive rates can be achieved. In particular each adaptive rate induces a set of so-called offset complexity measures, and obtaining small upper bounds on these quantities is sufficient to demonstrate achievability. A cornerstone of our analysis technique is the use of one-sided tail inequalities to bound suprema of offset random processes. Our framework recovers and improves a wide variety of adaptive bounds including quantile bounds, second-order data-dependent bounds, and small loss bounds. In addition we derive a new type of adaptive bound for online linear optimization based on the spectral norm, as well as a new online PAC-Bayes theorem that holds for countably infinite sets.
On Monotonicity of the Optimal Transmission Policy in Cross-layer Adaptive m-QAM Modulation
Authors: Ni Ding, Parastoo Sadeghi, Rodney A. Kennedy
Date: 2015-08-21 00:00:00

This paper considers a cross-layer adaptive modulation system that is modeled
as a Markov decision process (MDP). We study how to utilize the monotonicity of
the optimal transmission policy to relieve the computational complexity of
dynamic programming (DP). In this system, a scheduler controls the bit rate of
the m-quadrature amplitude modulation (m-QAM) in order to minimize the
long-term losses incurred by the queue overflow in the data link layer and the
transmission power consumption in the physical layer. The work is done in two
steps. Firstly, we observe the L-natural-convexity and submodularity of DP to
prove that the optimal policy is always nondecreasing in queue occupancy/state
and derive the sufficient condition for it to be nondecreasing in both queue
and channel states. We also show that, due to the L-natural-convexity of DP,
the variation of the optimal policy in queue state is restricted by a bounded
marginal effect: The increment of the optimal policy between adjacent queue
states is no greater than one. Secondly, we use the monotonicity results to
present two low complexity algorithms: monotonic policy iteration (MPI) based
on L-natural-convexity and discrete simultaneous perturbation stochastic
approximation (DSPSA). We run experiments to show that the time complexity of
MPI based on L-natural-convexity is much lower than that of DP and the
conventional MPI that is based on submodularity and DSPSA is able to adaptively
track the optimal policy when the system parameters change.
Circle-based Eye Center Localization (CECL)
Authors: Yustinus Eko Soelistio, Eric Postma, Alfons Maes
Date: 2015-06-15 00:00:00

We propose an improved eye center localization method based on the Hough
transform, called Circle-based Eye Center Localization (CECL) that is simple,
robust, and achieves accuracy on a par with typically more complex
state-of-the-art methods. The CECL method relies on color and shape cues that
distinguish the iris from other facial structures. The accuracy of the CECL
method is demonstrated through a comparison with 15 state-of-the-art eye center
localization methods against five error thresholds, as reported in the
literature. The CECL method achieved an accuracy of 80.8% to 99.4% and ranked
first for 2 of the 5 thresholds. It is concluded that the CECL method offers an
attractive alternative to existing methods for automatic eye center
localization.
Simple Text Mining for Sentiment Analysis of Political Figure Using Naive Bayes Classifier Method
Authors: Yustinus Eko Soelistio, Martinus Raditia Sigit Surendra
Date: 2015-08-21 00:00:00
Tasks: Articles, Sentiment Analysis

Text mining can be applied to many fields. One of the application is using
text mining in digital newspaper to do politic sentiment analysis. In this
paper sentiment analysis is applied to get information from digital news
articles about its positive or negative sentiment regarding particular
politician. This paper suggests a simple model to analyze digital newspaper
sentiment polarity using naive Bayes classifier method. The model uses a set of
initial data to begin with which will be updated when new information appears.
The model showed promising result when tested and can be implemented to some
other sentiment analysis problems.
Auto-Sizing Neural Networks: With Applications to n-gram Language Models
Authors: Kenton Murray, David Chiang
Date: 2015-08-20 00:00:00
Proceeding: EMNLP 2015 9
Tasks: Decoder, Language Modeling, Language Modelling, Machine Translation, Translation

Neural networks have been shown to improve performance across a range of
natural-language tasks. However, designing and training them can be
complicated. Frequently, researchers resort to repeated experimentation to pick
optimal settings. In this paper, we address the issue of choosing the correct
number of units in hidden layers. We introduce a method for automatically
adjusting network size by pruning out hidden units through $\ell_{\infty,1}$
and $\ell_{2,1}$ regularization. We apply this method to language modeling and
demonstrate its ability to correctly choose the number of hidden units while
maintaining perplexity. We also include these models in a machine translation
decoder and show that these smaller neural models maintain the significant
improvements of their unpruned versions.
Improving Image Restoration with Soft-Rounding
Authors: Xing Mei, Honggang Qi, Bao-Gang Hu, Siwei Lyu
Date: 2015-08-20 00:00:00
Proceeding: ICCV 2015 12
Tasks: Image Restoration, SSIM

Several important classes of images such as text, barcode and pattern images
have the property that pixels can only take a distinct subset of values. This
knowledge can benefit the restoration of such images, but it has not been
widely considered in current restoration methods. In this work, we describe an
effective and efficient approach to incorporate the knowledge of distinct pixel
values of the pristine images into the general regularized least squares
restoration framework. We introduce a new regularizer that attains zero at the
designated pixel values and becomes a quadratic penalty function in the
intervals between them. When incorporated into the regularized least squares
restoration framework, this regularizer leads to a simple and efficient step
that resembles and extends the rounding operation, which we term as
soft-rounding. We apply the soft-rounding enhanced solution to the restoration
of binary text/barcode images and pattern images with multiple distinct pixel
values. Experimental results show that soft-rounding enhanced restoration
methods achieve significant improvement in both visual quality and quantitative
measures (PSNR and SSIM). Furthermore, we show that this regularizer can also
benefit the restoration of general natural images.
Adjusted least squares fitting of algebraic hypersurfaces
Authors: Konstantin Usevich, Ivan Markovsky
Date: 2014-12-06 00:00:00

We consider the problem of fitting a set of points in Euclidean space by an
algebraic hypersurface. We assume that points on a true hypersurface, described
by a polynomial equation, are corrupted by zero mean independent Gaussian
noise, and we estimate the coefficients of the true polynomial equation. The
adjusted least squares estimator accounts for the bias present in the ordinary
least squares estimator. The adjusted least squares estimator is based on
constructing a quasi-Hankel matrix, which is a bias-corrected matrix of
moments. For the case of unknown noise variance, the estimator is defined as a
solution of a polynomial eigenvalue problem. In this paper, we present new
results on invariance properties of the adjusted least squares estimator and an
improved algorithm for computing the estimator for an arbitrary set of
monomials in the polynomial equation.
Using User Generated Online Photos to Estimate and Monitor Air Pollution in Major Cities
Authors: Yuncheng Li, Jifei Huang, Jiebo Luo
Date: 2015-08-20 00:00:00

With the rapid development of economy in China over the past decade, air
pollution has become an increasingly serious problem in major cities and caused
grave public health concerns in China. Recently, a number of studies have dealt
with air quality and air pollution. Among them, some attempt to predict and
monitor the air quality from different sources of information, ranging from
deployed physical sensors to social media. These methods are either too
expensive or unreliable, prompting us to search for a novel and effective way
to sense the air quality. In this study, we propose to employ the state of the
art in computer vision techniques to analyze photos that can be easily acquired
from online social media. Next, we establish the correlation between the haze
level computed directly from photos with the official PM 2.5 record of the
taken city at the taken time. Our experiments based on both synthetic and real
photos have shown the promise of this image-based approach to estimating and
monitoring air pollution.