pwc-archive/papers-with-abstracts huggingface dataset:
Factorbird - a Parameter Server Approach to Distributed Matrix Factorization
Authors: Sebastian Schelter, Venu Satuluri, Reza Zadeh
Date: 2014-11-03 00:00:00

We present Factorbird, a prototype of a parameter server approach for
factorizing large matrices with Stochastic Gradient Descent-based algorithms.
We designed Factorbird to meet the following desiderata: (a) scalability to
tall and wide matrices with dozens of billions of non-zeros, (b) extensibility
to different kinds of models and loss functions as long as they can be
optimized using Stochastic Gradient Descent (SGD), and (c) adaptability to both
batch and streaming scenarios. Factorbird uses a parameter server in order to
scale to models that exceed the memory of an individual machine, and employs
lock-free Hogwild!-style learning with a special partitioning scheme to
drastically reduce conflicting updates. We also discuss other aspects of the
design of our system such as how to efficiently grid search for hyperparameters
at scale. We present experiments of Factorbird on a matrix built from a subset
of Twitter's interaction graph, consisting of more than 38 billion non-zeros
and about 200 million rows and columns, which is to the best of our knowledge
the largest matrix on which factorization results have been reported in the
literature.
Bayesian feature selection with strongly-regularizing priors maps to the Ising Model
Authors: Charles K. Fisher, Pankaj Mehta
Date: 2014-11-03 00:00:00
Tasks: Bayesian Inference, feature selection, regression

Identifying small subsets of features that are relevant for prediction and/or
classification tasks is a central problem in machine learning and statistics.
The feature selection task is especially important, and computationally
difficult, for modern datasets where the number of features can be comparable
to, or even exceed, the number of samples. Here, we show that feature selection
with Bayesian inference takes a universal form and reduces to calculating the
magnetizations of an Ising model, under some mild conditions. Our results
exploit the observation that the evidence takes a universal form for
strongly-regularizing priors --- priors that have a large effect on the
posterior probability even in the infinite data limit. We derive explicit
expressions for feature selection for generalized linear models, a large class
of statistical techniques that include linear and logistic regression. We
illustrate the power of our approach by analyzing feature selection in a
logistic regression-based classifier trained to distinguish between the letters
B and D in the notMNIST dataset.
On Detecting Noun-Adjective Agreement Errors in Bulgarian Language Using GATE
Authors: Nadezhda Borisova, Grigor Iliev, Elena Karashtranova
Date: 2014-11-03 00:00:00
Tasks: Retrieval

In this article, we describe an approach for automatic detection of
noun-adjective agreement errors in Bulgarian texts by explaining the necessary
steps required to develop a simple Java-based language processing application.
For this purpose, we use the GATE language processing framework, which is
capable of analyzing texts in Bulgarian language and can be embedded in
software applications, accessed through a set of Java APIs. In our example
application we also demonstrate how to use the functionality of GATE to perform
regular expressions over annotations for detecting agreement errors in simple
noun phrases formed by two words - attributive adjective and a noun, where the
attributive adjective precedes the noun. The provided code samples can also be
used as a starting point for implementing natural language processing
functionalities in software applications related to language processing tasks
like detection, annotation and retrieval of word groups meeting a specific set
of criteria.
Affective Facial Expression Processing via Simulation: A Probabilistic Model
Authors: Jonathan Vitale, Mary-Anne Williams, Benjamin Johnston, Giuseppe Boccignone
Date: 2014-11-03 00:00:00

Understanding the mental state of other people is an important skill for
intelligent agents and robots to operate within social environments. However,
the mental processes involved in `mind-reading' are complex. One explanation of
such processes is Simulation Theory - it is supported by a large body of
neuropsychological research. Yet, determining the best computational model or
theory to use in simulation-style emotion detection, is far from being
understood.
  In this work, we use Simulation Theory and neuroscience findings on
Mirror-Neuron Systems as the basis for a novel computational model, as a way to
handle affective facial expressions. The model is based on a probabilistic
mapping of observations from multiple identities onto a single fixed identity
(`internal transcoding of external stimuli'), and then onto a latent space
(`phenomenological response'). Together with the proposed architecture we
present some promising preliminary results
Efficient Implementations of the Generalized Lasso Dual Path Algorithm
Authors: Taylor Arnold, Ryan Tibshirani
Date: 2014-05-13 00:00:00

We consider efficient implementations of the generalized lasso dual path
algorithm of Tibshirani and Taylor (2011). We first describe a generic approach
that covers any penalty matrix D and any (full column rank) matrix X of
predictor variables. We then describe fast implementations for the special
cases of trend filtering problems, fused lasso problems, and sparse fused lasso
problems, both with X=I and a general matrix X. These specialized
implementations offer a considerable improvement over the generic
implementation, both in terms of numerical stability and efficiency of the
solution path computation. These algorithms are all available for use in the
genlasso R package, which can be found in the CRAN repository.
Personalized News Recommendation with Context Trees
Authors: Florent Garcin, Christos Dimitrakakis, Boi Faltings
Date: 2013-03-04 00:00:00
Tasks: Articles, News Recommendation, Recommendation Systems

The profusion of online news articles makes it difficult to find interesting
articles, a problem that can be assuaged by using a recommender system to bring
the most relevant news stories to readers. However, news recommendation is
challenging because the most relevant articles are often new content seen by
few users. In addition, they are subject to trends and preference changes over
time, and in many cases we do not have sufficient information to profile the
reader.
  In this paper, we introduce a class of news recommendation systems based on
context trees. They can provide high-quality news recommendation to anonymous
visitors based on present browsing behaviour. We show that context-tree
recommender systems provide good prediction accuracy and recommendation
novelty, and they are sufficiently flexible to capture the unique properties of
news articles.
Unsupervised Deep Haar Scattering on Graphs
Authors: Xu Chen, Xiuyuan Cheng, Stéphane Mallat
Date: 2014-06-09 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Classification, Dimensionality Reduction, General Classification

The classification of high-dimensional data defined on graphs is particularly
difficult when the graph geometry is unknown. We introduce a Haar scattering
transform on graphs, which computes invariant signal descriptors. It is
implemented with a deep cascade of additions, subtractions and absolute values,
which iteratively compute orthogonal Haar wavelet transforms. Multiscale
neighborhoods of unknown graphs are estimated by minimizing an average total
variation, with a pair matching algorithm of polynomial complexity. Supervised
classification with dimension reduction is tested on data bases of scrambled
images, and for signals sampled on unknown irregular grids on a sphere.
Tagging Scientific Publications using Wikipedia and Natural Language Processing Tools. Comparison on the ArXiv Dataset
Authors: Michał Łopuszyński, Łukasz Bolikowski
Date: 2013-09-02 00:00:00
Tasks: BIG-bench Machine Learning, Clustering

In this work, we compare two simple methods of tagging scientific
publications with labels reflecting their content. As a first source of labels
Wikipedia is employed, second label set is constructed from the noun phrases
occurring in the analyzed corpus. We examine the statistical properties and the
effectiveness of both approaches on the dataset consisting of abstracts from
0.7 million of scientific documents deposited in the ArXiv preprint collection.
We believe that obtained tags can be later on applied as useful document
features in various machine learning tasks (document similarity, clustering,
topic modelling, etc.).
Unsupervised Keyword Extraction from Polish Legal Texts
Authors: Michał Jungiewicz, Michał Łopuszyński
Date: 2014-08-16 00:00:00
Tasks: Keyword Extraction

In this work, we present an application of the recently proposed unsupervised
keyword extraction algorithm RAKE to a corpus of Polish legal texts from the
field of public procurement. RAKE is essentially a language and domain
independent method. Its only language-specific input is a stoplist containing a
set of non-content words. The performance of the method heavily depends on the
choice of such a stoplist, which should be domain adopted. Therefore, we
complement RAKE algorithm with an automatic approach to selecting non-content
words, which is based on the statistical properties of term distribution.
Non Binary Local Gradient Contours for Face Recognition
Authors: Abdullah Gubbi, Mohammad Fazle Azeem, M. Sharmila Kumari
Date: 2014-11-03 00:00:00
Tasks: Attribute, Binarization, Face Recognition

As the features from the traditional Local Binary Patterns (LBP) and Local Directional Patterns (LDP) are found to be ineffective for face recognition, we have proposed a new approach derived on the basis of Information sets whereby the loss of information that occurs during the binarization is eliminated. The information sets expand the scope of fuzzy sets by connecting the attribute and the corresponding membership function value as a product. Since face is having smooth texture in a limited area, the extracted features must be highly discernible. To limit the number of features, we consider only the non overlapping windows. By the application of the information set theory we can reduce the number of feature of an image. The derived features are shown to work fairly well over eigenface, fisherface and LBP methods.
Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature
Authors: Tom Gunter, Michael A. Osborne, Roman Garnett, Philipp Hennig, Stephen J. Roberts
Date: 2014-11-03 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Active Learning, Numerical Integration

We propose a novel sampling framework for inference in probabilistic models:
an active learning approach that converges more quickly (in wall-clock time)
than Markov chain Monte Carlo (MCMC) benchmarks. The central challenge in
probabilistic inference is numerical integration, to average over ensembles of
models or unknown (hyper-)parameters (for example to compute the marginal
likelihood or a partition function). MCMC has provided approaches to numerical
integration that deliver state-of-the-art inference, but can suffer from sample
inefficiency and poor convergence diagnostics. Bayesian quadrature techniques
offer a model-based solution to such problems, but their uptake has been
hindered by prohibitive computation costs. We introduce a warped model for
probabilistic integrands (likelihoods) that are known to be non-negative,
permitting a cheap active learning scheme to optimally select sample locations.
Our algorithm is demonstrated to offer faster convergence (in seconds) relative
to simple Monte Carlo and annealed importance sampling on both synthetic and
real-world examples.
Sparsity Constrained Graph Regularized NMF for Spectral Unmixing of Hyperspectral Data
Authors: Roozbeh Rajabi, Hassan Ghassemian
Date: 2014-11-03 00:00:00

Hyperspectral images contain mixed pixels due to low spatial resolution of
hyperspectral sensors. Mixed pixels are pixels containing more than one
distinct material called endmembers. The presence percentages of endmembers in
mixed pixels are called abundance fractions. Spectral unmixing problem refers
to decomposing these pixels into a set of endmembers and abundance fractions.
Due to nonnegativity constraint on abundance fractions, nonnegative matrix
factorization methods (NMF) have been widely used for solving spectral unmixing
problem. In this paper we have used graph regularized NMF (GNMF) method
combined with sparseness constraint to decompose mixed pixels in hyperspectral
imagery. This method preserves the geometrical structure of data while
representing it in low dimensional space. Adaptive regularization parameter
based on temperature schedule in simulated annealing method also has been used
in this paper for the sparseness term. Proposed algorithm is applied on
synthetic and real datasets. Synthetic data is generated based on endmembers
from USGS spectral library. AVIRIS Cuprite dataset is used as real dataset for
evaluation of proposed method. Results are quantified based on spectral angle
distance (SAD) and abundance angle distance (AAD) measures. Results in
comparison with other methods show that the proposed method can unmix data more
effectively. Specifically for the Cuprite dataset, performance of the proposed
method is approximately 10% better than the VCA and Sparse NMF in terms of root
mean square of SAD.
Variational Gaussian Process State-Space Models
Authors: Roger Frigola, Yutian Chen, Carl E. Rasmussen
Date: 2014-06-18 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Gaussian Processes, State Space Models, Time Series, Time Series Analysis, Variational Inference

State-space models have been successfully used for more than fifty years in
different areas of science and engineering. We present a procedure for
efficient variational Bayesian learning of nonlinear state-space models based
on sparse Gaussian processes. The result of learning is a tractable posterior
over nonlinear dynamical systems. In comparison to conventional parametric
models, we offer the possibility to straightforwardly trade off model capacity
and computational cost whilst avoiding overfitting. Our main algorithm uses a
hybrid inference approach combining variational Bayes and sequential Monte
Carlo. We also present stochastic variational inference and online learning
approaches for fast learning with long time series.
Adaptive Image Denoising by Targeted Databases
Authors: Enming Luo, Stanley H. Chan, Truong Q. Nguyen
Date: 2014-06-30 00:00:00
Tasks: Denoising, Image Denoising

We propose a data-dependent denoising procedure to restore noisy images.
Different from existing denoising algorithms which search for patches from
either the noisy image or a generic database, the new algorithm finds patches
from a database that contains only relevant patches. We formulate the denoising
problem as an optimal filter design problem and make two contributions. First,
we determine the basis function of the denoising filter by solving a group
sparsity minimization problem. The optimization formulation generalizes
existing denoising algorithms and offers systematic analysis of the
performance. Improvement methods are proposed to enhance the patch search
process. Second, we determine the spectral coefficients of the denoising filter
by considering a localized Bayesian prior. The localized prior leverages the
similarity of the targeted database, alleviates the intensive Bayesian
computation, and links the new method to the classical linear minimum mean
squared error estimation. We demonstrate applications of the proposed method in
a variety of scenarios, including text images, multiview images and face
images. Experimental results show the superiority of the new algorithm over
existing methods.
Iterative Hessian sketch: Fast and accurate solution approximation for constrained least-squares
Authors: Mert Pilanci, Martin J. Wainwright
Date: 2014-11-03 00:00:00

We study randomized sketching methods for approximately solving least-squares
problem with a general convex constraint. The quality of a least-squares
approximation can be assessed in different ways: either in terms of the value
of the quadratic objective function (cost approximation), or in terms of some
distance measure between the approximate minimizer and the true minimizer
(solution approximation). Focusing on the latter criterion, our first main
result provides a general lower bound on any randomized method that sketches
both the data matrix and vector in a least-squares problem; as a surprising
consequence, the most widely used least-squares sketch is sub-optimal for
solution approximation. We then present a new method known as the iterative
Hessian sketch, and show that it can be used to obtain approximations to the
original least-squares problem using a projection dimension proportional to the
statistical complexity of the least-squares minimizer, and a logarithmic number
of iterations. We illustrate our general theory with simulations for both
unconstrained and constrained versions of least-squares, including
$\ell_1$-regularization and nuclear norm constraints. We also numerically
demonstrate the practicality of our approach in a real face expression
classification experiment.
Machine learning for many-body physics: The case of the Anderson impurity model
Authors: Louis-François Arsenault, Alejandro Lopez-Bezanilla, O. Anatole von Lilienfeld, Andrew J. Millis
Date: 2014-08-05 00:00:00
Tasks: BIG-bench Machine Learning

Machine learning methods are applied to finding the Green's function of the
Anderson impurity model, a basic model system of quantum many-body
condensed-matter physics. Different methods of parametrizing the Green's
function are investigated; a representation in terms of Legendre polynomials is
found to be superior due to its limited number of coefficients and its
applicability to state of the art methods of solution. The dependence of the
errors on the size of the training set is determined. The results indicate that
a machine learning approach to dynamical mean-field theory may be feasible.
What is usual in unusual videos? Trajectory snippet histograms for discovering unusualness
Authors: Ahmet Iscen, Anil Armagan, Pinar Duygulu
Date: 2014-01-03 00:00:00

Unusual events are important as being possible indicators of undesired
consequences. Moreover, unusualness in everyday life activities may also be
amusing to watch as proven by the popularity of such videos shared in social
media. Discovery of unusual events in videos is generally attacked as a problem
of finding usual patterns, and then separating the ones that do not resemble to
those. In this study, we address the problem from the other side, and try to
answer what type of patterns are shared among unusual videos that make them
resemble to each other regardless of the ongoing event. With this challenging
problem at hand, we propose a novel descriptor to encode the rapid motions in
videos utilizing densely extracted trajectories. The proposed descriptor, which
is referred to as trajectory snipped histograms, is used to distinguish unusual
videos from usual videos, and further exploited to discover snapshots in which
unusualness happen. Experiments on domain specific people falling videos and
unrestricted funny videos show the effectiveness of our method in capturing
unusualness.
A General Framework for Mixed Graphical Models
Authors: Eunho Yang, Pradeep Ravikumar, Genevera I. Allen, Yulia Baker, Ying-Wooi Wan, Zhandong Liu
Date: 2014-11-02 00:00:00

"Mixed Data" comprising a large number of heterogeneous variables (e.g.
count, binary, continuous, skewed continuous, among other data types) are
prevalent in varied areas such as genomics and proteomics, imaging genetics,
national security, social networking, and Internet advertising. There have been
limited efforts at statistically modeling such mixed data jointly, in part
because of the lack of computationally amenable multivariate distributions that
can capture direct dependencies between such mixed variables of different
types. In this paper, we address this by introducing a novel class of Block
Directed Markov Random Fields (BDMRFs). Using the basic building block of
node-conditional univariate exponential families from Yang et al. (2012), we
introduce a class of mixed conditional random field distributions, that are
then chained according to a block-directed acyclic graph to form our class of
Block Directed Markov Random Fields (BDMRFs). The Markov independence graph
structure underlying a BDMRF thus has both directed and undirected edges. We
introduce conditions under which these distributions exist and are
normalizable, study several instances of our models, and propose scalable
penalized conditional likelihood estimators with statistical guarantees for
recovering the underlying network structure. Simulations as well as an
application to learning mixed genomic networks from next generation sequencing
expression data and mutation data demonstrate the versatility of our methods.
Noisy Matrix Completion under Sparse Factor Models
Authors: Akshay Soni, Swayambhoo Jain, Jarvis Haupt, Stefano Gonella
Date: 2014-11-02 00:00:00
Tasks: Clustering, Dictionary Learning, Matrix Completion

This paper examines a general class of noisy matrix completion tasks where
the goal is to estimate a matrix from observations obtained at a subset of its
entries, each of which is subject to random noise or corruption. Our specific
focus is on settings where the matrix to be estimated is well-approximated by a
product of two (a priori unknown) matrices, one of which is sparse. Such
structural models - referred to here as "sparse factor models" - have been
widely used, for example, in subspace clustering applications, as well as in
contemporary sparse modeling and dictionary learning tasks. Our main
theoretical contributions are estimation error bounds for sparsity-regularized
maximum likelihood estimators for problems of this form, which are applicable
to a number of different observation noise or corruption models. Several
specific implications are examined, including scenarios where observations are
corrupted by additive Gaussian noise or additive heavier-tailed (Laplace)
noise, Poisson-distributed observations, and highly-quantized (e.g., one-bit)
observations. We also propose a simple algorithmic approach based on the
alternating direction method of multipliers for these tasks, and provide
experimental evidence to support our error analyses.
Deterministic Bayesian Information Fusion and the Analysis of its Performance
Authors: Gaurav Thakur
Date: 2013-11-15 00:00:00

This paper develops a mathematical and computational framework for analyzing
the expected performance of Bayesian data fusion, or joint statistical
inference, within a sensor network. We use variational techniques to obtain the
posterior expectation as the optimal fusion rule under a deterministic
constraint and a quadratic cost, and study the smoothness and other properties
of its classification performance. For a certain class of fusion problems, we
prove that this fusion rule is also optimal in a much wider sense and satisfies
strong asymptotic convergence results. We show how these results apply to a
variety of examples with Gaussian, exponential and other statistics, and
discuss computational methods for determining the fusion system's performance
in more general, large-scale problems. These results are motivated by studying
the performance of fusing multi-modal radar and acoustic sensors for detecting
explosive substances, but have broad applicability to other Bayesian decision
problems.