pwc-archive/papers-with-abstracts huggingface dataset:
Large-Scale Deep Learning on the YFCC100M Dataset
Authors: Karl Ni, Roger Pearce, Kofi Boakye, Brian Van Essen, Damian Borth, Barry Chen, Eric Wang
Date: 2015-02-11 00:00:00
Tasks: Deep Learning, GPU

We present a work-in-progress snapshot of learning with a 15 billion
parameter deep learning network on HPC architectures applied to the largest
publicly available natural image and video dataset released to-date. Recent
advancements in unsupervised deep neural networks suggest that scaling up such
networks in both model and training dataset size can yield significant
improvements in the learning of concepts at the highest layers. We train our
three-layer deep neural network on the Yahoo! Flickr Creative Commons 100M
dataset. The dataset comprises approximately 99.2 million images and 800,000
user-created videos from Yahoo's Flickr image and video sharing platform.
Training of our network takes eight days on 98 GPU nodes at the High
Performance Computing Center at Lawrence Livermore National Laboratory.
Encouraging preliminary results and future research directions are presented
and discussed.
Covariance estimation using conjugate gradient for 3D classification in Cryo-EM
Authors: Joakim andén, Eugene Katsevich, Amit Singer
Date: 2014-12-02 00:00:00
Tasks: 3D Classification, General Classification

Classifying structural variability in noisy projections of biological
macromolecules is a central problem in Cryo-EM. In this work, we build on a
previous method for estimating the covariance matrix of the three-dimensional
structure present in the molecules being imaged. Our proposed method allows for
incorporation of contrast transfer function and non-uniform distribution of
viewing angles, making it more suitable for real-world data. We evaluate its
performance on a synthetic dataset and an experimental dataset obtained by
imaging a 70S ribosome complex.
Reconstruction in the Labeled Stochastic Block Model
Authors: Marc Lelarge, Laurent Massoulié, Jiaming Xu
Date: 2015-02-11 00:00:00
Tasks: model, Stochastic Block Model, Two-sample testing

The labeled stochastic block model is a random graph model representing
networks with community structure and interactions of multiple types. In its
simplest form, it consists of two communities of approximately equal size, and
the edges are drawn and labeled at random with probability depending on whether
their two endpoints belong to the same community or not.
  It has been conjectured in \cite{Heimlicher12} that correlated reconstruction
(i.e.\ identification of a partition correlated with the true partition into
the underlying communities) would be feasible if and only if a model parameter
exceeds a threshold. We prove one half of this conjecture, i.e., reconstruction
is impossible when below the threshold. In the positive direction, we introduce
a weighted graph to exploit the label information. With a suitable choice of
weight function, we show that when above the threshold by a specific constant,
reconstruction is achieved by (1) minimum bisection, (2) a semidefinite
relaxation of minimum bisection, and (3) a spectral method combined with
removal of edges incident to vertices of high degree. Furthermore, we show that
hypothesis testing between the labeled stochastic block model and the labeled
Erd\H{o}s-R\'enyi random graph model exhibits a phase transition at the
conjectured reconstruction threshold.
Boost Phrase-level Polarity Labelling with Review-level Sentiment Classification
Authors: Yongfeng Zhang, Min Zhang, Yiqun Liu, Shaoping Ma
Date: 2015-02-11 00:00:00
Tasks: Classification, General Classification, Sentiment Analysis, Sentiment Classification

Sentiment analysis on user reviews helps to keep track of user reactions
towards products, and make advices to users about what to buy. State-of-the-art
review-level sentiment classification techniques could give pretty good
precisions of above 90%. However, current phrase-level sentiment analysis
approaches might only give sentiment polarity labelling precisions of around
70%~80%, which is far from satisfaction and restricts its application in many
practical tasks. In this paper, we focus on the problem of phrase-level
sentiment polarity labelling and attempt to bridge the gap between phrase-level
and review-level sentiment analysis. We investigate the inconsistency between
the numerical star ratings and the sentiment orientation of textual user
reviews. Although they have long been treated as identical, which serves as a
basic assumption in previous work, we find that this assumption is not
necessarily true. We further propose to leverage the results of review-level
sentiment classification to boost the performance of phrase-level polarity
labelling using a novel constrained convex optimization framework. Besides, the
framework is capable of integrating various kinds of information sources and
heuristics, while giving the global optimal solution due to its convexity.
Experimental results on both English and Chinese reviews show that our
framework achieves high labelling precisions of up to 89%, which is a
significant improvement from current approaches.
Statistical laws in linguistics
Authors: Eduardo G. Altmann, Martin Gerlach
Date: 2015-02-11 00:00:00
Tasks: Text Generation

Zipf's law is just one out of many universal laws proposed to describe
statistical regularities in language. Here we review and critically discuss how
these laws can be statistically interpreted, fitted, and tested (falsified).
The modern availability of large databases of written text allows for tests
with an unprecedent statistical accuracy and also a characterization of the
fluctuations around the typical behavior. We find that fluctuations are usually
much larger than expected based on simplifying statistical assumptions (e.g.,
independence and lack of correlations between observations).These
simplifications appear also in usual statistical tests so that the large
fluctuations can be erroneously interpreted as a falsification of the law.
Instead, here we argue that linguistic laws are only meaningful (falsifiable)
if accompanied by a model for which the fluctuations can be computed (e.g., a
generative model of the text). The large fluctuations we report show that the
constraints imposed by linguistic laws on the creativity process of text
generation are not as tight as one could expect.
Off-policy evaluation for MDPs with unknown structure
Authors: Assaf Hallak, François Schnitzler, Timothy Mann, Shie Mannor
Date: 2015-02-11 00:00:00
Tasks: Off-policy evaluation

Off-policy learning in dynamic decision problems is essential for providing
strong evidence that a new policy is better than the one in use. But how can we
prove superiority without testing the new policy? To answer this question, we
introduce the G-SCOPE algorithm that evaluates a new policy based on data
generated by the existing policy. Our algorithm is both computationally and
sample efficient because it greedily learns to exploit factored structure in
the dynamics of the environment. We present a finite sample analysis of our
approach and show through experiments that the algorithm scales well on
high-dimensional problems with few samples.
A Hybrid Approach for Improved Content-based Image Retrieval using Segmentation
Authors: Smarajit Bose, Amita Pal, Jhimli Mallick, Sunil Kumar, Pratyaydipta Rudra
Date: 2015-02-11 00:00:00
Tasks: Content-Based Image Retrieval, Image Retrieval, Retrieval

The objective of Content-Based Image Retrieval (CBIR) methods is essentially
to extract, from large (image) databases, a specified number of images similar
in visual and semantic content to a so-called query image. To bridge the
semantic gap that exists between the representation of an image by low-level
features (namely, colour, shape, texture) and its high-level semantic content
as perceived by humans, CBIR systems typically make use of the relevance
feedback (RF) mechanism. RF iteratively incorporates user-given inputs
regarding the relevance of retrieved images, to improve retrieval efficiency.
One approach is to vary the weights of the features dynamically via feature
reweighting. In this work, an attempt has been made to improve retrieval
accuracy by enhancing a CBIR system based on color features alone, through
implicit incorporation of shape information obtained through prior segmentation
of the images. Novel schemes for feature reweighting as well as for
initialization of the relevant set for improved relevance feedback, have also
been proposed for boosting performance of RF- based CBIR. At the same time, new
measures for evaluation of retrieval accuracy have been suggested, to overcome
the limitations of existing measures in the RF context. Results of extensive
experiments have been presented to illustrate the effectiveness of the proposed
approaches.
Hierarchical Dirichlet Scaling Process
Authors: Dongwoo Kim, Alice Oh
Date: 2014-03-22 00:00:00
Tasks: Articles

We present the \textit{hierarchical Dirichlet scaling process} (HDSP), a
Bayesian nonparametric mixed membership model. The HDSP generalizes the
hierarchical Dirichlet process (HDP) to model the correlation structure between
metadata in the corpus and mixture components. We construct the HDSP based on
the normalized gamma representation of the Dirichlet process, and this
construction allows incorporating a scaling function that controls the
membership probabilities of the mixture components. We develop two scaling
methods to demonstrate that different modeling assumptions can be expressed in
the HDSP. We also derive the corresponding approximate posterior inference
algorithms using variational Bayes. Through experiments on datasets of
newswire, medical journal articles, conference proceedings, and product
reviews, we show that the HDSP results in a better predictive performance than
labeled LDA, partially labeled LDA, and author topic model and a better
negative review classification performance than the supervised topic model and
SVM.
A HMAX with LLC for visual recognition
Authors: Kean Hong Lau, Yong Haur Tay, Fook Loong Lo
Date: 2015-02-10 00:00:00
Tasks: GPU, Transfer Learning

Today's high performance deep artificial neural networks (ANNs) rely heavily
on parameter optimization, which is sequential in nature and even with a
powerful GPU, would have taken weeks to train them up for solving challenging
tasks [22]. HMAX [17] has demonstrated that a simple high performing network
could be obtained without heavy optimization. In this paper, we had improved on
the existing best HMAX neural network [12] in terms of structural simplicity
and performance. Our design replaces the L1 minimization sparse coding (SC)
with a locality-constrained linear coding (LLC) [20] which has a lower
computational demand. We also put the simple orientation filter bank back into
the front layer of the network replacing PCA. Our system's performance has
improved over the existing architecture and reached 79.0% on the challenging
Caltech-101 [7] dataset, which is state-of-the-art for ANNs (without transfer
learning). From our empirical data, the main contributors to our system's
performance include an introduction of partial signal whitening, a spot
detector, and a spatial pyramid matching (SPM) [14] layer.
An Aggregation Method for Sparse Logistic Regression
Authors: Zhe Liu
Date: 2014-10-25 00:00:00
Tasks: General Classification, regression

$L_1$ regularized logistic regression has now become a workhorse of data
mining and bioinformatics: it is widely used for many classification problems,
particularly ones with many features. However, $L_1$ regularization typically
selects too many features and that so-called false positives are unavoidable.
In this paper, we demonstrate and analyze an aggregation method for sparse
logistic regression in high dimensions. This approach linearly combines the
estimators from a suitable set of logistic models with different underlying
sparsity patterns and can balance the predictive ability and model
interpretability. Numerical performance of our proposed aggregation method is
then investigated using simulation studies. We also analyze a published
genome-wide case-control dataset to further evaluate the usefulness of the
aggregation method in multilocus association mapping.
Gaussian Process Models for HRTF based Sound-Source Localization and Active-Learning
Authors: Yuancheng Luo, Dmitry N. Zotkin, Ramani Duraiswami
Date: 2015-02-11 00:00:00
Tasks: Active Learning, regression, Sound Source Localization

From a machine learning perspective, the human ability localize sounds can be
modeled as a non-parametric and non-linear regression problem between binaural
spectral features of sound received at the ears (input) and their sound-source
directions (output). The input features can be summarized in terms of the
individual's head-related transfer functions (HRTFs) which measure the spectral
response between the listener's eardrum and an external point in $3$D. Based on
these viewpoints, two related problems are considered: how can one achieve an
optimal sampling of measurements for training sound-source localization (SSL)
models, and how can SSL models be used to infer the subject's HRTFs in
listening tests. First, we develop a class of binaural SSL models based on
Gaussian process regression and solve a \emph{forward selection} problem that
finds a subset of input-output samples that best generalize to all SSL
directions. Second, we use an \emph{active-learning} approach that updates an
online SSL model for inferring the subject's SSL errors via headphones and a
graphical user interface. Experiments show that only a small fraction of HRTFs
are required for $5^{\circ}$ localization accuracy and that the learned HRTFs
are localized closer to their intended directions than non-individualized
HRTFs.
Kernel Task-Driven Dictionary Learning for Hyperspectral Image Classification
Authors: Soheil Bahrampour, Nasser M. Nasrabadi, Asok Ray, Kenneth W. Jenkins
Date: 2015-02-10 00:00:00
Tasks: Classification, Dictionary Learning, General Classification, Hyperspectral Image Classification, image-classification, Image Classification

Dictionary learning algorithms have been successfully used in both
reconstructive and discriminative tasks, where the input signal is represented
by a linear combination of a few dictionary atoms. While these methods are
usually developed under $\ell_1$ sparsity constrain (prior) in the input
domain, recent studies have demonstrated the advantages of sparse
representation using structured sparsity priors in the kernel domain. In this
paper, we propose a supervised dictionary learning algorithm in the kernel
domain for hyperspectral image classification. In the proposed formulation, the
dictionary and classifier are obtained jointly for optimal classification
performance. The supervised formulation is task-driven and provides learned
features from the hyperspectral data that are well suited for the
classification task. Moreover, the proposed algorithm uses a joint
($\ell_{12}$) sparsity prior to enforce collaboration among the neighboring
pixels. The simulation results illustrate the efficiency of the proposed
dictionary learning algorithm.
Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation
Authors: Qi Wei, Nicolas Dobigeon, Jean-Yves Tourneret
Date: 2015-02-10 00:00:00
Tasks: Form

This paper proposes a fast multi-band image fusion algorithm, which combines
a high-spatial low-spectral resolution image and a low-spatial high-spectral
resolution image. The well admitted forward model is explored to form the
likelihoods of the observations. Maximizing the likelihoods leads to solving a
Sylvester equation. By exploiting the properties of the circulant and
downsampling matrices associated with the fusion problem, a closed-form
solution for the corresponding Sylvester equation is obtained explicitly,
getting rid of any iterative update step. Coupled with the alternating
direction method of multipliers and the block coordinate descent method, the
proposed algorithm can be easily generalized to incorporate prior information
for the fusion problem, allowing a Bayesian estimator. Simulation results show
that the proposed algorithm achieves the same performance as existing
algorithms with the advantage of significantly decreasing the computational
complexity of these algorithms.
Global Convergence of Stochastic Gradient Descent for Some Non-convex Matrix Problems
Authors: Christopher De Sa, Kunle Olukotun, Christopher Ré
Date: 2014-11-05 00:00:00
Tasks: Matrix Completion

Stochastic gradient descent (SGD) on a low-rank factorization is commonly
employed to speed up matrix problems including matrix completion, subspace
tracking, and SDP relaxation. In this paper, we exhibit a step size scheme for
SGD on a low-rank least-squares problem, and we prove that, under broad
sampling conditions, our method converges globally from a random starting point
within $O(\epsilon^{-1} n \log n)$ steps with constant probability for
constant-rank problems. Our modification of SGD relates it to stochastic power
iteration. We also show experiments to illustrate the runtime and convergence
of the algorithm.
Empirically Estimable Classification Bounds Based on a New Divergence Measure
Authors: Visar Berisha, Alan Wisler, Alfred O. Hero, Andreas Spanias
Date: 2014-12-19 00:00:00
Tasks: Binary Classification, Classification, feature selection, General Classification

Information divergence functions play a critical role in statistics and
information theory. In this paper we show that a non-parametric f-divergence
measure can be used to provide improved bounds on the minimum binary
classification probability of error for the case when the training and test
data are drawn from the same distribution and for the case where there exists
some mismatch between training and test distributions. We confirm the
theoretical results by designing feature selection algorithms using the
criteria from these bounds and by evaluating the algorithms on a series of
pathological speech classification tasks.
Video Primal Sketch: A Unified Middle-Level Representation for Video
Authors: Zhi Han, Zongben Xu, Song-Chun Zhu
Date: 2015-02-10 00:00:00

This paper presents a middle-level video representation named Video Primal
Sketch (VPS), which integrates two regimes of models: i) sparse coding model
using static or moving primitives to explicitly represent moving corners,
lines, feature points, etc., ii) FRAME /MRF model reproducing feature
statistics extracted from input video to implicitly represent textured motion,
such as water and fire. The feature statistics include histograms of
spatio-temporal filters and velocity distributions. This paper makes three
contributions to the literature: i) Learning a dictionary of video primitives
using parametric generative models; ii) Proposing the Spatio-Temporal FRAME
(ST-FRAME) and Motion-Appearance FRAME (MA-FRAME) models for modeling and
synthesizing textured motion; and iii) Developing a parsimonious hybrid model
for generic video representation. Given an input video, VPS selects the proper
models automatically for different motion patterns and is compatible with
high-level action representations. In the experiments, we synthesize a number
of textured motion; reconstruct real videos using the VPS; report a series of
human perception experiments to verify the quality of reconstructed videos;
demonstrate how the VPS changes over the scale transition in videos; and
present the close connection between VPS and high-level action models.
Characterization and Compensation of Network-Level Anomalies in Mixed-Signal Neuromorphic Modeling Platforms
Authors: Mihai A. Petrovici, Bernhard Vogginger, Paul Müller, Oliver Breitwieser, Mikael Lundqvist, Lyle Muller, Matthias Ehrlich, Alain Destexhe, Anders Lansner, René Schüffny, Johannes Schemmel, Karlheinz Meier
Date: 2014-04-29 00:00:00

Advancing the size and complexity of neural network models leads to an ever
increasing demand for computational resources for their simulation.
Neuromorphic devices offer a number of advantages over conventional computing
architectures, such as high emulation speed or low power consumption, but this
usually comes at the price of reduced configurability and precision. In this
article, we investigate the consequences of several such factors that are
common to neuromorphic devices, more specifically limited hardware resources,
limited parameter configurability and parameter variations. Our final aim is to
provide an array of methods for coping with such inevitable distortion
mechanisms. As a platform for testing our proposed strategies, we use an
executable system specification (ESS) of the BrainScaleS neuromorphic system,
which has been designed as a universal emulation back-end for neuroscientific
modeling. We address the most essential limitations of this device in detail
and study their effects on three prototypical benchmark network models within a
well-defined, systematic workflow. For each network model, we start by defining
quantifiable functionality measures by which we then assess the effects of
typical hardware-specific distortion mechanisms, both in idealized software
simulations and on the ESS. For those effects that cause unacceptable
deviations from the original network dynamics, we suggest generic compensation
mechanisms and demonstrate their effectiveness. Both the suggested workflow and
the investigated compensation mechanisms are largely back-end independent and
do not require additional hardware configurability beyond the one required to
emulate the benchmark networks in the first place. We hereby provide a generic
methodological environment for configurable neuromorphic devices that are
targeted at emulating large-scale, functional neural networks.
Real Time Implementation of Spatial Filtering On FPGA
Authors: Chaitannya Supe
Date: 2015-02-10 00:00:00
Tasks: Face Detection

Field Programmable Gate Array (FPGA) technology has gained vital importance
mainly because of its parallel processing hardware which makes it ideal for
image and video processing. In this paper, a step by step approach to apply a
linear spatial filter on real time video frame sent by Omnivision OV7670 camera
using Zynq Evaluation and Development board based on Xilinx XC7Z020 has been
discussed. Face detection application was chosen to explain above procedure.
This procedure is applicable to most of the complex image processing algorithms
which needs to be implemented using FPGA.
Adjusting Leverage Scores by Row Weighting: A Practical Approach to Coherent Matrix Completion
Authors: Shusen Wang, Tong Zhang, Zhihua Zhang
Date: 2014-12-26 00:00:00
Tasks: Low-Rank Matrix Completion, Matrix Completion

Low-rank matrix completion is an important problem with extensive real-world
applications. When observations are uniformly sampled from the underlying
matrix entries, existing methods all require the matrix to be incoherent. This
paper provides the first working method for coherent matrix completion under
the standard uniform sampling model. Our approach is based on the weighted
nuclear norm minimization idea proposed in several recent work, and our key
contribution is a practical method to compute the weighting matrices so that
the leverage scores become more uniform after weighting. Under suitable
conditions, we are able to derive theoretical results, showing the
effectiveness of our approach. Experiments on synthetic data show that our
approach recovers highly coherent matrices with high precision, whereas the
standard unweighted method fails even on noise-free data.
Talk to the Hand: Generating a 3D Print from Photographs
Authors: Edward Aboufadel, Sylvanna V. Krawczyk, Melissa Sherman-Bennett
Date: 2015-02-10 00:00:00

This manuscript presents a linear algebra-based technique that only requires
two unique photographs from a digital camera to mathematically construct a 3D
surface representation which can then be 3D printed. Basic computer vision
theory and manufacturing principles are also briefly discussed.