pwc-archive/papers-with-abstracts huggingface dataset:
A Riemannian Framework for Statistical Analysis of Topological Persistence Diagrams
Authors: Rushil Anirudh, Vinay Venkataraman, Karthikeyan Natesan Ramamurthy, Pavan Turaga
Date: 2016-05-28 00:00:00
Tasks: Topological Data Analysis

Topological data analysis is becoming a popular way to study high dimensional
feature spaces without any contextual clues or assumptions. This paper concerns
itself with one popular topological feature, which is the number of
$d-$dimensional holes in the dataset, also known as the Betti$-d$ number. The
persistence of the Betti numbers over various scales is encoded into a
persistence diagram (PD), which indicates the birth and death times of these
holes as scale varies. A common way to compare PDs is by a point-to-point
matching, which is given by the $n$-Wasserstein metric. However, a big drawback
of this approach is the need to solve correspondence between points before
computing the distance; for $n$ points, the complexity grows according to
$\mathcal{O}($n$^3)$. Instead, we propose to use an entirely new framework
built on Riemannian geometry, that models PDs as 2D probability density
functions that are represented in the square-root framework on a Hilbert
Sphere. The resulting space is much more intuitive with closed form expressions
for common operations. The distance metric is 1) correspondence-free and also
2) independent of the number of points in the dataset. The complexity of
computing distance between PDs now grows according to $\mathcal{O}(K^2)$, for a
$K \times K$ discretization of $[0,1]^2$. This also enables the use of existing
machinery in differential geometry towards statistical analysis of PDs such as
computing the mean, geodesics, classification etc. We report competitive
results with the Wasserstein metric, at a much lower computational load,
indicating the favorable properties of the proposed approach.
Sparse Coding and Counting for Robust Visual Tracking
Authors: Risheng Liu, Jing Wang, Yiyang Wang, Zhixun Su, Yu Cai
Date: 2016-05-28 00:00:00
Tasks: Visual Tracking

In this paper, we propose a novel sparse coding and counting method under
Bayesian framwork for visual tracking. In contrast to existing methods, the
proposed method employs the combination of L0 and L1 norm to regularize the
linear coefficients of incrementally updated linear basis. The sparsity
constraint enables the tracker to effectively handle difficult challenges, such
as occlusion or image corruption. To achieve realtime processing, we propose a
fast and efficient numerical algorithm for solving the proposed model. Although
it is an NP-hard problem, the proposed accelerated proximal gradient (APG)
approach is guaranteed to converge to a solution quickly. Besides, we provide a
closed solution of combining L0 and L1 regularized representation to obtain
better sparsity. Experimental results on challenging video sequences
demonstrate that the proposed method achieves state-of-the-art results both in
accuracy and speed.
Computational Estimate Visualisation and Evaluation of Agent Classified Rules Learning System
Authors: Kennedy E. Ehimwenma, Martin Beer, Paul Crowther
Date: 2016-05-28 00:00:00

Student modelling and agent classified rules learning as applied in the
development of the intelligent Preassessment System has been presented in
[10],[11]. In this paper, we now demystify the theory behind the development of
the pre-assessment system followed by some computational experimentation and
graph visualisation of the agent classified rules learning algorithm in the
estimation and prediction of classified rules. In addition, we present some
preliminary results of the pre-assessment system evaluation. From the results,
it is gathered that the system has performed according to its design
specification.
Online Bayesian Collaborative Topic Regression
Authors: Chenghao Liu, Tao Jin, Steven C. H. Hoi, Peilin Zhao, Jianling Sun
Date: 2016-05-28 00:00:00
Tasks: Recommendation Systems, regression

Collaborative Topic Regression (CTR) combines ideas of probabilistic matrix
factorization (PMF) and topic modeling (e.g., LDA) for recommender systems,
which has gained increasing successes in many applications. Despite enjoying
many advantages, the existing CTR algorithms have some critical limitations.
First of all, they are often designed to work in a batch learning manner,
making them unsuitable to deal with streaming data or big data in real-world
recommender systems. Second, the document-specific topic proportions of LDA are
fed to the downstream PMF, but not reverse, which is sub-optimal as the rating
information is not exploited in discovering the low-dimensional representation
of documents and thus can result in a sub-optimal representation for
prediction. In this paper, we propose a novel scheme of Online Bayesian
Collaborative Topic Regression (OBCTR) which is efficient and scalable for
learning from data streams. Particularly, we {\it jointly} optimize the
combined objective function of both PMF and LDA in an online learning fashion,
in which both PMF and LDA tasks can be reinforced each other during the online
learning process. Our encouraging experimental results on real-world data
validate the effectiveness of the proposed method.
Video Key Frame Extraction using Entropy value as Global and Local Feature
Authors: Siddu P Algur, Vivek R
Date: 2016-05-28 00:00:00

Key frames play an important role in video annotation. It is one of the
widely used methods for video abstraction as this will help us for processing a
large set of video data with sufficient content representation in faster way.
In this paper a novel approach for key-frame extraction using entropy value is
proposed. The proposed approach classifies frames based on entropy values as
global feature and selects frame from each class as representative key-frame.
It also eliminates redundant frames from selected key-frames using entropy
value as local feature. Evaluation of the approach on several video clips has
been presented. Results show that the algorithm is successful in helping
annotators automatically identify video key-frames.
A Channelized Binning Method for Extraction of Dominant Color Pixel Value
Authors: Siddu P Algur, N H Ayachit, Vivek R
Date: 2016-05-28 00:00:00
Tasks: Quantization

The Color is one of the most important and easily identifiable features for
describing the visual content. The MPEG standard has developed a number of
descriptors that covers different aspects of the visual content. The Dominant
color descriptor is one of them. This paper proposes a channelized binning
approach a novel method for extraction of the dominant color pixel value which
is a variant of the dominant color descriptor. The Channelized binning method
treats the problem as a statistical problem and tries to avoid color
quantization and interpolation guessing of number and centroid of dominant
colors. Channelized binning is an iterative approach which automatically
estimates the number of dominant pixel values and their centroids. It operates
on 24 bit full RGB color space, by considering one color channel at a time and
hence avoiding the color quantization. Results show that the proposed method
can successfully extract dominant color pixel values.
Gauss quadrature for matrix inverse forms with applications
Authors: Chengtao Li, Suvrit Sra, Stefanie Jegelka
Date: 2015-12-07 00:00:00
Tasks: BIG-bench Machine Learning, Point Processes

We present a framework for accelerating a spectrum of machine learning
algorithms that require computation of bilinear inverse forms $u^\top A^{-1}u$,
where $A$ is a positive definite matrix and $u$ a given vector. Our framework
is built on Gauss-type quadrature and easily scales to large, sparse matrices.
Further, it allows retrospective computation of lower and upper bounds on
$u^\top A^{-1}u$, which in turn accelerates several algorithms. We prove that
these bounds tighten iteratively and converge at a linear (geometric) rate. To
our knowledge, ours is the first work to demonstrate these key properties of
Gauss-type quadrature, which is a classical and deeply studied topic. We
illustrate empirical consequences of our results by using quadrature to
accelerate machine learning tasks involving determinantal point processes and
submodular optimization, and observe tremendous speedups in several instances.
Fast DPP Sampling for Nyström with Application to Kernel Methods
Authors: Chengtao Li, Stefanie Jegelka, Suvrit Sra
Date: 2016-03-19 00:00:00
Tasks: Point Processes, regression

The Nystr\"om method has long been popular for scaling up kernel methods. Its
theoretical guarantees and empirical performance rely critically on the quality
of the landmarks selected. We study landmark selection for Nystr\"om using
Determinantal Point Processes (DPPs), discrete probability models that allow
tractable generation of diverse samples. We prove that landmarks selected via
DPPs guarantee bounds on approximation errors; subsequently, we analyze
implications for kernel ridge regression. Contrary to prior reservations due to
cubic complexity of DPPsampling, we show that (under certain conditions) Markov
chain DPP sampling requires only linear time in the size of the data. We
present several empirical results that support our theoretical analysis, and
demonstrate the superior performance of DPP-based landmark selection compared
with existing approaches.
Learning to Generate with Memory
Authors: Chongxuan Li, Jun Zhu, Bo Zhang
Date: 2016-02-24 00:00:00
Tasks: Density Estimation, Image Generation, Imputation, Representation Learning

Memory units have been widely used to enrich the capabilities of deep
networks on capturing long-term dependencies in reasoning and prediction tasks,
but little investigation exists on deep generative models (DGMs) which are good
at inferring high-level invariant representations from unlabeled data. This
paper presents a deep generative model with a possibly large external memory
and an attention mechanism to capture the local detail information that is
often lost in the bottom-up abstraction process in representation learning. By
adopting a smooth attention model, the whole network is trained end-to-end by
optimizing a variational bound of data likelihood via auto-encoding variational
Bayesian methods, where an asymmetric recognition network is learnt jointly to
infer high-level invariant representations. The asymmetric architecture can
reduce the competition between bottom-up invariant feature extraction and
top-down generation of instance details. Our experiments on several datasets
demonstrate that memory can significantly boost the performance of DGMs and
even achieve state-of-the-art results on various tasks, including density
estimation, image generation, and missing value imputation.
Muffled Semi-Supervised Learning
Authors: Akshay Balsubramani, Yoav Freund
Date: 2016-05-28 00:00:00
Tasks: regression

We explore a novel approach to semi-supervised learning. This approach is
contrary to the common approach in that the unlabeled examples serve to
"muffle," rather than enhance, the guidance provided by the labeled examples.
We provide several variants of the basic algorithm and show experimentally that
they can achieve significantly higher AUC than boosted trees, random forests
and logistic regression when unlabeled examples are available.
Stagewise Learning for Sparse Clustering of Discretely-Valued Data
Authors: Vincent Zhao, Steven W. Zucker
Date: 2015-06-09 00:00:00
Tasks: Clustering

The performance of EM in learning mixtures of product distributions often
depends on the initialization. This can be problematic in crowdsourcing and
other applications, e.g. when a small number of 'experts' are diluted by a
large number of noisy, unreliable participants. We develop a new EM algorithm
that is driven by these experts. In a manner that differs from other
approaches, we start from a single mixture class. The algorithm then develops
the set of 'experts' in a stagewise fashion based on a mutual information
criterion. At each stage EM operates on this subset of the players, effectively
regularizing the E rather than the M step. Experiments show that stagewise EM
outperforms other initialization techniques for crowdsourcing and neurosciences
applications, and can guide a full EM to results comparable to those obtained
knowing the exact distribution.
Weighted Residuals for Very Deep Networks
Authors: Falong Shen, Gang Zeng
Date: 2016-05-28 00:00:00
Tasks: GPU

Deep residual networks have recently shown appealing performance on many
challenging computer vision tasks. However, the original residual structure
still has some defects making it difficult to converge on very deep networks.
In this paper, we introduce a weighted residual network to address the
incompatibility between \texttt{ReLU} and element-wise addition and the deep
network initialization problem. The weighted residual network is able to learn
to combine residuals from different layers effectively and efficiently. The
proposed models enjoy a consistent improvement over accuracy and convergence
with increasing depths from 100+ layers to 1000+ layers. Besides, the weighted
residual networks have little more computation and GPU memory burden than the
original residual networks. The networks are optimized by projected stochastic
gradient descent. Experiments on CIFAR-10 have shown that our algorithm has a
\emph{faster convergence speed} than the original residual networks and reaches
a \emph{high accuracy} at 95.3\% with a 1192-layer model.
Efficient Sampling for k-Determinantal Point Processes
Authors: Chengtao Li, Stefanie Jegelka, Suvrit Sra
Date: 2015-09-04 00:00:00
Tasks: Diversity, Point Processes

Determinantal Point Processes (DPPs) are elegant probabilistic models of
repulsion and diversity over discrete sets of items. But their applicability to
large sets is hindered by expensive cubic-complexity matrix operations for
basic tasks such as sampling. In light of this, we propose a new method for
approximate sampling from discrete $k$-DPPs. Our method takes advantage of the
diversity property of subsets sampled from a DPP, and proceeds in two stages:
first it constructs coresets for the ground set of items; thereafter, it
efficiently samples subsets based on the constructed coresets. As opposed to
previous approaches, our algorithm aims to minimize the total variation
distance to the original distribution. Experiments on both synthetic and real
datasets indicate that our sampling algorithm works efficiently on large data
sets, and yields more accurate samples than previous approaches.
Benefits of depth in neural networks
Authors: Matus Telgarsky
Date: 2016-02-14 00:00:00

For any positive integer $k$, there exist neural networks with $\Theta(k^3)$
layers, $\Theta(1)$ nodes per layer, and $\Theta(1)$ distinct parameters which
can not be approximated by networks with $\mathcal{O}(k)$ layers unless they
are exponentially large --- they must possess $\Omega(2^k)$ nodes. This result
is proved here for a class of nodes termed "semi-algebraic gates" which
includes the common choices of ReLU, maximum, indicator, and piecewise
polynomial functions, therefore establishing benefits of depth against not just
standard networks with ReLU gates, but also convolutional networks with ReLU
and maximization gates, sum-product networks, and boosted decision trees (in
this last case with a stronger separation: $\Omega(2^{k^3})$ total tree nodes
are required).
Regret Analysis of the Finite-Horizon Gittins Index Strategy for Multi-Armed Bandits
Authors: Tor Lattimore
Date: 2015-11-18 00:00:00
Tasks: Multi-Armed Bandits, Thompson Sampling

I analyse the frequentist regret of the famous Gittins index strategy for
multi-armed bandits with Gaussian noise and a finite horizon. Remarkably it
turns out that this approach leads to finite-time regret guarantees comparable
to those available for the popular UCB algorithm. Along the way I derive
finite-time bounds on the Gittins index that are asymptotically exact and may
be of independent interest. I also discuss some computational issues and
present experimental results suggesting that a particular version of the
Gittins index strategy is a modest improvement on existing algorithms with
finite-time regret guarantees such as UCB and Thompson sampling.
Synthesized Classifiers for Zero-Shot Learning
Authors: Soravit Changpinyo, Wei-Lun Chao, Boqing Gong, Fei Sha
Date: 2016-03-02 00:00:00
Proceeding: CVPR 2016 6
Tasks: Object, Zero-Shot Learning

Given semantic descriptions of object classes, zero-shot learning aims to
accurately recognize objects of the unseen classes, from which no examples are
available at the training stage, by associating them to the seen classes, from
which labeled examples are provided. We propose to tackle this problem from the
perspective of manifold learning. Our main idea is to align the semantic space
that is derived from external information to the model space that concerns
itself with recognizing visual features. To this end, we introduce a set of
"phantom" object classes whose coordinates live in both the semantic space and
the model space. Serving as bases in a dictionary, they can be optimized from
labeled data such that the synthesized real object classifiers achieve optimal
discriminative performance. We demonstrate superior accuracy of our approach
over the state of the art on four benchmark datasets for zero-shot learning,
including the full ImageNet Fall 2011 dataset with more than 20,000 unseen
classes.
SPF-CellTracker: Tracking multiple cells with strongly-correlated moves using a spatial particle filter
Authors: Osamu Hirose, Shotaro Kawaguchi, Terumasa Tokunaga, Yu Toyoshima, Takayuki Teramoto, Sayuri Kuge, Takeshi Ishihara, Yuichi Iino, Ryo Yoshida
Date: 2015-08-26 00:00:00
Tasks: Cell Tracking

Tracking many cells in time-lapse 3D image sequences is an important
challenging task of bioimage informatics. Motivated by a study of brain-wide 4D
imaging of neural activity in C. elegans, we present a new method of multi-cell
tracking. Data types to which the method is applicable are characterized as
follows: (i) cells are imaged as globular-like objects, (ii) it is difficult to
distinguish cells based only on shape and size, (iii) the number of imaged
cells ranges in several hundreds, (iv) moves of nearly-located cells are
strongly correlated and (v) cells do not divide. We developed a tracking
software suite which we call SPF-CellTracker. Incorporating dependency on
cells' moves into prediction model is the key to reduce the tracking errors:
cell-switching and coalescence of tracked positions. We model target cells'
correlated moves as a Markov random field and we also derive a fast computation
algorithm, which we call spatial particle filter. With the live-imaging data of
nuclei of C. elegans neurons in which approximately 120 nuclei of neurons are
imaged, we demonstrate an advantage of the proposed method over the standard
particle filter and a method developed by Tokunaga et al. (2014).
Stacking With Auxiliary Features
Authors: Nazneen Fatema Rajani, Raymond J. Mooney
Date: 2016-05-27 00:00:00
Tasks: object-detection, Object Detection, slot-filling, Slot Filling

Ensembling methods are well known for improving prediction accuracy. However,
they are limited in the sense that they cannot discriminate among component
models effectively. In this paper, we propose stacking with auxiliary features
that learns to fuse relevant information from multiple systems to improve
performance. Auxiliary features enable the stacker to rely on systems that not
just agree on an output but also the provenance of the output. We demonstrate
our approach on three very different and difficult problems -- the Cold Start
Slot Filling, the Tri-lingual Entity Discovery and Linking and the ImageNet
object detection tasks. We obtain new state-of-the-art results on the first two
tasks and substantial improvements on the detection task, thus verifying the
power and generality of our approach.
Benchmarking Deep Reinforcement Learning for Continuous Control
Authors: Yan Duan, Xi Chen, Rein Houthooft, John Schulman, Pieter Abbeel
Date: 2016-04-22 00:00:00
Tasks: Action Triplet Recognition, Atari Games, Benchmarking, continuous-control, Continuous Control, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recently, researchers have made significant progress combining the advances
in deep learning for learning feature representations with reinforcement
learning. Some notable examples include training agents to play Atari games
based on raw pixel data and to acquire advanced manipulation skills using raw
sensory inputs. However, it has been difficult to quantify progress in the
domain of continuous control due to the lack of a commonly adopted benchmark.
In this work, we present a benchmark suite of continuous control tasks,
including classic tasks like cart-pole swing-up, tasks with very high state and
action dimensionality such as 3D humanoid locomotion, tasks with partial
observations, and tasks with hierarchical structure. We report novel findings
based on the systematic evaluation of a range of implemented reinforcement
learning algorithms. Both the benchmark and reference implementations are
released at https://github.com/rllab/rllab in order to facilitate experimental
reproducibility and to encourage adoption by other researchers.
Improved SVRG for Non-Strongly-Convex or Sum-of-Non-Convex Objectives
Authors: Zeyuan Allen-Zhu, Yang Yuan
Date: 2015-06-05 00:00:00
Tasks: regression

Many classical algorithms are found until several years later to outlive the
confines in which they were conceived, and continue to be relevant in
unforeseen settings. In this paper, we show that SVRG is one such method: being
originally designed for strongly convex objectives, it is also very robust in
non-strongly convex or sum-of-non-convex settings.
  More precisely, we provide new analysis to improve the state-of-the-art
running times in both settings by either applying SVRG or its novel variant.
Since non-strongly convex objectives include important examples such as Lasso
or logistic regression, and sum-of-non-convex objectives include famous
examples such as stochastic PCA and is even believed to be related to training
deep neural nets, our results also imply better performances in these
applications.
Even Faster Accelerated Coordinate Descent Using Non-Uniform Sampling
Authors: Zeyuan Allen-Zhu, Zheng Qu, Peter Richtárik, Yang Yuan
Date: 2015-12-30 00:00:00

Accelerated coordinate descent is widely used in optimization due to its
cheap per-iteration cost and scalability to large-scale problems. Up to a
primal-dual transformation, it is also the same as accelerated stochastic
gradient descent that is one of the central methods used in machine learning.
  In this paper, we improve the best known running time of accelerated
coordinate descent by a factor up to $\sqrt{n}$. Our improvement is based on a
clean, novel non-uniform sampling that selects each coordinate with a
probability proportional to the square root of its smoothness parameter. Our
proof technique also deviates from the classical estimation sequence technique
used in prior work. Our speed-up applies to important problems such as
empirical risk minimization and solving linear systems, both in theory and in
practice.