pwc-archive/papers-with-abstracts huggingface dataset:
"Mental Rotation" by Optimizing Transforming Distance
Authors: Weiguang Ding, Graham W. Taylor
Date: 2014-06-11 00:00:00
Tasks: Data Augmentation

The human visual system is able to recognize objects despite transformations
that can drastically alter their appearance. To this end, much effort has been
devoted to the invariance properties of recognition systems. Invariance can be
engineered (e.g. convolutional nets), or learned from data explicitly (e.g.
temporal coherence) or implicitly (e.g. by data augmentation). One idea that
has not, to date, been explored is the integration of latent variables which
permit a search over a learned space of transformations. Motivated by evidence
that people mentally simulate transformations in space while comparing
examples, so-called "mental rotation", we propose a transforming distance.
Here, a trained relational model actively transforms pairs of examples so that
they are maximally similar in some feature space yet respect the learned
transformational constraints. We apply our method to nearest-neighbour problems
on the Toronto Face Database and NORB.
Multi-Target Shrinkage
Authors: Daniel Bartz, Johannes Höhne, Klaus-Robert Müller
Date: 2014-12-05 00:00:00

Stein showed that the multivariate sample mean is outperformed by "shrinking"
to a constant target vector. Ledoit and Wolf extended this approach to the
sample covariance matrix and proposed a multiple of the identity as shrinkage
target. In a general framework, independent of a specific estimator, we extend
the shrinkage concept by allowing simultaneous shrinkage to a set of targets.
Application scenarios include settings with (A) additional data sets from
potentially similar distributions, (B) non-stationarity, (C) a natural grouping
of the data or (D) multiple alternative estimators which could serve as
targets.
  We show that this Multi-Target Shrinkage can be translated into a quadratic
program and derive conditions under which the estimation of the shrinkage
intensities yields optimal expected squared error in the limit. For the sample
mean and the sample covariance as specific instances, we derive conditions
under which the optimality of MTS is applicable. We consider two asymptotic
settings: the large dimensional limit (LDL), where the dimensionality and the
number of observations go to infinity at the same rate, and the finite
observations large dimensional limit (FOLDL), where only the dimensionality
goes to infinity while the number of observations remains constant. We then
show the effectiveness in extensive simulations and on real world data.
CoMIC: Good features for detection and matching at object boundaries
Authors: Swarna Kamlam Ravindran, Anurag Mittal
Date: 2014-12-05 00:00:00
Tasks: Object, Point Tracking

Feature or interest points typically use information aggregation in 2D
patches which does not remain stable at object boundaries when there is object
motion against a significantly varying background. Level or iso-intensity
curves are much more stable under such conditions, especially the longer ones.
In this paper, we identify stable portions on long iso-curves and detect
corners on them. Further, the iso-curve associated with a corner is used to
discard portions from the background and improve matching. Such CoMIC (Corners
on Maximally-stable Iso-intensity Curves) points yield superior results at the
object boundary regions compared to state-of-the-art detectors while performing
comparably at the interior regions as well. This is illustrated in exhaustive
matching experiments for both boundary and non-boundary regions in applications
such as stereo and point tracking for structure from motion in video sequences.
A parallel sampling based clustering
Authors: Aditya AV Sastry, Kalyan Netti
Date: 2014-12-05 00:00:00
Tasks: Clustering

The problem of automatically clustering data is an age old problem. People
have created numerous algorithms to tackle this problem. The execution time of
any of this algorithm grows with the number of input points and the number of
cluster centers required. To reduce the number of input points we could average
the points locally and use the means or the local centers as the input for
clustering. However since the required number of local centers is very high,
running the clustering algorithm on the entire dataset to obtain these
representational points is very time consuming. To remedy this problem, in this
paper we are proposing two subclustering schemes where by we subdivide the
dataset into smaller sets and run the clustering algorithm on the smaller
datasets to obtain the required number of datapoints to run our clustering
algorithm with. As we are subdividing the given dataset, we could run
clustering algorithm on each smaller piece of the dataset in parallel. We found
that both parallel and serial execution of this method to be much faster than
the original clustering algorithm and error in running the clustering algorithm
on a reduced set to be very less.
Quantile universal threshold: model selection at the detection edge for high-dimensional linear regression
Authors: Jairo Diaz-Rodriguez, Sylvain Sardy
Date: 2014-12-05 00:00:00
Tasks: compressed sensing, Model Selection, regression

To estimate a sparse linear model from data with Gaussian noise, consilience
from lasso and compressed sensing literatures is that thresholding estimators
like lasso and the Dantzig selector have the ability in some situations to
identify with high probability part of the significant covariates
asymptotically, and are numerically tractable thanks to convexity.
  Yet, the selection of a threshold parameter $\lambda$ remains crucial in
practice. To that aim we propose Quantile Universal Thresholding, a selection
of $\lambda$ at the detection edge. We show with extensive simulations and real
data that an excellent compromise between high true positive rate and low false
discovery rate is achieved, leading also to good predictive risk.
Person Re-identification by Saliency Learning
Authors: Rui Zhao, Wanli Ouyang, Xiaogang Wang
Date: 2014-12-05 00:00:00
Tasks: Patch Matching, Person Re-Identification

Human eyes can recognize person identities based on small salient regions,
i.e. human saliency is distinctive and reliable in pedestrian matching across
disjoint camera views. However, such valuable information is often hidden when
computing similarities of pedestrian images with existing approaches. Inspired
by our user study result of human perception on human saliency, we propose a
novel perspective for person re-identification based on learning human saliency
and matching saliency distribution. The proposed saliency learning and matching
framework consists of four steps: (1) To handle misalignment caused by drastic
viewpoint change and pose variations, we apply adjacency constrained patch
matching to build dense correspondence between image pairs. (2) We propose two
alternative methods, i.e. K-Nearest Neighbors and One-class SVM, to estimate a
saliency score for each image patch, through which distinctive features stand
out without using identity labels in the training procedure. (3) saliency
matching is proposed based on patch matching. Matching patches with
inconsistent saliency brings penalty, and images of the same identity are
recognized by minimizing the saliency matching cost. (4) Furthermore, saliency
matching is tightly integrated with patch matching in a unified structural
RankSVM learning framework. The effectiveness of our approach is validated on
the VIPeR dataset and the CUHK01 dataset. Our approach outperforms the
state-of-the-art person re-identification methods on both datasets.
Deep Structured learning for mass segmentation from Mammograms
Authors: Neeraj Dhungel, Gustavo Carneiro, Andrew P. Bradley
Date: 2014-10-27 00:00:00
Tasks: Mass Segmentation From Mammograms, Segmentation

In this paper, we present a novel method for the segmentation of breast
masses from mammograms exploring structured and deep learning. Specifically,
using structured support vector machine (SSVM), we formulate a model that
combines different types of potential functions, including one that classifies
image regions using deep learning. Our main goal with this work is to show the
accuracy and efficiency improvements that these relatively new techniques can
provide for the segmentation of breast masses from mammograms. We also propose
an easily reproducible quantitative analysis to as- sess the performance of
breast mass segmentation methodologies based on widely accepted accuracy and
running time measurements on public datasets, which will facilitate further
comparisons for this segmentation problem. In particular, we use two publicly
available datasets (DDSM-BCRP and INbreast) and propose the computa- tion of
the running time taken for the methodology to produce a mass segmentation given
an input image and the use of the Dice index to quantitatively measure the
segmentation accuracy. For both databases, we show that our proposed
methodology produces competitive results in terms of accuracy and running time.
A higher homotopic extension of persistent (co)homology
Authors: Estanislao Herscovich
Date: 2014-12-05 00:00:00
Tasks: Topological Data Analysis

Our objective in this article is to show a possibly interesting structure of
homotopic nature appearing in persistent (co)homology. Assuming that the
filtration of the (say) simplicial set embedded in a finite dimensional vector
space induces a multiplicative filtration (which would not be a so harsh
hypothesis in our setting) on the dg algebra given by the complex of simplicial
cochains, we may use a result by T. Kadeishvili to get a unique (up to
noncanonical equivalence) A_infinity-algebra structure on the complete
persistent cohomology of the filtered simplicial (or topological) set. We then
provide a construction of a (pseudo)metric on the set of all (generalized)
barcodes (that is, of all cohomological degrees) enriched with the
A_infinity-algebra structure stated before, refining the usual bottleneck
metric, and which is also independent of the particular A_infinity-algebra
structure chosen (among those equivalent to each other). We think that this
distance might deserve some attention for topological data analysis, for it in
particular can recognize different linking or foldings patterns, as in the
Borromean rings. As an aside, we give a simple proof of a result relating the
barcode structure between persistent homology and cohomology. This result was
observed in a recent article by V. de Silva, D. Morozov and M.
Vejdemo-Johansson under some restricted assumptions, which we do not suppose.
Analysis of a Reduced-Communication Diffusion LMS Algorithm
Authors: Reza Arablouei, Stefan Werner, Kutluyıl Doğançay, Yih-Fang Huang
Date: 2014-08-25 00:00:00

In diffusion-based algorithms for adaptive distributed estimation, each node
of an adaptive network estimates a target parameter vector by creating an
intermediate estimate and then combining the intermediate estimates available
within its closed neighborhood. We analyze the performance of a
reduced-communication diffusion least mean-square (RC-DLMS) algorithm, which
allows each node to receive the intermediate estimates of only a subset of its
neighbors at each iteration. This algorithm eases the usage of network
communication resources and delivers a trade-off between estimation performance
and communication cost. We show analytically that the RC-DLMS algorithm is
stable and convergent in both mean and mean-square senses. We also calculate
its theoretical steady-state mean-square deviation. Simulation results
demonstrate a good match between theory and experiment.
Efficient Representations for Life-Long Learning and Autoencoding
Authors: Maria-Florina Balcan, Avrim Blum, Santosh Vempala
Date: 2014-11-06 00:00:00

It has been a long-standing goal in machine learning, as well as in AI more
generally, to develop life-long learning systems that learn many different
tasks over time, and reuse insights from tasks learned, "learning to learn" as
they do so. In this work we pose and provide efficient algorithms for several
natural theoretical formulations of this goal. Specifically, we consider the
problem of learning many different target functions over time, that share
certain commonalities that are initially unknown to the learning algorithm. Our
aim is to learn new internal representations as the algorithm learns new target
functions, that capture this commonality and allow subsequent learning tasks to
be solved more efficiently and from less data. We develop efficient algorithms
for two very different kinds of commonalities that target functions might
share: one based on learning common low-dimensional and unions of
low-dimensional subspaces and one based on learning nonlinear Boolean
combinations of features. Our algorithms for learning Boolean feature
combinations additionally have a dual interpretation, and can be viewed as
giving an efficient procedure for constructing near-optimal sparse Boolean
autoencoders under a natural "anchor-set" assumption.
Reading Text in the Wild with Convolutional Neural Networks
Authors: Max Jaderberg, Karen Simonyan, Andrea Vedaldi, Andrew Zisserman
Date: 2014-12-04 00:00:00
Tasks: Image Retrieval, Region Proposal, Retrieval, Text Generation, Text Spotting

In this work we present an end-to-end system for text spotting -- localising
and recognising text in natural scene images -- and text based image retrieval.
This system is based on a region proposal mechanism for detection and deep
convolutional neural networks for recognition. Our pipeline uses a novel
combination of complementary proposal generation techniques to ensure high
recall, and a fast subsequent filtering stage for improving precision. For the
recognition and ranking of proposals, we train very large convolutional neural
networks to perform word recognition on the whole proposal region at the same
time, departing from the character classifier based systems of the past. These
networks are trained solely on data produced by a synthetic text generation
engine, requiring no human labelled data.
  Analysing the stages of our pipeline, we show state-of-the-art performance
throughout. We perform rigorous experiments across a number of standard
end-to-end text spotting benchmarks and text-based image retrieval datasets,
showing a large improvement over all previous methods. Finally, we demonstrate
a real-world application of our text spotting system to allow thousands of
hours of news footage to be instantly searchable via a text query.
Primal-Dual Algorithms for Non-negative Matrix Factorization with the Kullback-Leibler Divergence
Authors: Felipe Yanez, Francis Bach
Date: 2014-12-04 00:00:00
Tasks: Face Recognition, Music Source Separation

Non-negative matrix factorization (NMF) approximates a given matrix as a
product of two non-negative matrices. Multiplicative algorithms deliver
reliable results, but they show slow convergence for high-dimensional data and
may be stuck away from local minima. Gradient descent methods have better
behavior, but only apply to smooth losses such as the least-squares loss. In
this article, we propose a first-order primal-dual algorithm for non-negative
decomposition problems (where one factor is fixed) with the KL divergence,
based on the Chambolle-Pock algorithm. All required computations may be
obtained in closed form and we provide an efficient heuristic way to select
step-sizes. By using alternating optimization, our algorithm readily extends to
NMF and, on synthetic examples, face recognition or music source separation
datasets, it is either faster than existing algorithms, or leads to improved
local optima, or both.
Quantifying error in estimates of human brain fiber directions using Earth Mover's Distance
Authors: Charles Zheng, Franco Pestilli, Ariel Rokem
Date: 2014-11-19 00:00:00
Tasks: ARC

Diffusion-weighted MR imaging (DWI) is the only method we currently have to
measure connections between different parts of the human brain in vivo. To
elucidate the structure of these connections, algorithms for tracking bundles
of axonal fibers through the subcortical white matter rely on local estimates
of the fiber orientation distribution function (fODF) in different parts of the
brain. These functions describe the relative abundance of populations of axonal
fibers crossing each other in each location. Multiple models exist for
estimating fODFs. The quality of the resulting estimates can be quantified by
means of a suitable measure of distance on the space of fODFs. However, there
are multiple distance metrics that can be applied for this purpose, including
smoothed $L_p$ distances and the Wasserstein metrics. Here, we give four
reasons for the use of the Earth Mover's Distance (EMD) equipped with the
arc-length, as a distance metric. (continued)
Annotating Synapses in Large EM Datasets
Authors: Stephen M. Plaza, Toufiq Parag, Gary B. Huang, Donald J. Olbris, Mathew A. Saunders, Patricia K. Rivlin
Date: 2014-09-05 00:00:00

Reconstructing neuronal circuits at the level of synapses is a central
problem in neuroscience and becoming a focus of the emerging field of
connectomics. To date, electron microscopy (EM) is the most proven technique
for identifying and quantifying synaptic connections. As advances in EM make
acquiring larger datasets possible, subsequent manual synapse identification
({\em i.e.}, proofreading) for deciphering a connectome becomes a major time
bottleneck. Here we introduce a large-scale, high-throughput, and
semi-automated methodology to efficiently identify synapses. We successfully
applied our methodology to the Drosophila medulla optic lobe, annotating many
more synapses than previous connectome efforts. Our approaches are extensible
and will make the often complicated process of synapse identification
accessible to a wider-community of potential proofreaders.
Convolutional Neural Networks at Constrained Time Cost
Authors: Kaiming He, Jian Sun
Date: 2014-12-04 00:00:00
Proceeding: CVPR 2015 6

Though recent advanced convolutional neural networks (CNNs) have been
improving the image recognition accuracy, the models are getting more complex
and time-consuming. For real-world applications in industrial and commercial
scenarios, engineers and developers are often faced with the requirement of
constrained time budget. In this paper, we investigate the accuracy of CNNs
under constrained time cost. Under this constraint, the designs of the network
architectures should exhibit as trade-offs among the factors like depth,
numbers of filters, filter sizes, etc. With a series of controlled comparisons,
we progressively modify a baseline model while preserving its time complexity.
This is also helpful for understanding the importance of the factors in network
designs. We present an architecture that achieves very competitive accuracy in
the ImageNet dataset (11.8% top-5 error, 10-view test), yet is 20% faster than
"AlexNet" (16.0% top-5 error, 10-view test).
Deep Learning for Answer Sentence Selection
Authors: Lei Yu, Karl Moritz Hermann, Phil Blunsom, Stephen Pulman
Date: 2014-12-04 00:00:00
Tasks: Deep Learning, Feature Engineering, Open-Domain Question Answering, Question Answering, Sentence

Answer sentence selection is the task of identifying sentences that contain
the answer to a given question. This is an important problem in its own right
as well as in the larger context of open domain question answering. We propose
a novel approach to solving this task via means of distributed representations,
and learn to match questions with answers by considering their semantic
encoding. This contrasts prior work on this task, which typically relies on
classifiers with large numbers of hand-crafted syntactic and semantic features
and various external resources. Our approach does not require any feature
engineering nor does it involve specialist linguistic data, making this model
easily applicable to a wide range of domains and languages. Experimental
results on a standard benchmark dataset from TREC demonstrate that---despite
its simplicity---our model matches state of the art performance on the answer
sentence selection task.
Tight convex relaxations for sparse matrix factorization
Authors: Emile Richard, Guillaume Obozinski, Jean-Philippe Vert
Date: 2014-07-19 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Clustering

Based on a new atomic norm, we propose a new convex formulation for sparse
matrix factorization problems in which the number of nonzero elements of the
factors is assumed fixed and known. The formulation counts sparse PCA with
multiple factors, subspace clustering and low-rank sparse bilinear regression
as potential applications. We compute slow rates and an upper bound on the
statistical dimension of the suggested norm for rank 1 matrices, showing that
its statistical dimension is an order of magnitude smaller than the usual
$\ell\_1$-norm, trace norm and their combinations. Even though our convex
formulation is in theory hard and does not lead to provably polynomial time
algorithmic schemes, we propose an active set algorithm leveraging the
structure of the convex problem to solve it and show promising numerical
results.
End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
Authors: Jan Chorowski, Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio
Date: 2014-12-04 00:00:00
Tasks: Decoder, speech-recognition, Speech Recognition

We replace the Hidden Markov Model (HMM) which is traditionally used in in
continuous speech recognition with a bi-directional recurrent neural network
encoder coupled to a recurrent neural network decoder that directly emits a
stream of phonemes. The alignment between the input and output sequences is
established using an attention mechanism: the decoder emits each symbol based
on a context created with a subset of input symbols elected by the attention
mechanism. We report initial results demonstrating that this new approach
achieves phoneme error rates that are comparable to the state-of-the-art
HMM-based decoders, on the TIMIT dataset.
LightLDA: Big Topic Models on Modest Compute Clusters
Authors: Jinhui Yuan, Fei Gao, Qirong Ho, Wei Dai, Jinliang Wei, Xun Zheng, Eric P. Xing, Tie-Yan Liu, Wei-Ying Ma
Date: 2014-12-04 00:00:00
Tasks: Topic Models

When building large-scale machine learning (ML) programs, such as big topic
models or deep neural nets, one usually assumes such tasks can only be
attempted with industrial-sized clusters with thousands of nodes, which are out
of reach for most practitioners or academic researchers. We consider this
challenge in the context of topic modeling on web-scale corpora, and show that
with a modest cluster of as few as 8 machines, we can train a topic model with
1 million topics and a 1-million-word vocabulary (for a total of 1 trillion
parameters), on a document collection with 200 billion tokens -- a scale not
yet reported even with thousands of machines. Our major contributions include:
1) a new, highly efficient O(1) Metropolis-Hastings sampling algorithm, whose
running cost is (surprisingly) agnostic of model size, and empirically
converges nearly an order of magnitude faster than current state-of-the-art
Gibbs samplers; 2) a structure-aware model-parallel scheme, which leverages
dependencies within the topic model, yielding a sampling strategy that is
frugal on machine memory and network communication; 3) a differential
data-structure for model storage, which uses separate data structures for high-
and low-frequency words to allow extremely large models to fit in memory, while
maintaining high inference speed; and 4) a bounded asynchronous data-parallel
scheme, which allows efficient distributed processing of massive data via a
parameter server. Our distribution strategy is an instance of the
model-and-data-parallel programming model underlying the Petuum framework for
general distributed ML, and was implemented on top of the Petuum open-source
system. We provide experimental evidence showing how this development puts
massive models within reach on a small cluster while still enjoying
proportional time cost reductions with increasing cluster size, in comparison
with alternative options.