pwc-archive/papers-with-abstracts huggingface dataset:
Efficient Regularized Regression for Variable Selection with L0 Penalty
Authors: Zhenqiu Liu, Gang Li
Date: 2014-07-28 00:00:00
Tasks: regression, Variable Selection

Variable (feature, gene, model, which we use interchangeably) selections for
regression with high-dimensional BIGDATA have found many applications in
bioinformatics, computational biology, image processing, and engineering. One
appealing approach is the L0 regularized regression which penalizes the number
of nonzero features in the model directly. L0 is known as the most essential
sparsity measure and has nice theoretical properties, while the popular L1
regularization is only a best convex relaxation of L0. Therefore, it is natural
to expect that L0 regularized regression performs better than LASSO. However,
it is well-known that L0 optimization is NP-hard and computationally
challenging. Instead of solving the L0 problems directly, most publications so
far have tried to solve an approximation problem that closely resembles L0
regularization.
  In this paper, we propose an efficient EM algorithm (L0EM) that directly
solves the L0 optimization problem. $L_0$EM is efficient with high dimensional
data. It also provides a natural solution to all Lp p in [0,2] problems. The
regularized parameter can be either determined through cross-validation or AIC
and BIC. Theoretical properties of the L0-regularized estimator are given under
mild conditions that permit the number of variables to be much larger than the
sample size. We demonstrate our methods through simulation and high-dimensional
genomic data. The results indicate that L0 has better performance than LASSO
and L0 with AIC or BIC has similar performance as computationally intensive
cross-validation. The proposed algorithms are efficient in identifying the
non-zero variables with less-bias and selecting biologically important genes
and pathways with high dimensional BIGDATA.
A Fast Hierarchical Method for Multi-script and Arbitrary Oriented Scene Text Extraction
Authors: Lluis Gomez, Dimosthenis Karatzas
Date: 2014-07-28 00:00:00
Tasks: Clustering, Text Detection, Text Segmentation

Typography and layout lead to the hierarchical organisation of text in words,
text lines, paragraphs. This inherent structure is a key property of text in
any script and language, which has nonetheless been minimally leveraged by
existing text detection methods. This paper addresses the problem of text
segmentation in natural scenes from a hierarchical perspective. Contrary to
existing methods, we make explicit use of text structure, aiming directly to
the detection of region groupings corresponding to text within a hierarchy
produced by an agglomerative similarity clustering process over individual
regions. We propose an optimal way to construct such an hierarchy introducing a
feature space designed to produce text group hypotheses with high recall and a
novel stopping rule combining a discriminative classifier and a probabilistic
measure of group meaningfulness based in perceptual organization. Results
obtained over four standard datasets, covering text in variable orientations
and different languages, demonstrate that our algorithm, while being trained in
a single mixed dataset, outperforms state of the art methods in unconstrained
scenarios.
Automation of Mathematical Induction as part of the History of Logic
Authors: J Strother Moore, Claus-Peter Wirth
Date: 2013-09-24 00:00:00
Tasks: Mathematical Induction

We review the history of the automation of mathematical induction
Fine-grained Activity Recognition with Holistic and Pose based Features
Authors: Leonid Pishchulin, Mykhaylo Andriluka, Bernt Schiele
Date: 2014-06-07 00:00:00
Tasks: Activity Recognition

Holistic methods based on dense trajectories are currently the de facto
standard for recognition of human activities in video. Whether holistic
representations will sustain or will be superseded by higher level video
encoding in terms of body pose and motion is the subject of an ongoing debate.
In this paper we aim to clarify the underlying factors responsible for good
performance of holistic and pose-based representations. To that end we build on
our recent dataset leveraging the existing taxonomy of human activities. This
dataset includes 24,920 video snippets covering 410 human activities in total.
Our analysis reveals that holistic and pose-based methods are highly
complementary, and their performance varies significantly depending on the
activity. We find that holistic methods are mostly affected by the number and
speed of trajectories, whereas pose-based methods are mostly influenced by
viewpoint of the person. We observe striking performance differences across
activities: for certain activities results with pose-based features are more
than twice as accurate compared to holistic features, and vice versa. The best
performing approach in our comparison is based on the combination of holistic
and pose-based approaches, which again underlines their complementarity.
Sequential Click Prediction for Sponsored Search with Recurrent Neural Networks
Authors: Yuyu Zhang, Hanjun Dai, Chang Xu, Jun Feng, Taifeng Wang, Jiang Bian, Bin Wang, Tie-Yan Liu
Date: 2014-04-23 00:00:00
Tasks: Prediction

Click prediction is one of the fundamental problems in sponsored search. Most
of existing studies took advantage of machine learning approaches to predict ad
click for each event of ad view independently. However, as observed in the
real-world sponsored search system, user's behaviors on ads yield high
dependency on how the user behaved along with the past time, especially in
terms of what queries she submitted, what ads she clicked or ignored, and how
long she spent on the landing pages of clicked ads, etc. Inspired by these
observations, we introduce a novel framework based on Recurrent Neural Networks
(RNN). Compared to traditional methods, this framework directly models the
dependency on user's sequential behaviors into the click prediction process
through the recurrent structure in RNN. Large scale evaluations on the
click-through logs from a commercial search engine demonstrate that our
approach can significantly improve the click prediction accuracy, compared to
sequence-independent approaches.
'Almost Sure' Chaotic Properties of Machine Learning Methods
Authors: Nabarun Mondal, Partha P. Ghosh
Date: 2014-07-28 00:00:00
Tasks: BIG-bench Machine Learning

It has been demonstrated earlier that universal computation is 'almost
surely' chaotic. Machine learning is a form of computational fixed point
iteration, iterating over the computable function space. We showcase some
properties of this iteration, and establish in general that the iteration is
'almost surely' of chaotic nature. This theory explains the observation in the
counter intuitive properties of deep learning methods. This paper demonstrates
that these properties are going to be universal to any learning method.
kLog: A Language for Logical and Relational Learning with Kernels
Authors: Paolo Frasconi, Fabrizio Costa, Luc De Raedt, Kurt De Grave
Date: 2012-05-17 00:00:00
Tasks: General Classification, Inductive logic programming, Relational Reasoning

We introduce kLog, a novel approach to statistical relational learning.
Unlike standard approaches, kLog does not represent a probability distribution
directly. It is rather a language to perform kernel-based learning on
expressive logical and relational representations. kLog allows users to specify
learning problems declaratively. It builds on simple but powerful concepts:
learning from interpretations, entity/relationship data modeling, logic
programming, and deductive databases. Access by the kernel to the rich
representation is mediated by a technique we call graphicalization: the
relational representation is first transformed into a graph --- in particular,
a grounded entity/relationship diagram. Subsequently, a choice of graph kernel
defines the feature space. kLog supports mixed numerical and symbolic data, as
well as background knowledge in the form of Prolog or Datalog programs as in
inductive logic programming systems. The kLog framework can be applied to
tackle the same range of tasks that has made statistical relational learning so
popular, including classification, regression, multitask learning, and
collective classification. We also report about empirical comparisons, showing
that kLog can be either more accurate, or much faster at the same level of
accuracy, than Tilde and Alchemy. kLog is GPLv3 licensed and is available at
http://klog.dinfo.unifi.it along with tutorials.
A Numerical Optimization Algorithm Inspired by the Strawberry Plant
Authors: F. Merrikh-Bayat
Date: 2014-07-28 00:00:00

This paper proposes a new numerical optimization algorithm inspired by the
strawberry plant for solving complicated engineering problems. Plants like
strawberry develop both runners and roots for propagation and search for water
resources and minerals. In these plants, runners and roots can be thought of as
tools for global and local searches, respectively. The proposed algorithm has
three main differences with the trivial nature-inspired optimization
algorithms: duplication-elimination of the computational agents at all
iterations, subjecting all agents to both small and large movements from the
beginning to end, and the lack of communication (information exchange) between
agents. Moreover, it has the advantage of using only three parameters to be
tuned by user. This algorithm is applied to standard test functions and the
results are compared with GA and PSO. The proposed algorithm is also used to
solve an open problem in the field of robust control theory. These simulations
show that the proposed algorithm can very effectively solve complicated
optimization problems.
Non-parametric Image Registration of Airborne LiDAR, Hyperspectral and Photographic Imagery of Forests
Authors: Juheon Lee, Xiaohao Cai, Carola-Bibiane Schonlieb, David Coomes
Date: 2014-07-28 00:00:00
Tasks: Image Registration, Object Recognition

There is much current interest in using multi-sensor airborne remote sensing
to monitor the structure and biodiversity of forests. This paper addresses the
application of non-parametric image registration techniques to precisely align
images obtained from multimodal imaging, which is critical for the successful
identification of individual trees using object recognition approaches.
Non-parametric image registration, in particular the technique of optimizing
one objective function containing data fidelity and regularization terms,
provides flexible algorithms for image registration. Using a survey of
woodlands in southern Spain as an example, we show that non-parametric image
registration can be successful at fusing datasets when there is little prior
knowledge about how the datasets are interrelated (i.e. in the absence of
ground control points). The validity of non-parametric registration methods in
airborne remote sensing is demonstrated by a series of experiments. Precise
data fusion is a prerequisite to accurate recognition of objects within
airborne imagery, so non-parametric image registration could make a valuable
contribution to the analysis pipeline.
Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression
Authors: Tim Salimans, David A. Knowles
Date: 2012-06-28 00:00:00
Tasks: Form, regression

We propose a general algorithm for approximating nonstandard Bayesian
posterior distributions. The algorithm minimizes the Kullback-Leibler
divergence of an approximating distribution to the intractable posterior
distribution. Our method can be used to approximate any posterior distribution,
provided that it is given in closed form up to the proportionality constant.
The approximation can be any distribution in the exponential family or any
mixture of such distributions, which means that it can be made arbitrarily
precise. Several examples illustrate the speed and accuracy of our
approximation method in practice.
Classification of Human Ventricular Arrhythmia in High Dimensional Representation Spaces
Authors: Yaqub Alwan, Zoran Cvetkovic, Michael Curtis
Date: 2013-12-18 00:00:00
Tasks: Classification, General Classification, Rhythm, Vocal Bursts Intensity Prediction

We studied classification of human ECGs labelled as normal sinus rhythm,
ventricular fibrillation and ventricular tachycardia by means of support vector
machines in different representation spaces, using different observation
lengths. ECG waveform segments of duration 0.5-4 s, their Fourier magnitude
spectra, and lower dimensional projections of Fourier magnitude spectra were
used for classification. All considered representations were of much higher
dimension than in published studies. Classification accuracy improved with
segment duration up to 2 s, with 4 s providing little improvement. We found
that it is possible to discriminate between ventricular tachycardia and
ventricular fibrillation by the present approach with much shorter runs of ECG
(2 s, minimum 86% sensitivity per class) than previously imagined. Ensembles of
classifiers acting on 1 s segments taken over 5 s observation windows gave best
results, with sensitivities of detection for all classes exceeding 93%.
Beyond KernelBoost
Authors: Roberto Rigamonti, Vincent Lepetit, Pascal Fua
Date: 2014-07-28 00:00:00
Tasks: Clustering

In this Technical Report we propose a set of improvements with respect to the
KernelBoost classifier presented in [Becker et al., MICCAI 2013]. We start with
a scheme inspired by Auto-Context, but that is suitable in situations where the
lack of large training sets poses a potential problem of overfitting. The aim
is to capture the interactions between neighboring image pixels to better
regularize the boundaries of segmented regions. As in Auto-Context [Tu et al.,
PAMI 2009] the segmentation process is iterative and, at each iteration, the
segmentation results for the previous iterations are taken into account in
conjunction with the image itself. However, unlike in [Tu et al., PAMI 2009],
we organize our recursion so that the classifiers can progressively focus on
difficult-to-classify locations. This lets us exploit the power of the
decision-tree paradigm while avoiding over-fitting. In the context of this
architecture, KernelBoost represents a powerful building block due to its
ability to learn on the score maps coming from previous iterations. We first
introduce two important mechanisms to empower the KernelBoost classifier,
namely pooling and the clustering of positive samples based on the appearance
of the corresponding ground-truth. These operations significantly contribute to
increase the effectiveness of the system on biomedical images, where texture
plays a major role in the recognition of the different image components. We
then present some other techniques that can be easily integrated in the
KernelBoost framework to further improve the accuracy of the final
segmentation. We show extensive results on different medical image datasets,
including some multi-label tasks, on which our method is shown to outperform
state-of-the-art approaches. The resulting segmentations display high accuracy,
neat contours, and reduced noise.
Text Classification Using Association Rules, Dependency Pruning and Hyperonymization
Authors: Yannis Haralambous, Philippe Lenca
Date: 2014-07-28 00:00:00
Tasks: Classification, General Classification, text-classification, Text Classification

We present new methods for pruning and enhancing item- sets for text
classification via association rule mining. Pruning methods are based on
dependency syntax and enhancing methods are based on replacing words by their
hyperonyms of various orders. We discuss the impact of these methods, compared
to pruning based on tfidf rank of words.
Deep Networks with Internal Selective Attention through Feedback Connections
Authors: Marijn Stollenga, Jonathan Masci, Faustino Gomez, Juergen Schmidhuber
Date: 2014-07-11 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Deep Attention, General Classification

Traditional convolutional neural networks (CNN) are stationary and
feedforward. They neither change their parameters during evaluation nor use
feedback from higher to lower layers. Real brains, however, do. So does our
Deep Attention Selective Network (dasNet) architecture. DasNets feedback
structure can dynamically alter its convolutional filter sensitivities during
classification. It harnesses the power of sequential processing to improve
classification performance, by allowing the network to iteratively focus its
internal attention on some of its convolutional filters. Feedback is trained
through direct policy search in a huge million-dimensional parameter space,
through scalable natural evolution strategies (SNES). On the CIFAR-10 and
CIFAR-100 datasets, dasNet outperforms the previous state-of-the-art model.
Discovering Discriminative Cell Attributes for HEp-2 Specimen Image Classification
Authors: Arnold Wiliem, Peter Hobson, Brian C. Lovell
Date: 2014-07-28 00:00:00
Tasks: Attribute, Diagnostic, General Classification, image-classification, Image Classification

Recently, there has been a growing interest in developing Computer Aided
Diagnostic (CAD) systems for improving the reliability and consistency of
pathology test results. This paper describes a novel CAD system for the
Anti-Nuclear Antibody (ANA) test via Indirect Immunofluorescence protocol on
Human Epithelial Type 2 (HEp-2) cells. While prior works have primarily focused
on classifying cell images extracted from ANA specimen images, this work takes
a further step by focussing on the specimen image classification problem
itself. Our system is able to efficiently classify specimen images as well as
producing meaningful descriptions of ANA pattern class which helps physicians
to understand the differences between various ANA patterns. We achieve this
goal by designing a specimen-level image descriptor that: (1) is highly
discriminative; (2) has small descriptor length and (3) is semantically
meaningful at the cell level. In our work, a specimen image descriptor is
represented by its overall cell attribute descriptors. As such, we propose two
max-margin based learning schemes to discover cell attributes whilst still
maintaining the discrimination of the specimen image descriptor. Our learning
schemes differ from the existing discriminative attribute learning approaches
as they primarily focus on discovering image-level attributes. Comparative
evaluations were undertaken to contrast the proposed approach to various
state-of-the-art approaches on a novel HEp-2 cell dataset which was
specifically proposed for the specimen-level classification. Finally, we
showcase the ability of the proposed approach to provide textual descriptions
to explain ANA patterns.
A unified framework for thermal face recognition
Authors: Reza Shoja Ghiass, Ognjen Arandjelovic, Hakim Bendada, Xavier Maldague
Date: 2014-07-28 00:00:00
Tasks: Face Recognition

The reduction of the cost of infrared (IR) cameras in recent years has made IR imaging a highly viable modality for face recognition in practice. A particularly attractive advantage of IR-based over conventional, visible spectrum-based face recognition stems from its invariance to visible illumination. In this paper we argue that the main limitation of previous work on face recognition using IR lies in its ad hoc approach to treating different nuisance factors which affect appearance, prohibiting a unified approach that is capable of handling concurrent changes in multiple (or indeed all) major extrinsic sources of variability, which is needed in practice. We describe the first approach that attempts to achieve this - the framework we propose achieves outstanding recognition performance in the presence of variable (i) pose, (ii) facial expression, (iii) physiological state, (iv) partial occlusion due to eye-wear, and (v) quasi-occlusion due to facial hair growth.
A Poisson convolution model for characterizing topical content with word frequency and exclusivity
Authors: Edoardo M. Airoldi, Jonathan M Bischof
Date: 2012-06-18 00:00:00

An ongoing challenge in the analysis of document collections is how to
summarize content in terms of a set of inferred themes that can be interpreted
substantively in terms of topics. The current practice of parametrizing the
themes in terms of most frequent words limits interpretability by ignoring the
differential use of words across topics. We argue that words that are both
common and exclusive to a theme are more effective at characterizing topical
content. We consider a setting where professional editors have annotated
documents to a collection of topic categories, organized into a tree, in which
leaf-nodes correspond to the most specific topics. Each document is annotated
to multiple categories, at different levels of the tree. We introduce a
hierarchical Poisson convolution model to analyze annotated documents in this
setting. The model leverages the structure among categories defined by
professional editors to infer a clear semantic description for each topic in
terms of words that are both frequent and exclusive. We carry out a large
randomized experiment on Amazon Turk to demonstrate that topic summaries based
on the FREX score are more interpretable than currently established frequency
based summaries, and that the proposed model produces more efficient estimates
of exclusivity than with currently models. We also develop a parallelized
Hamiltonian Monte Carlo sampler that allows the inference to scale to millions
of documents.
Algorithms, Initializations, and Convergence for the Nonnegative Matrix Factorization
Authors: Amy N. Langville, Carl D. Meyer, Russell Albright, James Cox, David Duling
Date: 2014-07-28 00:00:00

It is well known that good initializations can improve the speed and accuracy
of the solutions of many nonnegative matrix factorization (NMF) algorithms.
Many NMF algorithms are sensitive with respect to the initialization of W or H
or both. This is especially true of algorithms of the alternating least squares
(ALS) type, including the two new ALS algorithms that we present in this paper.
We compare the results of six initialization procedures (two standard and four
new) on our ALS algorithms. Lastly, we discuss the practical issue of choosing
an appropriate convergence criterion.
Modular Belief Updates and Confusion about Measures of Certainty in Artificial Intelligence Research
Authors: Eric J. Horvitz, David Heckerman
Date: 2014-07-27 00:00:00

Over the last decade, there has been growing interest in the use or measures
or change in belief for reasoning with uncertainty in artificial intelligence
research. An important characteristic of several methodologies that reason with
changes in belief or belief updates, is a property that we term modularity. We
call updates that satisfy this property modular updates. Whereas probabilistic
measures of belief update - which satisfy the modularity property were first
discovered in the nineteenth century, knowledge and discussion of these
quantities remains obscure in artificial intelligence research. We define
modular updates and discuss their inappropriate use in two influential expert
systems.