pwc-archive/papers-with-abstracts huggingface dataset:
Solution Path Clustering with Adaptive Concave Penalty
Authors: Yuliya Marchetti, Qing Zhou
Date: 2014-04-24 00:00:00
Tasks: Clustering

Fast accumulation of large amounts of complex data has created a need for
more sophisticated statistical methodologies to discover interesting patterns
and better extract information from these data. The large scale of the data
often results in challenging high-dimensional estimation problems where only a
minority of the data shows specific grouping patterns. To address these
emerging challenges, we develop a new clustering methodology that introduces
the idea of a regularization path into unsupervised learning. A regularization
path for a clustering problem is created by varying the degree of sparsity
constraint that is imposed on the differences between objects via the minimax
concave penalty with adaptive tuning parameters. Instead of providing a single
solution represented by a cluster assignment for each object, the method
produces a short sequence of solutions that determines not only the cluster
assignment but also a corresponding number of clusters for each solution. The
optimization of the penalized loss function is carried out through an MM
algorithm with block coordinate descent. The advantages of this clustering
algorithm compared to other existing methods are as follows: it does not
require the input of the number of clusters; it is capable of simultaneously
separating irrelevant or noisy observations that show no grouping pattern,
which can greatly improve data interpretation; it is a general methodology that
can be applied to many clustering problems. We test this method on various
simulated datasets and on gene expression data, where it shows better or
competitive performance compared against several clustering methods.
Automated adaptive inference of coarse-grained dynamical models in systems biology
Authors: Bryan C. Daniels, Ilya Nemenman
Date: 2014-04-24 00:00:00

Cellular regulatory dynamics is driven by large and intricate networks of
interactions at the molecular scale, whose sheer size obfuscates understanding.
In light of limited experimental data, many parameters of such dynamics are
unknown, and thus models built on the detailed, mechanistic viewpoint overfit
and are not predictive. At the other extreme, simple ad hoc models of complex
processes often miss defining features of the underlying systems. Here we
propose an approach that instead constructs phenomenological, coarse-grained
models of network dynamics that automatically adapt their complexity to the
amount of available data. Such adaptive models lead to accurate predictions
even when microscopic details of the studied systems are unknown due to
insufficient data. The approach is computationally tractable, even for a
relatively large number of dynamical variables, allowing its software
realization, named Sir Isaac, to make successful predictions even when
important dynamic variables are unobserved. For example, it matches the known
phase space structure for simulated planetary motion data, avoids overfitting
in a complex biological signaling system, and produces accurate predictions for
a yeast glycolysis model with only tens of data points and over half of the
interacting species unobserved.
Scalable Similarity Learning using Large Margin Neighborhood Embedding
Authors: Zhaowen Wang, Jianchao Yang, Zhe Lin, Jonathan Brandt, Shiyu Chang, Thomas Huang
Date: 2014-04-24 00:00:00
Tasks: Metric Learning, Triplet

Classifying large-scale image data into object categories is an important
problem that has received increasing research attention. Given the huge amount
of data, non-parametric approaches such as nearest neighbor classifiers have
shown promising results, especially when they are underpinned by a learned
distance or similarity measurement. Although metric learning has been well
studied in the past decades, most existing algorithms are impractical to handle
large-scale data sets. In this paper, we present an image similarity learning
method that can scale well in both the number of images and the dimensionality
of image descriptors. To this end, similarity comparison is restricted to each
sample's local neighbors and a discriminative similarity measure is induced
from large margin neighborhood embedding. We also exploit the ensemble of
projections so that high-dimensional features can be processed in a set of
lower-dimensional subspaces in parallel without much performance compromise.
The similarity function is learned online using a stochastic gradient descent
algorithm in which the triplet sampling strategy is customized for quick
convergence of classification performance. The effectiveness of our proposed
model is validated on several data sets with scales varying from tens of
thousands to one million images. Recognition accuracies competitive with the
state-of-the-art performance are achieved with much higher efficiency and
scalability.
CoRE Kernels
Authors: Ping Li
Date: 2014-04-24 00:00:00
Tasks: General Classification

The term "CoRE kernel" stands for correlation-resemblance kernel. In many
applications (e.g., vision), the data are often high-dimensional, sparse, and
non-binary. We propose two types of (nonlinear) CoRE kernels for non-binary
sparse data and demonstrate the effectiveness of the new kernels through a
classification experiment. CoRE kernels are simple with no tuning parameters.
However, training nonlinear kernel SVM can be (very) costly in time and memory
and may not be suitable for truly large-scale industrial applications (e.g.
search). In order to make the proposed CoRE kernels more practical, we develop
basic probabilistic hashing algorithms which transform nonlinear kernels into
linear kernels.
An Analysis of State-Relevance Weights and Sampling Distributions on L1-Regularized Approximate Linear Programming Approximation Accuracy
Authors: Gavin Taylor, Connor Geer, David Piekut
Date: 2014-04-16 00:00:00

Recent interest in the use of $L_1$ regularization in the use of value
function approximation includes Petrik et al.'s introduction of
$L_1$-Regularized Approximate Linear Programming (RALP). RALP is unique among
$L_1$-regularized approaches in that it approximates the optimal value function
using off-policy samples. Additionally, it produces policies which outperform
those of previous methods, such as LSPI. RALP's value function approximation
quality is affected heavily by the choice of state-relevance weights in the
objective function of the linear program, and by the distribution from which
samples are drawn; however, there has been no discussion of these
considerations in the previous literature. In this paper, we discuss and
explain the effects of choices in the state-relevance weights and sampling
distribution on approximation quality, using both theoretical and experimental
illustrations. The results provide insight not only onto these effects, but
also provide intuition into the types of MDPs which are especially well suited
for approximation with RALP.
How to Construct Deep Recurrent Neural Networks
Authors: Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Yoshua Bengio
Date: 2013-12-20 00:00:00
Tasks: Language Modeling, Language Modelling

In this paper, we explore different ways to extend a recurrent neural network
(RNN) to a \textit{deep} RNN. We start by arguing that the concept of depth in
an RNN is not as clear as it is in feedforward neural networks. By carefully
analyzing and understanding the architecture of an RNN, however, we find three
points of an RNN which may be made deeper; (1) input-to-hidden function, (2)
hidden-to-hidden transition and (3) hidden-to-output function. Based on this
observation, we propose two novel architectures of a deep RNN which are
orthogonal to an earlier attempt of stacking multiple recurrent layers to build
a deep RNN (Schmidhuber, 1992; El Hihi and Bengio, 1996). We provide an
alternative interpretation of these deep RNNs using a novel framework based on
neural operators. The proposed deep RNNs are empirically evaluated on the tasks
of polyphonic music prediction and language modeling. The experimental result
supports our claim that the proposed deep RNNs benefit from the depth and
outperform the conventional, shallow RNNs.
Automatic Construction and Natural-Language Description of Nonparametric Regression Models
Authors: James Robert Lloyd, David Duvenaud, Roger Grosse, Joshua B. Tenenbaum, Zoubin Ghahramani
Date: 2014-02-18 00:00:00
Tasks: Gaussian Processes, regression, Time Series, Time Series Analysis

This paper presents the beginnings of an automatic statistician, focusing on
regression problems. Our system explores an open-ended space of statistical
models to discover a good explanation of a data set, and then produces a
detailed report with figures and natural-language text. Our approach treats
unknown regression functions nonparametrically using Gaussian processes, which
has two important consequences. First, Gaussian processes can model functions
in terms of high-level properties (e.g. smoothness, trends, periodicity,
changepoints). Taken together with the compositional structure of our language
of models this allows us to automatically describe functions in simple terms.
Second, the use of flexible nonparametric models and a rich language for
composing them in an open-ended manner also results in state-of-the-art
extrapolation performance evaluated over 13 real time series data sets from
various domains.
Unsupervised Text Extraction from G-Maps
Authors: Chandranath Adak
Date: 2014-04-24 00:00:00
Tasks: Clustering, Image Segmentation, Semantic Segmentation

This paper represents an text extraction method from Google maps, GIS
maps/images. Due to an unsupervised approach there is no requirement of any
prior knowledge or training set about the textual and non-textual parts. Fuzzy
CMeans clustering technique is used for image segmentation and Prewitt method
is used to detect the edges. Connected component analysis and gridding
technique enhance the correctness of the results. The proposed method reaches
98.5% accuracy level on the basis of experimental data sets.
Classifying pairs with trees for supervised biological network inference
Authors: Marie Schrynemackers, Louis Wehenkel, M. Madan Babu, Pierre Geurts
Date: 2014-04-24 00:00:00
Tasks: Clustering

Networks are ubiquitous in biology and computational approaches have been
largely investigated for their inference. In particular, supervised machine
learning methods can be used to complete a partially known network by
integrating various measurements. Two main supervised frameworks have been
proposed: the local approach, which trains a separate model for each network
node, and the global approach, which trains a single model over pairs of nodes.
Here, we systematically investigate, theoretically and empirically, the
exploitation of tree-based ensemble methods in the context of these two
approaches for biological network inference. We first formalize the problem of
network inference as classification of pairs, unifying in the process
homogeneous and bipartite graphs and discussing two main sampling schemes. We
then present the global and the local approaches, extending the later for the
prediction of interactions between two unseen network nodes, and discuss their
specializations to tree-based ensemble methods, highlighting their
interpretability and drawing links with clustering techniques. Extensive
computational experiments are carried out with these methods on various
biological networks that clearly highlight that these methods are competitive
with existing methods.
Rough Clustering Based Unsupervised Image Change Detection
Authors: Chandranath Adak
Date: 2014-04-24 00:00:00
Tasks: Change Detection, Clustering

This paper introduces an unsupervised technique to detect the changed region
of multitemporal images on a same reference plane with the help of rough
clustering. The proposed technique is a soft-computing approach, based on the
concept of rough set with rough clustering and Pawlak's accuracy. It is less
noisy and avoids pre-deterministic knowledge about the distribution of the
changed and unchanged regions. To show the effectiveness, the proposed
technique is compared with some other approaches.
Rényi Divergence and Kullback-Leibler Divergence
Authors: Tim van Erven, Peter Harremoës
Date: 2012-06-12 00:00:00

R\'enyi divergence is related to R\'enyi entropy much like Kullback-Leibler
divergence is related to Shannon's entropy, and comes up in many settings. It
was introduced by R\'enyi as a measure of information that satisfies almost the
same axioms as Kullback-Leibler divergence, and depends on a parameter that is
called its order. In particular, the R\'enyi divergence of order 1 equals the
Kullback-Leibler divergence.
  We review and extend the most important properties of R\'enyi divergence and
Kullback-Leibler divergence, including convexity, continuity, limits of
$\sigma$-algebras and the relation of the special order 0 to the Gaussian
dichotomy and contiguity. We also show how to generalize the Pythagorean
inequality to orders different from 1, and we extend the known equivalence
between channel capacity and minimax redundancy to continuous channel inputs
(for all orders) and present several other minimax results.
A Comparative study Between Fuzzy Clustering Algorithm and Hard Clustering Algorithm
Authors: Dibya Jyoti Bora, Dr. Anil Kumar Gupta
Date: 2014-04-24 00:00:00
Tasks: Clustering

Data clustering is an important area of data mining. This is an unsupervised
study where data of similar types are put into one cluster while data of
another types are put into different cluster. Fuzzy C means is a very important
clustering technique based on fuzzy logic. Also we have some hard clustering
techniques available like K-means among the popular ones. In this paper a
comparative study is done between Fuzzy clustering algorithm and hard
clustering algorithm
Statistical Decision Making for Optimal Budget Allocation in Crowd Labeling
Authors: Xi Chen, Qihang Lin, Dengyong Zhou
Date: 2014-03-12 00:00:00
Tasks: Decision Making

In crowd labeling, a large amount of unlabeled data instances are outsourced
to a crowd of workers. Workers will be paid for each label they provide, but
the labeling requester usually has only a limited amount of the budget. Since
data instances have different levels of labeling difficulty and workers have
different reliability, it is desirable to have an optimal policy to allocate
the budget among all instance-worker pairs such that the overall labeling
accuracy is maximized. We consider categorical labeling tasks and formulate the
budget allocation problem as a Bayesian Markov decision process (MDP), which
simultaneously conducts learning and decision making. Using the dynamic
programming (DP) recurrence, one can obtain the optimal allocation policy.
However, DP quickly becomes computationally intractable when the size of the
problem increases. To solve this challenge, we propose a computationally
efficient approximate policy, called optimistic knowledge gradient policy. Our
MDP is a quite general framework, which applies to both pull crowdsourcing
marketplaces with homogeneous workers and push marketplaces with heterogeneous
workers. It can also incorporate the contextual information of instances when
they are available. The experiments on both simulated and real data show that
the proposed policy achieves a higher labeling accuracy than other existing
policies at the same budget level.
Towards a Logic-Based Unifying Framework for Computing
Authors: Robert Kowalski, Fariba Sadri
Date: 2013-01-29 00:00:00

In this paper we propose a logic-based, framework inspired by artificial
intelligence, but scaled down for practical database and programming
applications. Computation in the framework is viewed as the task of generating
a sequence of state transitions, with the purpose of making an agent's goals
all true. States are represented by sets of atomic sentences (or facts),
representing the values of program variables, tuples in a coordination
language, facts in relational databases, or Herbrand models.
  In the model-theoretic semantics, the entire sequence of states and events
are combined into a single model-theoretic structure, by associating timestamps
with facts and events. But in the operational semantics, facts are updated
destructively, without timestamps. We show that the model generated by
destructive updates is identical to the model generated by reasoning with facts
containing timestamps. We also extend the model with intentional predicates and
composite event predicates defined by logic programs containing conditions in
first-order logic, which query the current state.
The fshape framework for the variability analysis of functional shapes
Authors: Benjamin Charlier, Nicolas Charon, Alain Trouvé
Date: 2014-04-24 00:00:00

This article introduces a full mathematical and numerical framework for
treating functional shapes (or fshapes) following the landmarks of shape spaces
and shape analysis. Functional shapes can be described as signal functions
supported on varying geometrical supports. Analysing variability of fshapes'
ensembles require the modelling and quantification of joint variations in
geometry and signal, which have been treated separately in previous approaches.
Instead, building on the ideas of shape spaces for purely geometrical objects,
we propose the extended concept of fshape bundles and define Riemannian metrics
for fshape metamorphoses to model geometrico-functional transformations within
these bundles. We also generalize previous works on data attachment terms based
on the notion of varifolds and demonstrate the utility of these distances.
Based on these, we propose variational formulations of the atlas estimation
problem on populations of fshapes and prove existence of solutions for the
different models. The second part of the article examines the numerical
implementation of the models by detailing discrete expressions for the metrics
and gradients and proposing an optimization scheme for the atlas estimation
problem. We present a few results of the methodology on a synthetic dataset as
well as on a population of retinal membranes with thickness maps.
Gradual Classical Logic for Attributed Objects
Authors: Ryuta Arisaka
Date: 2014-04-24 00:00:00
Tasks: Attribute, Relation

There is knowledge. There is belief. And there is tacit agreement.' 'We may
talk about objects. We may talk about attributes of the objects. Or we may talk
both about objects and their attributes.' This work inspects tacit agreements
on assumptions about the relation between objects and their attributes, and
studies a way of expressing them, presenting as the result what we term gradual
logic in which the sense of truth gradually shifts. It extends classical logic
instances with a new logical connective capturing the object-attribute
relation. A formal semantics is presented. Decidability is proved. Para-
consistent/epistemic/conditional/intensional/description/combined logics are
compared.
Maximum Margin Vector Correlation Filter
Authors: Vishnu Naresh Boddeti, B. V. K. Vijaya Kumar
Date: 2014-04-24 00:00:00
Tasks: object-detection, Object Detection

Correlation Filters (CFs) are a class of classifiers which are designed for
accurate pattern localization. Traditionally CFs have been used with scalar
features only, which limits their ability to be used with vector feature
representations like Gabor filter banks, SIFT, HOG, etc. In this paper we
present a new CF named Maximum Margin Vector Correlation Filter (MMVCF) which
extends the traditional CF designs to vector features. MMVCF further combines
the generalization capability of large margin based classifiers like Support
Vector Machines (SVMs) and the localization properties of CFs for better
robustness to outliers. We demonstrate the efficacy of MMVCF for object
detection and landmark localization on a variety of databases and demonstrate
that MMVCF consistently shows improved pattern localization capability in
comparison to SVMs.
Study design in causal models
Authors: Juha Karvanen
Date: 2012-11-13 00:00:00

The causal assumptions, the study design and the data are the elements
required for scientific inference in empirical research. The research is
adequately communicated only if all of these elements and their relations are
described precisely. Causal models with design describe the study design and
the missing data mechanism together with the causal structure and allow the
direct application of causal calculus in the estimation of the causal effects.
The flow of the study is visualized by ordering the nodes of the causal diagram
in two dimensions by their causal order and the time of the observation.
Conclusions whether a causal or observational relationship can be estimated
from the collected incomplete data can be made directly from the graph. Causal
models with design offer a systematic and unifying view scientific inference
and increase the clarity and speed of communication. Examples on the causal
models for a case-control study, a nested case-control study, a clinical trial
and a two-stage case-cohort study are presented.
Generating Natural Language Descriptions from OWL Ontologies: the NaturalOWL System
Authors: Ion Androutsopoulos, Gerasimos Lampouras, Dimitrios Galanis
Date: 2014-04-24 00:00:00
Tasks: Sentence, Text Generation

We present NaturalOWL, a natural language generation system that produces
texts describing individuals or classes of OWL ontologies. Unlike simpler OWL
verbalizers, which typically express a single axiom at a time in controlled,
often not entirely fluent natural language primarily for the benefit of domain
experts, we aim to generate fluent and coherent multi-sentence texts for
end-users. With a system like NaturalOWL, one can publish information in OWL on
the Web, along with automatically produced corresponding texts in multiple
languages, making the information accessible not only to computer programs and
domain experts, but also end-users. We discuss the processing stages of
NaturalOWL, the optional domain-dependent linguistic resources that the system
can use at each stage, and why they are useful. We also present trials showing
that when the domain-dependent llinguistic resources are available, NaturalOWL
produces significantly better texts compared to a simpler verbalizer, and that
the resources can be created with relatively light effort.
On Learning Where To Look
Authors: Marc'Aurelio Ranzato
Date: 2014-04-24 00:00:00

Current automatic vision systems face two major challenges: scalability and
extreme variability of appearance. First, the computational time required to
process an image typically scales linearly with the number of pixels in the
image, therefore limiting the resolution of input images to thumbnail size.
Second, variability in appearance and pose of the objects constitute a major
hurdle for robust recognition and detection. In this work, we propose a model
that makes baby steps towards addressing these challenges. We describe a
learning based method that recognizes objects through a series of glimpses.
This system performs an amount of computation that scales with the complexity
of the input rather than its number of pixels. Moreover, the proposed method is
potentially more robust to changes in appearance since its parameters are
learned in a data driven manner. Preliminary experiments on a handwritten
dataset of digits demonstrate the computational advantages of this approach.