pwc-archive/papers-with-abstracts huggingface dataset:
The Law of Total Odds
Authors: Dirk Tasche
Date: 2013-12-02 00:00:00
Tasks: Binary Classification, General Classification

The law of total probability may be deployed in binary classification
exercises to estimate the unconditional class probabilities if the class
proportions in the training set are not representative of the population class
proportions. We argue that this is not a conceptually sound approach and
suggest an alternative based on the new law of total odds. We quantify the bias
of the total probability estimator of the unconditional class probabilities and
show that the total odds estimator is unbiased. The sample version of the total
odds estimator is shown to coincide with a maximum-likelihood estimator known
from the literature. The law of total odds can also be used for transforming
the conditional class probabilities if independent estimates of the
unconditional class probabilities of the population are available.
  Keywords: Total probability, likelihood ratio, Bayes' formula, binary
classification, relative odds, unbiased estimator, supervised learning, dataset
shift.
On the number of response regions of deep feed forward networks with piece-wise linear activations
Authors: Razvan Pascanu, Guido Montufar, Yoshua Bengio
Date: 2013-12-20 00:00:00

This paper explores the complexity of deep feedforward networks with linear
pre-synaptic couplings and rectified linear activations. This is a contribution
to the growing body of work contrasting the representational power of deep and
shallow network architectures. In particular, we offer a framework for
comparing deep and shallow models that belong to the family of piecewise linear
functions based on computational geometry. We look at a deep rectifier
multi-layer perceptron (MLP) with linear outputs units and compare it with a
single layer version of the model. In the asymptotic regime, when the number of
inputs stays constant, if the shallow model has $kn$ hidden units and $n_0$
inputs, then the number of linear regions is $O(k^{n_0}n^{n_0})$. For a $k$
layer model with $n$ hidden units on each layer it is $\Omega(\left\lfloor
{n}/{n_0}\right\rfloor^{k-1}n^{n_0})$. The number
$\left\lfloor{n}/{n_0}\right\rfloor^{k-1}$ grows faster than $k^{n_0}$ when $n$
tends to infinity or when $k$ tends to infinity and $n \geq 2n_0$.
Additionally, even when $k$ is small, if we restrict $n$ to be $2n_0$, we can
show that a deep model has considerably more linear regions that a shallow one.
We consider this as a first step towards understanding the complexity of these
models and specifically towards providing suitable mathematical tools for
future analysis.
A Clockwork RNN
Authors: Jan Koutník, Klaus Greff, Faustino Gomez, Jürgen Schmidhuber
Date: 2014-02-14 00:00:00
Tasks: General Classification

Sequence prediction and classification are ubiquitous and challenging
problems in machine learning that can require identifying complex dependencies
between temporally distant inputs. Recurrent Neural Networks (RNNs) have the
ability, in theory, to cope with these temporal dependencies by virtue of the
short-term memory implemented by their recurrent (feedback) connections.
However, in practice they are difficult to train successfully when the
long-term memory is required. This paper introduces a simple, yet powerful
modification to the standard RNN architecture, the Clockwork RNN (CW-RNN), in
which the hidden layer is partitioned into separate modules, each processing
inputs at its own temporal granularity, making computations only at its
prescribed clock rate. Rather than making the standard RNN models more complex,
CW-RNN reduces the number of RNN parameters, improves the performance
significantly in the tasks tested, and speeds up the network evaluation. The
network is demonstrated in preliminary experiments involving two tasks: audio
signal generation and TIMIT spoken word classification, where it outperforms
both RNN and LSTM networks.
Generative Modelling for Unsupervised Score Calibration
Authors: Niko Brümmer, Daniel Garcia-Romero
Date: 2013-11-04 00:00:00

Score calibration enables automatic speaker recognizers to make
cost-effective accept / reject decisions. Traditional calibration requires
supervised data, which is an expensive resource. We propose a 2-component GMM
for unsupervised calibration and demonstrate good performance relative to a
supervised baseline on NIST SRE'10 and SRE'12. A Bayesian analysis demonstrates
that the uncertainty associated with the unsupervised calibration parameter
estimates is surprisingly small.
Stochastic Gradient Estimate Variance in Contrastive Divergence and Persistent Contrastive Divergence
Authors: Mathias Berglund, Tapani Raiko
Date: 2013-12-20 00:00:00

Contrastive Divergence (CD) and Persistent Contrastive Divergence (PCD) are
popular methods for training the weights of Restricted Boltzmann Machines.
However, both methods use an approximate method for sampling from the model
distribution. As a side effect, these approximations yield significantly
different biases and variances for stochastic gradient estimates of individual
data points. It is well known that CD yields a biased gradient estimate. In
this paper we however show empirically that CD has a lower stochastic gradient
estimate variance than exact sampling, while the mean of subsequent PCD
estimates has a higher variance than exact sampling. The results give one
explanation to the finding that CD can be used with smaller minibatches or
higher learning rates than PCD.
Authorship Analysis based on Data Compression
Authors: Daniele Cerra, Mihai Datcu, Peter Reinartz
Date: 2014-02-14 00:00:00
Tasks: Data Compression

This paper proposes to perform authorship analysis using the Fast Compression
Distance (FCD), a similarity measure based on compression with dictionaries
directly extracted from the written texts. The FCD computes a similarity
between two documents through an effective binary search on the intersection
set between the two related dictionaries. In the reported experiments the
proposed method is applied to documents which are heterogeneous in style,
written in five different languages and coming from different historical
periods. Results are comparable to the state of the art and outperform
traditional compression-based methods.
Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget
Authors: Anoop Korattikara, Yutian Chen, Max Welling
Date: 2013-04-19 00:00:00

Can we make Bayesian posterior MCMC sampling more efficient when faced with
very large datasets? We argue that computing the likelihood for N datapoints in
the Metropolis-Hastings (MH) test to reach a single binary decision is
computationally inefficient. We introduce an approximate MH rule based on a
sequential hypothesis test that allows us to accept or reject samples with high
confidence using only a fraction of the data required for the exact MH rule.
While this method introduces an asymptotic bias, we show that this bias can be
controlled and is more than offset by a decrease in variance due to our ability
to draw more samples per unit of time.
Machine Learning of Phonologically Conditioned Noun Declensions For Tamil Morphological Generators
Authors: K. Rajan, Dr. V. Ramalingam, Dr. M. Ganesan
Date: 2014-02-14 00:00:00
Tasks: BIG-bench Machine Learning, Text Generation

This paper presents machine learning solutions to a practical problem of
Natural Language Generation (NLG), particularly the word formation in
agglutinative languages like Tamil, in a supervised manner. The morphological
generator is an important component of Natural Language Processing in
Artificial Intelligence. It generates word forms given a root and affixes. The
morphophonemic changes like addition, deletion, alternation etc., occur when
two or more morphemes or words joined together. The Sandhi rules should be
explicitly specified in the rule based morphological analyzers and generators.
In machine learning framework, these rules can be learned automatically by the
system from the training samples and subsequently be applied for new inputs. In
this paper we proposed the machine learning models which learn the
morphophonemic rules for noun declensions from the given training data. These
models are trained to learn sandhi rules using various learning algorithms and
the performance of those algorithms are presented. From this we conclude that
machine learning of morphological processing such as word form generation can
be successfully learned in a supervised manner, without explicit description of
rules. The performance of Decision trees and Bayesian machine learning
algorithms on noun declensions are discussed.
An evaluative baseline for geo-semantic relatedness and similarity
Authors: Andrea Ballatore, Michela Bertolotto, David C. Wilson
Date: 2014-02-14 00:00:00
Tasks: Retrieval, Semantic Similarity, Semantic Textual Similarity

In geographic information science and semantics, the computation of semantic
similarity is widely recognised as key to supporting a vast number of tasks in
information integration and retrieval. By contrast, the role of geo-semantic
relatedness has been largely ignored. In natural language processing, semantic
relatedness is often confused with the more specific semantic similarity. In
this article, we discuss a notion of geo-semantic relatedness based on Lehrer's
semantic fields, and we compare it with geo-semantic similarity. We then
describe and validate the Geo Relatedness and Similarity Dataset (GeReSiD), a
new open dataset designed to evaluate computational measures of geo-semantic
relatedness and similarity. This dataset is larger than existing datasets of
this kind, and includes 97 geographic terms combined into 50 term pairs rated
by 203 human subjects. GeReSiD is available online and can be used as an
evaluation baseline to determine empirically to what degree a given
computational model approximates geo-semantic relatedness and similarity.
Active Clustering with Model-Based Uncertainty Reduction
Authors: Caiming Xiong, David Johnson, Jason J. Corso
Date: 2014-02-07 00:00:00
Tasks: Clustering, model

Semi-supervised clustering seeks to augment traditional clustering methods by
incorporating side information provided via human expertise in order to
increase the semantic meaningfulness of the resulting clusters. However, most
current methods are \emph{passive} in the sense that the side information is
provided beforehand and selected randomly. This may require a large number of
constraints, some of which could be redundant, unnecessary, or even detrimental
to the clustering results. Thus in order to scale such semi-supervised
algorithms to larger problems it is desirable to pursue an \emph{active}
clustering method---i.e. an algorithm that maximizes the effectiveness of the
available human labor by only requesting human input where it will have the
greatest impact. Here, we propose a novel online framework for active
semi-supervised spectral clustering that selects pairwise constraints as
clustering proceeds, based on the principle of uncertainty reduction. Using a
first-order Taylor expansion, we decompose the expected uncertainty reduction
problem into a gradient and a step-scale, computed via an application of matrix
perturbation theory and cluster-assignment entropy, respectively. The resulting
model is used to estimate the uncertainty reduction potential of each sample in
the dataset. We then present the human user with pairwise queries with respect
to only the best candidate sample. We evaluate our method using three different
image datasets (faces, leaves and dogs), a set of common UCI machine learning
datasets and a gene dataset. The results validate our decomposition formulation
and show that our method is consistently superior to existing state-of-the-art
techniques, as well as being robust to noise and to unknown numbers of
clusters.
Recovering Graph-Structured Activations using Adaptive Compressive Measurements
Authors: Akshay Krishnamurthy, James Sharpnack, Aarti Singh
Date: 2013-05-01 00:00:00

We study the localization of a cluster of activated vertices in a graph, from
adaptively designed compressive measurements. We propose a hierarchical
partitioning of the graph that groups the activated vertices into few
partitions, so that a top-down sensing procedure can identify these partitions,
and hence the activations, using few measurements. By exploiting the cluster
structure, we are able to provide localization guarantees at weaker signal to
noise ratios than in the unstructured setting. We complement this performance
guarantee with an information theoretic lower bound, providing a necessary
signal-to-noise ratio for any algorithm to successfully localize the cluster.
We verify our analysis with some simulations, demonstrating the practicality of
our algorithm.
Grid Topology Identification using Electricity Prices
Authors: Vassilis Kekatos, Georgios B. Giannakis, Ross Baldick
Date: 2013-12-02 00:00:00

The potential of recovering the topology of a grid using solely publicly
available market data is explored here. In contemporary whole-sale electricity
markets, real-time prices are typically determined by solving the
network-constrained economic dispatch problem. Under a linear DC model,
locational marginal prices (LMPs) correspond to the Lagrange multipliers of the
linear program involved. The interesting observation here is that the matrix of
spatiotemporally varying LMPs exhibits the following property: Once
premultiplied by the weighted grid Laplacian, it yields a low-rank and sparse
matrix. Leveraging this rich structure, a regularized maximum likelihood
estimator (MLE) is developed to recover the grid Laplacian from the LMPs. The
convex optimization problem formulated includes low rank- and
sparsity-promoting regularizers, and it is solved using a scalable algorithm.
Numerical tests on prices generated for the IEEE 14-bus benchmark provide
encouraging topology recovery results.
Hand-Eye and Robot-World Calibration by Global Polynomial Optimization
Authors: Jan Heller, Didier Henrion, Tomas Pajdla
Date: 2014-02-13 00:00:00

The need to relate measurements made by a camera to a different known
coordinate system arises in many engineering applications. Historically, it
appeared for the first time in the connection with cameras mounted on robotic
systems. This problem is commonly known as hand-eye calibration. In this paper,
we present several formulations of hand-eye calibration that lead to
multivariate polynomial optimization problems. We show that the method of
convex linear matrix inequality (LMI) relaxations can be used to effectively
solve these problems and to obtain globally optimal solutions. Further, we show
that the same approach can be used for the simultaneous hand-eye and
robot-world calibration. Finally, we validate the proposed solutions using both
synthetic and real datasets.
Relations on FP-Soft Sets Applied to Decision Making Problems
Authors: Irfan Deli, Naim Çağman
Date: 2014-02-13 00:00:00
Tasks: Decision Making

In this work, we first define relations on the fuzzy parametrized soft sets
and study their properties. We also give a decision making method based on
these relations. In approximate reasoning, relations on the fuzzy parametrized
soft sets have shown to be of a primordial importance. Finally, the method is
successfully applied to a problems that contain uncertainties.
A Unifying Survey of Reinforced, Sensitive and Stigmergic Agent-Based Approaches for E-GTSP
Authors: Camelia-M. Pintea
Date: 2012-08-24 00:00:00
Tasks: Combinatorial Optimization, Traveling Salesman Problem

The Generalized Traveling Salesman Problem (GTSP) is one of the NP-hard
combinatorial optimization problems. A variant of GTSP is E-GTSP where E,
meaning equality, has the constraint: exactly one node from a cluster of a
graph partition is visited. The main objective of the E-GTSP is to find a
minimum cost tour passing through exactly one node from each cluster of an
undirected graph. Agent-based approaches involving are successfully used
nowadays for solving real life complex problems. The aim of the current paper
is to illustrate some variants of agent-based algorithms including ant-based
models with specific properties for solving E-GTSP.
Gaussian Process Volatility Model
Authors: Yue Wu, Jose Miguel Hernandez Lobato, Zoubin Ghahramani
Date: 2014-02-13 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Gaussian Processes, model

The accurate prediction of time-changing variances is an important task in
the modeling of financial data. Standard econometric models are often limited
as they assume rigid functional relationships for the variances. Moreover,
function parameters are usually learned using maximum likelihood, which can
lead to overfitting. To address these problems we introduce a novel model for
time-changing variances using Gaussian Processes. A Gaussian Process (GP)
defines a distribution over functions, which allows us to capture highly
flexible functional relationships for the variances. In addition, we develop an
online algorithm to perform inference. The algorithm has two main advantages.
First, it takes a Bayesian approach, thereby avoiding overfitting. Second, it
is much quicker than current offline inference procedures. Finally, our new
model was evaluated on financial data and showed significant improvement in
predictive performance over current standard models.
Software Requirement Specification Using Reverse Speech Technology
Authors: Santhy Viswam, Sajeer Karattil
Date: 2014-02-13 00:00:00

Speech analysis had been taken to a new level with the discovery of Reverse
Speech (RS). RS is the discovery of hidden messages, referred as reversals, in
normal speech. Works are in progress for exploiting the relevance of RS in
different real world applications such as investigation, medical field etc. In
this paper we represent an innovative method for preparing a reliable Software
Requirement Specification (SRS) document with the help of reverse speech. As
SRS act as the backbone for the successful completion of any project, a
reliable method is needed to overcome the inconsistencies. Using RS such a
reliable method for SRS documentation was developed.
Squeezing bottlenecks: exploring the limits of autoencoder semantic representation capabilities
Authors: Parth Gupta, Rafael E. Banchs, Paolo Rosso
Date: 2014-02-13 00:00:00
Tasks: Sentence

We present a comprehensive study on the use of autoencoders for modelling
text data, in which (differently from previous studies) we focus our attention
on the following issues: i) we explore the suitability of two different models
bDA and rsDA for constructing deep autoencoders for text data at the sentence
level; ii) we propose and evaluate two novel metrics for better assessing the
text-reconstruction capabilities of autoencoders; and iii) we propose an
automatic method to find the critical bottleneck dimensionality for text
language representations (below which structural information is lost).
Event Structure of Transitive Verb: A MARVS perspective
Authors: Jia-Fei Hong, Kathleen Ahrens, Chu-Ren Huang
Date: 2014-02-13 00:00:00
Tasks: Attribute

Module-Attribute Representation of Verbal Semantics (MARVS) is a theory of
the representation of verbal semantics that is based on Mandarin Chinese data
(Huang et al. 2000). In the MARVS theory, there are two different types of
modules: Event Structure Modules and Role Modules. There are also two sets of
attributes: Event-Internal Attributes and Role-Internal Attributes, which are
linked to the Event Structure Module and the Role Module, respectively. In this
study, we focus on four transitive verbs as chi1(eat), wan2(play),
huan4(change) and shao1(burn) and explore their event structures by the MARVS
theory.
Regularization for Multiple Kernel Learning via Sum-Product Networks
Authors: Ziming Zhang
Date: 2014-02-13 00:00:00

In this paper, we are interested in constructing general graph-based
regularizers for multiple kernel learning (MKL) given a structure which is used
to describe the way of combining basis kernels. Such structures are represented
by sum-product networks (SPNs) in our method. Accordingly we propose a new
convex regularization method for MLK based on a path-dependent kernel weighting
function which encodes the entire SPN structure in our method. Under certain
conditions and from the view of probability, this function can be considered to
follow multinomial distributions over the weights associated with product nodes
in SPNs. We also analyze the convexity of our regularizer and the complexity of
our induced classifiers, and further propose an efficient wrapper algorithm to
optimize our formulation. In our experiments, we apply our method to ......
Prediction with Missing Data via Bayesian Additive Regression Trees
Authors: Adam Kapelner, Justin Bleich
Date: 2013-06-03 00:00:00
Tasks: Imputation, regression

We present a method for incorporating missing data in non-parametric
statistical learning without the need for imputation. We focus on a tree-based
method, Bayesian Additive Regression Trees (BART), enhanced with "Missingness
Incorporated in Attributes," an approach recently proposed incorporating
missingness into decision trees (Twala, 2008). This procedure takes advantage
of the partitioning mechanisms found in tree-based models. Simulations on
generated models and real data indicate that our proposed method can forecast
well on complicated missing-at-random and not-missing-at-random models as well
as models where missingness itself influences the response. Our procedure has
higher predictive performance and is more stable than competitors in many
cases. We also illustrate BART's abilities to incorporate missingness into
uncertainty intervals and to detect the influence of missingness on the model
fit.
Local Optima Networks: A New Model of Combinatorial Fitness Landscapes
Authors: Gabriela Ochoa, Sébastien Verel, Fabio Daolio, Marco Tomassini
Date: 2014-02-12 00:00:00
Tasks: Heuristic Search

This chapter overviews a recently introduced network-based model of
combinatorial landscapes: Local Optima Networks (LON). The model compresses the
information given by the whole search space into a smaller mathematical object
that is a graph having as vertices the local optima and as edges the possible
weighted transitions between them. Two definitions of edges have been proposed:
basin-transition and escape-edges, which capture relevant topological features
of the underlying search spaces. This network model brings a new set of metrics
to characterize the structure of combinatorial landscapes, those associated
with the science of complex networks. These metrics are described, and results
are presented of local optima network extraction and analysis for two selected
combinatorial landscapes: NK landscapes and the quadratic assignment problem.
Network features are found to correlate with and even predict the performance
of heuristic search algorithms operating on these problems.
Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization
Authors: Martin Slawski, Matthias Hein
Date: 2012-05-04 00:00:00

Least squares fitting is in general not useful for high-dimensional linear
models, in which the number of predictors is of the same or even larger order
of magnitude than the number of samples. Theory developed in recent years has
coined a paradigm according to which sparsity-promoting regularization is
regarded as a necessity in such setting. Deviating from this paradigm, we show
that non-negativity constraints on the regression coefficients may be similarly
effective as explicit regularization if the design matrix has additional
properties, which are met in several applications of non-negative least squares
(NNLS). We show that for these designs, the performance of NNLS with regard to
prediction and estimation is comparable to that of the lasso. We argue further
that in specific cases, NNLS may have a better $\ell_{\infty}$-rate in
estimation and hence also advantages with respect to support recovery when
combined with thresholding. From a practical point of view, NNLS does not
depend on a regularization parameter and is hence easier to use.