pwc-archive/papers-with-abstracts huggingface dataset:
Twist your logic with TouIST
Authors: Khaled Skander Ben Slimane, Alexis Comte, Olivier Gasquet, Abdelwahab Heba, Olivier Lezaud, Frederic Maris, Mael Valais
Date: 2015-07-14 00:00:00

SAT provers are powerful tools for solving real-sized logic problems, but
using them requires solid programming knowledge and may be seen w.r.t.\ logic
like assembly language w.r.t.\ programming. Something like a high level
language was missing to ease various users to take benefit of these tools. {\sc
\texttt {TouIST}}\ aims at filling this gap. It is devoted to propositional
logic and its main features are 1) to offer a high-level logic langage for
expressing succintly complex formulas (e.g.\ formulas describing Sudoku rules,
planification problems,\ldots) and 2) to find models to these formulas by using
the adequate powerful prover, which the user has no need to know about. It
consists in a friendly interface that offers several syntactic facilities and
which is connected with some sufficiently powerful provers allowing to
automatically solve big instances of difficult problems (such as time-tables or
Sudokus). It can interact with various provers: pure SAT solver but also SMT
provers (SAT modulo theories - like linear theory of reals, etc) and thus may
also be used by beginners for experiencing with pure propositional problems up
to graduate students or even researchers for solving planification problems
involving big sets of fluents and numerical constraints on them.
Neural CRF Parsing
Authors: Greg Durrett, Dan Klein
Date: 2015-07-13 00:00:00
Proceeding: IJCNLP 2015 7

This paper describes a parsing model that combines the exact dynamic
programming of CRF parsing with the rich nonlinear featurization of neural net
approaches. Our model is structurally a CRF that factors over anchored rule
productions, but instead of linear potential functions based on sparse
features, we use nonlinear potentials computed via a feedforward neural
network. Because potentials are still local to anchored rules, structured
inference (CKY) is unchanged from the sparse case. Computing gradients during
learning involves backpropagating an error signal formed from standard CRF
sufficient statistics (expected rule counts). Using only dense features, our
neural CRF already exceeds a strong baseline CRF model (Hall et al., 2014). In
combination with sparse features, our system achieves 91.1 F1 on section 23 of
the Penn Treebank, and more generally outperforms the best prior single parser
results on a range of languages.
Beyond Hartigan Consistency: Merge Distortion Metric for Hierarchical Clustering
Authors: Justin Eldridge, Mikhail Belkin, Yusu Wang
Date: 2015-06-21 00:00:00
Tasks: Clustering

Hierarchical clustering is a popular method for analyzing data which
associates a tree to a dataset. Hartigan consistency has been used extensively
as a framework to analyze such clustering algorithms from a statistical point
of view. Still, as we show in the paper, a tree which is Hartigan consistent
with a given density can look very different than the correct limit tree.
Specifically, Hartigan consistency permits two types of undesirable
configurations which we term over-segmentation and improper nesting. Moreover,
Hartigan consistency is a limit property and does not directly quantify
difference between trees.
  In this paper we identify two limit properties, separation and minimality,
which address both over-segmentation and improper nesting and together imply
(but are not implied by) Hartigan consistency. We proceed to introduce a merge
distortion metric between hierarchical clusterings and show that convergence in
our distance implies both separation and minimality. We also prove that uniform
separation and minimality imply convergence in the merge distortion metric.
Furthermore, we show that our merge distortion metric is stable under
perturbations of the density.
  Finally, we demonstrate applicability of these concepts by proving
convergence results for two clustering algorithms. First, we show convergence
(and hence separation and minimality) of the recent robust single linkage
algorithm of Chaudhuri and Dasgupta (2010). Second, we provide convergence
results on manifolds for topological split tree clustering.
Priors for Random Count Matrices Derived from a Family of Negative Binomial Processes
Authors: Mingyuan Zhou, Oscar Hernan Madrid Padilla, James G. Scott
Date: 2014-04-12 00:00:00
Tasks: feature selection, text-classification, Text Classification

We define a family of probability distributions for random count matrices
with a potentially unbounded number of rows and columns. The three
distributions we consider are derived from the gamma-Poisson, gamma-negative
binomial, and beta-negative binomial processes. Because the models lead to
closed-form Gibbs sampling update equations, they are natural candidates for
nonparametric Bayesian priors over count matrices. A key aspect of our analysis
is the recognition that, although the random count matrices within the family
are defined by a row-wise construction, their columns can be shown to be i.i.d.
This fact is used to derive explicit formulas for drawing all the columns at
once. Moreover, by analyzing these matrices' combinatorial structure, we
describe how to sequentially construct a column-i.i.d. random count matrix one
row at a time, and derive the predictive distribution of a new row count vector
with previously unseen features. We describe the similarities and differences
between the three priors, and argue that the greater flexibility of the gamma-
and beta- negative binomial processes, especially their ability to model
over-dispersed, heavy-tailed count data, makes these well suited to a wide
variety of real-world applications. As an example of our framework, we
construct a naive-Bayes text classifier to categorize a count vector to one of
several existing random count matrices of different categories. The classifier
supports an unbounded number of features, and unlike most existing methods, it
does not require a predefined finite vocabulary to be shared by all the
categories, and needs neither feature selection nor parameter tuning. Both the
gamma- and beta- negative binomial processes are shown to significantly
outperform the gamma-Poisson process for document categorization, with
comparable performance to other state-of-the-art supervised text classification
algorithms.
Scalable Bayesian Optimization Using Deep Neural Networks
Authors: Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Md. Mostofa Ali Patwary, Prabhat, Ryan P. Adams
Date: 2015-02-19 00:00:00
Tasks: Bayesian Optimization, Caption Generation, Gaussian Processes, global-optimization, Hyperparameter Optimization, Image Classification, Object Recognition

Bayesian optimization is an effective methodology for the global optimization
of functions with expensive evaluations. It relies on querying a distribution
over functions defined by a relatively cheap surrogate model. An accurate model
for this distribution over functions is critical to the effectiveness of the
approach, and is typically fit using Gaussian processes (GPs). However, since
GPs scale cubically with the number of observations, it has been challenging to
handle objectives whose optimization requires many evaluations, and as such,
massively parallelizing the optimization.
  In this work, we explore the use of neural networks as an alternative to GPs
to model distributions over functions. We show that performing adaptive basis
function regression with a neural network as the parametric form performs
competitively with state-of-the-art GP-based approaches, but scales linearly
with the number of data rather than cubically. This allows us to achieve a
previously intractable degree of parallelism, which we apply to large scale
hyperparameter optimization, rapidly finding competitive models on benchmark
object recognition tasks using convolutional networks, and image caption
generation using neural language models.
The mRMR variable selection method: a comparative study for functional data
Authors: José R. Berrendero, Antonio Cuevas, José L. Torrecilla
Date: 2015-07-13 00:00:00
Tasks: Variable Selection

The use of variable selection methods is particularly appealing in
statistical problems with functional data. The obvious general criterion for
variable selection is to choose the `most representative' or `most relevant'
variables. However, it is also clear that a purely relevance-oriented criterion
could lead to select many redundant variables. The mRMR (minimum Redundance
Maximum Relevance) procedure, proposed by Ding and Peng (2005) and Peng et al.
(2005) is an algorithm to systematically perform variable selection, achieving
a reasonable trade-off between relevance and redundancy. In its original form,
this procedure is based on the use of the so-called mutual information
criterion to assess relevance and redundancy. Keeping the focus on functional
data problems, we propose here a modified version of the mRMR method, obtained
by replacing the mutual information by the new association measure (called
distance correlation) suggested by Sz\'ekely et al. (2007). We have also
performed an extensive simulation study, including 1600 functional experiments
(100 functional models $\times$ 4 sample sizes $\times$ 4 classifiers) and
three real-data examples aimed at comparing the different versions of the mRMR
methodology. The results are quite conclusive in favor of the new proposed
alternative.
Individual performance calibration using physiological stress signals
Authors: Francisco Hernando-Gallego, Antonio Artés-Rodríguez
Date: 2015-07-13 00:00:00

The relation between performance and stress is described by the Yerkes-Dodson
Law but varies significantly between individuals. This paper describes a method
for determining the individual optimal performance as a function of
physiological signals. The method is based on attention and reasoning tests of
increasing complexity under monitoring of three physiological signals: Galvanic
Skin Response (GSR), Heart Rate (HR), and Electromyogram (EMG). Based on the
test results with 15 different individuals, we first show that two of the
signals, GSR and HR, have enough discriminative power to distinguish between
relax and stress periods. We then show a positive correlation between the
complexity level of the tests and the GSR and HR signals, and we finally
determine the optimal performance point as the signal level just before a
performance decrease. We also discuss the differences among signals depending
on the type of test.
Incremental LSTM-based Dialog State Tracker
Authors: Lukas Zilka, Filip Jurcicek
Date: 2015-07-13 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

A dialog state tracker is an important component in modern spoken dialog
systems. We present an incremental dialog state tracker, based on LSTM
networks. It directly uses automatic speech recognition hypotheses to track the
state. We also present the key non-standard aspects of the model that bring its
performance close to the state-of-the-art and experimentally analyze their
contribution: including the ASR confidence scores, abstracting scarcely
represented values, including transcriptions in the training data, and model
averaging.
Supervised Hierarchical Classification for Student Answer Scoring
Authors: Itziar Aldabe, Oier Lopez de Lacalle, Iñigo Lopez-Gazpio, Montse Maritxalar
Date: 2015-07-13 00:00:00
Tasks: Classification, General Classification

This paper describes a hierarchical system that predicts one label at a time
for automated student response analysis. For the task, we build a
classification binary tree that delays more easily confused labels to later
stages using hierarchical processes. In particular, the paper describes how the
hierarchical classifier has been built and how the classification task has been
broken down into binary subtasks. It finally discusses the motivations and
fundamentals of such an approach.
Zipf's law for word frequencies: word forms versus lemmas in long texts
Authors: Alvaro Corral, Gemma Boleda, Ramon Ferrer-i-Cancho
Date: 2014-07-31 00:00:00
Tasks: LEMMA, valid

Zipf's law is a fundamental paradigm in the statistics of written and spoken
natural language as well as in other communication systems. We raise the
question of the elementary units for which Zipf's law should hold in the most
natural way, studying its validity for plain word forms and for the
corresponding lemma forms. In order to have as homogeneous sources as possible,
we analyze some of the longest literary texts ever written, comprising four
different languages, with different levels of morphological complexity. In all
cases Zipf's law is fulfilled, in the sense that a power-law distribution of
word or lemma frequencies is valid for several orders of magnitude. We
investigate the extent to which the word-lemma transformation preserves two
parameters of Zipf's law: the exponent and the low-frequency cut-off. We are
not able to demonstrate a strict invariance of the tail, as for a few texts
both exponents deviate significantly, but we conclude that the exponents are
very similar, despite the remarkable transformation that going from words to
lemmas represents, considerably affecting all ranges of frequencies. In
contrast, the low-frequency cut-offs are less stable.
Direct Variational Perspective Shape from Shading with Cartesian Depth Parametrisation
Authors: Yong Chul Ju, Daniel Maurer, Michael Breuß, Andrés Bruhn
Date: 2015-05-22 00:00:00
Tasks: valid

Most of today's state-of-the-art methods for perspective shape from shading
are modelled in terms of partial differential equations (PDEs) of
Hamilton-Jacobi type. To improve the robustness of such methods w.r.t. noise
and missing data, first approaches have recently been proposed that seek to
embed the underlying PDE into a variational framework with data and smoothness
term. So far, however, such methods either make use of a radial depth
parametrisation that makes the regularisation hard to interpret from a
geometrical viewpoint or they consider indirect smoothness terms that require
additional consistency constraints to provide valid solutions. Moreover the
minimisation of such frameworks is an intricate task, since the underlying
energy is typically non-convex. In our paper we address all three of the
aforementioned issues. First, we propose a novel variational model that
operates directly on the Cartesian depth. In this context, we also point out a
common mistake in the derivation of the surface normal. Moreover, we employ a
direct second-order regulariser with edge-preservation property. This direct
regulariser yields by construction valid solutions without requiring additional
consistency constraints. Finally, we also propose a novel coarse-to-fine
minimisation framework based on an alternating explicit scheme. This framework
allows us to avoid local minima during the minimisation and thus to improve the
accuracy of the reconstruction. Experiments show the good quality of our model
as well as the usefulness of the proposed numerical scheme.
Information Compression, Intelligence, Computing, and Mathematics
Authors: J. Gerard Wolff
Date: 2013-10-31 00:00:00
Tasks: Chunking, Information Retrieval, Retrieval

This paper presents evidence for the idea that much of artificial
intelligence, human perception and cognition, mainstream computing, and
mathematics, may be understood as compression of information via the matching
and unification of patterns. This is the basis for the "SP theory of
intelligence", outlined in the paper and fully described elsewhere. Relevant
evidence may be seen: in empirical support for the SP theory; in some
advantages of information compression (IC) in terms of biology and engineering;
in our use of shorthands and ordinary words in language; in how we merge
successive views of any one thing; in visual recognition; in binocular vision;
in visual adaptation; in how we learn lexical and grammatical structures in
language; and in perceptual constancies. IC via the matching and unification of
patterns may be seen in both computing and mathematics: in IC via equations; in
the matching and unification of names; in the reduction or removal of
redundancy from unary numbers; in the workings of Post's Canonical System and
the transition function in the Universal Turing Machine; in the way computers
retrieve information from memory; in systems like Prolog; and in the
query-by-example technique for information retrieval. The chunking-with-codes
technique for IC may be seen in the use of named functions to avoid repetition
of computer code. The schema-plus-correction technique may be seen in functions
with parameters and in the use of classes in object-oriented programming. And
the run-length coding technique may be seen in multiplication, in division, and
in several other devices in mathematics and computing. The SP theory resolves
the apparent paradox of "decompression by compression". And computing and
cognition as IC is compatible with the uses of redundancy in such things as
backup copies to safeguard data and understanding speech in a noisy
environment.
Sparsity assisted solution to the twin image problem in phase retrieval
Authors: Charu Gaur, Baranidharan Mohan, Kedar Khare
Date: 2015-07-13 00:00:00
Tasks: Object, Retrieval

The iterative phase retrieval problem for complex-valued objects from Fourier
transform magnitude data is known to suffer from the twin image problem. In
particular, when the object support is centro-symmetric, the iterative solution
often stagnates such that the resultant complex image contains the features of
both the desired solution and its inverted and complex-conjugated replica. The
conventional approach to address the twin image problem is to modify the object
support during initial iterations which can possibly lead to elimination of one
of the twin images. However, at present there seems to be no deterministic
procedure to make sure that the twin image will always be very weak or absent.
In this work we make an important observation that the ideal solution without
the twin image is typically more sparse (in some suitable transform domain) as
compared to the stagnated solution containing the twin image. We further show
that introducing a sparsity enhancing step in the iterative algorithm can
address the twin image problem without the need to change the object support
throughout the iterative process even when the object support is
centro-symmetric. In a simulation study, we use binary and gray-scale pure
phase objects and illustrate the effectiveness of the sparsity assisted phase
recovery in the context of the twin image problem. The results have important
implications for a wide range of topics in Physics where the phase retrieval
problem plays a central role.
Score-based Causal Learning in Additive Noise Models
Authors: Christopher Nowzohour, Peter Bühlmann
Date: 2013-11-25 00:00:00
Tasks: Model Selection

Given data sampled from a number of variables, one is often interested in the
underlying causal relationships in the form of a directed acyclic graph. In the
general case, without interventions on some of the variables it is only
possible to identify the graph up to its Markov equivalence class. However, in
some situations one can find the true causal graph just from observational
data, for example in structural equation models with additive noise and
nonlinear edge functions. Most current methods for achieving this rely on
nonparametric independence tests. One of the problems there is that the null
hypothesis is independence, which is what one would like to get evidence for.
We take a different approach in our work by using a penalized likelihood as a
score for model selection. This is practically feasible in many settings and
has the advantage of yielding a natural ranking of the candidate models. When
making smoothness assumptions on the probability density space, we prove
consistency of the penalized maximum likelihood estimator. We also present
empirical results for simulated scenarios and real two-dimensional data sets
(cause-effect pairs) where we obtain similar results as other state-of-the-art
methods.
Quantitative Evaluation of Performance and Validity Indices for Clustering the Web Navigational Sessions
Authors: Zahid Ansari, M. F. Azeem, Waseem Ahmed, A. Vinaya Babu
Date: 2015-07-13 00:00:00
Tasks: Clustering

Clustering techniques are widely used in Web Usage Mining to capture similar
interests and trends among users accessing a Web site. For this purpose, web
access logs generated at a particular web site are preprocessed to discover the
user navigational sessions. Clustering techniques are then applied to group the
user session data into user session clusters, where intercluster similarities
are minimized while the intra cluster similarities are maximized. Since the
application of different clustering algorithms generally results in different
sets of cluster formation, it is important to evaluate the performance of these
methods in terms of accuracy and validity of the clusters, and also the time
required to generate them, using appropriate performance measures. This paper
describes various validity and accuracy measures including Dunn's Index, Davies
Bouldin Index, C Index, Rand Index, Jaccard Index, Silhouette Index, Fowlkes
Mallows and Sum of the Squared Error (SSE). We conducted the performance
evaluation of the following clustering techniques: k-Means, k-Medoids, Leader,
Single Link Agglomerative Hierarchical and DBSCAN. These techniques are
implemented and tested against the Web user navigational data. Finally their
performance results are presented and compared.
EigenGP: Gaussian Process Models with Adaptive Eigenfunctions
Authors: Hao Peng, Yuan Qi
Date: 2014-01-02 00:00:00
Tasks: Gaussian Processes

Gaussian processes (GPs) provide a nonparametric representation of functions.
However, classical GP inference suffers from high computational cost for big
data. In this paper, we propose a new Bayesian approach, EigenGP, that learns
both basis dictionary elements--eigenfunctions of a GP prior--and prior
precisions in a sparse finite model. It is well known that, among all
orthogonal basis functions, eigenfunctions can provide the most compact
representation. Unlike other sparse Bayesian finite models where the basis
function has a fixed form, our eigenfunctions live in a reproducing kernel
Hilbert space as a finite linear combination of kernel functions. We learn the
dictionary elements--eigenfunctions--and the prior precisions over these
elements as well as all the other hyperparameters from data by maximizing the
model marginal likelihood. We explore computational linear algebra to simplify
the gradient computation significantly. Our experimental results demonstrate
improved predictive performance of EigenGP over alternative sparse GP methods
as well as relevance vector machine.
First-order integer programming for MAP problems
Authors: James Cussens
Date: 2015-07-10 00:00:00

Finding the most probable (MAP) model in SRL frameworks such as Markov logic
and Problog can, in principle, be solved by encoding the problem as a
`grounded-out' mixed integer program (MIP). However, useful first-order
structure disappears in this process motivating the development of first-order
MIP approaches. Here we present mfoilp, one such approach. Since the syntax and
semantics of mfoilp is essentially the same as existing approaches we focus
here mainly on implementation and algorithmic issues. We start with the
(conceptually) simple problem of using a logic program to generate a MIP
instance before considering more ambitious exploitation of first-order
representations.