pwc-archive/papers-with-abstracts huggingface dataset:
Square Root Graphical Models: Multivariate Generalizations of Univariate Exponential Families that Permit Positive Dependencies
Authors: David I. Inouye, Pradeep Ravikumar, Inderjit S. Dhillon
Date: 2016-03-11 00:00:00
Tasks: parameter estimation

We develop Square Root Graphical Models (SQR), a novel class of parametric
graphical models that provides multivariate generalizations of univariate
exponential family distributions. Previous multivariate graphical models [Yang
et al. 2015] did not allow positive dependencies for the exponential and
Poisson generalizations. However, in many real-world datasets, variables
clearly have positive dependencies. For example, the airport delay time in New
York---modeled as an exponential distribution---is positively related to the
delay time in Boston. With this motivation, we give an example of our model
class derived from the univariate exponential distribution that allows for
almost arbitrary positive and negative dependencies with only a mild condition
on the parameter matrix---a condition akin to the positive definiteness of the
Gaussian covariance matrix. Our Poisson generalization allows for both positive
and negative dependencies without any constraints on the parameter values. We
also develop parameter estimation methods using node-wise regressions with
$\ell_1$ regularization and likelihood approximation methods using sampling.
Finally, we demonstrate our exponential generalization on a synthetic dataset
and a real-world dataset of airport delay times.
Iterative Instance Segmentation
Authors: Ke Li, Bharath Hariharan, Jitendra Malik
Date: 2015-11-26 00:00:00
Proceeding: CVPR 2016 6
Tasks: Instance Segmentation, Prediction, Segmentation, Semantic Segmentation, Structured Prediction

Existing methods for pixel-wise labelling tasks generally disregard the
underlying structure of labellings, often leading to predictions that are
visually implausible. While incorporating structure into the model should
improve prediction quality, doing so is challenging - manually specifying the
form of structural constraints may be impractical and inference often becomes
intractable even if structural constraints are given. We sidestep this problem
by reducing structured prediction to a sequence of unconstrained prediction
problems and demonstrate that this approach is capable of automatically
discovering priors on shape, contiguity of region predictions and smoothness of
region contours from data without any a priori specification. On the instance
segmentation task, this method outperforms the state-of-the-art, achieving a
mean $\mathrm{AP}^{r}$ of 63.6% at 50% overlap and 43.3% at 70% overlap.
Memory-Efficient Backpropagation Through Time
Authors: Audrūnas Gruslys, Remi Munos, Ivo Danihelka, Marc Lanctot, Alex Graves
Date: 2016-06-10 00:00:00
Proceeding: NeurIPS 2016 12

We propose a novel approach to reduce memory consumption of the
backpropagation through time (BPTT) algorithm when training recurrent neural
networks (RNNs). Our approach uses dynamic programming to balance a trade-off
between caching of intermediate results and recomputation. The algorithm is
capable of tightly fitting within almost any user-set memory budget while
finding an optimal execution policy minimizing the computational cost.
Computational devices have limited memory capacity and maximizing a
computational performance given a fixed memory budget is a practical use-case.
We provide asymptotic computational upper bounds for various regimes. The
algorithm is particularly effective for long sequences. For sequences of length
1000, our algorithm saves 95\% of memory usage while using only one third more
time per iteration than the standard BPTT.
Natural Language Comprehension with the EpiReader
Authors: Adam Trischler, Zheng Ye, Xingdi Yuan, Kaheer Suleman
Date: 2016-06-07 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Question Answering, Reading Comprehension

We present the EpiReader, a novel model for machine comprehension of text.
Machine comprehension of unstructured, real-world text is a major research goal
for natural language processing. Current tests of machine comprehension pose
questions whose answers can be inferred from some supporting text, and evaluate
a model's response to the questions. The EpiReader is an end-to-end neural
model comprising two components: the first component proposes a small set of
candidate answers after comparing a question to its supporting text, and the
second component formulates hypotheses using the proposed candidates and the
question, then reranks the hypotheses based on their estimated concordance with
the supporting text. We present experiments demonstrating that the EpiReader
sets a new state-of-the-art on the CNN and Children's Book Test machine
comprehension benchmarks, outperforming previous neural models by a significant
margin.
FOMTrace: Interactive Video Segmentation By Image Graphs and Fuzzy Object Models
Authors: Thiago Vallin Spina, Alexandre Xavier Falcão
Date: 2016-06-10 00:00:00
Tasks: Object, Object Tracking, Segmentation, Video Editing, Video Segmentation, Video Semantic Segmentation

Common users have changed from mere consumers to active producers of
multimedia data content. Video editing plays an important role in this
scenario, calling for simple segmentation tools that can handle fast-moving and
deformable video objects with possible occlusions, color similarities with the
background, among other challenges. We present an interactive video
segmentation method, named FOMTrace, which addresses the problem in an
effective and efficient way. From a user-provided object mask in a first frame,
the method performs semi-automatic video segmentation on a spatiotemporal
superpixel-graph, and then estimates a Fuzzy Object Model (FOM), which refines
segmentation of the second frame by constraining delineation on a pixel-graph
within a region where the object's boundary is expected to be. The user can
correct/accept the refined object mask in the second frame, which is then
similarly used to improve the spatiotemporal video segmentation of the
remaining frames. Both steps are repeated alternately, within interactive
response times, until the segmentation refinement of the final frame is
accepted by the user. Extensive experiments demonstrate FOMTrace's ability for
tracing objects in comparison with state-of-the-art approaches for interactive
video segmentation, supervised, and unsupervised object tracking.
Conditional Generation and Snapshot Learning in Neural Dialogue Systems
Authors: Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Lina M. Rojas-Barahona, Pei-Hao Su, Stefan Ultes, David Vandyke, Steve Young
Date: 2016-06-10 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Text Generation

Recently a variety of LSTM-based conditional language models (LM) have been
applied across a range of language generation tasks. In this work we study
various model architectures and different ways to represent and aggregate the
source information in an end-to-end neural dialogue system framework. A method
called snapshot learning is also proposed to facilitate learning from
supervised sequential signals by applying a companion cross-entropy objective
function to the conditioning vector. The experimental and analytical results
demonstrate firstly that competition occurs between the conditioning vector and
the LM, and the differing architectures provide different trade-offs between
the two. Secondly, the discriminative power and transparency of the
conditioning vector is key to providing both model interpretability and better
performance. Thirdly, snapshot learning leads to consistent performance
improvements independent of which architecture is used.
Neural Machine Translation of Rare Words with Subword Units
Authors: Rico Sennrich, Barry Haddow, Alexandra Birch
Date: 2015-08-31 00:00:00
Proceeding: ACL 2016 8
Tasks: Machine Translation, NMT, Segmentation, Translation

Neural machine translation (NMT) models typically operate with a fixed
vocabulary, but translation is an open-vocabulary problem. Previous work
addresses the translation of out-of-vocabulary words by backing off to a
dictionary. In this paper, we introduce a simpler and more effective approach,
making the NMT model capable of open-vocabulary translation by encoding rare
and unknown words as sequences of subword units. This is based on the intuition
that various word classes are translatable via smaller units than words, for
instance names (via character copying or transliteration), compounds (via
compositional translation), and cognates and loanwords (via phonological and
morphological transformations). We discuss the suitability of different word
segmentation techniques, including simple character n-gram models and a
segmentation based on the byte pair encoding compression algorithm, and
empirically show that subword models improve over a back-off dictionary
baseline for the WMT 15 translation tasks English-German and English-Russian by
1.1 and 1.3 BLEU, respectively.
WordNet2Vec: Corpora Agnostic Word Vectorization Method
Authors: Roman Bartusiak, Łukasz Augustyniak, Tomasz Kajdanowicz, Przemysław Kazienko, Maciej Piasecki
Date: 2016-06-10 00:00:00
Tasks: Clustering, General Classification, Position, Sentiment Analysis, Transfer Learning

A complex nature of big data resources demands new methods for structuring
especially for textual content. WordNet is a good knowledge source for
comprehensive abstraction of natural language as its good implementations exist
for many languages. Since WordNet embeds natural language in the form of a
complex network, a transformation mechanism WordNet2Vec is proposed in the
paper. It creates vectors for each word from WordNet. These vectors encapsulate
general position - role of a given word towards all other words in the natural
language. Any list or set of such vectors contains knowledge about the context
of its component within the whole language. Such word representation can be
easily applied to many analytic tasks like classification or clustering. The
usefulness of the WordNet2Vec method was demonstrated in sentiment analysis,
i.e. classification with transfer learning for the real Amazon opinion textual
dataset.
Automatic Genre and Show Identification of Broadcast Media
Authors: Mortaza Doulaty, Oscar Saz, Raymond W. M. Ng, Thomas Hain
Date: 2016-06-10 00:00:00

Huge amounts of digital videos are being produced and broadcast every day,
leading to giant media archives. Effective techniques are needed to make such
data accessible further. Automatic meta-data labelling of broadcast media is an
essential task for multimedia indexing, where it is standard to use multi-modal
input for such purposes. This paper describes a novel method for automatic
detection of media genre and show identities using acoustic features, textual
features or a combination thereof. Furthermore the inclusion of available
meta-data, such as time of broadcast, is shown to lead to very high
performance. Latent Dirichlet Allocation is used to model both acoustics and
text, yielding fixed dimensional representations of media recordings that can
then be used in Support Vector Machines based classification. Experiments are
conducted on more than 1200 hours of TV broadcasts from the British
Broadcasting Corporation (BBC), where the task is to categorise the broadcasts
into 8 genres or 133 show identities. On a 200-hour test set, accuracies of
98.6% and 85.7% were achieved for genre and show identification respectively,
using a combination of acoustic and textual features with meta-data.
A Lower Bound Analysis of Population-based Evolutionary Algorithms for Pseudo-Boolean Functions
Authors: Chao Qian, Yang Yu, Zhi-Hua Zhou
Date: 2016-06-10 00:00:00
Tasks: Evolutionary Algorithms

Evolutionary algorithms (EAs) are population-based general-purpose
optimization algorithms, and have been successfully applied in various
real-world optimization tasks. However, previous theoretical studies often
employ EAs with only a parent or offspring population and focus on specific
problems. Furthermore, they often only show upper bounds on the running time,
while lower bounds are also necessary to get a complete understanding of an
algorithm. In this paper, we analyze the running time of the
($\mu$+$\lambda$)-EA (a general population-based EA with mutation only) on the
class of pseudo-Boolean functions with a unique global optimum. By applying the
recently proposed switch analysis approach, we prove the lower bound $\Omega(n
\ln n+ \mu + \lambda n\ln\ln n/ \ln n)$ for the first time. Particularly on the
two widely-studied problems, OneMax and LeadingOnes, the derived lower bound
discloses that the ($\mu$+$\lambda$)-EA will be strictly slower than the
(1+1)-EA when the population size $\mu$ or $\lambda$ is above a moderate order.
Our results imply that the increase of population size, while usually desired
in practice, bears the risk of increasing the lower bound of the running time
and thus should be carefully considered.
Arbitrage-Free Combinatorial Market Making via Integer Programming
Authors: Christian Kroer, Miroslav Dudík, Sébastien Lahaie, Sivaraman Balakrishnan
Date: 2016-06-09 00:00:00
Tasks: Prediction

We present a new combinatorial market maker that operates arbitrage-free
combinatorial prediction markets specified by integer programs. Although the
problem of arbitrage-free pricing, while maintaining a bound on the subsidy
provided by the market maker, is #P-hard in the worst case, we posit that the
typical case might be amenable to modern integer programming (IP) solvers. At
the crux of our method is the Frank-Wolfe (conditional gradient) algorithm
which is used to implement a Bregman projection aligned with the market maker's
cost function, using an IP solver as an oracle. We demonstrate the tractability
and improved accuracy of our approach on real-world prediction market data from
combinatorial bets placed on the 2010 NCAA Men's Division I Basketball
Tournament, where the outcome space is of size 2^63. To our knowledge, this is
the first implementation and empirical evaluation of an arbitrage-free
combinatorial prediction market on this scale.
Gaussian Processes for Music Audio Modelling and Content Analysis
Authors: Pablo A. Alvarado, Dan Stowell
Date: 2016-06-03 00:00:00
Tasks: Beat Tracking, Gaussian Processes, Music Transcription

Real music signals are highly variable, yet they have strong statistical
structure. Prior information about the underlying physical mechanisms by which
sounds are generated and rules by which complex sound structure is constructed
(notes, chords, a complete musical score), can be naturally unified using
Bayesian modelling techniques. Typically algorithms for Automatic Music
Transcription independently carry out individual tasks such as multiple-F0
detection and beat tracking. The challenge remains to perform joint estimation
of all parameters. We present a Bayesian approach for modelling music audio,
and content analysis. The proposed methodology based on Gaussian processes
seeks joint estimation of multiple music concepts by incorporating into the
kernel prior information about non-stationary behaviour, dynamics, and rich
spectral content present in the modelled music signal. We illustrate the
benefits of this approach via two tasks: pitch estimation, and inferring
missing segments in a polyphonic audio recording.
Structured Factored Inference: A Framework for Automated Reasoning in Probabilistic Programming Languages
Authors: Avi Pfeffer, Brian Ruttenberg, William Kretschmer
Date: 2016-06-10 00:00:00
Tasks: Probabilistic Programming

Reasoning on large and complex real-world models is a computationally
difficult task, yet one that is required for effective use of many AI
applications. A plethora of inference algorithms have been developed that work
well on specific models or only on parts of general models. Consequently, a
system that can intelligently apply these inference algorithms to different
parts of a model for fast reasoning is highly desirable. We introduce a new
framework called structured factored inference (SFI) that provides the
foundation for such a system. Using models encoded in a probabilistic
programming language, SFI provides a sound means to decompose a model into
sub-models, apply an inference algorithm to each sub-model, and combine the
resulting information to answer a query. Our results show that SFI is nearly as
accurate as exact inference yet retains the benefits of approximate inference
methods.
Tunable Online MUS/MSS Enumeration
Authors: Jaroslav Bendik, Nikola Benes, Ivana Cerna, Jiri Barnat
Date: 2016-06-10 00:00:00

In various areas of computer science, the problem of dealing with a set of
constraints arises. If the set of constraints is unsatisfiable, one may ask for
a minimal description of the reason for this unsatisifi- ability. Minimal
unsatisifable subsets (MUSes) and maximal satisifiable subsets (MSSes) are two
kinds of such minimal descriptions. The goal of this work is the enumeration of
MUSes and MSSes for a given constraint system. As such full enumeration may be
intractable in general, we focus on building an online algorithm, which
produces MUSes/MSSes in an on-the-fly manner as soon as they are discovered.
The problem has been studied before even in its online version. However, our
algorithm uses a novel approach that is able to outperform current state-of-the
art algorithms for online MUS/MSS enumeration. Moreover, the performance of our
algorithm can be adjusted using tunable parameters. We evaluate the algorithm
on a set of benchmarks.
Predict or classify: The deceptive role of time-locking in brain signal classification
Authors: Marco Rusconi, Angelo Valleriani
Date: 2016-05-26 00:00:00
Tasks: General Classification

Several experimental studies claim to be able to predict the outcome of
simple decisions from brain signals measured before subjects are aware of their
decision. Often, these studies use multivariate pattern recognition methods
with the underlying assumption that the ability to classify the brain signal is
equivalent to predict the decision itself. Here we show instead that it is
possible to correctly classify a signal even if it does not contain any
predictive information about the decision. We first define a simple stochastic
model that mimics the random decision process between two equivalent
alternatives, and generate a large number of independent trials that contain no
choice-predictive information. The trials are first time-locked to the time
point of the final event and then classified using standard machine-learning
techniques. The resulting classification accuracy is above chance level long
before the time point of time-locking. We then analyze the same trials using
information theory. We demonstrate that the high classification accuracy is a
consequence of time-locking and that its time behavior is simply related to the
large relaxation time of the process. We conclude that when time-locking is a
crucial step in the analysis of neural activity patterns, both the emergence
and the timing of the classification accuracy are affected by structural
properties of the network that generates the signal.
A Scheme for Molecular Computation of Maximum Likelihood Estimators for Log-Linear Models
Authors: Manoj Gopalkrishnan
Date: 2015-06-10 00:00:00

We propose a novel molecular computing scheme for statistical inference. We
focus on the much-studied statistical inference problem of computing maximum
likelihood estimators for log-linear models. Our scheme takes log-linear models
to reaction systems, and the observed data to initial conditions, so that the
corresponding equilibrium of each reaction system encodes the corresponding
maximum likelihood estimator. The main idea is to exploit the coincidence
between thermodynamic entropy and statistical entropy. We map a Maximum Entropy
characterization of the maximum likelihood estimator onto a Maximum Entropy
characterization of the equilibrium concentrations for the reaction system.
This allows for an efficient encoding of the problem, and reveals that reaction
networks are superbly suited to statistical inference tasks. Such a scheme may
also provide a template to understanding how in vivo biochemical signaling
pathways integrate extensive information about their environment and history.
Reducing Overfitting in Deep Networks by Decorrelating Representations
Authors: Michael Cogswell, Faruk Ahmed, Ross Girshick, Larry Zitnick, Dhruv Batra
Date: 2015-11-19 00:00:00
Tasks: Data Augmentation

One major challenge in training Deep Neural Networks is preventing
overfitting. Many techniques such as data augmentation and novel regularizers
such as Dropout have been proposed to prevent overfitting without requiring a
massive amount of training data. In this work, we propose a new regularizer
called DeCov which leads to significantly reduced overfitting (as indicated by
the difference between train and val performance), and better generalization.
Our regularizer encourages diverse or non-redundant representations in Deep
Neural Networks by minimizing the cross-covariance of hidden activations. This
simple intuition has been explored in a number of past works but surprisingly
has never been applied as a regularizer in supervised learning. Experiments
across a range of datasets and network architectures show that this loss always
reduces overfitting while almost always maintaining or increasing
generalization performance and often improving performance over Dropout.
Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications
Authors: Ciprian Corneanu, Marc Oliu, Jeffrey F. Cohn, Sergio Escalera
Date: 2016-06-10 00:00:00
Tasks: Face Detection, Facial Expression Recognition, Facial Expression Recognition (FER)

Facial expressions are an important way through which humans interact
socially. Building a system capable of automatically recognizing facial
expressions from images and video has been an intense field of study in recent
years. Interpreting such expressions remains challenging and much research is
needed about the way they relate to human affect. This paper presents a general
overview of automatic RGB, 3D, thermal and multimodal facial expression
analysis. We define a new taxonomy for the field, encompassing all steps from
face detection to facial expression recognition, and describe and classify the
state of the art methods accordingly. We also present the important datasets
and the bench-marking of most influential methods. We conclude with a general
discussion about trends, important questions and future lines of research.
Towards Anthropo-inspired Computational Systems: the $P^3$ Model
Authors: Michael W. Bridges, Salvatore Distefano, Manuel Mazzara, Marat Minlebaev, Max Talanov, Jordi Vallverdú
Date: 2016-06-10 00:00:00
Tasks: Distributed Computing, Philosophy

This paper proposes a model which aim is providing a more coherent framework
for agents design. We identify three closely related anthropo-centered domains
working on separate functional levels. Abstracting from human physiology,
psychology, and philosophy we create the $P^3$ model to be used as a multi-tier
approach to deal with complex class of problems. The three layers identified in
this model have been named PhysioComputing, MindComputing, and MetaComputing.
Several instantiations of this model are finally presented related to different
IT areas such as artificial intelligence, distributed computing, software and
service engineering.