pwc-archive/papers-with-abstracts huggingface dataset:
A robust autoassociative memory with coupled networks of Kuramoto-type oscillators
Authors: Daniel Heger, Katharina Krischer
Date: 2016-04-07 00:00:00
Tasks: Vocal Bursts Type Prediction

Uncertain recognition success, unfavorable scaling of connection complexity
or dependence on complex external input impair the usefulness of current
oscillatory neural networks for pattern recognition or restrict technical
realizations to small networks. We propose a new network architecture of
coupled oscillators for pattern recognition which shows none of the mentioned
aws. Furthermore we illustrate the recognition process with simulation results
and analyze the new dynamics analytically: Possible output patterns are
isolated attractors of the system. Additionally, simple criteria for
recognition success are derived from a lower bound on the basins of attraction.
Teaching natural language to computers
Authors: Joseph Corneli, Miriam Corneli
Date: 2016-04-29 00:00:00

"Natural Language," whether spoken and attended to by humans, or processed
and generated by computers, requires networked structures that reflect creative
processes in semantic, syntactic, phonetic, linguistic, social, emotional, and
cultural modules. Being able to produce novel and useful behavior following
repeated practice gets to the root of both artificial intelligence and human
language. This paper investigates the modalities involved in language-like
applications that computers -- and programmers -- engage with, and aims to fine
tune the questions we ask to better account for context, self-awareness, and
embodiment.
Bootstrap-Based Regularization for Low-Rank Matrix Estimation
Authors: Julie Josse, Stefan Wager
Date: 2014-10-30 00:00:00

We develop a flexible framework for low-rank matrix estimation that allows us
to transform noise models into regularization schemes via a simple bootstrap
algorithm. Effectively, our procedure seeks an autoencoding basis for the
observed matrix that is stable with respect to the specified noise model; we
call the resulting procedure a stable autoencoder. In the simplest case, with
an isotropic noise model, our method is equivalent to a classical singular
value shrinkage estimator. For non-isotropic noise models, e.g., Poisson noise,
the method does not reduce to singular value shrinkage, and instead yields new
estimators that perform well in experiments. Moreover, by iterating our stable
autoencoding scheme, we can automatically generate low-rank estimates without
specifying the target rank as a tuning parameter.
Automatic Variational ABC
Authors: Alexander Moreno, Tameem Adel, Edward Meeds, James M. Rehg, Max Welling
Date: 2016-06-28 00:00:00
Tasks: Variational Inference

Approximate Bayesian Computation (ABC) is a framework for performing
likelihood-free posterior inference for simulation models. Stochastic
Variational inference (SVI) is an appealing alternative to the inefficient
sampling approaches commonly used in ABC. However, SVI is highly sensitive to
the variance of the gradient estimators, and this problem is exacerbated by
approximating the likelihood. We draw upon recent advances in variance
reduction for SV and likelihood-free inference using deterministic simulations
to produce low variance gradient estimators of the variational lower-bound. By
then exploiting automatic differentiation libraries we can avoid nearly all
model-specific derivations. We demonstrate performance on three problems and
compare to existing SVI algorithms. Our results demonstrate the correctness and
efficiency of our algorithm.
A Local Density-Based Approach for Local Outlier Detection
Authors: Bo Tang, Haibo He
Date: 2016-06-28 00:00:00
Tasks: Density Estimation, Object, Outlier Detection

This paper presents a simple but effective density-based outlier detection
approach with the local kernel density estimation (KDE). A Relative
Density-based Outlier Score (RDOS) is introduced to measure the local
outlierness of objects, in which the density distribution at the location of an
object is estimated with a local KDE method based on extended nearest neighbors
of the object. Instead of using only $k$ nearest neighbors, we further consider
reverse nearest neighbors and shared nearest neighbors of an object for density
distribution estimation. Some theoretical properties of the proposed RDOS
including its expected value and false alarm probability are derived. A
comprehensive experimental study on both synthetic and real-life data sets
demonstrates that our approach is more effective than state-of-the-art outlier
detection methods.
A Learning Algorithm for Relational Logistic Regression: Preliminary Results
Authors: Bahare Fatemi, Seyed Mehran Kazemi, David Poole
Date: 2016-06-28 00:00:00
Tasks: regression, Relational Reasoning

Relational logistic regression (RLR) is a representation of conditional
probability in terms of weighted formulae for modelling multi-relational data.
In this paper, we develop a learning algorithm for RLR models. Learning an RLR
model from data consists of two steps: 1- learning the set of formulae to be
used in the model (a.k.a. structure learning) and learning the weight of each
formula (a.k.a. parameter learning). For structure learning, we deploy Schmidt
and Murphy's hierarchical assumption: first we learn a model with simple
formulae, then more complex formulae are added iteratively only if all their
sub-formulae have proven effective in previous learned models. For parameter
learning, we convert the problem into a non-relational learning problem and use
an off-the-shelf logistic regression learning algorithm from Weka, an
open-source machine learning tool, to learn the weights. We also indicate how
hidden features about the individuals can be incorporated into RLR to boost the
learning performance. We compare our learning algorithm to other structure and
parameter learning algorithms in the literature, and compare the performance of
RLR models to standard logistic regression and RDN-Boost on a modified version
of the MovieLens data-set.
Feedforward Initialization for Fast Inference of Deep Generative Networks is biologically plausible
Authors: Yoshua Bengio, Benjamin Scellier, Olexa Bilaniuk, Joao Sacramento, Walter Senn
Date: 2016-06-06 00:00:00

We consider deep multi-layered generative models such as Boltzmann machines
or Hopfield nets in which computation (which implements inference) is both
recurrent and stochastic, but where the recurrence is not to model sequential
structure, only to perform computation. We find conditions under which a simple
feedforward computation is a very good initialization for inference, after the
input units are clamped to observed values. It means that after the feedforward
initialization, the recurrent network is very close to a fixed point of the
network dynamics, where the energy gradient is 0. The main condition is that
consecutive layers form a good auto-encoder, or more generally that different
groups of inputs into the unit (in particular, bottom-up inputs on one hand,
top-down inputs on the other hand) are consistent with each other, producing
the same contribution into the total weighted sum of inputs. In biological
terms, this would correspond to having each dendritic branch correctly
predicting the aggregate input from all the dendritic branches, i.e., the soma
potential. This is consistent with the prediction that the synaptic weights
into dendritic branches such as those of the apical and basal dendrites of
pyramidal cells are trained to minimize the prediction error made by the
dendritic branch when the target is the somatic activity. Whereas previous work
has shown how to achieve fast negative phase inference (when the model is
unclamped) in a predictive recurrent model, this contribution helps to achieve
fast positive phase inference (when the target output is clamped) in such
recurrent neural models.
Linguistic Input Features Improve Neural Machine Translation
Authors: Rico Sennrich, Barry Haddow
Date: 2016-06-09 00:00:00
Proceeding: WS 2016 8
Tasks: Decoder, Machine Translation, Translation

Neural machine translation has recently achieved impressive results, while
using little in the way of external linguistic information. In this paper we
show that the strong learning capability of neural MT models does not make
linguistic features redundant; they can be easily incorporated to provide
further improvements in performance. We generalize the embedding layer of the
encoder in the attentional encoder--decoder architecture to support the
inclusion of arbitrary features, in addition to the baseline word feature. We
add morphological features, part-of-speech tags, and syntactic dependency
labels as input features to English<->German, and English->Romanian neural
machine translation systems. In experiments on WMT16 training and test sets, we
find that linguistic input features improve model quality according to three
metrics: perplexity, BLEU and CHRF3. An open-source implementation of our
neural MT system is available, as are sample files and configurations.
Edinburgh Neural Machine Translation Systems for WMT 16
Authors: Rico Sennrich, Barry Haddow, Alexandra Birch
Date: 2016-06-09 00:00:00
Proceeding: WS 2016 8
Tasks: Decoder, Machine Translation, Translation

We participated in the WMT 2016 shared news translation task by building
neural translation systems for four language pairs, each trained in both
directions: English<->Czech, English<->German, English<->Romanian and
English<->Russian. Our systems are based on an attentional encoder-decoder,
using BPE subword segmentation for open-vocabulary translation with a fixed
vocabulary. We experimented with using automatic back-translations of the
monolingual News corpus as additional training data, pervasive dropout, and
target-bidirectional models. All reported methods give substantial
improvements, and we see improvements of 4.3--11.2 BLEU over our baseline
systems. In the human evaluation, our systems were the (tied) best constrained
system for 7 out of 8 translation directions in which we participated.
Network-Efficient Distributed Word2vec Training System for Large Vocabularies
Authors: Erik Ordentlich, Lee Yang, Andy Feng, Peter Cnudde, Mihajlo Grbovic, Nemanja Djuric, Vladan Radosavljevic, Gavin Owens
Date: 2016-06-27 00:00:00

Word2vec is a popular family of algorithms for unsupervised training of dense
vector representations of words on large text corpuses. The resulting vectors
have been shown to capture semantic relationships among their corresponding
words, and have shown promise in reducing a number of natural language
processing (NLP) tasks to mathematical operations on these vectors. While
heretofore applications of word2vec have centered around vocabularies with a
few million words, wherein the vocabulary is the set of words for which vectors
are simultaneously trained, novel applications are emerging in areas outside of
NLP with vocabularies comprising several 100 million words. Existing word2vec
training systems are impractical for training such large vocabularies as they
either require that the vectors of all vocabulary words be stored in the memory
of a single server or suffer unacceptable training latency due to massive
network data transfer. In this paper, we present a novel distributed, parallel
training system that enables unprecedented practical training of vectors for
vocabularies with several 100 million words on a shared cluster of commodity
servers, using far less network traffic than the existing solutions. We
evaluate the proposed system on a benchmark dataset, showing that the quality
of vectors does not degrade relative to non-distributed training. Finally, for
several quarters, the system has been deployed for the purpose of matching
queries to ads in Gemini, the sponsored search advertising platform at Yahoo,
resulting in significant improvement of business metrics.
A Variational Approximations-DIC Rubric for Parameter Estimation and Mixture Model Selection Within a Family Setting
Authors: Sanjeena Subedi, Paul D. McNicholas
Date: 2013-06-23 00:00:00
Tasks: Clustering, Model Selection, parameter estimation

Mixture model-based clustering has become an increasingly popular data analysis technique since its introduction over fifty years ago, and is now commonly utilized within a family setting. Families of mixture models arise when the component parameters, usually the component covariance (or scale) matrices, are decomposed and a number of constraints are imposed. Within the family setting, model selection involves choosing the member of the family, i.e., the appropriate covariance structure, in addition to the number of mixture components. To date, the Bayesian information criterion (BIC) has proved most effective for model selection, and the expectation-maximization (EM) algorithm is usually used for parameter estimation. In fact, this EM-BIC rubric has virtually monopolized the literature on families of mixture models. Deviating from this rubric, variational Bayes approximations are developed for parameter estimation and the deviance information criterion for model selection. The variational Bayes approach provides an alternate framework for parameter estimation by constructing a tight lower bound on the complex marginal likelihood and maximizing this lower bound by minimizing the associated Kullback-Leibler divergence. This approach is taken on the most commonly used family of Gaussian mixture models, and real and simulated data are used to compare the new approach to the EM-BIC rubric.
Dynamic Hierarchical Dirichlet Process for Abnormal Behaviour Detection in Video
Authors: Olga Isupova, Danil Kuzin, Lyudmila Mihaylova
Date: 2016-06-27 00:00:00
Tasks: Decision Making

This paper proposes a novel dynamic Hierarchical Dirichlet Process topic
model that considers the dependence between successive observations.
Conventional posterior inference algorithms for this kind of models require
processing of the whole data through several passes. It is computationally
intractable for massive or sequential data. We design the batch and online
inference algorithms, based on the Gibbs sampling, for the proposed model. It
allows to process sequential data, incrementally updating the model by a new
observation. The model is applied to abnormal behaviour detection in video
sequences. A new abnormality measure is proposed for decision making. The
proposed method is compared with the method based on the non- dynamic
Hierarchical Dirichlet Process, for which we also derive the online Gibbs
sampler and the abnormality measure. The results with synthetic and real data
show that the consideration of the dynamics in a topic model improves the
classification performance for abnormal behaviour detection.
Anomaly detection in video with Bayesian nonparametrics
Authors: Olga Isupova, Danil Kuzin, Lyudmila Mihaylova
Date: 2016-06-27 00:00:00
Tasks: Anomaly Detection, Decision Making, General Classification

A novel dynamic Bayesian nonparametric topic model for anomaly detection in
video is proposed in this paper. Batch and online Gibbs samplers are developed
for inference. The paper introduces a new abnormality measure for decision
making. The proposed method is evaluated on both synthetic and real data. The
comparison with a non-dynamic model shows the superiority of the proposed
dynamic one in terms of the classification performance for anomaly detection.
Exploiting the Short-term to Long-term Plasticity Transition in Memristive Nanodevice Learning Architectures
Authors: Christopher H. Bennett, Selina La Barbera, Adrien F. Vincent, Fabien Alibart, Damien Querlioz
Date: 2016-06-27 00:00:00
Tasks: Classification, General Classification

Memristive nanodevices offer new frontiers for computing systems that unite
arithmetic and memory operations on-chip. Here, we explore the integration of
electrochemical metallization cell (ECM) nanodevices with tunable filamentary
switching in nanoscale learning systems. Such devices offer a natural
transition between short-term plasticity (STP) and long-term plasticity (LTP).
In this work, we show that this property can be exploited to efficiently solve
noisy classification tasks. A single crossbar learning scheme is first
introduced and evaluated. Perfect classification is possible only for simple
input patterns, within critical timing parameters, and when device variability
is weak. To overcome these limitations, a dual-crossbar learning system partly
inspired by the extreme learning machine (ELM) approach is then introduced.
This approach outperforms a conventional ELM-inspired system when the first
layer is imprinted before training and testing, and especially so when
variability in device timing evolution is considered: variability is therefore
transformed from an issue to a feature. In attempting to classify the MNIST
database under the same conditions, conventional ELM obtains 84%
classification, the imprinted, uniform device system obtains 88%
classification, and the imprinted, variable device system reaches 92%
classification. We discuss benefits and drawbacks of both systems in terms of
energy, complexity, area imprint, and speed. All these results highlight that
tuning and exploiting intrinsic device timing parameters may be of central
interest to future bio-inspired approximate computing systems.
Distributed Submodular Maximization
Authors: Baharan Mirzasoleiman, Amin Karbasi, Rik Sarkar, Andreas Krause
Date: 2014-11-03 00:00:00
Tasks: Clustering

Many large-scale machine learning problems--clustering, non-parametric
learning, kernel machines, etc.--require selecting a small yet representative
subset from a large dataset. Such problems can often be reduced to maximizing a
submodular set function subject to various constraints. Classical approaches to
submodular optimization require centralized access to the full dataset, which
is impractical for truly large-scale problems. In this paper, we consider the
problem of submodular function maximization in a distributed fashion. We
develop a simple, two-stage protocol GreeDi, that is easily implemented using
MapReduce style computations. We theoretically analyze our approach, and show
that under certain natural conditions, performance close to the centralized
approach can be achieved. We begin with monotone submodular maximization
subject to a cardinality constraint, and then extend this approach to obtain
approximation guarantees for (not necessarily monotone) submodular maximization
subject to more general constraints including matroid or knapsack constraints.
In our extensive experiments, we demonstrate the effectiveness of our approach
on several applications, including sparse Gaussian process inference and
exemplar based clustering on tens of millions of examples using Hadoop.
Depth Estimation from Single Image using Sparse Representations
Authors: Yigit Oktar
Date: 2016-06-27 00:00:00
Tasks: Depth Estimation, Monocular Depth Estimation

Monocular depth estimation is an interesting and challenging problem as there
is no analytic mapping known between an intensity image and its depth map.
Recently there has been a lot of data accumulated through depth-sensing
cameras, in parallel to that researchers started to tackle this task using
various learning algorithms. In this paper, a deep sparse coding method is
proposed for monocular depth estimation along with an approach for
deterministic dictionary initialization.
Interpreting extracted rules from ensemble of trees: Application to computer-aided diagnosis of breast MRI
Authors: Cristina Gallego-Ortiz, Anne L. Martel
Date: 2016-06-27 00:00:00
Tasks: General Classification, Lesion Classification

High predictive performance and ease of use and interpretability are
important requirements for the applicability of a computer-aided diagnosis
(CAD) to human reading studies. We propose a CAD system specifically designed
to be more comprehensible to the radiologist reviewing screening breast MRI
studies. Multiparametric imaging features are combined to produce a CAD system
for differentiating cancerous and non-cancerous lesions. The complete system
uses a rule-extraction algorithm to present lesion classification results in an
easy to understand graph visualization.
Evaluating Informal-Domain Word Representations With UrbanDictionary
Authors: Naomi Saphra, Adam Lopez
Date: 2016-06-27 00:00:00
Proceeding: WS 2016 8
Tasks: Text Normalization

Existing corpora for intrinsic evaluation are not targeted towards tasks in
informal domains such as Twitter or news comment forums. We want to test
whether a representation of informal words fulfills the promise of eliding
explicit text normalization as a preprocessing step. One possible evaluation
metric for such domains is the proximity of spelling variants. We propose how
such a metric might be computed and how a spelling variant dataset can be
collected using UrbanDictionary.
Pragmatic factors in image description: the case of negations
Authors: Emiel van Miltenburg, Roser Morante, Desmond Elliott
Date: 2016-06-20 00:00:00
Proceeding: WS 2016 8
Tasks: Image Description, Negation

We provide a qualitative analysis of the descriptions containing negations
(no, not, n't, nobody, etc) in the Flickr30K corpus, and a categorization of
negation uses. Based on this analysis, we provide a set of requirements that an
image description system should have in order to generate negation sentences.
As a pilot experiment, we used our categorization to manually annotate
sentences containing negations in the Flickr30K corpus, with an agreement score
of K=0.67. With this paper, we hope to open up a broader discussion of
subjective language in image descriptions.
Two Discourse Driven Language Models for Semantics
Authors: Haoruo Peng, Dan Roth
Date: 2016-06-17 00:00:00
Proceeding: ACL 2016 8
Tasks: Cloze Test, Discourse Parsing, Language Modeling, Language Modelling, Natural Language Understanding, Vocal Bursts Valence Prediction

Natural language understanding often requires deep semantic knowledge.
Expanding on previous proposals, we suggest that some important aspects of
semantic knowledge can be modeled as a language model if done at an appropriate
level of abstraction. We develop two distinct models that capture semantic
frame chains and discourse information while abstracting over the specific
mentions of predicates and entities. For each model, we investigate four
implementations: a "standard" N-gram language model and three discriminatively
trained "neural" language models that generate embeddings for semantic frames.
The quality of the semantic language models (SemLM) is evaluated both
intrinsically, using perplexity and a narrative cloze test and extrinsically -
we show that our SemLM helps improve performance on semantic natural language
processing tasks such as co-reference resolution and discourse parsing.