pwc-archive/papers-with-abstracts huggingface dataset:
The Lifted Matrix-Space Model for Semantic Composition
Authors: WooJin Chung, Sheng-Fu Wang, Samuel R. Bowman
Date: 2017-11-09 00:00:00
Proceeding: CONLL 2018 10
Tasks: Semantic Composition, Sentence, Word Embeddings

Tree-structured neural network architectures for sentence encoding draw
inspiration from the approach to semantic composition generally seen in formal
linguistics, and have shown empirical improvements over comparable sequence
models by doing so. Moreover, adding multiplicative interaction terms to the
composition functions in these models can yield significant further
improvements. However, existing compositional approaches that adopt such a
powerful composition function scale poorly, with parameter counts exploding as
model dimension or vocabulary size grows. We introduce the Lifted Matrix-Space
model, which uses a global transformation to map vector word embeddings to
matrices, which can then be composed via an operation based on matrix-matrix
multiplication. Its composition function effectively transmits a larger number
of activations across layers with relatively few model parameters. We evaluate
our model on the Stanford NLI corpus, the Multi-Genre NLI corpus, and the
Stanford Sentiment Treebank and find that it consistently outperforms TreeLSTM
(Tai et al., 2015), the previous best known composition function for
tree-structured models.
Efficient-UCBV: An Almost Optimal Algorithm using Variance Estimates
Authors: Subhojyoti Mukherjee, K. P. Naveen, Nandan Sudarsanam, Balaraman Ravindran
Date: 2017-11-09 00:00:00
Tasks: Thompson Sampling

We propose a novel variant of the UCB algorithm (referred to as
Efficient-UCB-Variance (EUCBV)) for minimizing cumulative regret in the
stochastic multi-armed bandit (MAB) setting. EUCBV incorporates the arm
elimination strategy proposed in UCB-Improved \citep{auer2010ucb}, while taking
into account the variance estimates to compute the arms' confidence bounds,
similar to UCBV \citep{audibert2009exploration}. Through a theoretical analysis
we establish that EUCBV incurs a \emph{gap-dependent} regret bound of
{\scriptsize $O\left( \dfrac{K\sigma^2_{\max} \log (T\Delta^2
/K)}{\Delta}\right)$} after $T$ trials, where $\Delta$ is the minimal gap
between optimal and sub-optimal arms; the above bound is an improvement over
that of existing state-of-the-art UCB algorithms (such as UCB1, UCB-Improved,
UCBV, MOSS). Further, EUCBV incurs a \emph{gap-independent} regret bound of
{\scriptsize $O\left(\sqrt{KT}\right)$} which is an improvement over that of
UCB1, UCBV and UCB-Improved, while being comparable with that of MOSS and
OCUCB. Through an extensive numerical study we show that EUCBV significantly
outperforms the popular UCB variants (like MOSS, OCUCB, etc.) as well as
Thompson sampling and Bayes-UCB algorithms.
A Note on a Tight Lower Bound for MNL-Bandit Assortment Selection Models
Authors: Xi Chen, Yining Wang
Date: 2017-09-18 00:00:00

In this short note we consider a dynamic assortment planning problem under
the capacitated multinomial logit (MNL) bandit model. We prove a tight lower
bound on the accumulated regret that matches existing regret upper bounds for
all parameters (time horizon $T$, number of items $N$ and maximum assortment
capacity $K$) up to logarithmic factors. Our results close an $O(\sqrt{K})$ gap
between upper and lower regret bounds from existing works.
SHOPPER: A Probabilistic Model of Consumer Choice with Substitutes and Complements
Authors: Francisco J. R. Ruiz, Susan Athey, David M. Blei
Date: 2017-11-09 00:00:00
Tasks: counterfactual

We develop SHOPPER, a sequential probabilistic model of shopping data. SHOPPER uses interpretable components to model the forces that drive how a customer chooses products; in particular, we designed SHOPPER to capture how items interact with other items. We develop an efficient posterior inference algorithm to estimate these forces from large-scale data, and we analyze a large dataset from a major chain grocery store. We are interested in answering counterfactual queries about changes in prices. We found that SHOPPER provides accurate predictions even under price interventions, and that it helps identify complementary and substitutable pairs of products.
Toward Depth Estimation Using Mask-Based Lensless Cameras
Authors: M. Salman Asif
Date: 2017-11-09 00:00:00
Tasks: Depth Estimation

Recently, coded masks have been used to demonstrate a thin form-factor
lensless camera, FlatCam, in which a mask is placed immediately on top of a
bare image sensor. In this paper, we present an imaging model and algorithm to
jointly estimate depth and intensity information in the scene from a single or
multiple FlatCams. We use a light field representation to model the mapping of
3D scene onto the sensor in which light rays from different depths yield
different modulation patterns. We present a greedy depth pursuit algorithm to
search the 3D volume and estimate the depth and intensity of each pixel within
the camera field-of-view. We present simulation results to analyze the
performance of our proposed model and algorithm with different FlatCam
settings.
Deep Neural Networks to Enable Real-time Multimessenger Astrophysics
Authors: Daniel George, E. A. Huerta
Date: 2016-12-30 00:00:00
Tasks: Astronomy, parameter estimation, Time Series, Time Series Analysis, Transfer Learning

Gravitational wave astronomy has set in motion a scientific revolution. To
further enhance the science reach of this emergent field, there is a pressing
need to increase the depth and speed of the gravitational wave algorithms that
have enabled these groundbreaking discoveries. To contribute to this effort, we
introduce Deep Filtering, a new highly scalable method for end-to-end
time-series signal processing, based on a system of two deep convolutional
neural networks, which we designed for classification and regression to rapidly
detect and estimate parameters of signals in highly noisy time-series data
streams. We demonstrate a novel training scheme with gradually increasing noise
levels, and a transfer learning procedure between the two networks. We showcase
the application of this method for the detection and parameter estimation of
gravitational waves from binary black hole mergers. Our results indicate that
Deep Filtering significantly outperforms conventional machine learning
techniques, achieves similar performance compared to matched-filtering while
being several orders of magnitude faster thus allowing real-time processing of
raw big data with minimal resources. More importantly, Deep Filtering extends
the range of gravitational wave signals that can be detected with ground-based
gravitational wave detectors. This framework leverages recent advances in
artificial intelligence algorithms and emerging hardware architectures, such as
deep-learning-optimized GPUs, to facilitate real-time searches of gravitational
wave sources and their electromagnetic and astro-particle counterparts.
Fast Meta-Learning for Adaptive Hierarchical Classifier Design
Authors: Gerrit J. J. van den Burg, Alfred O. Hero
Date: 2017-11-09 00:00:00
Tasks: Binary Classification, Classification, General Classification, Meta-Learning

We propose a new splitting criterion for a meta-learning approach to
multiclass classifier design that adaptively merges the classes into a
tree-structured hierarchy of increasingly difficult binary classification
problems. The classification tree is constructed from empirical estimates of
the Henze-Penrose bounds on the pairwise Bayes misclassification rates that
rank the binary subproblems in terms of difficulty of classification. The
proposed empirical estimates of the Bayes error rate are computed from the
minimal spanning tree (MST) of the samples from each pair of classes. Moreover,
a meta-learning technique is presented for quantifying the one-vs-rest Bayes
error rate for each individual class from a single MST on the entire dataset.
Extensive simulations on benchmark datasets show that the proposed hierarchical
method can often be learned much faster than competing methods, while achieving
competitive accuracy.
Machine Learning Friendly Set Version of Johnson-Lindenstrauss Lemma
Authors: Mieczysław A. Kłopotek
Date: 2017-03-04 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, LEMMA

In this paper we make a novel use of the Johnson-Lindenstrauss Lemma. The
Lemma has an existential form saying that there exists a JL transformation $f$
of the data points into lower dimensional space such that all of them fall into
predefined error range $\delta$.
  We formulate in this paper a theorem stating that we can choose the target
dimensionality in a random projection type JL linear transformation in such a
way that with probability $1-\epsilon$ all of them fall into predefined error
range $\delta$ for any user-predefined failure probability $\epsilon$.
  This result is important for applications such a data clustering where we
want to have a priori dimensionality reducing transformation instead of trying
out a (large) number of them, as with traditional Johnson-Lindenstrauss Lemma.
In particular, we take a closer look at the $k$-means algorithm and prove that
a good solution in the projected space is also a good solution in the original
space. Furthermore, under proper assumptions local optima in the original space
are also ones in the projected space. We define also conditions for which
clusterability property of the original space is transmitted to the projected
space, so that special case algorithms for the original space are also
applicable in the projected space.
Learning Multi-Modal Word Representation Grounded in Visual Context
Authors: Éloi Zablocki, Benjamin Piwowarski, Laure Soulier, Patrick Gallinari
Date: 2017-11-09 00:00:00
Tasks: Word Embeddings

Representing the semantics of words is a long-standing problem for the
natural language processing community. Most methods compute word semantics
given their textual context in large corpora. More recently, researchers
attempted to integrate perceptual and visual features. Most of these works
consider the visual appearance of objects to enhance word representations but
they ignore the visual environment and context in which objects appear. We
propose to unify text-based techniques with vision-based techniques by
simultaneously leveraging textual and visual context to learn multimodal word
embeddings. We explore various choices for what can serve as a visual context
and present an end-to-end method to integrate visual context elements in a
multimodal skip-gram model. We provide experiments and extensive analysis of
the obtained results.
Scalable Log Determinants for Gaussian Process Kernel Learning
Authors: Kun Dong, David Eriksson, Hannes Nickisch, David Bindel, Andrew Gordon Wilson
Date: 2017-11-09 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Gaussian Processes, Point Processes

For applications as varied as Bayesian neural networks, determinantal point
processes, elliptical graphical models, and kernel learning for Gaussian
processes (GPs), one must compute a log determinant of an $n \times n$ positive
definite matrix, and its derivatives - leading to prohibitive
$\mathcal{O}(n^3)$ computations. We propose novel $\mathcal{O}(n)$ approaches
to estimating these quantities from only fast matrix vector multiplications
(MVMs). These stochastic approximations are based on Chebyshev, Lanczos, and
surrogate models, and converge quickly even for kernel matrices that have
challenging spectra. We leverage these approximations to develop a scalable
Gaussian process approach to kernel learning. We find that Lanczos is generally
superior to Chebyshev for kernel learning, and that a surrogate approach can be
highly efficient and accurate with popular kernels.
Worm-level Control through Search-based Reinforcement Learning
Authors: Mathias Lechner, Radu Grosu, Ramin M. Hasani
Date: 2017-11-09 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Through natural evolution, nervous systems of organisms formed near-optimal
structures to express behavior. Here, we propose an effective way to create
control agents, by \textit{re-purposing} the function of biological neural
circuit models, to govern similar real world applications. We model the
tap-withdrawal (TW) neural circuit of the nematode, \textit{C. elegans}, a
circuit responsible for the worm's reflexive response to external mechanical
touch stimulations, and learn its synaptic and neural parameters as a policy
for controlling the inverted pendulum problem. For reconfiguration of the
purpose of the TW neural circuit, we manipulate a search-based reinforcement
learning. We show that our neural policy performs as good as existing
traditional control theory and machine learning approaches. A video
demonstration of the performance of our method can be accessed at
\url{https://youtu.be/o-Ia5IVyff8}.
Regret Minimization in Behaviorally-Constrained Zero-Sum Games
Authors: Gabriele Farina, Christian Kroer, Tuomas Sandholm
Date: 2017-11-09 00:00:00
Proceeding: ICML 2017 8
Tasks: counterfactual

No-regret learning has emerged as a powerful tool for solving extensive-form
games. This was facilitated by the counterfactual-regret minimization (CFR)
framework, which relies on the instantiation of regret minimizers for simplexes
at each information set of the game. We use an instantiation of the CFR
framework to develop algorithms for solving behaviorally-constrained (and, as a
special case, perturbed in the Selten sense) extensive-form games, which allows
us to compute approximate Nash equilibrium refinements. Nash equilibrium
refinements are motivated by a major deficiency in Nash equilibrium: it
provides virtually no guarantees on how it will play in parts of the game tree
that are reached with zero probability. Refinements can mend this issue, but
have not been adopted in practice, mostly due to a lack of scalable algorithms.
We show that, compared to standard algorithms, our method finds solutions that
have substantially better refinement properties, while enjoying a convergence
rate that is comparable to that of state-of-the-art algorithms for Nash
equilibrium computation both in theory and practice.
Smooth Primal-Dual Coordinate Descent Algorithms for Nonsmooth Convex Optimization
Authors: Ahmet Alacaoglu, Quoc Tran-Dinh, Olivier Fercoq, Volkan Cevher
Date: 2017-11-09 00:00:00
Proceeding: NeurIPS 2017 12

We propose a new randomized coordinate descent method for a convex
optimization template with broad applications. Our analysis relies on a novel
combination of four ideas applied to the primal-dual gap function: smoothing,
acceleration, homotopy, and coordinate descent with non-uniform sampling. As a
result, our method features the first convergence rate guarantees among the
coordinate descent methods, that are the best-known under a variety of common
structure assumptions on the template. We provide numerical evidence to support
the theoretical results with a comparison to state-of-the-art algorithms.
Open-World Knowledge Graph Completion
Authors: Baoxu Shi, Tim Weninger
Date: 2017-11-09 00:00:00
Tasks: Entity Linking, Knowledge Graph Completion, Knowledge Graphs, Link Prediction, World Knowledge

Knowledge Graphs (KGs) have been applied to many tasks including Web search,
link prediction, recommendation, natural language processing, and entity
linking. However, most KGs are far from complete and are growing at a rapid
pace. To address these problems, Knowledge Graph Completion (KGC) has been
proposed to improve KGs by filling in its missing connections. Unlike existing
methods which hold a closed-world assumption, i.e., where KGs are fixed and new
entities cannot be easily added, in the present work we relax this assumption
and propose a new open-world KGC task. As a first attempt to solve this task we
introduce an open-world KGC model called ConMask. This model learns embeddings
of the entity's name and parts of its text-description to connect unseen
entities to the KG. To mitigate the presence of noisy text descriptions,
ConMask uses a relationship-dependent content masking to extract relevant
snippets and then trains a fully convolutional neural network to fuse the
extracted snippets with entities in the KG. Experiments on large data sets,
both old and new, show that ConMask performs well in the open-world KGC task
and even outperforms existing KGC models on the standard closed-world KGC task.
Material Classification in the Wild: Do Synthesized Training Data Generalise Better than Real-World Training Data?
Authors: Grigorios Kalliatakis, Anca Sticlaru, George Stamatiadis, Shoaib Ehsan, Ales Leonardis, Juergen Gall, Klaus D. McDonald-Maier
Date: 2017-11-09 00:00:00
Tasks: General Classification, Material Classification

We question the dominant role of real-world training images in the field of
material classification by investigating whether synthesized data can
generalise more effectively than real-world data. Experimental results on three
challenging real-world material databases show that the best performing
pre-trained convolutional neural network (CNN) architectures can achieve up to
91.03% mean average precision when classifying materials in cross-dataset
scenarios. We demonstrate that synthesized data achieve an improvement on mean
average precision when used as training data and in conjunction with
pre-trained CNN architectures, which spans from ~ 5% to ~ 19% across three
widely used material databases of real-world images.
Repairing Ontologies via Axiom Weakening
Authors: Nicolas Troquard, Roberto Confalonieri, Pietro Galliani, Rafael Penaloza, Daniele Porello, Oliver Kutz
Date: 2017-11-09 00:00:00

Ontology engineering is a hard and error-prone task, in which small changes
may lead to errors, or even produce an inconsistent ontology. As ontologies
grow in size, the need for automated methods for repairing inconsistencies
while preserving as much of the original knowledge as possible increases. Most
previous approaches to this task are based on removing a few axioms from the
ontology to regain consistency. We propose a new method based on weakening
these axioms to make them less restrictive, employing the use of refinement
operators. We introduce the theoretical framework for weakening DL ontologies,
propose algorithms to repair ontologies based on the framework, and provide an
analysis of the computational complexity. Through an empirical analysis made
over real-life ontologies, we show that our approach preserves significantly
more of the original knowledge of the ontology than removing axioms.
A Further Analysis of The Role of Heterogeneity in Coevolutionary Spatial Games
Authors: Marcos Cardinot, Josephine Griffith, Colm O'Riordan
Date: 2017-11-09 00:00:00

Heterogeneity has been studied as one of the most common explanations of the
puzzle of cooperation in social dilemmas. A large number of papers have been
published discussing the effects of increasing heterogeneity in structured
populations of agents, where it has been established that heterogeneity may
favour cooperative behaviour if it supports agents to locally coordinate their
strategies. In this paper, assuming an existing model of a heterogeneous
weighted network, we aim to further this analysis by exploring the relationship
(if any) between heterogeneity and cooperation. We adopt a weighted network
which is fully populated by agents playing both the Prisoner's Dilemma or the
Optional Prisoner's Dilemma games with coevolutionary rules, i.e., not only the
strategies but also the link weights evolve over time. Surprisingly, results
show that the heterogeneity of link weights (states) on their own does not
always promote cooperation; rather cooperation is actually favoured by the
increase in the number of overlapping states and not by the heterogeneity
itself. We believe that these results can guide further research towards a more
accurate analysis of the role of heterogeneity in social dilemmas.
Feature Sensitive Label Fusion with Random Walker for Atlas-based Image Segmentation
Authors: Siqi Bao, Albert C. S. Chung
Date: 2016-10-24 00:00:00
Tasks: Image Segmentation, Semantic Segmentation

In this paper, a novel label fusion method is proposed for brain magnetic
resonance image segmentation. This label fusion method is formulated on a
graph, which embraces both label priors from atlases and anatomical priors from
target image. To represent a pixel in a comprehensive way, three kinds of
feature vectors are generated, including intensity, gradient and structural
signature. To select candidate atlas nodes for fusion, rather than exact
searching, randomized k-d tree with spatial constraint is introduced as an
efficient approximation for high-dimensional feature matching. Feature
Sensitive Label Prior (FSLP), which takes both the consistency and variety of
different features into consideration, is proposed to gather atlas priors. As
FSLP is a non-convex problem, one heuristic approach is further designed to
solve it efficiently. Moreover, based on the anatomical knowledge, parts of the
target pixels are also employed as graph seeds to assist the label fusion
process and an iterative strategy is utilized to gradually update the label
map. The comprehensive experiments carried out on two publicly available
databases give results to demonstrate that the proposed method can obtain
better segmentation quality.
A random matrix analysis and improvement of semi-supervised learning for large dimensional data
Authors: Xiaoyi Mai, Romain Couillet
Date: 2017-11-09 00:00:00
Tasks: General Classification

This article provides an original understanding of the behavior of a class of
graph-oriented semi-supervised learning algorithms in the limit of large and
numerous data. It is demonstrated that the intuition at the root of these
methods collapses in this limit and that, as a result, most of them become
inconsistent. Corrective measures and a new data-driven parametrization scheme
are proposed along with a theoretical analysis of the asymptotic performances
of the resulting approach. A surprisingly close behavior between theoretical
performances on Gaussian mixture models and on real datasets is also
illustrated throughout the article, thereby suggesting the importance of the
proposed analysis for dealing with practical data. As a result, significant
performance gains are observed on practical data classification using the
proposed parametrization.
Measuring Catastrophic Forgetting in Neural Networks
Authors: Ronald Kemker, Marc McClure, Angelina Abitino, Tyler Hayes, Christopher Kanan
Date: 2017-08-07 00:00:00

Deep neural networks are used in many state-of-the-art systems for machine
perception. Once a network is trained to do a specific task, e.g., bird
classification, it cannot easily be trained to do new tasks, e.g.,
incrementally learning to recognize additional bird species or learning an
entirely different task such as flower recognition. When new tasks are added,
typical deep neural networks are prone to catastrophically forgetting previous
tasks. Networks that are capable of assimilating new information incrementally,
much like how humans form new memories over time, will be more efficient than
re-training the model from scratch each time a new task needs to be learned.
There have been multiple attempts to develop schemes that mitigate catastrophic
forgetting, but these methods have not been directly compared, the tests used
to evaluate them vary considerably, and these methods have only been evaluated
on small-scale problems (e.g., MNIST). In this paper, we introduce new metrics
and benchmarks for directly comparing five different mechanisms designed to
mitigate catastrophic forgetting in neural networks: regularization,
ensembling, rehearsal, dual-memory, and sparse-coding. Our experiments on
real-world images and sounds show that the mechanism(s) that are critical for
optimal performance vary based on the incremental training paradigm and type of
data being used, but they all demonstrate that the catastrophic forgetting
problem has yet to be solved.