pwc-archive/papers-with-abstracts huggingface dataset:
Analysis of Agent Expertise in Ms. Pac-Man using Value-of-Information-based Policies
Authors: Isaac J. Sledge, Jose C. Principe
Date: 2017-02-28 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Conventional reinforcement learning methods for Markov decision processes
rely on weakly-guided, stochastic searches to drive the learning process. It
can therefore be difficult to predict what agent behaviors might emerge. In
this paper, we consider an information-theoretic cost function for performing
constrained stochastic searches that promote the formation of risk-averse to
risk-favoring behaviors. This cost function is the value of information, which
provides the optimal trade-off between the expected return of a policy and the
policy's complexity; policy complexity is measured by number of bits and
controlled by a single hyperparameter on the cost function. As the policy
complexity is reduced, the agents will increasingly eschew risky actions. This
reduces the potential for high accrued rewards. As the policy complexity
increases, the agents will take actions, regardless of the risk, that can raise
the long-term rewards. The obtainable reward depends on a single, tunable
hyperparameter that regulates the degree of policy complexity.
  We evaluate the performance of value-of-information-based policies on a
stochastic version of Ms. Pac-Man. A major component of this paper is the
demonstration that ranges of policy complexity values yield different game-play
styles and explaining why this occurs. We also show that our
reinforcement-learning search mechanism is more efficient than the others we
utilize. This result implies that the value of information theory is
appropriate for framing the exploitation-exploration trade-off in reinforcement
learning.
AIDE: An algorithm for measuring the accuracy of probabilistic inference algorithms
Authors: Marco F. Cusumano-Towner, Vikash K. Mansinghka
Date: 2017-05-19 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Variational Inference

Approximate probabilistic inference algorithms are central to many fields.
Examples include sequential Monte Carlo inference in robotics, variational
inference in machine learning, and Markov chain Monte Carlo inference in
statistics. A key problem faced by practitioners is measuring the accuracy of
an approximate inference algorithm on a specific data set. This paper
introduces the auxiliary inference divergence estimator (AIDE), an algorithm
for measuring the accuracy of approximate inference algorithms. AIDE is based
on the observation that inference algorithms can be treated as probabilistic
models and the random variables used within the inference algorithm can be
viewed as auxiliary variables. This view leads to a new estimator for the
symmetric KL divergence between the approximating distributions of two
inference algorithms. The paper illustrates application of AIDE to algorithms
for inference in regression, hidden Markov, and Dirichlet process mixture
models. The experiments show that AIDE captures the qualitative behavior of a
broad class of inference algorithms and can detect failure modes of inference
algorithms that are missed by standard heuristics.
Low-shot Visual Recognition by Shrinking and Hallucinating Features
Authors: Bharath Hariharan, Ross Girshick
Date: 2016-06-09 00:00:00
Proceeding: ICCV 2017 10
Tasks: BIG-bench Machine Learning, Few-Shot Image Classification

Low-shot visual learning---the ability to recognize novel object categories
from very few examples---is a hallmark of human visual intelligence. Existing
machine learning approaches fail to generalize in the same way. To make
progress on this foundational problem, we present a low-shot learning benchmark
on complex images that mimics challenges faced by recognition systems in the
wild. We then propose a) representation regularization techniques, and b)
techniques to hallucinate additional training examples for data-starved
classes. Together, our methods improve the effectiveness of convolutional
networks in low-shot learning, improving the one-shot accuracy on novel classes
by 2.3x on the challenging ImageNet dataset.
Neural Expectation Maximization
Authors: Klaus Greff, Sjoerd van Steenkiste, Jürgen Schmidhuber
Date: 2017-08-11 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Clustering

Many real world tasks such as reasoning and physical interaction require
identification and manipulation of conceptual entities. A first step towards
solving these tasks is the automated discovery of distributed symbol-like
representations. In this paper, we explicitly formalize this problem as
inference in a spatial mixture model where each component is parametrized by a
neural network. Based on the Expectation Maximization framework we then derive
a differentiable clustering method that simultaneously learns how to group and
represent individual entities. We evaluate our method on the (sequential)
perceptual grouping task and find that it is able to accurately recover the
constituent objects. We demonstrate that the learned representations are useful
for next-step prediction.
Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols
Authors: Serhii Havrylov, Ivan Titov
Date: 2017-05-31 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Reinforcement Learning

Learning to communicate through interaction, rather than relying on explicit
supervision, is often considered a prerequisite for developing a general AI. We
study a setting where two agents engage in playing a referential game and, from
scratch, develop a communication protocol necessary to succeed in this game.
Unlike previous work, we require that messages they exchange, both at train and
test time, are in the form of a language (i.e. sequences of discrete symbols).
We compare a reinforcement learning approach and one using a differentiable
relaxation (straight-through Gumbel-softmax estimator) and observe that the
latter is much faster to converge and it results in more effective protocols.
Interestingly, we also observe that the protocol we induce by optimizing the
communication success exhibits a degree of compositionality and variability
(i.e. the same information can be phrased in different ways), both properties
characteristic of natural languages. As the ultimate goal is to ensure that
communication is accomplished in natural language, we also perform experiments
where we inject prior information about natural language into our model and
study properties of the resulting protocol.
Searching for Biophysically Realistic Parameters for Dynamic Neuron Models by Genetic Algorithms from Calcium Imaging Recording
Authors: Magdalena Fuchs, Manuel Zimmer, Radu Grosu, Ramin M. Hasani
Date: 2017-11-04 00:00:00
Tasks: Time Series, Time Series Analysis

Individual Neurons in the nervous systems exploit various dynamics. To
capture these dynamics for single neurons, we tune the parameters of an
electrophysiological model of nerve cells, to fit experimental data obtained by
calcium imaging. A search for the biophysical parameters of this model is
performed by means of a genetic algorithm, where the model neuron is exposed to
a predefined input current representing overall inputs from other parts of the
nervous system. The algorithm is then constrained for keeping the ion-channel
currents within reasonable ranges, while producing the best fit to a calcium
imaging time series of the AVA interneuron, from the brain of the soil-worm, C.
elegans. Our settings enable us to project a set of biophysical parameters to
the the neuron kinetics observed in neuronal imaging.
Gradient Episodic Memory for Continual Learning
Authors: David Lopez-Paz, Marc'Aurelio Ranzato
Date: 2017-06-26 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Continual Learning, Incremental Learning

One major obstacle towards AI is the poor ability of models to solve new problems quicker, and without forgetting previously acquired knowledge. To better understand this issue, we study the problem of continual learning, where the model observes, once and one by one, examples concerning a sequence of tasks. First, we propose a set of metrics to evaluate models learning over a continuum of data. These metrics characterize models not only by their test accuracy, but also in terms of their ability to transfer knowledge across tasks. Second, we propose a model for continual learning, called Gradient Episodic Memory (GEM) that alleviates forgetting, while allowing beneficial transfer of knowledge to previous tasks. Our experiments on variants of the MNIST and CIFAR-100 datasets demonstrate the strong performance of GEM when compared to the state-of-the-art.
The Case for Meta-Cognitive Machine Learning: On Model Entropy and Concept Formation in Deep Learning
Authors: Johan Loeckx
Date: 2017-11-04 00:00:00
Tasks: BIG-bench Machine Learning

Machine learning is usually defined in behaviourist terms, where external
validation is the primary mechanism of learning. In this paper, I argue for a
more holistic interpretation in which finding more probable, efficient and
abstract representations is as central to learning as performance. In other
words, machine learning should be extended with strategies to reason over its
own learning process, leading to so-called meta-cognitive machine learning. As
such, the de facto definition of machine learning should be reformulated in
these intrinsically multi-objective terms, taking into account not only the
task performance but also internal learning objectives. To this end, we suggest
a "model entropy function" to be defined that quantifies the efficiency of the
internal learning processes. It is conjured that the minimization of this model
entropy leads to concept formation. Besides philosophical aspects, some initial
illustrations are included to support the claims.
Deep Stacking Networks for Low-Resource Chinese Word Segmentation with Transfer Learning
Authors: Jingjing Xu, Xu sun, Sujian Li, Xiaoyan Cai, Bingzhen Wei
Date: 2017-11-04 00:00:00
Tasks: Chinese Word Segmentation, Transfer Learning

In recent years, neural networks have proven to be effective in Chinese word
segmentation. However, this promising performance relies on large-scale
training data. Neural networks with conventional architectures cannot achieve
the desired results in low-resource datasets due to the lack of labelled
training data. In this paper, we propose a deep stacking framework to improve
the performance on word segmentation tasks with insufficient data by
integrating datasets from diverse domains. Our framework consists of two parts,
domain-based models and deep stacking networks. The domain-based models are
used to learn knowledge from different datasets. The deep stacking networks are
designed to integrate domain-based models. To reduce model conflicts, we
innovatively add communication paths among models and design various structures
of deep stacking networks, including Gaussian-based Stacking Networks,
Concatenate-based Stacking Networks, Sequence-based Stacking Networks and
Tree-based Stacking Networks. We conduct experiments on six low-resource
datasets from various domains. Our proposed framework shows significant
performance improvements on all datasets compared with several strong
baselines.
On clustering network-valued data
Authors: Soumendu Sundar Mukherjee, Purnamrita Sarkar, Lizhen Lin
Date: 2016-06-08 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Clustering, Community Detection

Community detection, which focuses on clustering nodes or detecting
communities in (mostly) a single network, is a problem of considerable
practical interest and has received a great deal of attention in the research
community. While being able to cluster within a network is important, there are
emerging needs to be able to cluster multiple networks. This is largely
motivated by the routine collection of network data that are generated from
potentially different populations. These networks may or may not have node
correspondence. When node correspondence is present, we cluster networks by
summarizing a network by its graphon estimate, whereas when node correspondence
is not present, we propose a novel solution for clustering such networks by
associating a computationally feasible feature vector to each network based on
trace of powers of the adjacency matrix. We illustrate our methods using both
simulated and real data sets, and theoretical justifications are provided in
terms of consistency.
Positive-Unlabeled Learning with Non-Negative Risk Estimator
Authors: Ryuichi Kiryo, Gang Niu, Marthinus C. Du Plessis, Masashi Sugiyama
Date: 2017-03-02 00:00:00
Proceeding: NeurIPS 2017 12

From only positive (P) and unlabeled (U) data, a binary classifier could be
trained with PU learning, in which the state of the art is unbiased PU
learning. However, if its model is very flexible, empirical risks on training
data will go negative, and we will suffer from serious overfitting. In this
paper, we propose a non-negative risk estimator for PU learning: when getting
minimized, it is more robust against overfitting, and thus we are able to use
very flexible models (such as deep neural networks) given limited P data.
Moreover, we analyze the bias, consistency, and mean-squared-error reduction of
the proposed risk estimator, and bound the estimation error of the resulting
empirical risk minimizer. Experiments demonstrate that our risk estimator fixes
the overfitting problem of its unbiased counterparts.
Language as a matrix product state
Authors: Vasily Pestun, John Terilla, Yiannis Vlassopoulos
Date: 2017-11-04 00:00:00
Tasks: Translation

We propose a statistical model for natural language that begins by
considering language as a monoid, then representing it in complex matrices with
a compatible translation invariant probability measure. We interpret the
probability measure as arising via the Born rule from a translation invariant
matrix product state.
Separation-Free Super-Resolution from Compressed Measurements is Possible: an Orthonormal Atomic Norm Minimization Approach
Authors: Weiyu Xu, Jirong Yi, Soura Dasgupta, Jian-Feng Cai, Mathews Jacob, Myung Cho
Date: 2017-11-04 00:00:00
Tasks: compressed sensing, Super-Resolution

We consider the problem of recovering the superposition of $R$ distinct
complex exponential functions from compressed non-uniform time-domain samples.
Total Variation (TV) minimization or atomic norm minimization was proposed in
the literature to recover the $R$ frequencies or the missing data. However, it
is known that in order for TV minimization and atomic norm minimization to
recover the missing data or the frequencies, the underlying $R$ frequencies are
required to be well-separated, even when the measurements are noiseless. This
paper shows that the Hankel matrix recovery approach can super-resolve the $R$
complex exponentials and their frequencies from compressed non-uniform
measurements, regardless of how close their frequencies are to each other. We
propose a new concept of orthonormal atomic norm minimization (OANM), and
demonstrate that the success of Hankel matrix recovery in separation-free
super-resolution comes from the fact that the nuclear norm of a Hankel matrix
is an orthonormal atomic norm. More specifically, we show that, in traditional
atomic norm minimization, the underlying parameter values $\textbf{must}$ be
well separated to achieve successful signal recovery, if the atoms are changing
continuously with respect to the continuously-valued parameter. In contrast,
for the OANM, it is possible the OANM is successful even though the original
atoms can be arbitrarily close.
  As a byproduct of this research, we provide one matrix-theoretic inequality
of nuclear norm, and give its proof from the theory of compressed sensing.
Guiding the search in continuous state-action spaces by learning an action sampling distribution from off-target samples
Authors: Beomjoon Kim, Leslie Pack Kaelbling, Tomas Lozano-Perez
Date: 2017-11-04 00:00:00
Tasks: Generative Adversarial Network

In robotics, it is essential to be able to plan efficiently in
high-dimensional continuous state-action spaces for long horizons. For such
complex planning problems, unguided uniform sampling of actions until a path to
a goal is found is hopelessly inefficient, and gradient-based approaches often
fall short when the optimization manifold of a given problem is not smooth. In
this paper we present an approach that guides the search of a state-space
planner, such as A*, by learning an action-sampling distribution that can
generalize across different instances of a planning problem. The motivation is
that, unlike typical learning approaches for planning for continuous action
space that estimate a policy, an estimated action sampler is more robust to
error since it has a planner to fall back on. We use a Generative Adversarial
Network (GAN), and address an important issue: search experience consists of a
relatively large number of actions that are not on a solution path and a
relatively small number of actions that actually are on a solution path. We
introduce a new technique, based on an importance-ratio estimation method, for
using samples from a non-target distribution to make GAN learning more
data-efficient. We provide theoretical guarantees and empirical evaluation in
three challenging continuous robot planning problems to illustrate the
effectiveness of our algorithm.
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Authors: Balaji Lakshminarayanan, Alexander Pritzel, Charles Blundell
Date: 2016-12-05 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Image Classification, regression, Uncertainty Quantification

Deep neural networks (NNs) are powerful black box predictors that have
recently achieved impressive performance on a wide spectrum of tasks.
Quantifying predictive uncertainty in NNs is a challenging and yet unsolved
problem. Bayesian NNs, which learn a distribution over weights, are currently
the state-of-the-art for estimating predictive uncertainty; however these
require significant modifications to the training procedure and are
computationally expensive compared to standard (non-Bayesian) NNs. We propose
an alternative to Bayesian NNs that is simple to implement, readily
parallelizable, requires very little hyperparameter tuning, and yields high
quality predictive uncertainty estimates. Through a series of experiments on
classification and regression benchmarks, we demonstrate that our method
produces well-calibrated uncertainty estimates which are as good or better than
approximate Bayesian NNs. To assess robustness to dataset shift, we evaluate
the predictive uncertainty on test examples from known and unknown
distributions, and show that our method is able to express higher uncertainty
on out-of-distribution examples. We demonstrate the scalability of our method
by evaluating predictive uncertainty estimates on ImageNet.
Learning Hierarchical Information Flow with Recurrent Neural Modules
Authors: Danijar Hafner, Alex Irpan, James Davidson, Nicolas Heess
Date: 2017-06-18 00:00:00
Proceeding: NeurIPS 2017 12

We propose ThalNet, a deep learning model inspired by neocortical
communication via the thalamus. Our model consists of recurrent neural modules
that send features through a routing center, endowing the modules with the
flexibility to share features over multiple time steps. We show that our model
learns to route information hierarchically, processing input data by a chain of
modules. We observe common architectures, such as feed forward neural networks
and skip connections, emerging as special cases of our architecture, while
novel connectivity patterns are learned for the text8 compression task. Our
model outperforms standard recurrent neural networks on several sequential
benchmarks.
An Iterative Co-Saliency Framework for RGBD Images
Authors: Runmin Cong, Jianjun Lei, Huazhu Fu, Weisi Lin, Qingming Huang, Xiaochun Cao, Chunping Hou
Date: 2017-11-04 00:00:00
Tasks: Co-Salient Object Detection, Saliency Detection

As a newly emerging and significant topic in computer vision community,
co-saliency detection aims at discovering the common salient objects in
multiple related images. The existing methods often generate the co-saliency
map through a direct forward pipeline which is based on the designed cues or
initialization, but lack the refinement-cycle scheme. Moreover, they mainly
focus on RGB image and ignore the depth information for RGBD images. In this
paper, we propose an iterative RGBD co-saliency framework, which utilizes the
existing single saliency maps as the initialization, and generates the final
RGBD cosaliency map by using a refinement-cycle model. Three schemes are
employed in the proposed RGBD co-saliency framework, which include the addition
scheme, deletion scheme, and iteration scheme. The addition scheme is used to
highlight the salient regions based on intra-image depth propagation and
saliency propagation, while the deletion scheme filters the saliency regions
and removes the non-common salient regions based on interimage constraint. The
iteration scheme is proposed to obtain more homogeneous and consistent
co-saliency map. Furthermore, a novel descriptor, named depth shape prior, is
proposed in the addition scheme to introduce the depth information to enhance
identification of co-salient objects. The proposed method can effectively
exploit any existing 2D saliency model to work well in RGBD co-saliency
scenarios. The experiments on two RGBD cosaliency datasets demonstrate the
effectiveness of our proposed framework.
Repeated Inverse Reinforcement Learning
Authors: Kareem Amin, Nan Jiang, Satinder Singh
Date: 2017-05-15 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Imitation Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We introduce a novel repeated Inverse Reinforcement Learning problem: the
agent has to act on behalf of a human in a sequence of tasks and wishes to
minimize the number of tasks that it surprises the human by acting suboptimally
with respect to how the human would have acted. Each time the human is
surprised, the agent is provided a demonstration of the desired behavior by the
human. We formalize this problem, including how the sequence of tasks is
chosen, in a few different ways and provide some foundational results.
A Regularized Framework for Sparse and Structured Neural Attention
Authors: Vlad Niculae, Mathieu Blondel
Date: 2017-05-22 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Machine Translation, Natural Language Inference, Sentence, Sentence Summarization, Text Summarization, Translation

Modern neural networks are often augmented with an attention mechanism, which
tells the network where to focus within the input. We propose in this paper a
new framework for sparse and structured attention, building upon a smoothed max
operator. We show that the gradient of this operator defines a mapping from
real values to probabilities, suitable as an attention mechanism. Our framework
includes softmax and a slight generalization of the recently-proposed sparsemax
as special cases. However, we also show how our framework can incorporate
modern structured penalties, resulting in more interpretable attention
mechanisms, that focus on entire segments or groups of an input. We derive
efficient algorithms to compute the forward and backward passes of our
attention mechanisms, enabling their use in a neural network trained with
backpropagation. To showcase their potential as a drop-in replacement for
existing ones, we evaluate our attention mechanisms on three large-scale tasks:
textual entailment, machine translation, and sentence summarization. Our
attention mechanisms improve interpretability without sacrificing performance;
notably, on textual entailment and summarization, we outperform the standard
attention mechanisms based on softmax and sparsemax.