pwc-archive/papers-with-abstracts huggingface dataset:
Stability-certified reinforcement learning: A control-theoretic perspective
Authors: Ming Jin, Javad Lavaei
Date: 2018-10-26 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We investigate the important problem of certifying stability of reinforcement
learning policies when interconnected with nonlinear dynamical systems. We show
that by regulating the input-output gradients of policies, strong guarantees of
robust stability can be obtained based on a proposed semidefinite programming
feasibility problem. The method is able to certify a large set of stabilizing
controllers by exploiting problem-specific structures; furthermore, we analyze
and establish its (non)conservatism. Empirical evaluations on two decentralized
control tasks, namely multi-flight formation and power system frequency
regulation, demonstrate that the reinforcement learning agents can have high
performance within the stability-certified parameter space, and also exhibit
stable learning behaviors in the long run.
Information Bottleneck Methods for Distributed Learning
Authors: Parinaz Farajiparvar, Ahmad Beirami, Matthew Nokleby
Date: 2018-10-26 00:00:00

We study a distributed learning problem in which Alice sends a compressed
distillation of a set of training data to Bob, who uses the distilled version
to best solve an associated learning problem. We formalize this as a
rate-distortion problem in which the training set is the source and Bob's
cross-entropy loss is the distortion measure. We consider this problem for
unsupervised learning for batch and sequential data. In the batch data, this
problem is equivalent to the information bottleneck (IB), and we show that
reduced-complexity versions of standard IB methods solve the associated
rate-distortion problem. For the streaming data, we present a new algorithm,
which may be of independent interest, that solves the rate-distortion problem
for Gaussian sources. Furthermore, to improve the results of the iterative
algorithm for sequential data we introduce a two-pass version of this
algorithm. Finally, we show the dependency of the rate on the number of samples
$k$ required for Gaussian sources to ensure cross-entropy loss that scales
optimally with the growth of the training set.
Parsing Coordination for Spoken Language Understanding
Authors: Sanchit Agarwal, Rahul Goel, Tagyoung Chung, Abhishek Sethi, Arindam Mandal, Spyros Matsoukas
Date: 2018-10-26 00:00:00
Tasks: Spoken Language Understanding

Typical spoken language understanding systems provide narrow semantic parses
using a domain-specific ontology. The parses contain intents and slots that are
directly consumed by downstream domain applications. In this work we discuss
expanding such systems to handle compound entities and intents by introducing a
domain-agnostic shallow parser that handles linguistic coordination. We show
that our model for parsing coordination learns domain-independent and
slot-independent features and is able to segment conjunct boundaries of many
different phrasal categories. We also show that using adversarial training can
be effective for improving generalization across different slot types for
coordination parsing.
Empirical Evaluation of Contextual Policy Search with a Comparison-based Surrogate Model and Active Covariance Matrix Adaptation
Authors: Alexander Fabisch
Date: 2018-10-26 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Contextual policy search (CPS) is a class of multi-task reinforcement
learning algorithms that is particularly useful for robotic applications. A
recent state-of-the-art method is Contextual Covariance Matrix Adaptation
Evolution Strategies (C-CMA-ES). It is based on the standard black-box
optimization algorithm CMA-ES. There are two useful extensions of CMA-ES that
we will transfer to C-CMA-ES and evaluate empirically: ACM-ES, which uses a
comparison-based surrogate model, and aCMA-ES, which uses an active update of
the covariance matrix. We will show that improvements with these methods can be
impressive in terms of sample-efficiency, although this is not relevant any
more for the robotic domain.
Transfer of Deep Reactive Policies for MDP Planning
Authors: Aniket Bajpai, Sankalp Garg, Mausam
Date: 2018-10-26 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Deep Reinforcement Learning, Reinforcement Learning, Reinforcement Learning (RL), Transfer Learning

Domain-independent probabilistic planners input an MDP description in a
factored representation language such as PPDDL or RDDL, and exploit the
specifics of the representation for faster planning. Traditional algorithms
operate on each problem instance independently, and good methods for
transferring experience from policies of other instances of a domain to a new
instance do not exist. Recently, researchers have begun exploring the use of
deep reactive policies, trained via deep reinforcement learning (RL), for MDP
planning domains. One advantage of deep reactive policies is that they are more
amenable to transfer learning.
  In this paper, we present the first domain-independent transfer algorithm for
MDP planning domains expressed in an RDDL representation. Our architecture
exploits the symbolic state configuration and transition function of the domain
(available via RDDL) to learn a shared embedding space for states and
state-action pairs for all problem instances of a domain. We then learn an RL
agent in the embedding space, making a near zero-shot transfer possible, i.e.,
without much training on the new instance, and without using the domain
simulator at all. Experiments on three different benchmark domains underscore
the value of our transfer algorithm. Compared against planning from scratch,
and a state-of-the-art RL transfer algorithm, our transfer solution has
significantly superior learning curves.
Can Entropy Explain Successor Surprisal Effects in Reading?
Authors: Marten van Schijndel, Tal Linzen
Date: 2018-10-26 00:00:00
Proceeding: WS 2019 1
Tasks: Language Modeling, Language Modelling

Human reading behavior is sensitive to surprisal: more predictable words tend
to be read faster. Unexpectedly, this applies not only to the surprisal of the
word that is currently being read, but also to the surprisal of upcoming
(successor) words that have not been fixated yet. This finding has been
interpreted as evidence that readers can extract lexical information
parafoveally. Calling this interpretation into question, Angele et al. (2015)
showed that successor effects appear even in contexts in which those successor
words are not yet visible. They hypothesized that successor surprisal predicts
reading time because it approximates the reader's uncertainty about upcoming
words. We test this hypothesis on a reading time corpus using an LSTM language
model, and find that successor surprisal and entropy are independent predictors
of reading time. This independence suggests that entropy alone is unlikely to
be the full explanation for successor surprisal effects.
Accumulating Knowledge for Lifelong Online Learning
Authors: Changjian Shui, Ihsen Hedhli, Christian Gagné
Date: 2018-10-26 00:00:00
Tasks: Lifelong learning, Transfer Learning

Lifelong learning can be viewed as a continuous transfer learning procedure
over consecutive tasks, where learning a given task depends on accumulated
knowledge --- the so-called knowledge base. Most published work on lifelong
learning makes a batch processing of each task, implying that a data collection
step is required beforehand. We are proposing a new framework, lifelong online
learning, in which the learning procedure for each task is interactive. This is
done through a computationally efficient algorithm where the predicted result
for a given task is made by combining two intermediate predictions: by using
only the information from the current task and by relying on the accumulated
knowledge. In this work, two challenges are tackled: making no assumption on
the task generation distribution, and processing with a possibly unknown number
of instances for each task. We are providing a theoretical analysis of this
algorithm, with a cumulative error upper bound for each task. We find that
under some mild conditions, the algorithm can still benefit from a small
cumulative error even when facing few interactions. Moreover, we provide
experimental results on both synthetic and real datasets that validate the
correct behavior and practical usefulness of the proposed algorithm.
Named Person Coreference in English News
Authors: Oshin Agarwal, Sanjay Subramanian, Ani Nenkova, Dan Roth
Date: 2018-10-26 00:00:00
Tasks: coreference-resolution, Coreference Resolution, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

People are often entities of interest in tasks such as search and information
extraction. In these tasks, the goal is to find as much information as possible
about people specified by their name. However in text, some of the references
to people are by pronouns (she, his) or generic descriptions (the professor,
the German chancellor). It is therefore important that coreference resolution
systems are able to link these different types of mentions to the correct
person name. Here, we evaluate two state of the art coreference resolution
systems on the subtask of Named Person Coreference, in which we are interested
in identifying a person mentioned by name, along with all other mentions of the
person, by pronoun or generic noun phrase. Our analysis reveals that standard
coreference metrics do not reflect adequately the requirements in this task:
they do not penalize systems for not identifying any mentions by name and they
reward systems even if systems find correctly mentions to the same entity but
fail to link these to a proper name (she--the student---no name). We introduce
new metrics for evaluating named person coreference that address these
discrepancies. We present a simple rule-based named entity recognition driven
system, which outperforms the current state-of-the-art systems on these
task-specific metrics and performs on par with them on traditional coreference
evaluations. Finally, we present similar evaluation for coreference resolution
of other named entities and show that the rule-based approach is effective only
for person named coreference, not other named entity types.
Extractive Summarization of EHR Discharge Notes
Authors: Emily Alsentzer, Anne Kim
Date: 2018-10-26 00:00:00
Tasks: Decision Making, Extractive Summarization

Patient summarization is essential for clinicians to provide coordinated care
and practice effective communication. Automated summarization has the potential
to save time, standardize notes, aid clinical decision making, and reduce
medical errors. Here we provide an upper bound on extractive summarization of
discharge notes and develop an LSTM model to sequentially label topics of
history of present illness notes. We achieve an F1 score of 0.876, which
indicates that this model can be employed to create a dataset for evaluation of
extractive summarization methods.
Finding Answers from the Word of God: Domain Adaptation for Neural Networks in Biblical Question Answering
Authors: Helen Jiahe Zhao, Jiamou Liu
Date: 2018-10-26 00:00:00
Tasks: Domain Adaptation, Question Answering, Sentence, Transfer Learning, Translation

Question answering (QA) has significantly benefitted from deep learning
techniques in recent years. However, domain-specific QA remains a challenge due
to the significant amount of data required to train a neural network. This
paper studies the answer sentence selection task in the Bible domain and answer
questions by selecting relevant verses from the Bible. For this purpose, we
create a new dataset BibleQA based on bible trivia questions and propose three
neural network models for our task. We pre-train our models on a large-scale QA
dataset, SQuAD, and investigate the effect of transferring weights on model
accuracy. Furthermore, we also measure the model accuracies with different
answer context lengths and different Bible translations. We affirm that
transfer learning has a noticeable improvement in the model accuracy. We
achieve relatively good results with shorter context lengths, whereas longer
context lengths decreased model accuracy. We also find that using a more modern
Bible translation in the dataset has a positive effect on the task.
Quantum Entanglement in Corpuses of Documents
Authors: Lester Beltran, Suzette Geriente
Date: 2018-10-25 00:00:00

We show that data collected from corpuses of documents violate the
Clauser-Horne-Shimony-Holt version of Bell's inequality (CHSH inequality) and
therefore indicate the presence of quantum entanglement in their structure. We
obtain this result by considering two concepts and their combination and
coincidence operations consisting of searches of co-occurrences of exemplars of
these concepts in specific corpuses of documents. Measuring the frequencies of
these co-occurrences and calculating the relative frequencies as approximate
probabilities entering in the CHSH inequality, we obtain manifest violations of
the latter for all considered corpuses of documents. In comparing these
violations with those analogously obtained in an earlier work for the same
combined concepts in psychological coincidence experiments with human
participants, also violating the CHSH inequality, we identify the entanglement
as being carried by the meaning connection between the two considered concepts
within the combination they form. We explain the stronger violation for the
corpuses of documents, as compared to the violation in the psychology
experiments, as being due to the superior meaning domain of the human mind and,
on the other side, to the latter reaching a broader domain of meaning and being
possibly also actively influenced during the experimentation. We mention some
of the issues to be analyzed in future work such as the violations of the CHSH
inequality being larger than the `Cirel'son bound' for all of the considered
corpuses of documents.
Neuromodulation of Neuromorphic Circuits
Authors: Luka Ribar, Rodolphe Sepulchre
Date: 2018-05-15 00:00:00

We present a novel methodology to enable control of a neuromorphic circuit in
close analogy with the physiological neuromodulation of a single neuron. The
methodology is general in that it only relies on a parallel interconnection of
elementary voltage-controlled current sources. In contrast to controlling a
nonlinear circuit through the parameter tuning of a state-space model, our
approach is purely input-output. The circuit elements are controlled and
interconnected to shape the current-voltage characteristics (I-V curves) of the
circuit in prescribed timescales. In turn, shaping those I-V curves determines
the excitability properties of the circuit. We show that this methodology
enables both robust and accurate control of the circuit behavior and resembles
the biophysical mechanisms of neuromodulation. As a proof of concept, we
simulate a SPICE model composed of MOSFET transconductance amplifiers operating
in the weak inversion regime.
Topic-Specific Sentiment Analysis Can Help Identify Political Ideology
Authors: Sumit Bhatia, Deepak P
Date: 2018-10-30 00:00:00
Proceeding: WS 2018 10
Tasks: Articles, Sentiment Analysis

Ideological leanings of an individual can often be gauged by the sentiment
one expresses about different issues. We propose a simple framework that
represents a political ideology as a distribution of sentiment polarities
towards a set of topics. This representation can then be used to detect
ideological leanings of documents (speeches, news articles, etc.) based on the
sentiments expressed towards different topics. Experiments performed using a
widely used dataset show the promise of our proposed approach that achieves
comparable performance to other methods despite being much simpler and more
interpretable.
Exploration by Random Network Distillation
Authors: Yuri Burda, Harrison Edwards, Amos Storkey, Oleg Klimov
Date: 2018-10-30 00:00:00
Proceeding: ICLR 2019
Tasks: Atari Games, Deep Reinforcement Learning, Montezuma's Revenge, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Unsupervised Reinforcement Learning

We introduce an exploration bonus for deep reinforcement learning methods
that is easy to implement and adds minimal overhead to the computation
performed. The bonus is the error of a neural network predicting features of
the observations given by a fixed randomly initialized neural network. We also
introduce a method to flexibly combine intrinsic and extrinsic rewards. We find
that the random network distillation (RND) bonus combined with this increased
flexibility enables significant progress on several hard exploration Atari
games. In particular we establish state of the art performance on Montezuma's
Revenge, a game famously difficult for deep reinforcement learning methods. To
the best of our knowledge, this is the first method that achieves better than
average human performance on this game without using demonstrations or having
access to the underlying state of the game, and occasionally completes the
first level.
DropBlock: A regularization method for convolutional networks
Authors: Golnaz Ghiasi, Tsung-Yi Lin, Quoc V. Le
Date: 2018-10-30 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Image Classification, Object Detection

Deep neural networks often work well when they are over-parameterized and
trained with a massive amount of noise and regularization, such as weight decay
and dropout. Although dropout is widely used as a regularization technique for
fully connected layers, it is often less effective for convolutional layers.
This lack of success of dropout for convolutional layers is perhaps due to the
fact that activation units in convolutional layers are spatially correlated so
information can still flow through convolutional networks despite dropout. Thus
a structured form of dropout is needed to regularize convolutional networks. In
this paper, we introduce DropBlock, a form of structured dropout, where units
in a contiguous region of a feature map are dropped together. We found that
applying DropbBlock in skip connections in addition to the convolution layers
increases the accuracy. Also, gradually increasing number of dropped units
during training leads to better accuracy and more robust to hyperparameter
choices. Extensive experiments show that DropBlock works better than dropout in
regularizing convolutional networks. On ImageNet classification, ResNet-50
architecture with DropBlock achieves $78.13\%$ accuracy, which is more than
$1.6\%$ improvement on the baseline. On COCO detection, DropBlock improves
Average Precision of RetinaNet from $36.8\%$ to $38.4\%$.
ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension
Authors: Sheng Zhang, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Kevin Duh, Benjamin Van Durme
Date: 2018-10-30 00:00:00
Tasks: Common Sense Reasoning, Machine Reading Comprehension, Reading Comprehension

We present a large-scale dataset, ReCoRD, for machine reading comprehension
requiring commonsense reasoning. Experiments on this dataset demonstrate that
the performance of state-of-the-art MRC systems fall far behind human
performance. ReCoRD represents a challenge for future research to bridge the
gap between human and machine commonsense reading comprehension. ReCoRD is
available at http://nlp.jhu.edu/record.
Data Poisoning Attack against Unsupervised Node Embedding Methods
Authors: Mingjie Sun, Jian Tang, Huichen Li, Bo Li, Chaowei Xiao, Yao Chen, Dawn Song
Date: 2018-10-30 00:00:00
Tasks: Data Poisoning, Link Prediction

Unsupervised node embedding methods (e.g., DeepWalk, LINE, and node2vec) have
attracted growing interests given their simplicity and effectiveness. However,
although these methods have been proved effective in a variety of applications,
none of the existing work has analyzed the robustness of them. This could be
very risky if these methods are attacked by an adversarial party. In this
paper, we take the task of link prediction as an example, which is one of the
most fundamental problems for graph analysis, and introduce a data positioning
attack to node embedding methods. We give a complete characterization of
attacker's utilities and present efficient solutions to adversarial attacks for
two popular node embedding methods: DeepWalk and LINE. We evaluate our proposed
attack model on multiple real-world graphs. Experimental results show that our
proposed model can significantly affect the results of link prediction by
slightly changing the graph structures (e.g., adding or removing a few edges).
We also show that our proposed model is very general and can be transferable
across different embedding methods. Finally, we conduct a case study on a
coauthor network to better understand our attack method.
Image Restoration using Total Variation Regularized Deep Image Prior
Authors: Jiaming Liu, Yu Sun, Xiaojian Xu, Ulugbek S. Kamilov
Date: 2018-10-30 00:00:00
Tasks: Deblurring, Denoising, Image Denoising, Image Reconstruction, Image Restoration

In the past decade, sparsity-driven regularization has led to significant
improvements in image reconstruction. Traditional regularizers, such as total
variation (TV), rely on analytical models of sparsity. However, increasingly
the field is moving towards trainable models, inspired from deep learning. Deep
image prior (DIP) is a recent regularization framework that uses a
convolutional neural network (CNN) architecture without data-driven training.
This paper extends the DIP framework by combining it with the traditional TV
regularization. We show that the inclusion of TV leads to considerable
performance gains when tested on several traditional restoration tasks such as
image denoising and deblurring.
Divergence Network: Graphical calculation method of divergence functions
Authors: Tomohiro Nishiyama
Date: 2018-10-30 00:00:00

In this paper, we introduce directed networks called `divergence network' in
order to perform graphical calculation of divergence functions. By using the
divergence networks, we can easily understand the geometric meaning of
calculation results and grasp relations among divergence functions intuitively.
Discovering state-parameter mappings in subsurface models using generative adversarial networks
Authors: Alexander Y. Sun
Date: 2018-10-30 00:00:00
Tasks: Translation

A fundamental problem in geophysical modeling is related to the
identification and approximation of causal structures among physical processes.
However, resolving the bidirectional mappings between physical parameters and
model state variables (i.e., solving the forward and inverse problems) is
challenging, especially when parameter dimensionality is high. Deep learning
has opened a new door toward knowledge representation and complex pattern
identification. In particular, the recently introduced generative adversarial
networks (GANs) hold strong promises in learning cross-domain mappings for
image translation. This study presents a state-parameter identification GAN
(SPID-GAN) for simultaneously learning bidirectional mappings between a
high-dimensional parameter space and the corresponding model state space.
SPID-GAN is demonstrated using a series of representative problems from
subsurface flow modeling. Results show that SPID-GAN achieves satisfactory
performance in identifying the bidirectional state-parameter mappings,
providing a new deep-learning-based, knowledge representation paradigm for a
wide array of complex geophysical problems.