pwc-archive/papers-with-abstracts huggingface dataset:
Near Optimal Exploration-Exploitation in Non-Communicating Markov Decision Processes
Authors: Ronan Fruit, Matteo Pirotta, Alessandro Lazaric
Date: 2018-07-06 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Efficient Exploration

While designing the state space of an MDP, it is common to include states
that are transient or not reachable by any policy (e.g., in mountain car, the
product space of speed and position contains configurations that are not
physically reachable). This leads to defining weakly-communicating or
multi-chain MDPs. In this paper, we introduce \tucrl, the first algorithm able
to perform efficient exploration-exploitation in any finite Markov Decision
Process (MDP) without requiring any form of prior knowledge. In particular, for
any MDP with $S^{\texttt{C}}$ communicating states, $A$ actions and
$\Gamma^{\texttt{C}} \leq S^{\texttt{C}}$ possible communicating next states,
we derive a $\widetilde{O}(D^{\texttt{C}} \sqrt{\Gamma^{\texttt{C}}
S^{\texttt{C}} AT})$ regret bound, where $D^{\texttt{C}}$ is the diameter
(i.e., the longest shortest path) of the communicating part of the MDP. This is
in contrast with optimistic algorithms (e.g., UCRL, Optimistic PSRL) that
suffer linear regret in weakly-communicating MDPs, as well as posterior
sampling or regularised algorithms (e.g., REGAL), which require prior knowledge
on the bias span of the optimal policy to bias the exploration to achieve
sub-linear regret. We also prove that in weakly-communicating MDPs, no
algorithm can ever achieve a logarithmic growth of the regret without first
suffering a linear regret for a number of steps that is exponential in the
parameters of the MDP. Finally, we report numerical simulations supporting our
theoretical findings and showing how TUCRL overcomes the limitations of the
state-of-the-art.
End-to-End Race Driving with Deep Reinforcement Learning
Authors: Maximilian Jaritz, Raoul de Charette, Marin Toromanoff, Etienne Perot, Fawzi Nashashibi
Date: 2018-07-06 00:00:00
Tasks: Deep Reinforcement Learning, Domain Adaptation, Object Recognition, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Scene Understanding

We present research using the latest reinforcement learning algorithm for
end-to-end driving without any mediated perception (object recognition, scene
understanding). The newly proposed reward and learning strategies lead together
to faster convergence and more robust driving using only RGB image from a
forward facing camera. An Asynchronous Actor Critic (A3C) framework is used to
learn the car control in a physically and graphically realistic rally game,
with the agents evolving simultaneously on tracks with a variety of road
structures (turns, hills), graphics (seasons, location) and physics (road
adherence). A thorough evaluation is conducted and generalization is proven on
unseen tracks and using legal speed limits. Open loop tests on real sequences
of images show some domain adaption capability of our method.
Deep Back Projection for Sparse-View CT Reconstruction
Authors: Dong Hye Ye, Gregery T. Buzzard, Max Ruby, Charles A. Bouman
Date: 2018-07-06 00:00:00
Tasks: CT Reconstruction, Image Reconstruction

Filtered back projection (FBP) is a classical method for image reconstruction
from sinogram CT data. FBP is computationally efficient but produces lower
quality reconstructions than more sophisticated iterative methods, particularly
when the number of views is lower than the number required by the Nyquist rate.
In this paper, we use a deep convolutional neural network (CNN) to produce
high-quality reconstructions directly from sinogram data. A primary novelty of
our approach is that we first back project each view separately to form a stack
of back projections and then feed this stack as input into the convolutional
neural network. These single-view back projections convert the encoding of
sinogram data into the appropriate spatial location, which can then be
leveraged by the spatial invariance of the CNN to learn the reconstruction
effectively. We demonstrate the benefit of our CNN based back projection on
simulated sparse-view CT data over classical FBP.
A Variational Time Series Feature Extractor for Action Prediction
Authors: Maxime Chaveroche, Adrien Malaisé, Francis Colas, François Charpillet, Serena Ivaldi
Date: 2018-07-06 00:00:00
Tasks: Action Recognition, Prediction, Temporal Action Localization, Time Series, Time Series Analysis, Variational Inference

We propose a Variational Time Series Feature Extractor (VTSFE), inspired by
the VAE-DMP model of Chen et al., to be used for action recognition and
prediction. Our method is based on variational autoencoders. It improves
VAE-DMP in that it has a better noise inference model, a simpler transition
model constraining the acceleration in the trajectories of the latent space,
and a tighter lower bound for the variational inference. We apply the method
for classification and prediction of whole-body movements on a dataset with 7
tasks and 10 demonstrations per task, recorded with a wearable motion capture
suit. The comparison with VAE and VAE-DMP suggests the better performance of
our method for feature extraction. An open-source software implementation of
each method with TensorFlow is also provided. In addition, a more detailed
version of this work can be found in the indicated code repository. Although it
was meant to, the VTSFE hasn't been tested for action prediction, due to a lack
of time in the context of Maxime Chaveroche's Master thesis at INRIA.
Testing Untestable Neural Machine Translation: An Industrial Case
Authors: Wujie Zheng, Wenyu Wang, Dian Liu, Changrong Zhang, Qinsong Zeng, Yuetang Deng, Wei Yang, Pinjia He, Tao Xie
Date: 2018-07-06 00:00:00
Tasks: Machine Translation, NMT, Translation, valid

Neural Machine Translation (NMT) has been widely adopted recently due to its
advantages compared with the traditional Statistical Machine Translation (SMT).
However, an NMT system still often produces translation failures due to the
complexity of natural language and sophistication in designing neural networks.
While in-house black-box system testing based on reference translations (i.e.,
examples of valid translations) has been a common practice for NMT quality
assurance, an increasingly critical industrial practice, named in-vivo testing,
exposes unseen types or instances of translation failures when real users are
using a deployed industrial NMT system. To fill the gap of lacking test oracle
for in-vivo testing of an NMT system, in this paper, we propose a new approach
for automatically identifying translation failures, without requiring reference
translations for a translation task; our approach can directly serve as a test
oracle for in-vivo testing. Our approach focuses on properties of natural
language translation that can be checked systematically and uses information
from both the test inputs (i.e., the texts to be translated) and the test
outputs (i.e., the translations under inspection) of the NMT system. Our
evaluation conducted on real-world datasets shows that our approach can
effectively detect targeted property violations as translation failures. Our
experiences on deploying our approach in both production and development
environments of WeChat (a messenger app with over one billion monthly active
users) demonstrate high effectiveness of our approach along with high industry
impact.
Cause-Effect Deep Information Bottleneck For Systematically Missing Covariates
Authors: Sonali Parbhoo, Mario Wieser, Aleksander Wieczorek, Volker Roth
Date: 2018-07-06 00:00:00
Tasks: Causal Inference

Estimating the causal effects of an intervention from high-dimensional observational data is difficult due to the presence of confounding. The task is often complicated by the fact that we may have a systematic missingness in our data at test time. Our approach uses the information bottleneck to perform a low-dimensional compression of covariates by explicitly considering the relevance of information. Based on the sufficiently reduced covariate, we transfer the relevant information to cases where data is missing at test time, allowing us to reliably and accurately estimate the effects of an intervention, even where data is incomplete. Our results on causal inference benchmarks and a real application for treating sepsis show that our method achieves state-of-the art performance, without sacrificing interpretability.
Sum-Product Networks for Sequence Labeling
Authors: Martin Ratajczak, Sebastian Tschiatschek, Franz Pernkopf
Date: 2018-07-06 00:00:00
Tasks: Optical Character Recognition, Optical Character Recognition (OCR)

We consider higher-order linear-chain conditional random fields (HO-LC-CRFs)
for sequence modelling, and use sum-product networks (SPNs) for representing
higher-order input- and output-dependent factors. SPNs are a recently
introduced class of deep models for which exact and efficient inference can be
performed. By combining HO-LC-CRFs with SPNs, expressive models over both the
output labels and the hidden variables are instantiated while still enabling
efficient exact inference. Furthermore, the use of higher-order factors allows
us to capture relations of multiple input segments and multiple output labels
as often present in real-world data. These relations can not be modelled by the
commonly used first-order models and higher-order models with local factors
including only a single output label. We demonstrate the effectiveness of our
proposed models for sequence labeling. In extensive experiments, we outperform
other state-of-the-art methods in optical character recognition and achieve
competitive results in phone classification.
Optimal Sensor Data Fusion Architecture for Object Detection in Adverse Weather Conditions
Authors: Andreas Pfeuffer, Klaus Dietmayer
Date: 2018-07-06 00:00:00
Tasks: Object, object-detection, Object Detection, Sensor Fusion

A good and robust sensor data fusion in diverse weather conditions is a quite
challenging task. There are several fusion architectures in the literature,
e.g. the sensor data can be fused right at the beginning (Early Fusion), or
they can be first processed separately and then concatenated later (Late
Fusion). In this work, different fusion architectures are compared and
evaluated by means of object detection tasks, in which the goal is to recognize
and localize predefined objects in a stream of data. Usually, state-of-the-art
object detectors based on neural networks are highly optimized for good weather
conditions, since the well-known benchmarks only consist of sensor data
recorded in optimal weather conditions. Therefore, the performance of these
approaches decreases enormously or even fails in adverse weather conditions. In
this work, different sensor fusion architectures are compared for good and
adverse weather conditions for finding the optimal fusion architecture for
diverse weather situations. A new training strategy is also introduced such
that the performance of the object detector is greatly enhanced in adverse
weather scenarios or if a sensor fails. Furthermore, the paper responds to the
question if the detection accuracy can be increased further by providing the
neural network with a-priori knowledge such as the spatial calibration of the
sensors.
Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsing
Authors: Chen Liang, Mohammad Norouzi, Jonathan Berant, Quoc Le, Ni Lao
Date: 2018-07-06 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Combinatorial Optimization, Program Synthesis, Semantic Parsing, Structured Prediction

We present Memory Augmented Policy Optimization (MAPO), a simple and novel
way to leverage a memory buffer of promising trajectories to reduce the
variance of policy gradient estimate. MAPO is applicable to deterministic
environments with discrete actions, such as structured prediction and
combinatorial optimization tasks. We express the expected return objective as a
weighted sum of two terms: an expectation over the high-reward trajectories
inside the memory buffer, and a separate expectation over trajectories outside
the buffer. To make an efficient algorithm of MAPO, we propose: (1) memory
weight clipping to accelerate and stabilize training; (2) systematic
exploration to discover high-reward trajectories; (3) distributed sampling from
inside and outside of the memory buffer to scale up training. MAPO improves the
sample efficiency and robustness of policy gradient, especially on tasks with
sparse rewards. We evaluate MAPO on weakly supervised program synthesis from
natural language (semantic parsing). On the WikiTableQuestions benchmark, we
improve the state-of-the-art by 2.6%, achieving an accuracy of 46.3%. On the
WikiSQL benchmark, MAPO achieves an accuracy of 74.9% with only weak
supervision, outperforming several strong baselines with full supervision. Our
source code is available at
https://github.com/crazydonkey200/neural-symbolic-machines
JUMPER: Learning When to Make Classification Decisions in Reading
Authors: Xianggen Liu, Lili Mou, Haotian Cui, Zhengdong Lu, Sen Song
Date: 2018-07-06 00:00:00
Tasks: Classification, General Classification, Reinforcement Learning, text-classification, Text Classification

In early years, text classification is typically accomplished by
feature-based machine learning models; recently, deep neural networks, as a
powerful learning machine, make it possible to work with raw input as the text
stands. However, exiting end-to-end neural networks lack explicit
interpretation of the prediction. In this paper, we propose a novel framework,
JUMPER, inspired by the cognitive process of text reading, that models text
classification as a sequential decision process. Basically, JUMPER is a neural
system that scans a piece of text sequentially and makes classification
decisions at the time it wishes. Both the classification result and when to
make the classification are part of the decision process, which is controlled
by a policy network and trained with reinforcement learning. Experimental
results show that a properly trained JUMPER has the following properties: (1)
It can make decisions whenever the evidence is enough, therefore reducing total
text reading by 30-40% and often finding the key rationale of prediction. (2)
It achieves classification accuracy better than or comparable to
state-of-the-art models in several benchmark and industrial datasets.
Neural Document Summarization by Jointly Learning to Score and Select Sentences
Authors: Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, Tiejun Zhao
Date: 2018-07-06 00:00:00
Proceeding: ACL 2018 7
Tasks: Document Summarization, Extractive Document Summarization, Extractive Summarization, Extractive Text Summarization, Sentence

Sentence scoring and sentence selection are two main steps in extractive
document summarization systems. However, previous works treat them as two
separated subtasks. In this paper, we present a novel end-to-end neural network
framework for extractive document summarization by jointly learning to score
and select sentences. It first reads the document sentences with a hierarchical
encoder to obtain the representation of sentences. Then it builds the output
summary by extracting sentences one by one. Different from previous methods,
our approach integrates the selection strategy into the scoring model, which
directly predicts the relative importance given previously selected sentences.
Experiments on the CNN/Daily Mail dataset show that the proposed framework
significantly outperforms the state-of-the-art extractive summarization models.
A survey on policy search algorithms for learning robot controllers in a handful of trials
Authors: Konstantinos Chatzilygeroudis, Vassilis Vassiliades, Freek Stulp, Sylvain Calinon, Jean-Baptiste Mouret
Date: 2018-07-06 00:00:00
Tasks: Bayesian Optimization, Reinforcement Learning

Most policy search algorithms require thousands of training episodes to find an effective policy, which is often infeasible with a physical robot. This survey article focuses on the extreme other end of the spectrum: how can a robot adapt with only a handful of trials (a dozen) and a few minutes? By analogy with the word "big-data", we refer to this challenge as "micro-data reinforcement learning". We show that a first strategy is to leverage prior knowledge on the policy structure (e.g., dynamic movement primitives), on the policy parameters (e.g., demonstrations), or on the dynamics (e.g., simulators). A second strategy is to create data-driven surrogate models of the expected reward (e.g., Bayesian optimization) or the dynamical model (e.g., model-based policy search), so that the policy optimizer queries the model instead of the real system. Overall, all successful micro-data algorithms combine these two strategies by varying the kind of model and prior knowledge. The current scientific challenges essentially revolve around scaling up to complex robots (e.g., humanoids), designing generic priors, and optimizing the computing time.
Sequential Copying Networks
Authors: Qingyu Zhou, Nan Yang, Furu Wei, Ming Zhou
Date: 2018-07-06 00:00:00
Tasks: Decoder, Question Generation, Question-Generation, Sentence, Sentence Summarization, Text Generation

Copying mechanism shows effectiveness in sequence-to-sequence based neural
network models for text generation tasks, such as abstractive sentence
summarization and question generation. However, existing works on modeling
copying or pointing mechanism only considers single word copying from the
source sentences. In this paper, we propose a novel copying framework, named
Sequential Copying Networks (SeqCopyNet), which not only learns to copy single
words, but also copies sequences from the input sentence. It leverages the
pointer networks to explicitly select a sub-span from the source side to target
side, and integrates this sequential copying mechanism to the generation
process in the encoder-decoder paradigm. Experiments on abstractive sentence
summarization and question generation tasks show that the proposed SeqCopyNet
can copy meaningful spans and outperforms the baseline models.
Combinatorial Bandits for Incentivizing Agents with Dynamic Preferences
Authors: Tanner Fiez, Shreyas Sekar, Liyuan Zheng, Lillian J. Ratliff
Date: 2018-07-06 00:00:00

The design of personalized incentives or recommendations to improve user
engagement is gaining prominence as digital platform providers continually
emerge. We propose a multi-armed bandit framework for matching incentives to
users, whose preferences are unknown a priori and evolving dynamically in time,
in a resource constrained environment. We design an algorithm that combines
ideas from three distinct domains: (i) a greedy matching paradigm, (ii) the
upper confidence bound algorithm (UCB) for bandits, and (iii) mixing times from
the theory of Markov chains. For this algorithm, we provide theoretical bounds
on the regret and demonstrate its performance via both synthetic and realistic
(matching supply and demand in a bike-sharing platform) examples.
Combining SLAM with muti-spectral photometric stereo for real-time dense 3D reconstruction
Authors: Yuanhong Xu, Pei Dong, Junyu Dong, Lin Qi
Date: 2018-07-06 00:00:00
Tasks: 3D Reconstruction

Obtaining dense 3D reconstrution with low computational cost is one of the
important goals in the field of SLAM. In this paper we propose a dense 3D
reconstruction framework from monocular multispectral video sequences using
jointly semi-dense SLAM and Multispectral Photometric Stereo approaches.
Starting from multispectral video, SALM (a) reconstructs a semi-dense 3D shape
that will be densified;(b) recovers relative sparse depth map that is then fed
as prioris into optimization-based multispectral photometric stereo for a more
accurate dense surface normal recovery;(c)obtains camera pose that is
subsequently used for conversion of view in the process of fusion where we
combine the relative sparse point cloud with the dense surface normal using the
automated cross-scale fusion method proposed in this paper to get a dense point
cloud with subtle texture information. Experiments show that our method can
effectively obtain denser 3D reconstructions.
Sliced Recurrent Neural Networks
Authors: Zeping Yu, Gongshen Liu
Date: 2018-07-06 00:00:00
Proceeding: COLING 2018 8
Tasks: Sentiment Analysis

Recurrent neural networks have achieved great success in many NLP tasks.
However, they have difficulty in parallelization because of the recurrent
structure, so it takes much time to train RNNs. In this paper, we introduce
sliced recurrent neural networks (SRNNs), which could be parallelized by
slicing the sequences into many subsequences. SRNNs have the ability to obtain
high-level information through multiple layers with few extra parameters. We
prove that the standard RNN is a special case of the SRNN when we use linear
activation functions. Without changing the recurrent units, SRNNs are 136 times
as fast as standard RNNs and could be even faster when we train longer
sequences. Experiments on six largescale sentiment analysis datasets show that
SRNNs achieve better performance than standard RNNs.
Differentially Private Online Submodular Optimization
Authors: Adrian Rivera Cardoso, Rachel Cummings
Date: 2018-07-06 00:00:00

In this paper we develop the first algorithms for online submodular
minimization that preserve differential privacy under full information feedback
and bandit feedback. A sequence of $T$ submodular functions over a collection
of $n$ elements arrive online, and at each timestep the algorithm must choose a
subset of $[n]$ before seeing the function. The algorithm incurs a cost equal
to the function evaluated on the chosen set, and seeks to choose a sequence of
sets that achieves low expected regret.
  Our first result is in the full information setting, where the algorithm can
observe the entire function after making its decision at each timestep. We give
an algorithm in this setting that is $\epsilon$-differentially private and
achieves expected regret
$\tilde{O}\left(\frac{n^{3/2}\sqrt{T}}{\epsilon}\right)$. This algorithm works
by relaxing submodular function to a convex function using the Lovasz
extension, and then simulating an algorithm for differentially private online
convex optimization.
  Our second result is in the bandit setting, where the algorithm can only see
the cost incurred by its chosen set, and does not have access to the entire
function. This setting is significantly more challenging because the algorithm
does not receive enough information to compute the Lovasz extension or its
subgradients. Instead, we construct an unbiased estimate using a single-point
estimation, and then simulate private online convex optimization using this
estimate. Our algorithm using bandit feedback is $\epsilon$-differentially
private and achieves expected regret
$\tilde{O}\left(\frac{n^{3/2}T^{3/4}}{\epsilon}\right)$.
Outperforming Good-Turing: Preliminary Report
Authors: Amichai Painsky, Meir Feder
Date: 2018-07-06 00:00:00

Estimating a large alphabet probability distribution from a limited number of
samples is a fundamental problem in machine learning and statistics. A variety
of estimation schemes have been proposed over the years, mostly inspired by the
early work of Laplace and the seminal contribution of Good and Turing. One of
the basic assumptions shared by most commonly-used estimators is the unique
correspondence between the symbol's sample frequency and its estimated
probability. In this work we tackle this paradigmatic assumption; we claim that
symbols with "similar" frequencies shall be assigned the same estimated
probability value. This way we regulate the number of parameters and improve
generalization. In this preliminary report we show that by applying an ensemble
of such regulated estimators, we introduce a dramatic enhancement in the
estimation accuracy (typically up to 50%), compared to currently known methods.
An implementation of our suggested method is publicly available at the first
author's web-page.