pwc-archive/papers-with-abstracts huggingface dataset:
GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models
Authors: Jiaxuan You, Rex Ying, Xiang Ren, William L. Hamilton, Jure Leskovec
Date: 2018-02-24 00:00:00
Proceeding: ICML 2018 7
Tasks: Graph Generation

Modeling and generating graphs is fundamental for studying networks in
biology, engineering, and social sciences. However, modeling complex
distributions over graphs and then efficiently sampling from these
distributions is challenging due to the non-unique, high-dimensional nature of
graphs and the complex, non-local dependencies that exist between edges in a
given graph. Here we propose GraphRNN, a deep autoregressive model that
addresses the above challenges and approximates any distribution of graphs with
minimal assumptions about their structure. GraphRNN learns to generate graphs
by training on a representative set of graphs and decomposes the graph
generation process into a sequence of node and edge formations, conditioned on
the graph structure generated so far.
  In order to quantitatively evaluate the performance of GraphRNN, we introduce
a benchmark suite of datasets, baselines and novel evaluation metrics based on
Maximum Mean Discrepancy, which measure distances between sets of graphs. Our
experiments show that GraphRNN significantly outperforms all baselines,
learning to generate diverse graphs that match the structural characteristics
of a target set, while also scaling to graphs 50 times larger than previous
deep models.
Learning to track on-the-fly using a particle filter with annealed- weighted QPSO modeled after a singular Dirac delta potential
Authors: Saptarshi Sengupta, Richard Alan Peters II
Date: 2018-06-04 00:00:00
Tasks: Visual Tracking

This paper proposes an evolutionary Particle Filter with a memory guided
proposal step size update and an improved, fully-connected Quantum-behaved
Particle Swarm Optimization (QPSO) resampling scheme for visual tracking
applications. The proposal update step uses importance weights proportional to
velocities encountered in recent memory to limit the swarm movement within
probable regions of interest. The QPSO resampling scheme uses a fitness
weighted mean best update to bias the swarm towards the fittest section of
particles while also employing a simulated annealing operator to avoid subpar
fine tune during latter course of iterations. By moving particles closer to
high likelihood landscapes of the posterior distribution using such constructs,
the sample impoverishment problem that plagues the Particle Filter is mitigated
to a great extent. Experimental results using benchmark sequences imply that
the proposed method outperforms competitive candidate trackers such as the
Particle Filter and the traditional Particle Swarm Optimization based Particle
Filter on a suite of tracker performance indices.
Hierarchical Text Generation and Planning for Strategic Dialogue
Authors: Denis Yarats, Mike Lewis
Date: 2017-12-15 00:00:00
Proceeding: ICML 2018 7
Tasks: Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sentence, Text Generation

End-to-end models for goal-orientated dialogue are challenging to train,
because linguistic and strategic aspects are entangled in latent state vectors.
We introduce an approach to learning representations of messages in dialogues
by maximizing the likelihood of subsequent sentences and actions, which
decouples the semantics of the dialogue utterance from its linguistic
realization. We then use these latent sentence representations for hierarchical
language generation, planning and reinforcement learning. Experiments show that
our approach increases the end-task reward achieved by the model, improves the
effectiveness of long-term planning using rollouts, and allows self-play
reinforcement learning to improve decision making without diverging from human
language. Our hierarchical latent-variable model outperforms previous work both
linguistically and strategically.
New And Surprising Ways to Be Mean. Adversarial NPCs with Coupled Empowerment Minimisation
Authors: Christian Guckelsberger, Christoph Salge, Julian Togelius
Date: 2018-06-04 00:00:00

Creating Non-Player Characters (NPCs) that can react robustly to unforeseen
player behaviour or novel game content is difficult and time-consuming. This
hinders the design of believable characters, and the inclusion of NPCs in games
that rely heavily on procedural content generation. We have previously
addressed this challenge by means of empowerment, a model of intrinsic
motivation, and demonstrated how a coupled empowerment maximisation (CEM)
policy can yield generic, companion-like behaviour. In this paper, we extend
the CEM framework with a minimisation policy to give rise to adversarial
behaviour. We conduct a qualitative, exploratory study in a dungeon-crawler
game, demonstrating that CEM can exploit the affordances of different content
facets in adaptive adversarial behaviour without modifications to the policy.
Changes to the level design, underlying mechanics and our character's actions
do not threaten our NPC's robustness, but yield new and surprising ways to be
mean.
A General Framework for Bandit Problems Beyond Cumulative Objectives
Authors: Asaf Cassel, Shie Mannor, Assaf Zeevi
Date: 2018-06-04 00:00:00
Tasks: Multi-Armed Bandits

The stochastic multi-armed bandit (MAB) problem is a common model for sequential decision problems. In the standard setup, a decision maker has to choose at every instant between several competing arms, each of them provides a scalar random variable, referred to as a "reward." Nearly all research on this topic considers the total cumulative reward as the criterion of interest. This work focuses on other natural objectives that cannot be cast as a sum over rewards, but rather more involved functions of the reward stream. Unlike the case of cumulative criteria, in the problems we study here the oracle policy, that knows the problem parameters a priori and is used to "center" the regret, is not trivial. We provide a systematic approach to such problems, and derive general conditions under which the oracle policy is sufficiently tractable to facilitate the design of optimism-based (upper confidence bound) learning policies. These conditions elucidate an interesting interplay between the arm reward distributions and the performance metric. Our main findings are illustrated for several commonly used objectives such as conditional value-at-risk, mean-variance trade-offs, Sharpe-ratio, and more.
Factorized Adversarial Networks for Unsupervised Domain Adaptation
Authors: Jian Ren, Jianchao Yang, Ning Xu, David J. Foran
Date: 2018-06-04 00:00:00
Tasks: Domain Adaptation, General Classification, image-classification, Image Classification, Unsupervised Domain Adaptation

In this paper, we propose Factorized Adversarial Networks (FAN) to solve
unsupervised domain adaptation problems for image classification tasks. Our
networks map the data distribution into a latent feature space, which is
factorized into a domain-specific subspace that contains domain-specific
characteristics and a task-specific subspace that retains category information,
for both source and target domains, respectively. Unsupervised domain
adaptation is achieved by adversarial training to minimize the discrepancy
between the distributions of two task-specific subspaces from source and target
domains. We demonstrate that the proposed approach outperforms state-of-the-art
methods on multiple benchmark datasets used in the literature for unsupervised
domain adaptation. Furthermore, we collect two real-world tagging datasets that
are much larger than existing benchmark datasets, and get significant
improvement upon baselines, proving the practical value of our approach.
Investigating Human Priors for Playing Video Games
Authors: Rachit Dubey, Pulkit Agrawal, Deepak Pathak, Thomas L. Griffiths, Alexei A. Efros
Date: 2018-02-28 00:00:00
Proceeding: ICML 2018 7
Tasks: Decision Making

What makes humans so good at solving seemingly complex video games? Unlike
computers, humans bring in a great deal of prior knowledge about the world,
enabling efficient decision making. This paper investigates the role of human
priors for solving video games. Given a sample game, we conduct a series of
ablation studies to quantify the importance of various priors on human
performance. We do this by modifying the video game environment to
systematically mask different types of visual information that could be used by
humans as priors. We find that removal of some prior knowledge causes a drastic
degradation in the speed with which human players solve the game, e.g. from 2
minutes to over 20 minutes. Furthermore, our results indicate that general
priors, such as the importance of objects and visual consistency, are critical
for efficient game-play. Videos and the game manipulations are available at
https://rach0012.github.io/humanRL_website/
Learning Visually Grounded Sentence Representations
Authors: Douwe Kiela, Alexis Conneau, Allan Jabri, Maximilian Nickel
Date: 2017-07-19 00:00:00
Proceeding: NAACL 2018 6
Tasks: Language Modelling, Representation Learning, Retrieval, Sentence

We introduce a variety of models, trained on a supervised image captioning
corpus to predict the image features for a given caption, to perform sentence
representation grounding. We train a grounded sentence encoder that achieves
good performance on COCO caption and image retrieval and subsequently show that
this encoder can successfully be transferred to various NLP tasks, with
improved performance over text-only models. Lastly, we analyze the contribution
of grounding, and show that word embeddings learned by this system outperform
non-grounded ones.
Adversarial Reinforcement Learning Framework for Benchmarking Collision Avoidance Mechanisms in Autonomous Vehicles
Authors: Vahid Behzadan, Arslan Munir
Date: 2018-06-04 00:00:00
Tasks: Autonomous Navigation, Autonomous Vehicles, Benchmarking, Collision Avoidance, Deep Reinforcement Learning, Motion Planning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

With the rapidly growing interest in autonomous navigation, the body of
research on motion planning and collision avoidance techniques has enjoyed an
accelerating rate of novel proposals and developments. However, the complexity
of new techniques and their safety requirements render the bulk of current
benchmarking frameworks inadequate, thus leaving the need for efficient
comparison techniques unanswered. This work proposes a novel framework based on
deep reinforcement learning for benchmarking the behavior of collision
avoidance mechanisms under the worst-case scenario of dealing with an optimal
adversarial agent, trained to drive the system into unsafe states. We describe
the architecture and flow of this framework as a benchmarking solution, and
demonstrate its efficacy via a practical case study of comparing the
reliability of two collision avoidance mechanisms in response to intentional
collision attempts.
Playing Atari with Six Neurons
Authors: Giuseppe Cuccu, Julian Togelius, Philippe Cudre-Mauroux
Date: 2018-06-04 00:00:00
Tasks: Atari Games, Decision Making, Deep Reinforcement Learning, Quantization, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep reinforcement learning, applied to vision-based problems like Atari
games, maps pixels directly to actions; internally, the deep neural network
bears the responsibility of both extracting useful information and making
decisions based on it. By separating the image processing from decision-making,
one could better understand the complexity of each task, as well as potentially
find smaller policy representations that are easier for humans to understand
and may generalize better. To this end, we propose a new method for learning
policies and compact state representations separately but simultaneously for
policy approximation in reinforcement learning. State representations are
generated by an encoder based on two novel algorithms: Increasing Dictionary
Vector Quantization makes the encoder capable of growing its dictionary size
over time, to address new observations as they appear in an open-ended
online-learning context; Direct Residuals Sparse Coding encodes observations by
disregarding reconstruction error minimization, and aiming instead for highest
information inclusion. The encoder autonomously selects observations online to
train on, in order to maximize code sparsity. As the dictionary size increases,
the encoder produces increasingly larger inputs for the neural network: this is
addressed by a variation of the Exponential Natural Evolution Strategies
algorithm which adapts its probability distribution dimensionality along the
run. We test our system on a selection of Atari games using tiny neural
networks of only 6 to 18 neurons (depending on the game's controls). These are
still capable of achieving results comparable---and occasionally superior---to
state-of-the-art techniques which use two orders of magnitude more neurons.
Modeling Camera Effects to Improve Visual Learning from Synthetic Data
Authors: Alexandra Carlson, Katherine A. Skinner, Ram Vasudevan, Matthew Johnson-Roberson
Date: 2018-03-21 00:00:00
Tasks: object-detection, Object Detection

Recent work has focused on generating synthetic imagery to increase the size
and variability of training data for learning visual tasks in urban scenes.
This includes increasing the occurrence of occlusions or varying environmental
and weather effects. However, few have addressed modeling variation in the
sensor domain. Sensor effects can degrade real images, limiting
generalizability of network performance on visual tasks trained on synthetic
data and tested in real environments. This paper proposes an efficient,
automatic, physically-based augmentation pipeline to vary sensor effects
--chromatic aberration, blur, exposure, noise, and color cast-- for synthetic
imagery. In particular, this paper illustrates that augmenting synthetic
training datasets with the proposed pipeline reduces the domain gap between
synthetic and real domains for the task of object detection in urban driving
scenes.
gprHOG and the popularity of Histogram of Oriented Gradients (HOG) for Buried Threat Detection in Ground-Penetrating Radar
Authors: Daniel Reichman, Leslie M. Collins, Jordan M. Malof
Date: 2018-06-04 00:00:00
Tasks: GPR

Substantial research has been devoted to the development of algorithms that
automate buried threat detection (BTD) with ground penetrating radar (GPR)
data, resulting in a large number of proposed algorithms. One popular algorithm
GPR-based BTD, originally applied by Torrione et al., 2012, is the Histogram of
Oriented Gradients (HOG) feature. In a recent large-scale comparison among five
veteran institutions, a modified version of HOG referred to here as "gprHOG",
performed poorly compared to other modern algorithms. In this paper, we provide
experimental evidence demonstrating that the modifications to HOG that comprise
gprHOG result in a substantially better-performing algorithm. The results here,
in conjunction with the large-scale algorithm comparison, suggest that HOG is
not competitive with modern GPR-based BTD algorithms. Given HOG's popularity,
these results raise some questions about many existing studies, and suggest
gprHOG (and especially HOG) should be employed with caution in future studies.
Importance Sampling Policy Evaluation with an Estimated Behavior Policy
Authors: Josiah P. Hanna, Scott Niekum, Peter Stone
Date: 2018-06-04 00:00:00
Tasks: Off-policy evaluation

We consider the problem of off-policy evaluation in Markov decision processes. Off-policy evaluation is the task of evaluating the expected return of one policy with data generated by a different, behavior policy. Importance sampling is a technique for off-policy evaluation that re-weights off-policy returns to account for differences in the likelihood of the returns between the two policies. In this paper, we study importance sampling with an estimated behavior policy where the behavior policy estimate comes from the same set of data used to compute the importance sampling estimate. We find that this estimator often lowers the mean squared error of off-policy evaluation compared to importance sampling with the true behavior policy or using a behavior policy that is estimated from a separate data set. Intuitively, estimating the behavior policy in this way corrects for error due to sampling in the action-space. Our empirical results also extend to other popular variants of importance sampling and show that estimating a non-Markovian behavior policy can further lower large-sample mean squared error even when the true behavior policy is Markovian.
Design of optimal illumination patterns in single-pixel imaging using image dictionaries
Authors: Jun Feng, Shuming Jiao, Yang Gao, Ting Lei, Xiaocong Yuan
Date: 2018-06-04 00:00:00

Single-pixel imaging (SPI) has a major drawback that many sequential illuminations are required for capturing one single image with long acquisition time. Basis illumination patterns such as Fourier patterns and Hadamard patterns can achieve much better imaging efficiency than random patterns. But the performance is still sub-optimal since the basis patterns are fixed and non-adaptive for varying object images. This Letter proposes a novel scheme for designing and optimizing the illumination patterns adaptively from an image dictionary by extracting the common image features using principal component analysis (PCA). Simulation and experimental results reveal that our proposed scheme outperforms conventional Fourier SPI in terms of imaging efficiency.
Computing the Spatial Probability of Inclusion inside Partial Contours for Computer Vision Applications
Authors: Dominique Beaini, Sofiane Achiche, Fabrice Nonez, Maxime Raison
Date: 2018-06-04 00:00:00
Tasks: Clustering, Edge Detection, Image Segmentation, object-detection, Object Detection, Semantic Segmentation

In Computer Vision, edge detection is one of the favored approaches for feature and object detection in images since it provides information about their objects boundaries. Other region-based approaches use probabilistic analysis such as clustering and Markov random fields, but those methods cannot be used to analyze edges and their interaction. In fact, only image segmentation can produce regions based on edges, but it requires thresholding by simply separating the regions into binary in-out information. Hence, there is currently a gap between edge-based and region-based algorithms, since edges cannot be used to study the properties of a region and vice versa. The objective of this paper is to present a novel spatial probability analysis that allows determining the probability of inclusion inside a set of partial contours (strokes). To answer this objective, we developed a new approach that uses electromagnetic convolutions and repulsion optimization to compute the required probabilities. Hence, it becomes possible to generate a continuous space of probability based only on the edge information, thus bridging the gap between the edge-based methods and the region-based methods. The developed method is consistent with the fundamental properties of inclusion probabilities and its results are validated by comparing an image with the probability-based estimation given by our algorithm. The method can also be generalized to take into consideration the intensity of the edges or to be used for 3D shapes. This is the first documented method that allows computing a space of probability based on interacting edges, which opens the path to broader applications such as image segmentation and contour completion.
Less is More: Simultaneous View Classification and Landmark Detection for Abdominal Ultrasound Images
Authors: Zhoubing Xu, Yuankai Huo, Jin-Hyeong Park, Bennett Landman, Andy Milkowski, Sasa Grbic, Shaohua Zhou
Date: 2018-05-25 00:00:00
Tasks: Classification, General Classification, Multi-Task Learning

An abdominal ultrasound examination, which is the most common ultrasound
examination, requires substantial manual efforts to acquire standard abdominal
organ views, annotate the views in texts, and record clinically relevant organ
measurements. Hence, automatic view classification and landmark detection of
the organs can be instrumental to streamline the examination workflow. However,
this is a challenging problem given not only the inherent difficulties from the
ultrasound modality, e.g., low contrast and large variations, but also the
heterogeneity across tasks, i.e., one classification task for all views, and
then one landmark detection task for each relevant view. While convolutional
neural networks (CNN) have demonstrated more promising outcomes on ultrasound
image analytics than traditional machine learning approaches, it becomes
impractical to deploy multiple networks (one for each task) due to the limited
computational and memory resources on most existing ultrasound scanners. To
overcome such limits, we propose a multi-task learning framework to handle all
the tasks by a single network. This network is integrated to perform view
classification and landmark detection simultaneously; it is also equipped with
global convolutional kernels, coordinate constraints, and a conditional
adversarial module to leverage the performances. In an experimental study based
on 187,219 ultrasound images, with the proposed simplified approach we achieve
(1) view classification accuracy better than the agreement between two clinical
experts and (2) landmark-based measurement errors on par with inter-user
variability. The multi-task approach also benefits from sharing the feature
extraction during the training process across all tasks and, as a result,
outperforms the approaches that address each task individually.
Backdrop: Stochastic Backpropagation
Authors: Siavash Golkar, Kyle Cranmer
Date: 2018-06-04 00:00:00
Proceeding: ICLR 2019 5

We introduce backdrop, a flexible and simple-to-implement method, intuitively
described as dropout acting only along the backpropagation pipeline. Backdrop
is implemented via one or more masking layers which are inserted at specific
points along the network. Each backdrop masking layer acts as the identity in
the forward pass, but randomly masks parts of the backward gradient
propagation. Intuitively, inserting a backdrop layer after any convolutional
layer leads to stochastic gradients corresponding to features of that scale.
Therefore, backdrop is well suited for problems in which the data have a
multi-scale, hierarchical structure. Backdrop can also be applied to problems
with non-decomposable loss functions where standard SGD methods are not well
suited. We perform a number of experiments and demonstrate that backdrop leads
to significant improvements in generalization.