pwc-archive/papers-with-abstracts huggingface dataset:
Structured Control Nets for Deep Reinforcement Learning
Authors: Mario Srouji, Jian Zhang, Ruslan Salakhutdinov
Date: 2018-02-22 00:00:00
Proceeding: ICML 2018 7
Tasks: Decision Making, Deep Reinforcement Learning, MuJoCo, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

In recent years, Deep Reinforcement Learning has made impressive advances in
solving several important benchmark problems for sequential decision making.
Many control applications use a generic multilayer perceptron (MLP) for
non-vision parts of the policy network. In this work, we propose a new neural
network architecture for the policy network representation that is simple yet
effective. The proposed Structured Control Net (SCN) splits the generic MLP
into two separate sub-modules: a nonlinear control module and a linear control
module. Intuitively, the nonlinear control is for forward-looking and global
control, while the linear control stabilizes the local dynamics around the
residual of global control. We hypothesize that this will bring together the
benefits of both linear and nonlinear policies: improve training sample
efficiency, final episodic reward, and generalization of learned policy, while
requiring a smaller network and being generally applicable to different
training methods. We validated our hypothesis with competitive results on
simulations from OpenAI MuJoCo, Roboschool, Atari, and a custom 2D urban
driving environment, with various ablation and generalization tests, trained
with multiple black-box and policy gradient training methods. The proposed
architecture has the potential to improve upon broader control tasks by
incorporating problem specific priors into the architecture. As a case study,
we demonstrate much improved performance for locomotion tasks by emulating the
biological central pattern generators (CPGs) as the nonlinear part of the
architecture.
Sleep-deprived Fatigue Pattern Analysis using Large-Scale Selfies from Social Med
Authors: Xuefeng Peng, Jiebo Luo, Catherine Glenn, Li-Kai Chi, Jingyao Zhan
Date: 2018-02-22 00:00:00

The complexities of fatigue have drawn much attention from researchers across
various disciplines. Short-term fatigue may cause safety issue while driving;
thus, dynamic systems were designed to track driver fatigue. Long-term fatigue
could lead to chronic syndromes, and eventually affect individuals physical and
psychological health. Traditional methodologies of evaluating fatigue not only
require sophisticated equipment but also consume enormous time. In this paper,
we attempt to develop a novel and efficient method to predict individual's
fatigue rate by scrutinizing human facial cues. Our goal is to predict fatigue
rate based on a selfie. To associate the fatigue rate with user behaviors, we
have collected nearly 1-million timeline posts from 10,480 users on Instagram.
We first detect all the faces and identify their demographics using automatic
algorithms. Next, we investigate the fatigue distribution by weekday over
different age, gender, and ethnic groups. This work represents a promising way
to assess sleep-deprived fatigue, and our study provides a viable and efficient
computational framework for user fatigue modeling in large-scale via social
media.
Trust-PCL: An Off-Policy Trust Region Method for Continuous Control
Authors: Ofir Nachum, Mohammad Norouzi, Kelvin Xu, Dale Schuurmans
Date: 2017-07-06 00:00:00
Proceeding: ICLR 2018 1
Tasks: continuous-control, Continuous Control, Reinforcement Learning, Reinforcement Learning (RL)

Trust region methods, such as TRPO, are often used to stabilize policy
optimization algorithms in reinforcement learning (RL). While current trust
region strategies are effective for continuous control, they typically require
a prohibitively large amount of on-policy interaction with the environment. To
address this problem, we propose an off-policy trust region method, Trust-PCL.
The algorithm is the result of observing that the optimal policy and state
values of a maximum reward objective with a relative-entropy regularizer
satisfy a set of multi-step pathwise consistencies along any path. Thus,
Trust-PCL is able to maintain optimization stability while exploiting
off-policy data to improve sample efficiency. When evaluated on a number of
continuous control tasks, Trust-PCL improves the solution quality and sample
efficiency of TRPO.
Identifying Sources and Sinks in the Presence of Multiple Agents with Gaussian Process Vector Calculus
Authors: Adam D. Cobb, Richard Everett, Andrew Markham, Stephen J. Roberts
Date: 2018-02-22 00:00:00
Tasks: Gaussian Processes

In systems of multiple agents, identifying the cause of observed agent
dynamics is challenging. Often, these agents operate in diverse, non-stationary
environments, where models rely on hand-crafted environment-specific features
to infer influential regions in the system's surroundings. To overcome the
limitations of these inflexible models, we present GP-LAPLACE, a technique for
locating sources and sinks from trajectories in time-varying fields. Using
Gaussian processes, we jointly infer a spatio-temporal vector field, as well as
canonical vector calculus operations on that field. Notably, we do this from
only agent trajectories without requiring knowledge of the environment, and
also obtain a metric for denoting the significance of inferred causal features
in the environment by exploiting our probabilistic method. To evaluate our
approach, we apply it to both synthetic and real-world GPS data, demonstrating
the applicability of our technique in the presence of multiple agents, as well
as its superiority over existing methods.
Content-Based Citation Recommendation
Authors: Chandra Bhagavatula, Sergey Feldman, Russell Power, Waleed Ammar
Date: 2018-02-22 00:00:00
Proceeding: NAACL 2018 6
Tasks: Articles, Citation Recommendation

We present a content-based method for recommending citations in an academic
paper draft. We embed a given query document into a vector space, then use its
nearest neighbors as candidates, and rerank the candidates using a
discriminative model trained to distinguish between observed and unobserved
citations. Unlike previous work, our method does not require metadata such as
author names which can be missing, e.g., during the peer review process.
Without using metadata, our method outperforms the best reported results on
PubMed and DBLP datasets with relative improvements of over 18% in F1@20 and
over 22% in MRR. We show empirically that, although adding metadata improves
the performance on standard metrics, it favors self-citations which are less
useful in a citation recommendation setup. We release an online portal
(http://labs.semanticscholar.org/citeomatic/) for citation recommendation based
on our method, and a new dataset OpenCorpus of 7 million research articles to
facilitate future research on this task.
DeepASL: Enabling Ubiquitous and Non-Intrusive Word and Sentence-Level Sign Language Translation
Authors: Biyi Fang, Jillian Co, Mi Zhang
Date: 2018-02-21 00:00:00
Tasks: Sentence, Sign Language Translation, Translation

There is an undeniable communication barrier between deaf people and people
with normal hearing ability. Although innovations in sign language translation
technology aim to tear down this communication barrier, the majority of
existing sign language translation systems are either intrusive or constrained
by resolution or ambient lighting conditions. Moreover, these existing systems
can only perform single-sign ASL translation rather than sentence-level
translation, making them much less useful in daily-life communication
scenarios. In this work, we fill this critical gap by presenting DeepASL, a
transformative deep learning-based sign language translation technology that
enables ubiquitous and non-intrusive American Sign Language (ASL) translation
at both word and sentence levels. DeepASL uses infrared light as its sensing
mechanism to non-intrusively capture the ASL signs. It incorporates a novel
hierarchical bidirectional deep recurrent neural network (HB-RNN) and a
probabilistic framework based on Connectionist Temporal Classification (CTC)
for word-level and sentence-level ASL translation respectively. To evaluate its
performance, we have collected 7,306 samples from 11 participants, covering 56
commonly used ASL words and 100 ASL sentences. DeepASL achieves an average
94.5% word-level translation accuracy and an average 8.2% word error rate on
translating unseen ASL sentences. Given its promising performance, we believe
DeepASL represents a significant step towards breaking the communication
barrier between deaf people and hearing majority, and thus has the significant
potential to fundamentally change deaf people's lives.
Unicorn: Continual Learning with a Universal, Off-policy Agent
Authors: Daniel J. Mankowitz, Augustin Žídek, André Barreto, Dan Horgan, Matteo Hessel, John Quan, Junhyuk Oh, Hado van Hasselt, David Silver, Tom Schaul
Date: 2018-02-22 00:00:00
Tasks: Continual Learning, Lifelong learning

Some real-world domains are best characterized as a single task, but for
others this perspective is limiting. Instead, some tasks continually grow in
complexity, in tandem with the agent's competence. In continual learning, also
referred to as lifelong learning, there are no explicit task boundaries or
curricula. As learning agents have become more powerful, continual learning
remains one of the frontiers that has resisted quick progress. To test
continual learning capabilities we consider a challenging 3D domain with an
implicit sequence of tasks and sparse rewards. We propose a novel agent
architecture called Unicorn, which demonstrates strong continual learning and
outperforms several baseline agents on the proposed domain. The agent achieves
this by jointly representing and learning multiple policies efficiently, using
a parallel off-policy learning setup.
A Spatial Mapping Algorithm with Applications in Deep Learning-Based Structure Classification
Authors: Thomas Corcoran, Rafael Zamora-Resendiz, Xinlian Liu, Silvia Crivelli
Date: 2018-02-07 00:00:00
Tasks: Anomaly Detection, Benchmarking, Dimensionality Reduction, General Classification

Convolutional Neural Network (CNN)-based machine learning systems have made
breakthroughs in feature extraction and image recognition tasks in two
dimensions (2D). Although there is significant ongoing work to apply CNN
technology to domains involving complex 3D data, the success of such efforts
has been constrained, in part, by limitations in data representation
techniques. Most current approaches rely upon low-resolution 3D models,
strategic limitation of scope in the 3D space, or the application of lossy
projection techniques to allow for the use of 2D CNNs. To address this issue,
we present a mapping algorithm that converts 3D structures to 2D and 1D data
grids by mapping a traversal of a 3D space-filling curve to the traversal of
corresponding 2D and 1D curves. We explore the performance of 2D and 1D CNNs
trained on data encoded with our method versus comparable volumetric CNNs
operating upon raw 3D data from a popular benchmarking dataset. Our experiments
demonstrate that both 2D and 1D representations of 3D data generated via our
method preserve a significant proportion of the 3D data's features in forms
learnable by CNNs. Furthermore, we demonstrate that our method of encoding 3D
data into lower-dimensional representations allows for decreased CNN training
time cost, increased original 3D model rendering resolutions, and supports
increased numbers of data channels when compared to purely volumetric
approaches. This demonstration is accomplished in the context of a structural
biology classification task wherein we train 3D, 2D, and 1D CNNs on examples of
two homologous branches within the Ras protein family. The essential
contribution of this paper is the introduction of a dimensionality-reduction
method that may ease the application of powerful deep learning tools to domains
characterized by complex structural data.
Truncated Variational Sampling for "Black Box" Optimization of Generative Models
Authors: Jörg Lücke, Zhenwen Dai, Georgios Exarchakis
Date: 2017-12-21 00:00:00

We investigate the optimization of two probabilistic generative models with
binary latent variables using a novel variational EM approach. The approach
distinguishes itself from previous variational approaches by using latent
states as variational parameters. Here we use efficient and general purpose
sampling procedures to vary the latent states, and investigate the "black box"
applicability of the resulting optimization procedure. For general purpose
applicability, samples are drawn from approximate marginal distributions of the
considered generative model as well as from the model's prior distribution. As
such, variational sampling is defined in a generic form, and is directly
executable for a given model. As a proof of concept, we then apply the novel
procedure (A) to Binary Sparse Coding (a model with continuous observables),
and (B) to basic Sigmoid Belief Networks (which are models with binary
observables). Numerical experiments verify that the investigated approach
efficiently as well as effectively increases a variational free energy
objective without requiring any additional analytical steps.
Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting
Authors: Yaguang Li, Rose Yu, Cyrus Shahabi, Yan Liu
Date: 2017-07-06 00:00:00
Proceeding: ICLR 2018 1
Tasks: Decoder, Multivariate Time Series Forecasting, Spatio-Temporal Forecasting, Time Series Analysis, Time Series Forecasting, Time Series Prediction, Traffic Prediction

Spatiotemporal forecasting has various applications in neuroscience, climate
and transportation domain. Traffic forecasting is one canonical example of such
learning task. The task is challenging due to (1) complex spatial dependency on
road networks, (2) non-linear temporal dynamics with changing road conditions
and (3) inherent difficulty of long-term forecasting. To address these
challenges, we propose to model the traffic flow as a diffusion process on a
directed graph and introduce Diffusion Convolutional Recurrent Neural Network
(DCRNN), a deep learning framework for traffic forecasting that incorporates
both spatial and temporal dependency in the traffic flow. Specifically, DCRNN
captures the spatial dependency using bidirectional random walks on the graph,
and the temporal dependency using the encoder-decoder architecture with
scheduled sampling. We evaluate the framework on two real-world large scale
road network traffic datasets and observe consistent improvement of 12% - 15%
over state-of-the-art baselines.
Computationally and statistically efficient learning of causal Bayes nets using path queries
Authors: Kevin Bello, Jean Honorio
Date: 2017-06-02 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Causal Discovery

Causal discovery from empirical data is a fundamental problem in many scientific domains. Observational data allows for identifiability only up to Markov equivalence class. In this paper we first propose a polynomial time algorithm for learning the exact correctly-oriented structure of the transitive reduction of any causal Bayesian network with high probability, by using interventional path queries. Each path query takes as input an origin node and a target node, and answers whether there is a directed path from the origin to the target. This is done by intervening on the origin node and observing samples from the target node. We theoretically show the logarithmic sample complexity for the size of interventional data per path query, for continuous and discrete networks. We then show how to learn the transitive edges using also logarithmic sample complexity (albeit in time exponential in the maximum number of parents for discrete networks), which allows us to learn the full network. We further extend our work by reducing the number of interventional path queries for learning rooted trees. We also provide an analysis of imperfect interventions.
Semantically Decomposing the Latent Spaces of Generative Adversarial Networks
Authors: Chris Donahue, Zachary C. Lipton, Akshay Balsubramani, Julian McAuley
Date: 2017-05-22 00:00:00
Proceeding: ICLR 2018 1
Tasks: Face Verification, Image Generation

We propose a new algorithm for training generative adversarial networks that
jointly learns latent codes for both identities (e.g. individual humans) and
observations (e.g. specific photographs). By fixing the identity portion of the
latent codes, we can generate diverse images of the same subject, and by fixing
the observation portion, we can traverse the manifold of subjects while
maintaining contingent aspects such as lighting and pose. Our algorithm
features a pairwise training scheme in which each sample from the generator
consists of two images with a common identity code. Corresponding samples from
the real dataset consist of two distinct photographs of the same subject. In
order to fool the discriminator, the generator must produce pairs that are
photorealistic, distinct, and appear to depict the same individual. We augment
both the DCGAN and BEGAN approaches with Siamese discriminators to facilitate
pairwise training. Experiments with human judges and an off-the-shelf face
verification system demonstrate our algorithm's ability to generate convincing,
identity-matched photographs.
Recent Advances in Recurrent Neural Networks
Authors: Hojjat Salehinejad, Sharan Sankar, Joseph Barfett, Errol Colak, Shahrokh Valaee
Date: 2017-12-29 00:00:00
Tasks: Time Series, Time Series Analysis

Recurrent neural networks (RNNs) are capable of learning features and long
term dependencies from sequential and time-series data. The RNNs have a stack
of non-linear units where at least one connection between units forms a
directed cycle. A well-trained RNN can model any dynamical system; however,
training RNNs is mostly plagued by issues in learning long-term dependencies.
In this paper, we present a survey on RNNs and several new advances for
newcomers and professionals in the field. The fundamentals and recent advances
are explained and the research challenges are introduced.
Discriminative Localization in CNNs for Weakly-Supervised Segmentation of Pulmonary Nodules
Authors: Xinyang Feng, Jie Yang, Andrew F. Laine, Elsa D. Angelini
Date: 2017-07-04 00:00:00
Tasks: Computed Tomography (CT), General Classification, image-classification, Image Classification, Lung Cancer Diagnosis, Segmentation, Weakly supervised segmentation

Automated detection and segmentation of pulmonary nodules on lung computed
tomography (CT) scans can facilitate early lung cancer diagnosis. Existing
supervised approaches for automated nodule segmentation on CT scans require
voxel-based annotations for training, which are labor- and time-consuming to
obtain. In this work, we propose a weakly-supervised method that generates
accurate voxel-level nodule segmentation trained with image-level labels only.
By adapting a convolutional neural network (CNN) trained for image
classification, our proposed method learns discriminative regions from the
activation maps of convolution units at different scales, and identifies the
true nodule location with a novel candidate-screening framework. Experimental
results on the public LIDC-IDRI dataset demonstrate that, our weakly-supervised
nodule segmentation framework achieves competitive performance compared to a
fully-supervised CNN-based segmentation method.
Modular Multi-Objective Deep Reinforcement Learning with Decision Values
Authors: Tomasz Tajmajer
Date: 2017-04-21 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this work we present a method for using Deep Q-Networks (DQNs) in
multi-objective environments. Deep Q-Networks provide remarkable performance in
single objective problems learning from high-level visual state
representations. However, in many scenarios (e.g in robotics, games), the agent
needs to pursue multiple objectives simultaneously. We propose an architecture
in which separate DQNs are used to control the agent's behaviour with respect
to particular objectives. In this architecture we introduce decision values to
improve the scalarization of multiple DQNs into a single action. Our
architecture enables the decomposition of the agent's behaviour into
controllable and replaceable sub-behaviours learned by distinct modules.
Moreover, it allows to change the priorities of particular objectives
post-learning, while preserving the overall performance of the agent. To
evaluate our solution we used a game-like simulator in which an agent -
provided with high-level visual input - pursues multiple objectives in a 2D
world.
Characterizing Implicit Bias in Terms of Optimization Geometry
Authors: Suriya Gunasekar, Jason Lee, Daniel Soudry, Nathan Srebro
Date: 2018-02-22 00:00:00
Proceeding: ICML 2018 7
Tasks: General Classification, regression

We study the implicit bias of generic optimization methods, such as mirror descent, natural gradient descent, and steepest descent with respect to different potentials and norms, when optimizing underdetermined linear regression or separable linear classification problems. We explore the question of whether the specific global minimum (among the many possible global minima) reached by an algorithm can be characterized in terms of the potential or norm of the optimization geometry, and independently of hyperparameter choices such as step-size and momentum.
Structured low-rank matrix completion for forecasting in time series analysis
Authors: Jonathan Gillard, Konstantin Usevich
Date: 2018-02-22 00:00:00
Tasks: Low-Rank Matrix Completion, Matrix Completion, Missing Values, Time Series, Time Series Analysis

In this paper we consider the low-rank matrix completion problem with
specific application to forecasting in time series analysis. Briefly, the
low-rank matrix completion problem is the problem of imputing missing values of
a matrix under a rank constraint. We consider a matrix completion problem for
Hankel matrices and a convex relaxation based on the nuclear norm. Based on new
theoretical results and a number of numerical and real examples, we investigate
the cases when the proposed approach can work. Our results highlight the
importance of choosing a proper weighting scheme for the known observations.
Vector Field Based Neural Networks
Authors: Daniel Vieira, Fabio Rangel, Fabricio Firmino, Joao Paixao
Date: 2018-02-22 00:00:00
Tasks: General Classification

A novel Neural Network architecture is proposed using the mathematically and
physically rich idea of vector fields as hidden layers to perform nonlinear
transformations in the data. The data points are interpreted as particles
moving along a flow defined by the vector field which intuitively represents
the desired movement to enable classification. The architecture moves the data
points from their original configuration to anew one following the streamlines
of the vector field with the objective of achieving a final configuration where
classes are separable. An optimization problem is solved through gradient
descent to learn this vector field.