pwc-archive/papers-with-abstracts huggingface dataset:
Deterministic Policy Optimization by Combining Pathwise and Score Function Estimators for Discrete Action Spaces
Authors: Daniel Levy, Stefano Ermon
Date: 2017-11-21 00:00:00
Tasks: Acrobot, Imitation Learning

Policy optimization methods have shown great promise in solving complex
reinforcement and imitation learning tasks. While model-free methods are
broadly applicable, they often require many samples to optimize complex
policies. Model-based methods greatly improve sample-efficiency but at the cost
of poor generalization, requiring a carefully handcrafted model of the system
dynamics for each task. Recently, hybrid methods have been successful in
trading off applicability for improved sample-complexity. However, these have
been limited to continuous action spaces. In this work, we present a new hybrid
method based on an approximation of the dynamics as an expectation over the
next state under the current policy. This relaxation allows us to derive a
novel hybrid policy gradient estimator, combining score function and pathwise
derivative estimators, that is applicable to discrete action spaces. We show
significant gains in sample complexity, ranging between $1.7$ and $25\times$,
when learning parameterized policies on Cart Pole, Acrobot, Mountain Car and
Hand Mass. Our method is applicable to both discrete and continuous action
spaces, when competing pathwise methods are limited to the latter.
Clonal analysis of newborn hippocampal dentate granule cell proliferation and development in temporal lobe epilepsy
Authors: Shatrunjai P. Singh, Candi L. LaSarge, Amen An, John J. McAuliffe, Steve C. Danzer
Date: 2017-11-21 00:00:00

Hippocampal dentate granule cells are among the few neuronal cell types
generated throughout adult life in mammals. In the normal brain, new granule
cells are generated from progenitors in the subgranular zone and integrate in a
typical fashion. During the development of epilepsy, granule cell integration
is profoundly altered. The new cells migrate to ectopic locations and develop
misoriented basal dendrites. Although it has been established that these
abnormal cells are newly generated, it is not known whether they arise
ubiquitously throughout the progenitor cell pool or are derived from a smaller
number of bad actor progenitors. To explore this question, we conducted a
clonal analysis study in mice expressing the Brainbow fluorescent protein
reporter construct in dentate granule cell progenitors. Mice were examined 2
months after pilocarpine-induced status epilepticus, a treatment that leads to
the development of epilepsy. Brain sections were rendered translucent so that
entire hippocampi could be reconstructed and all fluorescently labeled cells
identified. Our findings reveal that a small number of progenitors produce the
majority of ectopic cells following status epilepticus, indicating that either
the affected progenitors or their local microenvironments have become
pathological. By contrast, granule cells with basal dendrites were equally
distributed among clonal groups. This indicates that these progenitors can
produce normal cells and suggests that global factors sporadically disrupt the
dendritic development of some new cells. Together, these findings strongly
predict that distinct mechanisms regulate different aspects
Multiple-Instance, Cascaded Classification for Keyword Spotting in Narrow-Band Audio
Authors: Ahmad AbdulKader, Kareem Nassar, Mohamed El-Geish, Daniel Galvez, Chetan Patil
Date: 2017-11-21 00:00:00
Tasks: General Classification, Keyword Spotting, Multiple Instance Learning

We propose using cascaded classifiers for a keyword spotting (KWS) task on narrow-band (NB), 8kHz audio acquired in non-IID environments -- a more challenging task than most state-of-the-art KWS systems face. We present a model that incorporates Deep Neural Networks (DNNs), cascading, multiple-feature representations, and multiple-instance learning. The cascaded classifiers handle the task's class imbalance and reduce power consumption on computationally-constrained devices via early termination. The KWS system achieves a false negative rate of 6% at an hourly false positive rate of 0.75
Efficient Probabilistic Performance Bounds for Inverse Reinforcement Learning
Authors: Daniel S. Brown, Scott Niekum
Date: 2017-07-03 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In the field of reinforcement learning there has been recent progress towards
safety and high-confidence bounds on policy performance. However, to our
knowledge, no practical methods exist for determining high-confidence policy
performance bounds in the inverse reinforcement learning setting---where the
true reward function is unknown and only samples of expert behavior are given.
We propose a sampling method based on Bayesian inverse reinforcement learning
that uses demonstrations to determine practical high-confidence upper bounds on
the $\alpha$-worst-case difference in expected return between any evaluation
policy and the optimal policy under the expert's unknown reward function. We
evaluate our proposed bound on both a standard grid navigation task and a
simulated driving task and achieve tighter and more accurate bounds than a
feature count-based baseline. We also give examples of how our proposed bound
can be utilized to perform risk-aware policy selection and risk-aware policy
improvement. Because our proposed bound requires several orders of magnitude
fewer demonstrations than existing high-confidence bounds, it is the first
practical method that allows agents that learn from demonstration to express
confidence in the quality of their learned policy.
Inverse Risk-Sensitive Reinforcement Learning
Authors: Lillian J. Ratliff, Eric Mazumdar
Date: 2017-03-29 00:00:00
Tasks: Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We address the problem of inverse reinforcement learning in Markov decision
processes where the agent is risk-sensitive. In particular, we model
risk-sensitivity in a reinforcement learning framework by making use of models
of human decision-making having their origins in behavioral psychology,
behavioral economics, and neuroscience. We propose a gradient-based inverse
reinforcement learning algorithm that minimizes a loss function defined on the
observed behavior. We demonstrate the performance of the proposed technique on
two examples, the first of which is the canonical Grid World example and the
second of which is a Markov decision process modeling passengers' decisions
regarding ride-sharing. In the latter, we use pricing and travel time data from
a ride-sharing company to construct the transition probabilities and rewards of
the Markov decision process.
Towards Automatic Learning of Procedures from Web Instructional Videos
Authors: Luowei Zhou, Chenliang Xu, Jason J. Corso
Date: 2017-03-28 00:00:00
Tasks: Dense Video Captioning, Procedure Learning, Segmentation, Video Captioning

The potential for agents, whether embodied or software, to learn by observing
other agents performing procedures involving objects and actions is rich.
Current research on automatic procedure learning heavily relies on action
labels or video subtitles, even during the evaluation phase, which makes them
infeasible in real-world scenarios. This leads to our question: can the
human-consensus structure of a procedure be learned from a large set of long,
unconstrained videos (e.g., instructional videos from YouTube) with only visual
evidence? To answer this question, we introduce the problem of procedure
segmentation--to segment a video procedure into category-independent procedure
segments. Given that no large-scale dataset is available for this problem, we
collect a large-scale procedure segmentation dataset with procedure segments
temporally localized and described; we use cooking videos and name the dataset
YouCook2. We propose a segment-level recurrent network for generating procedure
segments by modeling the dependencies across segments. The generated segments
can be used as pre-processing for other tasks, such as dense video captioning
and event parsing. We show in our experiments that the proposed model
outperforms competitive baselines in procedure segmentation.
Deep Long Short-Term Memory Adaptive Beamforming Networks For Multichannel Robust Speech Recognition
Authors: Zhong Meng, Shinji Watanabe, John R. Hershey, Hakan Erdogan
Date: 2017-11-21 00:00:00
Tasks: Robust Speech Recognition, speech-recognition, Speech Recognition

Far-field speech recognition in noisy and reverberant conditions remains a
challenging problem despite recent deep learning breakthroughs. This problem is
commonly addressed by acquiring a speech signal from multiple microphones and
performing beamforming over them. In this paper, we propose to use a recurrent
neural network with long short-term memory (LSTM) architecture to adaptively
estimate real-time beamforming filter coefficients to cope with non-stationary
environmental noise and dynamic nature of source and microphones positions
which results in a set of timevarying room impulse responses. The LSTM adaptive
beamformer is jointly trained with a deep LSTM acoustic model to predict senone
labels. Further, we use hidden units in the deep LSTM acoustic model to assist
in predicting the beamforming filter coefficients. The proposed system achieves
7.97% absolute gain over baseline systems with no beamforming on CHiME-3 real
evaluation set.
The Riemannian Geometry of Deep Generative Models
Authors: Hang Shao, Abhishek Kumar, P. Thomas Fletcher
Date: 2017-11-21 00:00:00
Tasks: Translation

Deep generative models learn a mapping from a low dimensional latent space to
a high-dimensional data space. Under certain regularity conditions, these
models parameterize nonlinear manifolds in the data space. In this paper, we
investigate the Riemannian geometry of these generated manifolds. First, we
develop efficient algorithms for computing geodesic curves, which provide an
intrinsic notion of distance between points on the manifold. Second, we develop
an algorithm for parallel translation of a tangent vector along a path on the
manifold. We show how parallel translation can be used to generate analogies,
i.e., to transport a change in one data point into a semantically similar
change of another data point. Our experiments on real image data show that the
manifolds learned by deep generative models, while nonlinear, are surprisingly
close to zero curvature. The practical implication is that linear paths in the
latent space closely approximate geodesics on the generated manifold. However,
further investigation into this phenomenon is warranted, to identify if there
are other architectures or datasets where curvature plays a more prominent
role. We believe that exploring the Riemannian geometry of deep generative
models, using the tools developed in this paper, will be an important step in
understanding the high-dimensional, nonlinear spaces these models learn.
Unsupervised Adaptation with Domain Separation Networks for Robust Speech Recognition
Authors: Zhong Meng, Zhuo Chen, Vadim Mazalov, Jinyu Li, Yifan Gong
Date: 2017-11-21 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Domain Adaptation, domain classification, General Classification, Robust Speech Recognition, speech-recognition, Speech Recognition, Unsupervised Domain Adaptation

Unsupervised domain adaptation of speech signal aims at adapting a
well-trained source-domain acoustic model to the unlabeled data from target
domain. This can be achieved by adversarial training of deep neural network
(DNN) acoustic models to learn an intermediate deep representation that is both
senone-discriminative and domain-invariant. Specifically, the DNN is trained to
jointly optimize the primary task of senone classification and the secondary
task of domain classification with adversarial objective functions. In this
work, instead of only focusing on learning a domain-invariant feature (i.e. the
shared component between domains), we also characterize the difference between
the source and target domain distributions by explicitly modeling the private
component of each domain through a private component extractor DNN. The private
component is trained to be orthogonal with the shared component and thus
implicitly increases the degree of domain-invariance of the shared component. A
reconstructor DNN is used to reconstruct the original speech feature from the
private and shared components as a regularization. This domain separation
framework is applied to the unsupervised environment adaptation task and
achieved 11.08% relative WER reduction from the gradient reversal layer
training, a representative adversarial training method, for automatic speech
recognition on CHiME-3 dataset.
Hybrid safe-strong rules for efficient optimization in lasso-type problems
Authors: Yaohui Zeng, Tianbao Yang, Patrick Breheny
Date: 2017-04-27 00:00:00
Tasks: Model Selection, Vocal Bursts Type Prediction

The lasso model has been widely used for model selection in data mining, machine learning, and high-dimensional statistical analysis. However, with the ultrahigh-dimensional, large-scale data sets now collected in many real-world applications, it is important to develop algorithms to solve the lasso that efficiently scale up to problems of this size. Discarding features from certain steps of the algorithm is a powerful technique for increasing efficiency and addressing the Big Data challenge. In this paper, we propose a family of hybrid safe-strong rules (HSSR) which incorporate safe screening rules into the sequential strong rule (SSR) to remove unnecessary computational burden. In particular, we present two instances of HSSR, namely SSR-Dome and SSR-BEDPP, for the standard lasso problem. We further extend SSR-BEDPP to the elastic net and group lasso problems to demonstrate the generalizability of the hybrid screening idea. Extensive numerical experiments with synthetic and real data sets are conducted for both the standard lasso and the group lasso problems. Results show that our proposed hybrid rules can substantially outperform existing state-of-the-art rules.
Relating Input Concepts to Convolutional Neural Network Decisions
Authors: Ning Xie, Md. Kamruzzaman Sarker, Derek Doran, Pascal Hitzler, Michael Raymer
Date: 2017-11-21 00:00:00
Tasks: Decision Making, Scene Recognition

Many current methods to interpret convolutional neural networks (CNNs) use
visualization techniques and words to highlight concepts of the input seemingly
relevant to a CNN's decision. The methods hypothesize that the recognition of
these concepts are instrumental in the decision a CNN reaches, but the nature
of this relationship has not been well explored. To address this gap, this
paper examines the quality of a concept's recognition by a CNN and the degree
to which the recognitions are associated with CNN decisions. The study
considers a CNN trained for scene recognition over the ADE20k dataset. It uses
a novel approach to find and score the strength of minimally distributed
representations of input concepts (defined by objects in scene images) across
late stage feature maps. Subsequent analysis finds evidence that concept
recognition impacts decision making. Strong recognition of concepts
frequently-occurring in few scenes are indicative of correct decisions, but
recognizing concepts common to many scenes may mislead the network.
Computational Social Choice and Computational Complexity: BFFs?
Authors: Lane A. Hemaspaandra
Date: 2017-10-30 00:00:00
Tasks: General Classification

We discuss the connection between computational social choice (comsoc) and
computational complexity. We stress the work so far on, and urge continued
focus on, two less-recognized aspects of this connection. Firstly, this is very
much a two-way street: Everyone knows complexity classification is used in
comsoc, but we also highlight benefits to complexity that have arisen from its
use in comsoc. Secondly, more subtle, less-known complexity tools often can be
very productively used in comsoc.
Personalization of Saliency Estimation
Authors: Bingqing Yu, James J. Clark
Date: 2017-11-21 00:00:00
Tasks: Saliency Prediction

Most existing saliency models use low-level features or task descriptions
when generating attention predictions. However, the link between observer
characteristics and gaze patterns is rarely investigated. We present a novel
saliency prediction technique which takes viewers' identities and personal
traits into consideration when modeling human attention. Instead of only
computing image salience for average observers, we consider the interpersonal
variation in the viewing behaviors of observers with different personal traits
and backgrounds. We present an enriched derivative of the GAN network, which is
able to generate personalized saliency predictions when fed with image stimuli
and specific information about the observer. Our model contains a generator
which generates grayscale saliency heat maps based on the image and an observer
label. The generator is paired with an adversarial discriminator which learns
to distinguish generated salience from ground truth salience. The discriminator
also has the observer label as an input, which contributes to the
personalization ability of our approach. We evaluate the performance of our
personalized salience model by comparison with a benchmark model along with
other un-personalized predictions, and illustrate improvements in prediction
accuracy for all tested observer groups.
Dynamic High Resolution Deformable Articulated Tracking
Authors: Aaron Walsman, Weilin Wan, Tanner Schmidt, Dieter Fox
Date: 2017-11-21 00:00:00
Tasks: GPU, Pose Estimation, Vocal Bursts Intensity Prediction

The last several years have seen significant progress in using depth cameras
for tracking articulated objects such as human bodies, hands, and robotic
manipulators. Most approaches focus on tracking skeletal parameters of a fixed
shape model, which makes them insufficient for applications that require
accurate estimates of deformable object surfaces. To overcome this limitation,
we present a 3D model-based tracking system for articulated deformable objects.
Our system is able to track human body pose and high resolution surface
contours in real time using a commodity depth sensor and GPU hardware. We
implement this as a joint optimization over a skeleton to account for changes
in pose, and over the vertices of a high resolution mesh to track the subject's
shape. Through experimental results we show that we are able to capture dynamic
sub-centimeter surface detail such as folds and wrinkles in clothing. We also
show that this shape estimation aids kinematic pose estimation by providing a
more accurate target to match against the point cloud. The end result is highly
accurate spatiotemporal and semantic information which is well suited for
physical human robot interaction as well as virtual and augmented reality
systems.
Generating Analytic Insights on Human Behaviour using Image Processing
Authors: Namit Juneja, Rajesh Kumar Muthu
Date: 2017-11-21 00:00:00

This paper proposes a method to track human figures in physical spaces and
then utilizes this data to generate several data points such as footfall
distribution, demographic analysis,heat maps as well as gender distribution.
The proposed framework aims to establish this while utilizing minimum
computational resources while remaining real time. It is often useful to have
information such as what kind of people visit a certain place or what hour of
the day experiences maximum activity, Such analysis can be used improve sales,
manage huge number of people as well as predict future behaviour. The proposed
framework is designed in a way such that it can take input streams from IP
cameras and use that to generate relevant data points using open source tools
such as OpenCV and raspberryPi.
WAYLA - Generating Images from Eye Movements
Authors: Bingqing Yu, James J. Clark
Date: 2017-11-21 00:00:00
Tasks: Generative Adversarial Network, Image Generation, Image Reconstruction, Image-to-Image Translation, Translation

We present a method for reconstructing images viewed by observers based only
on their eye movements. By exploring the relationships between gaze patterns
and image stimuli, the "What Are You Looking At?" (WAYLA) system learns to
synthesize photo-realistic images that are similar to the original pictures
being viewed. The WAYLA approach is based on the Conditional Generative
Adversarial Network (Conditional GAN) image-to-image translation technique of
Isola et al. We consider two specific applications - the first, of
reconstructing newspaper images from gaze heat maps, and the second, of
detailed reconstruction of images containing only text. The newspaper image
reconstruction process is divided into two image-to-image translation
operations, the first mapping gaze heat maps into image segmentations, and the
second mapping the generated segmentation into a newspaper image. We validate
the performance of our approach using various evaluation metrics, along with
human visual inspection. All results confirm the ability of our network to
perform image generation tasks using eye tracking data.
Kullback-Leibler Principal Component for Tensors is not NP-hard
Authors: Kejun Huang, Nicholas D. Sidiropoulos
Date: 2017-11-21 00:00:00

We study the problem of nonnegative rank-one approximation of a nonnegative
tensor, and show that the globally optimal solution that minimizes the
generalized Kullback-Leibler divergence can be efficiently obtained, i.e., it
is not NP-hard. This result works for arbitrary nonnegative tensors with an
arbitrary number of modes (including two, i.e., matrices). We derive a
closed-form expression for the KL principal component, which is easy to compute
and has an intuitive probabilistic interpretation. For generalized KL
approximation with higher ranks, the problem is for the first time shown to be
equivalent to multinomial latent variable modeling, and an iterative algorithm
is derived that resembles the expectation-maximization algorithm. On the Iris
dataset, we showcase how the derived results help us learn the model in an
\emph{unsupervised} manner, and obtain strikingly close performance to that
from supervised methods.