pwc-archive/papers-with-abstracts huggingface dataset:
Attention-based End-to-End Models for Small-Footprint Keyword Spotting
Authors: Changhao Shan, Junbo Zhang, Yujun Wang, Lei Xie
Date: 2018-03-29 00:00:00
Tasks: Keyword Spotting, Small-Footprint Keyword Spotting

In this paper, we propose an attention-based end-to-end neural approach for
small-footprint keyword spotting (KWS), which aims to simplify the pipelines of
building a production-quality KWS system. Our model consists of an encoder and
an attention mechanism. The encoder transforms the input signal into a high
level representation using RNNs. Then the attention mechanism weights the
encoder features and generates a fixed-length vector. Finally, by linear
transformation and softmax function, the vector becomes a score used for
keyword detection. We also evaluate the performance of different encoder
architectures, including LSTM, GRU and CRNN. Experiments on real-world wake-up
data show that our approach outperforms the recent Deep KWS approach by a large
margin and the best performance is achieved by CRNN. To be more specific, with
~84K parameters, our attention-based model achieves 1.02% false rejection rate
(FRR) at 1.0 false alarm (FA) per hour.
Neural Speed Reading via Skim-RNN
Authors: Minjoon Seo, Sewon Min, Ali Farhadi, Hannaneh Hajishirzi
Date: 2017-11-06 00:00:00
Proceeding: ICLR 2018 1
Tasks: CPU

Inspired by the principles of speed reading, we introduce Skim-RNN, a
recurrent neural network (RNN) that dynamically decides to update only a small
fraction of the hidden state for relatively unimportant input tokens. Skim-RNN
gives computational advantage over an RNN that always updates the entire hidden
state. Skim-RNN uses the same input and output interfaces as a standard RNN and
can be easily used instead of RNNs in existing models. In our experiments, we
show that Skim-RNN can achieve significantly reduced computational cost without
losing accuracy compared to standard RNNs across five different natural
language tasks. In addition, we demonstrate that the trade-off between accuracy
and speed of Skim-RNN can be dynamically controlled during inference time in a
stable manner. Our analysis also shows that Skim-RNN running on a single CPU
offers lower latency compared to standard RNNs on GPUs.
Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective
Authors: Jing Zhang, Tong Zhang, Yuchao Dai, Mehrtash Harandi, Richard Hartley
Date: 2018-03-29 00:00:00
Proceeding: CVPR 2018 6
Tasks: Benchmarking, Saliency Detection, Saliency Prediction, Unsupervised Saliency Detection

The success of current deep saliency detection methods heavily depends on the
availability of large-scale supervision in the form of per-pixel labeling. Such
supervision, while labor-intensive and not always possible, tends to hinder the
generalization ability of the learned models. By contrast, traditional
handcrafted features based unsupervised saliency detection methods, even though
have been surpassed by the deep supervised methods, are generally
dataset-independent and could be applied in the wild. This raises a natural
question that "Is it possible to learn saliency maps without using labeled data
while improving the generalization ability?". To this end, we present a novel
perspective to unsupervised saliency detection through learning from multiple
noisy labeling generated by "weak" and "noisy" unsupervised handcrafted
saliency methods. Our end-to-end deep learning framework for unsupervised
saliency detection consists of a latent saliency prediction module and a noise
modeling module that work collaboratively and are optimized jointly. Explicit
noise modeling enables us to deal with noisy saliency maps in a probabilistic
way. Extensive experimental results on various benchmarking datasets show that
our model not only outperforms all the unsupervised saliency methods with a
large margin but also achieves comparable performance with the recent
state-of-the-art supervised deep saliency methods.
Motion-Appearance Co-Memory Networks for Video Question Answering
Authors: Jiyang Gao, Runzhou Ge, Kan Chen, Ram Nevatia
Date: 2018-03-29 00:00:00
Proceeding: CVPR 2018 6
Tasks: Question Answering, Video Question Answering, Visual Question Answering (VQA)

Video Question Answering (QA) is an important task in understanding video
temporal structure. We observe that there are three unique attributes of video
QA compared with image QA: (1) it deals with long sequences of images
containing richer information not only in quantity but also in variety; (2)
motion and appearance information are usually correlated with each other and
able to provide useful attention cues to the other; (3) different questions
require different number of frames to infer the answer. Based these
observations, we propose a motion-appearance comemory network for video QA. Our
networks are built on concepts from Dynamic Memory Network (DMN) and introduces
new mechanisms for video QA. Specifically, there are three salient aspects: (1)
a co-memory attention mechanism that utilizes cues from both motion and
appearance to generate attention; (2) a temporal conv-deconv network to
generate multi-level contextual facts; (3) a dynamic fact ensemble method to
construct temporal representation dynamically for different questions. We
evaluate our method on TGIF-QA dataset, and the results outperform
state-of-the-art significantly on all four tasks of TGIF-QA.
TOM-Net: Learning Transparent Object Matting from a Single Image
Authors: Guan-Ying Chen, Kai Han, Kwan-Yee K. Wong
Date: 2018-03-13 00:00:00
Proceeding: CVPR 2018 6
Tasks: Decoder, Image Matting, Object, Transparent objects

This paper addresses the problem of transparent object matting. Existing
image matting approaches for transparent objects often require tedious
capturing procedures and long processing time, which limit their practical use.
In this paper, we first formulate transparent object matting as a refractive
flow estimation problem. We then propose a deep learning framework, called
TOM-Net, for learning the refractive flow. Our framework comprises two parts,
namely a multi-scale encoder-decoder network for producing a coarse prediction,
and a residual network for refinement. At test time, TOM-Net takes a single
image as input, and outputs a matte (consisting of an object mask, an
attenuation mask and a refractive flow field) in a fast feed-forward pass. As
no off-the-shelf dataset is available for transparent object matting, we create
a large-scale synthetic dataset consisting of 158K images of transparent
objects rendered in front of images sampled from the Microsoft COCO dataset. We
also collect a real dataset consisting of 876 samples using 14 transparent
objects and 60 background images. Promising experimental results have been
achieved on both synthetic and real data, which clearly demonstrate the
effectiveness of our approach.
A Causal And-Or Graph Model for Visibility Fluent Reasoning in Tracking Interacting Objects
Authors: Yuanlu Xu, Lei Qin, Xiaobai Liu, Jianwen Xie, Song-Chun Zhu
Date: 2017-09-16 00:00:00
Proceeding: CVPR 2018 6
Tasks: Visual Tracking

Tracking humans that are interacting with the other subjects or environment
remains unsolved in visual tracking, because the visibility of the human of
interests in videos is unknown and might vary over time. In particular, it is
still difficult for state-of-the-art human trackers to recover complete human
trajectories in crowded scenes with frequent human interactions. In this work,
we consider the visibility status of a subject as a fluent variable, whose
change is mostly attributed to the subject's interaction with the surrounding,
e.g., crossing behind another object, entering a building, or getting into a
vehicle, etc. We introduce a Causal And-Or Graph (C-AOG) to represent the
causal-effect relations between an object's visibility fluent and its
activities, and develop a probabilistic graph model to jointly reason the
visibility fluent change (e.g., from visible to invisible) and track humans in
videos. We formulate this joint task as an iterative search of a feasible
causal graph structure that enables fast search algorithm, e.g., dynamic
programming method. We apply the proposed method on challenging video sequences
to evaluate its capabilities of estimating visibility fluent changes of
subjects and tracking subjects of interests over time. Results with comparisons
demonstrate that our method outperforms the alternative trackers and can
recover complete trajectories of humans in complicated scenarios with frequent
human interactions.
Predictive Liability Models and Visualizations of High Dimensional Retail Employee Data
Authors: Richard R. Yang, Mike Borowczak
Date: 2017-07-14 00:00:00
Tasks: BIG-bench Machine Learning, Dimensionality Reduction, feature selection, regression, Vocal Bursts Intensity Prediction

Employee theft and dishonesty is a major contributor to loss in the retail
industry. Retailers have reported the need for more automated analytic tools to
assess the liability of their employees. In this work, we train and optimize
several machine learning models for regression prediction and analysis on this
data, which will help retailers identify and manage risky employees. Since the
data we use is very high dimensional, we use feature selection techniques to
identify the most contributing factors to an employee's assessed risk. We also
use dimension reduction and data embedding techniques to present this dataset
in a easy to interpret format.
Multi-Scale Structure-Aware Network for Human Pose Estimation
Authors: Lipeng Ke, Ming-Ching Chang, Honggang Qi, Siwei Lyu
Date: 2018-03-27 00:00:00
Proceeding: ECCV 2018 9
Tasks: Pose Estimation, regression

We develop a robust multi-scale structure-aware neural network for human pose
estimation. This method improves the recent deep conv-deconv hourglass models
with four key improvements: (1) multi-scale supervision to strengthen
contextual feature learning in matching body keypoints by combining feature
heatmaps across scales, (2) multi-scale regression network at the end to
globally optimize the structural matching of the multi-scale features, (3)
structure-aware loss used in the intermediate supervision and at the regression
to improve the matching of keypoints and respective neighbors to infer a
higher-order matching configurations, and (4) a keypoint masking training
scheme that can effectively fine-tune our network to robustly localize occluded
keypoints via adjacent matches. Our method can effectively improve
state-of-the-art pose estimation methods that suffer from difficulties in scale
varieties, occlusions, and complex multi-person scenarios. This multi-scale
supervision tightly integrates with the regression network to effectively (i)
localize keypoints using the ensemble of multi-scale features, and (ii) infer
global pose configuration by maximizing structural consistencies across
multiple keypoints and scales. The keypoint masking training enhances these
advantages to focus learning on hard occlusion samples. Our method achieves the
leading position in the MPII challenge leaderboard among the state-of-the-art
methods.
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
Authors: Agrim Gupta, Justin Johnson, Li Fei-Fei, Silvio Savarese, Alexandre Alahi
Date: 2018-03-29 00:00:00
Proceeding: CVPR 2018 6
Tasks: Collision Avoidance, Motion Forecasting, Multi-future Trajectory Prediction, Navigate, Self-Driving Cars, Trajectory Forecasting, Trajectory Prediction

Understanding human motion behavior is critical for autonomous moving
platforms (like self-driving cars and social robots) if they are to navigate
human-centric environments. This is challenging because human motion is
inherently multimodal: given a history of human motion paths, there are many
socially plausible ways that people could move in the future. We tackle this
problem by combining tools from sequence prediction and generative adversarial
networks: a recurrent sequence-to-sequence model observes motion histories and
predicts future behavior, using a novel pooling mechanism to aggregate
information across people. We predict socially plausible futures by training
adversarially against a recurrent discriminator, and encourage diverse
predictions with a novel variety loss. Through experiments on several datasets
we demonstrate that our approach outperforms prior work in terms of accuracy,
variety, collision avoidance, and computational complexity.
An Empirical Analysis of Constrained Support Vector Quantile Regression for Nonparametric Probabilistic Forecasting of Wind Power
Authors: Kostas Hatalis, Shalinee Kishore, Katya Scheinberg, Alberto Lamadrid
Date: 2018-03-29 00:00:00
Tasks: Decision Making, Form, Prediction Intervals, quantile regression, regression

Uncertainty analysis in the form of probabilistic forecasting can provide
significant improvements in decision-making processes in the smart power grid
for better integrating renewable energies such as wind. Whereas point
forecasting provides a single expected value, probabilistic forecasts provide
more information in the form of quantiles, prediction intervals, or full
predictive densities. This paper analyzes the effectiveness of an approach for
nonparametric probabilistic forecasting of wind power that combines support
vector machines and nonlinear quantile regression with non-crossing
constraints. A numerical case study is conducted using publicly available wind
data from the Global Energy Forecasting Competition 2014. Multiple quantiles
are estimated to form 20%, 40%, 60% and 80% prediction intervals which are
evaluated using the pinball loss function and reliability measures. Three
benchmark models are used for comparison where results demonstrate the proposed
approach leads to significantly better performance while preventing the problem
of overlapping quantile estimates.
Deep neural networks can be improved using human-derived contextual expectations
Authors: Harish Katti, Marius V. Peelen, S. P. Arun
Date: 2016-11-22 00:00:00
Tasks: object-detection, Object Detection

Real-world objects occur in specific contexts. Such context has been shown to
facilitate detection by constraining the locations to search. But can context
directly benefit object detection? To do so, context needs to be learned
independently from target features. This is impossible in traditional object
detection where classifiers are trained on images containing both target
features and surrounding context. In contrast, humans can learn context and
target features separately, such as when we see highways without cars. Here we
show for the first time that human-derived scene expectations can be used to
improve object detection performance in machines. To measure contextual
expectations, we asked human subjects to indicate the scale, location and
likelihood at which cars or people might occur in scenes without these objects.
Humans showed highly systematic expectations that we could accurately predict
using scene features. This allowed us to predict human expectations on novel
scenes without requiring manual annotation. On augmenting deep neural networks
with predicted human expectations, we obtained substantial gains in accuracy
for detecting cars and people (1-3%) as well as on detecting associated objects
(3-20%). In contrast, augmenting deep networks with other conventional features
yielded far smaller gains. This improvement was due to relatively poor matches
at highly likely locations being correctly labelled as target and conversely
strong matches at unlikely locations being correctly rejected as false alarms.
Taken together, our results show that augmenting deep neural networks with
human-derived context features improves their performance, suggesting that
humans learn scene context separately unlike deep networks.
Iterative Inversion of Deformation Vector Fields with Feedback Control
Authors: Abhishek Kumar Dubey, Alexandros-Stavros Iliopoulos, Xiaobai Sun, Fang-Fang Yin, Lei Ren
Date: 2016-10-27 00:00:00
Tasks: 4D reconstruction, Image Registration

Purpose: Often, the inverse deformation vector field (DVF) is needed together
with the corresponding forward DVF in 4D reconstruction and dose calculation,
adaptive radiation therapy, and simultaneous deformable registration. This
study aims at improving both accuracy and efficiency of iterative algorithms
for DVF inversion, and advancing our understanding of divergence and latency
conditions. Method: We introduce a framework of fixed-point iteration
algorithms with active feedback control for DVF inversion. Based on rigorous
convergence analysis, we design control mechanisms for modulating the inverse
consistency (IC) residual of the current iterate, to be used as feedback into
the next iterate. The control is designed adaptively to the input DVF with the
objective to enlarge the convergence area and expedite convergence. Three
particular settings of feedback control are introduced: constant value over the
domain throughout the iteration; alternating values between iteration steps;
and spatially variant values. We also introduce three spectral measures of the
displacement Jacobian for characterizing a DVF. These measures reveal the
critical role of what we term the non-translational displacement component
(NTDC) of the DVF. We carry out inversion experiments with an analytical DVF
pair, and with DVFs associated with thoracic CT images of 6 patients at end of
expiration and end of inspiration. Results: NTDC-adaptive iterations are shown
to attain a larger convergence region at a faster pace compared to previous
non-adaptive DVF inversion iteration algorithms. By our numerical experiments,
alternating control yields smaller IC residuals and inversion errors than
constant control. Spatially variant control renders smaller residuals and
errors by at least an order of magnitude, compared to other schemes, in no more
than 10 steps. Inversion results also show remarkable quantitative agreement
with analysis-based predictions. Conclusion: Our analysis captures properties
of DVF data associated with clinical CT images, and provides new understanding
of iterative DVF inversion algorithms with a simple residual feedback control.
Adaptive control is necessary and highly effective in the presence of non-small
NTDCs. The adaptive iterations or the spectral measures, or both, may
potentially be incorporated into deformable image registration methods.
Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methods
Authors: Deirdre Quillen, Eric Jang, Ofir Nachum, Chelsea Finn, Julian Ibarz, Sergey Levine
Date: 2018-02-28 00:00:00
Tasks: Deep Reinforcement Learning, Diversity, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Robotic Grasping

In this paper, we explore deep reinforcement learning algorithms for
vision-based robotic grasping. Model-free deep reinforcement learning (RL) has
been successfully applied to a range of challenging environments, but the
proliferation of algorithms makes it difficult to discern which particular
approach would be best suited for a rich, diverse task like grasping. To answer
this question, we propose a simulated benchmark for robotic grasping that
emphasizes off-policy learning and generalization to unseen objects. Off-policy
learning enables utilization of grasping data over a wide variety of objects,
and diversity is important to enable the method to generalize to new objects
that were not seen during training. We evaluate the benchmark tasks against a
variety of Q-function estimation methods, a method previously proposed for
robotic grasping with deep neural network models, and a novel approach based on
a combination of Monte Carlo return estimation and an off-policy correction.
Our results indicate that several simple methods provide a surprisingly strong
competitor to popular algorithms such as double Q-learning, and our analysis of
stability sheds light on the relative tradeoffs between the algorithms.
On Convergence of some Gradient-based Temporal-Differences Algorithms for Off-Policy Learning
Authors: Huizhen Yu
Date: 2017-12-27 00:00:00

We consider off-policy temporal-difference (TD) learning methods for policy
evaluation in Markov decision processes with finite spaces and discounted
reward criteria, and we present a collection of convergence results for several
gradient-based TD algorithms with linear function approximation. The algorithms
we analyze include: (i) two basic forms of two-time-scale gradient-based TD
algorithms, which we call GTD and which minimize the mean squared projected
Bellman error using stochastic gradient-descent; (ii) their "robustified"
biased variants; (iii) their mirror-descent versions which combine the
mirror-descent idea with TD learning; and (iv) a single-time-scale version of
GTD that solves minimax problems formulated for approximate policy evaluation.
  We derive convergence results for three types of stepsizes: constant
stepsize, slowly diminishing stepsize, as well as the standard type of
diminishing stepsize with a square-summable condition. For the first two types
of stepsizes, we apply the weak convergence method from stochastic
approximation theory to characterize the asymptotic behavior of the algorithms,
and for the standard type of stepsize, we analyze the algorithmic behavior with
respect to a stronger mode of convergence, almost sure convergence. Our
convergence results are for the aforementioned TD algorithms with three general
ways of setting their $\lambda$-parameters: (i) state-dependent $\lambda$; (ii)
a recently proposed scheme of using history-dependent $\lambda$ to keep the
eligibility traces of the algorithms bounded while allowing for relatively
large values of $\lambda$; and (iii) a composite scheme of setting the
$\lambda$-parameters that combines the preceding two schemes and allows a
broader class of generalized Bellman operators to be used for approximate
policy evaluation with TD methods.
Learning to Look around Objects for Top-View Representations of Outdoor Scenes
Authors: Samuel Schulter, Menghua Zhai, Nathan Jacobs, Manmohan Chandraker
Date: 2018-03-28 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D geometry, Semantic Segmentation

Given a single RGB image of a complex outdoor road scene in the perspective
view, we address the novel problem of estimating an occlusion-reasoned semantic
scene layout in the top-view. This challenging problem not only requires an
accurate understanding of both the 3D geometry and the semantics of the visible
scene, but also of occluded areas. We propose a convolutional neural network
that learns to predict occluded portions of the scene layout by looking around
foreground objects like cars or pedestrians. But instead of hallucinating RGB
values, we show that directly predicting the semantics and depths in the
occluded areas enables a better transformation into the top-view. We further
show that this initial top-view representation can be significantly enhanced by
learning priors and rules about typical road layouts from simulated or, if
available, map data. Crucially, training our model does not require costly or
subjective human annotations for occluded areas or the top-view, but rather
uses readily available annotations for standard semantic segmentation. We
extensively evaluate and analyze our approach on the KITTI and Cityscapes data
sets.