pwc-archive/papers-with-abstracts huggingface dataset:
Model-based Dashboards for Customer Analytics
Authors: Ryan Dew, Asim Ansari
Date: 2015-11-17 00:00:00
Tasks: model

Automating the customer analytics process is crucial for companies that
manage distinct customer bases. In such data-rich and dynamic environments,
visualization plays a key role in understanding events of interest. These ideas
have led to the popularity of analytics dashboards, yet academic research has
paid scant attention to these managerial needs. We develop a probabilistic,
nonparametric framework for understanding and predicting individual-level
spending using Gaussian process priors over latent functions that describe
customer spending along calendar time, interpurchase time, and customer
lifetime dimensions. These curves form a dashboard that provides a visual
model-based representation of purchasing dynamics that is easily
comprehensible. The model flexibly and automatically captures the form and
duration of the impact of events that influence spend propensity, even when
such events are unknown a-priori. We illustrate the use of our Gaussian Process
Propensity Model (GPPM) on data from two popular mobile games. We show that the
GPPM generalizes hazard and buy-till-you-die models by incorporating calendar
time dynamics while simultaneously accounting for recency and lifetime effects.
It therefore provides insights about spending propensity beyond those available
from these models. Finally, we show that the GPPM outperforms these benchmarks
both in fitting and forecasting real and simulated spend data.
Fourier ptychographic reconstruction using Poisson maximum likelihood and truncated Wirtinger gradient
Authors: Liheng Bian, Jinli Suo, Jaebum Chung, Xiaoze Ou, Changhuei Yang, Feng Chen, Qionghai Dai
Date: 2016-03-01 00:00:00
Tasks: Retrieval

Fourier ptychographic microscopy (FPM) is a novel computational coherent
imaging technique for high space-bandwidth product imaging. Mathematically,
Fourier ptychographic (FP) reconstruction can be implemented as a phase
retrieval optimization process, in which we only obtain low resolution
intensity images corresponding to the sub-bands of the sample's high resolution
(HR) spatial spectrum, and aim to retrieve the complex HR spectrum. In real
setups, the measurements always suffer from various degenerations such as
Gaussian noise, Poisson noise, speckle noise and pupil location error, which
would largely degrade the reconstruction. To efficiently address these
degenerations, we propose a novel FP reconstruction method under a gradient
descent optimization framework in this paper. The technique utilizes Poisson
maximum likelihood for better signal modeling, and truncated Wirtinger gradient
for error removal. Results on both simulated data and real data captured using
our laser FPM setup show that the proposed method outperforms other
state-of-the-art algorithms. Also, we have released our source code for
non-commercial use.
Crowdsourcing On-street Parking Space Detection
Authors: Ruizhi Liao, Cristian Roman, Peter Ball, Shumao Ou, Liping Chen
Date: 2016-03-01 00:00:00

As the number of vehicles continues to grow, parking spaces are at a premium
in city streets. Additionally, due to the lack of knowledge about street
parking spaces, heuristic circling the blocks not only costs drivers' time and
fuel, but also increases city congestion. In the wake of recent trend to build
convenient, green and energy-efficient smart cities, we rethink common
techniques adopted by high-profile smart parking systems, and present a
user-engaged (crowdsourcing) and sonar-based prototype to identify urban
on-street parking spaces. The prototype includes an ultrasonic sensor, a GPS
receiver and associated Arduino micro-controllers. It is mounted on the
passenger side of a car to measure the distance from the vehicle to the nearest
roadside obstacle. Multiple road tests are conducted around Wheatley, Oxford to
gather results and emulate the crowdsourcing approach. By extracting parked
vehicles' features from the collected trace, a supervised learning algorithm is
developed to estimate roadside parking occupancy and spot illegal parking
vehicles. A quantity estimation model is derived to calculate the required
number of sensing units to cover urban streets. The estimation is
quantitatively compared to a fixed sensing solution. The results show that the
crowdsourcing way would need substantially fewer sensors compared to the fixed
sensing system.
Convolutional Patch Representations for Image Retrieval: an Unsupervised Approach
Authors: Mattis Paulin, Julien Mairal, Matthijs Douze, Zaid Harchaoui, Florent Perronnin, Cordelia Schmid
Date: 2016-03-01 00:00:00
Tasks: image-classification, Image Classification, Image Retrieval, Retrieval

Convolutional neural networks (CNNs) have recently received a lot of
attention due to their ability to model local stationary structures in natural
images in a multi-scale fashion, when learning all model parameters with
supervision. While excellent performance was achieved for image classification
when large amounts of labeled visual data are available, their success for
un-supervised tasks such as image retrieval has been moderate so far. Our paper
focuses on this latter setting and explores several methods for learning patch
descriptors without supervision with application to matching and instance-level
retrieval. To that effect, we propose a new family of convolutional descriptors
for patch representation , based on the recently introduced convolutional
kernel networks. We show that our descriptor, named Patch-CKN, performs better
than SIFT as well as other convolutional networks learned by artificially
introducing supervision and is significantly faster to train. To demonstrate
its effectiveness, we perform an extensive evaluation on standard benchmarks
for patch and image retrieval where we obtain state-of-the-art results. We also
introduce a new dataset called RomePatches, which allows to simultaneously
study descriptor performance for patch and image retrieval.
A Nonlinear Adaptive Filter Based on the Model of Simple Multilinear Functionals
Authors: Felipe C. Pinheiro, Cássio G. Lopes
Date: 2016-03-01 00:00:00
Tasks: tensor algebra

Nonlinear adaptive filtering allows for modeling of some additional aspects
of a general system and usually relies on highly complex algorithms, such as
those based on the Volterra series. Through the use of the Kronecker product
and some basic facts of tensor algebra, we propose a simple model of
nonlinearity, one that can be interpreted as a product of the outputs of K FIR
linear filters, and compute its cost function together with its gradient, which
allows for some analysis of the optimization problem. We use these results it
in a stochastic gradient framework, from which we derive an LMS-like algorithm
and investigate the problems of multi-modality in the mean-square error surface
and the choice of adequate initial conditions. Its computational complexity is
calculated. The new algorithm is tested in a system identification setup and is
compared with other polynomial algorithms from the literature, presenting
favorable convergence and/or computational complexity.
Quantifying the vanishing gradient and long distance dependency problem in recursive neural networks and recursive LSTMs
Authors: Phong Le, Willem Zuidema
Date: 2016-03-01 00:00:00
Proceeding: WS 2016 8
Tasks: Sentence, Sentiment Analysis, Word Embeddings

Recursive neural networks (RNN) and their recently proposed extension
recursive long short term memory networks (RLSTM) are models that compute
representations for sentences, by recursively combining word embeddings
according to an externally provided parse tree. Both models thus, unlike
recurrent networks, explicitly make use of the hierarchical structure of a
sentence. In this paper, we demonstrate that RNNs nevertheless suffer from the
vanishing gradient and long distance dependency problem, and that RLSTMs
greatly improve over RNN's on these problems. We present an artificial learning
task that allows us to quantify the severity of these problems for both models.
We further show that a ratio of gradients (at the root node and a focal leaf
node) is highly indicative of the success of backpropagation at optimizing the
relevant weights low in the tree. This paper thus provides an explanation for
existing, superior results of RLSTMs on tasks such as sentiment analysis, and
suggests that the benefits of including hierarchical structure and of including
LSTM-style gating are complementary.
Delving Deeper into Convolutional Networks for Learning Video Representations
Authors: Nicolas Ballas, Li Yao, Chris Pal, Aaron Courville
Date: 2015-11-19 00:00:00
Tasks: Action Recognition, Decoder, Temporal Action Localization, Video Captioning

We propose an approach to learn spatio-temporal features in videos from
intermediate visual representations we call "percepts" using
Gated-Recurrent-Unit Recurrent Networks (GRUs).Our method relies on percepts
that are extracted from all level of a deep convolutional network trained on
the large ImageNet dataset. While high-level percepts contain highly
discriminative information, they tend to have a low-spatial resolution.
Low-level percepts, on the other hand, preserve a higher spatial resolution
from which we can model finer motion patterns. Using low-level percepts can
leads to high-dimensionality video representations. To mitigate this effect and
control the model number of parameters, we introduce a variant of the GRU model
that leverages the convolution operations to enforce sparse connectivity of the
model units and share parameters across the input spatial locations.
  We empirically validate our approach on both Human Action Recognition and
Video Captioning tasks. In particular, we achieve results equivalent to
state-of-art on the YouTube2Text dataset using a simpler text-decoder model and
without extra 3D CNN features.
Super-Resolution with Deep Convolutional Sufficient Statistics
Authors: Joan Bruna, Pablo Sprechmann, Yann Lecun
Date: 2015-11-18 00:00:00
Tasks: Bandwidth Extension, Image Super-Resolution, Structured Prediction, Super-Resolution

Inverse problems in image and audio, and super-resolution in particular, can
be seen as high-dimensional structured prediction problems, where the goal is
to characterize the conditional distribution of a high-resolution output given
its low-resolution corrupted observation. When the scaling ratio is small,
point estimates achieve impressive performance, but soon they suffer from the
regression-to-the-mean problem, result of their inability to capture the
multi-modality of this conditional distribution. Modeling high-dimensional
image and audio distributions is a hard task, requiring both the ability to
model complex geometrical structures and textured regions. In this paper, we
propose to use as conditional model a Gibbs distribution, where its sufficient
statistics are given by deep convolutional neural networks. The features
computed by the network are stable to local deformation, and have reduced
variance when the input is a stationary texture. These properties imply that
the resulting sufficient statistics minimize the uncertainty of the target
signals given the degraded observations, while being highly informative. The
filters of the CNN are initialized by multiscale complex wavelets, and then we
propose an algorithm to fine-tune them by estimating the gradient of the
conditional log-likelihood, which bears some similarities with Generative
Adversarial Networks. We evaluate experimentally the proposed approach in the
image super-resolution task, but the approach is general and could be used in
other challenging ill-posed problems such as audio bandwidth extension.
Numerical Approaches for Linear Left-invariant Diffusions on SE(2), their Comparison to Exact Solutions, and their Applications in Retinal Imaging
Authors: Jiong Zhang, Remco Duits, Gonzalo Sanguinetti, Bart M. ter Haar Romeny
Date: 2014-03-13 00:00:00

Left-invariant PDE-evolutions on the roto-translation group $SE(2)$ (and
their resolvent equations) have been widely studied in the fields of cortical
modeling and image analysis. They include hypo-elliptic diffusion (for contour
enhancement) proposed by Citti & Sarti, and Petitot, and they include the
direction process (for contour completion) proposed by Mumford. This paper
presents a thorough study and comparison of the many numerical approaches,
which, remarkably, is missing in the literature. Existing numerical approaches
can be classified into 3 categories: Finite difference methods, Fourier based
methods (equivalent to $SE(2)$-Fourier methods), and stochastic methods (Monte
Carlo simulations). There are also 3 types of exact solutions to the
PDE-evolutions that were derived explicitly (in the spatial Fourier domain) in
previous works by Duits and van Almsick in 2005. Here we provide an overview of
these 3 types of exact solutions and explain how they relate to each of the 3
numerical approaches. We compute relative errors of all numerical approaches to
the exact solutions, and the Fourier based methods show us the best performance
with smallest relative errors. We also provide an improvement of Mathematica
algorithms for evaluating Mathieu-functions, crucial in implementations of the
exact solutions. Furthermore, we include an asymptotical analysis of the
singularities within the kernels and we propose a probabilistic extension of
underlying stochastic processes that overcomes the singular behavior in the
origin of time-integrated kernels. Finally, we show retinal imaging
applications of combining left-invariant PDE-evolutions with invertible
orientation scores.
Deep Spatial Autoencoders for Visuomotor Learning
Authors: Chelsea Finn, Xin Yu Tan, Yan Duan, Trevor Darrell, Sergey Levine, Pieter Abbeel
Date: 2015-09-21 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning provides a powerful and flexible framework for
automated acquisition of robotic motion skills. However, applying reinforcement
learning requires a sufficiently detailed representation of the state,
including the configuration of task-relevant objects. We present an approach
that automates state-space construction by learning a state representation
directly from camera images. Our method uses a deep spatial autoencoder to
acquire a set of feature points that describe the environment for the current
task, such as the positions of objects, and then learns a motion skill with
these feature points using an efficient reinforcement learning method based on
local linear models. The resulting controller reacts continuously to the
learned feature points, allowing the robot to dynamically manipulate objects in
the world with closed-loop control. We demonstrate our method with a PR2 robot
on tasks that include pushing a free-standing toy block, picking up a bag of
rice using a spatula, and hanging a loop of rope on a hook at various
positions. In each task, our method automatically learns to track task-relevant
objects and manipulate their configuration with the robot's arm.
Driver Gaze Region Estimation Without Using Eye Movement
Authors: Lex Fridman, Philipp Langhans, Joonbum Lee, Bryan Reimer
Date: 2015-07-16 00:00:00

Automated estimation of the allocation of a driver's visual attention may be
a critical component of future Advanced Driver Assistance Systems. In theory,
vision-based tracking of the eye can provide a good estimate of gaze location.
In practice, eye tracking from video is challenging because of sunglasses,
eyeglass reflections, lighting conditions, occlusions, motion blur, and other
factors. Estimation of head pose, on the other hand, is robust to many of these
effects, but cannot provide as fine-grained of a resolution in localizing the
gaze. However, for the purpose of keeping the driver safe, it is sufficient to
partition gaze into regions. In this effort, we propose a system that extracts
facial features and classifies their spatial configuration into six regions in
real-time. Our proposed method achieves an average accuracy of 91.4% at an
average decision rate of 11 Hz on a dataset of 50 drivers from an on-road
study.
Dual Smoothing and Level Set Techniques for Variational Matrix Decomposition
Authors: Aleksandr Y. Aravkin, Stephen Becker
Date: 2016-03-01 00:00:00

We focus on the robust principal component analysis (RPCA) problem, and
review a range of old and new convex formulations for the problem and its
variants. We then review dual smoothing and level set techniques in convex
optimization, present several novel theoretical results, and apply the
techniques on the RPCA problem. In the final sections, we show a range of
numerical experiments for simulated and real-world problems.
Learning to retrieve out-of-vocabulary words in speech recognition
Authors: Imran Sheikh, Irina Illina, Dominique Fohr, Georges Linarès
Date: 2015-11-17 00:00:00
Tasks: Retrieval, speech-recognition, Speech Recognition

Many Proper Names (PNs) are Out-Of-Vocabulary (OOV) words for speech
recognition systems used to process diachronic audio data. To help recovery of
the PNs missed by the system, relevant OOV PNs can be retrieved out of the many
OOVs by exploiting semantic context of the spoken content. In this paper, we
propose two neural network models targeted to retrieve OOV PNs relevant to an
audio document: (a) Document level Continuous Bag of Words (D-CBOW), (b)
Document level Continuous Bag of Weighted Words (D-CBOW2). Both these models
take document words as input and learn with an objective to maximise the
retrieval of co-occurring OOV PNs. With the D-CBOW2 model we propose a new
approach in which the input embedding layer is augmented with a context anchor
layer. This layer learns to assign importance to input words and has the
ability to capture (task specific) key-words in a bag-of-word neural network
model. With experiments on French broadcast news videos we show that these two
models outperform the baseline methods based on raw embeddings from LDA,
Skip-gram and Paragraph Vectors. Combining the D-CBOW and D-CBOW2 models gives
faster convergence during training.
Event Search and Analytics: Detecting Events in Semantically Annotated Corpora for Search and Analytics
Authors: Dhruv Gupta
Date: 2016-03-01 00:00:00

In this article, I present the questions that I seek to answer in my PhD
research. I posit to analyze natural language text with the help of semantic
annotations and mine important events for navigating large text corpora.
Semantic annotations such as named entities, geographic locations, and temporal
expressions can help us mine events from the given corpora. These events thus
provide us with useful means to discover the locked knowledge in them. I pose
three problems that can help unlock this knowledge vault in semantically
annotated text corpora: i. identifying important events; ii. semantic search;
and iii. event analytics.
Image denoising based on improved data-driven sparse representation
Authors: Dai-Qiang Chen
Date: 2015-02-11 00:00:00
Tasks: Denoising, Image Denoising, Image Restoration

Sparse representation of images under certain transform domain has been
playing a fundamental role in image restoration tasks. One such representative
method is the widely used wavelet tight frame systems. Instead of adopting
fixed filters for constructing a tight frame to sparsely model any input image,
a data-driven tight frame was proposed for the sparse representation of images,
and shown to be very efficient for image denoising very recently. However, in
this method the number of framelet filters used for constructing a tight frame
is the same as the length of filters. In fact, through further investigation it
is found that part of these filters are unnecessary and even harmful to the
recovery effect due to the influence of noise. Therefore, an improved
data-driven sparse representation systems constructed with much less number of
filters are proposed. Numerical results on denoising experiments demonstrate
that the proposed algorithm overall outperforms the original data-driven tight
frame construction scheme on both the recovery quality and computational time.
Multi-task Sequence to Sequence Learning
Authors: Minh-Thang Luong, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, Lukasz Kaiser
Date: 2015-11-19 00:00:00
Tasks: Caption Generation, Decoder, Machine Translation, Multi-Task Learning, Translation

Sequence to sequence learning has recently emerged as a new paradigm in
supervised learning. To date, most of its applications focused on only one task
and not much work explored this framework for multiple tasks. This paper
examines three multi-task learning (MTL) settings for sequence to sequence
models: (a) the oneto-many setting - where the encoder is shared between
several tasks such as machine translation and syntactic parsing, (b) the
many-to-one setting - useful when only the decoder can be shared, as in the
case of translation and image caption generation, and (c) the many-to-many
setting - where multiple encoders and decoders are shared, which is the case
with unsupervised objectives and translation. Our results show that training on
a small amount of parsing and image caption data can improve the translation
quality between English and German by up to 1.5 BLEU points over strong
single-task baselines on the WMT benchmarks. Furthermore, we have established a
new state-of-the-art result in constituent parsing with 93.0 F1. Lastly, we
reveal interesting properties of the two unsupervised learning objectives,
autoencoder and skip-thought, in the MTL context: autoencoder helps less in
terms of perplexities but more on BLEU scores compared to skip-thought.
Reasoning about Entailment with Neural Attention
Authors: Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiský, Phil Blunsom
Date: 2015-09-22 00:00:00
Tasks: Natural Language Inference

While most approaches to automatically recognizing entailment relations have
used classifiers employing hand engineered features derived from complex
natural language processing pipelines, in practice their performance has been
only slightly better than bag-of-word pair classifiers using only lexical
similarity. The only attempt so far to build an end-to-end differentiable
neural network for entailment failed to outperform such a simple similarity
classifier. In this paper, we propose a neural model that reads two sentences
to determine entailment using long short-term memory units. We extend this
model with a word-by-word neural attention mechanism that encourages reasoning
over entailments of pairs of words and phrases. Furthermore, we present a
qualitative analysis of attention weights produced by this model, demonstrating
such reasoning capabilities. On a large entailment dataset this model
outperforms the previous best neural model and a classifier with engineered
features by a substantial margin. It is the first generic end-to-end
differentiable system that achieves state-of-the-art accuracy on a textual
entailment dataset.
Herding as a Learning System with Edge-of-Chaos Dynamics
Authors: Yutian Chen, Max Welling
Date: 2016-02-09 00:00:00

Herding defines a deterministic dynamical system at the edge of chaos. It
generates a sequence of model states and parameters by alternating parameter
perturbations with state maximizations, where the sequence of states can be
interpreted as "samples" from an associated MRF model. Herding differs from
maximum likelihood estimation in that the sequence of parameters does not
converge to a fixed point and differs from an MCMC posterior sampling approach
in that the sequence of states is generated deterministically. Herding may be
interpreted as a"perturb and map" method where the parameter perturbations are
generated using a deterministic nonlinear dynamical system rather than randomly
from a Gumbel distribution. This chapter studies the distinct statistical
characteristics of the herding algorithm and shows that the fast convergence
rate of the controlled moments may be attributed to edge of chaos dynamics. The
herding algorithm can also be generalized to models with latent variables and
to a discriminative learning setting. The perceptron cycling theorem ensures
that the fast moment matching property is preserved in the more general
framework.