pwc-archive/papers-with-abstracts huggingface dataset:
Database Learning: Toward a Database that Becomes Smarter Every Time
Authors: Yongjoo Park, Ahmad Shahab Tajik, Michael Cafarella, Barzan Mozafari
Date: 2017-03-16 00:00:00

In today's databases, previous query answers rarely benefit answering future
queries. For the first time, to the best of our knowledge, we change this
paradigm in an approximate query processing (AQP) context. We make the
following observation: the answer to each query reveals some degree of
knowledge about the answer to another query because their answers stem from the
same underlying distribution that has produced the entire dataset. Exploiting
and refining this knowledge should allow us to answer queries more
analytically, rather than by reading enormous amounts of raw data. Also,
processing more queries should continuously enhance our knowledge of the
underlying distribution, and hence lead to increasingly faster response times
for future queries.
  We call this novel idea---learning from past query answers---Database
Learning. We exploit the principle of maximum entropy to produce answers, which
are in expectation guaranteed to be more accurate than existing sample-based
approximations. Empowered by this idea, we build a query engine on top of Spark
SQL, called Verdict. We conduct extensive experiments on real-world query
traces from a large customer of a major database vendor. Our results
demonstrate that Verdict supports 73.7% of these queries, speeding them up by
up to 23.0x for the same accuracy level compared to existing AQP systems.
Gradient-based Regularization Parameter Selection for Problems with Non-smooth Penalty Functions
Authors: Jean Feng, Noah Simon
Date: 2017-03-28 00:00:00
Tasks: regression

In high-dimensional and/or non-parametric regression problems, regularization
(or penalization) is used to control model complexity and induce desired
structure. Each penalty has a weight parameter that indicates how strongly the
structure corresponding to that penalty should be enforced. Typically the
parameters are chosen to minimize the error on a separate validation set using
a simple grid search or a gradient-free optimization method. It is more
efficient to tune parameters if the gradient can be determined, but this is
often difficult for problems with non-smooth penalty functions. Here we show
that for many penalized regression problems, the validation loss is actually
smooth almost-everywhere with respect to the penalty parameters. We can
therefore apply a modified gradient descent algorithm to tune parameters.
Through simulation studies on example regression problems, we find that
increasing the number of penalty parameters and tuning them using our method
can decrease the generalization error.
Age Progression/Regression by Conditional Adversarial Autoencoder
Authors: Zhifei Zhang, Yang song, Hairong Qi
Date: 2017-02-27 00:00:00
Proceeding: CVPR 2017 7
Tasks: Attribute, regression

"If I provide you a face image of mine (without telling you the actual age
when I took the picture) and a large amount of face images that I crawled
(containing labeled faces of different ages but not necessarily paired), can
you show me what I would look like when I am 80 or what I was like when I was
5?" The answer is probably a "No." Most existing face aging works attempt to
learn the transformation between age groups and thus would require the paired
samples as well as the labeled query image. In this paper, we look at the
problem from a generative modeling perspective such that no paired samples is
required. In addition, given an unlabeled image, the generative model can
directly produce the image with desired age attribute. We propose a conditional
adversarial autoencoder (CAAE) that learns a face manifold, traversing on which
smooth age progression and regression can be realized simultaneously. In CAAE,
the face is first mapped to a latent vector through a convolutional encoder,
and then the vector is projected to the face manifold conditional on age
through a deconvolutional generator. The latent vector preserves personalized
face features (i.e., personality) and the age condition controls progression
vs. regression. Two adversarial networks are imposed on the encoder and
generator, respectively, forcing to generate more photo-realistic faces.
Experimental results demonstrate the appealing performance and flexibility of
the proposed framework by comparing with the state-of-the-art and ground truth.
Deep scattering transform applied to note onset detection and instrument recognition
Authors: D. Cazau, G. Revillon, O. Adam
Date: 2017-03-28 00:00:00
Tasks: Information Retrieval, Instrument Recognition, Music Information Retrieval, Music Transcription, Onset Detection, Retrieval

Automatic Music Transcription (AMT) is one of the oldest and most
well-studied problems in the field of music information retrieval. Within this
challenging research field, onset detection and instrument recognition take
important places in transcription systems, as they respectively help to
determine exact onset times of notes and to recognize the corresponding
instrument sources. The aim of this study is to explore the usefulness of
multiscale scattering operators for these two tasks on plucked string
instrument and piano music. After resuming the theoretical background and
illustrating the key features of this sound representation method, we evaluate
its performances comparatively to other classical sound representations. Using
both MIDI-driven datasets with real instrument samples and real musical pieces,
scattering is proved to outperform other sound representations for these AMT
subtasks, putting forward its richer sound representation and invariance
properties.
Particle Filtering for PLCA model with Application to Music Transcription
Authors: D. Cazau, G. Revillon, W. Yuancheng, O. Adam
Date: 2017-03-28 00:00:00
Tasks: Music Transcription, parameter estimation

Automatic Music Transcription (AMT) consists in automatically estimating the
notes in an audio recording, through three attributes: onset time, duration and
pitch. Probabilistic Latent Component Analysis (PLCA) has become very popular
for this task. PLCA is a spectrogram factorization method, able to model a
magnitude spectrogram as a linear combination of spectral vectors from a
dictionary. Such methods use the Expectation-Maximization (EM) algorithm to
estimate the parameters of the acoustic model. This algorithm presents
well-known inherent defaults (local convergence, initialization dependency),
making EM-based systems limited in their applications to AMT, particularly in
regards to the mathematical form and number of priors. To overcome such limits,
we propose in this paper to employ a different estimation framework based on
Particle Filtering (PF), which consists in sampling the posterior distribution
over larger parameter ranges. This framework proves to be more robust in
parameter estimation, more flexible and unifying in the integration of prior
knowledge in the system. Note-level transcription accuracies of 61.8 $\%$ and
59.5 $\%$ were achieved on evaluation sound datasets of two different
instrument repertoires, including the classical piano (from MAPS dataset) and
the marovany zither, and direct comparisons to previous PLCA-based approaches
are provided. Steps for further development are also outlined.
Fast Second-Order Stochastic Backpropagation for Variational Inference
Authors: Kai Fan, Ziteng Wang, Jeff Beck, James Kwok, Katherine Heller
Date: 2015-09-09 00:00:00
Tasks: regression, Variational Inference

We propose a second-order (Hessian or Hessian-free) based optimization method
for variational inference inspired by Gaussian backpropagation, and argue that
quasi-Newton optimization can be developed as well. This is accomplished by
generalizing the gradient computation in stochastic backpropagation via a
reparametrization trick with lower complexity. As an illustrative example, we
apply this approach to the problems of Bayesian logistic regression and
variational auto-encoder (VAE). Additionally, we compute bounds on the
estimator variance of intractable expectations for the family of Lipschitz
continuous function. Our method is practical, scalable and model free. We
demonstrate our method on several real-world datasets and provide comparisons
with other stochastic gradient methods to show substantial enhancement in
convergence rates.
Separation of Concerns in Reinforcement Learning
Authors: Harm van Seijen, Mehdi Fatemi, Joshua Romoff, Romain Laroche
Date: 2016-12-15 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we propose a framework for solving a single-agent task by
using multiple agents, each focusing on different aspects of the task. This
approach has two main advantages: 1) it allows for training specialized agents
on different parts of the task, and 2) it provides a new way to transfer
knowledge, by transferring trained agents. Our framework generalizes the
traditional hierarchical decomposition, in which, at any moment in time, a
single agent has control until it has solved its particular subtask. We
illustrate our framework with empirical experiments on two domains.
Unifying the Stochastic Spectral Descent for Restricted Boltzmann Machines with Bernoulli or Gaussian Inputs
Authors: Kai Fan
Date: 2017-03-28 00:00:00

Stochastic gradient descent based algorithms are typically used as the
general optimization tools for most deep learning models. A Restricted
Boltzmann Machine (RBM) is a probabilistic generative model that can be stacked
to construct deep architectures. For RBM with Bernoulli inputs, non-Euclidean
algorithm such as stochastic spectral descent (SSD) has been specifically
designed to speed up the convergence with improved use of the gradient
estimation by sampling methods. However, the existing algorithm and
corresponding theoretical justification depend on the assumption that the
possible configurations of inputs are finite, like binary variables. The
purpose of this paper is to generalize SSD for Gaussian RBM being capable of
mod- eling continuous data, regardless of the previous assumption. We propose
the gradient descent methods in non-Euclidean space of parameters, via de-
riving the upper bounds of logarithmic partition function for RBMs based on
Schatten-infinity norm. We empirically show that the advantage and improvement
of SSD over stochastic gradient descent (SGD).
License Plate Detection and Recognition Using Deeply Learned Convolutional Neural Networks
Authors: Syed Zain Masood, Guang Shu, Afshin Dehghan, Enrique. G. Ortiz
Date: 2017-03-21 00:00:00
Tasks: License Plate Detection

This work details Sighthounds fully automated license plate detection and
recognition system. The core technology of the system is built using a sequence
of deep Convolutional Neural Networks (CNNs) interlaced with accurate and
efficient algorithms. The CNNs are trained and fine-tuned so that they are
robust under different conditions (e.g. variations in pose, lighting,
occlusion, etc.) and can work across a variety of license plate templates (e.g.
sizes, backgrounds, fonts, etc). For quantitative analysis, we show that our
system outperforms the leading license plate detection and recognition
technology i.e. ALPR on several benchmarks. Our system is available to
developers through the Sighthound Cloud API at
https://www.sighthound.com/products/cloud
Improving Named Entity Recognition for Chinese Social Media with Word Segmentation Representation Learning
Authors: Nanyun Peng, Mark Dredze
Date: 2016-03-02 00:00:00
Proceeding: ACL 2016 8
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Representation Learning, Segmentation

Named entity recognition, and other information extraction tasks, frequently
use linguistic features such as part of speech tags or chunkings. For languages
where word boundaries are not readily identified in text, word segmentation is
a key first step to generating features for an NER system. While using word
boundary tags as features are helpful, the signals that aid in identifying
these boundaries may provide richer information for an NER system. New
state-of-the-art word segmentation systems use neural models to learn
representations for predicting word boundaries. We show that these same
representations, jointly trained with an NER system, yield significant
improvements in NER for Chinese social media. In our experiments, jointly
training NER and word segmentation with an LSTM-CRF model yields nearly 5%
absolute improvement over previously published results.
Collective Anomaly Detection based on Long Short Term Memory Recurrent Neural Network
Authors: Loic Bontemps, Van Loi Cao, James McDermott, Nhien-An Le-Khac
Date: 2017-03-28 00:00:00
Tasks: Anomaly Detection, Intrusion Detection, Time Series, Time Series Analysis

Intrusion detection for computer network systems becomes one of the most
critical tasks for network administrators today. It has an important role for
organizations, governments and our society due to its valuable resources on
computer networks. Traditional misuse detection strategies are unable to detect
new and unknown intrusion. Besides, anomaly detection in network security is
aim to distinguish between illegal or malicious events and normal behavior of
network systems. Anomaly detection can be considered as a classification
problem where it builds models of normal network behavior, which it uses to
detect new patterns that significantly deviate from the model. Most of the cur-
rent research on anomaly detection is based on the learning of normally and
anomaly behaviors. They do not take into account the previous, re- cent events
to detect the new incoming one. In this paper, we propose a real time
collective anomaly detection model based on neural network learning and feature
operating. Normally a Long Short Term Memory Recurrent Neural Network (LSTM
RNN) is trained only on normal data and it is capable of predicting several
time steps ahead of an input. In our approach, a LSTM RNN is trained with
normal time series data before performing a live prediction for each time step.
Instead of considering each time step separately, the observation of prediction
errors from a certain number of time steps is now proposed as a new idea for
detecting collective anomalies. The prediction errors from a number of the
latest time steps above a threshold will indicate a collective anomaly. The
model is built on a time series version of the KDD 1999 dataset. The
experiments demonstrate that it is possible to offer reliable and efficient for
collective anomaly detection.
Feature Analysis and Selection for Training an End-to-End Autonomous Vehicle Controller Using the Deep Learning Approach
Authors: Shun Yang, Wenshuo Wang, Chang Liu, Kevin Deng, J. Karl Hedrick
Date: 2017-03-28 00:00:00
Tasks: Autonomous Vehicles, feature selection

Deep learning-based approaches have been widely used for training controllers
for autonomous vehicles due to their powerful ability to approximate nonlinear
functions or policies. However, the training process usually requires large
labeled data sets and takes a lot of time. In this paper, we analyze the
influences of features on the performance of controllers trained using the
convolutional neural networks (CNNs), which gives a guideline of feature
selection to reduce computation cost. We collect a large set of data using The
Open Racing Car Simulator (TORCS) and classify the image features into three
categories (sky-related, roadside-related, and road-related features).We then
design two experimental frameworks to investigate the importance of each single
feature for training a CNN controller.The first framework uses the training
data with all three features included to train a controller, which is then
tested with data that has one feature removed to evaluate the feature's
effects. The second framework is trained with the data that has one feature
excluded, while all three features are included in the test data. Different
driving scenarios are selected to test and analyze the trained controllers
using the two experimental frameworks. The experiment results show that (1) the
road-related features are indispensable for training the controller, (2) the
roadside-related features are useful to improve the generalizability of the
controller to scenarios with complicated roadside information, and (3) the
sky-related features have limited contribution to train an end-to-end
autonomous vehicle controller.
Semantic Compositional Networks for Visual Captioning
Authors: Zhe Gan, Chuang Gan, Xiaodong He, Yunchen Pu, Kenneth Tran, Jianfeng Gao, Lawrence Carin, Li Deng
Date: 2016-11-23 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Captioning, Semantic Composition, TAG

A Semantic Compositional Network (SCN) is developed for image captioning, in
which semantic concepts (i.e., tags) are detected from the image, and the
probability of each tag is used to compose the parameters in a long short-term
memory (LSTM) network. The SCN extends each weight matrix of the LSTM to an
ensemble of tag-dependent weight matrices. The degree to which each member of
the ensemble is used to generate an image caption is tied to the
image-dependent probability of the corresponding tag. In addition to captioning
images, we also extend the SCN to generate captions for video clips. We
qualitatively analyze semantic composition in SCNs, and quantitatively evaluate
the algorithm on three benchmark datasets: COCO, Flickr30k, and Youtube2Text.
Experimental results show that the proposed method significantly outperforms
prior state-of-the-art approaches, across multiple evaluation metrics.
A World of Difference: Divergent Word Interpretations among People
Authors: Tianran Hu, Ruihua Song, Maya Abtahian, Philip Ding, Xing Xie, Jiebo Luo
Date: 2017-03-08 00:00:00

Divergent word usages reflect differences among people. In this paper, we
present a novel angle for studying word usage divergence -- word
interpretations. We propose an approach that quantifies semantic differences in
interpretations among different groups of people. The effectiveness of our
approach is validated by quantitative evaluations. Experiment results indicate
that divergences in word interpretations exist. We further apply the approach
to two well studied types of differences between people -- gender and region.
The detected words with divergent interpretations reveal the unique features of
specific groups of people. For gender, we discover that certain different
interests, social attitudes, and characters between males and females are
reflected in their divergent interpretations of many words. For region, we find
that specific interpretations of certain words reveal the geographical and
cultural features of different regions.
Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation
Authors: Benjamin Scellier, Yoshua Bengio
Date: 2016-02-16 00:00:00
Tasks: Prediction

We introduce Equilibrium Propagation, a learning framework for energy-based
models. It involves only one kind of neural computation, performed in both the
first phase (when the prediction is made) and the second phase of training
(after the target or prediction error is revealed). Although this algorithm
computes the gradient of an objective function just like Backpropagation, it
does not need a special computation or circuit for the second phase, where
errors are implicitly propagated. Equilibrium Propagation shares similarities
with Contrastive Hebbian Learning and Contrastive Divergence while solving the
theoretical issues of both algorithms: our algorithm computes the gradient of a
well defined objective function. Because the objective function is defined in
terms of local perturbations, the second phase of Equilibrium Propagation
corresponds to only nudging the prediction (fixed point, or stationary
distribution) towards a configuration that reduces prediction error. In the
case of a recurrent multi-layer supervised network, the output units are
slightly nudged towards their target in the second phase, and the perturbation
introduced at the output layer propagates backward in the hidden layers. We
show that the signal 'back-propagated' during this second phase corresponds to
the propagation of error derivatives and encodes the gradient of the objective
function, when the synaptic update corresponds to a standard form of
spike-timing dependent plasticity. This work makes it more plausible that a
mechanism similar to Backpropagation could be implemented by brains, since
leaky integrator neural computation performs both inference and error
back-propagation in our model. The only local difference between the two phases
is whether synaptic changes are allowed or not.
Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network
Authors: Nasim Souly, Concetto Spampinato, Mubarak Shah
Date: 2017-03-28 00:00:00
Tasks: Benchmarking, Clustering, General Classification, Generative Adversarial Network, Semantic Segmentation, Weakly supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation

Semantic segmentation has been a long standing challenging task in computer
vision. It aims at assigning a label to each image pixel and needs significant
number of pixellevel annotated data, which is often unavailable. To address
this lack, in this paper, we leverage, on one hand, massive amount of available
unlabeled or weakly labeled data, and on the other hand, non-real images
created through Generative Adversarial Networks. In particular, we propose a
semi-supervised framework ,based on Generative Adversarial Networks (GANs),
which consists of a generator network to provide extra training examples to a
multi-class classifier, acting as discriminator in the GAN framework, that
assigns sample a label y from the K possible classes or marks it as a fake
sample (extra class). The underlying idea is that adding large fake visual data
forces real samples to be close in the feature space, enabling a bottom-up
clustering process, which, in turn, improves multiclass pixel classification.
To ensure higher quality of generated images for GANs with consequent improved
pixel classification, we extend the above framework by adding weakly annotated
data, i.e., we provide class level information to the generator. We tested our
approaches on several challenging benchmarking visual datasets, i.e. PASCAL,
SiftFLow, Stanford and CamVid, achieving competitive performance also compared
to state-of-the-art semantic segmentation method
Efficient Two-Dimensional Sparse Coding Using Tensor-Linear Combination
Authors: Fei Jiang, Xiao-Yang Liu, Hongtao Lu, Ruimin Shen
Date: 2017-03-28 00:00:00
Tasks: Denoising, Vocal Bursts Valence Prediction

Sparse coding (SC) is an automatic feature extraction and selection technique
that is widely used in unsupervised learning. However, conventional SC
vectorizes the input images, which breaks apart the local proximity of pixels
and destructs the elementary object structures of images. In this paper, we
propose a novel two-dimensional sparse coding (2DSC) scheme that represents the
input images as the tensor-linear combinations under a novel algebraic
framework. 2DSC learns much more concise dictionaries because it uses the
circular convolution operator, since the shifted versions of atoms learned by
conventional SC are treated as the same ones. We apply 2DSC to natural images
and demonstrate that 2DSC returns meaningful dictionaries for large patches.
Moreover, for mutli-spectral images denoising, the proposed 2DSC reduces
computational costs with competitive performance in comparison with the
state-of-the-art algorithms.