pwc-archive/papers-with-abstracts huggingface dataset:
Let Features Decide for Themselves: Feature Mask Network for Person Re-identification
Authors: Guodong Ding, Salman Khan, Zhenmin Tang, Fatih Porikli
Date: 2017-11-20 00:00:00
Tasks: Person Re-Identification, Retrieval

Person re-identification aims at establishing the identity of a pedestrian
from a gallery that contains images of multiple people obtained from a
multi-camera system. Many challenges such as occlusions, drastic lighting and
pose variations across the camera views, indiscriminate visual appearances,
cluttered backgrounds, imperfect detections, motion blur, and noise make this
task highly challenging. While most approaches focus on learning features and
metrics to derive better representations, we hypothesize that both local and
global contextual cues are crucial for an accurate identity matching. To this
end, we propose a Feature Mask Network (FMN) that takes advantage of ResNet
high-level features to predict a feature map mask and then imposes it on the
low-level features to dynamically reweight different object parts for a locally
aware feature representation. This serves as an effective attention mechanism
by allowing the network to focus on local details selectively. Given the
resemblance of person re-identification with classification and retrieval
tasks, we frame the network training as a multi-task objective optimization,
which further improves the learned feature descriptions. We conduct experiments
on Market-1501, DukeMTMC-reID and CUHK03 datasets, where the proposed approach
respectively achieves significant improvements of $5.3\%$, $9.1\%$ and $10.7\%$
in mAP measure relative to the state-of-the-art.
Interactive, Intelligent Tutoring for Auxiliary Constructions in Geometry Proofs
Authors: Ke Wang, Zhendong Su
Date: 2017-11-20 00:00:00
Tasks: Automated Theorem Proving

Geometry theorem proving forms a major and challenging component in the K-12
mathematics curriculum. A particular difficult task is to add auxiliary
constructions (i.e, additional lines or points) to aid proof discovery.
Although there exist many intelligent tutoring systems proposed for geometry
proofs, few teach students how to find auxiliary constructions. And the few
exceptions are all limited by their underlying reasoning processes for
supporting auxiliary constructions. This paper tackles these weaknesses of
prior systems by introducing an interactive geometry tutor, the Advanced
Geometry Proof Tutor (AGPT). It leverages a recent automated geometry prover to
provide combined benefits that any geometry theorem prover or intelligent
tutoring system alone cannot accomplish. In particular, AGPT not only can
automatically process images of geometry problems directly, but also can
interactively train and guide students toward discovering auxiliary
constructions on their own. We have evaluated AGPT via a pilot study with 78
high school students. The study results show that, on training students how to
find auxiliary constructions, there is no significant perceived difference
between AGPT and human tutors, and AGPT is significantly more effective than
the state-of-the-art geometry solver that produces human-readable proofs.
Spectral-Spatial Feature Extraction and Classification by ANN Supervised with Center Loss in Hyperspectral Imagery
Authors: Alan J. X. Guo, Fei Zhu
Date: 2017-11-20 00:00:00
Tasks: Face Recognition, General Classification

In this paper, we propose a spectral-spatial feature extraction and
classification framework based on artificial neuron network (ANN) in the
context of hyperspectral imagery. With limited labeled samples, only spectral
information is exploited for training and spatial context is integrated
posteriorly at the testing stage. Taking advantage of recent advances in face
recognition, a joint supervision symbol that combines softmax loss and center
loss is adopted to train the proposed network, by which intra-class features
are gathered while inter-class variations are enlarged. Based on the learned
architecture, the extracted spectrum-based features are classified by a center
classifier. Moreover, to fuse the spectral and spatial information, an adaptive
spectral-spatial center classifier is developed, where multiscale neighborhoods
are considered simultaneously, and the final label is determined using an
adaptive voting strategy. Finally, experimental results on three well-known
datasets validate the effectiveness of the proposed methods compared with the
state-of-the-art approaches.
Exact heat kernel on a hypersphere and its applications in kernel SVM
Authors: Chenchao Zhao, Jun S. Song
Date: 2017-02-05 00:00:00
Tasks: Imputation

Many contemporary statistical learning methods assume a Euclidean feature
space. This paper presents a method for defining similarity based on
hyperspherical geometry and shows that it often improves the performance of
support vector machine compared to other competing similarity measures.
Specifically, the idea of using heat diffusion on a hypersphere to measure
similarity has been previously proposed, demonstrating promising results based
on a heuristic heat kernel obtained from the zeroth order parametrix expansion;
however, how well this heuristic kernel agrees with the exact hyperspherical
heat kernel remains unknown. This paper presents a higher order parametrix
expansion of the heat kernel on a unit hypersphere and discusses several
problems associated with this expansion method. We then compare the heuristic
kernel with an exact form of the heat kernel expressed in terms of a uniformly
and absolutely convergent series in high-dimensional angular momentum
eigenmodes. Being a natural measure of similarity between sample points
dwelling on a hypersphere, the exact kernel often shows superior performance in
kernel SVM classifications applied to text mining, tumor somatic mutation
imputation, and stock market analysis.
BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems
Authors: Zachary Lipton, Xiujun Li, Jianfeng Gao, Lihong Li, Faisal Ahmed, Li Deng
Date: 2017-11-15 00:00:00
Tasks: Deep Reinforcement Learning, Efficient Exploration, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Task-Oriented Dialogue Systems, Thompson Sampling

We present a new algorithm that significantly improves the efficiency of
exploration for deep Q-learning agents in dialogue systems. Our agents explore
via Thompson sampling, drawing Monte Carlo samples from a Bayes-by-Backprop
neural network. Our algorithm learns much faster than common exploration
strategies such as \epsilon-greedy, Boltzmann, bootstrapping, and
intrinsic-reward-based ones. Additionally, we show that spiking the replay
buffer with experiences from just a few successful episodes can make Q-learning
feasible when it might otherwise fail.
Competing Bandits: Learning under Competition
Authors: Yishay Mansour, Aleksandrs Slivkins, Zhiwei Steven Wu
Date: 2017-02-27 00:00:00

Most modern systems strive to learn from interactions with users, and many
engage in exploration: making potentially suboptimal choices for the sake of
acquiring new information. We initiate a study of the interplay between
exploration and competition--how such systems balance the exploration for
learning and the competition for users. Here the users play three distinct
roles: they are customers that generate revenue, they are sources of data for
learning, and they are self-interested agents which choose among the competing
systems. In our model, we consider competition between two multi-armed bandit
algorithms faced with the same bandit instance. Users arrive one by one and
choose among the two algorithms, so that each algorithm makes progress if and
only if it is chosen. We ask whether and to what extent competition
incentivizes the adoption of better bandit algorithms. We investigate this
issue for several models of user response, as we vary the degree of rationality
and competitiveness in the model. Our findings are closely related to the
"competition vs. innovation" relationship, a well-studied theme in economics.
Art of singular vectors and universal adversarial perturbations
Authors: Valentin Khrulkov, Ivan Oseledets
Date: 2017-09-11 00:00:00
Proceeding: CVPR 2018 6
Tasks: image-classification, Image Classification

Vulnerability of Deep Neural Networks (DNNs) to adversarial attacks has been
attracting a lot of attention in recent studies. It has been shown that for
many state of the art DNNs performing image classification there exist
universal adversarial perturbations --- image-agnostic perturbations mere
addition of which to natural images with high probability leads to their
misclassification. In this work we propose a new algorithm for constructing
such universal perturbations. Our approach is based on computing the so-called
$(p, q)$-singular vectors of the Jacobian matrices of hidden layers of a
network. Resulting perturbations present interesting visual patterns, and by
using only 64 images we were able to construct universal perturbations with
more than 60 \% fooling rate on the dataset consisting of 50000 images. We also
investigate a correlation between the maximal singular value of the Jacobian
matrix and the fooling rate of the corresponding singular vector, and show that
the constructed perturbations generalize across networks.
Maximum A Posteriori Inference in Sum-Product Networks
Authors: Jun Mei, Yong Jiang, Kewei Tu
Date: 2017-08-16 00:00:00

Sum-product networks (SPNs) are a class of probabilistic graphical models
that allow tractable marginal inference. However, the maximum a posteriori
(MAP) inference in SPNs is NP-hard. We investigate MAP inference in SPNs from
both theoretical and algorithmic perspectives. For the theoretical part, we
reduce general MAP inference to its special case without evidence and hidden
variables; we also show that it is NP-hard to approximate the MAP problem to
$2^{n^\epsilon}$ for fixed $0 \leq \epsilon < 1$, where $n$ is the input size.
For the algorithmic part, we first present an exact MAP solver that runs
reasonably fast and could handle SPNs with up to 1k variables and 150k arcs in
our experiments. We then present a new approximate MAP solver with a good
balance between speed and accuracy, and our comprehensive experiments on
real-world datasets show that it has better overall performance than existing
approximate solvers.
Fast Recurrent Fully Convolutional Networks for Direct Perception in Autonomous Driving
Authors: Yiqi Hou, Sascha Hornauer, Karl Zipser
Date: 2017-11-17 00:00:00
Tasks: Autonomous Driving, General Classification, Image Segmentation, Semantic Segmentation

Deep convolutional neural networks (CNNs) have been shown to perform
extremely well at a variety of tasks including subtasks of autonomous driving
such as image segmentation and object classification. However, networks
designed for these tasks typically require vast quantities of training data and
long training periods to converge. We investigate the design rationale behind
end-to-end driving network designs by proposing and comparing three small and
computationally inexpensive deep end-to-end neural network models that generate
driving control signals directly from input images. In contrast to prior work
that segments the autonomous driving task, our models take on a novel approach
to the autonomous driving problem by utilizing deep and thin Fully
Convolutional Nets (FCNs) with recurrent neural nets and low parameter counts
to tackle a complex end-to-end regression task predicting both steering and
acceleration commands. In addition, we include layers optimized for
classification to allow the networks to implicitly learn image semantics. We
show that the resulting networks use 3x fewer parameters than the most recent
comparable end-to-end driving network and 500x fewer parameters than the
AlexNet variations and converge both faster and to lower losses while
maintaining robustness against overfitting.
Concept Drift Detection and Adaptation with Hierarchical Hypothesis Testing
Authors: Shujian Yu, Zubin Abraham, Heng Wang, Mohak Shah, Yantao Wei, José C. Príncipe
Date: 2017-07-25 00:00:00
Tasks: Drift Detection, General Classification, Two-sample testing

A fundamental issue for statistical classification models in a streaming
environment is that the joint distribution between predictor and response
variables changes over time (a phenomenon also known as concept drifts), such
that their classification performance deteriorates dramatically. In this paper,
we first present a hierarchical hypothesis testing (HHT) framework that can
detect and also adapt to various concept drift types (e.g., recurrent or
irregular, gradual or abrupt), even in the presence of imbalanced data labels.
A novel concept drift detector, namely Hierarchical Linear Four Rates (HLFR),
is implemented under the HHT framework thereafter. By substituting a
widely-acknowledged retraining scheme with an adaptive training strategy, we
further demonstrate that the concept drift adaptation capability of HLFR can be
significantly boosted. The theoretical analysis on the Type-I and Type-II
errors of HLFR is also performed. Experiments on both simulated and real-world
datasets illustrate that our methods outperform state-of-the-art methods in
terms of detection precision, detection delay as well as the adaptability
across different concept drift types.
Modeling Epistemological Principles for Bias Mitigation in AI Systems: An Illustration in Hiring Decisions
Authors: Marisa Vasconcelos, Carlos Cardonha, Bernardo Gonçalves
Date: 2017-11-20 00:00:00
Tasks: Decision Making, Fairness

Artificial Intelligence (AI) has been used extensively in automatic decision
making in a broad variety of scenarios, ranging from credit ratings for loans
to recommendations of movies. Traditional design guidelines for AI models focus
essentially on accuracy maximization, but recent work has shown that
economically irrational and socially unacceptable scenarios of discrimination
and unfairness are likely to arise unless these issues are explicitly
addressed. This undesirable behavior has several possible sources, such as
biased datasets used for training that may not be detected in black-box models.
After pointing out connections between such bias of AI and the problem of
induction, we focus on Popper's contributions after Hume's, which offer a
logical theory of preferences. An AI model can be preferred over others on
purely rational grounds after one or more attempts at refutation based on
accuracy and fairness. Inspired by such epistemological principles, this paper
proposes a structured approach to mitigate discrimination and unfairness caused
by bias in AI systems. In the proposed computational framework, models are
selected and enhanced after attempts at refutation. To illustrate our
discussion, we focus on hiring decision scenarios where an AI system filters in
which job applicants should go to the interview phase.
Zero-Shot Learning via Class-Conditioned Deep Generative Models
Authors: Wenlin Wang, Yunchen Pu, Vinay Kumar Verma, Kai Fan, Yizhe Zhang, Changyou Chen, Piyush Rai, Lawrence Carin
Date: 2017-11-15 00:00:00
Tasks: Few-Shot Learning, Zero-Shot Learning

We present a deep generative model for learning to predict classes not seen
at training time. Unlike most existing methods for this problem, that represent
each class as a point (via a semantic embedding), we represent each seen/unseen
class using a class-specific latent-space distribution, conditioned on class
attributes. We use these latent-space distributions as a prior for a supervised
variational autoencoder (VAE), which also facilitates learning highly
discriminative feature representations for the inputs. The entire framework is
learned end-to-end using only the seen-class training data. The model infers
corresponding attributes of a test image by maximizing the VAE lower bound; the
inferred attributes may be linked to labels not seen when training. We further
extend our model to a (1) semi-supervised/transductive setting by leveraging
unlabeled unseen-class data via an unsupervised learning module, and (2)
few-shot learning where we also have a small number of labeled inputs from the
unseen classes. We compare our model with several state-of-the-art methods
through a comprehensive set of experiments on a variety of benchmark data sets.
Compression-Based Regularization with an Application to Multi-Task Learning
Authors: Matías Vera, Leonardo Rey Vega, Pablo Piantanida
Date: 2017-11-19 00:00:00
Tasks: Multi-Task Learning, Text Categorization

This paper investigates, from information theoretic grounds, a learning
problem based on the principle that any regularity in a given dataset can be
exploited to extract compact features from data, i.e., using fewer bits than
needed to fully describe the data itself, in order to build meaningful
representations of a relevant content (multiple labels). We begin by
introducing the noisy lossy source coding paradigm with the log-loss fidelity
criterion which provides the fundamental tradeoffs between the
\emph{cross-entropy loss} (average risk) and the information rate of the
features (model complexity). Our approach allows an information theoretic
formulation of the \emph{multi-task learning} (MTL) problem which is a
supervised learning framework in which the prediction models for several
related tasks are learned jointly from common representations to achieve better
generalization performance. Then, we present an iterative algorithm for
computing the optimal tradeoffs and its global convergence is proven provided
that some conditions hold. An important property of this algorithm is that it
provides a natural safeguard against overfitting, because it minimizes the
average risk taking into account a penalization induced by the model
complexity. Remarkably, empirical results illustrate that there exists an
optimal information rate minimizing the \emph{excess risk} which depends on the
nature and the amount of available training data. An application to
hierarchical text categorization is also investigated, extending previous
works.
Combination of LMS Adaptive Filters with Coefficients Feedback
Authors: Luiz. F. O. Chamon, Cassio G. Lopes
Date: 2016-08-10 00:00:00

Parallel combinations of adaptive filters have been effectively used to
improve the performance of adaptive algorithms and address well-known
trade-offs, such as convergence rate vs. steady-state error. Nevertheless,
typical combinations suffer from a convergence stagnation issue due to the fact
that the component filters run independently. Solutions to this issue usually
involve conditional transfers of coefficients between filters, which although
effective, are hard to generalize to combinations with more filters or when
there is no clearly faster adaptive filter. In this work, a more natural
solution is proposed by cyclically feeding back the combined coefficient vector
to all component filters. Besides coping with convergence stagnation, this new
topology improves tracking and supervisor stability, and bridges an important
conceptual gap between combinations of adaptive filters and variable step size
schemes. We analyze the steady-state, tracking, and transient performance of
this topology for LMS component filters and supervisors with generic activation
functions. Numerical examples are used to illustrate how coefficients feedback
can improve the performance of parallel combinations at a small computational
overhead.
Attend and Diagnose: Clinical Time Series Analysis using Attention Models
Authors: Huan Song, Deepta Rajan, Jayaraman J. Thiagarajan, Andreas Spanias
Date: 2017-11-10 00:00:00
Tasks: Sand, Time Series, Time Series Analysis

With widespread adoption of electronic health records, there is an increased
emphasis for predictive models that can effectively deal with clinical
time-series data. Powered by Recurrent Neural Network (RNN) architectures with
Long Short-Term Memory (LSTM) units, deep neural networks have achieved
state-of-the-art results in several clinical prediction tasks. Despite the
success of RNNs, its sequential nature prohibits parallelized computing, thus
making it inefficient particularly when processing long sequences. Recently,
architectures which are based solely on attention mechanisms have shown
remarkable success in transduction tasks in NLP, while being computationally
superior. In this paper, for the first time, we utilize attention models for
clinical time-series modeling, thereby dispensing recurrence entirely. We
develop the \textit{SAnD} (Simply Attend and Diagnose) architecture, which
employs a masked, self-attention mechanism, and uses positional encoding and
dense interpolation strategies for incorporating temporal order. Furthermore,
we develop a multi-task variant of \textit{SAnD} to jointly infer models with
multiple diagnosis tasks. Using the recent MIMIC-III benchmark datasets, we
demonstrate that the proposed approach achieves state-of-the-art performance in
all tasks, outperforming LSTM models and classical baselines with
hand-engineered features.
Statistical inference using SGD
Authors: Tianyang Li, Liu Liu, Anastasios Kyrillidis, Constantine Caramanis
Date: 2017-05-21 00:00:00

We present a novel method for frequentist statistical inference in
$M$-estimation problems, based on stochastic gradient descent (SGD) with a
fixed step size: we demonstrate that the average of such SGD sequences can be
used for statistical inference, after proper scaling. An intuitive analysis
using the Ornstein-Uhlenbeck process suggests that such averages are
asymptotically normal. From a practical perspective, our SGD-based inference
procedure is a first order method, and is well-suited for large scale problems.
To show its merits, we apply it to both synthetic and real datasets, and
demonstrate that its accuracy is comparable to classical statistical methods,
while requiring potentially far less computation.
Differentially Private Mixture of Generative Neural Networks
Authors: Gergely Acs, Luca Melis, Claude Castelluccia, Emiliano De Cristofaro
Date: 2017-09-13 00:00:00

Generative models are used in a wide range of applications building on large
amounts of contextually rich information. Due to possible privacy violations of
the individuals whose data is used to train these models, however, publishing
or sharing generative models is not always viable. In this paper, we present a
novel technique for privately releasing generative models and entire
high-dimensional datasets produced by these models. We model the generator
distribution of the training data with a mixture of $k$ generative neural
networks. These are trained together and collectively learn the generator
distribution of a dataset. Data is divided into $k$ clusters, using a novel
differentially private kernel $k$-means, then each cluster is given to separate
generative neural networks, such as Restricted Boltzmann Machines or
Variational Autoencoders, which are trained only on their own cluster using
differentially private gradient descent. We evaluate our approach using the
MNIST dataset, as well as call detail records and transit datasets, showing
that it produces realistic synthetic samples, which can also be used to
accurately compute arbitrary number of counting queries.