pwc-archive/papers-with-abstracts huggingface dataset:
Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks
Authors: Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, Ananthram Swami
Date: 2015-11-14 00:00:00
Tasks: Autonomous Vehicles, BIG-bench Machine Learning

Deep learning algorithms have been shown to perform extremely well on many
classical machine learning problems. However, recent studies have shown that
deep learning, like other machine learning techniques, is vulnerable to
adversarial samples: inputs crafted to force a deep neural network (DNN) to
provide adversary-selected outputs. Such attacks can seriously undermine the
security of the system supported by the DNN, sometimes with devastating
consequences. For example, autonomous vehicles can be crashed, illicit or
illegal content can bypass content filters, or biometric authentication systems
can be manipulated to allow improper access. In this work, we introduce a
defensive mechanism called defensive distillation to reduce the effectiveness
of adversarial samples on DNNs. We analytically investigate the
generalizability and robustness properties granted by the use of defensive
distillation when training DNNs. We also empirically study the effectiveness of
our defense mechanisms on two DNNs placed in adversarial settings. The study
shows that defensive distillation can reduce effectiveness of sample creation
from 95% to less than 0.5% on a studied DNN. Such dramatic gains can be
explained by the fact that distillation leads gradients used in adversarial
sample creation to be reduced by a factor of 10^30. We also find that
distillation increases the average minimum number of features that need to be
modified to create adversarial samples by about 800% on one of the DNNs we
tested.
A Variational Perspective on Accelerated Methods in Optimization
Authors: Andre Wibisono, Ashia C. Wilson, Michael. I. Jordan
Date: 2016-03-14 00:00:00

Accelerated gradient methods play a central role in optimization, achieving
optimal rates in many settings. While many generalizations and extensions of
Nesterov's original acceleration method have been proposed, it is not yet clear
what is the natural scope of the acceleration concept. In this paper, we study
accelerated methods from a continuous-time perspective. We show that there is a
Lagrangian functional that we call the \emph{Bregman Lagrangian} which
generates a large class of accelerated methods in continuous time, including
(but not limited to) accelerated gradient descent, its non-Euclidean extension,
and accelerated higher-order gradient methods. We show that the continuous-time
limit of all of these methods correspond to traveling the same curve in
spacetime at different speeds. From this perspective, Nesterov's technique and
many of its generalizations can be viewed as a systematic way to go from the
continuous-time curves generated by the Bregman Lagrangian to a family of
discrete-time accelerated algorithms.
Graph Based Sinogram Denoising for Tomographic Reconstructions
Authors: Faisal Mahmood, Nauman Shahid, Pierre Vandergheynst, Ulf Skoglund
Date: 2016-03-14 00:00:00
Tasks: Denoising, Tomographic Reconstructions

Limited data and low dose constraints are common problems in a variety of
tomographic reconstruction paradigms which lead to noisy and incomplete data.
Over the past few years sinogram denoising has become an essential
pre-processing step for low dose Computed Tomographic (CT) reconstructions. We
propose a novel sinogram denoising algorithm inspired by the modern field of
signal processing on graphs. Graph based methods often perform better than
standard filtering operations since they can exploit the signal structure. This
makes the sinogram an ideal candidate for graph based denoising since it
generally has a piecewise smooth structure. We test our method with a variety
of phantoms and different reconstruction methods. Our numerical study shows
that the proposed algorithm improves the performance of analytical filtered
back-projection (FBP) and iterative methods ART (Kaczmarz) and SIRT
(Cimmino).We observed that graph denoised sinogram always minimizes the error
measure and improves the accuracy of the solution as compared to regular
reconstructions.
Studying a set of properties of inconsistency indices for pairwise comparisons
Authors: Matteo Brunelli
Date: 2015-07-31 00:00:00

Pairwise comparisons between alternatives are a well-established tool to
decompose decision problems into smaller and more easily tractable
sub-problems. However, due to our limited rationality, the subjective
preferences expressed by decision makers over pairs of alternatives can hardly
ever be consistent. Therefore, several inconsistency indices have been proposed
in the literature to quantify the extent of the deviation from complete
consistency. Only recently, a set of properties has been proposed to define a
family of functions representing inconsistency indices. The scope of this paper
is twofold. Firstly, it expands the set of properties by adding and justifying
a new one. Secondly, it continues the study of inconsistency indices to check
whether or not they satisfy the above mentioned properties. Out of the four
indices considered in this paper, in its present form, two fail to satisfy some
properties. An adjusted version of one index is proposed so that it fulfills
them.
Graphical Representation for Heterogeneous Face Recognition
Authors: Chunlei Peng, Xinbo Gao, Nannan Wang, Jie Li
Date: 2015-03-02 00:00:00
Tasks: Face Recognition, Heterogeneous Face Recognition

Heterogeneous face recognition (HFR) refers to matching face images acquired
from different sources (i.e., different sensors or different wavelengths) for
identification. HFR plays an important role in both biometrics research and
industry. In spite of promising progresses achieved in recent years, HFR is
still a challenging problem due to the difficulty to represent two
heterogeneous images in a homogeneous manner. Existing HFR methods either
represent an image ignoring the spatial information, or rely on a
transformation procedure which complicates the recognition task. Considering
these problems, we propose a novel graphical representation based HFR method
(G-HFR) in this paper. Markov networks are employed to represent heterogeneous
image patches separately, which takes the spatial compatibility between
neighboring image patches into consideration. A coupled representation
similarity metric (CRSM) is designed to measure the similarity between obtained
graphical representations. Extensive experiments conducted on multiple HFR
scenarios (viewed sketch, forensic sketch, near infrared image, and thermal
infrared image) show that the proposed method outperforms state-of-the-art
methods.
Top-$K$ Ranking from Pairwise Comparisons: When Spectral Ranking is Optimal
Authors: Minje Jang, Sunghyun Kim, Changho Suh, Sewoong Oh
Date: 2016-03-14 00:00:00

We explore the top-$K$ rank aggregation problem. Suppose a collection of
items is compared in pairs repeatedly, and we aim to recover a consistent
ordering that focuses on the top-$K$ ranked items based on partially revealed
preference information. We investigate the Bradley-Terry-Luce model in which
one ranks items according to their perceived utilities modeled as noisy
observations of their underlying true utilities. Our main contributions are
two-fold. First, in a general comparison model where item pairs to compare are
given a priori, we attain an upper and lower bound on the sample size for
reliable recovery of the top-$K$ ranked items. Second, more importantly,
extending the result to a random comparison model where item pairs to compare
are chosen independently with some probability, we show that in slightly
restricted regimes, the gap between the derived bounds reduces to a constant
factor, hence reveals that a spectral method can achieve the minimax optimality
on the (order-wise) sample size required for top-$K$ ranking. That is to say,
we demonstrate a spectral method alone to be sufficient to achieve the
optimality and advantageous in terms of computational complexity, as it does
not require an additional stage of maximum likelihood estimation that a
state-of-the-art scheme employs to achieve the optimality. We corroborate our
main results by numerical experiments.
Regression-based Hypergraph Learning for Image Clustering and Classification
Authors: Sheng Huang, Dan Yang, Bo Liu, Xiaohong Zhang
Date: 2016-03-14 00:00:00
Tasks: Classification, Clustering, General Classification, Image Clustering, regression

Inspired by the recently remarkable successes of Sparse Representation (SR),
Collaborative Representation (CR) and sparse graph, we present a novel
hypergraph model named Regression-based Hypergraph (RH) which utilizes the
regression models to construct the high quality hypergraphs. Moreover, we plug
RH into two conventional hypergraph learning frameworks, namely hypergraph
spectral clustering and hypergraph transduction, to present Regression-based
Hypergraph Spectral Clustering (RHSC) and Regression-based Hypergraph
Transduction (RHT) models for addressing the image clustering and
classification issues. Sparse Representation and Collaborative Representation
are employed to instantiate two RH instances and their RHSC and RHT algorithms.
The experimental results on six popular image databases demonstrate that the
proposed RH learning algorithms achieve promising image clustering and
classification performances, and also validate that RH can inherit the
desirable properties from both hypergraph models and regression models.
Exploratory Gradient Boosting for Reinforcement Learning in Complex Domains
Authors: David Abel, Alekh Agarwal, Fernando Diaz, Akshay Krishnamurthy, Robert E. Schapire
Date: 2016-03-14 00:00:00
Tasks: Minecraft, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

High-dimensional observations and complex real-world dynamics present major
challenges in reinforcement learning for both function approximation and
exploration. We address both of these challenges with two complementary
techniques: First, we develop a gradient-boosting style, non-parametric
function approximator for learning on $Q$-function residuals. And second, we
propose an exploration strategy inspired by the principles of state abstraction
and information acquisition under uncertainty. We demonstrate the empirical
effectiveness of these techniques, first, as a preliminary check, on two
standard tasks (Blackjack and $n$-Chain), and then on two much larger and more
realistic tasks with high-dimensional observation spaces. Specifically, we
introduce two benchmarks built within the game Minecraft where the observations
are pixel arrays of the agent's visual field. A combination of our two
algorithmic techniques performs competitively on the standard
reinforcement-learning tasks while consistently and substantially outperforming
baselines on the two tasks with high-dimensional observation spaces. The new
function approximator, exploration strategy, and evaluation benchmarks are each
of independent interest in the pursuit of reinforcement-learning methods that
scale to real-world domains.
Multi-modal Tracking for Object based SLAM
Authors: Prateek Singhal, Ruffin White, Henrik Christensen
Date: 2016-03-14 00:00:00
Tasks: Object, Object Tracking, Visual Object Tracking, Visual Odometry

We present an on-line 3D visual object tracking framework for monocular
cameras by incorporating spatial knowledge and uncertainty from semantic
mapping along with high frequency measurements from visual odometry. Using a
combination of vision and odometry that are tightly integrated we can increase
the overall performance of object based tracking for semantic mapping. We
present a framework for integration of the two data-sources into a coherent
framework through information based fusion/arbitration. We demonstrate the
framework in the context of OmniMapper[1] and present results on 6 challenging
sequences over multiple objects compared to data obtained from a motion capture
systems. We are able to achieve a mean error of 0.23m for per frame tracking
showing 9% relative error less than state of the art tracker.
Learning Binary Codes and Binary Weights for Efficient Classification
Authors: Fumin Shen, Yadong Mu, Wei Liu, Yang Yang, Heng Tao Shen
Date: 2016-03-14 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Image Retrieval

This paper proposes a generic formulation that significantly expedites the
training and deployment of image classification models, particularly under the
scenarios of many image categories and high feature dimensions. As a defining
property, our method represents both the images and learned classifiers using
binary hash codes, which are simultaneously learned from the training data.
Classifying an image thereby reduces to computing the Hamming distance between
the binary codes of the image and classifiers and selecting the class with
minimal Hamming distance. Conventionally, compact hash codes are primarily used
for accelerating image search. Our work is first of its kind to represent
classifiers using binary codes. Specifically, we formulate multi-class image
classification as an optimization problem over binary variables. The
optimization alternatively proceeds over the binary classifiers and image hash
codes. Profiting from the special property of binary codes, we show that the
sub-problems can be efficiently solved through either a binary quadratic
program (BQP) or linear program. In particular, for attacking the BQP problem,
we propose a novel bit-flipping procedure which enjoys high efficacy and local
optimality guarantee. Our formulation supports a large family of empirical loss
functions and is here instantiated by exponential / hinge losses. Comprehensive
evaluations are conducted on several representative image benchmarks. The
experiments consistently observe reduced complexities of model training and
deployment, without sacrifice of accuracies.
A Stochastic Approach to STDP
Authors: Runchun Wang, Chetan Singh Thakur, Tara Julia Hamilton, Jonathan Tapson, André van Schaik
Date: 2016-03-13 00:00:00
Tasks: 8k

We present a digital implementation of the Spike Timing Dependent Plasticity
(STDP) learning rule. The proposed digital implementation consists of an
exponential decay generator array and a STDP adaptor array. On the arrival of a
pre- and post-synaptic spike, the STDP adaptor will send a digital spike to the
decay generator. The decay generator will then generate an exponential decay,
which will be used by the STDP adaptor to perform the weight adaption. The
exponential decay, which is computational expensive, is efficiently implemented
by using a novel stochastic approach, which we analyse and characterise here.
We use a time multiplexing approach to achieve 8192 (8k) virtual STDP adaptors
and decay generators with only one physical implementation of each. We have
validated our stochastic STDP approach with measurement results of a balanced
excitation/inhibition experiment. Our stochastic approach is ideal for
implementing the STDP learning rule in large-scale spiking neural networks
running in real time.
Exploring the Neural Algorithm of Artistic Style
Authors: Yaroslav Nikulin, Roman Novak
Date: 2016-02-23 00:00:00
Tasks: Style Transfer

We explore the method of style transfer presented in the article "A Neural
Algorithm of Artistic Style" by Leon A. Gatys, Alexander S. Ecker and Matthias
Bethge (arXiv:1508.06576).
  We first demonstrate the power of the suggested style space on a few
examples. We then vary different hyper-parameters and program properties that
were not discussed in the original paper, among which are the recognition
network used, starting point of the gradient descent and different ways to
partition style and content layers. We also give a brief comparison of some of
the existing algorithm implementations and deep learning frameworks used.
  To study the style space further we attempt to generate synthetic images by
maximizing a single entry in one of the Gram matrices $\mathcal{G}_l$ and some
interesting results are observed. Next, we try to mimic the sparsity and
intensity distribution of Gram matrices obtained from a real painting and
generate more complex textures.
  Finally, we propose two new style representations built on top of network's
features and discuss how one could be used to achieve local and potentially
content-aware style transfer.
A Grothendieck-type inequality for local maxima
Authors: Andrea Montanari
Date: 2016-03-13 00:00:00
Tasks: Vocal Bursts Type Prediction

A large number of problems in optimization, machine learning, signal
processing can be effectively addressed by suitable semidefinite programming
(SDP) relaxations. Unfortunately, generic SDP solvers hardly scale beyond
instances with a few hundreds variables (in the underlying combinatorial
problem). On the other hand, it has been observed empirically that an effective
strategy amounts to introducing a (non-convex) rank constraint, and solving the
resulting smooth optimization problem by ascent methods. This non-convex
problem has --generically-- a large number of local maxima, and the reason for
this success is therefore unclear.
  This paper provides rigorous support for this approach. For the problem of
maximizing a linear functional over the elliptope, we prove that all local
maxima are within a small gap from the SDP optimum. In several problems of
interest, arbitrarily small relative error can be achieved by taking the rank
constraint $k$ to be of order one, independently of the problem size.
Revisiting Active Perception
Authors: Ruzena Bajcsy, Yiannis Aloimonos, John K. Tsotsos
Date: 2016-03-08 00:00:00

Despite the recent successes in robotics, artificial intelligence and
computer vision, a complete artificial agent necessarily must include active
perception. A multitude of ideas and methods for how to accomplish this have
already appeared in the past, their broader utility perhaps impeded by
insufficient computational power or costly hardware. The history of these
ideas, perhaps selective due to our perspectives, is presented with the goal of
organizing the past literature and highlighting the seminal contributions. We
argue that those contributions are as relevant today as they were decades ago
and, with the state of modern computational tools, are poised to find new life
in the robotic perception systems of the next decade.
Triplet Similarity Embedding for Face Verification
Authors: Swami Sankaranarayanan, Azadeh Alavi, Rama Chellappa
Date: 2016-02-10 00:00:00
Tasks: Face Verification, Triplet

In this work, we present an unconstrained face verification algorithm and
evaluate it on the recently released IJB-A dataset that aims to push the
boundaries of face verification methods. The proposed algorithm couples a deep
CNN-based approach with a low-dimensional discriminative embedding learnt using
triplet similarity constraints in a large margin fashion. Aside from yielding
performance improvement, this embedding provides significant advantages in
terms of memory and post-processing operations like hashing and visualization.
Experiments on the IJB-A dataset show that the proposed algorithm outperforms
state of the art methods in verification and identification metrics, while
requiring less training time.
Divide and Conquer Local Average Regression
Authors: Xiangyu Chang, Shao-Bo Lin, Yao Wang
Date: 2016-01-23 00:00:00
Tasks: regression

The divide and conquer strategy, which breaks a massive data set into a se-
ries of manageable data blocks, and then combines the independent results of
data blocks to obtain a final decision, has been recognized as a
state-of-the-art method to overcome challenges of massive data analysis. In
this paper, we merge the divide and conquer strategy with local average
regression methods to infer the regressive relationship of input-output pairs
from a massive data set. After theoretically analyzing the pros and cons, we
find that although the divide and conquer local average regression can reach
the optimal learning rate, the restric- tion to the number of data blocks is a
bit strong, which makes it only feasible for small number of data blocks. We
then propose two variants to lessen (or remove) this restriction. Our results
show that these variants can achieve the optimal learning rate with much milder
restriction (or without such restriction). Extensive experimental studies are
carried out to verify our theoretical assertions.
Image and Depth from a Single Defocused Image Using Coded Aperture Photography
Authors: Mina Masoudifar, Hamid Reza Pourreza
Date: 2016-03-13 00:00:00
Tasks: Deblurring, Depth Estimation, Image Quality Assessment

Depth from defocus and defocus deblurring from a single image are two
challenging problems that are derived from the finite depth of field in
conventional cameras. Coded aperture imaging is one of the techniques that is
used for improving the results of these two problems. Up to now, different
methods have been proposed for improving the results of either defocus
deblurring or depth estimation. In this paper, a multi-objective function is
proposed for evaluating and designing aperture patterns with the aim of
improving the results of both depth from defocus and defocus deblurring.
Pattern evaluation is performed by considering the scene illumination condition
and camera system specification. Based on the proposed criteria, a single
asymmetric pattern is designed that is used for restoring a sharp image and a
depth map from a single input. Since the designed pattern is asymmetric,
defocus objects on the two sides of the focal plane can be distinguished. Depth
estimation is performed by using a new algorithm, which is based on image
quality assessment criteria and can distinguish between blurred objects lying
in front or behind the focal plane. Extensive simulations as well as
experiments on a variety of real scenes are conducted to compare our aperture
with previously proposed ones.
Deep Interactive Object Selection
Authors: Ning Xu, Brian Price, Scott Cohen, Jimei Yang, Thomas Huang
Date: 2016-03-13 00:00:00
Proceeding: CVPR 2016 6
Tasks: Interactive Segmentation, Object

Interactive object selection is a very important research problem and has
many applications. Previous algorithms require substantial user interactions to
estimate the foreground and background distributions. In this paper, we present
a novel deep learning based algorithm which has a much better understanding of
objectness and thus can reduce user interactions to just a few clicks. Our
algorithm transforms user provided positive and negative clicks into two
Euclidean distance maps which are then concatenated with the RGB channels of
images to compose (image, user interactions) pairs. We generate many of such
pairs by combining several random sampling strategies to model user click
patterns and use them to fine tune deep Fully Convolutional Networks (FCNs).
Finally the output probability maps of our FCN 8s model is integrated with
graph cut optimization to refine the boundary segments. Our model is trained on
the PASCAL segmentation dataset and evaluated on other datasets with different
object classes. Experimental results on both seen and unseen objects clearly
demonstrate that our algorithm has a good generalization ability and is
superior to all existing interactive object selection approaches.