pwc-archive/papers-with-abstracts huggingface dataset:
Learning Robust Deep Face Representation
Authors: Xiang Wu
Date: 2015-07-17 00:00:00
Tasks: Face Recognition

With the development of convolution neural network, more and more researchers
focus their attention on the advantage of CNN for face recognition task. In
this paper, we propose a deep convolution network for learning a robust face
representation. The deep convolution net is constructed by 4 convolution
layers, 4 max pooling layers and 2 fully connected layers, which totally
contains about 4M parameters. The Max-Feature-Map activation function is used
instead of ReLU because the ReLU might lead to the loss of information due to
the sparsity while the Max-Feature-Map can get the compact and discriminative
feature vectors. The model is trained on CASIA-WebFace dataset and evaluated on
LFW dataset. The result on LFW achieves 97.77% on unsupervised setting for
single net.
Parallel Magnetic Resonance Imaging
Authors: Martin Uecker
Date: 2015-01-25 00:00:00
Tasks: compressed sensing, Image Reconstruction

The main disadvantage of Magnetic Resonance Imaging (MRI) are its long scan
times and, in consequence, its sensitivity to motion. Exploiting the
complementary information from multiple receive coils, parallel imaging is able
to recover images from under-sampled k-space data and to accelerate the
measurement. Because parallel magnetic resonance imaging can be used to
accelerate basically any imaging sequence it has many important applications.
Parallel imaging brought a fundamental shift in image reconstruction: Image
reconstruction changed from a simple direct Fourier transform to the solution
of an ill-conditioned inverse problem. This work gives an overview of image
reconstruction from the perspective of inverse problems. After introducing
basic concepts such as regularization, discretization, and iterative
reconstruction, advanced topics are discussed including algorithms for
auto-calibration, the connection to approximation theory, and the combination
with compressed sensing.
Multiscale Adaptive Representation of Signals: I. The Basic Framework
Authors: Cheng Tai, Weinan E
Date: 2015-07-17 00:00:00
Tasks: Computational Efficiency, Denoising, Dictionary Learning, Object Recognition

We introduce a framework for designing multi-scale, adaptive, shift-invariant
frames and bi-frames for representing signals. The new framework, called
AdaFrame, improves over dictionary learning-based techniques in terms of
computational efficiency at inference time. It improves classical multi-scale
basis such as wavelet frames in terms of coding efficiency. It provides an
attractive alternative to dictionary learning-based techniques for low level
signal processing tasks, such as compression and denoising, as well as high
level tasks, such as feature extraction for object recognition. Connections
with deep convolutional networks are also discussed. In particular, the
proposed framework reveals a drawback in the commonly used approach for
visualizing the activations of the intermediate layers in convolutional
networks, and suggests a natural alternative.
Deep Multimodal Speaker Naming
Authors: Yongtao Hu, Jimmy Ren, Jingwen Dai, Chang Yuan, Li Xu, Wenping Wang
Date: 2015-07-17 00:00:00
Tasks: Face Alignment

Automatic speaker naming is the problem of localizing as well as identifying
each speaking character in a TV/movie/live show video. This is a challenging
problem mainly attributes to its multimodal nature, namely face cue alone is
insufficient to achieve good performance. Previous multimodal approaches to
this problem usually process the data of different modalities individually and
merge them using handcrafted heuristics. Such approaches work well for simple
scenes, but fail to achieve high performance for speakers with large appearance
variations. In this paper, we propose a novel convolutional neural networks
(CNN) based learning framework to automatically learn the fusion function of
both face and audio cues. We show that without using face tracking, facial
landmark localization or subtitle/transcript, our system with robust multimodal
feature extraction is able to achieve state-of-the-art speaker naming
performance evaluated on two diverse TV series. The dataset and implementation
of our algorithm are publicly available online.
RBIR Based on Signature Graph
Authors: Thanh The Van, Thanh Manh Le
Date: 2015-07-17 00:00:00
Tasks: Image Retrieval, Retrieval

This paper approaches the image retrieval system on the base of visual
features local region RBIR (region-based image retrieval). First of all, the
paper presents a method for extracting the interest points based on
Harris-Laplace to create the feature region of the image. Next, in order to
reduce the storage space and speed up query image, the paper builds the binary
signature structure to describe the visual content of image. Based on the
image's binary signature, the paper builds the SG (signature graph) to classify
and store image's binary signatures. Since then, the paper builds the image
retrieval algorithm on SG through the similar measure EMD (earth mover's
distance) between the image's binary signatures. Last but not least, the paper
gives an image retrieval model RBIR, experiments and assesses the image
retrieval method on Corel image database over 10,000 images.
Lens Factory: Automatic Lens Generation Using Off-the-shelf Components
Authors: Libin Sun, Brian Guenter, Neel Joshi, Patrick Therien, James Hays
Date: 2015-06-30 00:00:00

Custom optics is a necessity for many imaging applications. Unfortunately,
custom lens design is costly (thousands to tens of thousands of dollars), time
consuming (10-12 weeks typical lead time), and requires specialized optics
design expertise. By using only inexpensive, off-the-shelf lens components the
Lens Factory automatic design system greatly reduces cost and time. Design,
ordering of parts, delivery, and assembly can be completed in a few days, at a
cost in the low hundreds of dollars. Lens design constraints, such as focal
length and field of view, are specified in terms familiar to the graphics
community so no optics expertise is necessary. Unlike conventional lens design
systems, which only use continuous optimization methods, Lens Factory adds a
discrete optimization stage. This stage searches the combinatorial space of
possible combinations of lens elements to find novel designs, evolving simple
canonical lens designs into more complex, better designs. Intelligent pruning
rules make the combinatorial search feasible. We have designed and built
several high performance optical systems which demonstrate the practicality of
the system.
Exploratory topic modeling with distributional semantics
Authors: Samuel Rönnqvist
Date: 2015-07-16 00:00:00
Tasks: Semantic Similarity, Semantic Textual Similarity

As we continue to collect and store textual data in a multitude of domains,
we are regularly confronted with material whose largely unknown thematic
structure we want to uncover. With unsupervised, exploratory analysis, no prior
knowledge about the content is required and highly open-ended tasks can be
supported. In the past few years, probabilistic topic modeling has emerged as a
popular approach to this problem. Nevertheless, the representation of the
latent topics as aggregations of semi-coherent terms limits their
interpretability and level of detail.
  This paper presents an alternative approach to topic modeling that maps
topics as a network for exploration, based on distributional semantics using
learned word vectors. From the granular level of terms and their semantic
similarity relations global topic structures emerge as clustered regions and
gradients of concepts. Moreover, the paper discusses the visual interactive
representation of the topic map, which plays an important role in supporting
its exploration.
Deep Learning and Music Adversaries
Authors: Corey Kereliuk, Bob L. Sturm, Jan Larsen
Date: 2015-07-16 00:00:00
Tasks: Deep Learning, Object Recognition, valid

An adversary is essentially an algorithm intent on making a classification
system perform in some particular way given an input, e.g., increase the
probability of a false negative. Recent work builds adversaries for deep
learning systems applied to image object recognition, which exploits the
parameters of the system to find the minimal perturbation of the input image
such that the network misclassifies it with high confidence. We adapt this
approach to construct and deploy an adversary of deep learning systems applied
to music content analysis. In our case, however, the input to the systems is
magnitude spectral frames, which requires special care in order to produce
valid input audio signals from network-derived perturbations. For two different
train-test partitionings of two benchmark datasets, and two different deep
architectures, we find that this adversary is very effective in defeating the
resulting systems. We find the convolutional networks are more robust, however,
compared with systems based on a majority vote over individually classified
audio frames. Furthermore, we integrate the adversary into the training of new
deep systems, but do not find that this improves their resilience against the
same adversary.
Recursive Sparse Point Process Regression with Application to Spectrotemporal Receptive Field Plasticity Analysis
Authors: Alireza Sheikhattar, Jonathan B. Fritz, Shihab A. Shamma, Behtash Babadi
Date: 2015-07-16 00:00:00
Tasks: compressed sensing, regression, Time Series, Time Series Analysis

We consider the problem of estimating the sparse time-varying parameter
vectors of a point process model in an online fashion, where the observations
and inputs respectively consist of binary and continuous time series. We
construct a novel objective function by incorporating a forgetting factor
mechanism into the point process log-likelihood to enforce adaptivity and
employ $\ell_1$-regularization to capture the sparsity. We provide a rigorous
analysis of the maximizers of the objective function, which extends the
guarantees of compressed sensing to our setting. We construct two recursive
filters for online estimation of the parameter vectors based on proximal
optimization techniques, as well as a novel filter for recursive computation of
statistical confidence regions. Simulation studies reveal that our algorithms
outperform several existing point process filters in terms of trackability,
goodness-of-fit and mean square error. We finally apply our filtering
algorithms to experimentally recorded spiking data from the ferret primary
auditory cortex during attentive behavior in a click rate discrimination task.
Our analysis provides new insights into the time-course of the spectrotemporal
receptive field plasticity of the auditory neurons.
Iterative Subsampling in Solution Path Clustering of Noisy Big Data
Authors: Yuliya Marchetti, Qing Zhou
Date: 2014-12-04 00:00:00
Tasks: Clustering, Computational Efficiency

We develop an iterative subsampling approach to improve the computational
efficiency of our previous work on solution path clustering (SPC). The SPC
method achieves clustering by concave regularization on the pairwise distances
between cluster centers. This clustering method has the important capability to
recognize noise and to provide a short path of clustering solutions; however,
it is not sufficiently fast for big datasets. Thus, we propose a method that
iterates between clustering a small subsample of the full data and sequentially
assigning the other data points to attain orders of magnitude of computational
savings. The new method preserves the ability to isolate noise, includes a
solution selection mechanism that ultimately provides one clustering solution
with an estimated number of clusters, and is shown to be able to extract small
tight clusters from noisy data. The method's relatively minor losses in
accuracy are demonstrated through simulation studies, and its ability to handle
large datasets is illustrated through applications to gene expression datasets.
An R package, SPClustering, for the SPC method with iterative subsampling is
available at http://www.stat.ucla.edu/~zhou/Software.html.
A Dependency-Based Neural Network for Relation Classification
Authors: Yang Liu, Furu Wei, Sujian Li, Heng Ji, Ming Zhou, Houfeng Wang
Date: 2015-07-16 00:00:00
Proceeding: IJCNLP 2015 7
Tasks: Classification, General Classification, Relation, Relation Classification

Previous research on relation classification has verified the effectiveness
of using dependency shortest paths or subtrees. In this paper, we further
explore how to make full use of the combination of these dependency
information. We first propose a new structure, termed augmented dependency path
(ADP), which is composed of the shortest dependency path between two entities
and the subtrees attached to the shortest path. To exploit the semantic
representation behind the ADP structure, we develop dependency-based neural
networks (DepNN): a recursive neural network designed to model the subtrees,
and a convolutional neural network to capture the most important features on
the shortest path. Experiments on the SemEval-2010 dataset show that our
proposed method achieves state-of-art results.
Optimizing the computation of overriding
Authors: Piero Andrea Bonatti, Iliana Mineva Petrova, Luigi Sauro
Date: 2015-07-16 00:00:00

We introduce optimization techniques for reasoning in DLN---a recently
introduced family of nonmonotonic description logics whose characterizing
features appear well-suited to model the applicative examples naturally arising
in biomedical domains and semantic web access control policies. Such
optimizations are validated experimentally on large KBs with more than 30K
axioms. Speedups exceed 1 order of magnitude. For the first time, response
times compatible with real-time reasoning are obtained with nonmonotonic KBs of
this size.
Constructing Binary Descriptors with a Stochastic Hill Climbing Search
Authors: Nenad Markuš, Igor S. Pandžić, Jörgen Ahlberg
Date: 2015-01-20 00:00:00

Binary descriptors of image patches provide processing speed advantages and
require less storage than methods that encode the patch appearance with a
vector of real numbers. We provide evidence that, despite its simplicity, a
stochastic hill climbing bit selection procedure for descriptor construction
defeats recently proposed alternatives on a standard discriminative power
benchmark. The method is easy to implement and understand, has no free
parameters that need fine tuning, and runs fast.
Upper-Confidence-Bound Algorithms for Active Learning in Multi-Armed Bandits
Authors: Alexandra Carpentier, Alessandro Lazaric, Mohammad Ghavamzadeh, Rémi Munos, Peter Auer, András Antos
Date: 2015-07-16 00:00:00
Tasks: Active Learning, Multi-Armed Bandits

In this paper, we study the problem of estimating uniformly well the mean
values of several distributions given a finite budget of samples. If the
variance of the distributions were known, one could design an optimal sampling
strategy by collecting a number of independent samples per distribution that is
proportional to their variance. However, in the more realistic case where the
distributions are not known in advance, one needs to design adaptive sampling
strategies in order to select which distribution to sample from according to
the previously observed samples. We describe two strategies based on pulling
the distributions a number of times that is proportional to a high-probability
upper-confidence-bound on their variance (built from previous observed samples)
and report a finite-sample performance analysis on the excess estimation error
compared to the optimal allocation. We show that the performance of these
allocation strategies depends not only on the variances but also on the full
shape of the distributions.
Scalable Gaussian Process Classification via Expectation Propagation
Authors: Daniel Hernández-Lobato, José Miguel Hernández-Lobato
Date: 2015-07-16 00:00:00
Tasks: Classification, General Classification

Variational methods have been recently considered for scaling the training
process of Gaussian process classifiers to large datasets. As an alternative,
we describe here how to train these classifiers efficiently using expectation
propagation. The proposed method allows for handling datasets with millions of
data instances. More precisely, it can be used for (i) training in a
distributed fashion where the data instances are sent to different nodes in
which the required computations are carried out, and for (ii) maximizing an
estimate of the marginal likelihood using a stochastic approximation of the
gradient. Several experiments indicate that the method described is competitive
with the variational approach.
Diagnosing State-Of-The-Art Object Proposal Methods
Authors: Hongyuan Zhu, Shijian Lu, Jianfei Cai, Quangqing Lee
Date: 2015-07-16 00:00:00
Tasks: Object, object-detection, Object Detection

Object proposal has become a popular paradigm to replace exhaustive sliding
window search in current top-performing methods in PASCAL VOC and ImageNet.
Recently, Hosang et al. conduct the first unified study of existing methods' in
terms of various image-level degradations. On the other hand, the vital
question "what object-level characteristics really affect existing methods'
performance?" is not yet answered. Inspired by Hoiem et al.'s work in
categorical object detection, this paper conducts the first meta-analysis of
various object-level characteristics' impact on state-of-the-art object
proposal methods. Specifically, we examine the effects of object size, aspect
ratio, iconic view, color contrast, shape regularity and texture. We also
analyse existing methods' localization accuracy and latency for various PASCAL
VOC object classes. Our study reveals the limitations of existing methods in
terms of non-iconic view, small object size, low color contrast, shape
regularity etc. Based on our observations, lessons are also learned and shared
with respect to the selection of existing object proposal technologies as well
as the design of the future ones.
On the Convergence of Stochastic Variational Inference in Bayesian Networks
Authors: Ulrich Paquet
Date: 2015-07-16 00:00:00
Tasks: Variational Inference

We highlight a pitfall when applying stochastic variational inference to
general Bayesian networks. For global random variables approximated by an
exponential family distribution, natural gradient steps, commonly starting from
a unit length step size, are averaged to convergence. This useful insight into
the scaling of initial step sizes is lost when the approximation factorizes
across a general Bayesian network, and care must be taken to ensure practical
convergence. We experimentally investigate how much of the baby (well-scaled
steps) is thrown out with the bath water (exact gradients).
Towards Predicting First Daily Departure Times: a Gaussian Modeling Approach for Load Shift Forecasting
Authors: Nicholas H. Kirk, Ilya Dianov
Date: 2015-07-16 00:00:00

This work provides two statistical Gaussian forecasting methods for
predicting First Daily Departure Times (FDDTs) of everyday use electric
vehicles. This is important in smart grid applications to understand
disconnection times of such mobile storage units, for instance to forecast
storage of non dispatchable loads (e.g. wind and solar power). We provide a
review of the relevant state-of-the-art driving behavior features towards FDDT
prediction, to then propose an approximated Gaussian method which qualitatively
forecasts how many vehicles will depart within a given time frame, by assuming
that departure times follow a normal distribution. This method considers
sampling sessions as Poisson distributions which are superimposed to obtain a
single approximated Gaussian model. Given the Gaussian distribution assumption
of the departure times, we also model the problem with Gaussian Mixture Models
(GMM), in which the priorly set number of clusters represents the desired time
granularity. Evaluation has proven that for the dataset tested, low error and
high confidence ($\approx 95\%$) is possible for 15 and 10 minute intervals,
and that GMM outperforms traditional modeling but is less generalizable across
datasets, as it is a closer fit to the sampling data. Conclusively we discuss
future possibilities and practical applications of the discussed model.
Massively Parallel Methods for Deep Reinforcement Learning
Authors: Arun Nair, Praveen Srinivasan, Sam Blackwell, Cagdas Alcicek, Rory Fearon, Alessandro De Maria, Vedavyas Panneershelvam, Mustafa Suleyman, Charles Beattie, Stig Petersen, Shane Legg, Volodymyr Mnih, Koray Kavukcuoglu, David Silver
Date: 2015-07-15 00:00:00
Tasks: Atari Games, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We present the first massively distributed architecture for deep
reinforcement learning. This architecture uses four main components: parallel
actors that generate new behaviour; parallel learners that are trained from
stored experience; a distributed neural network to represent the value function
or behaviour policy; and a distributed store of experience. We used our
architecture to implement the Deep Q-Network algorithm (DQN). Our distributed
algorithm was applied to 49 games from Atari 2600 games from the Arcade
Learning Environment, using identical hyperparameters. Our performance
surpassed non-distributed DQN in 41 of the 49 games and also reduced the
wall-time required to achieve these results by an order of magnitude on most
games.
How to Center Binary Deep Boltzmann Machines
Authors: Jan Melchior, Asja Fischer, Laurenz Wiskott
Date: 2013-11-06 00:00:00

This work analyzes centered binary Restricted Boltzmann Machines (RBMs) and
binary Deep Boltzmann Machines (DBMs), where centering is done by subtracting
offset values from visible and hidden variables. We show analytically that (i)
centering results in a different but equivalent parameterization for artificial
neural networks in general, (ii) the expected performance of centered binary
RBMs/DBMs is invariant under simultaneous flip of data and offsets, for any
offset value in the range of zero to one, (iii) centering can be reformulated
as a different update rule for normal binary RBMs/DBMs, and (iv) using the
enhanced gradient is equivalent to setting the offset values to the average
over model and data mean. Furthermore, numerical simulations suggest that (i)
optimal generative performance is achieved by subtracting mean values from
visible as well as hidden variables, (ii) centered RBMs/DBMs reach
significantly higher log-likelihood values than normal binary RBMs/DBMs, (iii)
centering variants whose offsets depend on the model mean, like the enhanced
gradient, suffer from severe divergence problems, (iv) learning is stabilized
if an exponentially moving average over the batch means is used for the offset
values instead of the current batch mean, which also prevents the enhanced
gradient from diverging, (v) centered RBMs/DBMs reach higher LL values than
normal RBMs/DBMs while having a smaller norm of the weight matrix, (vi)
centering leads to an update direction that is closer to the natural gradient
and that the natural gradient is extremly efficient for training RBMs, (vii)
centering dispense the need for greedy layer-wise pre-training of DBMs, (viii)
furthermore we show that pre-training often even worsen the results
independently whether centering is used or not, and (ix) centering is also
beneficial for auto encoders.