pwc-archive/papers-with-abstracts huggingface dataset:
Can We Boost the Power of the Viola-Jones Face Detector Using Pre-processing? An Empirical Study
Authors: Mahmoud Afifi, Marwa Nasser, Mostafa Korashy, Katherine Rohde, Aly Abdelrahim
Date: 2017-09-22 00:00:00
Tasks: Deblurring, Face Detection

The Viola-Jones face detection algorithm was (and still is) a quite popular
face detector. In spite of the numerous face detection techniques that have
been recently presented, there are many research works that are still based on
the Viola-Jones algorithm because of its simplicity. In this paper, we study
the influence of a set of blind pre-processing methods on the face detection
rate using the Viola-Jones algorithm. We focus on two aspects of improvement,
specifically badly illuminated faces and blurred faces. Many methods for
lighting invariant and deblurring are used in order to improve the detection
accuracy. We want to avoid using blind pre-processing methods that may obstruct
the face detector. To that end, we perform two sets of experiments. The first
set is performed to avoid any blind pre-processing method that may hurt the
face detector. The second set is performed to study the effect of the selected
pre-processing methods on images that suffer from hard conditions. We present
two manners of applying the pre-processing method to the image prior to being
used by the Viola-Jones face detector. Four different datasets are used to draw
a coherent conclusion about the potential improvement caused by using prior
enhanced images. The results demonstrate that some of the pre-processing
methods may hurt the accuracy of Viola-Jones face detection algorithm. However,
other pre-processing methods have an evident positive impact on the accuracy of
the face detector. Overall, we recommend three simple and fast blind
photometric normalization methods as a pre-processing step in order to improve
the accuracy of the pre-trained Viola-Jones face detector.
Long-Range Correlation Underlying Childhood Language and Generative Models
Authors: Kumiko Tanaka-Ishii
Date: 2017-12-11 00:00:00
Tasks: Time Series, Time Series Analysis

Long-range correlation, a property of time series exhibiting long-term
memory, is mainly studied in the statistical physics domain and has been
reported to exist in natural language. Using a state-of-the-art method for such
analysis, long-range correlation is first shown to occur in long CHILDES data
sets. To understand why, Bayesian generative models of language, originally
proposed in the cognitive scientific domain, are investigated. Among
representative models, the Simon model was found to exhibit surprisingly good
long-range correlation, but not the Pitman-Yor model. Since the Simon model is
known not to correctly reflect the vocabulary growth of natural language, a
simple new model is devised as a conjunct of the Simon and Pitman-Yor models,
such that long-range correlation holds with a correct vocabulary growth rate.
The investigation overall suggests that uniform sampling is one cause of
long-range correlation and could thus have a relation with actual linguistic
processes.
Lifting high-dimensional nonlinear models with Gaussian regressors
Authors: Christos Thrampoulidis, Ankit Singh Rawat
Date: 2017-12-11 00:00:00
Tasks: Vocal Bursts Intensity Prediction

We study the problem of recovering a structured signal $\mathbf{x}_0$ from
high-dimensional data $\mathbf{y}_i=f(\mathbf{a}_i^T\mathbf{x}_0)$ for some
nonlinear (and potentially unknown) link function $f$, when the regressors
$\mathbf{a}_i$ are iid Gaussian. Brillinger (1982) showed that ordinary
least-squares estimates $\mathbf{x}_0$ up to a constant of proportionality
$\mu_\ell$, which depends on $f$. Recently, Plan & Vershynin (2015) extended
this result to the high-dimensional setting deriving sharp error bounds for the
generalized Lasso. Unfortunately, both least-squares and the Lasso fail to
recover $\mathbf{x}_0$ when $\mu_\ell=0$. For example, this includes all even
link functions. We resolve this issue by proposing and analyzing an alternative
convex recovery method. In a nutshell, our method treats such link functions as
if they were linear in a lifted space of higher-dimension. Interestingly, our
error analysis captures the effect of both the nonlinearity and the problem's
geometry in a few simple summary parameters.
Distributed Solution of Large-Scale Linear Systems via Accelerated Projection-Based Consensus
Authors: Navid Azizan-Ruhi, Farshad Lahouti, Salman Avestimehr, Babak Hassibi
Date: 2017-08-04 00:00:00

Solving a large-scale system of linear equations is a key step at the heart
of many algorithms in machine learning, scientific computing, and beyond. When
the problem dimension is large, computational and/or memory constraints make it
desirable, or even necessary, to perform the task in a distributed fashion. In
this paper, we consider a common scenario in which a taskmaster intends to
solve a large-scale system of linear equations by distributing subsets of the
equations among a number of computing machines/cores. We propose an accelerated
distributed consensus algorithm, in which at each iteration every machine
updates its solution by adding a scaled version of the projection of an error
signal onto the nullspace of its system of equations, and where the taskmaster
conducts an averaging over the solutions with momentum. The convergence
behavior of the proposed algorithm is analyzed in detail and analytically shown
to compare favorably with the convergence rate of alternative distributed
methods, namely distributed gradient descent, distributed versions of
Nesterov's accelerated gradient descent and heavy-ball method, the block
Cimmino method, and ADMM. On randomly chosen linear systems, as well as on
real-world data sets, the proposed method offers significant speed-up relative
to all the aforementioned methods. Finally, our analysis suggests a novel
variation of the distributed heavy-ball method, which employs a particular
distributed preconditioning, and which achieves the same theoretical
convergence rate as the proposed consensus-based method.
Robust Deep Reinforcement Learning with Adversarial Attacks
Authors: Anay Pattanaik, Zhenyi Tang, Shuijing Liu, Gautham Bommannan, Girish Chowdhary
Date: 2017-12-11 00:00:00
Tasks: Deep Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper proposes adversarial attacks for Reinforcement Learning (RL) and
then improves the robustness of Deep Reinforcement Learning algorithms (DRL) to
parameter uncertainties with the help of these attacks. We show that even a
naively engineered attack successfully degrades the performance of DRL
algorithm. We further improve the attack using gradient information of an
engineered loss function which leads to further degradation in performance.
These attacks are then leveraged during training to improve the robustness of
RL within robust control framework. We show that this adversarial training of
DRL algorithms like Deep Double Q learning and Deep Deterministic Policy
Gradients leads to significant increase in robustness to parameter variations
for RL benchmarks such as Cart-pole, Mountain Car, Hopper and Half Cheetah
environment.
Towards Fully Environment-Aware UAVs: Real-Time Path Planning with Online 3D Wind Field Prediction in Complex Terrain
Authors: Philipp Oettershagen, Florian Achermann, Benjamin Müller, Daniel Schneider, Roland Siegwart
Date: 2017-12-10 00:00:00
Tasks: Motion Planning

Today, low-altitude fixed-wing Unmanned Aerial Vehicles (UAVs) are largely
limited to primitively follow user-defined waypoints. To allow fully-autonomous
remote missions in complex environments, real-time environment-aware navigation
is required both with respect to terrain and strong wind drafts. This paper
presents two relevant initial contributions: First, the literature's first-ever
3D wind field prediction method which can run in real time onboard a UAV is
presented. The approach retrieves low-resolution global weather data, and uses
potential flow theory to adjust the wind field such that terrain boundaries,
mass conservation, and the atmospheric stratification are observed. A
comparison with 1D LIDAR data shows an overall wind error reduction of 23% with
respect to the zero-wind assumption that is mostly used for UAV path planning
today. However, given that the vertical winds are not resolved accurately
enough further research is required and identified. Second, a sampling-based
path planner that considers the aircraft dynamics in non-uniform wind
iteratively via Dubins airplane paths is presented. Performance optimizations,
e.g. obstacle-aware sampling and fast 2.5D-map collision checks, render the
planner 50% faster than the Open Motion Planning Library (OMPL) implementation.
Test cases in Alpine terrain show that the wind-aware planning performs up to
50x less iterations than shortest-path planning and is thus slower in low
winds, but that it tends to deliver lower-cost paths in stronger winds. More
importantly, in contrast to the shortest-path planner, it always delivers
collision-free paths. Overall, our initial research demonstrates the
feasibility of 3D wind field prediction from a UAV and the advantages of
wind-aware planning. This paves the way for follow-up research on
fully-autonomous environment-aware navigation of UAVs in real-life missions and
complex terrain.
Sensitivity Analysis for Predictive Uncertainty in Bayesian Neural Networks
Authors: Stefan Depeweg, José Miguel Hernández-Lobato, Steffen Udluft, Thomas Runkler
Date: 2017-12-10 00:00:00
Tasks: Sensitivity

We derive a novel sensitivity analysis of input variables for predictive
epistemic and aleatoric uncertainty. We use Bayesian neural networks with
latent variables as a model class and illustrate the usefulness of our
sensitivity analysis on real-world datasets. Our method increases the
interpretability of complex black-box probabilistic models.
Shape optimization in laminar flow with a label-guided variational autoencoder
Authors: Stephan Eismann, Stefan Bartzsch, Stefano Ermon
Date: 2017-12-10 00:00:00
Tasks: Bayesian Optimization, regression

Computational design optimization in fluid dynamics usually requires to solve
non-linear partial differential equations numerically. In this work, we explore
a Bayesian optimization approach to minimize an object's drag coefficient in
laminar flow based on predicting drag directly from the object shape. Jointly
training an architecture combining a variational autoencoder mapping shapes to
latent representations and Gaussian process regression allows us to generate
improved shapes in the two dimensional case we consider.
Learning K-way D-dimensional Discrete Code For Compact Embedding Representations
Authors: Ting Chen, Martin Renqiang Min, Yizhou Sun
Date: 2017-11-08 00:00:00
Tasks: Language Modeling, Language Modelling

Embedding methods such as word embedding have become pillars for many
applications containing discrete structures. Conventional embedding methods
directly associate each symbol with a continuous embedding vector, which is
equivalent to applying linear transformation based on "one-hot" encoding of the
discrete symbols. Despite its simplicity, such approach yields number of
parameters that grows linearly with the vocabulary size and can lead to
overfitting. In this work we propose a much more compact K-way D-dimensional
discrete encoding scheme to replace the "one-hot" encoding. In "KD encoding",
each symbol is represented by a $D$-dimensional code, and each of its dimension
has a cardinality of $K$. The final symbol embedding vector can be generated by
composing the code embedding vectors. To learn the semantically meaningful
code, we derive a relaxed discrete optimization technique based on stochastic
gradient descent. By adopting the new coding system, the efficiency of
parameterization can be significantly improved (from linear to logarithmic),
and this can also mitigate the over-fitting problem. In our experiments with
language modeling, the number of embedding parameters can be reduced by 97\%
while achieving similar or better performance.
Sketch Layer Separation in Multi-Spectral Historical Document Images
Authors: AmirAbbas Davari, Armin Häberle, Vincent Christlein, Andreas Maier, Christian Riess
Date: 2017-12-10 00:00:00
Tasks: Diagnostic

High-resolution imaging has delivered new prospects for detecting the
material composition and structure of cultural treasures. Despite the various
techniques for analysis, a significant diagnostic gap remained in the range of
available research capabilities for works on paper. Old master drawings were
mostly composed in a multi-step manner with various materials. This resulted in
the overlapping of different layers which made the subjacent strata difficult
to differentiate. The separation of stratified layers using imaging methods
could provide insights into the artistic work processes and help answer
questions about the object, its attribution, or in identifying forgeries. The
pattern recognition procedure was tested with mock replicas to achieve the
separation and the capability of displaying concealed red chalk under ink. In
contrast to RGB-sensor based imaging, the multi- or hyperspectral technology
allows accurate layer separation by recording the characteristic signatures of
the material's reflectance. The risk of damage to the artworks as a result of
the examination can be reduced by using combinations of defined spectra for
lightning and image capturing. By guaranteeing the maximum level of
readability, our results suggest that the technique can be applied to a broader
range of objects and assist in diagnostic research into cultural treasures in
the future.
Tracking using Numerous Anchor points
Authors: Tanushri Chakravorty, Guillaume-Alexandre Bilodeau, Eric Granger
Date: 2017-02-07 00:00:00
Tasks: Object, Object Localization, Visual Tracking

In this paper, an online adaptive model-free tracker is proposed to track
single objects in video sequences to deal with real-world tracking challenges
like low-resolution, object deformation, occlusion and motion blur. The novelty
lies in the construction of a strong appearance model that captures features
from the initialized bounding box and then are assembled into anchor-point
features. These features memorize the global pattern of the object and have an
internal star graph-like structure. These features are unique and flexible and
helps tracking generic and deformable objects with no limitation on specific
objects. In addition, the relevance of each feature is evaluated online using
short-term consistency and long-term consistency. These parameters are adapted
to retain consistent features that vote for the object location and that deal
with outliers for long-term tracking scenarios. Additionally, voting in a
Gaussian manner helps in tackling inherent noise of the tracking system and in
accurate object localization. Furthermore, the proposed tracker uses pairwise
distance measure to cope with scale variations and combines pixel-level binary
features and global weighted color features for model update. Finally,
experimental results on a visual tracking benchmark dataset are presented to
demonstrate the effectiveness and competitiveness of the proposed tracker.
An Attention-Based Deep Net for Learning to Rank
Authors: Baiyang Wang, Diego Klabjan
Date: 2017-02-20 00:00:00
Tasks: Decoder, Image Retrieval, Information Retrieval, Learning-To-Rank, Retrieval

In information retrieval, learning to rank constructs a machine-based ranking
model which given a query, sorts the search results by their degree of
relevance or importance to the query. Neural networks have been successfully
applied to this problem, and in this paper, we propose an attention-based deep
neural network which better incorporates different embeddings of the queries
and search results with an attention-based mechanism. This model also applies a
decoder mechanism to learn the ranks of the search results in a listwise
fashion. The embeddings are trained with convolutional neural networks or the
word2vec model. We demonstrate the performance of this model with image
retrieval and text querying data sets.
Quantifying Translation-Invariance in Convolutional Neural Networks
Authors: Eric Kauderer-Abrams
Date: 2017-12-10 00:00:00
Tasks: Data Augmentation, Object Recognition, Translation

A fundamental problem in object recognition is the development of image
representations that are invariant to common transformations such as
translation, rotation, and small deformations. There are multiple hypotheses
regarding the source of translation invariance in CNNs. One idea is that
translation invariance is due to the increasing receptive field size of neurons
in successive convolution layers. Another possibility is that invariance is due
to the pooling operation. We develop a simple a tool, the
translation-sensitivity map, which we use to visualize and quantify the
translation-invariance of various architectures. We obtain the surprising
result that architectural choices such as the number of pooling layers and the
convolution filter size have only a secondary effect on the
translation-invariance of a network. Our analysis identifies training data
augmentation as the most important factor in obtaining translation-invariant
representations of images using convolutional neural networks.
SolarisNet: A Deep Regression Network for Solar Radiation Prediction
Authors: Subhadip Dey, Sawon Pratiher, Saon Banerjee, Chanchal Kumar Mukherjee
Date: 2017-11-22 00:00:00
Tasks: Prediction, regression

Effective utilization of photovoltaic (PV) plants requires weather
variability robust global solar radiation (GSR) forecasting models. Random
weather turbulence phenomena coupled with assumptions of clear sky model as
suggested by Hottel pose significant challenges to parametric & non-parametric
models in GSR conversion rate estimation. Also, a decent GSR estimate requires
costly high-tech radiometer and expert dependent instrument handling and
measurements, which are subjective. As such, a computer aided monitoring (CAM)
system to evaluate PV plant operation feasibility by employing smart grid past
data analytics and deep learning is developed. Our algorithm, SolarisNet is a
6-layer deep neural network trained on data collected at two weather stations
located near Kalyani metrological site, West Bengal, India. The daily GSR
prediction performance using SolarisNet outperforms the existing state of art
and its efficacy in inferring past GSR data insights to comprehend daily and
seasonal GSR variability along with its competence for short term forecasting
is discussed.
Effects of Limiting Memory Capacity on the Behaviour of Exemplar Dynamics
Authors: B. Goodman, P. F. Tupper
Date: 2017-03-10 00:00:00

Exemplar models are a popular class of models used to describe language
change. Here we study how limiting the memory capacity of an individual in
these models affects the system's behaviour. In particular we demonstrate the
effect this change has on the extinction of categories. Previous work in
exemplar dynamics has not addressed this question. In order to investigate
this, we will inspect a simplified exemplar model. We will prove for the
simplified model that all the sound categories but one will always become
extinct, whether memory storage is limited or not. However, computer
simulations show that changing the number of stored memories alters how fast
categories become extinct.
Compiling Deep Learning Models for Custom Hardware Accelerators
Authors: Andre Xian Ming Chang, Aliasger Zaidy, Vinayak Gokhale, Eugenio Culurciello
Date: 2017-08-01 00:00:00
Tasks: Deep Learning, object-detection, Object Detection

Convolutional neural networks (CNNs) are the core of most state-of-the-art
deep learning algorithms specialized for object detection and classification.
CNNs are both computationally complex and embarrassingly parallel. Two
properties that leave room for potential software and hardware optimizations
for embedded systems. Given a programmable hardware accelerator with a CNN
oriented custom instructions set, the compiler's task is to exploit the
hardware's full potential, while abiding with the hardware constraints and
maintaining generality to run different CNN models with varying workload
properties. Snowflake is an efficient and scalable hardware accelerator
implemented on programmable logic devices. It implements a control pipeline for
a custom instruction set. The goal of this paper is to present Snowflake's
compiler that generates machine level instructions from Torch7 model
description files. The main software design points explored in this work are:
model structure parsing, CNN workload breakdown, loop rearrangement for memory
bandwidth optimizations and memory access balancing. The performance achieved
by compiler generated instructions matches against hand optimized code for
convolution layers. Generated instructions also efficiently execute AlexNet and
ResNet18 inference on Snowflake. Snowflake with $256$ processing units was
synthesized on Xilinx's Zynq XC7Z045 FPGA. At $250$ MHz, AlexNet achieved in
$93.6$ frames/s and $1.2$ GB/s of off-chip memory bandwidth, and $21.4$
frames/s and $2.2$ GB/s for ResNet18. Total on-chip power is $5$ W.
DGCNN: Disordered Graph Convolutional Neural Network Based on the Gaussian Mixture Model
Authors: Bo Wu, Yang Liu, Bo Lang, Lei Huang
Date: 2017-12-10 00:00:00
Tasks: Graph Classification, Graph Similarity, Retrieval

Convolutional neural networks (CNNs) can be applied to graph similarity
matching, in which case they are called graph CNNs. Graph CNNs are attracting
increasing attention due to their effectiveness and efficiency. However, the
existing convolution approaches focus only on regular data forms and require
the transfer of the graph or key node neighborhoods of the graph into the same
fixed form. During this transfer process, structural information of the graph
can be lost, and some redundant information can be incorporated. To overcome
this problem, we propose the disordered graph convolutional neural network
(DGCNN) based on the mixed Gaussian model, which extends the CNN by adding a
preprocessing layer called the disordered graph convolutional layer (DGCL). The
DGCL uses a mixed Gaussian function to realize the mapping between the
convolution kernel and the nodes in the neighborhood of the graph. The output
of the DGCL is the input of the CNN. We further implement a
backward-propagation optimization process of the convolutional layer by which
we incorporate the feature-learning model of the irregular node neighborhood
structure into the network. Thereafter, the optimization of the convolution
kernel becomes part of the neural network learning process. The DGCNN can
accept arbitrary scaled and disordered neighborhood graph structures as the
receptive fields of CNNs, which reduces information loss during graph
transformation. Finally, we perform experiments on multiple standard graph
datasets. The results show that the proposed method outperforms the
state-of-the-art methods in graph classification and retrieval.
On Convergence and Stability of GANs
Authors: Naveen Kodali, Jacob Abernethy, James Hays, Zsolt Kira
Date: 2017-05-19 00:00:00
Proceeding: ICLR 2018 1

We propose studying GAN training dynamics as regret minimization, which is in
contrast to the popular view that there is consistent minimization of a
divergence between real and generated distributions. We analyze the convergence
of GAN training from this new point of view to understand why mode collapse
happens. We hypothesize the existence of undesirable local equilibria in this
non-convex game to be responsible for mode collapse. We observe that these
local equilibria often exhibit sharp gradients of the discriminator function
around some real data points. We demonstrate that these degenerate local
equilibria can be avoided with a gradient penalty scheme called DRAGAN. We show
that DRAGAN enables faster training, achieves improved stability with fewer
mode collapses, and leads to generator networks with better modeling
performance across a variety of architectures and objective functions.