pwc-archive/papers-with-abstracts huggingface dataset:
A matching based clustering algorithm for categorical data
Authors: Ruben A. Gevorgyan, Yenok B. Hakobyan
Date: 2018-12-09 00:00:00
Tasks: Clustering, Feature Importance

Cluster analysis is one of the essential tasks in data mining and knowledge
discovery. Each type of data poses unique challenges in achieving relatively
efficient partitioning of the data into homogeneous groups. While the
algorithms for numeric data are relatively well studied in the literature,
there are still challenges to address in case of categorical data. The main
issue is the unordered structure of categorical data, which makes the
implementation of the standard concepts of clustering algorithms difficult. For
instance, the assessment of distance between objects, the selection of
representatives for categorical data is not as straightforward as for
continuous data. Therefore, this paper presents a new framework for
partitioning categorical data, which does not use the distance measure as a key
concept. The Matching based clustering algorithm is designed based on the
similarity matrix and a framework for updating the latter using the feature
importance criteria. The experimental results show this algorithm can serve as
an alternative to existing ones and can be an efficient knowledge discovery
tool.
Towards Neural Network Patching: Evaluating Engagement-Layers and Patch-Architectures
Authors: Sebastian Kauschke, David Hermann Lehmann
Date: 2018-12-09 00:00:00

In this report we investigate fundamental requirements for the application of
classifier patching on neural networks. Neural network patching is an approach
for adapting neural network models to handle concept drift in nonstationary
environments. Instead of creating or updating the existing network to
accommodate concept drift, neural network patching leverages the inner layers
of the network as well as its output to learn a patch that enhances the
classification and corrects errors caused by the drift. It learns (i) a
predictor that estimates whether the original network will misclassify an
instance, and (ii) a patching network that fixes the misclassification. Neural
network patching is based on the idea that the original network can still
classify a majority of instances well, and that the inner feature
representations encoded in the deep network aid the classifier to cope with
unseen or changed inputs. In order to apply this kind of patching, we evaluate
different engagement layers and patch architectures in this report, and find a
set of generally applicable heuristics, which aid in parametrizing the patching
procedure.
A note on solving nonlinear optimization problems in variable precision
Authors: S. Gratton, Ph. L. Toint
Date: 2018-12-09 00:00:00

This short note considers an efficient variant of the trust-region algorithm
with dynamic accuracy proposed Carter (1993) and Conn, Gould and Toint (2000)
as a tool for very high-performance computing, an area where it is critical to
allow multi-precision computations for keeping the energy dissipation under
control. Numerical experiments are presented indicating that the use of the
considered method can bring substantial savings in objective function's and
gradient's evaluation "energy costs" by efficiently exploiting multi-precision
computations.
A Comparison of Embedded Deep Learning Methods for Person Detection
Authors: Chloe Eunhyang Kim, Mahdi Maktab Dar Oghaz, Jiri Fajtl, Vasileios Argyriou, Paolo Remagnino
Date: 2018-12-09 00:00:00
Tasks: Deep Learning, GPU, Human Detection

Recent advancements in parallel computing, GPU technology and deep learning
provide a new platform for complex image processing tasks such as person
detection to flourish. Person detection is fundamental preliminary operation
for several high level computer vision tasks. One industry that can
significantly benefit from person detection is retail. In recent years, various
studies attempt to find an optimal solution for person detection using neural
networks and deep learning. This study conducts a comparison among the state of
the art deep learning base object detector with the focus on person detection
performance in indoor environments. Performance of various implementations of
YOLO, SSD, RCNN, R-FCN and SqueezeDet have been assessed using our in-house
proprietary dataset which consists of over 10 thousands indoor images captured
form shopping malls, retails and stores. Experimental results indicate that,
Tiny YOLO-416 and SSD (VGG-300) are the fastest and Faster-RCNN (Inception
ResNet-v2) and R-FCN (ResNet-101) are the most accurate detectors investigated
in this study. Further analysis shows that YOLO v3-416 delivers relatively
accurate result in a reasonable amount of time, which makes it an ideal model
for person detection in embedded platforms.
A New Variational Model for Joint Image Reconstruction and Motion Estimation in Spatiotemporal Imaging
Authors: Chong Chen, Barbara Gris, Ozan Öktem
Date: 2018-12-09 00:00:00
Tasks: Image Reconstruction, Image Registration, Motion Estimation, Optical Flow Estimation

We propose a new variational model for joint image reconstruction and motion
estimation in spatiotemporal imaging, which is investigated along a general
framework that we present with shape theory. This model consists of two
components, one for conducting modified static image reconstruction, and the
other performs sequentially indirect image registration. For the latter, we
generalize the large deformation diffeomorphic metric mapping framework into
the sequentially indirect registration setting. The proposed model is compared
theoretically against alternative approaches (optical flow based model and
diffeomorphic motion models), and we demonstrate that the proposed model has
desirable properties in terms of the optimal solution. The theoretical
derivations and efficient algorithms are also presented for a time-discretized
scenario of the proposed model, which show that the optimal solution of the
time-discretized version is consistent with that of the time-continuous one,
and most of the computational components is the easy-implemented linearized
deformation. The complexity of the algorithm is analyzed as well. This work is
concluded by some numerical examples in 2D space + time tomography with very
sparse and/or highly noisy data.
Applying Nature-Inspired Optimization Algorithms for Selecting Important Timestamps to Reduce Time Series Dimensionality
Authors: Muhammad Marwan Muhammad Fuad
Date: 2018-12-09 00:00:00
Tasks: Clustering, Time Series, Time Series Analysis

Time series data account for a major part of data supply available today.
Time series mining handles several tasks such as classification, clustering,
query-by-content, prediction, and others. Performing data mining tasks on raw
time series is inefficient as these data are high-dimensional by nature.
Instead, time series are first pre-processed using several techniques before
different data mining tasks can be performed on them. In general, there are two
main approaches to reduce time series dimensionality, the first is what we call
landmark methods. These methods are based on finding characteristic features in
the target time series. The second is based on data transformations. These
methods transform the time series from the original space into a reduced space,
where they can be managed more efficiently. The method we present in this paper
applies a third approach, as it projects a time series onto a lower-dimensional
space by selecting important points in the time series. The novelty of our
method is that these points are not chosen according to a geometric criterion,
which is subjective in most cases, but through an optimization process. The
other important characteristic of our method is that these important points are
selected on a dataset-level and not on a single time series-level. The direct
advantage of this strategy is that the distance defined on the low-dimensional
space lower bounds the original distance applied to raw data. This enables us
to apply the popular GEMINI algorithm. The promising results of our experiments
on a wide variety of time series datasets, using different optimizers, and
applied to the two major data mining tasks, validate our new method.
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
Authors: Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, Kurt Keutzer
Date: 2018-12-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: GPU, Image Classification, Neural Architecture Search

Designing accurate and efficient ConvNets for mobile devices is challenging because the design space is combinatorially large. Due to this, previous neural architecture search (NAS) methods are computationally expensive. ConvNet architecture optimality depends on factors such as input resolution and target devices. However, existing approaches are too expensive for case-by-case redesigns. Also, previous work focuses primarily on reducing FLOPs, but FLOP count does not always reflect actual latency. To address these, we propose a differentiable neural architecture search (DNAS) framework that uses gradient-based methods to optimize ConvNet architectures, avoiding enumerating and training individual architectures separately as in previous methods. FBNets, a family of models discovered by DNAS surpass state-of-the-art models both designed manually and generated automatically. FBNet-B achieves 74.1% top-1 accuracy on ImageNet with 295M FLOPs and 23.1 ms latency on a Samsung S8 phone, 2.4x smaller and 1.5x faster than MobileNetV2-1.3 with similar accuracy. Despite higher accuracy and lower latency than MnasNet, we estimate FBNet-B's search cost is 420x smaller than MnasNet's, at only 216 GPU-hours. Searched for different resolutions and channel sizes, FBNets achieve 1.5% to 6.4% higher accuracy than MobileNetV2. The smallest FBNet achieves 50.2% accuracy and 2.9 ms latency (345 frames per second) on a Samsung S8. Over a Samsung-optimized FBNet, the iPhone-X-optimized model achieves a 1.4x speedup on an iPhone X.
Area-preserving mapping of 3D ultrasound carotid artery images using density-equalizing reference map
Authors: Gary P. T. Choi, Bernard Chiu, Chris H. Rycroft
Date: 2018-12-09 00:00:00
Tasks: ARC

Carotid atherosclerosis is a focal disease at the bifurcations of the carotid
artery. To quantitatively monitor the local changes in the
vessel-wall-plus-plaque thickness (VWT) and compare the VWT distributions for
different patients or for the same patients at different ultrasound scanning
sessions, a mapping technique is required to adjust for the geometric
variability of different carotid artery models. In this work, we propose a
novel method called density-equalizing reference map (DERM) for mapping 3D
carotid surfaces to a standardized 2D carotid template, with an emphasis on
preserving the local geometry of the carotid surface by minimizing the local
area distortion. The initial map was generated by a previously described
arc-length scaling (ALS) mapping method, which projects a 3D carotid surface
onto a 2D non-convex L-shaped domain. A smooth and area-preserving flattened
map was subsequently constructed by deforming the ALS map using the proposed
algorithm that combines the density-equalizing map and the reference map
techniques. This combination allows, for the first time, one-to-one mapping
from a 3D surface to a standardized non-convex planar domain in an
area-preserving manner. Evaluations using 20 carotid surface models show that
the proposed method reduced the area distortion of the flattening maps by over
80% as compared to the ALS mapping method.
Real-Time Referring Expression Comprehension by Single-Stage Grounding Network
Authors: Xinpeng Chen, Lin Ma, Jingyuan Chen, Zequn Jie, Wei Liu, Jiebo Luo
Date: 2018-12-09 00:00:00
Tasks: Attribute, Referring Expression, Referring Expression Comprehension

In this paper, we propose a novel end-to-end model, namely Single-Stage
Grounding network (SSG), to localize the referent given a referring expression
within an image. Different from previous multi-stage models which rely on
object proposals or detected regions, our proposed model aims to comprehend a
referring expression through one single stage without resorting to region
proposals as well as the subsequent region-wise feature extraction.
Specifically, a multimodal interactor is proposed to summarize the local region
features regarding the referring expression attentively. Subsequently, a
grounder is proposed to localize the referring expression within the given
image directly. For further improving the localization accuracy, a guided
attention mechanism is proposed to enforce the grounder to focus on the central
region of the referent. Moreover, by exploiting and predicting visual attribute
information, the grounder can further distinguish the referent objects within
an image and thereby improve the model performance. Experiments on RefCOCO,
RefCOCO+, and RefCOCOg datasets demonstrate that our proposed SSG without
relying on any region proposals can achieve comparable performance with other
advanced models. Furthermore, our SSG outperforms the previous models and
achieves the state-of-art performance on the ReferItGame dataset. More
importantly, our SSG is time efficient and can ground a referring expression in
a 416*416 image from the RefCOCO dataset in 25ms (40 referents per second) on
average with a Nvidia Tesla P40, accomplishing more than 9* speedups over the
existing multi-stage models.
Zero Initialization of modified Gated Recurrent Encoder-Decoder Network for Short Term Load Forecasting
Authors: Vedanshu, M M Tripathi
Date: 2018-12-09 00:00:00
Tasks: Decoder, Load Forecasting

Single layer Feedforward Neural Network(FNN) is used many a time as a last
layer in models such as seq2seq or could be a simple RNN network. The
importance of such layer is to transform the output to our required dimensions.
When it comes to weights and biases initialization, there is no such specific
technique that could speed up the learning process. We could depend on deep
network initialization techniques such as Xavier or He initialization. But such
initialization fails to show much improvement in learning speed or accuracy. In
this paper we propose Zero Initialization (ZI) for weights of a single layer
network. We first test this technique with on a simple RNN network and compare
the results against Xavier, He and Identity initialization. As a final test we
implement it on a seq2seq network. It was found that ZI considerably reduces
the number of epochs used and improve the accuracy. The developed model has
been applied for short-term load forecasting using the load data of Australian
Energy Market. The model is able to forecast the day ahead load accurately with
error of 0.94%.
Learning Transferable Adversarial Examples via Ghost Networks
Authors: Yingwei Li, Song Bai, Yuyin Zhou, Cihang Xie, Zhishuai Zhang, Alan Yuille
Date: 2018-12-09 00:00:00
Tasks: Adversarial Attack

Recent development of adversarial attacks has proven that ensemble-based methods outperform traditional, non-ensemble ones in black-box attack. However, as it is computationally prohibitive to acquire a family of diverse models, these methods achieve inferior performance constrained by the limited number of models to be ensembled. In this paper, we propose Ghost Networks to improve the transferability of adversarial examples. The critical principle of ghost networks is to apply feature-level perturbations to an existing model to potentially create a huge set of diverse models. After that, models are subsequently fused by longitudinal ensemble. Extensive experimental results suggest that the number of networks is essential for improving the transferability of adversarial examples, but it is less necessary to independently train different networks and ensemble them in an intensive aggregation way. Instead, our work can be used as a computationally cheap and easily applied plug-in to improve adversarial approaches both in single-model and multi-model attack, compatible with residual and non-residual networks. By reproducing the NeurIPS 2017 adversarial competition, our method outperforms the No.1 attack submission by a large margin, demonstrating its effectiveness and efficiency. Code is available at https://github.com/LiYingwei/ghost-network.
Learning Multiplication-free Linear Transformations
Authors: Cristian Rusu
Date: 2018-12-09 00:00:00
Tasks: Dictionary Learning

In this paper, we propose several dictionary learning algorithms for sparse representations that also impose specific structures on the learned dictionaries such that they are numerically efficient to use: reduced number of addition/multiplications and even avoiding multiplications altogether. We base our work on factorizations of the dictionary in highly structured basic building blocks (binary orthonormal, scaling and shear transformations) for which we can write closed-form solutions to the optimization problems that we consider. We show the effectiveness of our methods on image data where we can compare against well-known numerically efficient transforms such as the fast Fourier and the fast discrete cosine transforms.
Feature Denoising for Improving Adversarial Robustness
Authors: Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Yuille, Kaiming He
Date: 2018-12-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: Adversarial Defense, Adversarial Robustness, General Classification, image-classification, Image Classification

Adversarial attacks to image classification systems present challenges to
convolutional networks and opportunities for understanding them. This study
suggests that adversarial perturbations on images lead to noise in the features
constructed by these networks. Motivated by this observation, we develop new
network architectures that increase adversarial robustness by performing
feature denoising. Specifically, our networks contain blocks that denoise the
features using non-local means or other filters; the entire networks are
trained end-to-end. When combined with adversarial training, our feature
denoising networks substantially improve the state-of-the-art in adversarial
robustness in both white-box and black-box attack settings. On ImageNet, under
10-iteration PGD white-box attacks where prior art has 27.9% accuracy, our
method achieves 55.7%; even under extreme 2000-iteration PGD white-box attacks,
our method secures 42.6% accuracy. Our method was ranked first in Competition
on Adversarial Attacks and Defenses (CAAD) 2018 --- it achieved 50.6%
classification accuracy on a secret, ImageNet-like test dataset against 48
unknown attackers, surpassing the runner-up approach by ~10%. Code is available
at https://github.com/facebookresearch/ImageNet-Adversarial-Training.
Binary Input Layer: Training of CNN models with binary input data
Authors: Robert Dürichen, Thomas Rocznik, Oliver Renz, Christian Peters
Date: 2018-12-09 00:00:00
Tasks: Binarization

For the efficient execution of deep convolutional neural networks (CNN) on
edge devices, various approaches have been presented which reduce the bit width
of the network parameters down to 1 bit. Binarization of the first layer was
always excluded, as it leads to a significant error increase. Here, we present
the novel concept of binary input layer (BIL), which allows the usage of binary
input data by learning bit specific binary weights. The concept is evaluated on
three datasets (PAMAP2, SVHN, CIFAR-10). Our results show that this approach is
in particular beneficial for multimodal datasets (PAMAP2) where it outperforms
networks using full precision weights in the first layer by 1:92 percentage
points (pp) while consuming only 2 % of the chip area.
Beyond Domain Adaptation: Unseen Domain Encapsulation via Universal Non-volume Preserving Models
Authors: Thanh-Dat Truong, Chi Nhan Duong, Khoa Luu, Minh-Triet Tran, Minh Do
Date: 2018-12-09 00:00:00
Tasks: Domain Adaptation, Domain Generalization, Face Recognition, Pedestrian Detection

Recognition across domains has recently become an active topic in the
research community. However, it has been largely overlooked in the problem of
recognition in new unseen domains. Under this condition, the delivered deep
network models are unable to be updated, adapted or fine-tuned. Therefore,
recent deep learning techniques, such as: domain adaptation, feature
transferring, and fine-tuning, cannot be applied. This paper presents a novel
Universal Non-volume Preserving approach to the problem of domain
generalization in the context of deep learning. The proposed method can be
easily incorporated with any other ConvNet framework within an end-to-end deep
network design to improve the performance. On digit recognition, we benchmark
on four popular digit recognition databases, i.e. MNIST, USPS, SVHN and
MNIST-M. The proposed method is also experimented on face recognition on
Extended Yale-B, CMU-PIE and CMU-MPIE databases and compared against other the
state-of-the-art methods. In the problem of pedestrian detection, we
empirically observe that the proposed method learns models that improve
performance across a priori unknown data distributions.
AutoGAN: Robust Classifier Against Adversarial Attacks
Authors: Blerta Lindqvist, Shridatt Sugrim, Rauf Izmailov
Date: 2018-12-08 00:00:00
Tasks: Generative Adversarial Network

Classifiers fail to classify correctly input images that have been
purposefully and imperceptibly perturbed to cause misclassification. This
susceptability has been shown to be consistent across classifiers, regardless
of their type, architecture or parameters. Common defenses against adversarial
attacks modify the classifer boundary by training on additional adversarial
examples created in various ways. In this paper, we introduce AutoGAN, which
counters adversarial attacks by enhancing the lower-dimensional manifold
defined by the training data and by projecting perturbed data points onto it.
AutoGAN mitigates the need for knowing the attack type and magnitude as well as
the need for having adversarial samples of the attack. Our approach uses a
Generative Adversarial Network (GAN) with an autoencoder generator and a
discriminator that also serves as a classifier. We test AutoGAN against
adversarial samples generated with state-of-the-art Fast Gradient Sign Method
(FGSM) as well as samples generated with random Gaussian noise, both using the
MNIST dataset. For different magnitudes of perturbation in training and
testing, AutoGAN can surpass the accuracy of FGSM method by up to 25\% points
on samples perturbed using FGSM. Without an augmented training dataset, AutoGAN
achieves an accuracy of 89\% compared to 1\% achieved by FGSM method on FGSM
testing adversarial samples.
Semantically-Aware Attentive Neural Embeddings for Image-based Visual Localization
Authors: Zachary Seymour, Karan Sikka, Han-Pang Chiu, Supun Samarasekera, Rakesh Kumar
Date: 2018-12-08 00:00:00
Tasks: Deep Attention, Image-Based Localization, Visual Localization

We present an approach that combines appearance and semantic information for 2D image-based localization (2D-VL) across large perceptual changes and time lags. Compared to appearance features, the semantic layout of a scene is generally more invariant to appearance variations. We use this intuition and propose a novel end-to-end deep attention-based framework that utilizes multimodal cues to generate robust embeddings for 2D-VL. The proposed attention module predicts a shared channel attention and modality-specific spatial attentions to guide the embeddings to focus on more reliable image regions. We evaluate our model against state-of-the-art (SOTA) methods on three challenging localization datasets. We report an average (absolute) improvement of $19\%$ over current SOTA for 2D-VL. Furthermore, we present an extensive study demonstrating the contribution of each component of our model, showing $8$--$15\%$ and $4\%$ improvement from adding semantic information and our proposed attention module. We finally show the predicted attention maps to offer useful insights into our model.