pwc-archive/papers-with-abstracts huggingface dataset:
A Contextual Bandit Bake-off
Authors: Alberto Bietti, Alekh Agarwal, John Langford
Date: 2018-02-12 00:00:00
Tasks: Diversity

Contextual bandit algorithms are essential for solving many real-world interactive machine learning problems. Despite multiple recent successes on statistically and computationally efficient methods, the practical behavior of these algorithms is still poorly understood. We leverage the availability of large numbers of supervised learning datasets to empirically evaluate contextual bandit algorithms, focusing on practical methods that learn by relying on optimization oracles from supervised learning. We find that a recent method (Foster et al., 2018) using optimism under uncertainty works the best overall. A surprisingly close second is a simple greedy baseline that only explores implicitly through the diversity of contexts, followed by a variant of Online Cover (Agarwal et al., 2014) which tends to be more conservative but robust to problem specification by design. Along the way, we also evaluate various components of contextual bandit algorithm design such as loss estimators. Overall, this is a thorough study and review of contextual bandit methodology.
Privacy Aware Offloading of Deep Neural Networks
Authors: Sam Leroux, Tim Verbelen, Pieter Simoens, Bart Dhoedt
Date: 2018-05-30 00:00:00

Deep neural networks require large amounts of resources which makes them hard
to use on resource constrained devices such as Internet-of-things devices.
Offloading the computations to the cloud can circumvent these constraints but
introduces a privacy risk since the operator of the cloud is not necessarily
trustworthy. We propose a technique that obfuscates the data before sending it
to the remote computation node. The obfuscated data is unintelligible for a
human eavesdropper but can still be classified with a high accuracy by a neural
network trained on unobfuscated images.
Towards Adversarial Configurations for Software Product Lines
Authors: Paul Temple, Mathieu Acher, Battista Biggio, Jean-Marc Jézéquel, Fabio Roli
Date: 2018-05-30 00:00:00
Tasks: BIG-bench Machine Learning, valid

Ensuring that all supposedly valid configurations of a software product line
(SPL) lead to well-formed and acceptable products is challenging since it is
most of the time impractical to enumerate and test all individual products of
an SPL. Machine learning classifiers have been recently used to predict the
acceptability of products associated with unseen configurations. For some
configurations, a tiny change in their feature values can make them pass from
acceptable to non-acceptable regarding users' requirements and vice-versa. In
this paper, we introduce the idea of leveraging these specific configurations
and their positions in the feature space to improve the classifier and
therefore the engineering of an SPL. Starting from a variability model, we
propose to use Adversarial Machine Learning techniques to create new,
adversarial configurations out of already known configurations by modifying
their feature values. Using an industrial video generator we show how
adversarial configurations can improve not only the classifier, but also the
variability model, the variability implementation, and the testing oracle.
Generalizing to Unseen Domains via Adversarial Data Augmentation
Authors: Riccardo Volpi, Hongseok Namkoong, Ozan Sener, John Duchi, Vittorio Murino, Silvio Savarese
Date: 2018-05-30 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Data Augmentation, Semantic Segmentation

We are concerned with learning models that generalize well to different
\emph{unseen} domains. We consider a worst-case formulation over data
distributions that are near the source domain in the feature space. Only using
training data from a single source distribution, we propose an iterative
procedure that augments the dataset with examples from a fictitious target
domain that is "hard" under the current model. We show that our iterative
scheme is an adaptive data augmentation method where we append adversarial
examples at each iteration. For softmax losses, we show that our method is a
data-dependent regularization scheme that behaves differently from classical
regularizers that regularize towards zero (e.g., ridge or lasso). On digit
recognition and semantic segmentation tasks, our method learns models improve
performance across a range of a priori unknown target domains.
Robustifying Models Against Adversarial Attacks by Langevin Dynamics
Authors: Vignesh Srinivasan, Arturo Marban, Klaus-Robert Müller, Wojciech Samek, Shinichi Nakajima
Date: 2018-05-30 00:00:00
Tasks: Denoising

Adversarial attacks on deep learning models have compromised their performance considerably. As remedies, a lot of defense methods were proposed, which however, have been circumvented by newer attacking strategies. In the midst of this ensuing arms race, the problem of robustness against adversarial attacks still remains unsolved. This paper proposes a novel, simple yet effective defense strategy where adversarial samples are relaxed onto the underlying manifold of the (unknown) target class distribution. Specifically, our algorithm drives off-manifold adversarial samples towards high density regions of the data generating distribution of the target class by the Metroplis-adjusted Langevin algorithm (MALA) with perceptual boundary taken into account. Although the motivation is similar to projection methods, e.g., Defense-GAN, our algorithm, called MALA for DEfense (MALADE), is equipped with significant dispersion - projection is distributed broadly, and therefore any whitebox attack cannot accurately align the input so that the MALADE moves it to a targeted untrained spot where the model predicts a wrong label. In our experiments, MALADE exhibited state-of-the-art performance against various elaborate attacking strategies.
xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
Authors: Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, Guangzhong Sun
Date: 2018-03-14 00:00:00
Tasks: Click-Through Rate Prediction, Recommendation Systems

Combinatorial features are essential for the success of many commercial
models. Manually crafting these features usually comes with high cost due to
the variety, volume and velocity of raw data in web-scale systems.
Factorization based models, which measure interactions in terms of vector
product, can learn patterns of combinatorial features automatically and
generalize to unseen features as well. With the great success of deep neural
networks (DNNs) in various fields, recently researchers have proposed several
DNN-based factorization model to learn both low- and high-order feature
interactions. Despite the powerful ability of learning an arbitrary function
from data, plain DNNs generate feature interactions implicitly and at the
bit-wise level. In this paper, we propose a novel Compressed Interaction
Network (CIN), which aims to generate feature interactions in an explicit
fashion and at the vector-wise level. We show that the CIN share some
functionalities with convolutional neural networks (CNNs) and recurrent neural
networks (RNNs). We further combine a CIN and a classical DNN into one unified
model, and named this new model eXtreme Deep Factorization Machine (xDeepFM).
On one hand, the xDeepFM is able to learn certain bounded-degree feature
interactions explicitly; on the other hand, it can learn arbitrary low- and
high-order feature interactions implicitly. We conduct comprehensive
experiments on three real-world datasets. Our results demonstrate that xDeepFM
outperforms state-of-the-art models. We have released the source code of
xDeepFM at \url{https://github.com/Leavingseason/xDeepFM}.
Neural Discrete Representation Learning
Authors: Aaron van den Oord, Oriol Vinyals, Koray Kavukcuoglu
Date: 2017-11-02 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Decoder, Representation Learning

Learning useful representations without supervision remains a key challenge
in machine learning. In this paper, we propose a simple yet powerful generative
model that learns such discrete representations. Our model, the Vector
Quantised-Variational AutoEncoder (VQ-VAE), differs from VAEs in two key ways:
the encoder network outputs discrete, rather than continuous, codes; and the
prior is learnt rather than static. In order to learn a discrete latent
representation, we incorporate ideas from vector quantisation (VQ). Using the
VQ method allows the model to circumvent issues of "posterior collapse" --
where the latents are ignored when they are paired with a powerful
autoregressive decoder -- typically observed in the VAE framework. Pairing
these representations with an autoregressive prior, the model can generate high
quality images, videos, and speech as well as doing high quality speaker
conversion and unsupervised learning of phonemes, providing further evidence of
the utility of the learnt representations.
Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks
Authors: Hardik Sharma, Jongse Park, Naveen Suda, Liangzhen Lai, Benson Chau, Joon Kyung Kim, Vikas Chandra, Hadi Esmaeilzadeh
Date: 2017-12-05 00:00:00
Tasks: GPU

Fully realizing the potential of acceleration for Deep Neural Networks (DNNs)
requires understanding and leveraging algorithmic properties. This paper builds
upon the algorithmic insight that bitwidth of operations in DNNs can be reduced
without compromising their classification accuracy. However, to prevent
accuracy loss, the bitwidth varies significantly across DNNs and it may even be
adjusted for each layer. Thus, a fixed-bitwidth accelerator would either offer
limited benefits to accommodate the worst-case bitwidth requirements, or lead
to a degradation in final accuracy. To alleviate these deficiencies, this work
introduces dynamic bit-level fusion/decomposition as a new dimension in the
design of DNN accelerators. We explore this dimension by designing Bit Fusion,
a bit-flexible accelerator, that constitutes an array of bit-level processing
elements that dynamically fuse to match the bitwidth of individual DNN layers.
This flexibility in the architecture enables minimizing the computation and the
communication at the finest granularity possible with no loss in accuracy. We
evaluate the benefits of BitFusion using eight real-world feed-forward and
recurrent DNNs. The proposed microarchitecture is implemented in Verilog and
synthesized in 45 nm technology. Using the synthesis results and cycle accurate
simulation, we compare the benefits of Bit Fusion to two state-of-the-art DNN
accelerators, Eyeriss and Stripes. In the same area, frequency, and process
technology, BitFusion offers 3.9x speedup and 5.1x energy savings over Eyeriss.
Compared to Stripes, BitFusion provides 2.6x speedup and 3.9x energy reduction
at 45 nm node when BitFusion area and frequency are set to those of Stripes.
Scaling to GPU technology node of 16 nm, BitFusion almost matches the
performance of a 250-Watt Titan Xp, which uses 8-bit vector instructions, while
BitFusion merely consumes 895 milliwatts of power.
Why Is My Classifier Discriminatory?
Authors: Irene Chen, Fredrik D. Johansson, David Sontag
Date: 2018-05-30 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Fairness

Recent attempts to achieve fairness in predictive models focus on the balance
between fairness and accuracy. In sensitive applications such as healthcare or
criminal justice, this trade-off is often undesirable as any increase in
prediction error could have devastating consequences. In this work, we argue
that the fairness of predictions should be evaluated in context of the data,
and that unfairness induced by inadequate samples sizes or unmeasured
predictive variables should be addressed through data collection, rather than
by constraining the model. We decompose cost-based metrics of discrimination
into bias, variance, and noise, and propose actions aimed at estimating and
reducing each term. Finally, we perform case-studies on prediction of income,
mortality, and review ratings, confirming the value of this analysis. We find
that data collection is often a means to reduce discrimination without
sacrificing accuracy.
Streamlined Deployment for Quantized Neural Networks
Authors: Yaman Umuroglu, Magnus Jahre
Date: 2017-09-12 00:00:00

Running Deep Neural Network (DNN) models on devices with limited
computational capability is a challenge due to large compute and memory
requirements. Quantized Neural Networks (QNNs) have emerged as a potential
solution to this problem, promising to offer most of the DNN accuracy benefits
with much lower computational cost. However, harvesting these benefits on
existing mobile CPUs is a challenge since operations on highly quantized
datatypes are not natively supported in most instruction set architectures
(ISAs). In this work, we first describe a streamlining flow to convert all QNN
inference operations to integer ones. Afterwards, we provide techniques based
on processing one bit position at a time (bit-serial) to show how QNNs can be
efficiently deployed using common bitwise operations. We demonstrate the
potential of QNNs on mobile CPUs with microbenchmarks and on a quantized
AlexNet, which is 3.5x faster than an optimized 8-bit baseline. Our bit-serial
matrix multiplication library is available on GitHub at https://git.io/vhshn
Using Neural Generative Models to Release Synthetic Twitter Corpora with Reduced Stylometric Identifiability of Users
Authors: Alexander G. Ororbia II, Fridolin Linder, Joshua Snoke
Date: 2016-06-03 00:00:00
Tasks: Translation

We present a method for generating synthetic versions of Twitter data using
neural generative models. The goal is protecting individuals in the source data
from stylometric re-identification attacks while still releasing data that
carries research value. Specifically, we generate tweet corpora that maintain
user-level word distributions by augmenting the neural language models with
user-specific components. We compare our approach to two standard text data
protection methods: redaction and iterative translation. We evaluate the three
methods on measures of risk and utility. We define risk following the
stylometric models of re-identification, and we define utility based on two
general word distribution measures and two common text analysis research tasks.
We find that neural models are able to significantly lower risk over previous
methods with little cost to utility. We also demonstrate that the neural models
allow data providers to actively control the risk-utility trade-off through
model tuning parameters. This work presents promising results for a new tool
addressing the problem of privacy for free text and sharing social media data
in a way that respects privacy and is ethically responsible.
Adaptive Network Sparsification with Dependent Variational Beta-Bernoulli Dropout
Authors: Juho Lee, Saehoon Kim, Jaehong Yoon, Hae Beom Lee, Eunho Yang, Sung Ju Hwang
Date: 2018-05-28 00:00:00

While variational dropout approaches have been shown to be effective for
network sparsification, they are still suboptimal in the sense that they set
the dropout rate for each neuron without consideration of the input data. With
such input-independent dropout, each neuron is evolved to be generic across
inputs, which makes it difficult to sparsify networks without accuracy loss. To
overcome this limitation, we propose adaptive variational dropout whose
probabilities are drawn from sparsity-inducing beta Bernoulli prior. It allows
each neuron to be evolved either to be generic or specific for certain inputs,
or dropped altogether. Such input-adaptive sparsity-inducing dropout allows the
resulting network to tolerate larger degree of sparsity without losing its
expressive power by removing redundancies among features. We validate our
dependent variational beta-Bernoulli dropout on multiple public datasets, on
which it obtains significantly more compact networks than baseline methods,
with consistent accuracy improvements over the base networks.
l0-norm Based Centers Selection for Training Fault Tolerant RBF Networks and Selecting Centers
Authors: Hao Wang, Chi-Sing Leung, Hing Cheung So, Ruibin Feng, Zifa Han
Date: 2018-05-30 00:00:00

The aim of this paper is to train an RBF neural network and select centers
under concurrent faults. It is well known that fault tolerance is a very
attractive property for neural networks. And center selection is an important
procedure during the training process of an RBF neural network. In this paper,
we devise two novel algorithms to address these two issues simultaneously. Both
of them are based on the ADMM framework. In the first method, the minimax
concave penalty (MCP) function is introduced to select centers. In the second
method, an l0-norm term is directly used, and the hard threshold (HT) is
utilized to address the l0-norm term. Under several mild conditions, we can
prove that both methods can globally converge to a unique limit point.
Simulation results show that, under concurrent fault, the proposed algorithms
are superior to many existing methods.
Automatic generation of object shapes with desired functionalities
Authors: Mihai Andries, Atabak Dehban, José Santos-Victor
Date: 2018-05-30 00:00:00
Tasks: Object

3D objects (artefacts) are made to fulfill functions. Designing an object
often starts with defining a list of functionalities that it should provide,
also known as functional requirements. Today, the design of 3D object models is
still a slow and largely artisanal activity, with few Computer-Aided Design
(CAD) tools existing to aid the exploration of the design solution space. To
accelerate the design process, we introduce an algorithm for generating object
shapes with desired functionalities. Following the concept of form follows
function, we assume that existing object shapes were rationally chosen to
provide desired functionalities. First, we use an artificial neural network to
learn a function-to-form mapping by analysing a dataset of objects labeled with
their functionalities. Then, we combine forms providing one or more desired
functions, generating an object shape that is expected to provide all of them.
Finally, we verify in simulation whether the generated object possesses the
desired functionalities, by defining and executing functionality tests on it.
MolGAN: An implicit generative model for small molecular graphs
Authors: Nicola De Cao, Thomas Kipf
Date: 2018-05-30 00:00:00
Tasks: Graph Matching, Reinforcement Learning, valid

Deep generative models for graph-structured data offer a new angle on the problem of chemical synthesis: by optimizing differentiable models that directly generate molecular graphs, it is possible to side-step expensive search procedures in the discrete and vast space of chemical structures. We introduce MolGAN, an implicit, likelihood-free generative model for small molecular graphs that circumvents the need for expensive graph matching procedures or node ordering heuristics of previous likelihood-based methods. Our method adapts generative adversarial networks (GANs) to operate directly on graph-structured data. We combine our approach with a reinforcement learning objective to encourage the generation of molecules with specific desired chemical properties. In experiments on the QM9 chemical database, we demonstrate that our model is capable of generating close to 100% valid compounds. MolGAN compares favorably both to recent proposals that use string-based (SMILES) representations of molecules and to a likelihood-based method that directly generates graphs, albeit being susceptible to mode collapse. Code at https://github.com/nicola-decao/MolGAN
Automatic Large-Scale Data Acquisition via Crowdsourcing for Crosswalk Classification: A Deep Learning Approach
Authors: Rodrigo F. Berriel, Franco Schmidt Rossi, Alberto F. de Souza, Thiago Oliveira-Santos
Date: 2018-05-30 00:00:00
Tasks: General Classification

Correctly identifying crosswalks is an essential task for the driving
activity and mobility autonomy. Many crosswalk classification, detection and
localization systems have been proposed in the literature over the years. These
systems use different perspectives to tackle the crosswalk classification
problem: satellite imagery, cockpit view (from the top of a car or behind the
windshield), and pedestrian perspective. Most of the works in the literature
are designed and evaluated using small and local datasets, i.e. datasets that
present low diversity. Scaling to large datasets imposes a challenge for the
annotation procedure. Moreover, there is still need for cross-database
experiments in the literature because it is usually hard to collect the data in
the same place and conditions of the final application. In this paper, we
present a crosswalk classification system based on deep learning. For that,
crowdsourcing platforms, such as OpenStreetMap and Google Street View, are
exploited to enable automatic training via automatic acquisition and annotation
of a large-scale database. Additionally, this work proposes a comparison study
of models trained using fully-automatic data acquisition and annotation against
models that were partially annotated. Cross-database experiments were also
included in the experimentation to show that the proposed methods enable use
with real world applications. Our results show that the model trained on the
fully-automatic database achieved high overall accuracy (94.12%), and that a
statistically significant improvement (to 96.30%) can be achieved by manually
annotating a specific part of the database. Finally, the results of the
cross-database experiments show that both models are robust to the many
variations of image and scenarios, presenting a consistent behavior.
Short-term Load Forecasting with Deep Residual Networks
Authors: Kunjin Chen, Kunlong Chen, Qin Wang, Ziyu He, Jun Hu, Jinliang He
Date: 2018-05-30 00:00:00
Tasks: Load Forecasting

We present in this paper a model for forecasting short-term power loads based
on deep residual networks. The proposed model is able to integrate domain
knowledge and researchers' understanding of the task by virtue of different
neural network building blocks. Specifically, a modified deep residual network
is formulated to improve the forecast results. Further, a two-stage ensemble
strategy is used to enhance the generalization capability of the proposed
model. We also apply the proposed model to probabilistic load forecasting using
Monte Carlo dropout. Three public datasets are used to prove the effectiveness
of the proposed model. Multiple test cases and comparison with existing models
show that the proposed model is able to provide accurate load forecasting
results and has high generalization capability.
Fast L1-Minimization Algorithm for Sparse Approximation Based on an Improved LPNN-LCA framework
Authors: Hao Wang, Ruibin Feng, Chi-Sing Leung
Date: 2018-05-30 00:00:00

The aim of sparse approximation is to estimate a sparse signal according to
the measurement matrix and an observation vector. It is widely used in data
analytics, image processing, and communication, etc. Up to now, a lot of
research has been done in this area, and many off-the-shelf algorithms have
been proposed. However, most of them cannot offer a real-time solution. To some
extent, this shortcoming limits its application prospects. To address this
issue, we devise a novel sparse approximation algorithm based on Lagrange
programming neural network (LPNN), locally competitive algorithm (LCA), and
projection theorem. LPNN and LCA are both analog neural network which can help
us get a real-time solution. The non-differentiable objective function can be
solved by the concept of LCA. Utilizing the projection theorem, we further
modify the dynamics and proposed a new system with global asymptotic stability.
Simulation results show that the proposed sparse approximation method has the
real-time solutions with satisfactory MSEs.