pwc-archive/papers-with-abstracts huggingface dataset:
Design Automation for Efficient Deep Learning Computing
Authors: Song Han, Han Cai, Ligeng Zhu, Ji Lin, Kuan Wang, Zhijian Liu, Yujun Lin
Date: 2019-04-24 00:00:00
Tasks: Deep Learning, Quantization

Efficient deep learning computing requires algorithm and hardware co-design
to enable specialization: we usually need to change the algorithm to reduce
memory footprint and improve energy efficiency. However, the extra degree of
freedom from the algorithm makes the design space much larger: it's not only
about designing the hardware but also about how to tweak the algorithm to best
fit the hardware. Human engineers can hardly exhaust the design space by
heuristics. It's labor consuming and sub-optimal. We propose design automation
techniques for efficient neural networks. We investigate automatically
designing specialized fast models, auto channel pruning, and auto
mixed-precision quantization. We demonstrate such learning-based, automated
design achieves superior performance and efficiency than rule-based human
design. Moreover, we shorten the design cycle by 200x than previous work, so
that we can afford to design specialized neural network models for different
hardware platforms.
Understanding Art through Multi-Modal Retrieval in Paintings
Authors: Noa Garcia, Benjamin Renoust, Yuta Nakashima
Date: 2019-04-24 00:00:00
Tasks: Art Analysis, Retrieval

In computer vision, visual arts are often studied from a purely aesthetics
perspective, mostly by analysing the visual appearance of an artistic
reproduction to infer its style, its author, or its representative features. In
this work, however, we explore art from both a visual and a language
perspective. Our aim is to bridge the gap between the visual appearance of an
artwork and its underlying meaning, by jointly analysing its aesthetics and its
semantics. We introduce the use of multi-modal techniques in the field of
automatic art analysis by 1) collecting a multi-modal dataset with fine-art
paintings and comments, and 2) exploring robust visual and textual
representations in artistic images.
Defocused images removal of axial overlapping scattering particles by using three-dimensional nonlinear diffusion based on digital holography
Authors: Wei-Na Li, Zhengyun Zhang, Jianshe Ma, Xiaohao Wang, Ping Su
Date: 2019-04-24 00:00:00
Tasks: Position

We propose a three-dimensional nonlinear diffusion method to implement the similar autofocusing function of multiple micro-objects and simultaneously remove the defocused images, which can distinguish the locations of certain sized scattering particles that are overlapping along z-axis. It is applied to all of the reconstruction slices that are generated from the captured hologram after each back propagation. For certain small sized particles, the maxima of maximum gradient magnitude of each reconstruction slice appears at the ground truth z position after applying the proposed scheme when the reconstruction range along z-axis is sufficiently long and the reconstruction depth spacing is sufficiently fine. Therefore, the reconstructed image at ground truth z position is remained, while the defocused images are diffused out. The results demonstrated that the proposed scheme can diffuse out the defocused images which are 20 um away from the ground truth z position in spite of that several scattering particles with different diameters are completely overlapping along z-axis with a distance of 800 um when the hologram pixel pitch is 2 um. It also demonstrated that the sparsity distribution of the ground truth z slice cannot be affected by the sparsity distribution of corresponding defocused images when the diameter of the particle is not more than 35um and the reconstruction depth spacing is not less than 20 um.
Condition-Transforming Variational AutoEncoder for Conversation Response Generation
Authors: Yu-Ping Ruan, Zhen-Hua Ling, Quan Liu, Zhigang Chen, Nitin Indurkhya
Date: 2019-04-24 00:00:00
Tasks: Diversity, Response Generation

This paper proposes a new model, called condition-transforming variational
autoencoder (CTVAE), to improve the performance of conversation response
generation using conditional variational autoencoders (CVAEs). In conventional
CVAEs , the prior distribution of latent variable z follows a multivariate
Gaussian distribution with mean and variance modulated by the input conditions.
Previous work found that this distribution tends to become condition
independent in practical application. In our proposed CTVAE model, the latent
variable z is sampled by performing a non-lineartransformation on the
combination of the input conditions and the samples from a
condition-independent prior distribution N (0; I). In our objective
evaluations, the CTVAE model outperforms the CVAE model on fluency metrics and
surpasses a sequence-to-sequence (Seq2Seq) model on diversity metrics. In
subjective preference tests, our proposed CTVAE model performs significantly
better than CVAE and Seq2Seq models on generating fluency, informative and
topic relevant responses.
A Comparison Study of Credit Card Fraud Detection: Supervised versus Unsupervised
Authors: Xuetong Niu, Li Wang, Xulei Yang
Date: 2019-04-24 00:00:00
Tasks: Anomaly Detection, Fraud Detection, Unsupervised Anomaly Detection

Credit card has become popular mode of payment for both online and offline
purchase, which leads to increasing daily fraud transactions. An Efficient
fraud detection methodology is therefore essential to maintain the reliability
of the payment system. In this study, we perform a comparison study of credit
card fraud detection by using various supervised and unsupervised approaches.
Specifically, 6 supervised classification models, i.e., Logistic Regression
(LR), K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Decision Tree
(DT), Random Forest (RF), Extreme Gradient Boosting (XGB), as well as 4
unsupervised anomaly detection models, i.e., One-Class SVM (OCSVM),
Auto-Encoder (AE), Restricted Boltzmann Machine (RBM), and Generative
Adversarial Networks (GAN), are explored in this study. We train all these
models on a public credit card transaction dataset from Kaggle website, which
contains 492 frauds out of 284,807 transactions. The labels of the transactions
are used for supervised learning models only. The performance of each model is
evaluated through 5-fold cross validation in terms of Area Under the Receiver
Operating Curves (AUROC). Within supervised approaches, XGB and RF obtain the
best performance with AUROC = 0.989 and AUROC = 0.988, respectively. While for
unsupervised approaches, RBM achieves the best performance with AUROC = 0.961,
followed by GAN with AUROC = 0.954. The experimental results show that
supervised models perform slightly better than unsupervised models in this
study. Anyway, unsupervised approaches are still promising for credit card
fraud transaction detection due to the insufficient annotation and the data
imbalance issue in real-world applications.
Autonomous Voltage Control for Grid Operation Using Deep Reinforcement Learning
Authors: Ruisheng Diao, Zhiwei Wang, Di Shi, Qianyun Chang, Jiajun Duan, Xiaohu Zhang
Date: 2019-04-24 00:00:00
Tasks: AI Agent, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Modern power grids are experiencing grand challenges caused by the stochastic
and dynamic nature of growing renewable energy and demand response. Traditional
theoretical assumptions and operational rules may be violated, which are
difficult to be adapted by existing control systems due to the lack of
computational power and accurate grid models for use in real time, leading to
growing concerns in the secure and economic operation of the power grid.
Existing operational control actions are typically determined offline, which
are less optimized. This paper presents a novel paradigm, Grid Mind, for
autonomous grid operational controls using deep reinforcement learning. The
proposed AI agent for voltage control can learn its control policy through
interactions with massive offline simulations, and adapts its behavior to new
changes including not only load/generation variations but also topological
changes. A properly trained agent is tested on the IEEE 14-bus system with tens
of thousands of scenarios, and promising performance is demonstrated in
applying autonomous voltage controls for secure grid operation.
Neural Collaborative Subspace Clustering
Authors: Tong Zhang, Pan Ji, Mehrtash Harandi, Wenbing Huang, Hongdong Li
Date: 2019-04-24 00:00:00
Tasks: Clustering

We introduce the Neural Collaborative Subspace Clustering, a neural model
that discovers clusters of data points drawn from a union of low-dimensional
subspaces. In contrast to previous attempts, our model runs without the aid of
spectral clustering. This makes our algorithm one of the kinds that can
gracefully scale to large datasets. At its heart, our neural model benefits
from a classifier which determines whether a pair of points lies on the same
subspace or not. Essential to our model is the construction of two affinity
matrices, one from the classifier and the other from a notion of subspace
self-expressiveness, to supervise training in a collaborative scheme. We
thoroughly assess and contrast the performance of our model against various
state-of-the-art clustering algorithms including deep subspace-based ones.
Realizing Petabyte Scale Acoustic Modeling
Authors: Sree Hari Krishnan Parthasarathi, Nitin Sivakrishnan, Pranav Ladkat, Nikko Strom
Date: 2019-04-24 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Large scale machine learning (ML) systems such as the Alexa automatic speech
recognition (ASR) system continue to improve with increasing amounts of
manually transcribed training data. Instead of scaling manual transcription to
impractical levels, we utilize semi-supervised learning (SSL) to learn acoustic
models (AM) from the vast firehose of untranscribed audio data. Learning an AM
from 1 Million hours of audio presents unique ML and system design challenges.
We present the design and evaluation of a highly scalable and resource
efficient SSL system for AM. Employing the student/teacher learning paradigm,
we focus on the student learning subsystem: a scalable and robust data pipeline
that generates features and targets from raw audio, and an efficient model
pipeline, including the distributed trainer, that builds a student model. Our
evaluations show that, even without extensive hyper-parameter tuning, we obtain
relative accuracy improvements in the 10 to 20$\%$ range, with higher gains in
noisier conditions. The end-to-end processing time of this SSL system was 12
days, and several components in this system can trivially scale linearly with
more compute resources.
Kernel Mean Embedding of Instance-wise Predictions in Multiple Instance Regression
Authors: Thomas Uriot
Date: 2019-04-24 00:00:00
Tasks: regression

In this paper, we propose an extension to an existing algorithm (instance-MIR) which tackles the multiple instance regression (MIR) problem, also known as distribution regression. The MIR setting arises when the data is a collection of bags, where each bag consists of several instances which correspond to the same and unique real-valued label. The goal of a MIR algorithm is to find a mapping from the instances of an unseen bag to its target value. The instance-MIR algorithm treats all the instances separately and maps each instance to a label. The final bag label is then taken as the mean or the median of the predictions for that given bag. While it is conceptually simple, taking a single statistic to summarize the distribution of the labels in each bag is a limitation. In spite of this performance bottleneck, the instance-MIR algorithm has been shown to be competitive when compared to the current state-of-the-art methods. We address the aforementioned issue by computing the kernel mean embeddings of the distributions of the predicted labels, for each bag, and learn a regressor from these embeddings to the bag label. We test our algorithm (instance-kme-MIR) on five real world datasets and obtain better results than the baseline instance-MIR across all the datasets, while achieving state-of-the-art results on two of the datasets.
Integer Programming for Learning Directed Acyclic Graphs from Continuous Data
Authors: Hasan Manzour, Simge Küçükyavuz, Ali Shojaie
Date: 2019-04-23 00:00:00

Learning directed acyclic graphs (DAGs) from data is a challenging task both
in theory and in practice, because the number of possible DAGs scales
superexponentially with the number of nodes. In this paper, we study the
problem of learning an optimal DAG from continuous observational data. We cast
this problem in the form of a mathematical programming model which can
naturally incorporate a super-structure in order to reduce the set of possible
candidate DAGs. We use the penalized negative log-likelihood score function
with both $\ell_0$ and $\ell_1$ regularizations and propose a new mixed-integer
quadratic optimization (MIQO) model, referred to as a layered network (LN)
formulation. The LN formulation is a compact model, which enjoys as tight an
optimal continuous relaxation value as the stronger but larger formulations
under a mild condition. Computational results indicate that the proposed
formulation outperforms existing mathematical formulations and scales better
than available algorithms that can solve the same problem with only $\ell_1$
regularization. In particular, the LN formulation clearly outperforms existing
methods in terms of computational time needed to find an optimal DAG in the
presence of a sparse super-structure.
Quantum-assisted associative adversarial network: Applying quantum annealing in deep learning
Authors: Max Wilson, Thomas Vandal, Tad Hogg, Eleanor Rieffel
Date: 2019-04-23 00:00:00
Tasks: Deep Learning

We present an algorithm for learning a latent variable generative model via
generative adversarial learning where the canonical uniform noise input is
replaced by samples from a graphical model. This graphical model is learned by
a Boltzmann machine which learns low-dimensional feature representation of data
extracted by the discriminator. A quantum annealer, the D-Wave 2000Q, is used
to sample from this model. This algorithm joins a growing family of algorithms
that use a quantum annealing subroutine in deep learning, and provides a
framework to test the advantages of quantum-assisted learning in GANs. Fully
connected, symmetric bipartite and Chimera graph topologies are compared on a
reduced stochastically binarized MNIST dataset, for both classical and quantum
annealing sampling methods. The quantum-assisted associative adversarial
network successfully learns a generative model of the MNIST dataset for all
topologies, and is also applied to the LSUN dataset bedrooms class for the
Chimera topology. Evaluated using the Fr\'{e}chet inception distance and
inception score, the quantum and classical versions of the algorithm are found
to have equivalent performance for learning an implicit generative model of the
MNIST dataset.
Deep Q-Learning for Nash Equilibria: Nash-DQN
Authors: Philippe Casgrain, Brian Ning, Sebastian Jaimungal
Date: 2019-04-23 00:00:00
Tasks: Q-Learning, Reinforcement Learning

Model-free learning for multi-agent stochastic games is an active area of research. Existing reinforcement learning algorithms, however, are often restricted to zero-sum games, and are applicable only in small state-action spaces or other simplified settings. Here, we develop a new data efficient Deep-Q-learning methodology for model-free learning of Nash equilibria for general-sum stochastic games. The algorithm uses a local linear-quadratic expansion of the stochastic game, which leads to analytically solvable optimal actions. The expansion is parametrized by deep neural networks to give it sufficient flexibility to learn the environment without the need to experience all state-action pairs. We study symmetry properties of the algorithm stemming from label-invariant stochastic games and as a proof of concept, apply our algorithm to learning optimal trading strategies in competitive electronic markets.
ML-KFHE: Multi-label ensemble classification algorithm exploiting sensor fusion properties of the Kalman filter
Authors: Arjun Pakrashi, Brian Mac Namee
Date: 2019-04-23 00:00:00
Tasks: General Classification, Multi-class Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Sensor Fusion

Despite the success of ensemble classification methods in multi-class classification problems, ensemble methods based on approaches other than bagging have not been widely explored for multi-label classification problems. The Kalman Filter-based Heuristic Ensemble (KFHE) is an ensemble method that exploits the sensor fusion properties of the Kalman filter to combine several classifier models, and that has been shown to be very effective. This work proposes a multi-label version of KFHE, ML-KFHE, demonstrating the effectiveness of the KFHE method on multi-label datasets. Two variants are introduced based on the underlying component classifier algorithm, ML-KFHE-HOMER, and ML-KFHE-CC which uses HOMER and Classifier Chain (CC) as the underlying multi-label algorithms respectively. ML-KFHE-HOMER and ML-KFHE-CC sequentially train multiple HOMER and CC multi-label classifiers and aggregate their outputs using the sensor fusion properties of the Kalman filter. Extensive experiments and detailed analysis were performed on thirteen multi-label datasets and eight other algorithms, which included state-of-the-art ensemble methods. The results show, for both versions, the ML-KFHE framework improves the predictive performance significantly with respect to bagged combinations of HOMER (named E-HOMER), also introduced in this paper, and bagged combination of CC, Ensemble Classifier Chains (ECC), thus demonstrating the effectiveness of ML-KFHE. Also, the ML-KFHE-HOMER variant was found to perform consistently and significantly better than the compared multi-label methods including existing approaches based on ensembles.
CascadeML: An Automatic Neural Network Architecture Evolution and Training Algorithm for Multi-label Classification
Authors: Arjun Pakrashi, Brian Mac Namee
Date: 2019-04-23 00:00:00
Tasks: AutoML, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

Multi-label classification is an approach which allows a datapoint to be
labelled with more than one class at the same time. A common but trivial
approach is to train individual binary classifiers per label, but the
performance can be improved by considering associations within the labels. Like
with any machine learning algorithm, hyperparameter tuning is important to
train a good multi-label classifier model. The task of selecting the best
hyperparameter settings for an algorithm is an optimisation problem. Very
limited work has been done on automatic hyperparameter tuning and AutoML in the
multi-label domain. This paper attempts to fill this gap by proposing a neural
network algorithm, CascadeML, to train multi-label neural network based on
cascade neural networks. This method requires minimal or no hyperparameter
tuning and also considers pairwise label associations. The cascade algorithm
grows the network architecture incrementally in a two phase process as it
learns the weights using adaptive first order gradient algorithm, therefore
omitting the requirement of preselecting the number of hidden layers, nodes and
the learning rate. The method was tested on 10 multi-label datasets and
compared with other multi-label classification algorithms. Results show that
CascadeML performs very well without hyperparameter tuning.
Deconstructing the Filter Bubble: User Decision-Making and Recommender Systems
Authors: Guy Aridor, Duarte Goncalves, Shan Sikdar
Date: 2019-04-23 00:00:00
Tasks: Decision Making, Recommendation Systems

We study a model of user decision-making in the context of recommender systems via numerical simulation. Our model provides an explanation for the findings of Nguyen, et. al (2014), where, in environments where recommender systems are typically deployed, users consume increasingly similar items over time even without recommendation. We find that recommendation alleviates these natural filter-bubble effects, but that it also leads to an increase in homogeneity across users, resulting in a trade-off between homogenizing across-user consumption and diversifying within-user consumption. Finally, we discuss how our model highlights the importance of collecting data on user beliefs and their evolution over time both to design better recommendations and to further understand their impact.
A neural network-based framework for financial model calibration
Authors: Shuaiqiang Liu, Anastasia Borovykh, Lech A. Grzelak, Cornelis W. Oosterlee
Date: 2019-04-23 00:00:00
Tasks: BIG-bench Machine Learning, global-optimization

A data-driven approach called CaNN (Calibration Neural Network) is proposed
to calibrate financial asset price models using an Artificial Neural Network
(ANN). Determining optimal values of the model parameters is formulated as
training hidden neurons within a machine learning framework, based on available
financial option prices. The framework consists of two parts: a forward pass in
which we train the weights of the ANN off-line, valuing options under many
different asset model parameter settings; and a backward pass, in which we
evaluate the trained ANN-solver on-line, aiming to find the weights of the
neurons in the input layer. The rapid on-line learning of implied volatility by
ANNs, in combination with the use of an adapted parallel global optimization
method, tackles the computation bottleneck and provides a fast and reliable
technique for calibrating model parameters while avoiding, as much as possible,
getting stuck in local minima. Numerical experiments confirm that this
machine-learning framework can be employed to calibrate parameters of
high-dimensional stochastic volatility models efficiently and accurately.
Block-distributed Gradient Boosted Trees
Authors: Theodore Vasiloudis, Hyunsu Cho, Henrik Boström
Date: 2019-04-23 00:00:00
Tasks: Click-Through Rate Prediction, Learning-To-Rank

The Gradient Boosted Tree (GBT) algorithm is one of the most popular machine learning algorithms used in production, for tasks that include Click-Through Rate (CTR) prediction and learning-to-rank. To deal with the massive datasets available today, many distributed GBT methods have been proposed. However, they all assume a row-distributed dataset, addressing scalability only with respect to the number of data points and not the number of features, and increasing communication cost for high-dimensional data. In order to allow for scalability across both the data point and feature dimensions, and reduce communication cost, we propose block-distributed GBTs. We achieve communication efficiency by making full use of the data sparsity and adapting the Quickscorer algorithm to the block-distributed setting. We evaluate our approach using datasets with millions of features, and demonstrate that we are able to achieve multiple orders of magnitude reduction in communication cost for sparse data, with no loss in accuracy, while providing a more scalable design. As a result, we are able to reduce the training time for high-dimensional data, and allow more cost-effective scale-out without the need for expensive network communication.