pwc-archive/papers-with-abstracts huggingface dataset:
Off-Policy Evaluation of Probabilistic Identity Data in Lookalike Modeling
Authors: Randell Cotta, Mingyang Hu, Dan Jiang, Peizhou Liao
Date: 2019-01-04 00:00:00
Tasks: Marketing, Off-policy evaluation

We evaluate the impact of probabilistically-constructed digital identity data
collected from Sep. to Dec. 2017 (approx.), in the context of
Lookalike-targeted campaigns. The backbone of this study is a large set of
probabilistically-constructed "identities", represented as small bags of
cookies and mobile ad identifiers with associated metadata, that are likely all
owned by the same underlying user. The identity data allows to generate
"identity-based", rather than "identifier-based", user models, giving a fuller
picture of the interests of the users underlying the identifiers. We employ
off-policy techniques to evaluate the potential of identity-powered lookalike
models without incurring the risk of allowing untested models to direct large
amounts of ad spend or the large cost of performing A/B tests. We add to
historical work on off-policy evaluation by noting a significant type of
"finite-sample bias" that occurs for studies combining modestly-sized datasets
and evaluation metrics involving rare events (e.g., conversions). We illustrate
this bias using a simulation study that later informs the handling of inverse
propensity weights in our analyses on real data. We demonstrate significant
lift in identity-powered lookalikes versus an identity-ignorant baseline: on
average ~70% lift in conversion rate. This rises to factors of ~(4-32)x for
identifiers having little data themselves, but that can be inferred to belong
to users with substantial data to aggregate across identifiers. This implies
that identity-powered user modeling is especially important in the context of
identifiers having very short lifespans (i.e., frequently churned cookies). Our
work motivates and informs the use of probabilistically-constructed identities
in marketing. It also deepens the canon of examples in which off-policy
learning has been employed to evaluate the complex systems of the internet
economy.
Visual Reasoning of Feature Attribution with Deep Recurrent Neural Networks
Authors: Chuan Wang, Takeshi Onishi, Keiichi Nemoto, Kwan-Liu Ma
Date: 2019-01-17 00:00:00
Tasks: Classification, General Classification, Visual Reasoning

Deep Recurrent Neural Network (RNN) has gained popularity in many sequence
classification tasks. Beyond predicting a correct class for each data instance,
data scientists also want to understand what differentiating factors in the
data have contributed to the classification during the learning process. We
present a visual analytics approach to facilitate this task by revealing the
RNN attention for all data instances, their temporal positions in the
sequences, and the attribution of variables at each value level. We demonstrate
with real-world datasets that our approach can help data scientists to
understand such dynamics in deep RNNs from the training results, hence guiding
their modeling process.
CodeX: Bit-Flexible Encoding for Streaming-based FPGA Acceleration of DNNs
Authors: Mohammad Samragh, Mojan Javaheripi, Farinaz Koushanfar
Date: 2019-01-17 00:00:00
Tasks: Reinforcement Learning

This paper proposes CodeX, an end-to-end framework that facilitates encoding,
bitwidth customization, fine-tuning, and implementation of neural networks on
FPGA platforms. CodeX incorporates nonlinear encoding to the computation flow
of neural networks to save memory. The encoded features demand significantly
lower storage compared to the raw full-precision activation values; therefore,
the execution flow of CodeX hardware engine is completely performed within the
FPGA using on-chip streaming buffers with no access to the off-chip DRAM. We
further propose a fully-automated algorithm inspired by reinforcement learning
which determines the customized encoding bitwidth across network layers. CodeX
full-stack framework comprises of a compiler which takes a high-level Python
description of an arbitrary neural network architecture. The compiler then
instantiates the corresponding elements from CodeX Hardware library for FPGA
implementation. Proof-of-concept evaluations on MNIST, SVHN, and CIFAR-10
datasets demonstrate an average of 4.65x throughput improvement compared to
stand-alone weight encoding. We further compare CodeX with six existing
full-precision DNN accelerators on ImageNet, showing an average of 3.6x and
2.54x improvement in throughput and performance-per-watt, respectively.
Disentangling Video with Independent Prediction
Authors: William F. Whitney, Rob Fergus
Date: 2019-01-17 00:00:00
Tasks: Prediction

We propose an unsupervised variational model for disentangling video into
independent factors, i.e. each factor's future can be predicted from its past
without considering the others. We show that our approach often learns factors
which are interpretable as objects in a scene.
Quadratic Autoencoder (Q-AE) for Low-dose CT Denoising
Authors: Fenglei Fan, Hongming Shan, Mannudeep K. Kalra, Ramandeep Singh, Guhan Qian, Matthew Getzin, Yueyang Teng, Juergen Hahn, Ge Wang
Date: 2019-01-17 00:00:00
Tasks: Decoder, Deep Learning, Denoising, Diversity, Image Denoising

Inspired by complexity and diversity of biological neurons, our group proposed quadratic neurons by replacing the inner product in current artificial neurons with a quadratic operation on input data, thereby enhancing the capability of an individual neuron. Along this direction, we are motivated to evaluate the power of quadratic neurons in popular network architectures, simulating human-like learning in the form of quadratic-neuron-based deep learning. Our prior theoretical studies have shown important merits of quadratic neurons and networks in representation, efficiency, and interpretability. In this paper, we use quadratic neurons to construct an encoder-decoder structure, referred as the quadratic autoencoder, and apply it to low-dose CT denoising. The experimental results on the Mayo low-dose CT dataset demonstrate the utility of quadratic autoencoder in terms of image denoising and model efficiency. To our best knowledge, this is the first time that the deep learning approach is implemented with a new type of neurons and demonstrates a significant potential in the medical imaging field.
Virtual-to-Real-World Transfer Learning for Robots on Wilderness Trails
Authors: Michael L. Iuzzolino, Michael E. Walker, Daniel Szafir
Date: 2019-01-17 00:00:00
Tasks: Autonomous Navigation, Deep Learning, General Classification, Management, Transfer Learning, Weather Forecasting

Robots hold promise in many scenarios involving outdoor use, such as
search-and-rescue, wildlife management, and collecting data to improve
environment, climate, and weather forecasting. However, autonomous navigation
of outdoor trails remains a challenging problem. Recent work has sought to
address this issue using deep learning. Although this approach has achieved
state-of-the-art results, the deep learning paradigm may be limited due to a
reliance on large amounts of annotated training data. Collecting and curating
training datasets may not be feasible or practical in many situations,
especially as trail conditions may change due to seasonal weather variations,
storms, and natural erosion. In this paper, we explore an approach to address
this issue through virtual-to-real-world transfer learning using a variety of
deep learning models trained to classify the direction of a trail in an image.
Our approach utilizes synthetic data gathered from virtual environments for
model training, bypassing the need to collect a large amount of real images of
the outdoors. We validate our approach in three main ways. First, we
demonstrate that our models achieve classification accuracies upwards of 95% on
our synthetic data set. Next, we utilize our classification models in the
control system of a simulated robot to demonstrate feasibility. Finally, we
evaluate our models on real-world trail data and demonstrate the potential of
virtual-to-real-world transfer learning.
Machine learning with neural networks
Authors: B. Mehlig
Date: 2019-01-17 00:00:00
Tasks: BIG-bench Machine Learning, Object Recognition, Time Series Analysis

These are lecture notes for a course on machine learning with neural networks for scientists and engineers that I have given at Gothenburg University and Chalmers Technical University in Gothenburg, Sweden. The material is organised into three parts: Hopfield networks, supervised learning of labeled data, and learning algorithms for unlabeled data sets. Part I introduces stochastic recurrent networks: Hopfield networks and Boltzmann machines. The analysis of their learning rules sets the scene for the later parts. Part II describes supervised learning with multilayer perceptrons and convolutional neural networks. This part starts with a simple geometrical interpretation of the learning rule and leads to the recent successes of convolutional networks in object recognition, recurrent networks in language processing, and reservoir computers in time-series analysis. Part III explains what neural networks can learn about data that is not labeled. This part begins with a description of unsupervised learning techniques for clustering of data, non-linear projections, and embeddings. A section on autoencoders explains how to learn without labels using convolutional networks, and the last chapter is dedicated to reinforcement learning. The overall goal of the course is to explain the fundamental principles that allow neural networks to learn, emphasising ideas and concepts that are common to all three parts. The present version does not contain exercises (copyright owned by Cambridge University Press). The complete book is available at https://www.cambridge.org/gb/academic/subjects/physics/statistical-physics/machine-learning-neural-networks-introduction-scientists-and-engineers?format=HB.
Efficient Matrix Profile Computation Using Different Distance Functions
Authors: Reza Akbarinia, Bertrand Cloez
Date: 2019-01-17 00:00:00
Tasks: Time Series, Time Series Analysis

Matrix profile has been recently proposed as a promising technique to the
problem of all-pairs-similarity search on time series. Efficient algorithms
have been proposed for computing it, e.g., STAMP, STOMP and SCRIMP++. All these
algorithms use the z-normalized Euclidean distance to measure the distance
between subsequences. However, as we observed, for some datasets other
Euclidean measurements are more useful for knowledge discovery from time
series. In this paper, we propose efficient algorithms for computing matrix
profile for a general class of Euclidean distances. We first propose a simple
but efficient algorithm called AAMP for computing matrix profile with the
"pure" (non-normalized) Euclidean distance. Then, we extend our algorithm for
the p-norm distance. We also propose an algorithm, called ACAMP, that uses the
same principle as AAMP, but for the case of z-normalized Euclidean distance. We
implemented our algorithms, and evaluated their performance through
experimentation. The experiments show excellent performance results. For
example, they show that AAMP is very efficient for computing matrix profile for
non-normalized Euclidean distances. The results also show that the ACAMP
algorithm is significantly faster than SCRIMP++ (the state of the art matrix
profile algorithm) for the case of z-normalized Euclidean distance.
The Oracle of DLphi
Authors: Dominik Alfke, Weston Baines, Jan Blechschmidt, Mauricio J. del Razo Sarmina, Amnon Drory, Dennis Elbrächter, Nando Farchmin, Matteo Gambara, Silke Glas, Philipp Grohs, Peter Hinz, Danijel Kivaranovic, Christian Kümmerle, Gitta Kutyniok, Sebastian Lunz, Jan Macdonald, Ryan Malthaner, Gregory Naisat, Ariel Neufeld, Philipp Christian Petersen, Rafael Reisenhofer, Jun-Da Sheng, Laura Thesing, Philipp Trunschke, Johannes von Lindheim, David Weber, Melanie Weber
Date: 2019-01-17 00:00:00
Tasks: General Classification

We present a novel technique based on deep learning and set theory which
yields exceptional classification and prediction results. Having access to a
sufficiently large amount of labelled training data, our methodology is capable
of predicting the labels of the test data almost always even if the training
data is entirely unrelated to the test data. In other words, we prove in a
specific setting that as long as one has access to enough data points, the
quality of the data is irrelevant.
Attentive Neural Processes
Authors: Hyunjik Kim, andriy mnih, Jonathan Schwarz, Marta Garnelo, Ali Eslami, Dan Rosenbaum, Oriol Vinyals, Yee Whye Teh
Date: 2019-01-17 00:00:00
Proceeding: ICLR 2019 5
Tasks: regression

Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by learning to map a context set of observed input-output pairs to a distribution over regression functions. Each function models the distribution of the output given an input, conditioned on the context. NPs have the benefit of fitting observed data efficiently with linear complexity in the number of context input-output pairs, and can learn a wide family of conditional distributions; they learn predictive distributions conditioned on context sets of arbitrary size. Nonetheless, we show that NPs suffer a fundamental drawback of underfitting, giving inaccurate predictions at the inputs of the observed data they condition on. We address this issue by incorporating attention into NPs, allowing each input location to attend to the relevant context points for the prediction. We show that this greatly improves the accuracy of predictions, results in noticeably faster training, and expands the range of functions that can be modelled.
Applying SVGD to Bayesian Neural Networks for Cyclical Time-Series Prediction and Inference
Authors: Xinyu Hu, Paul Szerlip, Theofanis Karaletsos, Rohit Singh
Date: 2019-01-17 00:00:00
Tasks: regression, Sensitivity, Time Series, Time Series Analysis, Time Series Prediction

A regression-based BNN model is proposed to predict spatiotemporal quantities
like hourly rider demand with calibrated uncertainties. The main contributions
of this paper are (i) A feed-forward deterministic neural network (DetNN)
architecture that predicts cyclical time series data with sensitivity to
anomalous forecasting events; (ii) A Bayesian framework applying SVGD to train
large neural networks for such tasks, capable of producing time series
predictions as well as measures of uncertainty surrounding the predictions.
Experiments show that the proposed BNN reduces average estimation error by 10%
across 8 U.S. cities compared to a fine-tuned multilayer perceptron (MLP), and
4% better than the same network architecture trained without SVGD.
Diverse mini-batch Active Learning
Authors: Fedor Zhdanov
Date: 2019-01-17 00:00:00
Tasks: Active Learning, Clustering, Diversity, General Classification, Informativeness

We study the problem of reducing the amount of labeled training data required
to train supervised classification models. We approach it by leveraging Active
Learning, through sequential selection of examples which benefit the model
most. Selecting examples one by one is not practical for the amount of training
examples required by the modern Deep Learning models. We consider the
mini-batch Active Learning setting, where several examples are selected at
once. We present an approach which takes into account both informativeness of
the examples for the model, as well as the diversity of the examples in a
mini-batch. By using the well studied K-means clustering algorithm, this
approach scales better than the previously proposed approaches, and achieves
comparable or better performance.
Fleet Prognosis with Physics-informed Recurrent Neural Networks
Authors: Renato Giorgiani Nascimento, Felipe A. C. Viana
Date: 2019-01-16 00:00:00
Tasks: Graph Regression, Graph-to-Sequence, Physics-informed machine learning, Prognosis

Services and warranties of large fleets of engineering assets is a very
profitable business. The success of companies in that area is often related to
predictive maintenance driven by advanced analytics. Therefore, accurate
modeling, as a way to understand how the complex interactions between operating
conditions and component capability define useful life, is key for services
profitability. Unfortunately, building prognosis models for large fleets is a
daunting task as factors such as duty cycle variation, harsh environments,
inadequate maintenance, and problems with mass production can lead to large
discrepancies between designed and observed useful lives. This paper introduces
a novel physics-informed neural network approach to prognosis by extending
recurrent neural networks to cumulative damage models. We propose a new
recurrent neural network cell designed to merge physics-informed and
data-driven layers. With that, engineers and scientists have the chance to use
physics-informed layers to model parts that are well understood (e.g., fatigue
crack growth) and use data-driven layers to model parts that are poorly
characterized (e.g., internal loads). A simple numerical experiment is used to
present the main features of the proposed physics-informed recurrent neural
network for damage accumulation. The test problem consist of predicting fatigue
crack length for a synthetic fleet of airplanes subject to different mission
mixes. The model is trained using full observation inputs (far-field loads) and
very limited observation of outputs (crack length at inspection for only a
portion of the fleet). The results demonstrate that our proposed hybrid
physics-informed recurrent neural network is able to accurately model fatigue
crack growth even when the observed distribution of crack length does not match
with the (unobservable) fleet distribution.
Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization
Authors: Xiao Fu, Shahana Ibrahim, Hoi-To Wai, Cheng Gao, Kejun Huang
Date: 2019-01-16 00:00:00
Tasks: Stochastic Optimization

This work considers the problem of computing the canonical polyadic decomposition (CPD) of large tensors. Prior works mostly leverage data sparsity to handle this problem, which is not suitable for handling dense tensors that often arise in applications such as medical imaging, computer vision, and remote sensing. Stochastic optimization is known for its low memory cost and per-iteration complexity when handling dense data. However, exisiting stochastic CPD algorithms are not flexible enough to incorporate a variety of constraints/regularizations that are of interest in signal and data analytics. Convergence properties of many such algorithms are also unclear. In this work, we propose a stochastic optimization framework for large-scale CPD with constraints/regularizations. The framework works under a doubly randomized fashion, and can be regarded as a judicious combination of randomized block coordinate descent (BCD) and stochastic proximal gradient (SPG). The algorithm enjoys lightweight updates and small memory footprint. In addition, this framework entails considerable flexibility---many frequently used regularizers and constraints can be readily handled under the proposed scheme. The approach is also supported by convergence analysis. Numerical results on large-scale dense tensors are employed to showcase the effectiveness of the proposed approach.
Response to "Visual Dialogue without Vision or Dialogue" (Massiceti et al., 2018)
Authors: Abhishek Das, Devi Parikh, Dhruv Batra
Date: 2019-01-16 00:00:00
Tasks: Visual Dialog

In a recent workshop paper, Massiceti et al. presented a baseline model and
subsequent critique of Visual Dialog (Das et al., CVPR 2017) that raises what
we believe to be unfounded concerns about the dataset and evaluation. This
article intends to rebut the critique and clarify potential confusions for
practitioners and future participants in the Visual Dialog challenge.
Differentially-Private Two-Party Egocentric Betweenness Centrality
Authors: Leyla Roohi, Benjamin I. P. Rubinstein, Vanessa Teague
Date: 2019-01-16 00:00:00
Tasks: Vocal Bursts Valence Prediction

We describe a novel protocol for computing the egocentric betweenness
centrality of a node when relevant edge information is spread between two
mutually distrusting parties such as two telecommunications providers. While
each node belongs to one network or the other, its ego network might include
edges unknown to its network provider. We develop a protocol of
differentially-private mechanisms to hide each network's internal edge
structure from the other; and contribute a new two-stage stratified sampler for
exponential improvement to time and space efficiency. Empirical results on
several open graph data sets demonstrate practical relative error rates while
delivering strong privacy guarantees, such as 16% error on a Facebook data set.
Evolving embodied intelligence from materials to machines
Authors: David Howard, Agoston E. Eiben, Danielle Frances Kennedy, Jean-Baptiste Mouret, Philip Valencia, Dave Winkler
Date: 2019-01-17 00:00:00

Natural lifeforms specialise to their environmental niches across many
levels; from low-level features such as DNA and proteins, through to
higher-level artefacts including eyes, limbs, and overarching body plans. We
propose Multi-Level Evolution (MLE), a bottom-up automatic process that designs
robots across multiple levels and niches them to tasks and environmental
conditions. MLE concurrently explores constituent molecular and material
'building blocks', as well as their possible assemblies into specialised
morphological and sensorimotor configurations. MLE provides a route to fully
harness a recent explosion in available candidate materials and ongoing
advances in rapid manufacturing processes. We outline a feasible MLE
architecture that realises this vision, highlight the main roadblocks and how
they may be overcome, and show robotic applications to which MLE is
particularly suited. By forming a research agenda to stimulate discussion
between researchers in related fields, we hope to inspire the pursuit of
multi-level robotic design all the way from material to machine.
Genetic Algorithms and the Traveling Salesman Problem a historical Review
Authors: Jan Scholz
Date: 2019-01-17 00:00:00
Tasks: Traveling Salesman Problem

In this paper a highly abstracted view on the historical development of
Genetic Algorithms for the Traveling Salesman Problem is given. In a meta-data
analysis three phases in the development can be distinguished. First
exponential growth in interest till 1996 can be observed, growth stays linear
till 2011 and after that publications deteriorate. These three phases are
examined and the major milestones are presented. Lastly an outlook to future
work in this field is infered.
Boolean matrix factorization meets consecutive ones property
Authors: Nikolaj Tatti, Pauli Miettinen
Date: 2019-01-17 00:00:00

Boolean matrix factorization is a natural and a popular technique for
summarizing binary matrices. In this paper, we study a problem of Boolean
matrix factorization where we additionally require that the factor matrices
have consecutive ones property (OBMF). A major application of this optimization
problem comes from graph visualization: standard techniques for visualizing
graphs are circular or linear layout, where nodes are ordered in circle or on a
line. A common problem with visualizing graphs is clutter due to too many
edges. The standard approach to deal with this is to bundle edges together and
represent them as ribbon. We also show that we can use OBMF for edge bundling
combined with circular or linear layout techniques.
  We demonstrate that not only this problem is NP-hard but we cannot have a
polynomial-time algorithm that yields a multiplicative approximation guarantee
(unless P = NP). On the positive side, we develop a greedy algorithm where at
each step we look for the best 1-rank factorization. Since even obtaining
1-rank factorization is NP-hard, we propose an iterative algorithm where we fix
one side and and find the other, reverse the roles, and repeat. We show that
this step can be done in linear time using pq-trees. We also extend the problem
to cyclic ones property and symmetric factorizations. Our experiments show that
our algorithms find high-quality factorizations and scale well.