pwc-archive/papers-with-abstracts huggingface dataset:
DKN: Deep Knowledge-Aware Network for News Recommendation
Authors: Hongwei Wang, Fuzheng Zhang, Xing Xie, Minyi Guo
Date: 2018-01-25 00:00:00
Tasks: Click-Through Rate Prediction, Common Sense Reasoning, News Recommendation, Recommendation Systems

Online news recommender systems aim to address the information explosion of
news and make personalized recommendation for users. In general, news language
is highly condensed, full of knowledge entities and common sense. However,
existing methods are unaware of such external knowledge and cannot fully
discover latent knowledge-level connections among news. The recommended results
for a user are consequently limited to simple patterns and cannot be extended
reasonably. Moreover, news recommendation also faces the challenges of high
time-sensitivity of news and dynamic diversity of users' interests. To solve
the above problems, in this paper, we propose a deep knowledge-aware network
(DKN) that incorporates knowledge graph representation into news
recommendation. DKN is a content-based deep recommendation framework for
click-through rate prediction. The key component of DKN is a multi-channel and
word-entity-aligned knowledge-aware convolutional neural network (KCNN) that
fuses semantic-level and knowledge-level representations of news. KCNN treats
words and entities as multiple channels, and explicitly keeps their alignment
relationship during convolution. In addition, to address users' diverse
interests, we also design an attention module in DKN to dynamically aggregate a
user's history with respect to current candidate news. Through extensive
experiments on a real online news platform, we demonstrate that DKN achieves
substantial gains over state-of-the-art deep recommendation models. We also
validate the efficacy of the usage of knowledge in DKN.
Weighted Community Detection and Data Clustering Using Message Passing
Authors: Cheng Shi, Yanchen Liu, Pan Zhang
Date: 2018-01-30 00:00:00
Tasks: Bayesian Inference, Clustering, Community Detection

Grouping objects into clusters based on similarities or weights between them
is one of the most important problems in science and engineering. In this work,
by extending message passing algorithms and spectral algorithms proposed for
unweighted community detection problem, we develop a non-parametric method
based on statistical physics, by mapping the problem to Potts model at the
critical temperature of spin glass transition and applying belief propagation
to solve the marginals corresponding to the Boltzmann distribution. Our
algorithm is robust to over-fitting and gives a principled way to determine
whether there are significant clusters in the data and how many clusters there
are. We apply our method to different clustering tasks and use extensive
numerical experiments to illustrate the advantage of our method over existing
algorithms. In the community detection problem in weighted and directed
networks, we show that our algorithm significantly outperforms existing
algorithms. In the clustering problem when the data was generated by mixture
models in the sparse regime we show that our method works to the theoretical
limit of detectability and gives accuracy very close to that of the optimal
Bayesian inference. In the semi-supervised clustering problem, our method only
needs several labels to work perfectly in classic datasets. Finally, we further
develop Thouless-Anderson-Palmer equations which reduce heavily the computation
complexity in dense-networks but gives almost the same performance as belief
propagation.
Robustness of classification ability of spiking neural networks
Authors: Jie Yang, Pingping Zhang, Yan Liu
Date: 2018-01-30 00:00:00
Tasks: Classification, General Classification

It is well-known that the robustness of artificial neural networks (ANNs) is
important for their wide ranges of applications. In this paper, we focus on the
robustness of the classification ability of a spiking neural network which
receives perturbed inputs. Actually, the perturbation is allowed to be
arbitrary styles. However, Gaussian perturbation and other regular ones have
been rarely investigated. For classification problems, the closer to the
desired point, the more perturbed points there are in the input space. In
addition, the perturbation may be periodic. Based on these facts, we only
consider sinusoidal and Gaussian perturbations in this paper. With the
SpikeProp algorithm, we perform extensive experiments on the classical XOR
problem and other three benchmark datasets. The numerical results show that
there is not significant reduction in the classification ability of the network
if the input signals are subject to sinusoidal and Gaussian perturbations.
Hierarchical Spatial Transformer Network
Authors: Chang Shu, Xi Chen, Qiwei Xie, Hua Han
Date: 2018-01-29 00:00:00
Tasks: Optical Flow Estimation

Computer vision researchers have been expecting that neural networks have
spatial transformation ability to eliminate the interference caused by
geometric distortion for a long time. Emergence of spatial transformer network
makes dream come true. Spatial transformer network and its variants can handle
global displacement well, but lack the ability to deal with local spatial
variance. Hence how to achieve a better manner of deformation in the neural
network has become a pressing matter of the moment. To address this issue, we
analyze the advantages and disadvantages of approximation theory and optical
flow theory, then we combine them to propose a novel way to achieve image
deformation and implement it with a hierarchical convolutional neural network.
This new approach solves for a linear deformation along with an optical flow
field to model image deformation. In the experiments of cluttered MNIST
handwritten digits classification and image plane alignment, our method
outperforms baseline methods by a large margin.
Multiresolution Kernel Approximation for Gaussian Process Regression
Authors: Yi Ding, Risi Kondor, Jonathan Eskreis-Winkler
Date: 2017-08-07 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: regression

Gaussian process regression generally does not scale to beyond a few
thousands data points without applying some sort of kernel approximation
method. Most approximations focus on the high eigenvalue part of the spectrum
of the kernel matrix, $K$, which leads to bad performance when the length scale
of the kernel is small. In this paper we introduce Multiresolution Kernel
Approximation (MKA), the first true broad bandwidth kernel approximation
algorithm. Important points about MKA are that it is memory efficient, and it
is a direct method, which means that it also makes it easy to approximate
$K^{-1}$ and $\mathop{\textrm{det}}(K)$.
Learning to Emulate an Expert Projective Cone Scheduler
Authors: Neal Master
Date: 2018-01-30 00:00:00
Tasks: Scheduling

Projective cone scheduling defines a large class of rate-stabilizing policies
for queueing models relevant to several applications. While there exists
considerable theory on the properties of projective cone schedulers, there is
little practical guidance on choosing the parameters that define them. In this
paper, we propose an algorithm for designing an automated projective cone
scheduling system based on observations of an expert projective cone scheduler.
We show that the estimated scheduling policy is able to emulate the expert in
the sense that the average loss realized by the learned policy will converge to
zero. Specifically, for a system with $n$ queues observed over a time horizon
$T$, the average loss for the algorithm is $O(\ln(T)\sqrt{\ln(n)/T})$. This
upper bound holds regardless of the statistical characteristics of the system.
The algorithm uses the multiplicative weights update method and can be applied
online so that additional observations of the expert scheduler can be used to
improve an existing estimate of the policy. This provides a data-driven method
for designing a scheduling policy based on observations of a human expert. We
demonstrate the efficacy of the algorithm with a simple numerical example and
discuss several extensions.
Machine learning for graph-based representations of three-dimensional discrete fracture networks
Authors: Manuel Valera, Zhengyang Guo, Priscilla Kelly, Sean Matz, Vito Adrian Cantu, Allon G. Percus, Jeffrey D. Hyman, Gowri Srinivasan, Hari S. Viswanathan
Date: 2017-05-27 00:00:00
Tasks: BIG-bench Machine Learning, Uncertainty Quantification

Structural and topological information play a key role in modeling flow and
transport through fractured rock in the subsurface. Discrete fracture network
(DFN) computational suites such as dfnWorks are designed to simulate flow and
transport in such porous media. Flow and transport calculations reveal that a
small backbone of fractures exists, where most flow and transport occurs.
Restricting the flowing fracture network to this backbone provides a
significant reduction in the network's effective size. However, the particle
tracking simulations needed to determine the reduction are computationally
intensive. Such methods may be impractical for large systems or for robust
uncertainty quantification of fracture networks, where thousands of forward
simulations are needed to bound system behavior.
  In this paper, we develop an alternative network reduction approach to
characterizing transport in DFNs, by combining graph theoretical and machine
learning methods. We consider a graph representation where nodes signify
fractures and edges denote their intersections. Using random forest and support
vector machines, we rapidly identify a subnetwork that captures the flow
patterns of the full DFN, based primarily on node centrality features in the
graph. Our supervised learning techniques train on particle-tracking backbone
paths found by dfnWorks, but run in negligible time compared to those
simulations. We find that our predictions can reduce the network to
approximately 20% of its original size, while still generating breakthrough
curves consistent with those of the original network.
Bayesian Nonparametric Kernel-Learning
Authors: Junier Oliva, Avinava Dubey, Andrew G. Wilson, Barnabas Poczos, Jeff Schneider, Eric P. Xing
Date: 2015-06-29 00:00:00

Kernel methods are ubiquitous tools in machine learning. However, there is
often little reason for the common practice of selecting a kernel a priori.
Even if a universal approximating kernel is selected, the quality of the finite
sample estimator may be greatly affected by the choice of kernel. Furthermore,
when directly applying kernel methods, one typically needs to compute a $N
\times N$ Gram matrix of pairwise kernel evaluations to work with a dataset of
$N$ instances. The computation of this Gram matrix precludes the direct
application of kernel methods on large datasets, and makes kernel learning
especially difficult. In this paper we introduce Bayesian nonparmetric
kernel-learning (BaNK), a generic, data-driven framework for scalable learning
of kernels. BaNK places a nonparametric prior on the spectral distribution of
random frequencies allowing it to both learn kernels and scale to large
datasets. We show that this framework can be used for large scale regression
and classification tasks. Furthermore, we show that BaNK outperforms several
other scalable approaches for kernel learning on a variety of real world
datasets.
Personalized Survival Prediction with Contextual Explanation Networks
Authors: Maruan Al-Shedivat, Avinava Dubey, Eric P. Xing
Date: 2018-01-30 00:00:00
Tasks: Prediction, Survival Prediction

Accurate and transparent prediction of cancer survival times on the level of
individual patients can inform and improve patient care and treatment
practices. In this paper, we design a model that concurrently learns to
accurately predict patient-specific survival distributions and to explain its
predictions in terms of patient attributes such as clinical tests or
assessments. Our model is flexible and based on a recurrent network, can handle
various modalities of data including temporal measurements, and yet constructs
and uses simple explanations in the form of patient- and time-specific linear
regression. For analysis, we use two publicly available datasets and show that
our networks outperform a number of baselines in prediction while providing a
way to inspect the reasons behind each prediction.
Parallel Tracking and Verifying
Authors: Heng Fan, Haibin Ling
Date: 2018-01-30 00:00:00
Tasks: Object Tracking, Visual Object Tracking

Being intensively studied, visual object tracking has witnessed great
advances in either speed (e.g., with correlation filters) or accuracy (e.g.,
with deep features). Real-time and high accuracy tracking algorithms, however,
remain scarce. In this paper we study the problem from a new perspective and
present a novel parallel tracking and verifying (PTAV) framework, by taking
advantage of the ubiquity of multi-thread techniques and borrowing ideas from
the success of parallel tracking and mapping in visual SLAM. The proposed PTAV
framework is typically composed of two components, a (base) tracker T and a
verifier V, working in parallel on two separate threads. The tracker T aims to
provide a super real-time tracking inference and is expected to perform well
most of the time; by contrast, the verifier V validates the tracking results
and corrects T when needed. The key innovation is that, V does not work on
every frame but only upon the requests from T; on the other end, T may adjust
the tracking according to the feedback from V. With such collaboration, PTAV
enjoys both the high efficiency provided by T and the strong discriminative
power by V. Meanwhile, to adapt V to object appearance changes over time, we
maintain a dynamic target template pool for adaptive verification, resulting in
further performance improvements. In our extensive experiments on popular
benchmarks including OTB2015, TC128, UAV20L and VOT2016, PTAV achieves the best
tracking accuracy among all real-time trackers, and in fact even outperforms
many deep learning based algorithms. Moreover, as a general framework, PTAV is
very flexible with great potentials for future improvement and generalization.
The Intriguing Properties of Model Explanations
Authors: Maruan Al-Shedivat, Avinava Dubey, Eric P. Xing
Date: 2018-01-30 00:00:00
Tasks: model

Linear approximations to the decision boundary of a complex model have become
one of the most popular tools for interpreting predictions. In this paper, we
study such linear explanations produced either post-hoc by a few recent methods
or generated along with predictions with contextual explanation networks
(CENs). We focus on two questions: (i) whether linear explanations are always
consistent or can be misleading, and (ii) when integrated into the prediction
process, whether and how explanations affect the performance of the model. Our
analysis sheds more light on certain properties of explanations produced by
different methods and suggests that learning models that explain and predict
jointly is often advantageous.
Mix-and-Match Tuning for Self-Supervised Semantic Segmentation
Authors: Xiaohang Zhan, Ziwei Liu, Ping Luo, Xiaoou Tang, Chen Change Loy
Date: 2017-12-02 00:00:00
Tasks: Colorization, Image Colorization, Image Segmentation, Segmentation, Semantic Segmentation, Triplet

Deep convolutional networks for semantic image segmentation typically require
large-scale labeled data, e.g. ImageNet and MS COCO, for network pre-training.
To reduce annotation efforts, self-supervised semantic segmentation is recently
proposed to pre-train a network without any human-provided labels. The key of
this new form of learning is to design a proxy task (e.g. image colorization),
from which a discriminative loss can be formulated on unlabeled data. Many
proxy tasks, however, lack the critical supervision signals that could induce
discriminative representation for the target image segmentation task. Thus
self-supervision's performance is still far from that of supervised
pre-training. In this study, we overcome this limitation by incorporating a
"mix-and-match" (M&M) tuning stage in the self-supervision pipeline. The
proposed approach is readily pluggable to many self-supervision methods and
does not use more annotated samples than the original process. Yet, it is
capable of boosting the performance of target image segmentation task to
surpass fully-supervised pre-trained counterpart. The improvement is made
possible by better harnessing the limited pixel-wise annotations in the target
dataset. Specifically, we first introduce the "mix" stage, which sparsely
samples and mixes patches from the target set to reflect rich and diverse local
patch statistics of target images. A "match" stage then forms a class-wise
connected graph, which can be used to derive a strong triplet-based
discriminative loss for fine-tuning the network. Our paradigm follows the
standard practice in existing self-supervised studies and no extra data or
label is required. With the proposed M&M approach, for the first time, a
self-supervision method can achieve comparable or even better performance
compared to its ImageNet pre-trained counterpart on both PASCAL VOC2012 dataset
and CityScapes dataset.
Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning
Authors: Scott Cyphers, Arjun K. Bansal, Anahita Bhiwandiwalla, Jayaram Bobba, Matthew Brookhart, Avijit Chakraborty, Will Constable, Christian Convey, Leona Cook, Omar Kanawi, Robert Kimball, Jason Knight, Nikolay Korovaiko, Varun Kumar, Yixing Lao, Christopher R. Lishka, Jaikrishnan Menon, Jennifer Myers, Sandeep Aswath Narayana, Adam Procter, Tristan J. Webb
Date: 2018-01-24 00:00:00
Tasks: CPU, Deep Learning, graph partitioning, Management, Math

The Deep Learning (DL) community sees many novel topologies published each
year. Achieving high performance on each new topology remains challenging, as
each requires some level of manual effort. This issue is compounded by the
proliferation of frameworks and hardware platforms. The current approach, which
we call "direct optimization", requires deep changes within each framework to
improve the training performance for each hardware backend (CPUs, GPUs, FPGAs,
ASICs) and requires $\mathcal{O}(fp)$ effort; where $f$ is the number of
frameworks and $p$ is the number of platforms. While optimized kernels for
deep-learning primitives are provided via libraries like Intel Math Kernel
Library for Deep Neural Networks (MKL-DNN), there are several compiler-inspired
ways in which performance can be further optimized. Building on our experience
creating neon (a fast deep learning library on GPUs), we developed Intel
nGraph, a soon to be open-sourced C++ library to simplify the realization of
optimized deep learning performance across frameworks and hardware platforms.
Initially-supported frameworks include TensorFlow, MXNet, and Intel neon
framework. Initial backends are Intel Architecture CPUs (CPU), the Intel(R)
Nervana Neural Network Processor(R) (NNP), and NVIDIA GPUs. Currently supported
compiler optimizations include efficient memory management and data layout
abstraction. In this paper, we describe our overall architecture and its core
components. In the future, we envision extending nGraph API support to a wider
range of frameworks, hardware (including FPGAs and ASICs), and compiler
optimizations (training versus inference optimizations, multi-node and
multi-device scaling via efficient sub-graph partitioning, and HW-specific
compounding of operations).
Predicting Rapid Fire Growth (Flashover) Using Conditional Generative Adversarial Networks
Authors: Kyongsik Yun, Jessi Bustos, Thomas Lu
Date: 2018-01-30 00:00:00
Tasks: Image Enhancement

A flashover occurs when a fire spreads very rapidly through crevices due to
intense heat. Flashovers present one of the most frightening and challenging
fire phenomena to those who regularly encounter them: firefighters.
Firefighters' safety and lives often depend on their ability to predict
flashovers before they occur. Typical pre-flashover fire characteristics
include dark smoke, high heat, and rollover ("angel fingers") and can be
quantified by color, size, and shape. Using a color video stream from a
firefighter's body camera, we applied generative adversarial neural networks
for image enhancement. The neural networks were trained to enhance very dark
fire and smoke patterns in videos and monitor dynamic changes in smoke and fire
areas. Preliminary tests with limited flashover training videos showed that we
predicted a flashover as early as 55 seconds before it occurred.
Contextual Explanation Networks
Authors: Maruan Al-Shedivat, Avinava Dubey, Eric P. Xing
Date: 2017-05-29 00:00:00
Proceeding: ICLR 2018 1
Tasks: Image Classification, Interpretability Techniques for Deep Learning, Prediction, Sentiment Analysis, Survival Analysis, text-classification, Text Classification, valid

Modern learning algorithms excel at producing accurate but complex models of the data. However, deploying such models in the real-world requires extra care: we must ensure their reliability, robustness, and absence of undesired biases. This motivates the development of models that are equally accurate but can be also easily inspected and assessed beyond their predictive performance. To this end, we introduce contextual explanation networks (CEN)---a class of architectures that learn to predict by generating and utilizing intermediate, simplified probabilistic models. Specifically, CENs generate parameters for intermediate graphical models which are further used for prediction and play the role of explanations. Contrary to the existing post-hoc model-explanation tools, CENs learn to predict and to explain simultaneously. Our approach offers two major advantages: (i) for each prediction valid, instance-specific explanation is generated with no computational overhead and (ii) prediction via explanation acts as a regularizer and boosts performance in data-scarce settings. We analyze the proposed framework theoretically and experimentally. Our results on image and text classification and survival analysis tasks demonstrate that CENs are not only competitive with the state-of-the-art methods but also offer additional insights behind each prediction, that can be valuable for decision support. We also show that while post-hoc methods may produce misleading explanations in certain cases, CENs are consistent and allow to detect such cases systematically.
Discrete Autoencoders for Sequence Models
Authors: Łukasz Kaiser, Samy Bengio
Date: 2018-01-29 00:00:00
Proceeding: ICLR 2018 1
Tasks: Language Modeling, Language Modelling, Machine Translation, Translation

Recurrent models for sequences have been recently successful at many tasks,
especially for language modeling and machine translation. Nevertheless, it
remains challenging to extract good representations from these models. For
instance, even though language has a clear hierarchical structure going from
characters through words to sentences, it is not apparent in current language
models. We propose to improve the representation in sequence models by
augmenting current approaches with an autoencoder that is forced to compress
the sequence through an intermediate discrete latent space. In order to
propagate gradients though this discrete representation we introduce an
improved semantic hashing technique. We show that this technique performs well
on a newly proposed quantitative efficiency measure. We also analyze latent
codes produced by the model showing how they correspond to words and phrases.
Finally, we present an application of the autoencoder-augmented model to
generating diverse translations.