pwc-archive/papers-with-abstracts huggingface dataset:
Revisiting Random Binning Features: Fast Convergence and Strong Parallelizability
Authors: Lingfei Wu, Ian E. H. Yen, Jie Chen, Rui Yan
Date: 2018-09-14 00:00:00

Kernel method has been developed as one of the standard approaches for
nonlinear learning, which however, does not scale to large data set due to its
quadratic complexity in the number of samples. A number of kernel approximation
methods have thus been proposed in the recent years, among which the random
features method gains much popularity due to its simplicity and direct
reduction of nonlinear problem to a linear one. The Random Binning (RB)
feature, proposed in the first random-feature paper \cite{rahimi2007random},
has drawn much less attention than the Random Fourier (RF) feature. In this
work, we observe that the RB features, with right choice of optimization
solver, could be orders-of-magnitude more efficient than other random features
and kernel approximation methods under the same requirement of accuracy. We
thus propose the first analysis of RB from the perspective of optimization,
which by interpreting RB as a Randomized Block Coordinate Descent in the
infinite-dimensional space, gives a faster convergence rate compared to that of
other random features. In particular, we show that by drawing $R$ random grids
with at least $\kappa$ number of non-empty bins per grid in expectation, RB
method achieves a convergence rate of $O(1/(\kappa R))$, which not only
sharpens its $O(1/\sqrt{R})$ rate from Monte Carlo analysis, but also shows a
$\kappa$ times speedup over other random features under the same analysis
framework. In addition, we demonstrate another advantage of RB in the
L1-regularized setting, where unlike other random features, a RB-based
Coordinate Descent solver can be parallelized with guaranteed speedup
proportional to $\kappa$. Our extensive experiments demonstrate the superior
performance of the RB features over other random features and kernel
approximation methods. Our code and data is available at {
\url{https://github.com/teddylfwu/RB_GEN}}.
Neural Network Topologies for Sparse Training
Authors: Ryan A. Robinett, Jeremy Kepner
Date: 2018-09-14 00:00:00

The sizes of deep neural networks (DNNs) are rapidly outgrowing the capacity
of hardware to store and train them. Research over the past few decades has
explored the prospect of sparsifying DNNs before, during, and after training by
pruning edges from the underlying topology. The resulting neural network is
known as a sparse neural network. More recent work has demonstrated the
remarkable result that certain sparse DNNs can train to the same precision as
dense DNNs at lower runtime and storage cost. An intriguing class of these
sparse DNNs is the X-Nets, which are initialized and trained upon a sparse
topology with neither reference to a parent dense DNN nor subsequent pruning.
We present an algorithm that deterministically generates sparse DNN topologies
that, as a whole, are much more diverse than X-Net topologies, while preserving
X-Nets' desired characteristics.
Graph Pattern Mining and Learning through User-defined Relations (Extended Version)
Authors: Carlos H. C. Teixeira, Leonardo Cotta, Bruno Ribeiro, Wagner Meira Jr
Date: 2018-09-14 00:00:00
Tasks: Graph Neural Network, Stochastic Optimization

In this work we propose R-GPM, a parallel computing framework for graph pattern mining (GPM) through a user-defined subgraph relation. More specifically, we enable the computation of statistics of patterns through their subgraph classes, generalizing traditional GPM methods. R-GPM provides efficient estimators for these statistics by employing a MCMC sampling algorithm combined with several optimizations. We provide both theoretical guarantees and empirical evaluations of our estimators in application scenarios such as stochastic optimization of deep high-order graph neural network models and pattern (motif) counting. We also propose and evaluate optimizations that enable improvements of our estimators accuracy, while reducing their computational costs in up to 3-orders-of-magnitude. Finally,we show that R-GPM is scalable, providing near-linear speedups on 44 cores in all of our tests.
Follow Me at the Edge: Mobility-Aware Dynamic Service Placement for Mobile Edge Computing
Authors: Tao Ouyang, Zhi Zhou, Xu Chen
Date: 2018-09-14 00:00:00
Tasks: Cloud Computing, Edge-computing

Mobile edge computing is a new computing paradigm, which pushes cloud
computing capabilities away from the centralized cloud to the network edge.
However, with the sinking of computing capabilities, the new challenge incurred
by user mobility arises: since end-users typically move erratically, the
services should be dynamically migrated among multiple edges to maintain the
service performance, i.e., user-perceived latency. Tackling this problem is
non-trivial since frequent service migration would greatly increase the
operational cost. To address this challenge in terms of the performance-cost
trade-off, in this paper we study the mobile edge service performance
optimization problem under long-term cost budget constraint. To address user
mobility which is typically unpredictable, we apply Lyapunov optimization to
decompose the long-term optimization problem into a series of real-time
optimization problems which do not require a priori knowledge such as user
mobility. As the decomposed problem is NP-hard, we first design an
approximation algorithm based on Markov approximation to seek a near-optimal
solution. To make our solution scalable and amenable to future 5G application
scenario with large-scale user devices, we further propose a distributed
approximation scheme with greatly reduced time complexity, based on the
technique of best response update. Rigorous theoretical analysis and extensive
evaluations demonstrate the efficacy of the proposed centralized and
distributed schemes.
Unsupervised Abstractive Sentence Summarization using Length Controlled Variational Autoencoder
Authors: Raphael Schumann
Date: 2018-09-14 00:00:00
Tasks: Decoder, Sentence, Sentence Summarization, Unsupervised Sentence Summarization

In this work we present an unsupervised approach to summarize sentences in
abstractive way using Variational Autoencoder (VAE). VAE are known to learn a
semantically rich latent variable, representing high dimensional input. VAEs
are trained by learning to reconstruct the input from the probabilistic latent
variable. Explicitly providing the information about output length during
training influences the VAE to not encode this information and thus can be
manipulated during inference. Instructing the decoder to produce a shorter
output sequence leads to expressing the input sentence with fewer words. We
show on different summarization data sets, that these shorter sentences can not
beat a simple baseline but yield higher ROUGE scores than trying to reconstruct
the whole sentence.
VoxelMorph: A Learning Framework for Deformable Medical Image Registration
Authors: Guha Balakrishnan, Amy Zhao, Mert R. Sabuncu, John Guttag, Adrian V. Dalca
Date: 2018-09-14 00:00:00
Tasks: Deformable Medical Image Registration, Diffeomorphic Medical Image Registration, Image Registration, Medical Image Analysis, Medical Image Registration

We present VoxelMorph, a fast learning-based framework for deformable, pairwise medical image registration. Traditional registration methods optimize an objective function for each pair of images, which can be time-consuming for large datasets or rich deformation models. In contrast to this approach, and building on recent learning-based methods, we formulate registration as a function that maps an input image pair to a deformation field that aligns these images. We parameterize the function via a convolutional neural network (CNN), and optimize the parameters of the neural network on a set of images. Given a new pair of scans, VoxelMorph rapidly computes a deformation field by directly evaluating the function. In this work, we explore two different training strategies. In the first (unsupervised) setting, we train the model to maximize standard image matching objective functions that are based on the image intensities. In the second setting, we leverage auxiliary segmentations available in the training data. We demonstrate that the unsupervised model's accuracy is comparable to state-of-the-art methods, while operating orders of magnitude faster. We also show that VoxelMorph trained with auxiliary data improves registration accuracy at test time, and evaluate the effect of training set size on registration. Our method promises to speed up medical image analysis and processing pipelines, while facilitating novel directions in learning-based registration and its applications. Our code is freely available at voxelmorph.csail.mit.edu.
A Variational Observation Model of 3D Object for Probabilistic Semantic SLAM
Authors: H. W. Yu, B. H. Le
Date: 2018-09-14 00:00:00
Tasks: Bayesian Inference, Object, object-detection, Object Detection, Scene Understanding, Semantic SLAM

We present a Bayesian object observation model for complete probabilistic
semantic SLAM. Recent studies on object detection and feature extraction have
become important for scene understanding and 3D mapping. However, 3D shape of
the object is too complex to formulate the probabilistic observation model;
therefore, performing the Bayesian inference of the object-oriented features as
well as their pose is less considered. Besides, when the robot equipped with an
RGB mono camera only observes the projected single view of an object, a
significant amount of the 3D shape information is abandoned. Due to these
limitations, semantic SLAM and viewpoint-independent loop closure using
volumetric 3D object shape is challenging. In order to enable the complete
formulation of probabilistic semantic SLAM, we approximate the observation
model of a 3D object with a tractable distribution. We also estimate the
variational likelihood from the 2D image of the object to exploit its observed
single view. In order to evaluate the proposed method, we perform pose and
feature estimation, and demonstrate that the automatic loop closure works
seamlessly without additional loop detector in various environments.
Automatic Catchphrase Extraction from Legal Case Documents via Scoring using Deep Neural Networks
Authors: Vu Tran, Minh Le Nguyen, Ken Satoh
Date: 2018-09-14 00:00:00

In this paper, we present a method of automatic catchphrase extracting from
legal case documents. We utilize deep neural networks for constructing scoring
model of our extraction system. We achieve comparable performance with systems
using corpus-wide and citation information which we do not use in our system.
Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translation
Authors: Brian Thompson, Huda Khayrallah, Antonios Anastasopoulos, Arya D. McCarthy, Kevin Duh, Rebecca Marvin, Paul McNamee, Jeremy Gwinnup, Tim Anderson, Philipp Koehn
Date: 2018-09-14 00:00:00
Proceeding: WS 2018 10
Tasks: Decoder, Domain Adaptation, Machine Translation, Translation

To better understand the effectiveness of continued training, we analyze the
major components of a neural machine translation system (the encoder, decoder,
and each embedding space) and consider each component's contribution to, and
capacity for, domain adaptation. We find that freezing any single component
during continued training has minimal impact on performance, and that
performance is surprisingly good when a single component is adapted while
holding the rest of the model fixed. We also find that continued training does
not move the model very far from the out-of-domain model, compared to a
sensitivity analysis metric, suggesting that the out-of-domain model can
provide a good generic initialization for the new domain.
Enhanced Optic Disk and Cup Segmentation with Glaucoma Screening from Fundus Images using Position encoded CNNs
Authors: Vismay Agrawal, Avinash Kori, Varghese Alex, Ganapathy Krishnamurthi
Date: 2018-09-14 00:00:00
Tasks: General Classification, Position, Segmentation

In this manuscript, we present a robust method for glaucoma screening from
fundus images using an ensemble of convolutional neural networks (CNNs). The
pipeline comprises of first segmenting the optic disk and optic cup from the
fundus image, then extracting a patch centered around the optic disk and
subsequently feeding to the classification network to differentiate the image
as diseased or healthy. In the segmentation network, apart from the image, we
make use of spatial co-ordinate (X \& Y) space so as to learn the structure of
interest better. The classification network is composed of a DenseNet201 and a
ResNet18 which were pre-trained on a large cohort of natural images. On the
REFUGE validation data (n=400), the segmentation network achieved a dice score
of 0.88 and 0.64 for optic disc and optic cup respectively. For the tasking
differentiating images affected with glaucoma from healthy images, the area
under the ROC curve was observed to be 0.85.
Model-Based Reinforcement Learning via Meta-Policy Optimization
Authors: Ignasi Clavera, Jonas Rothfuss, John Schulman, Yasuhiro Fujita, Tamim Asfour, Pieter Abbeel
Date: 2018-09-14 00:00:00
Tasks: Model-based Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Model-based reinforcement learning approaches carry the promise of being data
efficient. However, due to challenges in learning dynamics models that
sufficiently match the real-world dynamics, they struggle to achieve the same
asymptotic performance as model-free methods. We propose Model-Based
Meta-Policy-Optimization (MB-MPO), an approach that foregoes the strong
reliance on accurate learned dynamics models. Using an ensemble of learned
dynamic models, MB-MPO meta-learns a policy that can quickly adapt to any model
in the ensemble with one policy gradient step. This steers the meta-policy
towards internalizing consistent dynamics predictions among the ensemble while
shifting the burden of behaving optimally w.r.t. the model discrepancies
towards the adaptation step. Our experiments show that MB-MPO is more robust to
model imperfections than previous model-based approaches. Finally, we
demonstrate that our approach is able to match the asymptotic performance of
model-free methods while requiring significantly less experience.
A Time Series Graph Cut Image Segmentation Scheme for Liver Tumors
Authors: Laramie Paxton, Yufeng Cao, Kevin R. Vixie, Yu-An Wang, Brian Hobbs, Chaan Ng
Date: 2018-09-13 00:00:00
Tasks: Image Segmentation, Liver Segmentation, Semantic Segmentation, Time Series, Time Series Analysis

Tumor detection in biomedical imaging is a time-consuming process for medical
professionals and is not without errors. Thus in recent decades, researchers
have developed algorithmic techniques for image processing using a wide variety
of mathematical methods, such as statistical modeling, variational techniques,
and machine learning. In this paper, we propose a semi-automatic method for
liver segmentation of 2D CT scans into three labels denoting healthy, vessel,
or tumor tissue based on graph cuts. First, we create a feature vector for each
pixel in a novel way that consists of the 59 intensity values in the time
series data and propose a simplified perimeter cost term in the energy
functional. We normalize the data and perimeter terms in the functional to
expedite the graph cut without having to optimize the scaling parameter
$\lambda$. In place of a training process, predetermined tissue means are
computed based on sample regions identified by expert radiologists. The
proposed method also has the advantage of being relatively simple to implement
computationally. It was evaluated against the ground truth on a clinical CT
dataset of 10 tumors and yielded segmentations with a mean Dice similarity
coefficient (DSC) of .77 and mean volume overlap error (VOE) of 36.7%. The
average processing time was 1.25 minutes per slice.
CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learning
Authors: Jiachen Yang, Alireza Nakhaei, David Isele, Kikuo Fujimura, Hongyuan Zha
Date: 2018-09-13 00:00:00
Proceeding: ICLR 2020 1
Tasks: Autonomous Vehicles, Efficient Exploration, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

A variety of cooperative multi-agent control problems require agents to achieve individual goals while contributing to collective success. This multi-goal multi-agent setting poses difficulties for recent algorithms, which primarily target settings with a single global reward, due to two new challenges: efficient exploration for learning both individual goal attainment and cooperation for others' success, and credit-assignment for interactions between actions and goals of different agents. To address both challenges, we restructure the problem into a novel two-stage curriculum, in which single-agent goal attainment is learned prior to learning multi-agent cooperation, and we derive a new multi-goal multi-agent policy gradient with a credit function for localized credit assignment. We use a function augmentation scheme to bridge value and policy functions across the curriculum. The complete architecture, called CM3, learns significantly faster than direct adaptations of existing algorithms on three challenging multi-goal multi-agent problems: cooperative navigation in difficult formations, negotiating multi-vehicle lane changes in the SUMO traffic simulator, and strategic cooperation in a Checkers environment.
Explainable time series tweaking via irreversible and reversible temporal transformations
Authors: Isak Karlsson, Jonathan Rebane, Panagiotis Papapetrou, Aristides Gionis
Date: 2018-09-13 00:00:00
Tasks: General Classification, Time Series, Time Series Analysis, Time Series Classification

Time series classification has received great attention over the past decade
with a wide range of methods focusing on predictive performance by exploiting
various types of temporal features. Nonetheless, little emphasis has been
placed on interpretability and explainability. In this paper, we formulate the
novel problem of explainable time series tweaking, where, given a time series
and an opaque classifier that provides a particular classification decision for
the time series, we want to find the minimum number of changes to be performed
to the given time series so that the classifier changes its decision to another
class. We show that the problem is NP-hard, and focus on two instantiations of
the problem, which we refer to as reversible and irreversible time series
tweaking. The classifier under investigation is the random shapelet forest
classifier. Moreover, we propose two algorithmic solutions for the two problems
along with simple optimizations, as well as a baseline solution using the
nearest neighbor classifier. An extensive experimental evaluation on a variety
of real datasets demonstrates the usefulness and effectiveness of our problem
formulation and solutions.
Distinguishing Between Roles of Football Players in Play-by-play Match Event Data
Authors: Bart Aalbers, Jan Van Haaren
Date: 2018-09-13 00:00:00

Over the last few decades, the player recruitment process in professional
football has evolved into a multi-billion industry and has thus become of vital
importance. To gain insights into the general level of their candidate
reinforcements, many professional football clubs have access to extensive video
footage and advanced statistics. However, the question whether a given player
would fit the team's playing style often still remains unanswered. In this
paper, we aim to bridge that gap by proposing a set of 21 player roles and
introducing a method for automatically identifying the most applicable roles
for each player from play-by-play event data collected during matches.
Deterministic Inequalities for Smooth M-estimators
Authors: Arun Kumar Kuchibhotla
Date: 2018-09-13 00:00:00

Ever since the proof of asymptotic normality of maximum likelihood estimator
by Cramer (1946), it has been understood that a basic technique of the Taylor
series expansion suffices for asymptotics of $M$-estimators with
smooth/differentiable loss function. Although the Taylor series expansion is a
purely deterministic tool, the realization that the asymptotic normality
results can also be made deterministic (and so finite sample) received far less
attention. With the advent of big data and high-dimensional statistics, the
need for finite sample results has increased. In this paper, we use the
(well-known) Banach fixed point theorem to derive various deterministic
inequalities that lead to the classical results when studied under randomness.
In addition, we provide applications of these deterministic inequalities for
crossvalidation/subsampling, marginal screening and uniform-in-submodel results
that are very useful for post-selection inference and in the study of
post-regularization estimators. Our results apply to many classical estimators,
in particular, generalized linear models, non-linear regression and cox
proportional hazards model. Extensions to non-smooth and constrained problems
are also discussed.
Defensive Dropout for Hardening Deep Neural Networks under Adversarial Attacks
Authors: Siyue Wang, Xiao Wang, Pu Zhao, Wujie Wen, David Kaeli, Peter Chin, Xue Lin
Date: 2018-09-13 00:00:00

Deep neural networks (DNNs) are known vulnerable to adversarial attacks. That
is, adversarial examples, obtained by adding delicately crafted distortions
onto original legal inputs, can mislead a DNN to classify them as any target
labels. This work provides a solution to hardening DNNs under adversarial
attacks through defensive dropout. Besides using dropout during training for
the best test accuracy, we propose to use dropout also at test time to achieve
strong defense effects. We consider the problem of building robust DNNs as an
attacker-defender two-player game, where the attacker and the defender know
each others' strategies and try to optimize their own strategies towards an
equilibrium. Based on the observations of the effect of test dropout rate on
test accuracy and attack success rate, we propose a defensive dropout algorithm
to determine an optimal test dropout rate given the neural network model and
the attacker's strategy for generating adversarial examples.We also investigate
the mechanism behind the outstanding defense effects achieved by the proposed
defensive dropout. Comparing with stochastic activation pruning (SAP), another
defense method through introducing randomness into the DNN model, we find that
our defensive dropout achieves much larger variances of the gradients, which is
the key for the improved defense effects (much lower attack success rate). For
example, our defensive dropout can reduce the attack success rate from 100% to
13.89% under the currently strongest attack i.e., C&W attack on MNIST dataset.
On the Strength of Character Language Models for Multilingual Named Entity Recognition
Authors: Xiaodong Yu, Stephen Mayhew, Mark Sammons, Dan Roth
Date: 2018-09-13 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Multilingual Named Entity Recognition, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Character-level patterns have been widely used as features in English Named
Entity Recognition (NER) systems. However, to date there has been no direct
investigation of the inherent differences between name and non-name tokens in
text, nor whether this property holds across multiple languages. This paper
analyzes the capabilities of corpus-agnostic Character-level Language Models
(CLMs) in the binary task of distinguishing name tokens from non-name tokens.
We demonstrate that CLMs provide a simple and powerful model for capturing
these differences, identifying named entity tokens in a diverse set of
languages at close to the performance of full NER systems. Moreover, by adding
very simple CLM-based features we can significantly improve the performance of
an off-the-shelf NER system for multiple languages.