pwc-archive/papers-with-abstracts huggingface dataset:
Multi-Information Source Optimization
Authors: Matthias Poloczek, Jialei Wang, Peter I. Frazier
Date: 2016-03-01 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Bayesian Optimization, Reinforcement Learning

We consider Bayesian optimization of an expensive-to-evaluate black-box
objective function, where we also have access to cheaper approximations of the
objective. In general, such approximations arise in applications such as
reinforcement learning, engineering, and the natural sciences, and are subject
to an inherent, unknown bias. This model discrepancy is caused by an inadequate
internal model that deviates from reality and can vary over the domain, making
the utilization of these approximations a non-trivial task.
  We present a novel algorithm that provides a rigorous mathematical treatment
of the uncertainties arising from model discrepancies and noisy observations.
Its optimization decisions rely on a value of information analysis that extends
the Knowledge Gradient factor to the setting of multiple information sources
that vary in cost: each sampling decision maximizes the predicted benefit per
unit cost.
  We conduct an experimental evaluation that demonstrates that the method
consistently outperforms other state-of-the-art techniques: it finds designs of
considerably higher objective value and additionally inflicts less cost in the
exploration process.
When Saliency Meets Sentiment: Understanding How Image Content Invokes Emotion and Sentiment
Authors: Honglin Zheng, Tianlang Chen, Jiebo Luo
Date: 2016-11-14 00:00:00
Tasks: Saliency Detection, Sentiment Analysis, Sentiment Classification

Sentiment analysis is crucial for extracting social signals from social media
content. Due to the prevalence of images in social media, image sentiment
analysis is receiving increasing attention in recent years. However, most
existing systems are black-boxes that do not provide insight on how image
content invokes sentiment and emotion in the viewers. Psychological studies
have confirmed that salient objects in an image often invoke emotions. In this
work, we investigate more fine-grained and more comprehensive interaction
between visual saliency and visual sentiment. In particular, we partition
images in several primary scene-type dimensions, including: open-closed,
natural-manmade, indoor-outdoor, and face-noface. Using state of the art
saliency detection algorithm and sentiment classification algorithm, we examine
how the sentiment of the salient region(s) in an image relates to the overall
sentiment of the image. The experiments on a representative image emotion
dataset have shown interesting correlation between saliency and sentiment in
different scene types and in turn shed light on the mechanism of visual
sentiment evocation.
How to scale distributed deep learning?
Authors: Peter H. Jin, Qiaochu Yuan, Forrest Iandola, Kurt Keutzer
Date: 2016-11-14 00:00:00
Tasks: Deep Learning, General Classification

Training time on large datasets for deep neural networks is the principal
workflow bottleneck in a number of important applications of deep learning,
such as object classification and detection in automatic driver assistance
systems (ADAS). To minimize training time, the training of a deep neural
network must be scaled beyond a single machine to as many machines as possible
by distributing the optimization method used for training. While a number of
approaches have been proposed for distributed stochastic gradient descent
(SGD), at the current time synchronous approaches to distributed SGD appear to
be showing the greatest performance at large scale. Synchronous scaling of SGD
suffers from the need to synchronize all processors on each gradient step and
is not resilient in the face of failing or lagging processors. In asynchronous
approaches using parameter servers, training is slowed by contention to the
parameter server. In this paper we compare the convergence of synchronous and
asynchronous SGD for training a modern ResNet network architecture on the
ImageNet classification problem. We also propose an asynchronous method,
gossiping SGD, that aims to retain the positive features of both systems by
replacing the all-reduce collective operation of synchronous training with a
gossip aggregation algorithm. We find, perhaps counterintuitively, that
asynchronous SGD, including both elastic averaging and gossiping, converges
faster at fewer nodes (up to about 32 nodes), whereas synchronous SGD scales
better to more nodes (up to about 100 nodes).
Earliness-Aware Deep Convolutional Networks for Early Time Series Classification
Authors: Wenlin Wang, Changyou Chen, Wenqi Wang, Piyush Rai, Lawrence Carin
Date: 2016-11-14 00:00:00
Tasks: Classification, Early  Classification, General Classification, Time Series, Time Series Analysis, Time Series Classification

We present Earliness-Aware Deep Convolutional Networks (EA-ConvNets), an
end-to-end deep learning framework, for early classification of time series
data. Unlike most existing methods for early classification of time series
data, that are designed to solve this problem under the assumption of the
availability of a good set of pre-defined (often hand-crafted) features, our
framework can jointly perform feature learning (by learning a deep hierarchy of
\emph{shapelets} capturing the salient characteristics in each time series),
along with a dynamic truncation model to help our deep feature learning
architecture focus on the early parts of each time series. Consequently, our
framework is able to make highly reliable early predictions, outperforming
various state-of-the-art methods for early time series classification, while
also being competitive when compared to the state-of-the-art time series
classification algorithms that work with \emph{fully observed} time series
data. To the best of our knowledge, the proposed framework is the first to
perform data-driven (deep) feature learning in the context of early
classification of time series data. We perform a comprehensive set of
experiments, on several benchmark data sets, which demonstrate that our method
yields significantly better predictions than various state-of-the-art methods
designed for early time series classification. In addition to obtaining high
accuracies, our experiments also show that the learned deep shapelets based
features are also highly interpretable and can help gain better understanding
of the underlying characteristics of time series data.
Lost in Space: Geolocation in Event Data
Authors: Sophie J. Lee, Howard Liu, Michael D. Ward
Date: 2016-11-14 00:00:00
Tasks: Articles, feature selection

Extracting the "correct" location information from text data, i.e.,
determining the place of event, has long been a goal for automated text
processing. To approximate human-like coding schema, we introduce a supervised
machine learning algorithm that classifies each location word to be either
correct or incorrect. We use news articles collected from around the world
(Integrated Crisis Early Warning System [ICEWS] data and Open Event Data
Alliance [OEDA] data) to test our algorithm that consists of two stages. In the
feature selection stage, we extract contextual information from texts, namely,
the N-gram patterns for location words, the frequency of mention, and the
context of the sentences containing location words. In the classification
stage, we use three classifiers to estimate the model parameters in the
training set and then to predict whether a location word in the test set news
articles is the place of the event. The validation results show that our
algorithm improves the accuracy rate of the current geolocation methods of
dictionary approach by as much as 25%.
Splitting matters: how monotone transformation of predictor variables may improve the predictions of decision tree models
Authors: Tal Galili, Isaac Meilijson
Date: 2016-11-14 00:00:00

It is widely believed that the prediction accuracy of decision tree models is
invariant under any strictly monotone transformation of the individual
predictor variables. However, this statement may be false when predicting new
observations with values that were not seen in the training-set and are close
to the location of the split point of a tree rule. The sensitivity of the
prediction error to the split point interpolation is high when the split point
of the tree is estimated based on very few observations, reaching 9%
misclassification error when only 10 observations are used for constructing a
split, and shrinking to 1% when relying on 100 observations. This study
compares the performance of alternative methods for split point interpolation
and concludes that the best choice is taking the mid-point between the two
closest points to the split point of the tree. Furthermore, if the (continuous)
distribution of the predictor variable is known, then using its probability
integral for transforming the variable ("quantile transformation") will reduce
the model's interpolation error by up to about a half on average. Accordingly,
this study provides guidelines for both developers and users of decision tree
models (including bagging and random forest).
3-D Convolutional Neural Networks for Glioblastoma Segmentation
Authors: Darvin Yi, Mu Zhou, Zhao Chen, Olivier Gevaert
Date: 2016-11-14 00:00:00
Tasks: Segmentation

Convolutional Neural Networks (CNN) have emerged as powerful tools for
learning discriminative image features. In this paper, we propose a framework
of 3-D fully CNN models for Glioblastoma segmentation from multi-modality MRI
data. By generalizing CNN models to true 3-D convolutions in learning 3-D tumor
MRI data, the proposed approach utilizes a unique network architecture to
decouple image pixels. Specifically, we design a convolutional layer with
pre-defined Difference- of-Gaussian (DoG) filters to perform true 3-D
convolution incorporating local neighborhood information at each pixel. We then
use three trained convolutional layers that act to decouple voxels from the
initial 3-D convolution. The proposed framework allows identification of
high-level tumor structures on MRI. We evaluate segmentation performance on the
BRATS segmentation dataset with 274 tumor samples. Extensive experimental
results demonstrate encouraging performance of the proposed approach comparing
to the state-of-the-art methods. Our data-driven approach achieves a median
Dice score accuracy of 89% in whole tumor glioblastoma segmentation, revealing
a generalized low-bias possibility to learn from medium-size MRI datasets.
Benchmarking Quantum Hardware for Training of Fully Visible Boltzmann Machines
Authors: Dmytro Korenkevych, Yanbo Xue, Zhengbing Bian, Fabian Chudak, William G. Macready, Jason Rolfe, Evgeny Andriyash
Date: 2016-11-14 00:00:00
Tasks: Benchmarking

Quantum annealing (QA) is a hardware-based heuristic optimization and
sampling method applicable to discrete undirected graphical models. While
similar to simulated annealing, QA relies on quantum, rather than thermal,
effects to explore complex search spaces. For many classes of problems, QA is
known to offer computational advantages over simulated annealing. Here we
report on the ability of recent QA hardware to accelerate training of fully
visible Boltzmann machines. We characterize the sampling distribution of QA
hardware, and show that in many cases, the quantum distributions differ
significantly from classical Boltzmann distributions. In spite of this
difference, training (which seeks to match data and model statistics) using
standard classical gradient updates is still effective. We investigate the use
of QA for seeding Markov chains as an alternative to contrastive divergence
(CD) and persistent contrastive divergence (PCD). Using $k=50$ Gibbs steps, we
show that for problems with high-energy barriers between modes, QA-based seeds
can improve upon chains with CD and PCD initializations. For these hard
problems, QA gradient estimates are more accurate, and allow for faster
learning. Furthermore, and interestingly, even the case of raw QA samples (that
is, $k=0$) achieved similar improvements. We argue that this relates to the
fact that we are training a quantum rather than classical Boltzmann
distribution in this case. The learned parameters give rise to hardware QA
distributions closely approximating classical Boltzmann distributions that are
hard to train with CD/PCD.
Efficient Globally Convergent Stochastic Optimization for Canonical Correlation Analysis
Authors: Weiran Wang, Jialei Wang, Dan Garber, Nathan Srebro
Date: 2016-04-07 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Stochastic Optimization

We study the stochastic optimization of canonical correlation analysis (CCA),
whose objective is nonconvex and does not decouple over training samples.
Although several stochastic gradient based optimization algorithms have been
recently proposed to solve this problem, no global convergence guarantee was
provided by any of them. Inspired by the alternating least squares/power
iterations formulation of CCA, and the shift-and-invert preconditioning method
for PCA, we propose two globally convergent meta-algorithms for CCA, both of
which transform the original problem into sequences of least squares problems
that need only be solved approximately. We instantiate the meta-algorithms with
state-of-the-art SGD methods and obtain time complexities that significantly
improve upon that of previous work. Experimental results demonstrate their
superior performance.
Ranking medical jargon in electronic health record notes by adapted distant supervision
Authors: Jinying Chen, Abhyuday N. Jagannatha, Samah J. Jarad, Hong Yu
Date: 2016-11-14 00:00:00

Objective: Allowing patients to access their own electronic health record
(EHR) notes through online patient portals has the potential to improve
patient-centered care. However, medical jargon, which abounds in EHR notes, has
been shown to be a barrier for patient EHR comprehension. Existing knowledge
bases that link medical jargon to lay terms or definitions play an important
role in alleviating this problem but have low coverage of medical jargon in
EHRs. We developed a data-driven approach that mines EHRs to identify and rank
medical jargon based on its importance to patients, to support the building of
EHR-centric lay language resources.
  Methods: We developed an innovative adapted distant supervision (ADS) model
based on support vector machines to rank medical jargon from EHRs. For distant
supervision, we utilized the open-access, collaborative consumer health
vocabulary, a large, publicly available resource that links lay terms to
medical jargon. We explored both knowledge-based features from the Unified
Medical Language System and distributed word representations learned from
unlabeled large corpora. We evaluated the ADS model using physician-identified
important medical terms.
  Results: Our ADS model significantly surpassed two state-of-the-art automatic
term recognition methods, TF*IDF and C-Value, yielding 0.810 ROC-AUC versus
0.710 and 0.667, respectively. Our model identified 10K important medical
jargon terms after ranking over 100K candidate terms mined from over 7,500 EHR
narratives.
  Conclusion: Our work is an important step towards enriching lexical resources
that link medical jargon to lay terms/definitions to support patient EHR
comprehension. The identified medical jargon terms and their rankings are
available upon request.
Practical Secure Aggregation for Federated Learning on User-Held Data
Authors: Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, Karn Seth
Date: 2016-11-14 00:00:00
Tasks: Federated Learning

Secure Aggregation protocols allow a collection of mutually distrust parties,
each holding a private value, to collaboratively compute the sum of those
values without revealing the values themselves. We consider training a deep
neural network in the Federated Learning model, using distributed stochastic
gradient descent across user-held training data on mobile devices, wherein
Secure Aggregation protects each user's model gradient. We design a novel,
communication-efficient Secure Aggregation protocol for high-dimensional data
that tolerates up to 1/3 users failing to complete the protocol. For 16-bit
input values, our protocol offers 1.73x communication expansion for $2^{10}$
users and $2^{20}$-dimensional vectors, and 1.98x expansion for $2^{14}$ users
and $2^{24}$ dimensional vectors.
Analysis of the Entropy-guided Switching Trimmed Mean Deviation-based Anisotropic Diffusion filter
Authors: Uche A. Nnolim
Date: 2016-04-21 00:00:00

This report describes the experimental analysis of a proposed switching
filter-anisotropic diffusion hybrid for the filtering of the fixed value (salt
and pepper) impulse noise (FVIN). The filter works well at both low and high
noise densities though it was specifically designed for high noise density
levels. The filter combines the switching mechanism of decision-based filters
and the partial differential equation-based formulation to yield a powerful
system capable of recovering the image signals at very high noise levels.
Experimental results indicate that the filter surpasses other filters,
especially at very high noise levels. Additionally, its adaptive nature ensures
that the performance is guided by the metrics obtained from the noisy input
image. The filter algorithm is of both global and local nature, where the
former is chosen to reduce computation time and complexity, while the latter is
used for best results.
An Evaluation of Information Sharing Parking Guidance Policies Using a Bayesian Approach
Authors: Xinyi Wu, Kartik Balkumar, Qi Luo, Robert Hampshire, Romesh Saigal
Date: 2016-11-14 00:00:00
Tasks: Management, Parking Space Occupancy

Real-time parking occupancy information is critical for a parking management
system to facilitate drivers to park more efficiently. Recent advances in
connected and automated vehicle technologies enable sensor-equipped cars (probe
cars) to detect and broadcast available parking spaces when driving through
parking lots. In this paper, we evaluate the impact of market penetration of
probe cars on the system performance, and investigate different parking
guidance policies to improve the data acquisition process. We adopt a
simulation-based approach to impose four policies on an off- street parking lot
influencing the behavior of probe cars to park in assigned parking spaces. This
in turn effects the scanning route and the parking space occupancy estimations.
The last policy we propose is a near-optimal guidance strategy that maximizes
the information gain of posteriors. The results suggest that an efficient
information gathering policy can compensate for low penetration of connected
and automated vehicles. We also highlight the policy trade-off that occur while
attempting to maximize information gain through explorations and improve
assignment accuracy through exploitations. Our results can assist urban policy
makers in designing and managing smart parking systems.
On numerical approximation schemes for expectation propagation
Authors: Alexis Roche
Date: 2016-11-14 00:00:00

Several numerical approximation strategies for the expectation-propagation
algorithm are studied in the context of large-scale learning: the Laplace
method, a faster variant of it, Gaussian quadrature, and a deterministic
version of variational sampling (i.e., combining quadrature with variational
approximation). Experiments in training linear binary classifiers show that the
expectation-propagation algorithm converges best using variational sampling,
while it also converges well using Laplace-style methods with smooth factors
but tends to be unstable with non-differentiable ones. Gaussian quadrature
yields unstable behavior or convergence to a sub-optimal solution in most
experiments.
Automatic discovery of discriminative parts as a quadratic assignment problem
Authors: Ronan Sicre, Julien Rabin, Yannis Avrithis, Teddy Furon, Frederic Jurie
Date: 2016-11-14 00:00:00
Tasks: General Classification, image-classification, Image Classification

Part-based image classification consists in representing categories by small
sets of discriminative parts upon which a representation of the images is
built. This paper addresses the question of how to automatically learn such
parts from a set of labeled training images. The training of parts is cast as a
quadratic assignment problem in which optimal correspondences between image
regions and parts are automatically learned. The paper analyses different
assignment strategies and thoroughly evaluates them on two public datasets:
Willow actions and MIT 67 scenes. State-of-the art results are obtained on
these datasets.
Deep Convolutional Neural Network Design Patterns
Authors: Leslie N. Smith, Nicholay Topin
Date: 2016-11-02 00:00:00
Tasks: Deep Learning

Recent research in the deep learning field has produced a plethora of new
architectures. At the same time, a growing number of groups are applying deep
learning to new applications. Some of these groups are likely to be composed of
inexperienced deep learning practitioners who are baffled by the dizzying array
of architecture choices and therefore opt to use an older architecture (i.e.,
Alexnet). Here we attempt to bridge this gap by mining the collective knowledge
contained in recent deep learning research to discover underlying principles
for designing neural network architectures. In addition, we describe several
architectural innovations, including Fractal of FractalNet network, Stagewise
Boosting Networks, and Taylor Series Networks (our Caffe code and prototxt
files is available at https://github.com/iPhysicist/CNNDesignPatterns). We hope
others are inspired to build on our preliminary work.
Feature Engineering and Ensemble Modeling for Paper Acceptance Rank Prediction
Authors: Yujie Qian, Yinpeng Dong, Ye Ma, Hailong Jin, Juanzi Li
Date: 2016-11-14 00:00:00
Tasks: Dimensionality Reduction, Feature Engineering

Measuring research impact and ranking academic achievement are important and
challenging problems. Having an objective picture of research institution is
particularly valuable for students, parents and funding agencies, and also
attracts attention from government and industry. KDD Cup 2016 proposes the
paper acceptance rank prediction task, in which the participants are asked to
rank the importance of institutions based on predicting how many of their
papers will be accepted at the 8 top conferences in computer science. In our
work, we adopt a three-step feature engineering method, including basic
features definition, finding similar conferences to enhance the feature set,
and dimension reduction using PCA. We propose three ranking models and the
ensemble methods for combining such models. Our experiment verifies the
effectiveness of our approach. In KDD Cup 2016, we achieved the overall rank of
the 2nd place.
On Explore-Then-Commit Strategies
Authors: Aurélien Garivier, Emilie Kaufmann, Tor Lattimore
Date: 2016-05-29 00:00:00
Proceeding: NeurIPS 2016 12

We study the problem of minimising regret in two-armed bandit problems with
Gaussian rewards. Our objective is to use this simple setting to illustrate
that strategies based on an exploration phase (up to a stopping time) followed
by exploitation are necessarily suboptimal. The results hold regardless of
whether or not the difference in means between the two arms is known. Besides
the main message, we also refine existing deviation inequalities, which allow
us to design fully sequential strategies with finite-time regret guarantees
that are (a) asymptotically optimal as the horizon grows and (b) order-optimal
in the minimax sense. Furthermore we provide empirical evidence that the theory
also holds in practice and discuss extensions to non-gaussian and
multiple-armed case.