pwc-archive/papers-with-abstracts huggingface dataset:
Applying an Ensemble Learning Method for Improving Multi-label Classification Performance
Authors: Amirreza Mahdavi-Shahri, Mahboobeh Houshmand, Mahdi Yaghoobi, Mehrdad Jalali
Date: 2018-01-07 00:00:00
Tasks: Classification, Ensemble Learning, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

In recent years, multi-label classification problem has become a
controversial issue. In this kind of classification, each sample is associated
with a set of class labels. Ensemble approaches are supervised learning
algorithms in which an operator takes a number of learning algorithms, namely
base-level algorithms and combines their outcomes to make an estimation. The
simplest form of ensemble learning is to train the base-level algorithms on
random subsets of data and then let them vote for the most popular
classifications or average the predictions of the base-level algorithms. In
this study, an ensemble learning method is proposed for improving multi-label
classification evaluation criteria. We have compared our method with well-known
base-level algorithms on some data sets. Experiment results show the proposed
approach outperforms the base well-known classifiers for the multi-label
classification problem.
Covariant Compositional Networks For Learning Graphs
Authors: Risi Kondor, Hy Truong Son, Horace Pan, Brandon Anderson, Shubhendu Trivedi
Date: 2018-01-07 00:00:00
Proceeding: ICLR 2018 1
Tasks: Graph Learning

Most existing neural networks for learning graphs address permutation
invariance by conceiving of the network as a message passing scheme, where each
node sums the feature vectors coming from its neighbors. We argue that this
imposes a limitation on their representation power, and instead propose a new
general architecture for representing objects consisting of a hierarchy of
parts, which we call Covariant Compositional Networks (CCNs). Here, covariance
means that the activation of each neuron must transform in a specific way under
permutations, similarly to steerability in CNNs. We achieve covariance by
making each activation transform according to a tensor representation of the
permutation group, and derive the corresponding tensor aggregation rules that
each neuron must implement. Experiments show that CCNs can outperform competing
methods on standard graph learning benchmarks.
Deep Bidirectional and Unidirectional LSTM Recurrent Neural Network for Network-wide Traffic Speed Prediction
Authors: Zhiyong Cui, Ruimin Ke, Ziyuan Pu, Yinhai Wang
Date: 2018-01-07 00:00:00
Tasks: Missing Values, Time Series, Time Series Analysis

Short-term traffic forecasting based on deep learning methods, especially long short-term memory (LSTM) neural networks, has received much attention in recent years. However, the potential of deep learning methods in traffic forecasting has not yet fully been exploited in terms of the depth of the model architecture, the spatial scale of the prediction area, and the predictive power of spatial-temporal data. In this paper, a deep stacked bidirectional and unidirectional LSTM (SBU- LSTM) neural network architecture is proposed, which considers both forward and backward dependencies in time series data, to predict network-wide traffic speed. A bidirectional LSTM (BDLSM) layer is exploited to capture spatial features and bidirectional temporal dependencies from historical data. To the best of our knowledge, this is the first time that BDLSTMs have been applied as building blocks for a deep architecture model to measure the backward dependency of traffic data for prediction. The proposed model can handle missing values in input data by using a masking mechanism. Further, this scalable model can predict traffic speed for both freeway and complex urban traffic networks. Comparisons with other classical and state-of-the-art models indicate that the proposed SBU-LSTM neural network achieves superior prediction performance for the whole traffic network in both accuracy and robustness.
Architecture Based Classification of Leaf Images
Authors: Mahmoud Sadeghi, Ali Zakerolhosseini, Ali Sonboli
Date: 2018-01-07 00:00:00
Tasks: Classification, General Classification

Plant classification and identification has so far been an important and
difficult task. In this paper, an efficient and systematic approach for
extracting the leaf architecture characters from captured digital images is
proposed. The input image is first pre-processed in five steps to be prepared
for feature extraction. In the second stage, methods for extracting different
architectural features are studied using various mathematical and computational
methods. Also, the classification rules for mapping the calculated values of
each feature to semantic botanical terms in proposed. Compared with previous
studies, the proposed method combines extracted features of an image with
specific knowledge of leaf architecture in the domain of botany to provide a
comprehensive framework for both computer engineers and botanist. Finally,
Based on the proposed method, experiments on the classification of the
ImagerCLEF 2012 dataset has been performed with promising results.
Gradient Diversity: a Key Ingredient for Scalable Distributed Learning
Authors: Dong Yin, Ashwin Pananjady, Max Lam, Dimitris Papailiopoulos, Kannan Ramchandran, Peter Bartlett
Date: 2017-06-18 00:00:00
Tasks: Diversity, Quantization

It has been experimentally observed that distributed implementations of
mini-batch stochastic gradient descent (SGD) algorithms exhibit speedup
saturation and decaying generalization ability beyond a particular batch-size.
In this work, we present an analysis hinting that high similarity between
concurrently processed gradients may be a cause of this performance
degradation. We introduce the notion of gradient diversity that measures the
dissimilarity between concurrent gradient updates, and show its key role in the
performance of mini-batch SGD. We prove that on problems with high gradient
diversity, mini-batch SGD is amenable to better speedups, while maintaining the
generalization performance of serial (one sample) SGD. We further establish
lower bounds on convergence where mini-batch SGD slows down beyond a particular
batch-size, solely due to the lack of gradient diversity. We provide
experimental evidence indicating the key role of gradient diversity in
distributed learning, and discuss how heuristics like dropout, Langevin
dynamics, and quantization can improve it.
Convex Relaxations for Pose Graph Optimization with Outliers
Authors: Luca Carlone, Giuseppe C. Calafiore
Date: 2018-01-07 00:00:00

Pose Graph Optimization involves the estimation of a set of poses from
pairwise measurements and provides a formalization for many problems arising in
mobile robotics and geometric computer vision. In this paper, we consider the
case in which a subset of the measurements fed to pose graph optimization is
spurious. Our first contribution is to develop robust estimators that can cope
with heavy-tailed measurement noise, hence increasing robustness to the
presence of outliers. Since the resulting estimators require solving nonconvex
optimization problems, we further develop convex relaxations that approximately
solve those problems via semidefinite programming. We then provide conditions
under which the proposed relaxations are exact. Contrarily to existing
approaches, our convex relaxations do not rely on the availability of an
initial guess for the unknown poses, hence they are more suitable for setups in
which such guess is not available (e.g., multi-robot localization, recovery
after localization failure). We tested the proposed techniques in extensive
simulations, and we show that some of the proposed relaxations are indeed tight
(i.e., they solve the original nonconvex problem 10 exactly) and ensure
accurate estimation in the face of a large number of outliers.
Representation Learning with Autoencoders for Electronic Health Records: A Comparative Study
Authors: Najibesadat Sadati, Milad Zafar Nezhad, Ratna Babu Chinnam, Dongxiao Zhu
Date: 2018-01-06 00:00:00
Tasks: Representation Learning, Small Data Image Classification

Increasing volume of Electronic Health Records (EHR) in recent years provides great opportunities for data scientists to collaborate on different aspects of healthcare research by applying advanced analytics to these EHR clinical data. A key requirement however is obtaining meaningful insights from high dimensional, sparse and complex clinical data. Data science approaches typically address this challenge by performing feature learning in order to build more reliable and informative feature representations from clinical data followed by supervised learning. In this paper, we propose a predictive modeling approach based on deep learning based feature representations and word embedding techniques. Our method uses different deep architectures (stacked sparse autoencoders, deep belief network, adversarial autoencoders and variational autoencoders) for feature representation in higher-level abstraction to obtain effective and robust features from EHRs, and then build prediction models on top of them. Our approach is particularly useful when the unlabeled data is abundant whereas labeled data is scarce. We investigate the performance of representation learning through a supervised learning approach. Our focus is to present a comparative study to evaluate the performance of different deep architectures through supervised learning and provide insights in the choice of deep feature representation techniques. Our experiments demonstrate that for small data sets, stacked sparse autoencoder demonstrates a superior generality performance in prediction due to sparsity regularization whereas variational autoencoders outperform the competing approaches for large data sets due to its capability of learning the representation distribution.
Improving utility of brain tumor confocal laser endomicroscopy: objective value assessment and diagnostic frame detection with convolutional neural networks
Authors: Mohammadhassan Izadyyazdanabadi, Evgenii Belykh, Nikolay Martirosyan, Jennifer Eschbacher, Peter Nakaji, Yezhou Yang, Mark C. Preul
Date: 2018-01-06 00:00:00
Tasks: Diagnostic, Specificity

Confocal laser endomicroscopy (CLE), although capable of obtaining images at
cellular resolution during surgery of brain tumors in real time, creates as
many non-diagnostic as diagnostic images. Non-useful images are often distorted
due to relative motion between probe and brain or blood artifacts. Many images,
however, simply lack diagnostic features immediately informative to the
physician. Examining all the hundreds or thousands of images from a single case
to discriminate diagnostic images from nondiagnostic ones can be tedious.
Providing a real-time diagnostic value assessment of images (fast enough to be
used during the surgical acquisition process and accurate enough for the
pathologist to rely on) to automatically detect diagnostic frames would
streamline the analysis of images and filter useful images for the
pathologist/surgeon. We sought to automatically classify images as diagnostic
or non-diagnostic. AlexNet, a deep-learning architecture, was used in a 4-fold
cross validation manner. Our dataset includes 16,795 images (8572 nondiagnostic
and 8223 diagnostic) from 74 CLE-aided brain tumor surgery patients. The ground
truth for all the images is provided by the pathologist. Average model accuracy
on test data was 91% overall (90.79 % accuracy, 90.94 % sensitivity and 90.87 %
specificity). To evaluate the model reliability we also performed receiver
operating characteristic (ROC) analysis yielding 0.958 average for the area
under ROC curve (AUC). These results demonstrate that a deeply trained AlexNet
network can achieve a model that reliably and quickly recognizes diagnostic CLE
images.
Perturbative Black Box Variational Inference
Authors: Robert Bamler, Cheng Zhang, Manfred Opper, Stephan Mandt
Date: 2017-09-21 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Gaussian Processes, Variational Inference

Black box variational inference (BBVI) with reparameterization gradients
triggered the exploration of divergence measures other than the
Kullback-Leibler (KL) divergence, such as alpha divergences. In this paper, we
view BBVI with generalized divergences as a form of estimating the marginal
likelihood via biased importance sampling. The choice of divergence determines
a bias-variance trade-off between the tightness of a bound on the marginal
likelihood (low bias) and the variance of its gradient estimators. Drawing on
variational perturbation theory of statistical physics, we use these insights
to construct a family of new variational bounds. Enumerated by an odd integer
order $K$, this family captures the standard KL bound for $K=1$, and converges
to the exact marginal likelihood as $K\to\infty$. Compared to
alpha-divergences, our reparameterization gradients have a lower variance. We
show in experiments on Gaussian Processes and Variational Autoencoders that the
new bounds are more mass covering, and that the resulting posterior covariances
are closer to the true posterior and lead to higher likelihoods on held-out
data.
Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples
Authors: Haw-Shiuan Chang, Erik Learned-Miller, Andrew McCallum
Date: 2017-04-24 00:00:00
Proceeding: NeurIPS 2017 12

Self-paced learning and hard example mining re-weight training instances to
improve learning accuracy. This paper presents two improved alternatives based
on lightweight estimates of sample uncertainty in stochastic gradient descent
(SGD): the variance in predicted probability of the correct class across
iterations of mini-batch SGD, and the proximity of the correct class
probability to the decision threshold. Extensive experimental results on six
datasets show that our methods reliably improve accuracy in various network
architectures, including additional gains on top of other popular training
techniques, such as residual learning, momentum, ADAM, batch normalization,
dropout, and distillation.
On the inherent competition between valid and spurious inductive inferences in Boolean data
Authors: M. Andrecut
Date: 2018-01-06 00:00:00
Tasks: valid

Inductive inference is the process of extracting general rules from specific
observations. This problem also arises in the analysis of biological networks,
such as genetic regulatory networks, where the interactions are complex and the
observations are incomplete. A typical task in these problems is to extract
general interaction rules as combinations of Boolean covariates, that explain a
measured response variable. The inductive inference process can be considered
as an incompletely specified Boolean function synthesis problem. This
incompleteness of the problem will also generate spurious inferences, which are
a serious threat to valid inductive inference rules. Using random Boolean data
as a null model, here we attempt to measure the competition between valid and
spurious inductive inference rules from a given data set. We formulate two
greedy search algorithms, which synthesize a given Boolean response variable in
a sparse disjunct normal form, and respectively a sparse generalized algebraic
normal form of the variables from the observation data, and we evaluate
numerically their performance.
Explorations in an English Poetry Corpus: A Neurocognitive Poetics Perspective
Authors: Arthur M. Jacobs
Date: 2018-01-06 00:00:00
Tasks: Diversity, Sentiment Analysis

This paper describes a corpus of about 3000 English literary texts with about
250 million words extracted from the Gutenberg project that span a range of
genres from both fiction and non-fiction written by more than 130 authors
(e.g., Darwin, Dickens, Shakespeare). Quantitative Narrative Analysis (QNA) is
used to explore a cleaned subcorpus, the Gutenberg English Poetry Corpus (GEPC)
which comprises over 100 poetic texts with around 2 million words from about 50
authors (e.g., Keats, Joyce, Wordsworth). Some exemplary QNA studies show
author similarities based on latent semantic analysis, significant topics for
each author or various text-analytic metrics for George Eliot's poem 'How Lisa
Loved the King' and James Joyce's 'Chamber Music', concerning e.g. lexical
diversity or sentiment analysis. The GEPC is particularly suited for research
in Digital Humanities, Natural Language Processing or Neurocognitive Poetics,
e.g. as training and test corpus, or for stimulus development and control.
An Ontology for Satellite Databases
Authors: Robert J. Rovetto
Date: 2018-01-06 00:00:00

This paper demonstrates the development of ontology for satellite databases.
First, I create a computational ontology for the Union of Concerned Scientists
(UCS) Satellite Database (UCSSD for short), called the UCS Satellite Ontology
(or UCSSO). Second, in developing UCSSO I show that The Space Situational
Awareness Ontology (SSAO) (Rovetto and Kelso 2016)--an existing space domain
reference ontology--and related ontology work by the author (Rovetto 2015,
2016) can be used either (i) with a database-specific local ontology such as
UCSSO, or (ii) in its stead. In case (i), local ontologies such as UCSSO can
reuse SSAO terms, perform term mappings, or extend it. In case (ii), the
author's orbital space ontology work, such as the SSAO, is usable by the UCSSD
and organizations with other space object catalogs, as a reference ontology
suite providing a common semantically-rich domain model. The SSAO, UCSSO, and
the broader Orbital Space Environment Domain Ontology project is online at
http://purl.org/space-ontology and GitHub. This ontology effort aims, in part,
to provide accurate formal representations of the domain for various
applications. Ontology engineering has the potential to facilitate the sharing
and integration of satellite data from federated databases and sensors for
safer spaceflight.
A Unified Primal Dual Active Set Algorithm for Nonconvex Sparse Recovery
Authors: Jian Huang, Yuling Jiao, Bangti Jin, Jin Liu, Xiliang Lu, Can Yang
Date: 2013-10-04 00:00:00

In this paper, we consider the problem of recovering a sparse signal based on
penalized least squares formulations. We develop a novel algorithm of
primal-dual active set type for a class of nonconvex sparsity-promoting
penalties, including $\ell^0$, bridge, smoothly clipped absolute deviation,
capped $\ell^1$ and minimax concavity penalty. First we establish the existence
of a global minimizer for the related optimization problems. Then we derive a
novel necessary optimality condition for the global minimizer using the
associated thresholding operator. The solutions to the optimality system are
coordinate-wise minimizers, and under minor conditions, they are also local
minimizers. Upon introducing the dual variable, the active set can be
determined using the primal and dual variables together. Further, this relation
lends itself to an iterative algorithm of active set type which at each step
involves first updating the primal variable only on the active set and then
updating the dual variable explicitly. When combined with a continuation
strategy on the regularization parameter, the primal dual active set method is
shown to converge globally to the underlying regression target under certain
regularity conditions. Extensive numerical experiments with both simulated and
real data demonstrate its superior performance in efficiency and accuracy
compared with the existing sparse recovery methods.
Batched High-dimensional Bayesian Optimization via Structural Kernel Learning
Authors: Zi Wang, Chengtao Li, Stefanie Jegelka, Pushmeet Kohli
Date: 2017-03-06 00:00:00
Proceeding: ICML 2017 8
Tasks: Bayesian Optimization, Point Processes, Vocal Bursts Intensity Prediction

Optimization of high-dimensional black-box functions is an extremely
challenging problem. While Bayesian optimization has emerged as a popular
approach for optimizing black-box functions, its applicability has been limited
to low-dimensional problems due to its computational and statistical challenges
arising from high-dimensional settings. In this paper, we propose to tackle
these challenges by (1) assuming a latent additive structure in the function
and inferring it properly for more efficient and effective BO, and (2)
performing multiple evaluations in parallel to reduce the number of iterations
required by the method. Our novel approach learns the latent structure with
Gibbs sampling and constructs batched queries using determinantal point
processes. Experimental validations on both synthetic and real-world functions
demonstrate that the proposed method outperforms the existing state-of-the-art
approaches.
Translating Neuralese
Authors: Jacob Andreas, Anca Dragan, Dan Klein
Date: 2017-04-23 00:00:00
Proceeding: ACL 2017 7
Tasks: Machine Translation, Translation

Several approaches have recently been proposed for learning decentralized
deep multiagent policies that coordinate via a differentiable communication
channel. While these policies are effective for many tasks, interpretation of
their induced communication strategies has remained a challenge. Here we
propose to interpret agents' messages by translating them. Unlike in typical
machine translation problems, we have no parallel data to learn from. Instead
we develop a translation model based on the insight that agent messages and
natural language strings mean the same thing if they induce the same belief
about the world in a listener. We present theoretical guarantees and empirical
evidence that our approach preserves both the semantics and pragmatics of
messages by ensuring that players communicating through a translation layer do
not suffer a substantial loss in reward relative to players with a common
language.
ReMotENet: Efficient Relevant Motion Event Detection for Large-scale Home Surveillance Videos
Authors: Ruichi Yu, Hongcheng Wang, Larry S. Davis
Date: 2018-01-06 00:00:00
Tasks: CPU, Event Detection, GPU, object-detection, Object Detection

This paper addresses the problem of detecting relevant motion caused by
objects of interest (e.g., person and vehicles) in large scale home
surveillance videos. The traditional method usually consists of two separate
steps, i.e., detecting moving objects with background subtraction running on
the camera, and filtering out nuisance motion events (e.g., trees, cloud,
shadow, rain/snow, flag) with deep learning based object detection and tracking
running on cloud. The method is extremely slow and therefore not cost
effective, and does not fully leverage the spatial-temporal redundancies with a
pre-trained off-the-shelf object detector. To dramatically speedup relevant
motion event detection and improve its performance, we propose a novel network
for relevant motion event detection, ReMotENet, which is a unified, end-to-end
data-driven method using spatial-temporal attention-based 3D ConvNets to
jointly model the appearance and motion of objects-of-interest in a video.
ReMotENet parses an entire video clip in one forward pass of a neural network
to achieve significant speedup. Meanwhile, it exploits the properties of home
surveillance videos, e.g., relevant motion is sparse both spatially and
temporally, and enhances 3D ConvNets with a spatial-temporal attention model
and reference-frame subtraction to encourage the network to focus on the
relevant moving objects. Experiments demonstrate that our method can achieve
comparable or event better performance than the object detection based method
but with three to four orders of magnitude speedup (up to 20k times) on GPU
devices. Our network is efficient, compact and light-weight. It can detect
relevant motion on a 15s surveillance video clip within 4-8 milliseconds on a
GPU and a fraction of second (0.17-0.39) on a CPU with a model size of less
than 1MB.