pwc-archive/papers-with-abstracts huggingface dataset:
Rate-Distortion Bounds on Bayes Risk in Supervised Learning
Authors: Matthew Nokleby, Ahmad Beirami, Robert Calderbank
Date: 2016-05-08 00:00:00

We present an information-theoretic framework for bounding the number of
labeled samples needed to train a classifier in a parametric Bayesian setting.
We derive bounds on the average $L_p$ distance between the learned classifier
and the true maximum a posteriori classifier, which are well-established
surrogates for the excess classification error due to imperfect learning. We
provide lower and upper bounds on the rate-distortion function, using $L_p$
loss as the distortion measure, of a maximum a priori classifier in terms of
the differential entropy of the posterior distribution and a quantity called
the interpolation dimension, which characterizes the complexity of the
parametric distribution family. In addition to expressing the information
content of a classifier in terms of lossy compression, the rate-distortion
function also expresses the minimum number of bits a learning machine needs to
extract from training data to learn a classifier to within a specified $L_p$
tolerance. We use results from universal source coding to express the
information content in the training data in terms of the Fisher information of
the parametric family and the number of training samples available. The result
is a framework for computing lower bounds on the Bayes $L_p$ risk. This
framework complements the well-known probably approximately correct (PAC)
framework, which provides minimax risk bounds involving the Vapnik-Chervonenkis
dimension or Rademacher complexity. Whereas the PAC framework provides upper
bounds the risk for the worst-case data distribution, the proposed
rate-distortion framework lower bounds the risk averaged over the data
distribution. We evaluate the bounds for a variety of data models, including
categorical, multinomial, and Gaussian models. In each case the bounds are
provably tight orderwise, and in two cases we prove that the bounds are tight
up to multiplicative constants.
Nonparametric independence testing via mutual information
Authors: Thomas B. Berrett, Richard J. Samworth
Date: 2017-11-17 00:00:00

We propose a test of independence of two multivariate random vectors, given a
sample from the underlying population. Our approach, which we call MINT, is
based on the estimation of mutual information, whose decomposition into joint
and marginal entropies facilitates the use of recently-developed efficient
entropy estimators derived from nearest neighbour distances. The proposed
critical values, which may be obtained from simulation (in the case where one
marginal is known) or resampling, guarantee that the test has nominal size, and
we provide local power analyses, uniformly over classes of densities whose
mutual information satisfies a lower bound. Our ideas may be extended to
provide a new goodness-of-fit tests of normal linear models based on assessing
the independence of our vector of covariates and an appropriately-defined
notion of an error vector. The theory is supported by numerical studies on both
simulated and real data.
Spatio-Temporal Data Mining: A Survey of Problems and Methods
Authors: Gowtham Atluri, Anuj Karpatne, Vipin Kumar
Date: 2017-11-13 00:00:00
Tasks: Anomaly Detection, Change Detection, Clustering, Epidemiology, Survey

Large volumes of spatio-temporal data are increasingly collected and studied
in diverse domains including, climate science, social sciences, neuroscience,
epidemiology, transportation, mobile health, and Earth sciences.
Spatio-temporal data differs from relational data for which computational
approaches are developed in the data mining community for multiple decades, in
that both spatial and temporal attributes are available in addition to the
actual measurements/attributes. The presence of these attributes introduces
additional challenges that needs to be dealt with. Approaches for mining
spatio-temporal data have been studied for over a decade in the data mining
community. In this article we present a broad survey of this relatively young
field of spatio-temporal data mining. We discuss different types of
spatio-temporal data and the relevant data mining questions that arise in the
context of analyzing each of these datasets. Based on the nature of the data
mining problem studied, we classify literature on spatio-temporal data mining
into six major categories: clustering, predictive learning, change detection,
frequent pattern mining, anomaly detection, and relationship mining. We discuss
the various forms of spatio-temporal data mining problems in each of these
categories.
Segmenting Brain Tumors with Symmetry
Authors: Hejia Zhang, Xia Zhu, Theodore L. Willke
Date: 2017-11-17 00:00:00
Tasks: Brain Tumor Segmentation, Medical Image Segmentation, Segmentation, Tumor Segmentation

We explore encoding brain symmetry into a neural network for a brain tumor
segmentation task. A healthy human brain is symmetric at a high level of
abstraction, and the high-level asymmetric parts are more likely to be tumor
regions. Paying more attention to asymmetries has the potential to boost the
performance in brain tumor segmentation. We propose a method to encode brain
symmetry into existing neural networks and apply the method to a
state-of-the-art neural network for medical imaging segmentation. We evaluate
our symmetry-encoded network on the dataset from a brain tumor segmentation
challenge and verify that the new model extracts information in the training
images more efficiently than the original model.
Bayesian Inference over the Stiefel Manifold via the Givens Representation
Authors: Arya A Pourzanjani, Richard M Jiang, Brian Mitchell, Paul J. Atzberger, Linda R. Petzold
Date: 2017-10-25 00:00:00
Tasks: Bayesian Inference, Dimensionality Reduction

We introduce an approach based on the Givens representation for posterior inference in statistical models with orthogonal matrix parameters, such as factor models and probabilistic principal component analysis (PPCA). We show how the Givens representation can be used to develop practical methods for transforming densities over the Stiefel manifold into densities over subsets of Euclidean space. We show how to deal with issues arising from the topology of the Stiefel manifold and how to inexpensively compute the change-of-measure terms. We introduce an auxiliary parameter approach that limits the impact of topological issues. We provide both analysis of our methods and numerical examples demonstrating the effectiveness of the approach. We also discuss how our Givens representation can be used to define general classes of distributions over the space of orthogonal matrices. We then give demonstrations on several examples showing how the Givens approach performs in practice in comparison with other methods.
Find Your Own Way: Weakly-Supervised Segmentation of Path Proposals for Urban Autonomy
Authors: Dan Barnes, Will Maddern, Ingmar Posner
Date: 2016-10-05 00:00:00
Tasks: Autonomous Driving, Segmentation, Semantic Segmentation, Weakly supervised segmentation

We present a weakly-supervised approach to segmenting proposed drivable paths
in images with the goal of autonomous driving in complex urban environments.
Using recorded routes from a data collection vehicle, our proposed method
generates vast quantities of labelled images containing proposed paths and
obstacles without requiring manual annotation, which we then use to train a
deep semantic segmentation network. With the trained network we can segment
proposed paths and obstacles at run-time using a vehicle equipped with only a
monocular camera without relying on explicit modelling of road or lane
markings. We evaluate our method on the large-scale KITTI and Oxford RobotCar
datasets and demonstrate reliable path proposal and obstacle segmentation in a
wide variety of environments under a range of lighting, weather and traffic
conditions. We illustrate how the method can generalise to multiple path
proposals at intersections and outline plans to incorporate the system into a
framework for autonomous urban driving.
Depth Assisted Full Resolution Network for Single Image-based View Synthesis
Authors: Xiaodong Cun, Feng Xu, Chi-Man Pun, Hao Gao
Date: 2017-11-17 00:00:00
Tasks: Depth Estimation

Researches in novel viewpoint synthesis majorly focus on interpolation from
multi-view input images. In this paper, we focus on a more challenging and
ill-posed problem that is to synthesize novel viewpoints from one single input
image. To achieve this goal, we propose a novel deep learning-based technique.
We design a full resolution network that extracts local image features with the
same resolution of the input, which contributes to derive high resolution and
prevent blurry artifacts in the final synthesized images. We also involve a
pre-trained depth estimation network into our system, and thus 3D information
is able to be utilized to infer the flow field between the input and the target
image. Since the depth network is trained by depth order information between
arbitrary pairs of points in the scene, global image features are also involved
into our system. Finally, a synthesis layer is used to not only warp the
observed pixels to the desired positions but also hallucinate the missing
pixels with recorded pixels. Experiments show that our technique performs well
on images of various scenes, and outperforms the state-of-the-art techniques.
VAE Learning via Stein Variational Gradient Descent
Authors: Yunchen Pu, Zhe Gan, Ricardo Henao, Chunyuan Li, Shaobo Han, Lawrence Carin
Date: 2017-04-18 00:00:00
Proceeding: NeurIPS 2017 12

A new method for learning variational autoencoders (VAEs) is developed, based
on Stein variational gradient descent. A key advantage of this approach is that
one need not make parametric assumptions about the form of the encoder
distribution. Performance is further enhanced by integrating the proposed
encoder with importance sampling. Excellent performance is demonstrated across
multiple unsupervised and semi-supervised problems, including semi-supervised
analysis of the ImageNet data, demonstrating the scalability of the model to
large datasets.
Superpixels Based Segmentation and SVM Based Classification Method to Distinguish Five Diseases from Normal Regions in Wireless Capsule Endoscopy
Authors: Omid Haji Maghsoudi
Date: 2017-11-17 00:00:00
Tasks: Clustering, General Classification, Semantic Segmentation, Specificity, Superpixels

Wireless Capsule Endoscopy (WCE) is relatively a new technology to examine
the entire GI trace. During an examination, it captures more than 55,000
frames. Reviewing all these images is time-consuming and prone to human error.
It has been a challenge to develop intelligent methods assisting physicians to
review the frames. The WCE frames are captured in 8-bit color depths which
provides enough a color range to detect abnormalities. Here, superpixel based
methods are proposed to segment five diseases including: bleeding, Crohn's
disease, Lymphangiectasia, Xanthoma, and Lymphoid hyperplasia. Two superpixels
methods are compared to provide semantic segmentation of these prolific
diseases: simple linear iterative clustering (SLIC) and quick shift (QS). The
segmented superpixels were classified into two classes (normal and abnormal) by
support vector machine (SVM) using texture and color features. For both
superpixel methods, the accuracy, specificity, sensitivity, and precision
(SLIC, QS) were around 92%, 93%, 93%, and 88%, respectively. However, SLIC was
dramatically faster than QS.
AxonDeepSeg: automatic axon and myelin segmentation from microscopy data using convolutional neural networks
Authors: Aldo Zaimi, Maxime Wabartha, Victor Herman, Pierre-Louis Antonsanti, Christian Samuel Perone, Julien Cohen-Adad
Date: 2017-11-03 00:00:00
Proceeding: Nature Scientific Reports 2018 2
Tasks: Segmentation

Segmentation of axon and myelin from microscopy images of the nervous system
provides useful quantitative information about the tissue microstructure, such
as axon density and myelin thickness. This could be used for instance to
document cell morphometry across species, or to validate novel non-invasive
quantitative magnetic resonance imaging techniques. Most currently-available
segmentation algorithms are based on standard image processing and usually
require multiple processing steps and/or parameter tuning by the user to adapt
to different modalities. Moreover, only few methods are publicly available. We
introduce AxonDeepSeg, an open-source software that performs axon and myelin
segmentation of microscopic images using deep learning. AxonDeepSeg features:
(i) a convolutional neural network architecture; (ii) an easy training
procedure to generate new models based on manually-labelled data and (iii) two
ready-to-use models trained from scanning electron microscopy (SEM) and
transmission electron microscopy (TEM). Results show high pixel-wise accuracy
across various species: 85% on rat SEM, 81% on human SEM, 95% on mice TEM and
84% on macaque TEM. Segmentation of a full rat spinal cord slice is computed
and morphological metrics are extracted and compared against the literature.
AxonDeepSeg is freely available at https://github.com/neuropoly/axondeepseg
Evolving soft locomotion in aquatic and terrestrial environments: effects of material properties and environmental transitions
Authors: Francesco Corucci, Nick Cheney, Francesco Giorgio-Serchi, Josh Bongard, Cecilia Laschi
Date: 2017-11-17 00:00:00

Designing soft robots poses considerable challenges: automated design
approaches may be particularly appealing in this field, as they promise to
optimize complex multi-material machines with very little or no human
intervention. Evolutionary soft robotics is concerned with the application of
optimization algorithms inspired by natural evolution in order to let soft
robots (both morphologies and controllers) spontaneously evolve within
physically-realistic simulated environments, figuring out how to satisfy a set
of objectives defined by human designers. In this paper a powerful evolutionary
system is put in place in order to perform a broad investigation on the
free-form evolution of walking and swimming soft robots in different
environments. Three sets of experiments are reported, tackling different
aspects of the evolution of soft locomotion. The first two sets explore the
effects of different material properties on the evolution of terrestrial and
aquatic soft locomotion: particularly, we show how different materials lead to
the evolution of different morphologies, behaviors, and energy-performance
tradeoffs. It is found that within our simplified physics world stiffer robots
evolve more sophisticated and effective gaits and morphologies on land, while
softer ones tend to perform better in water. The third set of experiments
starts investigating the effect and potential benefits of major environmental
transitions (land - water) during evolution. Results provide interesting
morphological exaptation phenomena, and point out a potential asymmetry between
land-water and water-land transitions: while the first type of transition
appears to be detrimental, the second one seems to have some beneficial
effects.
Deep Local Binary Patterns
Authors: Kelwin Fernandes, Jaime S. Cardoso
Date: 2017-11-17 00:00:00
Tasks: Texture Classification, Translation

Local Binary Pattern (LBP) is a traditional descriptor for texture analysis
that gained attention in the last decade. Being robust to several properties
such as invariance to illumination translation and scaling, LBPs achieved
state-of-the-art results in several applications. However, LBPs are not able to
capture high-level features from the image, merely encoding features with low
abstraction levels. In this work, we propose Deep LBP, which borrow ideas from
the deep learning community to improve LBP expressiveness. By using
parametrized data-driven LBP, we enable successive applications of the LBP
operators with increasing abstraction levels. We validate the relevance of the
proposed idea in several datasets from a wide range of applications. Deep LBP
improved the performance of traditional and multiscale LBP in all cases.
Compressing Word Embeddings via Deep Compositional Code Learning
Authors: Raphael Shu, Hideki Nakayama
Date: 2017-11-03 00:00:00
Proceeding: ICLR 2018 1
Tasks: Machine Translation, Quantization, Sentiment Analysis, Translation, Word Embeddings

Natural language processing (NLP) models often require a massive number of
parameters for word embeddings, resulting in a large storage or memory
footprint. Deploying neural NLP models to mobile devices requires compressing
the word embeddings without any significant sacrifices in performance. For this
purpose, we propose to construct the embeddings with few basis vectors. For
each word, the composition of basis vectors is determined by a hash code. To
maximize the compression rate, we adopt the multi-codebook quantization
approach instead of binary coding scheme. Each code is composed of multiple
discrete numbers, such as (3, 2, 1, 8), where the value of each component is
limited to a fixed range. We propose to directly learn the discrete codes in an
end-to-end neural network by applying the Gumbel-softmax trick. Experiments
show the compression rate achieves 98% in a sentiment analysis task and 94% ~
99% in machine translation tasks without performance loss. In both tasks, the
proposed method can improve the model performance by slightly lowering the
compression rate. Compared to other approaches such as character-level
segmentation, the proposed method is language-independent and does not require
modifications to the network architecture.
Cautious NMPC with Gaussian Process Dynamics for Autonomous Miniature Race Cars
Authors: Lukas Hewing, Alexander Liniger, Melanie N. Zeilinger
Date: 2017-11-17 00:00:00
Tasks: Model Predictive Control

This paper presents an adaptive high performance control method for
autonomous miniature race cars. Racing dynamics are notoriously hard to model
from first principles, which is addressed by means of a cautious nonlinear
model predictive control (NMPC) approach that learns to improve its dynamics
model from data and safely increases racing performance. The approach makes use
of a Gaussian Process (GP) and takes residual model uncertainty into account
through a chance constrained formulation. We present a sparse GP approximation
with dynamically adjusting inducing inputs, enabling a real-time implementable
controller. The formulation is demonstrated in simulations, which show
significant improvement with respect to both lap time and constraint
satisfaction compared to an NMPC without model learning.
Learning to Play Othello with Deep Neural Networks
Authors: Paweł Liskowski, Wojciech Jaśkowski, Krzysztof Krawiec
Date: 2017-11-17 00:00:00

Achieving superhuman playing level by AlphaGo corroborated the capabilities
of convolutional neural architectures (CNNs) for capturing complex spatial
patterns. This result was to a great extent due to several analogies between Go
board states and 2D images CNNs have been designed for, in particular
translational invariance and a relatively large board. In this paper, we verify
whether CNN-based move predictors prove effective for Othello, a game with
significantly different characteristics, including a much smaller board size
and complete lack of translational invariance. We compare several CNN
architectures and board encodings, augment them with state-of-the-art
extensions, train on an extensive database of experts' moves, and examine them
with respect to move prediction accuracy and playing strength. The empirical
evaluation confirms high capabilities of neural move predictors and suggests a
strong correlation between prediction accuracy and playing strength. The best
CNNs not only surpass all other 1-ply Othello players proposed to date but
defeat (2-ply) Edax, the best open-source Othello player.
A generic and fast C++ optimization framework
Authors: Ryan R. Curtin, Shikhar Bhardwaj, Marcus Edel, Yannis Mentekidis
Date: 2017-11-17 00:00:00
Tasks: BIG-bench Machine Learning

The development of the mlpack C++ machine learning library
(http://www.mlpack.org/) has required the design and implementation of a
flexible, robust optimization system that is able to solve the types of
arbitrary optimization problems that may arise all throughout machine learning
problems. In this paper, we present the generic optimization framework that we
have designed for mlpack. A key priority in the design was ease of
implementation of both new optimizers and new objective functions to be
optimized; therefore, implementation of a new optimizer requires only one
method and implementation of a new objective function requires at most four
functions. This leads to simple and intuitive code, which, for fast prototyping
and experimentation, is of paramount importance. When compared to optimization
frameworks of other libraries, we find that mlpack's supports more types of
objective functions, is able to make optimizations that other frameworks do
not, and seamlessly supports user-defined objective functions and optimizers.
Memory Based Online Learning of Deep Representations from Video Streams
Authors: Federico Pernici, Federico Bartoli, Matteo Bruni, Alberto del Bimbo
Date: 2017-11-17 00:00:00
Proceeding: CVPR 2018 6
Tasks: Face Identification

We present a novel online unsupervised method for face identity learning from
video streams. The method exploits deep face descriptors together with a memory
based learning mechanism that takes advantage of the temporal coherence of
visual data. Specifically, we introduce a discriminative feature matching
solution based on Reverse Nearest Neighbour and a feature forgetting strategy
that detect redundant features and discard them appropriately while time
progresses. It is shown that the proposed learning procedure is asymptotically
stable and can be effectively used in relevant applications like multiple face
identification and tracking from unconstrained video streams. Experimental
results show that the proposed method achieves comparable results in the task
of multiple face tracking and better performance in face identification with
offline approaches exploiting future information. Code will be publicly
available.
Calibration of Distributionally Robust Empirical Optimization Models
Authors: Jun-Ya Gotoh, Michael Jong Kim, Andrew E. B. Lim
Date: 2017-11-17 00:00:00
Tasks: Sensitivity

We study the out-of-sample properties of robust empirical optimization problems with smooth $\phi$-divergence penalties and smooth concave objective functions, and develop a theory for data-driven calibration of the non-negative "robustness parameter" $\delta$ that controls the size of the deviations from the nominal model. Building on the intuition that robust optimization reduces the sensitivity of the expected reward to errors in the model by controlling the spread of the reward distribution, we show that the first-order benefit of ``little bit of robustness" (i.e., $\delta$ small, positive) is a significant reduction in the variance of the out-of-sample reward while the corresponding impact on the mean is almost an order of magnitude smaller. One implication is that substantial variance (sensitivity) reduction is possible at little cost if the robustness parameter is properly calibrated. To this end, we introduce the notion of a robust mean-variance frontier to select the robustness parameter and show that it can be approximated using resampling methods like the bootstrap. Our examples show that robust solutions resulting from "open loop" calibration methods (e.g., selecting a $90\%$ confidence level regardless of the data and objective function) can be very conservative out-of-sample, while those corresponding to the robustness parameter that optimizes an estimate of the out-of-sample expected reward (e.g., via the bootstrap) with no regard for the variance are often insufficiently robust.