pwc-archive/papers-with-abstracts huggingface dataset:
Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile
Authors: Panayotis Mertikopoulos, Bruno Lecouat, Houssam Zenati, Chuan-Sheng Foo, Vijay Chandrasekhar, Georgios Piliouras
Date: 2018-07-07 00:00:00

Owing to their connection with generative adversarial networks (GANs),
saddle-point problems have recently attracted considerable interest in machine
learning and beyond. By necessity, most theoretical guarantees revolve around
convex-concave (or even linear) problems; however, making theoretical inroads
towards efficient GAN training depends crucially on moving beyond this classic
framework. To make piecemeal progress along these lines, we analyze the
behavior of mirror descent (MD) in a class of non-monotone problems whose
solutions coincide with those of a naturally associated variational inequality
- a property which we call coherence. We first show that ordinary, "vanilla" MD
converges under a strict version of this condition, but not otherwise; in
particular, it may fail to converge even in bilinear models with a unique
solution. We then show that this deficiency is mitigated by optimism: by taking
an "extra-gradient" step, optimistic mirror descent (OMD) converges in all
coherent problems. Our analysis generalizes and extends the results of
Daskalakis et al. (2018) for optimistic gradient descent (OGD) in bilinear
problems, and makes concrete headway for establishing convergence beyond
convex-concave games. We also provide stochastic analogues of these results,
and we validate our analysis by numerical experiments in a wide array of GAN
models (including Gaussian mixture models, as well as the CelebA and CIFAR-10
datasets).
Reservoir Computing Universality With Stochastic Inputs
Authors: Lukas Gonon, Juan-Pablo Ortega
Date: 2018-07-07 00:00:00

The universal approximation properties with respect to $L ^p $-type criteria
of three important families of reservoir computers with stochastic
discrete-time semi-infinite inputs is shown. First, it is proved that linear
reservoir systems with either polynomial or neural network readout maps are
universal. More importantly, it is proved that the same property holds for two
families with linear readouts, namely, trigonometric state-affine systems and
echo state networks, which are the most widely used reservoir systems in
applications. The linearity in the readouts is a key feature in supervised
machine learning applications. It guarantees that these systems can be used in
high-dimensional situations and in the presence of large datasets. The $L ^p $
criteria used in this paper allow the formulation of universality results that
do not necessarily impose almost sure uniform boundedness in the inputs or the
fading memory property in the filter that needs to be approximated.
Predicting Infant Motor Development Status using Day Long Movement Data from Wearable Sensors
Authors: David Goodfellow, Ruoyu Zhi, Rebecca Funke, Jose Carlos Pulido, Maja Mataric, Beth A. Smith
Date: 2018-07-07 00:00:00

Infants with a variety of complications at or before birth are classified as
being at risk for developmental delays (AR). As they grow older, they are
followed by healthcare providers in an effort to discern whether they are on a
typical or impaired developmental trajectory. Often, it is difficult to make an
accurate determination early in infancy as infants with typical development
(TD) display high variability in their developmental trajectories both in
content and timing. Studies have shown that spontaneous movements have the
potential to differentiate typical and atypical trajectories early in life
using sensors and kinematic analysis systems. In this study, machine learning
classification algorithms are used to take inertial movement from wearable
sensors placed on an infant for a day and predict if the infant is AR or TD,
thus further establishing the connection between early spontaneous movement and
developmental trajectory.
Gradient Hyperalignment for multi-subject fMRI data alignment
Authors: Tonglin Xu, Muhammad Yousefnezhad, Daoqiang Zhang
Date: 2018-07-07 00:00:00
Tasks: Brain Decoding, General Classification, Multi-Subject Fmri Data Alignment

Multi-subject fMRI data analysis is an interesting and challenging problem in
human brain decoding studies. The inherent anatomical and functional
variability across subjects make it necessary to do both anatomical and
functional alignment before classification analysis. Besides, when it comes to
big data, time complexity becomes a problem that cannot be ignored. This paper
proposes Gradient Hyperalignment (Gradient-HA) as a gradient-based functional
alignment method that is suitable for multi-subject fMRI datasets with large
amounts of samples and voxels. The advantage of Gradient-HA is that it can
solve independence and high dimension problems by using Independent Component
Analysis (ICA) and Stochastic Gradient Ascent (SGA). Validation using
multi-classification tasks on big data demonstrates that Gradient-HA method has
less time complexity and better or comparable performance compared with other
state-of-the-art functional alignment methods.
Anytime Neural Prediction via Slicing Networks Vertically
Authors: Hankook Lee, Jinwoo Shin
Date: 2018-07-07 00:00:00
Tasks: Prediction

The pioneer deep neural networks (DNNs) have emerged to be deeper or wider
for improving their accuracy in various applications of artificial
intelligence. However, DNNs are often too heavy to deploy in practice, and it
is often required to control their architectures dynamically given computing
resource budget, i.e., anytime prediction. While most existing approaches have
focused on training multiple shallow sub-networks jointly, we study training
thin sub-networks instead. To this end, we first build many inclusive thin
sub-networks (of the same depth) under a minor modification of existing
multi-branch DNNs, and found that they can significantly outperform the
state-of-art dense architecture for anytime prediction. This is remarkable due
to their simplicity and effectiveness, but training many thin sub-networks
jointly faces a new challenge on training complexity. To address the issue, we
also propose a novel DNN architecture by forcing a certain sparsity pattern on
multi-branch network parameters, making them train efficiently for the purpose
of anytime prediction. In our experiments on the ImageNet dataset, its
sub-networks have up to $43.3\%$ smaller sizes (FLOPs) compared to those of the
state-of-art anytime model with respect to the same accuracy. Finally, we also
propose an alternative task under the proposed architecture using a
hierarchical taxonomy, which brings a new angle for anytime prediction.
Synthetic Sampling for Multi-Class Malignancy Prediction
Authors: Matthew Yung, Eli T. Brown, Alexander Rasin, Jacob D. Furst, Daniela S. Raicu
Date: 2018-07-07 00:00:00
Tasks: General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Prediction, Sensitivity

We explore several oversampling techniques for an imbalanced multi-label
classification problem, a setting often encountered when developing models for
Computer-Aided Diagnosis (CADx) systems. While most CADx systems aim to
optimize classifiers for overall accuracy without considering the relative
distribution of each class, we look into using synthetic sampling to increase
per-class performance when predicting the degree of malignancy. Using low-level
image features and a random forest classifier, we show that using synthetic
oversampling techniques increases the sensitivity of the minority classes by an
average of 7.22% points, with as much as a 19.88% point increase in sensitivity
for a particular minority class. Furthermore, the analysis of low-level image
feature distributions for the synthetic nodules reveals that these nodules can
provide insights on how to preprocess image data for better classification
performance or how to supplement the original datasets when more data
acquisition is feasible.
From Text to Topics in Healthcare Records: An Unsupervised Graph Partitioning Methodology
Authors: M. Tarik Altuncu, Erik Mayer, Sophia N. Yaliraki, Mauricio Barahona
Date: 2018-07-07 00:00:00
Tasks: Community Detection, Descriptive, graph partitioning

Electronic Healthcare Records contain large volumes of unstructured data,
including extensive free text. Yet this source of detailed information often
remains under-used because of a lack of methodologies to extract interpretable
content in a timely manner. Here we apply network-theoretical tools to analyse
free text in Hospital Patient Incident reports from the National Health
Service, to find clusters of documents with similar content in an unsupervised
manner at different levels of resolution. We combine deep neural network
paragraph vector text-embedding with multiscale Markov Stability community
detection applied to a sparsified similarity graph of document vectors, and
showcase the approach on incident reports from Imperial College Healthcare NHS
Trust, London. The multiscale community structure reveals different levels of
meaning in the topics of the dataset, as shown by descriptive terms extracted
from the clusters of records. We also compare a posteriori against hand-coded
categories assigned by healthcare personnel, and show that our approach
outperforms LDA-based models. Our content clusters exhibit good correspondence
with two levels of hand-coded categories, yet they also provide further medical
detail in certain areas and reveal complementary descriptors of incidents
beyond the external classification taxonomy.
Fast and Accurate Point Cloud Registration using Trees of Gaussian Mixtures
Authors: Ben Eckart, Kihwan Kim, Jan Kautz
Date: 2018-07-06 00:00:00
Tasks: Autonomous Navigation, GPU, Point Cloud Registration, Scene Recognition

Point cloud registration sits at the core of many important and challenging
3D perception problems including autonomous navigation, SLAM, object/scene
recognition, and augmented reality. In this paper, we present a new
registration algorithm that is able to achieve state-of-the-art speed and
accuracy through its use of a hierarchical Gaussian Mixture Model (GMM)
representation. Our method constructs a top-down multi-scale representation of
point cloud data by recursively running many small-scale data likelihood
segmentations in parallel on a GPU. We leverage the resulting representation
using a novel PCA-based optimization criterion that adaptively finds the best
scale to perform data association between spatial subsets of point cloud data.
Compared to previous Iterative Closest Point and GMM-based techniques, our
tree-based point association algorithm performs data association in
logarithmic-time while dynamically adjusting the level of detail to best match
the complexity and spatial distribution characteristics of local scene
geometry. In addition, unlike other GMM methods that restrict covariances to be
isotropic, our new PCA-based optimization criterion well-approximates the true
MLE solution even when fully anisotropic Gaussian covariances are used.
Efficient data association, multi-scale adaptability, and a robust MLE
approximation produce an algorithm that is up to an order of magnitude both
faster and more accurate than current state-of-the-art on a wide variety of 3D
datasets captured from LiDAR to structured light.
Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences
Authors: Motonobu Kanagawa, Philipp Hennig, Dino Sejdinovic, Bharath K. Sriperumbudur
Date: 2018-07-06 00:00:00
Tasks: Gaussian Processes, regression

This paper is an attempt to bridge the conceptual gaps between researchers
working on the two widely used approaches based on positive definite kernels:
Bayesian learning or inference using Gaussian processes on the one side, and
frequentist kernel methods based on reproducing kernel Hilbert spaces on the
other. It is widely known in machine learning that these two formalisms are
closely related; for instance, the estimator of kernel ridge regression is
identical to the posterior mean of Gaussian process regression. However, they
have been studied and developed almost independently by two essentially
separate communities, and this makes it difficult to seamlessly transfer
results between them. Our aim is to overcome this potential difficulty. To this
end, we review several old and new results and concepts from either side, and
juxtapose algorithmic quantities from each framework to highlight close
similarities. We also provide discussions on subtle philosophical and
theoretical differences between the two approaches.
The Goldilocks zone: Towards better understanding of neural network loss landscapes
Authors: Stanislav Fort, Adam Scherlis
Date: 2018-07-06 00:00:00

We explore the loss landscape of fully-connected and convolutional neural
networks using random, low-dimensional hyperplanes and hyperspheres. Evaluating
the Hessian, $H$, of the loss function on these hypersurfaces, we observe 1) an
unusual excess of the number of positive eigenvalues of $H$, and 2) a large
value of $\mathrm{Tr}(H) / ||H||$ at a well defined range of configuration
space radii, corresponding to a thick, hollow, spherical shell we refer to as
the \textit{Goldilocks zone}. We observe this effect for fully-connected neural
networks over a range of network widths and depths on MNIST and CIFAR-10
datasets with the $\mathrm{ReLU}$ and $\tanh$ non-linearities, and a similar
effect for convolutional networks. Using our observations, we demonstrate a
close connection between the Goldilocks zone, measures of local
convexity/prevalence of positive curvature, and the suitability of a network
initialization. We show that the high and stable accuracy reached when
optimizing on random, low-dimensional hypersurfaces is directly related to the
overlap between the hypersurface and the Goldilocks zone, and as a corollary
demonstrate that the notion of intrinsic dimension is initialization-dependent.
We note that common initialization techniques initialize neural networks in
this particular region of unusually high convexity/prevalence of positive
curvature, and offer a geometric intuition for their success. Furthermore, we
demonstrate that initializing a neural network at a number of points and
selecting for high measures of local convexity such as $\mathrm{Tr}(H) /
||H||$, number of positive eigenvalues of $H$, or low initial loss, leads to
statistically significantly faster training on MNIST. Based on our
observations, we hypothesize that the Goldilocks zone contains an unusually
high density of suitable initialization configurations.
Guided Proceduralization: Optimizing Geometry Processing and Grammar Extraction for Architectural Models
Authors: Ilke Demir, Daniel G. Aliaga
Date: 2018-07-06 00:00:00

We describe a guided proceduralization framework that optimizes geometry
processing on architectural input models to extract target grammars. We aim to
provide efficient artistic workflows by creating procedural representations
from existing 3D models, where the procedural expressiveness is controlled by
the user. Architectural reconstruction and modeling tasks have been handled as
either time consuming manual processes or procedural generation with difficult
control and artistic influence. We bridge the gap between creation and
generation by converting existing manually modeled architecture to procedurally
editable parametrized models, and carrying the guidance to procedural domain by
letting the user define the target procedural representation. Additionally, we
propose various applications of such procedural representations, including
guided completion of point cloud models, controllable 3D city modeling, and
other benefits of procedural modeling.
Quasi-Dilemmas for Artificial Moral Agents
Authors: Daniel Kasenberg, Vasanth Sarathy, Thomas Arnold, Matthias Scheutz, Tom Williams
Date: 2018-07-06 00:00:00

In this paper we describe moral quasi-dilemmas (MQDs): situations similar to
moral dilemmas, but in which an agent is unsure whether exploring the plan
space or the world may reveal a course of action that satisfies all moral
requirements. We argue that artificial moral agents (AMAs) should be built to
handle MQDs (in particular, by exploring the plan space rather than immediately
accepting the inevitability of the moral dilemma), and that MQDs may be useful
for evaluating AMA architectures.
Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry
Authors: Nan Yang, Rui Wang, Jörg Stückler, Daniel Cremers
Date: 2018-07-06 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Reconstruction, Depth Estimation, Depth Prediction, Monocular Visual Odometry, Motion Estimation, Visual Odometry

Monocular visual odometry approaches that purely rely on geometric cues are
prone to scale drift and require sufficient motion parallax in successive
frames for motion estimation and 3D reconstruction. In this paper, we propose
to leverage deep monocular depth prediction to overcome limitations of
geometry-based monocular visual odometry. To this end, we incorporate deep
depth predictions into Direct Sparse Odometry (DSO) as direct virtual stereo
measurements. For depth prediction, we design a novel deep network that refines
predicted depth from a single image in a two-stage process. We train our
network in a semi-supervised way on photoconsistency in stereo images and on
consistency with accurate sparse depth reconstructions from Stereo DSO. Our
deep predictions excel state-of-the-art approaches for monocular depth on the
KITTI benchmark. Moreover, our Deep Virtual Stereo Odometry clearly exceeds
previous monocular and deep learning based methods in accuracy. It even
achieves comparable performance to the state-of-the-art stereo methods, while
only relying on a single camera.
Automated and Interpretable Patient ECG Profiles for Disease Detection, Tracking, and Discovery
Authors: Geoffrey H. Tison, Jeffrey Zhang, Francesca N. Delling, Rahul C. Deo
Date: 2018-07-06 00:00:00
Tasks: Binary Classification, Decision Making, Diagnostic, Rhythm

The electrocardiogram or ECG has been in use for over 100 years and remains
the most widely performed diagnostic test to characterize cardiac structure and
electrical activity. We hypothesized that parallel advances in computing power,
innovations in machine learning algorithms, and availability of large-scale
digitized ECG data would enable extending the utility of the ECG beyond its
current limitations, while at the same time preserving interpretability, which
is fundamental to medical decision-making. We identified 36,186 ECGs from the
UCSF database that were 1) in normal sinus rhythm and 2) would enable training
of specific models for estimation of cardiac structure or function or detection
of disease. We derived a novel model for ECG segmentation using convolutional
neural networks (CNN) and Hidden Markov Models (HMM) and evaluated its output
by comparing electrical interval estimates to 141,864 measurements from the
clinical workflow. We built a 725-element patient-level ECG profile using
downsampled segmentation data and trained machine learning models to estimate
left ventricular mass, left atrial volume, mitral annulus e' and to detect and
track four diseases: pulmonary arterial hypertension (PAH), hypertrophic
cardiomyopathy (HCM), cardiac amyloid (CA), and mitral valve prolapse (MVP).
CNN-HMM derived ECG segmentation agreed with clinical estimates, with median
absolute deviations (MAD) as a fraction of observed value of 0.6% for heart
rate and 4% for QT interval. Patient-level ECG profiles enabled quantitative
estimates of left ventricular and mitral annulus e' velocity with good
discrimination in binary classification models of left ventricular hypertrophy
and diastolic function. Models for disease detection ranged from AUROC of 0.94
to 0.77 for MVP. Top-ranked variables for all models included known ECG
characteristics along with novel predictors of these traits/diseases.
M-ADDA: Unsupervised Domain Adaptation with Deep Metric Learning
Authors: Issam Laradji, Reza Babanezhad
Date: 2018-07-06 00:00:00
Tasks: Domain Adaptation, Metric Learning, Triplet, Unsupervised Domain Adaptation

Unsupervised domain adaptation techniques have been successful for a wide
range of problems where supervised labels are limited. The task is to classify
an unlabeled `target' dataset by leveraging a labeled `source' dataset that
comes from a slightly similar distribution. We propose metric-based adversarial
discriminative domain adaptation (M-ADDA) which performs two main steps. First,
it uses a metric learning approach to train the source model on the source
dataset by optimizing the triplet loss function. This results in clusters where
embeddings of the same label are close to each other and those with different
labels are far from one another. Next, it uses the adversarial approach (as
that used in ADDA \cite{2017arXiv170205464T}) to make the extracted features
from the source and target datasets indistinguishable. Simultaneously, we
optimize a novel loss function that encourages the target dataset's embeddings
to form clusters. While ADDA and M-ADDA use similar architectures, we show that
M-ADDA performs significantly better on the digits adaptation datasets of MNIST
and USPS. This suggests that using metric-learning for domain adaptation can
lead to large improvements in classification accuracy for the domain adaptation
task. The code is available at \url{https://github.com/IssamLaradji/M-ADDA}.
3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
Authors: Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, Taco Cohen
Date: 2018-07-06 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: General Classification

We present a convolutional network that is equivariant to rigid body motions.
The model uses scalar-, vector-, and tensor fields over 3D Euclidean space to
represent data, and equivariant convolutions to map between such
representations. These SE(3)-equivariant convolutions utilize kernels which are
parameterized as a linear combination of a complete steerable kernel basis,
which is derived analytically in this paper. We prove that equivariant
convolutions are the most general equivariant linear maps between fields over
R^3. Our experimental results confirm the effectiveness of 3D Steerable CNNs
for the problem of amino acid propensity prediction and protein structure
classification, both of which have inherent SE(3) symmetry.
Fully Scalable Gaussian Processes using Subspace Inducing Inputs
Authors: Aristeidis Panos, Petros Dellaportas, Michalis K. Titsias
Date: 2018-07-06 00:00:00
Tasks: Extreme Multi-Label Classification, Gaussian Processes, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

We introduce fully scalable Gaussian processes, an implementation scheme that
tackles the problem of treating a high number of training instances together
with high dimensional input data. Our key idea is a representation trick over
the inducing variables called subspace inducing inputs. This is combined with
certain matrix-preconditioning based parametrizations of the variational
distributions that lead to simplified and numerically stable variational lower
bounds. Our illustrative applications are based on challenging extreme
multi-label classification problems with the extra burden of the very large
number of class labels. We demonstrate the usefulness of our approach by
presenting predictive performances together with low computational times in
datasets with extremely large number of instances and input dimensions.