pwc-archive/papers-with-abstracts huggingface dataset:
PAC-Bayes Control: Learning Policies that Provably Generalize to Novel Environments
Authors: Anirudha Majumdar, Alec Farid, Anoopkumar Sonar
Date: 2018-06-11 00:00:00

Our goal is to learn control policies for robots that provably generalize well to novel environments given a dataset of example environments. The key technical idea behind our approach is to leverage tools from generalization theory in machine learning by exploiting a precise analogy (which we present in the form of a reduction) between generalization of control policies to novel environments and generalization of hypotheses in the supervised learning setting. In particular, we utilize the Probably Approximately Correct (PAC)-Bayes framework, which allows us to obtain upper bounds that hold with high probability on the expected cost of (stochastic) control policies across novel environments. We propose policy learning algorithms that explicitly seek to minimize this upper bound. The corresponding optimization problem can be solved using convex optimization (Relative Entropy Programming in particular) in the setting where we are optimizing over a finite policy space. In the more general setting of continuously parameterized policies (e.g., neural network policies), we minimize this upper bound using stochastic gradient descent. We present simulated results of our approach applied to learning (1) reactive obstacle avoidance policies and (2) neural network-based grasping policies. We also present hardware results for the Parrot Swing drone navigating through different obstacle environments. Our examples demonstrate the potential of our approach to provide strong generalization guarantees for robotic systems with continuous state and action spaces, complicated (e.g., nonlinear) dynamics, rich sensory inputs (e.g., depth images), and neural network-based policies.
Insights on representational similarity in neural networks with canonical correlation
Authors: Ari S. Morcos, Maithra Raghu, Samy Bengio
Date: 2018-06-14 00:00:00
Proceeding: NeurIPS 2018 12

Comparing different neural network representations and determining how
representations evolve over time remain challenging open questions in our
understanding of the function of neural networks. Comparing representations in
neural networks is fundamentally difficult as the structure of representations
varies greatly, even across groups of networks trained on identical tasks, and
over the course of training. Here, we develop projection weighted CCA
(Canonical Correlation Analysis) as a tool for understanding neural networks,
building off of SVCCA, a recently proposed method (Raghu et al., 2017). We
first improve the core method, showing how to differentiate between signal and
noise, and then apply this technique to compare across a group of CNNs,
demonstrating that networks which generalize converge to more similar
representations than networks which memorize, that wider networks converge to
more similar solutions than narrow networks, and that trained networks with
identical topology but different learning rates converge to distinct clusters
with diverse representations. We also investigate the representational dynamics
of RNNs, across both training and sequential timesteps, finding that RNNs
converge in a bottom-up pattern over the course of training and that the hidden
state is highly variable over the course of a sequence, even when accounting
for linear transforms. Together, these results provide new insights into the
function of CNNs and RNNs, and demonstrate the utility of using CCA to
understand representations.
Enhanced Local Binary Patterns for Automatic Face Recognition
Authors: Pavel Král, Ladislav Lenc, Antonín Vrba
Date: 2017-02-10 00:00:00
Tasks: Face Recognition

This paper presents a novel automatic face recognition approach based on
local binary patterns. This descriptor considers a local neighbourhood of a
pixel to compute the feature vector values. This method is not very robust to
handle image noise, variances and different illumination conditions. We address
these issues by proposing a novel descriptor which considers more pixels and
different neighbourhoods to compute the feature vector values. The proposed
method is evaluated on two benchmark corpora, namely UFI and FERET face
datasets. We experimentally show that our approach outperforms state-of-the-art
methods and is efficient particularly in the real conditions where the above
mentioned issues are obvious. We further show that the proposed method handles
well one training sample issue and is also robust to the image resolution.
Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model
Authors: Soheil Kolouri, Phillip E. Pope, Charles E. Martin, Gustavo K. Rohde
Date: 2018-04-05 00:00:00
Tasks: model

In this paper we study generative modeling via autoencoders while using the
elegant geometric properties of the optimal transport (OT) problem and the
Wasserstein distances. We introduce Sliced-Wasserstein Autoencoders (SWAE),
which are generative models that enable one to shape the distribution of the
latent space into any samplable probability distribution without the need for
training an adversarial network or defining a closed-form for the distribution.
In short, we regularize the autoencoder loss with the sliced-Wasserstein
distance between the distribution of the encoded training samples and a
predefined samplable distribution. We show that the proposed formulation has an
efficient numerical solution that provides similar capabilities to Wasserstein
Autoencoders (WAE) and Variational Autoencoders (VAE), while benefiting from an
embarrassingly simple implementation.
Detecting Speech Act Types in Developer Question/Answer Conversations During Bug Repair
Authors: Andrew Wood, Paige Rodeghero, Ameer Armaly, Collin McMillan
Date: 2018-06-13 00:00:00

This paper targets the problem of speech act detection in conversations about
bug repair. We conduct a "Wizard of Oz" experiment with 30 professional
programmers, in which the programmers fix bugs for two hours, and use a
simulated virtual assistant for help. Then, we use an open coding manual
annotation procedure to identify the speech act types in the conversations.
Finally, we train and evaluate a supervised learning algorithm to automatically
detect the speech act types in the conversations. In 30 two-hour conversations,
we made 2459 annotations and uncovered 26 speech act types. Our automated
detection achieved 69% precision and 50% recall. The key application of this
work is to advance the state of the art for virtual assistants in software
engineering. Virtual assistant technology is growing rapidly, though
applications in software engineering are behind those in other areas, largely
due to a lack of relevant data and experiments. This paper targets this problem
in the area of developer Q/A conversations about bug repair.
Probabilistic Tools for the Analysis of Randomized Optimization Heuristics
Authors: Benjamin Doerr
Date: 2018-01-20 00:00:00

This chapter collects several probabilistic tools that proved to be useful in the analysis of randomized search heuristics. This includes classic material like Markov, Chebyshev and Chernoff inequalities, but also lesser known topics like stochastic domination and coupling or Chernoff bounds for geometrically distributed random variables and for negatively correlated random variables. Most of the results presented here have appeared previously, some, however, only in recent conference publications. While the focus is on collecting tools for the analysis of randomized search heuristics, many of these may be useful as well in the analysis of classic randomized algorithms or discrete random structures.
Using Clinical Narratives and Structured Data to Identify Distant Recurrences in Breast Cancer
Authors: Zexian Zeng, Ankita Roy, Xiaoyu Li, Sasa Espino, Susan Clare, Seema Khan, Yuan Luo
Date: 2018-06-13 00:00:00
Tasks: Computational Phenotyping

Accurately identifying distant recurrences in breast cancer from the
Electronic Health Records (EHR) is important for both clinical care and
secondary analysis. Although multiple applications have been developed for
computational phenotyping in breast cancer, distant recurrence identification
still relies heavily on manual chart review. In this study, we aim to develop a
model that identifies distant recurrences in breast cancer using clinical
narratives and structured data from EHR. We apply MetaMap to extract features
from clinical narratives and also retrieve structured clinical data from EHR.
Using these features, we train a support vector machine model to identify
distant recurrences in breast cancer patients. We train the model using 1,396
double-annotated subjects and validate the model using 599 double-annotated
subjects. In addition, we validate the model on a set of 4,904 single-annotated
subjects as a generalization test. We obtained a high area under curve (AUC)
score of 0.92 (SD=0.01) in the cross-validation using the training dataset,
then obtained AUC scores of 0.95 and 0.93 in the held-out test and
generalization test using 599 and 4,904 samples respectively. Our model can
accurately and efficiently identify distant recurrences in breast cancer by
combining features extracted from unstructured clinical narratives and
structured clinical data.
Natural Language Processing for EHR-Based Computational Phenotyping
Authors: Zexian Zeng, Yu Deng, Xiaoyu Li, Tristan Naumann, Yuan Luo
Date: 2018-06-13 00:00:00
Tasks: Computational Phenotyping

This article reviews recent advances in applying natural language processing
(NLP) to Electronic Health Records (EHRs) for computational phenotyping.
NLP-based computational phenotyping has numerous applications including
diagnosis categorization, novel phenotype discovery, clinical trial screening,
pharmacogenomics, drug-drug interaction (DDI) and adverse drug event (ADE)
detection, as well as genome-wide and phenome-wide association studies.
Significant progress has been made in algorithm development and resource
construction for computational phenotyping. Among the surveyed methods,
well-designed keyword search and rule-based systems often achieve good
performance. However, the construction of keyword and rule lists requires
significant manual effort, which is difficult to scale. Supervised machine
learning models have been favored because they are capable of acquiring both
classification patterns and structures from data. Recently, deep learning and
unsupervised learning have received growing attention, with the former favored
for its performance and the latter for its ability to find novel phenotypes.
Integrating heterogeneous data sources have become increasingly important and
have shown promise in improving model performance. Often better performance is
achieved by combining multiple modalities of information. Despite these many
advances, challenges and opportunities remain for NLP-based computational
phenotyping, including better model interpretability and generalizability, and
proper characterization of feature relations in clinical narratives
To understand deep learning we need to understand kernel learning
Authors: Mikhail Belkin, Siyuan Ma, Soumik Mandal
Date: 2018-02-05 00:00:00
Proceeding: ICML 2018 7
Tasks: Deep Learning, Generalization Bounds

Generalization performance of classifiers in deep learning has recently
become a subject of intense study. Deep models, typically over-parametrized,
tend to fit the training data exactly. Despite this "overfitting", they perform
well on test data, a phenomenon not yet fully understood.
  The first point of our paper is that strong performance of overfitted
classifiers is not a unique feature of deep learning. Using six real-world and
two synthetic datasets, we establish experimentally that kernel machines
trained to have zero classification or near zero regression error perform very
well on test data, even when the labels are corrupted with a high level of
noise. We proceed to give a lower bound on the norm of zero loss solutions for
smooth kernels, showing that they increase nearly exponentially with data size.
We point out that this is difficult to reconcile with the existing
generalization bounds. Moreover, none of the bounds produce non-trivial results
for interpolating solutions.
  Second, we show experimentally that (non-smooth) Laplacian kernels easily fit
random labels, a finding that parallels results for ReLU neural networks. In
contrast, fitting noisy data requires many more epochs for smooth Gaussian
kernels. Similar performance of overfitted Laplacian and Gaussian classifiers
on test, suggests that generalization is tied to the properties of the kernel
function rather than the optimization process.
  Certain key phenomena of deep learning are manifested similarly in kernel
methods in the modern "overfitted" regime. The combination of the experimental
and theoretical results presented in this paper indicates a need for new
theoretical ideas for understanding properties of classical kernel methods. We
argue that progress on understanding deep learning will be difficult until more
tractable "shallow" kernel methods are better understood.
Age and Gender Classification From Ear Images
Authors: Dogucan Yaman, Fevziye Irem Eyiokur, Nurdan Sezgin, Hazim Kemal Ekenel
Date: 2018-06-14 00:00:00
Tasks: Age And Gender Classification, Age Classification, Age Estimation, Classification, Domain Adaptation, Gender Classification, General Classification

In this paper, we present a detailed analysis on extracting soft biometric
traits, age and gender, from ear images. Although there have been a few
previous work on gender classification using ear images, to the best of our
knowledge, this study is the first work on age classification from ear images.
In the study, we have utilized both geometric features and appearance-based
features for ear representation. The utilized geometric features are based on
eight anthropometric landmarks and consist of 14 distance measurements and two
area calculations. The appearance-based methods employ deep convolutional
neural networks for representation and classification. The well-known
convolutional neural network models, namely, AlexNet, VGG-16, GoogLeNet, and
SqueezeNet have been adopted for the study. They have been fine-tuned on a
large-scale ear dataset that has been built from the profile and
close-to-profile face images in the Multi-PIE face dataset. This way, we have
performed a domain adaptation. The updated models have been fine-tuned once
more time on the small-scale target ear dataset, which contains only around 270
ear images for training. According to the experimental results,
appearance-based methods have been found to be superior to the methods based on
geometric features. We have achieved 94\% accuracy for gender classification,
whereas 52\% accuracy has been obtained for age classification. These results
indicate that ear images provide useful cues for age and gender classification,
however, further work is required for age estimation.
PsyPhy: A Psychophysics Driven Evaluation Framework for Visual Recognition
Authors: Brandon RichardWebster, Samuel E. Anthony, Walter J. Scheirer
Date: 2016-11-19 00:00:00

By providing substantial amounts of data and standardized evaluation
protocols, datasets in computer vision have helped fuel advances across all
areas of visual recognition. But even in light of breakthrough results on
recent benchmarks, it is still fair to ask if our recognition algorithms are
doing as well as we think they are. The vision sciences at large make use of a
very different evaluation regime known as Visual Psychophysics to study visual
perception. Psychophysics is the quantitative examination of the relationships
between controlled stimuli and the behavioral responses they elicit in
experimental test subjects. Instead of using summary statistics to gauge
performance, psychophysics directs us to construct item-response curves made up
of individual stimulus responses to find perceptual thresholds, thus allowing
one to identify the exact point at which a subject can no longer reliably
recognize the stimulus class. In this article, we introduce a comprehensive
evaluation framework for visual recognition models that is underpinned by this
methodology. Over millions of procedurally rendered 3D scenes and 2D images, we
compare the performance of well-known convolutional neural networks. Our
results bring into question recent claims of human-like performance, and
provide a path forward for correcting newly surfaced algorithmic deficiencies.
Using Search Queries to Understand Health Information Needs in Africa
Authors: Rediet Abebe, Shawndra Hill, Jennifer Wortman Vaughan, Peter M. Small, H. Andrew Schwartz
Date: 2018-06-14 00:00:00
Tasks: Misconceptions

The lack of comprehensive, high-quality health data in developing nations
creates a roadblock for combating the impacts of disease. One key challenge is
understanding the health information needs of people in these nations. Without
understanding people's everyday needs, concerns, and misconceptions, health
organizations and policymakers lack the ability to effectively target education
and programming efforts. In this paper, we propose a bottom-up approach that
uses search data from individuals to uncover and gain insight into health
information needs in Africa. We analyze Bing searches related to HIV/AIDS,
malaria, and tuberculosis from all 54 African nations. For each disease, we
automatically derive a set of common search themes or topics, revealing a
wide-spread interest in various types of information, including disease
symptoms, drugs, concerns about breastfeeding, as well as stigma, beliefs in
natural cures, and other topics that may be hard to uncover through traditional
surveys. We expose the different patterns that emerge in health information
needs by demographic groups (age and sex) and country. We also uncover
discrepancies in the quality of content returned by search engines to users by
topic. Combined, our results suggest that search data can help illuminate
health information needs in Africa and inform discussions on health policy and
targeted education efforts both on- and offline.
A Sauer-Shelah-Perles Lemma for Sumsets
Authors: Zeev Dvir, Shay Moran
Date: 2018-06-14 00:00:00
Tasks: LEMMA

We show that any family of subsets $A\subseteq 2^{[n]}$ satisfies $\lvert
A\rvert \leq O\bigl(n^{\lceil{d}/{2}\rceil}\bigr)$, where $d$ is the VC
dimension of $\{S\triangle T \,\vert\, S,T\in A\}$, and $\triangle$ is the
symmetric difference operator. We also observe that replacing $\triangle$ by
either $\cup$ or $\cap$ fails to satisfy an analogous statement. Our proof is
based on the polynomial method; specifically, on an argument due to [Croot,
Lev, Pach '17].
Efficient sampling for Gaussian linear regression with arbitrary priors
Authors: P. Richard Hahn, Jingyu He, Hedibert Lopes
Date: 2018-06-14 00:00:00
Tasks: regression

This paper develops a slice sampler for Bayesian linear regression models
with arbitrary priors. The new sampler has two advantages over current
approaches. One, it is faster than many custom implementations that rely on
auxiliary latent variables, if the number of regressors is large. Two, it can
be used with any prior with a density function that can be evaluated up to a
normalizing constant, making it ideal for investigating the properties of new
shrinkage priors without having to develop custom sampling algorithms. The new
sampler takes advantage of the special structure of the linear regression
likelihood, allowing it to produce better effective sample size per second than
common alternative approaches.
Learning Influence-Receptivity Network Structure with Guarantee
Authors: Ming Yu, Varun Gupta, Mladen Kolar
Date: 2018-06-14 00:00:00
Tasks: Community Detection

Traditional works on community detection from observations of information
cascade assume that a single adjacency matrix parametrizes all the observed
cascades. However, in reality the connection structure usually does not stay
the same across cascades. For example, different people have different topics
of interest, therefore the connection structure depends on the
information/topic content of the cascade. In this paper we consider the case
where we observe a sequence of noisy adjacency matrices triggered by
information/event with different topic distributions. We propose a novel latent
model using the intuition that a connection is more likely to exist between two
nodes if they are interested in similar topics, which are common with the
information/event. Specifically, we endow each node with two node-topic
vectors: an influence vector that measures how influential/authoritative they
are on each topic; and a receptivity vector that measures how
receptive/susceptible they are to each topic. We show how these two node-topic
structures can be estimated from observed adjacency matrices with theoretical
guarantee on estimation error, in cases where the topic distributions of the
information/event are known, as well as when they are unknown. Experiments on
synthetic and real data demonstrate the effectiveness of our model and superior
performance compared to state-of-the-art methods.
Action Learning for 3D Point Cloud Based Organ Segmentation
Authors: Xia Zhong, Mario Amrehn, Nishant Ravikumar, Shuqing Chen, Norbert Strobel, Annette Birkhold, Markus Kowarschik, Rebecca Fahrig, Andreas Maier
Date: 2018-06-14 00:00:00
Tasks: Organ Segmentation, Q-Learning, Transfer Learning

We propose a novel point cloud based 3D organ segmentation pipeline utilizing
deep Q-learning. In order to preserve shape properties, the learning process is
guided using a statistical shape model. The trained agent directly predicts
piece-wise linear transformations for all vertices in each iteration. This
mapping between the ideal transformation for an object outline estimation is
learned based on image features. To this end, we introduce aperture features
that extract gray values by sampling the 3D volume within the cone centered
around the associated vertex and its normal vector. Our approach is also
capable of estimating a hierarchical pyramid of non rigid deformations for
multi-resolution meshes. In the application phase, we use a marginal approach
to gradually estimate affine as well as non-rigid transformations. We performed
extensive evaluations to highlight the robust performance of our approach on a
variety of challenge data as well as clinical data. Additionally, our method
has a run time ranging from 0.3 to 2.7 seconds to segment each organ. In
addition, we show that the proposed method can be applied to different organs,
X-ray based modalities, and scanning protocols without the need of transfer
learning. As we learn actions, even unseen reference meshes can be processed as
demonstrated in an example with the Visible Human. From this we conclude that
our method is robust, and we believe that our method can be successfully
applied to many more applications, in particular, in the interventional imaging
space.