pwc-archive/papers-with-abstracts huggingface dataset:
ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning
Authors: Maarten Sap, Ronan LeBras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A. Smith, Yejin Choi
Date: 2018-10-31 00:00:00
Tasks: Relation

We present ATOMIC, an atlas of everyday commonsense reasoning, organized
through 877k textual descriptions of inferential knowledge. Compared to
existing resources that center around taxonomic knowledge, ATOMIC focuses on
inferential knowledge organized as typed if-then relations with variables
(e.g., "if X pays Y a compliment, then Y will likely return the compliment").
We propose nine if-then relation types to distinguish causes vs. effects,
agents vs. themes, voluntary vs. involuntary events, and actions vs. mental
states. By generatively training on the rich inferential knowledge described in
ATOMIC, we show that neural models can acquire simple commonsense capabilities
and reason about previously unseen events. Experimental results demonstrate
that multitask models that incorporate the hierarchical structure of if-then
relation types lead to more accurate inference compared to models trained in
isolation, as measured by both automatic and human evaluation.
Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation
Authors: Matthew O'Kelly, Aman Sinha, Hongseok Namkoong, John Duchi, Russ Tedrake
Date: 2018-10-31 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Autonomous Driving

While recent developments in autonomous vehicle (AV) technology highlight
substantial progress, we lack tools for rigorous and scalable testing.
Real-world testing, the $\textit{de facto}$ evaluation environment, places the
public in danger, and, due to the rare nature of accidents, will require
billions of miles in order to statistically validate performance claims. We
implement a simulation framework that can test an entire modern autonomous
driving system, including, in particular, systems that employ deep-learning
perception and control algorithms. Using adaptive importance-sampling methods
to accelerate rare-event probability evaluation, we estimate the probability of
an accident under a base distribution governing standard traffic behavior. We
demonstrate our framework on a highway scenario, accelerating system evaluation
by $2$-$20$ times over naive Monte Carlo sampling methods and $10$-$300
\mathsf{P}$ times (where $\mathsf{P}$ is the number of processors) over
real-world testing.
Democratizing Production-Scale Distributed Deep Learning
Authors: Minghuang Ma, Hadi Pouransari, Daniel Chao, Saurabh Adya, Santiago Akle Serrano, Yi Qin, Dan Gimnicher, Dominic Walsh
Date: 2018-10-31 00:00:00
Tasks: Deep Learning

The interest and demand for training deep neural networks have been
experiencing rapid growth, spanning a wide range of applications in both
academia and industry. However, training them distributed and at scale remains
difficult due to the complex ecosystem of tools and hardware involved. One
consequence is that the responsibility of orchestrating these complex
components is often left to one-off scripts and glue code customized for
specific problems. To address these restrictions, we introduce \emph{Alchemist}
- an internal service built at Apple from the ground up for \emph{easy},
\emph{fast}, and \emph{scalable} distributed training. We discuss its design,
implementation, and examples of running different flavors of distributed
training. We also present case studies of its internal adoption in the
development of autonomous systems, where training times have been reduced by
10x to keep up with the ever-growing data collection.
Testing Halfspaces over Rotation-Invariant Distributions
Authors: Nathaniel Harms
Date: 2018-10-31 00:00:00

We present an algorithm for testing halfspaces over arbitrary, unknown
rotation-invariant distributions. Using $\tilde O(\sqrt{n}\epsilon^{-7})$
random examples of an unknown function $f$, the algorithm determines with high
probability whether $f$ is of the form $f(x) = sign(\sum_i w_ix_i-t)$ or is
$\epsilon$-far from all such functions. This sample size is significantly
smaller than the well-known requirement of $\Omega(n)$ samples for learning
halfspaces, and known lower bounds imply that our sample size is optimal (in
its dependence on $n$) up to logarithmic factors. The algorithm is
distribution-free in the sense that it requires no knowledge of the
distribution aside from the promise of rotation invariance. To prove the
correctness of this algorithm we present a theorem relating the distance
between a function and a halfspace to the distance between their centers of
mass, that applies to arbitrary distributions.
Dirichlet Variational Autoencoder for Text Modeling
Authors: Yijun Xiao, Tiancheng Zhao, William Yang Wang
Date: 2018-10-31 00:00:00

We introduce an improved variational autoencoder (VAE) for text modeling with
topic information explicitly modeled as a Dirichlet latent variable. By
providing the proposed model topic awareness, it is more superior at
reconstructing input texts. Furthermore, due to the inherent interactions
between the newly introduced Dirichlet variable and the conventional
multivariate Gaussian variable, the model is less prone to KL divergence
vanishing. We derive the variational lower bound for the new model and conduct
experiments on four different data sets. The results show that the proposed
model is superior at text reconstruction across the latent space and
classifications on learned representations have higher test accuracies.
Towards a Simple Approach to Multi-step Model-based Reinforcement Learning
Authors: Kavosh Asadi, Evan Cater, Dipendra Misra, Michael L. Littman
Date: 2018-10-31 00:00:00
Tasks: Model-based Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

When environmental interaction is expensive, model-based reinforcement
learning offers a solution by planning ahead and avoiding costly mistakes.
Model-based agents typically learn a single-step transition model. In this
paper, we propose a multi-step model that predicts the outcome of an action
sequence with variable length. We show that this model is easy to learn, and
that the model can make policy-conditional predictions. We report preliminary
results that show a clear advantage for the multi-step model compared to its
one-step counterpart.
Measuring Issue Ownership using Word Embeddings
Authors: Amaru Cuba Gyllensten, Magnus Sahlgren
Date: 2018-10-31 00:00:00
Proceeding: WS 2018 10
Tasks: Document Embedding, Word Embeddings

Sentiment and topic analysis are common methods used for social media
monitoring. Essentially, these methods answers questions such as, "what is
being talked about, regarding X", and "what do people feel, regarding X". In
this paper, we investigate another venue for social media monitoring, namely
issue ownership and agenda setting, which are concepts from political science
that have been used to explain voter choice and electoral outcomes. We argue
that issue alignment and agenda setting can be seen as a kind of semantic
source similarity of the kind "how similar is source A to issue owner P, when
talking about issue X", and as such can be measured using word/document
embedding techniques. We present work in progress towards measuring that kind
of conditioned similarity, and introduce a new notion of similarity for
predictive embeddings. We then test this method by measuring the similarity
between politically aligned media and political parties, conditioned on
bloc-specific issues.
Compressing physical properties of atomic species for improving predictive chemistry
Authors: John E. Herr, Kevin Koh, Kun Yao, John Parkhill
Date: 2018-10-31 00:00:00
Tasks: BIG-bench Machine Learning

The answers to many unsolved problems lie in the intractable chemical space
of molecules and materials. Machine learning techniques are rapidly growing in
popularity as a way to compress and explore chemical space efficiently. One of
the most important aspects of machine learning techniques is representation
through the feature vector, which should contain the most important descriptors
necessary to make accurate predictions, not least of which is the atomic
species in the molecule or material. In this work we introduce a compressed
representation of physical properties for atomic species we call the elemental
modes. The elemental modes provide an excellent representation by capturing
many of the nuances of the periodic table and the similarity of atomic species.
We apply the elemental modes to several different tasks for machine learning
algorithms and show that they enable us to make improvements to these tasks
even beyond simply achieving higher accuracy predictions.
A Mixture Model Based Defense for Data Poisoning Attacks Against Naive Bayes Spam Filters
Authors: David J. Miller, Xinyi Hu, Zhen Xiang, George Kesidis
Date: 2018-10-31 00:00:00
Tasks: Data Poisoning

Naive Bayes spam filters are highly susceptible to data poisoning attacks.
Here, known spam sources/blacklisted IPs exploit the fact that their received
emails will be treated as (ground truth) labeled spam examples, and used for
classifier training (or re-training). The attacking source thus generates
emails that will skew the spam model, potentially resulting in great
degradation in classifier accuracy. Such attacks are successful mainly because
of the poor representation power of the naive Bayes (NB) model, with only a
single (component) density to represent spam (plus a possible attack). We
propose a defense based on the use of a mixture of NB models. We demonstrate
that the learned mixture almost completely isolates the attack in a second NB
component, with the original spam component essentially unchanged by the
attack. Our approach addresses both the scenario where the classifier is being
re-trained in light of new data and, significantly, the more challenging
scenario where the attack is embedded in the original spam training set. Even
for weak attack strengths, BIC-based model order selection chooses a
two-component solution, which invokes the mixture-based defense. Promising
results are presented on the TREC 2005 spam corpus.
Regularized Fourier Ptychography using an Online Plug-and-Play Algorithm
Authors: Yu Sun, Shiqi Xu, Yunzhe Li, Lei Tian, Brendt Wohlberg, Ulugbek S. Kamilov
Date: 2018-10-31 00:00:00
Tasks: Image Reconstruction

The plug-and-play priors (PnP) framework has been recently shown to achieve
state-of-the-art results in regularized image reconstruction by leveraging a
sophisticated denoiser within an iterative algorithm. In this paper, we propose
a new online PnP algorithm for Fourier ptychographic microscopy (FPM) based on
the fast iterative shrinkage/threshold algorithm (FISTA). Specifically, the
proposed algorithm uses only a subset of measurements, which makes it scalable
to a large set of measurements. We validate the algorithm by showing that it
can lead to significant performance gains on both simulated and experimental
data.
A task in a suit and a tie: paraphrase generation with semantic augmentation
Authors: Su Wang, Rahul Gupta, Nancy Chang, Jason Baldridge
Date: 2018-10-31 00:00:00
Tasks: Paraphrase Generation

Paraphrasing is rooted in semantics. We show the effectiveness of
transformers (Vaswani et al. 2017) for paraphrase generation and further
improvements by incorporating PropBank labels via a multi-encoder. Evaluating
on MSCOCO and WikiAnswers, we find that transformers are fast and effective,
and that semantic augmentation for both transformers and LSTMs leads to sizable
2-3 point gains in BLEU, METEOR and TER. More importantly, we find surprisingly
large gains on human evaluations compared to previous models. Nevertheless,
manual inspection of generated paraphrases reveals ample room for improvement:
even our best model produces human-acceptable paraphrases for only 28% of
captions from the CHIA dataset (Sharma et al. 2018), and it fails spectacularly
on sentences from Wikipedia. Overall, these results point to the potential for
incorporating semantics in the task while highlighting the need for stronger
evaluation.
Face Recognition: From Traditional to Deep Learning Methods
Authors: Daniel Sáez Trigueros, Li Meng, Margaret Hartnett
Date: 2018-10-31 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Face Recognition

Starting in the seventies, face recognition has become one of the most
researched topics in computer vision and biometrics. Traditional methods based
on hand-crafted features and traditional machine learning techniques have
recently been superseded by deep neural networks trained with very large
datasets. In this paper we provide a comprehensive and up-to-date literature
review of popular face recognition methods including both traditional
(geometry-based, holistic, feature-based and hybrid methods) and deep learning
methods.
Dimensionality Reduction has Quantifiable Imperfections: Two Geometric Bounds
Authors: Kry Yik Chau Lui, Gavin Weiguang Ding, Ruitong Huang, Robert J. McCann
Date: 2018-10-31 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Dimensionality Reduction, Information Retrieval, Retrieval, Vocal Bursts Valence Prediction

In this paper, we investigate Dimensionality reduction (DR) maps in an
information retrieval setting from a quantitative topology point of view. In
particular, we show that no DR maps can achieve perfect precision and perfect
recall simultaneously. Thus a continuous DR map must have imperfect precision.
We further prove an upper bound on the precision of Lipschitz continuous DR
maps. While precision is a natural measure in an information retrieval setting,
it does not measure `how' wrong the retrieved data is. We therefore propose a
new measure based on Wasserstein distance that comes with similar theoretical
guarantee. A key technical step in our proofs is a particular optimization
problem of the $L_2$-Wasserstein distance over a constrained set of
distributions. We provide a complete solution to this optimization problem,
which can be of independent interest on the technical side.
Generating Photo-Realistic Training Data to Improve Face Recognition Accuracy
Authors: Daniel Sáez Trigueros, Li Meng, Margaret Hartnett
Date: 2018-10-31 00:00:00
Tasks: Face Recognition, Generative Adversarial Network

In this paper we investigate the feasibility of using synthetic data to
augment face datasets. In particular, we propose a novel generative adversarial
network (GAN) that can disentangle identity-related attributes from
non-identity-related attributes. This is done by training an embedding network
that maps discrete identity labels to an identity latent space that follows a
simple prior distribution, and training a GAN conditioned on samples from that
distribution. Our proposed GAN allows us to augment face datasets by generating
both synthetic images of subjects in the training set and synthetic images of
new subjects not in the training set. By using recent advances in GAN training,
we show that the synthetic images generated by our model are photo-realistic,
and that training with augmented datasets can indeed increase the accuracy of
face recognition models as compared with models trained with real images alone.
The Price of Fair PCA: One Extra Dimension
Authors: Samira Samadi, Uthaipon Tantipongpipat, Jamie Morgenstern, Mohit Singh, Santosh Vempala
Date: 2018-10-31 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Dimensionality Reduction

We investigate whether the standard dimensionality reduction technique of PCA
inadvertently produces data representations with different fidelity for two
different populations. We show on several real-world data sets, PCA has higher
reconstruction error on population A than on B (for example, women versus men
or lower- versus higher-educated individuals). This can happen even when the
data set has a similar number of samples from A and B. This motivates our study
of dimensionality reduction techniques which maintain similar fidelity for A
and B. We define the notion of Fair PCA and give a polynomial-time algorithm
for finding a low dimensional representation of the data which is
nearly-optimal with respect to this measure. Finally, we show on real-world
data sets that our algorithm can be used to efficiently generate a fair low
dimensional representation of the data.
On the Persistence of Clustering Solutions and True Number of Clusters in a Dataset
Authors: Amber Srivastava, Mayank Baranwal, Srinivasa Salapaka
Date: 2018-10-31 00:00:00
Tasks: Clustering

Typically clustering algorithms provide clustering solutions with
prespecified number of clusters. The lack of a priori knowledge on the true
number of underlying clusters in the dataset makes it important to have a
metric to compare the clustering solutions with different number of clusters.
This article quantifies a notion of persistence of clustering solutions that
enables comparing solutions with different number of clusters. The persistence
relates to the range of data-resolution scales over which a clustering solution
persists; it is quantified in terms of the maximum over two-norms of all the
associated cluster-covariance matrices. Thus we associate a persistence value
for each element in a set of clustering solutions with different number of
clusters. We show that the datasets where natural clusters are a priori known,
the clustering solutions that identify the natural clusters are most persistent
- in this way, this notion can be used to identify solutions with true number
of clusters. Detailed experiments on a variety of standard and synthetic
datasets demonstrate that the proposed persistence-based indicator outperforms
the existing approaches, such as, gap-statistic method, $X$-means, $G$-means,
$PG$-means, dip-means algorithms and information-theoretic method, in
accurately identifying the clustering solutions with true number of clusters.
Interestingly, our method can be explained in terms of the phase-transition
phenomenon in the deterministic annealing algorithm, where the number of
distinct cluster centers changes (bifurcates) with respect to an annealing
parameter.
An Evolutionary Algorithm with Crossover and Mutation for Model-Based Clustering
Authors: Sharon M. McNicholas, Paul D. McNicholas, Daniel A. Ashlock
Date: 2018-10-31 00:00:00
Tasks: Clustering, General Classification, parameter estimation

An evolutionary algorithm (EA) is developed as an alternative to the EM algorithm for parameter estimation in model-based clustering. This EA facilitates a different search of the fitness landscape, i.e., the likelihood surface, utilizing both crossover and mutation. Furthermore, this EA represents an efficient approach to "hard" model-based clustering and so it can be viewed as a sort of generalization of the k-means algorithm, which is itself equivalent to a restricted Gaussian mixture model. The EA is illustrated on several datasets, and its performance is compared to other hard clustering approaches and model-based clustering via the EM algorithm.
SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning
Authors: Daoming Lyu, Fangkai Yang, Bo Liu, Steven Gustafson
Date: 2018-10-31 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Scheduling

Deep reinforcement learning (DRL) has gained great success by learning
directly from high-dimensional sensory inputs, yet is notorious for the lack of
interpretability. Interpretability of the subtasks is critical in hierarchical
decision-making as it increases the transparency of black-box-style DRL
approach and helps the RL practitioners to understand the high-level behavior
of the system better. In this paper, we introduce symbolic planning into DRL
and propose a framework of Symbolic Deep Reinforcement Learning (SDRL) that can
handle both high-dimensional sensory inputs and symbolic planning. The
task-level interpretability is enabled by relating symbolic actions to
options.This framework features a planner -- controller -- meta-controller
architecture, which takes charge of subtask scheduling, data-driven subtask
learning, and subtask evaluation, respectively. The three components
cross-fertilize each other and eventually converge to an optimal symbolic plan
along with the learned subtasks, bringing together the advantages of long-term
planning capability with symbolic knowledge and end-to-end reinforcement
learning directly from a high-dimensional sensory input. Experimental results
validate the interpretability of subtasks, along with improved data efficiency
compared with state-of-the-art approaches.
The UEA multivariate time series classification archive, 2018
Authors: Anthony Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, Eamonn Keogh
Date: 2018-10-31 00:00:00
Tasks: Classification, General Classification, Time Series, Time Series Analysis, Time Series Classification

In 2002, the UCR time series classification archive was first released with
sixteen datasets. It gradually expanded, until 2015 when it increased in size
from 45 datasets to 85 datasets. In October 2018 more datasets were added,
bringing the total to 128. The new archive contains a wide range of problems,
including variable length series, but it still only contains univariate time
series classification problems. One of the motivations for introducing the
archive was to encourage researchers to perform a more rigorous evaluation of
newly proposed time series classification (TSC) algorithms. It has worked: most
recent research into TSC uses all 85 datasets to evaluate algorithmic advances.
Research into multivariate time series classification, where more than one
series are associated with each class label, is in a position where univariate
TSC research was a decade ago. Algorithms are evaluated using very few datasets
and claims of improvement are not based on statistical comparisons. We aim to
address this problem by forming the first iteration of the MTSC archive, to be
hosted at the website www.timeseriesclassification.com. Like the univariate
archive, this formulation was a collaborative effort between researchers at the
University of East Anglia (UEA) and the University of California, Riverside
(UCR). The 2018 vintage consists of 30 datasets with a wide range of cases,
dimensions and series lengths. For this first iteration of the archive we
format all data to be of equal length, include no series with missing data and
provide train/test splits.
Deep Generative Model with Beta Bernoulli Process for Modeling and Learning Confounding Factors
Authors: Prashnna K Gyawali, Cameron Knight, Sandesh Ghimire, B. Milan Horacek, John L. Sapp, Linwei Wang
Date: 2018-10-31 00:00:00
Tasks: Representation Learning

While deep representation learning has become increasingly capable of separating task-relevant representations from other confounding factors in the data, two significant challenges remain. First, there is often an unknown and potentially infinite number of confounding factors coinciding in the data. Second, not all of these factors are readily observable. In this paper, we present a deep conditional generative model that learns to disentangle a task-relevant representation from an unknown number of confounding factors that may grow infinitely. This is achieved by marrying the representational power of deep generative models with Bayesian non-parametric factor models, where a supervised deterministic encoder learns task-related representation and a probabilistic encoder with an Indian Buffet Process (IBP) learns the unknown number of unobservable confounding factors. We tested the presented model in two datasets: a handwritten digit dataset (MNIST) augmented with colored digits and a clinical ECG dataset with significant inter-subject variations and augmented with signal artifacts. These diverse data sets highlighted the ability of the presented model to grow with the complexity of the data and identify the absence or presence of unobserved confounding factors.