pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised Dimension Selection using a Blue Noise Spectrum
Authors: Jayaraman J. Thiagarajan, Rushil Anirudh, Rahul Sridhar, Peer-Timo Bremer
Date: 2018-10-31 00:00:00
Tasks: Dimensionality Reduction

Unsupervised dimension selection is an important problem that seeks to reduce
dimensionality of data, while preserving the most useful characteristics. While
dimensionality reduction is commonly utilized to construct low-dimensional
embeddings, they produce feature spaces that are hard to interpret. Further, in
applications such as sensor design, one needs to perform reduction directly in
the input domain, instead of constructing transformed spaces. Consequently,
dimension selection (DS) aims to solve the combinatorial problem of identifying
the top-$k$ dimensions, which is required for effective experiment design,
reducing data while keeping it interpretable, and designing better sensing
mechanisms. In this paper, we develop a novel approach for DS based on graph
signal analysis to measure feature influence. By analyzing synthetic graph
signals with a blue noise spectrum, we show that we can measure the importance
of each dimension. Using experiments in supervised learning and image masking,
we demonstrate the superiority of the proposed approach over existing
techniques in capturing crucial characteristics of high dimensional spaces,
using only a small subset of the original features.
Understanding Deep Neural Networks through Input Uncertainties
Authors: Jayaraman J. Thiagarajan, Irene Kim, Rushil Anirudh, Peer-Timo Bremer
Date: 2018-10-31 00:00:00
Tasks: Prediction, Sensitivity

Techniques for understanding the functioning of complex machine learning
models are becoming increasingly popular, not only to improve the validation
process, but also to extract new insights about the data via exploratory
analysis. Though a large class of such tools currently exists, most assume that
predictions are point estimates and use a sensitivity analysis of these
estimates to interpret the model. Using lightweight probabilistic networks we
show how including prediction uncertainties in the sensitivity analysis leads
to: (i) more robust and generalizable models; and (ii) a new approach for model
interpretation through uncertainty decomposition. In particular, we introduce a
new regularization that takes both the mean and variance of a prediction into
account and demonstrate that the resulting networks provide improved
generalization to unseen data. Furthermore, we propose a new technique to
explain prediction uncertainties through uncertainties in the input domain,
thus providing new ways to validate and interpret deep learning models.
Multiple Measurement Vectors Problem: A Decoupling Property and its Applications
Authors: Saeid Haghighatshoar, Giuseppe Caire
Date: 2018-10-31 00:00:00
Tasks: compressed sensing

We study a Compressed Sensing (CS) problem known as Multiple Measurement
Vectors (MMV) problem, which arises in joint estimation of multiple signal
realizations when the signal samples have a common (joint) sparse support over
a fixed known dictionary. Although there is a vast literature on the analysis
of MMV, it is not yet fully known how the number of signal samples and their
statistical correlations affects the performance of the joint estimation in
MMV. Moreover, in many instances of MMV the underlying sparsifying dictionary
may not be precisely known, and it is still an open problem to quantify how the
dictionary mismatch may affect the estimation performance.
  In this paper, we focus on $\ell_{2,1}$-norm regularized least squares
($\ell_{2,1}$-LS) as a well-known and widely-used MMV algorithm in the
literature. We prove an interesting decoupling property for $\ell_{2,1}$-LS,
where we show that it can be decomposed into two phases: i) use all the signal
samples to estimate the signal covariance matrix (coupled phase), ii) plug in
the resulting covariance estimate as the true covariance matrix into the
Minimum Mean Squared Error (MMSE) estimator to reconstruct each signal sample
individually (decoupled phase). As a consequence of this decomposition, we are
able to provide further insights on the performance of $\ell_{2,1}$-LS for MMV.
In particular, we address how the signal correlations and dictionary mismatch
affects its performance. Moreover, we show that by using the decoupling
property one can obtain a variety of MMV algorithms with performances even
better than that of $\ell_{2,1}$-LS. We also provide numerical simulations to
validate our theoretical results.
Extracting Linguistic Resources from the Web for Concept-to-Text Generation
Authors: Gerasimos Lampouras, Ion Androutsopoulos
Date: 2018-10-31 00:00:00
Tasks: Concept-To-Text Generation, Sentence, Text Generation

Many concept-to-text generation systems require domain-specific linguistic
resources to produce high quality texts, but manually constructing these
resources can be tedious and costly. Focusing on NaturalOWL, a publicly
available state of the art natural language generator for OWL ontologies, we
propose methods to extract from the Web sentence plans and natural language
names, two of the most important types of domain-specific linguistic resources
used by the generator. Experiments show that texts generated using linguistic
resources extracted by our methods in a semi-automatic manner, with minimal
human involvement, are perceived as being almost as good as texts generated
using manually authored linguistic resources, and much better than texts
produced by using linguistic resources extracted from the relation and entity
identifiers of the ontology.
You May Not Need Attention
Authors: Ofir Press, Noah A. Smith
Date: 2018-10-31 00:00:00
Tasks: Decoder, NMT, Translation

In NMT, how far can we get without attention and without separate encoding
and decoding? To answer that question, we introduce a recurrent neural
translation model that does not use attention and does not have a separate
encoder and decoder. Our eager translation model is low-latency, writing target
tokens as soon as it reads the first source token, and uses constant memory
during decoding. It performs on par with the standard attention-based model of
Bahdanau et al. (2014), and better on long sentences.
On The Inductive Bias of Words in Acoustics-to-Word Models
Authors: Hao Tang, James Glass
Date: 2018-10-31 00:00:00
Tasks: Inductive Bias

Acoustics-to-word models are end-to-end speech recognizers that use words as
targets without relying on pronunciation dictionaries or graphemes. These
models are notoriously difficult to train due to the lack of linguistic
knowledge. It is also unclear how the amount of training data impacts the
optimization and generalization of such models. In this work, we study the
optimization and generalization of acoustics-to-word models under different
amounts of training data. In addition, we study three types of inductive bias,
leveraging a pronunciation dictionary, word boundary annotations, and
constraints on word durations. We find that constraining word durations leads
to the most improvement. Finally, we analyze the word embedding space learned
by the model, and find that the space has a structure dominated by the
pronunciation of words. This suggests that the contexts of words, instead of
their phonetic structure, should be the future focus of inductive bias in
acoustics-to-word models.
Unsupervised Identification of Disease Marker Candidates in Retinal OCT Imaging Data
Authors: Philipp Seeböck, Sebastian M. Waldstein, Sophie Klimscha, Hrvoje Bogunovic, Thomas Schlegl, Bianca S. Gerendas, René Donner, Ursula Schmidt-Erfurth, Georg Langs
Date: 2018-10-31 00:00:00
Tasks: Binary Classification, Clustering, Denoising, Management, Prognosis

The identification and quantification of markers in medical images is
critical for diagnosis, prognosis, and disease management. Supervised machine
learning enables the detection and exploitation of findings that are known a
priori after annotation of training examples by experts. However, supervision
does not scale well, due to the amount of necessary training examples, and the
limitation of the marker vocabulary to known entities. In this proof-of-concept
study, we propose unsupervised identification of anomalies as candidates for
markers in retinal Optical Coherence Tomography (OCT) imaging data without a
constraint to a priori definitions. We identify and categorize marker
candidates occurring frequently in the data, and demonstrate that these markers
show predictive value in the task of detecting disease. A careful qualitative
analysis of the identified data driven markers reveals how their quantifiable
occurrence aligns with our current understanding of disease course, in early-
and late age-related macular degeneration (AMD) patients. A multi-scale deep
denoising autoencoder is trained on healthy images, and a one-class support
vector machine identifies anomalies in new data. Clustering in the anomalies
identifies stable categories. Using these markers to classify healthy-, early
AMD- and late AMD cases yields an accuracy of 81.40%. In a second binary
classification experiment on a publicly available data set (healthy vs.
intermediate AMD) the model achieves an area under the ROC curve of 0.944.
Differentiable MPC for End-to-end Planning and Control
Authors: Brandon Amos, Ivan Dario Jimenez Rodriguez, Jacob Sacks, Byron Boots, J. Zico Kolter
Date: 2018-10-31 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Imitation Learning, Model Predictive Control, Reinforcement Learning

We present foundations for using Model Predictive Control (MPC) as a differentiable policy class for reinforcement learning in continuous state and action spaces. This provides one way of leveraging and combining the advantages of model-free and model-based approaches. Specifically, we differentiate through MPC by using the KKT conditions of the convex approximation at a fixed point of the controller. Using this strategy, we are able to learn the cost and dynamics of a controller via end-to-end learning. Our experiments focus on imitation learning in the pendulum and cartpole domains, where we learn the cost and dynamics terms of an MPC policy class. We show that our MPC policies are significantly more data-efficient than a generic neural network and that our method is superior to traditional system identification in a setting where the expert is unrealizable.
Accelerating SGD with momentum for over-parameterized learning
Authors: Chaoyue Liu, Mikhail Belkin
Date: 2018-10-31 00:00:00
Proceeding: ICLR 2020 1

Nesterov SGD is widely used for training modern neural networks and other machine learning models. Yet, its advantages over SGD have not been theoretically clarified. Indeed, as we show in our paper, both theoretically and empirically, Nesterov SGD with any parameter selection does not in general provide acceleration over ordinary SGD. Furthermore, Nesterov SGD may diverge for step sizes that ensure convergence of ordinary SGD. This is in contrast to the classical results in the deterministic scenario, where the same step size ensures accelerated convergence of the Nesterov's method over optimal gradient descent. To address the non-acceleration issue, we introduce a compensation term to Nesterov SGD. The resulting algorithm, which we call MaSS, converges for same step sizes as SGD. We prove that MaSS obtains an accelerated convergence rates over SGD for any mini-batch size in the linear setting. For full batch, the convergence rate of MaSS matches the well-known accelerated rate of the Nesterov's method. We also analyze the practically important question of the dependence of the convergence rate and optimal hyper-parameters on the mini-batch size, demonstrating three distinct regimes: linear scaling, diminishing returns and saturation. Experimental evaluation of MaSS for several standard architectures of deep networks, including ResNet and convolutional networks, shows improved performance over SGD, Nesterov SGD and Adam.
Picking Apart Story Salads
Authors: Su Wang, Eric Holgate, Greg Durrett, Katrin Erk
Date: 2018-10-31 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Clustering

During natural disasters and conflicts, information about what happened is
often confusing, messy, and distributed across many sources. We would like to
be able to automatically identify relevant information and assemble it into
coherent narratives of what happened. To make this task accessible to neural
models, we introduce Story Salads, mixtures of multiple documents that can be
generated at scale. By exploiting the Wikipedia hierarchy, we can generate
salads that exhibit challenging inference problems. Story salads give rise to a
novel, challenging clustering task, where the objective is to group sentences
from the same narratives. We demonstrate that simple bag-of-words similarity
clustering falls short on this task and that it is necessary to take into
account global context and coherence.
Performance assessment of the deep learning technologies in grading glaucoma severity
Authors: Yi Zhen, Lei Wang, Han Liu, Jian Zhang, Jiantao Pu
Date: 2018-10-31 00:00:00

Objective: To validate and compare the performance of eight available deep
learning architectures in grading the severity of glaucoma based on color
fundus images. Materials and Methods: We retrospectively collected a dataset of
5978 fundus images and their glaucoma severities were annotated by the
consensus of two experienced ophthalmologists. We preprocessed the images to
generate global and local regions of interest (ROIs), namely the global
field-of-view images and the local disc region images. We then divided the
generated images into three independent sub-groups for training, validation,
and testing purposes. With the datasets, eight convolutional neural networks
(CNNs) (i.e., VGG16, VGG19, ResNet, DenseNet, InceptionV3, InceptionResNet,
Xception, and NASNetMobile) were trained separately to grade glaucoma severity,
and validated quantitatively using the area under the receiver operating
characteristic (ROC) curve and the quadratic kappa score. Results: The CNNs,
except VGG16 and VGG19, achieved average kappa scores of 80.36% and 78.22% when
trained from scratch on global and local ROIs, and 85.29% and 82.72% when
fine-tuned using the pre-trained weights, respectively. VGG16 and VGG19
achieved reasonable accuracy when trained from scratch, but they failed when
using pre-trained weights for global and local ROIs. Among these CNNs, the
DenseNet had the highest classification accuracy (i.e., 75.50%) based on
pre-trained weights when using global ROIs, as compared to 65.50% when using
local ROIs. Conclusion: The experiments demonstrated the feasibility of the
deep learning technology in grading glaucoma severity. In particular, global
field-of-view images contain relatively richer information that may be critical
for glaucoma assessment, suggesting that we should use the entire field-of-view
of a fundus image for training a deep learning network.
Analyzing biological and artificial neural networks: challenges with opportunities for synergy?
Authors: David G. T. Barrett, Ari S. Morcos, Jakob H. Macke
Date: 2018-10-31 00:00:00
Tasks: Object Recognition

Deep neural networks (DNNs) transform stimuli across multiple processing
stages to produce representations that can be used to solve complex tasks, such
as object recognition in images. However, a full understanding of how they
achieve this remains elusive. The complexity of biological neural networks
substantially exceeds the complexity of DNNs, making it even more challenging
to understand the representations that they learn. Thus, both machine learning
and computational neuroscience are faced with a shared challenge: how can we
analyze their representations in order to understand how they solve complex
tasks?
  We review how data-analysis concepts and techniques developed by
computational neuroscientists can be useful for analyzing representations in
DNNs, and in turn, how recently developed techniques for analysis of DNNs can
be useful for understanding representations in biological neural networks. We
explore opportunities for synergy between the two fields, such as the use of
DNNs as in-silico model systems for neuroscience, and how this synergy can lead
to new hypotheses about the operating principles of biological neural networks.
Multiparty Dynamics and Failure Modes for Machine Learning and Artificial Intelligence
Authors: David Manheim
Date: 2018-10-16 00:00:00
Tasks: BIG-bench Machine Learning

An important challenge for safety in machine learning and artificial
intelligence systems is a~set of related failures involving specification
gaming, reward hacking, fragility to distributional shifts, and Goodhart's or
Campbell's law. This paper presents additional failure modes for interactions
within multi-agent systems that are closely related. These multi-agent failure
modes are more complex, more problematic, and less well understood than the
single-agent case, and are also already occurring, largely unnoticed. After
motivating the discussion with examples from poker-playing artificial
intelligence (AI), the paper explains why these failure modes are in some
senses unavoidable. Following this, the paper categorizes failure modes,
provides definitions, and cites examples for each of the modes: accidental
steering, coordination failures, adversarial misalignment, input spoofing and
filtering, and goal co-option or direct hacking. The paper then discusses how
extant literature on multi-agent AI fails to address these failure modes, and
identifies work which may be useful for the mitigation of these failure modes.
Privacy Preserving Multi-Agent Planning with Provable Guarantees
Authors: Amos Beimel, Ronen I. Brafman
Date: 2018-10-31 00:00:00
Tasks: Privacy Preserving

In privacy-preserving multi-agent planning, a group of agents attempt to
cooperatively solve a multi-agent planning problem while maintaining private
their data and actions. Although much work was carried out in this area in past
years, its theoretical foundations have not been fully worked out.
Specifically, although algorithms with precise privacy guarantees exist, even
their most efficient implementations are not fast enough on realistic
instances, whereas for practical algorithms no meaningful privacy guarantees
exist. Secure-MAFS, a variant of the multi-agent forward search algorithm
(MAFS) is the only practical algorithm to attempt to offer more precise
guarantees, but only in very limited settings and with proof sketches only. In
this paper we formulate a precise notion of secure computation for search-based
algorithms and prove that Secure MAFS has this property in all domains.
Multimodal Machine Learning for Automated ICD Coding
Authors: Keyang Xu, Mike Lam, Jingzhi Pang, Xin Gao, Charlotte Band, Piyush Mathur, Frank Papay, Ashish K. Khanna, Jacek B. Cywinski, Kamal Maheshwari, Pengtao Xie, Eric Xing
Date: 2018-10-31 00:00:00
Tasks: BIG-bench Machine Learning, Diagnostic

This study presents a multimodal machine learning model to predict ICD-10 diagnostic codes. We developed separate machine learning models that can handle data from different modalities, including unstructured text, semi-structured text and structured tabular data. We further employed an ensemble method to integrate all modality-specific models to generate ICD-10 codes. Key evidence was also extracted to make our prediction more convincing and explainable. We used the Medical Information Mart for Intensive Care III (MIMIC -III) dataset to validate our approach. For ICD code prediction, our best-performing model (micro-F1 = 0.7633, micro-AUC = 0.9541) significantly outperforms other baseline models including TF-IDF (micro-F1 = 0.6721, micro-AUC = 0.7879) and Text-CNN model (micro-F1 = 0.6569, micro-AUC = 0.9235). For interpretability, our approach achieves a Jaccard Similarity Coefficient (JSC) of 0.1806 on text data and 0.3105 on tabular data, where well-trained physicians achieve 0.2780 and 0.5002 respectively.
MULAN: A Blind and Off-Grid Method for Multichannel Echo Retrieval
Authors: Helena Peic Tukuljac, Antoine Deleforge, Rémi Gribonval
Date: 2018-10-31 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: compressed sensing, Information Retrieval, Retrieval

This paper addresses the general problem of blind echo retrieval, i.e., given
M sensors measuring in the discrete-time domain M mixtures of K delayed and
attenuated copies of an unknown source signal, can the echo locations and
weights be recovered? This problem has broad applications in fields such as
sonars, seismol-ogy, ultrasounds or room acoustics. It belongs to the broader
class of blind channel identification problems, which have been intensively
studied in signal processing. Existing methods in the literature proceed in two
steps: (i) blind estimation of sparse discrete-time filters and (ii) echo
information retrieval by peak-picking on filters. The precision of these
methods is fundamentally limited by the rate at which the signals are sampled:
estimated echo locations are necessary on-grid, and since true locations never
match the sampling grid, the weight estimation precision is impacted. This is
the so-called basis-mismatch problem in compressed sensing. We propose a
radically different approach to the problem, building on the framework of
finite-rate-of-innovation sampling. The approach operates directly in the
parameter-space of echo locations and weights, and enables near-exact blind and
off-grid echo retrieval from discrete-time measurements. It is shown to
outperform conventional methods by several orders of magnitude in precision.
Learning to Represent Edits
Authors: Pengcheng Yin, Graham Neubig, Miltiadis Allamanis, Marc Brockschmidt, Alexander L. Gaunt
Date: 2018-10-31 00:00:00
Proceeding: ICLR 2019 5

We introduce the problem of learning distributed representations of edits. By
combining a "neural editor" with an "edit encoder", our models learn to
represent the salient information of an edit and can be used to apply edits to
new inputs. We experiment on natural language and source code edit data. Our
evaluation yields promising results that suggest that our neural network models
learn to capture the structure and semantics of edits. We hope that this
interesting task and data source will inspire other researchers to work further
on this problem.
Boosting for Comparison-Based Learning
Authors: Michaël Perrot, Ulrike Von Luxburg
Date: 2018-10-31 00:00:00
Tasks: Object, Triplet

We consider the problem of classification in a comparison-based setting: given a set of objects, we only have access to triplet comparisons of the form "object $x_i$ is closer to object $x_j$ than to object $x_k$." In this paper we introduce TripletBoost, a new method that can learn a classifier just from such triplet comparisons. The main idea is to aggregate the triplets information into weak classifiers, which can subsequently be boosted to a strong classifier. Our method has two main advantages: (i) it is applicable to data from any metric space, and (ii) it can deal with large scale problems using only passively obtained and noisy triplets. We derive theoretical generalization guarantees and a lower bound on the number of necessary triplets, and we empirically show that our method is both competitive with state of the art approaches and resistant to noise.