pwc-archive/papers-with-abstracts huggingface dataset:
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Authors: Jonathan Shen, Patrick Nguyen, Yonghui Wu, Zhifeng Chen, Mia X. Chen, Ye Jia, Anjuli Kannan, Tara Sainath, Yuan Cao, Chung-Cheng Chiu, Yanzhang He, Jan Chorowski, Smit Hinsu, Stella Laurenzo, James Qin, Orhan Firat, Wolfgang Macherey, Suyog Gupta, Ankur Bapna, Shuyuan Zhang, Ruoming Pang, Ron J. Weiss, Rohit Prabhavalkar, Qiao Liang, Benoit Jacob, Bowen Liang, HyoukJoong Lee, Ciprian Chelba, Sébastien Jean, Bo Li, Melvin Johnson, Rohan Anil, Rajat Tibrewal, Xiaobing Liu, Akiko Eriguchi, Navdeep Jaitly, Naveen Ari, Colin Cherry, Parisa Haghani, Otavio Good, Youlong Cheng, Raziel Alvarez, Isaac Caswell, Wei-Ning Hsu, Zongheng Yang, Kuan-Chieh Wang, Ekaterina Gonina, Katrin Tomanek, Ben Vanik, Zelin Wu, Llion Jones, Mike Schuster, Yanping Huang, Dehao Chen, Kazuki Irie, George Foster, John Richardson, Klaus Macherey, Antoine Bruguier, Heiga Zen, Colin Raffel, Shankar Kumar, Kanishka Rao, David Rybach, Matthew Murray, Vijayaditya Peddinti, Maxim Krikun, Michiel A. U. Bacchiani, Thomas B. Jablin, Rob Suderman, Ian Williams, Benjamin Lee, Deepti Bhatia, Justin Carlson, Semih Yavuz, Yu Zhang, Ian McGraw, Max Galkin, Qi Ge, Golan Pundak, Chad Whipkey, Todd Wang, Uri Alon, Dmitry Lepikhin, Ye Tian, Sara Sabour, William Chan, Shubham Toshniwal, Baohua Liao, Michael Nirschl, Pat Rondon
Date: 2019-02-21 00:00:00
Tasks: Sequence-To-Sequence Speech Recognition

Lingvo is a Tensorflow framework offering a complete solution for
collaborative deep learning research, with a particular focus towards
sequence-to-sequence models. Lingvo models are composed of modular building
blocks that are flexible and easily extensible, and experiment configurations
are centralized and highly customizable. Distributed training and quantized
inference are supported directly within the framework, and it contains existing
implementations of a large number of utilities, helper functions, and the
newest research ideas. Lingvo has been used in collaboration by dozens of
researchers in more than 20 papers over the last two years. This document
outlines the underlying design of Lingvo and serves as an introduction to the
various pieces of the framework, while also offering examples of advanced
features that showcase the capabilities of the framework.
Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applications
Authors: Songtao Lu, Ioannis Tsaknakis, Mingyi Hong, Yongxin Chen
Date: 2019-02-21 00:00:00

The min-max problem, also known as the saddle point problem, is a class of optimization problems which minimizes and maximizes two subsets of variables simultaneously. This class of problems can be used to formulate a wide range of signal processing and communication (SPCOM) problems. Despite its popularity, most existing theory for this class has been mainly developed for problems with certain special convex-concave structure. Therefore, it cannot be used to guide the algorithm design for many interesting problems in SPCOM, where various kinds of non-convexity arise. In this work, we consider a block-wise one-sided non-convex min-max problem, in which the minimization problem consists of multiple blocks and is non-convex, while the maximization problem is (strongly) concave. We propose a class of simple algorithms named Hybrid Block Successive Approximation (HiBSA), which alternatingly perform gradient descent-type steps for the minimization blocks and gradient ascent-type steps for the maximization problem. A key element in the proposed algorithm is the use of certain regularization and penalty sequences, which stabilize the algorithm and ensure convergence. We show that HiBSA converges to some properly defined first-order stationary solutions with quantifiable global rates. To validate the efficiency of the proposed algorithms, we conduct numerical tests on a number of problems, including the robust learning problem, the non-convex min-utility maximization problems, and certain wireless jamming problem arising in interfering channels.
Bayes Optimal Early Stopping Policies for Black-Box Optimization
Authors: Matthew Streeter
Date: 2019-02-21 00:00:00

We derive an optimal policy for adaptively restarting a randomized algorithm,
based on observed features of the run-so-far, so as to minimize the expected
time required for the algorithm to successfully terminate. Given a suitable
Bayesian prior, this result can be used to select the optimal black-box
optimization algorithm from among a large family of algorithms that includes
random search, Successive Halving, and Hyperband. On CIFAR-10 and ImageNet
hyperparameter tuning problems, the proposed policies offer up to a factor of
13 improvement over random search in terms of expected time to reach a given
target accuracy, and up to a factor of 3 improvement over a baseline adaptive
policy that terminates a run whenever its accuracy is below-median.
Capuchin: Causal Database Repair for Algorithmic Fairness
Authors: Babak Salimi, Luke Rodriguez, Bill Howe, Dan Suciu
Date: 2019-02-21 00:00:00
Tasks: Fairness

Fairness is increasingly recognized as a critical component of machine learning systems. However, it is the underlying data on which these systems are trained that often reflect discrimination, suggesting a database repair problem. Existing treatments of fairness rely on statistical correlations that can be fooled by statistical anomalies, such as Simpson's paradox. Proposals for causality-based definitions of fairness can correctly model some of these situations, but they require specification of the underlying causal models. In this paper, we formalize the situation as a database repair problem, proving sufficient conditions for fair classifiers in terms of admissible variables as opposed to a complete causal model. We show that these conditions correctly capture subtle fairness violations. We then use these conditions as the basis for database repair algorithms that provide provable fairness guarantees about classifiers trained on their training labels. We evaluate our algorithms on real data, demonstrating improvement over the state of the art on multiple fairness metrics proposed in the literature while retaining high utility.
End-to-End Jet Classification of Quarks and Gluons with the CMS Open Data
Authors: Michael Andrews, John Alison, Sitong An, Patrick Bryant, Bjorn Burkle, Sergei Gleyzer, Meenakshi Narain, Manfred Paulini, Barnabas Poczos, Emanuele Usai
Date: 2019-02-21 00:00:00
Tasks: General Classification

We describe the construction of end-to-end jet image classifiers based on simulated low-level detector data to discriminate quark- vs. gluon-initiated jets with high-fidelity simulated CMS Open Data. We highlight the importance of precise spatial information and demonstrate competitive performance to existing state-of-the-art jet classifiers. We further generalize the end-to-end approach to event-level classification of quark vs. gluon di-jet QCD events. We compare the fully end-to-end approach to using hand-engineered features and demonstrate that the end-to-end algorithm is robust against the effects of underlying event and pile-up.
An Online Decision-Theoretic Pipeline for Responder Dispatch
Authors: Ayan Mukhopadhyay, Geoffrey Pettet, Chinmaya Samal, Abhishek Dubey, Yevgeniy Vorobeychik
Date: 2019-02-21 00:00:00

The problem of dispatching emergency responders to service traffic accidents,
fire, distress calls and crimes plagues urban areas across the globe. While
such problems have been extensively looked at, most approaches are offline.
Such methodologies fail to capture the dynamically changing environments under
which critical emergency response occurs, and therefore, fail to be implemented
in practice. Any holistic approach towards creating a pipeline for effective
emergency response must also look at other challenges that it subsumes -
predicting when and where incidents happen and understanding the changing
environmental dynamics. We describe a system that collectively deals with all
these problems in an online manner, meaning that the models get updated with
streaming data sources. We highlight why such an approach is crucial to the
effectiveness of emergency response, and present an algorithmic framework that
can compute promising actions for a given decision-theoretic model for
responder dispatch. We argue that carefully crafted heuristic measures can
balance the trade-off between computational time and the quality of solutions
achieved and highlight why such an approach is more scalable and tractable than
traditional approaches. We also present an online mechanism for incident
prediction, as well as an approach based on recurrent neural networks for
learning and predicting environmental features that affect responder dispatch.
We compare our methodology with prior state-of-the-art and existing dispatch
strategies in the field, which show that our approach results in a reduction in
response time with a drastic reduction in computational time.
Convolutional Analysis Operator Learning: Dependence on Training Data
Authors: Il Yong Chun, David Hong, Ben Adcock, Jeffrey A. Fessler
Date: 2019-02-21 00:00:00
Tasks: Open-Ended Question Answering, Operator learning

Convolutional analysis operator learning (CAOL) enables the unsupervised training of (hierarchical) convolutional sparsifying operators or autoencoders from large datasets. One can use many training images for CAOL, but a precise understanding of the impact of doing so has remained an open question. This paper presents a series of results that lend insight into the impact of dataset size on the filter update in CAOL. The first result is a general deterministic bound on errors in the estimated filters, and is followed by a bound on the expected errors as the number of training samples increases. The second result provides a high probability analogue. The bounds depend on properties of the training data, and we investigate their empirical values with real data. Taken together, these results provide evidence for the potential benefit of using more training data in CAOL.
Quantifying Perceptual Distortion of Adversarial Examples
Authors: Matt Jordan, Naren Manoj, Surbhi Goel, Alexandros G. Dimakis
Date: 2019-02-21 00:00:00
Tasks: SSIM

Recent work has shown that additive threat models, which only permit the
addition of bounded noise to the pixels of an image, are insufficient for fully
capturing the space of imperceivable adversarial examples. For example, small
rotations and spatial transformations can fool classifiers, remain
imperceivable to humans, but have large additive distance from the original
images. In this work, we leverage quantitative perceptual metrics like LPIPS
and SSIM to define a novel threat model for adversarial attacks.
  To demonstrate the value of quantifying the perceptual distortion of
adversarial examples, we present and employ a unifying framework fusing
different attack styles. We first prove that our framework results in images
that are unattainable by attack styles in isolation. We then perform
adversarial training using attacks generated by our framework to demonstrate
that networks are only robust to classes of adversarial perturbations they have
been trained against, and combination attacks are stronger than any of their
individual components. Finally, we experimentally demonstrate that our combined
attacks retain the same perceptual distortion but induce far higher
misclassification rates when compared against individual attacks.
Latent Translation: Crossing Modalities by Bridging Generative Models
Authors: Yingtao Tian, Jesse Engel
Date: 2019-02-21 00:00:00
Tasks: Machine Translation, Translation

End-to-end optimization has achieved state-of-the-art performance on many
specific problems, but there is no straight-forward way to combine pretrained
models for new problems. Here, we explore improving modularity by learning a
post-hoc interface between two existing models to solve a new task.
Specifically, we take inspiration from neural machine translation, and cast the
challenging problem of cross-modal domain transfer as unsupervised translation
between the latent spaces of pretrained deep generative models. By abstracting
away the data representation, we demonstrate that it is possible to transfer
across different modalities (e.g., image-to-audio) and even different types of
generative models (e.g., VAE-to-GAN). We compare to state-of-the-art techniques
and find that a straight-forward variational autoencoder is able to best bridge
the two generative models through learning a shared latent space. We can
further impose supervised alignment of attributes in both domains with a
classifier in the shared latent space. Through qualitative and quantitative
evaluations, we demonstrate that locality and semantic alignment are preserved
through the transfer process, as indicated by high transfer accuracies and
smooth interpolations within a class. Finally, we show this modular structure
speeds up training of new interface models by several orders of magnitude by
decoupling it from expensive retraining of base generative models.
WebProtégé: A Cloud-Based Ontology Editor
Authors: Matthew Horridge, Rafael S. Gonçalves, Csongor I. Nyulas, Tania Tudorache, Mark A. Musen
Date: 2019-02-21 00:00:00

We present WebProt\'eg\'e, a tool to develop ontologies represented in the
Web Ontology Language (OWL). WebProt\'eg\'e is a cloud-based application that
allows users to collaboratively edit OWL ontologies, and it is available for
use at https://webprotege.stanford.edu. WebProt\'ege\'e currently hosts more
than 68,000 OWL ontology projects and has over 50,000 user accounts. In this
paper, we detail the main new features of the latest version of WebProt\'eg\'e.
UQ-CHI: An Uncertainty Quantification-Based Contemporaneous Health Index for Degenerative Disease Monitoring
Authors: Aven Samareh, Shuai Huang
Date: 2019-02-21 00:00:00
Tasks: Decision Making, Management, Uncertainty Quantification

Developing knowledge-driven contemporaneous health index (CHI) that can
precisely reflect the underlying patient across the course of the condition's
progression holds a unique value, like facilitating a range of clinical
decision-making opportunities. This is particularly important for monitoring
degenerative condition such as Alzheimer's disease (AD), where the condition of
the patient will decay over time. Detecting early symptoms and progression
sign, and continuous severity evaluation, are all essential for disease
management. While a few methods have been developed in the literature,
uncertainty quantification of those health index models has been largely
neglected. To ensure the continuity of the care, we should be more explicit
about the level of confidence in model outputs. Ideally, decision-makers should
be provided with recommendations that are robust in the face of substantial
uncertainty about future outcomes. In this paper, we aim at filling this gap by
developing an uncertainty quantification based contemporaneous longitudinal
index, named UQ-CHI, with a particular focus on continuous patient monitoring
of degenerative conditions. Our method is to combine convex optimization and
Bayesian learning using the maximum entropy learning (MEL) framework,
integrating uncertainty on labels as well. Our methodology also provides
closed-form solutions in some important decision making tasks, e.g., such as
predicting the label of a new sample. Numerical studies demonstrate the
effectiveness of the propose UQ-CHI method in prediction accuracy, monitoring
efficacy, and unique advantages if uncertainty quantification is enabled
practice.
Lung Cancer Detection using Co-learning from Chest CT Images and Clinical Demographics
Authors: Jiachen Wang, Riqiang Gao, Yuankai Huo, Shunxing Bao, Yunxi Xiong, Sanja L. Antic, Travis J. Osterman, Pierre P. Massion, Bennett A. Landman
Date: 2019-02-21 00:00:00
Tasks: Computed Tomography (CT)

Early detection of lung cancer is essential in reducing mortality. Recent
studies have demonstrated the clinical utility of low-dose computed tomography
(CT) to detect lung cancer among individuals selected based on very limited
clinical information. However, this strategy yields high false positive rates,
which can lead to unnecessary and potentially harmful procedures. To address
such challenges, we established a pipeline that co-learns from detailed
clinical demographics and 3D CT images. Toward this end, we leveraged data from
the Consortium for Molecular and Cellular Characterization of Screen-Detected
Lesions (MCL), which focuses on early detection of lung cancer. A 3D
attention-based deep convolutional neural net (DCNN) is proposed to identify
lung cancer from the chest CT scan without prior anatomical location of the
suspicious nodule. To improve upon the non-invasive discrimination between
benign and malignant, we applied a random forest classifier to a dataset
integrating clinical information to imaging data. The results show that the AUC
obtained from clinical demographics alone was 0.635 while the attention network
alone reached an accuracy of 0.687. In contrast when applying our proposed
pipeline integrating clinical and imaging variables, we reached an AUC of 0.787
on the testing dataset. The proposed network both efficiently captures
anatomical information for classification and also generates attention maps
that explain the features that drive performance.
An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise
Authors: Yeming Wen, Kevin Luk, Maxime Gazeau, Guodong Zhang, Harris Chan, Jimmy Ba
Date: 2019-02-21 00:00:00
Tasks: Stochastic Optimization

The choice of batch-size in a stochastic optimization algorithm plays a substantial role for both optimization and generalization. Increasing the batch-size used typically improves optimization but degrades generalization. To address the problem of improving generalization while maintaining optimal convergence in large-batch training, we propose to add covariance noise to the gradients. We demonstrate that the learning performance of our method is more accurately captured by the structure of the covariance matrix of the noise rather than by the variance of gradients. Moreover, over the convex-quadratic, we prove in theory that it can be characterized by the Frobenius norm of the noise matrix. Our empirical studies with standard deep learning model-architectures and datasets shows that our method not only improves generalization performance in large-batch training, but furthermore, does so in a way where the optimization performance remains desirable and the training duration is not elongated.
Overcoming Multi-Model Forgetting
Authors: Yassine Benyahia, Kaicheng Yu, Kamil Bennani-Smires, Martin Jaggi, Anthony Davison, Mathieu Salzmann, Claudiu Musat
Date: 2019-02-21 00:00:00
Proceeding: ICLR 2019 5
Tasks: model, Neural Architecture Search

We identify a phenomenon, which we refer to as multi-model forgetting, that
occurs when sequentially training multiple deep networks with partially-shared
parameters; the performance of previously-trained models degrades as one
optimizes a subsequent one, due to the overwriting of shared parameters. To
overcome this, we introduce a statistically-justified weight plasticity loss
that regularizes the learning of a model's shared parameters according to their
importance for the previous models, and demonstrate its effectiveness when
training two models sequentially and for neural architecture search. Adding
weight plasticity in neural architecture search preserves the best models to
the end of the search and yields improved results in both natural language
processing and computer vision tasks.
Online Multi-Object Tracking with Instance-Aware Tracker and Dynamic Model Refreshment
Authors: Peng Chu, Heng Fan, Chiu C. Tan, Haibin Ling
Date: 2019-02-21 00:00:00
Tasks: Multi-Object Tracking, Object Tracking, Online Multi-Object Tracking

Recent progresses in model-free single object tracking (SOT) algorithms have
largely inspired applying SOT to \emph{multi-object tracking} (MOT) to improve
the robustness as well as relieving dependency on external detector. However,
SOT algorithms are generally designed for distinguishing a target from its
environment, and hence meet problems when a target is spatially mixed with
similar objects as observed frequently in MOT. To address this issue, in this
paper we propose an instance-aware tracker to integrate SOT techniques for MOT
by encoding awareness both within and between target models. In particular, we
construct each target model by fusing information for distinguishing target
both from background and other instances (tracking targets). To conserve
uniqueness of all target models, our instance-aware tracker considers response
maps from all target models and assigns spatial locations exclusively to
optimize the overall accuracy. Another contribution we make is a dynamic model
refreshing strategy learned by a convolutional neural network. This strategy
helps to eliminate initialization noise as well as to adapt to the variation of
target size and appearance. To show the effectiveness of the proposed approach,
it is evaluated on the popular MOT15 and MOT16 challenge benchmarks. On both
benchmarks, our approach achieves the best overall performances in comparison
with published results.
Blind Hyperspectral-Multispectral Image Fusion via Graph Laplacian Regularization
Authors: Chandrajit Bajaj, Tianming Wang
Date: 2019-02-21 00:00:00
Tasks: Super-Resolution

Fusing a low-resolution hyperspectral image (HSI) and a high-resolution
multispectral image (MSI) of the same scene leads to a super-resolution image
(SRI), which is information rich spatially and spectrally. In this paper, we
super-resolve the HSI using the graph Laplacian defined on the MSI. Unlike many
existing works, we don't assume prior knowledge about the spatial degradation
from SRI to HSI, nor a perfectly aligned HSI and MSI pair. Our algorithm
progressively alternates between finding the blur kernel and fusing HSI with
MSI, generating accurate estimations of the blur kernel and the SRI at
convergence. Experiments on various datasets demonstrate the advantages of the
proposed algorithm in the quality of fusion and its capability in dealing with
unknown spatial degradation.
Towards Visually Grounded Sub-Word Speech Unit Discovery
Authors: David Harwath, James Glass
Date: 2019-02-21 00:00:00

In this paper, we investigate the manner in which interpretable sub-word
speech units emerge within a convolutional neural network model trained to
associate raw speech waveforms with semantically related natural image scenes.
We show how diphone boundaries can be superficially extracted from the
activation patterns of intermediate layers of the model, suggesting that the
model may be leveraging these events for the purpose of word recognition. We
present a series of experiments investigating the information encoded by these
events.
Web Links Prediction And Category-Wise Recommendation Based On Browser History
Authors: Ashadullah Shawon, Syed Tauhid Zuhori, Firoz Mahmud, Md. Jamil-Ur Rahman
Date: 2019-02-21 00:00:00
Tasks: General Classification, Hyperparameter Optimization, Link Prediction, Prediction

A web browser should not be only for browsing web pages but also help users
to find out their target websites and recommend similar type websites based on
their behavior. Throughout this paper, we propose two methods to make a web
browser more intelligent about link prediction which works during typing on
address-bar and recommendation of websites according to several categories. Our
proposed link prediction system is actually frecency prediction which is
predicted based on the first visit, last visit and URL counts. But recommend
system is the most challenging as it is needed to classify web URLs according
to names without visiting web pages. So we use existing model for URL
classification. The only existing approach gives unsatisfactory results and low
accuracy. So we add hyperparameter optimization with an existing approach that
finds the best parameters for existing URL classification model and gives
better accuracy. In this paper, we propose a category wise recommendation
system using frecency value and the total visit of individual URL category.