pwc-archive/papers-with-abstracts huggingface dataset:
Conv-codes: Audio Hashing For Bird Species Classification
Authors: Anshul Thakur, Pulkit Sharma, Vinayak Abrol, Padmanabhan Rajan
Date: 2019-02-07 00:00:00
Tasks: Classification, Clustering, General Classification

In this work, we propose a supervised, convex representation based audio
hashing framework for bird species classification. The proposed framework
utilizes archetypal analysis, a matrix factorization technique, to obtain
convex-sparse representations of a bird vocalization. These convex
representations are hashed using Bloom filters with non-cryptographic hash
functions to obtain compact binary codes, designated as conv-codes. The
conv-codes extracted from the training examples are clustered using
class-specific k-medoids clustering with Jaccard coefficient as the similarity
metric. A hash table is populated using the cluster centers as keys while hash
values/slots are pointers to the species identification information. During
testing, the hash table is searched to find the species information
corresponding to a cluster center that exhibits maximum similarity with the
test conv-code. Hence, the proposed framework classifies a bird vocalization in
the conv-code space and requires no explicit classifier or reconstruction error
calculations. Apart from that, based on min-hash and direct addressing, we also
propose a variant of the proposed framework that provides faster and effective
classification. The performances of both these frameworks are compared with
existing bird species classification frameworks on the audio recordings of 50
different bird species.
CHIP: Channel-wise Disentangled Interpretation of Deep Convolutional Neural Networks
Authors: Xinrui Cui, Dan Wang, Z. Jane Wang
Date: 2019-02-07 00:00:00
Tasks: image-classification, Image Classification

With the widespread applications of deep convolutional neural networks (DCNNs), it becomes increasingly important for DCNNs not only to make accurate predictions but also to explain how they make their decisions. In this work, we propose a CHannel-wise disentangled InterPretation (CHIP) model to give the visual interpretation to the predictions of DCNNs. The proposed model distills the class-discriminative importance of channels in networks by utilizing the sparse regularization. Here, we first introduce the network perturbation technique to learn the model. The proposed model is capable to not only distill the global perspective knowledge from networks but also present the class-discriminative visual interpretation for specific predictions of networks. It is noteworthy that the proposed model is able to interpret different layers of networks without re-training. By combining the distilled interpretation knowledge in different layers, we further propose the Refined CHIP visual interpretation that is both high-resolution and class-discriminative. Experimental results on the standard dataset demonstrate that the proposed model provides promising visual interpretation for the predictions of networks in image classification task compared with existing visual interpretation methods. Besides, the proposed method outperforms related approaches in the application of ILSVRC 2015 weakly-supervised localization task.
Cost-Effective Incentive Allocation via Structured Counterfactual Inference
Authors: Romain Lopez, Chenchen Li, Xiang Yan, Junwu Xiong, Michael. I. Jordan, Yuan Qi, Le Song
Date: 2019-02-07 00:00:00
Tasks: counterfactual, Counterfactual Inference, Domain Adaptation, Marketing

We address a practical problem ubiquitous in modern marketing campaigns, in which a central agent tries to learn a policy for allocating strategic financial incentives to customers and observes only bandit feedback. In contrast to traditional policy optimization frameworks, we take into account the additional reward structure and budget constraints common in this setting, and develop a new two-step method for solving this constrained counterfactual policy optimization problem. Our method first casts the reward estimation problem as a domain adaptation problem with supplementary structure, and then subsequently uses the estimators for optimizing the policy with constraints. We also establish theoretical error bounds for our estimation procedure and we empirically show that the approach leads to significant improvement on both synthetic and real datasets.
A Simple Baseline for Bayesian Uncertainty in Deep Learning
Authors: Wesley Maddox, Timur Garipov, Pavel Izmailov, Dmitry Vetrov, Andrew Gordon Wilson
Date: 2019-02-07 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Bayesian Inference, Deep Learning, Transfer Learning, Uncertainty Quantification

We propose SWA-Gaussian (SWAG), a simple, scalable, and general purpose approach for uncertainty representation and calibration in deep learning. Stochastic Weight Averaging (SWA), which computes the first moment of stochastic gradient descent (SGD) iterates with a modified learning rate schedule, has recently been shown to improve generalization in deep learning. With SWAG, we fit a Gaussian using the SWA solution as the first moment and a low rank plus diagonal covariance also derived from the SGD iterates, forming an approximate posterior distribution over neural network weights; we then sample from this Gaussian distribution to perform Bayesian model averaging. We empirically find that SWAG approximates the shape of the true posterior, in accordance with results describing the stationary distribution of SGD iterates. Moreover, we demonstrate that SWAG performs well on a wide variety of tasks, including out of sample detection, calibration, and transfer learning, in comparison to many popular alternatives including MC dropout, KFAC Laplace, SGLD, and temperature scaling.
End-to-end losses based on speaker basis vectors and all-speaker hard negative mining for speaker verification
Authors: Hee-Soo Heo, Jee-weon Jung, IL-Ho Yang, Sung-Hyun Yoon, Hye-jin Shim, Ha-Jin Yu
Date: 2019-02-07 00:00:00
Tasks: All, Metric Learning, Speaker Verification

In recent years, speaker verification has primarily performed using deep neural networks that are trained to output embeddings from input features such as spectrograms or Mel-filterbank energies. Studies that design various loss functions, including metric learning have been widely explored. In this study, we propose two end-to-end loss functions for speaker verification using the concept of speaker bases, which are trainable parameters. One loss function is designed to further increase the inter-speaker variation, and the other is designed to conduct the identical concept with hard negative mining. Each speaker basis is designed to represent the corresponding speaker in the process of training deep neural networks. In contrast to the conventional loss functions that can consider only a limited number of speakers included in a mini-batch, the proposed loss functions can consider all the speakers in the training set regardless of the mini-batch composition. In particular, the proposed loss functions enable hard negative mining and calculations of between-speaker variations with consideration of all speakers. Through experiments on VoxCeleb1 and VoxCeleb2 datasets, we confirmed that the proposed loss functions could supplement conventional softmax and center loss functions.
Extending Stein's unbiased risk estimator to train deep denoisers with correlated pairs of noisy images
Authors: Magauiya Zhussip, Shakarim Soltanayev, Se Young Chun
Date: 2019-02-07 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Denoising, Image Restoration

Recently, Stein's unbiased risk estimator (SURE) has been applied to unsupervised training of deep neural network Gaussian denoisers that outperformed classical non-deep learning based denoisers and yielded comparable performance to those trained with ground truth. While SURE requires only one noise realization per image for training, it does not take advantage of having multiple noise realizations per image when they are available (e.g., two uncorrelated noise realizations per image for Noise2Noise). Here, we propose an extended SURE (eSURE) to train deep denoisers with correlated pairs of noise realizations per image and applied it to the case with two uncorrelated realizations per image to achieve better performance than SURE based method and comparable results to Noise2Noise. Then, we further investigated the case with imperfect ground truth (i.e., mild noise in ground truth) that may be obtained considering painstaking, time-consuming, and even expensive processes of collecting ground truth images with multiple noisy images. For the case of generating noisy training data by adding synthetic noise to imperfect ground truth to yield correlated pairs of images, our proposed eSURE based training method outperformed conventional SURE based method as well as Noise2Noise.
Empirically Accelerating Scaled Gradient Projection Using Deep Neural Network For Inverse Problems In Image Processing
Authors: Byung Hyun Lee, Se Young Chun
Date: 2019-02-07 00:00:00
Tasks: Deblurring, Image Inpainting, Image Reconstruction

Recently, deep neural networks (DNNs) have shown advantages in accelerating optimization algorithms. One approach is to unfold finite number of iterations of conventional optimization algorithms and to learn parameters in the algorithms. However, these are forward methods and are indeed neither iterative nor convergent. Here, we present a novel DNN-based convergent iterative algorithm that accelerates conventional optimization algorithms. We train a DNN to yield parameters in scaled gradient projection method. So far, these parameters have been chosen heuristically, but have shown to be crucial for good empirical performance. In simulation results, the proposed method significantly improves the empirical convergence rate over conventional optimization methods for various large-scale inverse problems in image processing.
Effectiveness of LSTMs in Predicting Congestive Heart Failure Onset
Authors: Sunil Mallya, Marc Overhage, Navneet Srivastava, Tatsuya Arai, Cole Erdman
Date: 2019-02-07 00:00:00
Tasks: Attribute, Feature Importance, Medical Diagnosis

In this paper we present a Recurrent neural networks (RNN) based architecture
that achieves an AUCROC of 0.9147 for predicting the onset of Congestive Heart
Failure (CHF) 15 months in advance using a 12-month observation window on a
large cohort of 216,394 patients. We believe this to be the largest study in
CHF onset prediction with respect to the number of CHF case patients in the
cohort and the test set (3,332 CHF patients) on which the AUC metrics are
reported. We explore the extent to which LSTM (Long Short Term Memory) based
model, a variant of RNNs, can accurately predict the onset of CHF when compared
to known linear baselines like Logistic Regression, Random Forests and deep
learning based models such as Multi-Layer Perceptron and Convolutional Neural
Networks. We utilize demographics, medical diagnosis and procedure data from
21,405 CHF and 194,989 control patients to as our features. We describe our
feature embedding strategy for medical diagnosis codes that accommodates the
sparse, irregular, longitudinal, and high-dimensional characteristics of EHR
data. We empirically show that LSTMs can capture the longitudinal aspects of
EHR data better than the proposed baselines. As an attempt to interpret the
model, we present a temporal data analysis-based technique on false positives
to attribute feature importance. A model capable of predicting the onset of
congestive heart failure months in the future with this level of accuracy and
precision can support efforts of practitioners to implement risk factor
reduction strategies and researchers to begin to systematically evaluate
interventions to potentially delay or avert development of the disease with
high mortality, morbidity and significant costs.
Artificial Intelligence for Prosthetics - challenge solutions
Authors: Łukasz Kidziński, Carmichael Ong, Sharada Prasanna Mohanty, Jennifer Hicks, Sean F. Carroll, Bo Zhou, Hongsheng Zeng, Fan Wang, Rongzhong Lian, Hao Tian, Wojciech Jaśkowski, Garrett Andersen, Odd Rune Lykkebø, Nihat Engin Toklu, Pranav Shyam, Rupesh Kumar Srivastava, Sergey Kolesnikov, Oleksii Hrinchuk, Anton Pechenko, Mattias Ljungström, Zhen Wang, Xu Hu, Zehong Hu, Minghui Qiu, Jun Huang, Aleksei Shpilman, Ivan Sosin, Oleg Svidchenko, Aleksandra Malysheva, Daniel Kudenko, Lance Rane, Aditya Bhatt, Zhengfei Wang, Penghui Qi, Zeyang Yu, Peng Peng, Quan Yuan, Wenxin Li, Yunsheng Tian, Ruihan Yang, Pingchuan Ma, Shauharda Khadka, Somdeb Majumdar, Zach Dwiel, Yinyin Liu, Evren Tumer, Jeremy Watson, Marcel Salathé, Sergey Levine, Scott Delp
Date: 2019-02-07 00:00:00
Tasks: Deep Reinforcement Learning, Imitation Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In the NeurIPS 2018 Artificial Intelligence for Prosthetics challenge,
participants were tasked with building a controller for a musculoskeletal model
with a goal of matching a given time-varying velocity vector. Top participants
were invited to describe their algorithms. In this work, we describe the
challenge and present thirteen solutions that used deep reinforcement learning
approaches. Many solutions use similar relaxations and heuristics, such as
reward shaping, frame skipping, discretization of the action space, symmetry,
and policy blending. However, each team implemented different modifications of
the known algorithms by, for example, dividing the task into subtasks, learning
low-level control, or by incorporating expert knowledge and using imitation
learning.
A Scale Invariant Flatness Measure for Deep Network Minima
Authors: Akshay Rangamani, Nam H. Nguyen, Abhishek Kumar, Dzung Phan, Sang H. Chin, Trac. D. Tran
Date: 2019-02-06 00:00:00

It has been empirically observed that the flatness of minima obtained from
training deep networks seems to correlate with better generalization. However,
for deep networks with positively homogeneous activations, most measures of
sharpness/flatness are not invariant to rescaling of the network parameters,
corresponding to the same function. This means that the measure of
flatness/sharpness can be made as small or as large as possible through
rescaling, rendering the quantitative measures meaningless. In this paper we
show that for deep networks with positively homogenous activations, these
rescalings constitute equivalence relations, and that these equivalence
relations induce a quotient manifold structure in the parameter space. Using
this manifold structure and an appropriate metric, we propose a Hessian-based
measure for flatness that is invariant to rescaling. We use this new measure to
confirm the proposition that Large-Batch SGD minima are indeed sharper than
Small-Batch SGD minima.
Dynamic-Weighted Simplex Strategy for Learning Enabled Cyber Physical Systems
Authors: Shreyas Ramakrishna, Charles Hartsell, Matthew P Burruss, Gabor Karsai, Abhishek Dubey
Date: 2019-02-06 00:00:00
Tasks: Autonomous Driving, Q-Learning, Reinforcement Learning

Cyber Physical Systems (CPS) have increasingly started using Learning Enabled Components (LECs) for performing perception-based control tasks. The simple design approach, and their capability to continuously learn has led to their widespread use in different autonomous applications. Despite their simplicity and impressive capabilities, these models are difficult to assure, which makes their use challenging. The problem of assuring CPS with untrusted controllers has been achieved using the Simplex Architecture. This architecture integrates the system to be assured with a safe controller and provides a decision logic to switch between the decisions of these controllers. However, the key challenges in using the Simplex Architecture are: (1) designing an effective decision logic, and (2) sudden transitions between controller decisions lead to inconsistent system performance. To address these research challenges, we make three key contributions: (1) \textit{dynamic-weighted simplex strategy} -- we introduce ``weighted simplex strategy" as the weighted ensemble extension of the classical Simplex Architecture. We then provide a reinforcement learning based mechanism to find dynamic ensemble weights, (2) \textit{middleware framework} -- we design a framework that allows the use of the dynamic-weighted simplex strategy, and provides a resource manager to monitor the computational resources, and (3) \textit{hardware testbed} -- we design a remote-controlled car testbed called DeepNNCar to test and demonstrate the aforementioned key concepts. Using the hardware, we show that the dynamic-weighted simplex strategy has 60\% fewer out-of-track occurrences (soft constraint violations), while demonstrating higher optimized speed (performance) of 0.4 m/s during indoor driving than the original LEC driven system.
Principal Model Analysis Based on Partial Least Squares
Authors: Qiwei Xie, Liang Tang, Weifu Li, Vijay John, Yong Hu
Date: 2019-02-06 00:00:00
Tasks: Dimensionality Reduction, General Classification, model, regression

Motivated by the Bagging Partial Least Squares (PLS) and Principal Component
Analysis (PCA) algorithms, we propose a Principal Model Analysis (PMA) method
in this paper. In the proposed PMA algorithm, the PCA and the PLS are combined.
In the method, multiple PLS models are trained on sub-training sets, derived
from the original training set based on the random sampling with replacement
method. The regression coefficients of all the sub-PLS models are fused in a
joint regression coefficient matrix. The final projection direction is then
estimated by performing the PCA on the joint regression coefficient matrix. The
proposed PMA method is compared with other traditional dimension reduction
methods, such as PLS, Bagging PLS, Linear discriminant analysis (LDA) and
PLS-LDA. Experimental results on six public datasets show that our proposed
method can achieve better classification performance and is usually more
stable.
Fast Hyperparameter Tuning using Bayesian Optimization with Directional Derivatives
Authors: Tinu Theckel Joy, Santu Rana, Sunil Gupta, Svetha Venkatesh
Date: 2019-02-06 00:00:00
Tasks: Bayesian Optimization, Learning Theory, PAC learning, Small Data Image Classification

In this paper we develop a Bayesian optimization based hyperparameter tuning
framework inspired by statistical learning theory for classifiers. We utilize
two key facts from PAC learning theory; the generalization bound will be higher
for a small subset of data compared to the whole, and the highest accuracy for
a small subset of data can be achieved with a simple model. We initially tune
the hyperparameters on a small subset of training data using Bayesian
optimization. While tuning the hyperparameters on the whole training data, we
leverage the insights from the learning theory to seek more complex models. We
realize this by using directional derivative signs strategically placed in the
hyperparameter search space to seek a more complex model than the one obtained
with small data. We demonstrate the performance of our method on the tasks of
tuning the hyperparameters of several machine learning algorithms.
A Bayesian Approach for Accurate Classification-Based Aggregates
Authors: Q. A. Meertens, C. G. H. Diks, H. J. van den Herik, F W Takes
Date: 2019-02-06 00:00:00
Tasks: Bayesian Inference, Classification, General Classification, Multi-class Classification

In this paper, we study the accuracy of values aggregated over classes
predicted by a classification algorithm. The problem is that the resulting
aggregates (e.g., sums of a variable) are known to be biased. The bias can be
large even for highly accurate classification algorithms, in particular when
dealing with class-imbalanced data. To correct this bias, the algorithm's
classification error rates have to be estimated. In this estimation, two issues
arise when applying existing bias correction methods. First, inaccuracies in
estimating classification error rates have to be taken into account. Second,
impermissible estimates, such as a negative estimate for a positive value, have
to be dismissed. We show that both issues are relevant in applications where
the true labels are known only for a small set of data points. We propose a
novel bias correction method using Bayesian inference. The novelty of our
method is that it imposes constraints on the model parameters. We show that our
method solves the problem of biased classification-based aggregates as well as
the two issues above, in the general setting of multi-class classification. In
the empirical evaluation, using a binary classifier on a real-world dataset of
company tax returns, we show that our method outperforms existing methods in
terms of mean squared error.
On $L_2$-consistency of nearest neighbor matching
Authors: James Sharpnack
Date: 2019-02-06 00:00:00
Tasks: Causal Inference, Domain Adaptation, Imputation, reinforcement-learning, Reinforcement Learning (RL)

Biased sampling and missing data complicates statistical problems ranging from causal inference to reinforcement learning. We often correct for biased sampling of summary statistics with matching methods and importance weighting. In this paper, we study nearest neighbor matching (NNM), which makes estimates of population quantities from biased samples by substituting unobserved variables with their nearest neighbors in the biased sample. We show that NNM is $L_2$-consistent in the absence of smoothness and boundedness assumptions in finite dimensions. We discuss applications of NNM, outline the barriers to generalizing this work to separable metric spaces, and compare this result to inverse probability weighting.
On the Variance of Unbiased Online Recurrent Optimization
Authors: Tim Cooijmans, James Martens
Date: 2019-02-06 00:00:00

The recently proposed Unbiased Online Recurrent Optimization algorithm (UORO,
arXiv:1702.05043) uses an unbiased approximation of RTRL to achieve fully
online gradient-based learning in RNNs. In this work we analyze the variance of
the gradient estimate computed by UORO, and propose several possible changes to
the method which reduce this variance both in theory and practice. We also
contribute significantly to the theoretical and intuitive understanding of UORO
(and its existing variance reduction technique), and demonstrate a fundamental
connection between its gradient estimate and the one that would be computed by
REINFORCE if small amounts of noise were added to the RNN's hidden units.
Adversarial Domain Adaptation for Stance Detection
Authors: Brian Xu, Mitra Mohtarami, James Glass
Date: 2019-02-06 00:00:00
Tasks: Domain Adaptation, Fact Checking, Stance Detection

This paper studies the problem of stance detection which aims to predict the
perspective (or stance) of a given document with respect to a given claim.
Stance detection is a major component of automated fact checking. As annotating
stances in different domains is a tedious and costly task, automatic methods
based on machine learning are viable alternatives. In this paper, we focus on
adversarial domain adaptation for stance detection where we assume there exists
sufficient labeled data in the source domain and limited labeled data in the
target domain. Extensive experiments on publicly available datasets show the
effectiveness of our domain adaption model in transferring knowledge for
accurate stance detection across domains.