pwc-archive/papers-with-abstracts huggingface dataset:
Encoding Categorical Variables with Conjugate Bayesian Models for WeWork Lead Scoring Engine
Authors: Austin Slakey, Daniel Salas, Yoni Schamroth
Date: 2019-04-30 00:00:00
Tasks: Binary Classification, Computational Efficiency

Applied Data Scientists throughout various industries are commonly faced with
the challenging task of encoding high-cardinality categorical features into
digestible inputs for machine learning algorithms. This paper describes a
Bayesian encoding technique developed for WeWork's lead scoring engine which
outputs the probability of a person touring one of our office spaces based on
interaction, enrichment, and geospatial data. We present a paradigm for
ensemble modeling which mitigates the need to build complicated preprocessing
and encoding schemes for categorical variables. In particular, domain-specific
conjugate Bayesian models are employed as base learners for features in a
stacked ensemble model. For each column of a categorical feature matrix we fit
a problem-specific prior distribution, for example, the Beta distribution for a
binary classification problem. In order to analytically derive the moments of
the posterior distribution, we update the prior with the conjugate likelihood
of the corresponding target variable for each unique value of the given
categorical feature. This function of column and value encodes the categorical
feature matrix so that the final learner in the ensemble model ingests
low-dimensional numerical input. Experimental results on both curated and real
world datasets demonstrate impressive accuracy and computational efficiency on
a variety of problem archetypes. Particularly, for the lead scoring engine at
WeWork -- where some categorical features have as many as 300,000 levels -- we
have seen an AUC improvement from 0.87 to 0.97 through implementing conjugate
Bayesian model encoding.
Adversarial Training and Robustness for Multiple Perturbations
Authors: Florian Tramèr, Dan Boneh
Date: 2019-04-30 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Adversarial Robustness

Defenses against adversarial examples, such as adversarial training, are typically tailored to a single perturbation type (e.g., small $\ell_\infty$-noise). For other perturbations, these defenses offer no guarantees and, at times, even increase the model's vulnerability. Our aim is to understand the reasons underlying this robustness trade-off, and to train models that are simultaneously robust to multiple perturbation types. We prove that a trade-off in robustness to different types of $\ell_p$-bounded and spatial perturbations must exist in a natural and simple statistical setting. We corroborate our formal analysis by demonstrating similar robustness trade-offs on MNIST and CIFAR10. Building upon new multi-perturbation adversarial training schemes, and a novel efficient attack for finding $\ell_1$-bounded adversarial examples, we show that no model trained against multiple attacks achieves robustness competitive with that of models trained on each attack individually. In particular, we uncover a pernicious gradient-masking phenomenon on MNIST, which causes adversarial training with first-order $\ell_\infty, \ell_1$ and $\ell_2$ adversaries to achieve merely $50\%$ accuracy. Our results question the viability and computational scalability of extending adversarial robustness, and adversarial training, to multiple perturbation types.
Convergence rates for ordinal embedding
Authors: Jordan S. Ellenberg, Lalit Jain
Date: 2019-04-30 00:00:00

We prove optimal bounds for the convergence rate of ordinal embedding (also
known as non-metric multidimensional scaling) in the 1-dimensional case. The
examples witnessing optimality of our bounds arise from a result in additive
number theory on sets of integers with no three-term arithmetic progressions.
We also carry out some computational experiments aimed at developing a sense of
what the convergence rate for ordinal embedding might look like in higher
dimensions.
A Study on Action Detection in the Wild
Authors: Yubo Zhang, Pavel Tokmakov, Martial Hebert, Cordelia Schmid
Date: 2019-04-29 00:00:00
Tasks: Action Detection

The recent introduction of the AVA dataset for action detection has caused a renewed interest to this problem. Several approaches have been recently proposed that improved the performance. However, all of them have ignored the main difficulty of the AVA dataset - its realistic distribution of training and test examples. This dataset was collected by exhaustive annotation of human action in uncurated videos. As a result, the most common categories, such as `stand' or `sit', contain tens of thousands of examples, whereas rare ones have only dozens. In this work we study the problem of action detection in a highly-imbalanced dataset. Differently from previous work on handling long-tail category distributions, we begin by analyzing the imbalance in the test set. We demonstrate that the standard AP metric is not informative for the categories in the tail, and propose an alternative one - Sampled AP. Armed with this new measure, we study the problem of transferring representations from the data-rich head to the rare tail categories and propose a simple but effective approach.
"Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations
Authors: Yu-jia Zhang, Kuangyan Song, Yiming Sun, Sarah Tan, Madeleine Udell
Date: 2019-04-29 00:00:00
Tasks: Fairness, General Classification, text-classification, Text Classification

Methods for interpreting machine learning black-box models increase the outcomes' transparency and in turn generates insight into the reliability and fairness of the algorithms. However, the interpretations themselves could contain significant uncertainty that undermines the trust in the outcomes and raises concern about the model's reliability. Focusing on the method "Local Interpretable Model-agnostic Explanations" (LIME), we demonstrate the presence of two sources of uncertainty, namely the randomness in its sampling procedure and the variation of interpretation quality across different input data points. Such uncertainty is present even in models with high training and test accuracy. We apply LIME to synthetic data and two public data sets, text classification in 20 Newsgroup and recidivism risk-scoring in COMPAS, to support our argument.
Learning Image Information for eCommerce Queries
Authors: Utkarsh Porwal
Date: 2019-04-29 00:00:00
Tasks: Information Retrieval, Retrieval

Computing similarity between a query and a document is fundamental in any
information retrieval system. In search engines, computing query-document
similarity is an essential step in both retrieval and ranking stages. In eBay
search, document is an item and the query-item similarity can be computed by
comparing different facets of the query-item pair. Query text can be compared
with the text of the item title. Likewise, a category constraint applied on the
query can be compared with the listing category of the item. However, images
are one signal that are usually present in the items but are not present in the
query. Images are one of the most intuitive signals used by users to determine
the relevance of the item given a query. Including this signal in estimating
similarity between the query-item pair is likely to improve the relevance of
the search engine. We propose a novel way of deriving image information for
queries. We attempt to learn image information for queries from item images
instead of generating explicit image features or an image for queries. We use
canonical correlation analysis (CCA) to learn a new subspace where projecting
the original data will give us a new query and item representation. We
hypothesize that this new query representation will also have image information
about the query. We estimate the query-item similarity using a vector space
model and report the performance of the proposed method on eBay's search data.
We show 11.89\% relevance improvement over the baseline using area under the
receiver operating characteristic curve (AUROC) as the evaluation metric. We
also show 3.1\% relevance improvement over the baseline with area under the
precision recall curve (AUPRC) .
Optical Transient Object Classification in Wide Field Small Aperture Telescopes with Neural Networks
Authors: Peng Jia, Yifei Zhao, Gang Xue, Dongmei Cai
Date: 2019-04-29 00:00:00
Tasks: Classification, Ensemble Learning, General Classification

Wide field small aperture telescopes are working horses for fast sky
surveying. Transient discovery is one of their main tasks. Classification of
candidate transient images between real sources and artifacts with high
accuracy is an important step for transient discovery. In this paper, we
propose two transient classification methods based on neural networks. The
first method uses the convolutional neural network without pooling layers to
classify transient images with low sampling rate. The second method assumes
transient images as one dimensional signals and is based on recurrent neural
networks with long short term memory and leaky ReLu activation function in each
detection layer. Testing with real observation data, we find that although
these two methods can both achieve more than 94% classification accuracy, they
have different classification properties for different targets. Based on this
result, we propose to use the ensemble learning method to further increase the
classification accuracy to more than 97%.
Unsupervised Extraction of Phenotypes from Cancer Clinical Notes for Association Studies
Authors: Stefan G. Stark, Stephanie L. Hyland, Melanie F. Pradier, Kjong Lehmann, Andreas Wicki, Fernando Perez Cruz, Julia E. Vogt, Gunnar Rätsch
Date: 2019-04-29 00:00:00
Tasks: Clustering

The recent adoption of Electronic Health Records (EHRs) by health care providers has introduced an important source of data that provides detailed and highly specific insights into patient phenotypes over large cohorts. These datasets, in combination with machine learning and statistical approaches, generate new opportunities for research and clinical care. However, many methods require the patient representations to be in structured formats, while the information in the EHR is often locked in unstructured texts designed for human readability. In this work, we develop the methodology to automatically extract clinical features from clinical narratives from large EHR corpora without the need for prior knowledge. We consider medical terms and sentences appearing in clinical narratives as atomic information units. We propose an efficient clustering strategy suitable for the analysis of large text corpora and to utilize the clusters to represent information about the patient compactly. To demonstrate the utility of our approach, we perform an association study of clinical features with somatic mutation profiles from 4,007 cancer patients and their tumors. We apply the proposed algorithm to a dataset consisting of about 65 thousand documents with a total of about 3.2 million sentences. We identify 341 significant statistical associations between the presence of somatic mutations and clinical features. We annotated these associations according to their novelty, and report several known associations. We also propose 32 testable hypotheses where the underlying biological mechanism does not appear to be known but plausible. These results illustrate that the automated discovery of clinical features is possible and the joint analysis of clinical and genetic datasets can generate appealing new hypotheses.
A neural network based on SPD manifold learning for skeleton-based hand gesture recognition
Authors: Xuan Son Nguyen, Luc Brun, Olivier Lézoray, Sébastien Bougleux
Date: 2019-04-29 00:00:00
Proceeding: CVPR 2019 6
Tasks: Gesture Recognition, Hand Gesture Recognition, Hand-Gesture Recognition

This paper proposes a new neural network based on SPD manifold learning for
skeleton-based hand gesture recognition. Given the stream of hand's joint
positions, our approach combines two aggregation processes on respectively
spatial and temporal domains. The pipeline of our network architecture consists
in three main stages. The first stage is based on a convolutional layer to
increase the discriminative power of learned features. The second stage relies
on different architectures for spatial and temporal Gaussian aggregation of
joint features. The third stage learns a final SPD matrix from skeletal data. A
new type of layer is proposed for the third stage, based on a variant of
stochastic gradient descent on Stiefel manifolds. The proposed network is
validated on two challenging datasets and shows state-of-the-art accuracies on
both datasets.
Improving Mechanical Ventilator Clinical Decision Support Systems with A Machine Learning Classifier for Determining Ventilator Mode
Authors: Gregory B. Rehm, Brooks T. Kuhn, Jimmy Nguyen, Nicholas R. Anderson, Chen-Nee Chuah, Jason Y. Adams
Date: 2019-04-29 00:00:00
Tasks: BIG-bench Machine Learning

Clinical decision support systems (CDSS) will play an in-creasing role in
improving the quality of medical care for critically ill patients. However, due
to limitations in current informatics infrastructure, CDSS do not always have
com-plete information on state of supporting physiologic monitor-ing devices,
which can limit the input data available to CDSS. This is especially true in
the use case of mechanical ventilation (MV), where current CDSS have no
knowledge of critical ventilation settings, such as ventilation mode. To enable
MV CDSS to make accurate recommendations related to ventilator mode, we
developed a highly performant ma-chine learning model that is able to perform
per-breath clas-sification of 5 of the most widely used ventilation modes in
the USA with an average F1-score of 97.52%. We also show how our approach makes
methodologic improvements over previous work and that it is highly robust to
missing data caused by software/sensor error.
Convolutional nets for reconstructing neural circuits from brain images acquired by serial section electron microscopy
Authors: Kisuk Lee, Nicholas Turner, Thomas Macrina, Jingpeng Wu, Ran Lu, H. Sebastian Seung
Date: 2019-04-29 00:00:00
Tasks: Boundary Detection

Neural circuits can be reconstructed from brain images acquired by serial
section electron microscopy. Image analysis has been performed by manual labor
for half a century, and efforts at automation date back almost as far.
Convolutional nets were first applied to neuronal boundary detection a dozen
years ago, and have now achieved impressive accuracy on clean images. Robust
handling of image defects is a major outstanding challenge. Convolutional nets
are also being employed for other tasks in neural circuit reconstruction:
finding synapses and identifying synaptic partners, extending or pruning
neuronal reconstructions, and aligning serial section images to create a 3D
image stack. Computational systems are being engineered to handle petavoxel
images of cubic millimeter brain volumes.
Predictive Situation Awareness for Ebola Virus Disease using a Collective Intelligence Multi-Model Integration Platform: Bayes Cloud
Authors: Cheol Young Park, Shou Matsumoto, Jubyung Ha, YoungWon Park
Date: 2019-04-29 00:00:00
Tasks: Decision Making

The humanity has been facing a plethora of challenges associated with infectious diseases, which kill more than 6 million people a year. Although continuous efforts have been applied to relieve the potential damages from such misfortunate events, it is unquestionable that there are many persisting challenges yet to overcome. One related issue we particularly address here is the assessment and prediction of such epidemics. In this field of study, traditional and ad-hoc models frequently fail to provide proper predictive situation awareness (PSAW), characterized by understanding the current situations and predicting the future situations. Comprehensive PSAW for infectious disease can support decision making and help to hinder disease spread. In this paper, we develop a computing system platform focusing on collective intelligence causal modeling, in order to support PSAW in the domain of infectious disease. Analyses of global epidemics require integration of multiple different data and models, which can be originated from multiple independent researchers. These models should be integrated to accurately assess and predict the infectious disease in terms of holistic view. The system shall provide three main functions: (1) collaborative causal modeling, (2) causal model integration, and (3) causal model reasoning. These functions are supported by subject-matter expert and artificial intelligence (AI), with uncertainty treatment. Subject-matter experts, as collective intelligence, develop causal models and integrate them as one joint causal model. The integrated causal model shall be used to reason about: (1) the past, regarding how the causal factors have occurred; (2) the present, regarding how the spread is going now; and (3) the future, regarding how it will proceed. Finally, we introduce one use case of predictive situation awareness for the Ebola virus disease.
New optimization algorithms for neural network training using operator splitting techniques
Authors: Cristian Daniel Alecsa, Titus Pinta, Imre Boros
Date: 2019-04-29 00:00:00
Tasks: General Classification

In the following paper we present a new type of optimization algorithms adapted for neural network training. These algorithms are based upon sequential operator splitting technique for some associated dynamical systems. Furthermore, we investigate through numerical simulations the empirical rate of convergence of these iterative schemes toward a local minimum of the loss function, with some suitable choices of the underlying hyper-parameters. We validate the convergence of these optimizers using the results of the accuracy and of the loss function on the MNIST, MNIST-Fashion and CIFAR 10 classification datasets.
Learning Raw Image Denoising with Bayer Pattern Unification and Bayer Preserving Augmentation
Authors: Jiaming Liu, Chi-Hao Wu, Yuzhi Wang, Qin Xu, Yuqian Zhou, Haibin Huang, Chuan Wang, Shaofan Cai, Yifan Ding, Haoqiang Fan, Jue Wang
Date: 2019-04-29 00:00:00
Tasks: Data Augmentation, Denoising, Image Augmentation, Image Denoising, SSIM, valid

In this paper, we present new data pre-processing and augmentation techniques for DNN-based raw image denoising. Compared with traditional RGB image denoising, performing this task on direct camera sensor readings presents new challenges such as how to effectively handle various Bayer patterns from different data sources, and subsequently how to perform valid data augmentation with raw images. To address the first problem, we propose a Bayer pattern unification (BayerUnify) method to unify different Bayer patterns. This allows us to fully utilize a heterogeneous dataset to train a single denoising model instead of training one model for each pattern. Furthermore, while it is essential to augment the dataset to improve model generalization and performance, we discovered that it is error-prone to modify raw images by adapting augmentation methods designed for RGB images. Towards this end, we present a Bayer preserving augmentation (BayerAug) method as an effective approach for raw image augmentation. Combining these data processing technqiues with a modified U-Net, our method achieves a PSNR of 52.11 and a SSIM of 0.9969 in NTIRE 2019 Real Image Denoising Challenge, demonstrating the state-of-the-art performance. Our code is available at https://github.com/Jiaming-Liu/BayerUnifyAug.
Generative models as parsimonious descriptions of sensorimotor loops
Authors: Manuel Baltieri, Christopher L. Buckley
Date: 2019-04-29 00:00:00

The Bayesian brain hypothesis, predictive processing and variational free
energy minimisation are typically used to describe perceptual processes based
on accurate generative models of the world. However, generative models need not
be veridical representations of the environment. We suggest that they can (and
should) be used to describe sensorimotor relationships relevant for behaviour
rather than precise accounts of the world.
Learning to Find Common Objects Across Few Image Collections
Authors: Amirreza Shaban, Amir Rahimi, Shray Bansal, Stephen Gould, Byron Boots, Richard Hartley
Date: 2019-04-29 00:00:00
Proceeding: ICCV 2019 10
Tasks: Few-Shot Learning, Object, Object Recognition

Given a collection of bags where each bag is a set of images, our goal is to select one image from each bag such that the selected images are from the same object class. We model the selection as an energy minimization problem with unary and pairwise potential functions. Inspired by recent few-shot learning algorithms, we propose an approach to learn the potential functions directly from the data. Furthermore, we propose a fast greedy inference algorithm for energy minimization. We evaluate our approach on few-shot common object recognition as well as object co-localization tasks. Our experiments show that learning the pairwise and unary terms greatly improves the performance of the model over several well-known methods for these tasks. The proposed greedy optimization algorithm achieves performance comparable to state-of-the-art structured inference algorithms while being ~10 times faster. The code is publicly available on https://github.com/haamoon/finding_common_object.
Advancing GraphSAGE with A Data-Driven Node Sampling
Authors: Jihun Oh, Kyunghyun Cho, Joan Bruna
Date: 2019-04-29 00:00:00
Tasks: General Classification, Graph Neural Network, Node Classification, Reinforcement Learning

As an efficient and scalable graph neural network, GraphSAGE has enabled an
inductive capability for inferring unseen nodes or graphs by aggregating
subsampled local neighborhoods and by learning in a mini-batch gradient descent
fashion. The neighborhood sampling used in GraphSAGE is effective in order to
improve computing and memory efficiency when inferring a batch of target nodes
with diverse degrees in parallel. Despite this advantage, the default uniform
sampling suffers from high variance in training and inference, leading to
sub-optimum accuracy. We propose a new data-driven sampling approach to reason
about the real-valued importance of a neighborhood by a non-linear regressor,
and to use the value as a criterion for subsampling neighborhoods. The
regressor is learned using a value-based reinforcement learning. The implied
importance for each combination of vertex and neighborhood is inductively
extracted from the negative classification loss output of GraphSAGE. As a
result, in an inductive node classification benchmark using three datasets, our
method enhanced the baseline using the uniform sampling, outperforming recent
variants of a graph neural network in accuracy.
Recurrent Neural Networks in the Eye of Differential Equations
Authors: Murphy Yuezhen Niu, Lior Horesh, Isaac Chuang
Date: 2019-04-29 00:00:00
Tasks: Numerical Integration

To understand the fundamental trade-offs between training stability, temporal
dynamics and architectural complexity of recurrent neural networks~(RNNs), we
directly analyze RNN architectures using numerical methods of ordinary
differential equations~(ODEs). We define a general family of RNNs--the
ODERNNs--by relating the composition rules of RNNs to integration methods of
ODEs at discrete time steps. We show that the degree of RNN's functional
nonlinearity $n$ and the range of its temporal memory $t$ can be mapped to the
corresponding stage of Runge-Kutta recursion and the order of time-derivative
of the ODEs. We prove that popular RNN architectures, such as LSTM and URNN,
fit into different orders of $n$-$t$-ODERNNs. This exact correspondence between
RNN and ODE helps us to establish the sufficient conditions for RNN training
stability and facilitates more flexible top-down designs of new RNN
architectures using large varieties of toolboxes from numerical integration of
ODEs. We provide such an example: Quantum-inspired Universal computing Neural
Network~(QUNN), which reduces the required number of training parameters from
polynomial in both data length and temporal memory length to only linear in
temporal memory length.